.. ******************************************************************************
.. * Copyright 2019 Intel Corporation
.. *
.. * Licensed under the Apache License, Version 2.0 (the "License");
.. * you may not use this file except in compliance with the License.
.. * You may obtain a copy of the License at
.. *
.. * http://www.apache.org/licenses/LICENSE-2.0
.. *
.. * Unless required by applicable law or agreed to in writing, software
.. * distributed under the License is distributed on an "AS IS" BASIS,
.. * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
.. * See the License for the specific language governing permissions and
.. * limitations under the License.
.. *******************************************************************************/
.. _svm_multi_class:
Multi-class Classifier
======================
.. toctree::
:glob:
:maxdepth: 4
While some classification algorithms naturally permit the use of more
than two classes, some algorithms, such as Support Vector Machines
(SVM), are by nature solving a two-class problem only. These
two-class (or binary) classifiers can be turned into multi-class
classifiers by using different strategies, such as One-Against-Rest
or One-Against-One.
|product| implements a Multi-Class Classifier using the
One-Against-One strategy.
Multi-class classifiers, such as SVM, are based on two-class
classifiers, which are integral components of the models trained with
the corresponding multi-class classifier algorithms.
Details
*******
Given :math:`n` feature vectors :math:`x_1 = (x_{11}, \ldots, x_{1p}), \ldots, x_n = (x_{n1}, \ldots, x_{np})`
of size :math:`p`, the number of classes :math:`K`, and a vector of class labels
:math:`y = (y_1, \ldots, y_n)`, where :math:`y_i \in \{0, 1, \ldots, K-1\}`,
the problem is to build a multi-class classifier using a two-class
(binary) classifier, such as a two-class SVM.
Training Stage
--------------
The model is trained with the One-Against-One method that uses the
binary classification described in
[Hsu02]_ as follows:
For each pair of classes :math:`(i, j)`, train a binary classifier, such
as SVM. The total number of such binary classifiers is :math:`\frac{K(K-1)}{2}`.
Prediction Stage
----------------
Given a new feature vector :math:`x_i`, the classifier determines
the class to which the vector belongs.
|product| provides two methods for class label prediction:
- Wu method. According to the algorithm 2 for computation of the
class probabilities described in [Wu04]_.
The library returns the index of the class with the largest
probability.
- Vote-based method. If the binary classifier predicts the
feature vector to be in :math:`i`-th class, the number of votes for the
class i is increased by one, otherwise the vote is given to the
j-th class. If two classes have equal numbers of votes, the
class with the smallest index is selected.
Usage of Training Alternative
*****************************
To build a Multi-class Classifier model using methods of the Model Builder class of Multi-class Classifier,
complete the following steps:
- Create a Multi-class Classifier model builder using a constructor with the required number of features and classes.
- Use the ``setTwoClassClassifierModel`` method for each pair of classes to add the pre-trained two-class classifiers to the model.
In the parameters to the method specify the classes' indices and the pointer to the pre-trained two-class classifier for this pair of classes.
You need to do this for each pair of classes, because the One-Against-One strategy is used.
- Use the ``getModel`` method to get the trained Multi-class Classifier model.
- Use the ``getStatus`` method to check the status of the model building process.
If ``DAAL_NOTHROW_EXCEPTIONS`` macros is defined, the status report contains the list of errors
that describe the problems API encountered (in case of API runtime failure).
Examples
--------
.. tabs::
.. tab:: oneAPI C++
Batch Processing
- :ref:`cpp_svm_two_class_thunder_dense_batch.cpp`
.. tab:: C++ (CPU)
Batch Processing
- :cpp_example:`svm_multi_class_model_builder.cpp `
.. tab:: Python*
:daal4py_example:`svm_multi_class_model_builder.py`
Batch Processing
****************
Multi-class classifier follows the general workflow described in :ref:`classification_usage_model`.
Training
--------
At the training stage, a multi-class classifier has the following
parameters:
.. tabularcolumns:: |\Y{0.25}|\Y{0.3}|\Y{0.45}|
.. list-table:: Training Parameters for Multi-class Classifier (Batch Processing)
:widths: 10 20 30
:header-rows: 1
:align: left
:class: longtable
* - Parameter
- Default Value
- Description
* - ``algorithmFPType``
- ``float``
- The floating-point type that the algorithm uses for intermediate
computations. Can be ``float`` or ``double``.
* - ``method``
- ``defaultDense``
- The computation method used by the multi-class classifier. The only
training method supported so far is One-Against-One.
* - ``training``
- Pointer to an object of the SVM training class
- Pointer to the training algorithm of the two-class classifier. By default, the SVM two-class classifier is used.
* - ``nClasses``
- Not applicable
- The number of classes. A required parameter.
Prediction
----------
At the prediction stage, a multi-class classifier has the following parameters:
.. tabularcolumns:: |\Y{0.15}|\Y{0.15}|\Y{0.15}|\Y{0.55}|
.. list-table:: Prediction Parameters for Multi-class Classifier (Batch Processing)
:widths: 10 10 10 30
:header-rows: 1
:align: left
:class: longtable
* - Parameter
- Method
- Default Value
- Description
* - ``algorithmFPType``
- ``defaultDense`` or ``voteBased``
- ``float``
- The floating-point type that the algorithm uses for intermediate
computations. Can be ``float`` or ``double``.
* - ``pmethod``
- Not applicable
- ``defaultDense``
- Available methods for multi-class classifier prediction stage:
- ``defaultDense`` - the method described in [Wu04]_
- ``voteBased`` - the method based on the votes obtained from two-class classifiers.
* - ``tmethod``
- ``defaultDense`` or ``voteBased``
- `training::oneAgainstOne`
- The computation method that was used to train the multi-class classifier model.
* - ``prediction``
- ``defaultDense`` or ``voteBased``
- Pointer to an object of the SVM prediction class
- Pointer to the prediction algorithm of the two-class classifier. By default, the SVM two-class classifier is used.
* - ``nClasses``
- ``defaultDense`` or ``voteBased``
- Not applicable
- The number of classes. A required parameter.
* - ``maxIterations``
- ``defaultDense``
- :math:`100`
- The maximal number of iterations for the algorithm.
* - ``accuracyThreshold``
- ``defaultDense``
- 1.0e-12
- The prediction accuracy.
* - ``resultsToEvaluate``
- ``voteBased``
- ``computeClassLabels``
- The 64-bit integer flag that specifies which extra characteristics of the decision function to compute.
Provide one of the following values to request a single characteristic
or use bitwise OR to request a combination of the characteristics:
- ``computeClassLabels`` for `prediction`
- ``computeDecisionFunction`` for `decisionFunction`
Output
------
In addition to classifier output, multiclass classifier calculates the result described below.
Pass the ``Result ID`` as a parameter to the methods that access the result of your algorithm.
For more details, see :ref:`algorithms`.
.. tabularcolumns:: |\Y{0.2}|\Y{0.8}|
.. list-table:: Output for Multi-class Classifier (Batch Processing)
:widths: 10 60
:header-rows: 1
:align: left
* - Result ID
- Result
* - ``decisionFunction``
- A numeric table of size :math:`n \times \frac{K(K-1)}{2}` containing the results of the decision function
computed for all binary models when the ``computeDecisionFunction`` option is enabled.
.. note::
If `resultsToEvaluate` does not contain `computeDecisionFunction`, the result of `decisionFunction` table is `NULL`.
By default, each numeric table of this result is an object of the ``HomogenNumericTable`` class,
but you can define the result as an object of any class derived from ``NumericTable``
except for ``PackedSymmetricMatrix`` and ``PackedTriangularMatrix``.
Examples
--------
.. tabs::
.. tab:: C++ (CPU)
Batch Processing:
- :cpp_example:`svm_multi_class_boser_csr_batch.cpp `
- :cpp_example:`svm_multi_class_boser_dense_batch.cpp `
- :cpp_example:`svm_multi_class_thunder_csr_batch.cpp `
- :cpp_example:`svm_multi_class_thunder_dense_batch.cpp `
.. tab:: Python*
Batch Processing:
- :daal4py_example:`svm_multiclass.py`