Multiclass Classifier¶
While some classification algorithms naturally permit the use of more than two classes, some algorithms, such as Support Vector Machines (SVM), are by nature solving a twoclass problem only. These twoclass (or binary) classifiers can be turned into multiclass classifiers by using different strategies, such as OneAgainstRest or OneAgainstOne.
oneDAL implements a MultiClass Classifier using the OneAgainstOne strategy.
Multiclass classifiers, such as SVM, are based on twoclass classifiers, which are integral components of the models trained with the corresponding multiclass classifier algorithms.
Details¶
Given \(n\) feature vectors \(x_1 = (x_{11}, \ldots, x_{1p}), \ldots, x_n = (x_{n1}, \ldots, x_{np})\) of size \(p\), the number of classes \(K\), and a vector of class labels \(y = (y_1, \ldots, y_n)\), where \(y_i \in \{0, 1, \ldots, K1\}\), the problem is to build a multiclass classifier using a twoclass (binary) classifier, such as a twoclass SVM.
Training Stage¶
The model is trained with the OneAgainstOne method that uses the binary classification described in [Hsu02] as follows: For each pair of classes \((i, j)\), train a binary classifier, such as SVM. The total number of such binary classifiers is \(\frac{K(K1)}{2}\).
Prediction Stage¶
Given a new feature vector \(x_i\), the classifier determines the class to which the vector belongs.
oneDAL provides two methods for class label prediction:
Wu method. According to the algorithm 2 for computation of the class probabilities described in [Wu04]. The library returns the index of the class with the largest probability.
Votebased method. If the binary classifier predicts the feature vector to be in \(i\)th class, the number of votes for the class i is increased by one, otherwise the vote is given to the jth class. If two classes have equal numbers of votes, the class with the smallest index is selected.
Usage of Training Alternative¶
To build a Multiclass Classifier model using methods of the Model Builder class of Multiclass Classifier, complete the following steps:
Create a Multiclass Classifier model builder using a constructor with the required number of features and classes.
Use the
setTwoClassClassifierModel
method for each pair of classes to add the pretrained twoclass classifiers to the model. In the parameters to the method specify the classes’ indices and the pointer to the pretrained twoclass classifier for this pair of classes. You need to do this for each pair of classes, because the OneAgainstOne strategy is used.Use the
getModel
method to get the trained Multiclass Classifier model.Use the
getStatus
method to check the status of the model building process. IfDAAL_NOTHROW_EXCEPTIONS
macros is defined, the status report contains the list of errors that describe the problems API encountered (in case of API runtime failure).
Examples¶
Batch Processing
Batch Processing
Batch Processing¶
Multiclass classifier follows the general workflow described in Classification Usage Model.
Training¶
At the training stage, a multiclass classifier has the following parameters:
Parameter 
Default Value 
Description 



The floatingpoint type that the algorithm uses for intermediate
computations. Can be 


The computation method used by the multiclass classifier. The only training method supported so far is OneAgainstOne. 

Pointer to an object of the SVM training class 
Pointer to the training algorithm of the twoclass classifier. By default, the SVM twoclass classifier is used. 

Not applicable 
The number of classes. A required parameter. 
Prediction¶
At the prediction stage, a multiclass classifier has the following parameters:
Parameter 
Method 
Default Value 
Description 




The floatingpoint type that the algorithm uses for intermediate
computations. Can be 

Not applicable 

Available methods for multiclass classifier prediction stage:



training::oneAgainstOne 
The computation method that was used to train the multiclass classifier model. 


Pointer to an object of the SVM prediction class 
Pointer to the prediction algorithm of the twoclass classifier. By default, the SVM twoclass classifier is used. 


Not applicable 
The number of classes. A required parameter. 


\(100\) 
The maximal number of iterations for the algorithm. 


1.0e12 
The prediction accuracy. 



The 64bit integer flag that specifies which extra characteristics of the decision function to compute. Provide one of the following values to request a single characteristic or use bitwise OR to request a combination of the characteristics:

Output¶
In addition to classifier output, multiclass classifier calculates the result described below.
Pass the Result ID
as a parameter to the methods that access the result of your algorithm.
For more details, see Algorithms.
Result ID 
Result 


A numeric table of size \(n \times \frac{K(K1)}{2}\) containing the results of the decision function
computed for all binary models when the 
Note
If resultsToEvaluate does not contain computeDecisionFunction, the result of decisionFunction table is NULL.
By default, each numeric table of this result is an object of the HomogenNumericTable
class,
but you can define the result as an object of any class derived from NumericTable
except for PackedSymmetricMatrix
and PackedTriangularMatrix
.
Examples¶
Batch Processing:
Note
There is no support for Java on GPU.
Batch Processing:
Batch Processing: