Kernel Functions#

Note

Kernel functions are also available with oneAPI interfaces:

Kernel functions form a class of algorithms for pattern analysis. The main characteristic of kernel functions is a distinct approach to this problem. Instead of reducing the dimension of the original data, kernel functions map the data into higher-dimensional spaces in order to make the data more easily separable there.

Linear Kernel#

A linear kernel is the simplest kernel function.

Problem Statement#

Given a set \(X\) of \(n\) feature vectors \(x_1 = (x_{11}, \ldots, x_{1p}), \ldots, x_n = (x_{n1}, \ldots, x_{np})\) of dimension \(p\) and a set \(Y\) of \(m\) feature vectors \(y_1 = (y_{11}, \ldots, y_{1p}), \ldots, y_m = (y_{m1}, \ldots, x_{mp})\), the problem is to compute the linear kernel function \(K(x_i,, y_i)\) for any pair of input vectors: \(K(x_i, y_i) = k {X_i}^T y_i + b\).

Batch Processing#

Algorithm Input#

The linear kernel function accepts the input described below. Pass the Input ID as a parameter to the methods that provide input for your algorithm.

Algorithm Input for Linear Kernel (Batch Processing)#

Input ID

Input

X

Pointer to the \(n \times p\) numeric table that represents the matrix X. This table can be an object of any class derived from NumericTable.

Y

Pointer to the \(m \times p\) numeric table that represents the matrix Y. This table can be an object of any class derived from NumericTable.

Algorithm Parameters#

The linear kernel function has the following parameters:

Algorithm Parameters for Linear Kernel (Batch Processing)#

Parameter

Default Value

Description

algorithmFPType

float

The floating-point type that the algorithm uses for intermediate computations. Can be float or double.

method

defaultDense

Available computation methods:

  • defaultDense - default performance-oriented method

  • fastCSR - performance-oriented method for CSR numeric tables

computationMode

matrixMatrix

Computation mode for the kernel function. Can be:

For CPU:

  • vectorVector - compute the kernel function for given feature vectors \(x_i\) and \(y_j\)

  • matrixVector - compute the kernel function for all vectors in the set \(X\) and a given feature vector \(y_j\)

  • matrixMatrix - compute the kernel function for all vectors in the sets \(X\) and \(Y\). In oneDAL, this mode requires equal numbers of observations in both input tables: \(n = m\).

For GPU:

  • matrixMatrix - compute the kernel function for all vectors in the sets \(X\) and \(Y\). In oneDAL, this mode requires equal numbers of observations in both input tables: \(n = m\).

rowIndexX

\(0\)

Index i of the vector in the set \(X\) for the vectorVector computation mode.

rowIndexY

\(0\)

Index \(j\) of the vector in the set \(Y\) for the vectorVector or matrixVector computation mode.

rowIndexResult

\(0\)

Row index in the values numeric table to locate the result of the computation for the vectorVector computation mode.

\(k\)

\(1.0\)

The coefficient \(k\) of the linear kernel.

\(b\)

\(0.0\)

The coefficient \(b\) of the linear kernel.

Algorithm Output#

The linear kernel function calculates the results described below. Pass the Result ID as a parameter to the methods that access the results of your algorithm.

Algorithm Output for Linear Kernel (Batch Processing)#

Result ID

Result

values

Pointer to the \(n \times m\) numeric table with the values of the kernel function.

Note

By default, this result is an object of the HomogenNumericTable class, but you can define the result as an object of any class derived from NumericTable except PackedSymmetricMatrix, PackedTriangularMatrix, and CSRNumericTable.

Examples#

Radial Basis Function Kernel#

The Radial Basis Function (RBF) kernel is a popular kernel function used in kernelized learning algorithms.

Problem Statement#

Given a set \(X\) of \(n\) feature vectors \(x_1 = (x_{11}, \ldots, x_{1p}), \ldots, x_n = (x_{n1}, \ldots, x_{np})\) of dimension \(p\) and a set \(Y\) of \(m\) feature vectors \(y_1 = (y_{11}, \ldots, y_{1p}), \ldots, y_m = (y_{m1}, \ldots, x_{mp})\), the problem is to compute the RBF kernel function \(K(x_i,, y_i)\) for any pair of input vectors:

\[K\left({x}_{i},{y}_{j}\right)=exp\left(-\frac{{\left(\|{x}_{i}-{y}_{j}\|\right)}^{2}}{2{\sigma }^{2}}\right)\]

Batch Processing#

Algorithm Input#

The RBF kernel accepts the input described below. Pass the Input ID as a parameter to the methods that provide input for your algorithm.

Algorithm Input for Radial Basis Function Kernel (Batch Processing)#

Input ID

Input

\(X\)

Pointer to the \(n \times p\) numeric table that represents the matrix \(X\). This table can be an object of any class derived from NumericTable.

\(Y\)

Pointer to the \(m \times p\) numeric table that represents the matrix \(Y\). This table can be an object of any class derived from NumericTable.

Algorithm Parameters#

The RBF kernel has the following parameters:

Algorithm Parameters for Radial Basis Function Kernel (Batch Processing)#

Parameter

Default Value

Description

algorithmFPType

float

The floating-point type that the algorithm uses for intermediate computations. Can be float or double.

method

defaultDense

Available computation methods:

  • defaultDense - default performance-oriented method

  • fastCSR - performance-oriented method for CSR numeric tables

computationMode

matrixMatrix

Computation mode for the kernel function. Can be:

For CPU:

  • vectorVector - compute the kernel function for given feature vectors \(x_i\) and \(y_j\)

  • matrixVector - compute the kernel function for all vectors in the set \(X\) and a given feature vector \(y_j\)

  • matrixMatrix - compute the kernel function for all vectors in the sets \(X\) and \(Y\). In oneDAL, this mode requires equal numbers of observations in both input tables: \(n = m\).

For GPU:

  • matrixMatrix - compute the kernel function for all vectors in the sets \(X\) and \(Y\). In oneDAL, this mode requires equal numbers of observations in both input tables: \(n = m\).

rowIndexX

\(0\)

Index \(i\) of the vector in the set \(X\) for the vectorVector computation mode.

rowIndexY

\(0\)

Index \(j\) of the vector in the set \(Y\) for the vectorVector or matrixVector computation mode.

rowIndexResult

\(0\)

Row index in the values numeric table to locate the result of the computation for the vectorVector computation mode.

sigma

\(1.0\)

The coefficient \(\sigma\) of the RBF kernel.

Algorithm Output#

The RBF kernel calculates the results described below. Pass the Result ID as a parameter to the methods that access the results of your algorithm.

Algorithm Output for Radial Basis Function Kernel (Batch Processing)#

Result ID

Result

values

Pointer to the \(n \times m\) numeric table with the values of the kernel function.

Note

By default, this result is an object of the HomogenNumericTable class, but you can define the result as an object of any class derived from NumericTable except PackedSymmetricMatrix, PackedTriangularMatrix, and CSRNumericTable.

Examples#

Batch Processing: