Objective Function

In oneDAL, the objective function represents an interface of objective functions \(K(\theta) = F(\theta) + M(\theta)\), where \(F(\theta)\) is a smooth and \(M(\theta)\) is a non-smooth functions, that accepts input argument \(\theta \in R^{p}\) and returns:

• The value of objective function, \(y = K(\theta)\)

• The value of \(M(\theta)\), \(y_{ns} = M(\theta)\)

• The gradient of \(F(\theta)\):

  \[g(\theta) = \nabla F(\theta) = \{ \frac{\partial F}{\partial \theta_1}, \ldots, \frac{\partial F}{\partial \theta_p} \}\]

• The Hessian of \(F(\theta)\):

  \[\begin{split}H = = \nabla^2 F(\theta) = {\nabla }^{2}{F}_{i}=\left[\begin{array}{ccc}\frac{\partial {F}_{i}} {\partial {\theta }_{1}\partial {\theta }_{1}}& \cdots & \frac{\partial {F}_{i}} {\partial {\theta }_{1}\partial {\theta }_{p}}\\ ⋮& \ddots & ⋮\\ \frac{\partial {F}_{i}}{\partial p\partial {\theta }_{1}}& \cdots & \frac{\partial {F}_{i}}{\partial {\theta }_{p}\partial {\theta }_{p}}\end{array}\right]\end{split}\]

• The objective function specific projection of proximal operator (see [MSE, Log-Loss, Cross-Entropy] for details):

  \[ \begin{align}\begin{aligned}\text{prox}_{\eta}^{M} (x) = \text{argmin}_{u \in R^p} (M(u) + \frac{1}{2 \eta} |u - x|_2^2)\\x \in R^p\end{aligned}\end{align} \]

• The objective function specific Lipschitz constant, \(\text{constantOfLipschitz} \leq |\nabla| F(\theta)\).

Objective functions

• Computation
• Sum of Functions
• Mean Squared Error Algorithm
• Objective Function with Precomputed Characteristics Algorithm
• Logistic Loss
• Cross-entropy Loss

Note

On GPU, only Logistic Loss and Cross-entropy Loss are supported, Mean Squared Error Algorithm is not supported.