Objective Function¶

In oneDAL, the objective function represents an interface of objective functions $$K(\theta) = F(\theta) + M(\theta)$$, where $$F(\theta)$$ is a smooth and $$M(\theta)$$ is a non-smooth functions, that accepts input argument $$\theta \in R^{p}$$ and returns:

• The value of objective function, $$y = K(\theta)$$

• The value of $$M(\theta)$$, $$y_{ns} = M(\theta)$$

• The gradient of $$F(\theta)$$:

$g(\theta) = \nabla F(\theta) = \{ \frac{\partial F}{\partial \theta_1}, \ldots, \frac{\partial F}{\partial \theta_p} \}$
• The Hessian of $$F(\theta)$$:

$\begin{split}H = = \nabla^2 F(\theta) = {\nabla }^{2}{F}_{i}=\left[\begin{array}{ccc}\frac{\partial {F}_{i}} {\partial {\theta }_{1}\partial {\theta }_{1}}& \cdots & \frac{\partial {F}_{i}} {\partial {\theta }_{1}\partial {\theta }_{p}}\\ ⋮& \ddots & ⋮\\ \frac{\partial {F}_{i}}{\partial p\partial {\theta }_{1}}& \cdots & \frac{\partial {F}_{i}}{\partial {\theta }_{p}\partial {\theta }_{p}}\end{array}\right]\end{split}$
• The objective function specific projection of proximal operator (see [MSE, Log-Loss, Cross-Entropy] for details):

\begin{align}\begin{aligned}\text{prox}_{\eta}^{M} (x) = \text{argmin}_{u \in R^p} (M(u) + \frac{1}{2 \eta} |u - x|_2^2)\\x \in R^p\end{aligned}\end{align}
• The objective function specific Lipschwitz constant, $$\text{constantOfLipschitz} \leq |\nabla| F(\theta)$$.

Note

On GPU, only Logistic Loss and Cross-entropy Loss are supported, Mean Squared Error Algorithm is not supported.