Objective Function¶
In oneDAL, the objective function represents an interface of objective functions \(K(\theta) = F(\theta) + M(\theta)\), where \(F(\theta)\) is a smooth and \(M(\theta)\) is a non-smooth functions, that accepts input argument \(\theta \in R^{p}\) and returns:
The value of objective function, \(y = K(\theta)\)
The value of \(M(\theta)\), \(y_{ns} = M(\theta)\)
The gradient of \(F(\theta)\):
\[g(\theta) = \nabla F(\theta) = \{ \frac{\partial F}{\partial \theta_1}, \ldots, \frac{\partial F}{\partial \theta_p} \}\]The Hessian of \(F(\theta)\):
\[\begin{split}H = = \nabla^2 F(\theta) = {\nabla }^{2}{F}_{i}=\left[\begin{array}{ccc}\frac{\partial {F}_{i}} {\partial {\theta }_{1}\partial {\theta }_{1}}& \cdots & \frac{\partial {F}_{i}} {\partial {\theta }_{1}\partial {\theta }_{p}}\\ ⋮& \ddots & ⋮\\ \frac{\partial {F}_{i}}{\partial p\partial {\theta }_{1}}& \cdots & \frac{\partial {F}_{i}}{\partial {\theta }_{p}\partial {\theta }_{p}}\end{array}\right]\end{split}\]The objective function specific projection of proximal operator (see [MSE, Log-Loss, Cross-Entropy] for details):
\[ \begin{align}\begin{aligned}\text{prox}_{\eta}^{M} (x) = \text{argmin}_{u \in R^p} (M(u) + \frac{1}{2 \eta} |u - x|_2^2)\\x \in R^p\end{aligned}\end{align} \]The objective function specific Lipschitz constant, \(\text{constantOfLipschitz} \leq |\nabla| F(\theta)\).
Objective functions
Note
On GPU, only Logistic Loss and Cross-entropy Loss are supported, Mean Squared Error Algorithm is not supported.