Optimizers#
This chapter describes optimizers implemented in oneDAL.
Newton-CG Optimizer#
The Newton-CG optimizer minimizes the convex function iteratively using its gradient and hessian-product operator.
Operation |
Computational methods |
|
Mathematical Formulation#
Computing#
The Newton-CG optimizer, also known as the hessian-free optimizer, minimizes convex functions without calculating the Hessian matrix. Instead, it uses a Hessian product matrix operator. In the Newton method, the descent direction is calculated using the formula \(d_k = -H_k^{-1} g_k\), where \(g_k, H_k\) are the gradient and hessian matrix of the loss function on the \(k\)-th iteration. The Newton-CG method uses the Conjugate Gradients solver to find the approximate solution to the equation \(H_k d_k = -g_k\). This solver can find solutions to the system of linear equations \(Ax = b\) taking vector \(b\) and functor \(f(p) = Ap\) as input.
Computation Method: dense#
The method defines the Newton-CG optimizer used by other algorithms for convex optimization. There is no separate computation mode to minimize a function manually.
Programming Interface#
Refer to API Reference: Newton-CG optimizer.