Contents

Linear Regression

Linear regression is a method for modeling the relationship between a dependent variable (which may be a vector) and one or more explanatory variables by fitting linear equations to observed data.

Details

Let \((x_1, \ldots, x_p)\) be a vector of input variables and \(y=(y_1, \ldots, y_k)\) be the response. For each \(j=1, \ldots ,k\), the linear regression model has the format [Hastie2009]:

\[y_j = \beta_{0j} + \beta_{1j} x_1 + \ldots + \beta_{pj} x_p\]

Here \(x_i\), \(i=1, \ldots,p\), are referred to as independent variables, and \(y_j\) are referred to as dependent variables or responses.

Linear regression is called:

  • Simple Linear Regression (if there is only one explanatory variable)

  • Multiple Linear Regression (if the number of explanatory variables \(p > 1\))

Training Stage

Let \((x_{11}, \ldots, x_{1p}, y_1, \ldots, x_{n1}, \ldots, x_{np}, y_n)\) be a set of training data, \(n \gg p\). The matrix \(X\) of size \(n \times p\) contains observations \(x_{ij}\), \(i=1, \ldots, n\), \(j = 1, \ldots, p\) of independent variables.

To estimate the coefficients \((\beta_{0j}, \ldots, \beta_{pj})\) one these methods can be used:

  • Normal Equation system

  • QR matrix decomposition

Prediction Stage

Linear regression based prediction is done for input vector \((x_1, \ldots, x_p)\) using the equation \(y_j = \beta_{0j} + \beta_{1j}x_1 + \ldots + \beta_{pj}x_p\) for each \(j=1, \ldots, k\).

Usage of Training Alternative

To build a Linear Regression model using methods of the Model Builder class of Linear Regression, complete the following steps:

  • Create a Linear Regression model builder using a constructor with the required number of responses and features.

  • Use the setBeta method to add the set of pre-calculated coefficients to the model. Specify random access iterators to the first and the last element of the set of coefficients [ISO/IEC 14882:2011 §24.2.7]_.

    Note

    If your set of coefficients does not contain an intercept, interceptFlag is automatically set to False, and to True, otherwise.

  • Use the getModel method to get the trained Linear Regression model.

  • Use the getStatus method to check the status of the model building process. If DAAL_NOTHROW_EXCEPTIONS macros is defined, the status report contains the list of errors that describe the problems API encountered (in case of API runtime failure).

Note

If after calling the getModel method you use the setBeta method to update coefficients, the initial model will be automatically updated with the new \(\beta\) coefficients.

Examples

previous

Linear and Ridge Regressions

next

Ridge Regression