Least Squares Fit

Description

The Least Squares Fit performs an ordinary least squares regression based on user-specified variables and options. The algorithm will fit an equation to the data of the form:

y = a + b₁x₁
+ b₂x₂
+ b₃x₃
+ ... +

The least squares fit algorithm will output the determined coefficients, a and b, as well as the estimation errors associated with each. Data associated with the regression, including the covariance matrix and the residuals, can be viewed after the regression is performed.

Usage

To perform a least squares regression, the appropriate data must be chosen. To select a variable as either dependent or independent, click the check box associated with each in the least squares window. Only one variable may be chosen as the dependent variable.

Compute Regression Menu

After selecting the desired options from the "Options" menu, the regression can be performed by selecting "Compute Regression..." from the "Perform Fit" menu. Once performed, a text window will open, showing the results of the regression.

After performing the regression, the independent and dependent check boxes in the least squares window will no longer be changeable. Also, the "Options" menu associated with this regression will be deactivated. The results window can be closed at any time; to reopen the results, simply select the "Compute Regression..." menu item again.

Options

Variances

The least squares fit allows for two types of variances to be computed for output. The Ordinary Least Squares variances are determined from the product of the variance of the residuals and the solution matrix, (X'X)^-1. The White's Robust Variance Estimates compute the variances based on a more conservative equation.

Extended Output

This menu allows for the selection of extended statistics to be printed. Currently, the only available option is the Durbin-Watson Statistic for time-series analysis.

Autocorrelation

This menu contains the necessary controls for performing a least squares fit accounting for autocorrelation. See the Least Squares Fit (Autocorrelation) page for more information.

Include Constant

By unchecking this box, the constant term is omitted from the regression procedure.

Ignore Matrix Condition

Selecting this option disables the safety check for matrix conditioning prior to attempting to solve the least-squares problem. More information can be found on the Condition page. The solution matrix tends to appear poorly condition when colinearity exists in series. The consequence of proceeding when conditioning is poor can range from highly-biased results to complete software failure.

Output

The least squares fit will generate a table of coefficient values as well as the errors associated with each. The results of the fit, including the R² value are also output. An example from a least squares fit is shown below:

Results of the Multiple Regression Model
Regression Variable: C
---------------------------------------------------------------
Sum Squared of the Residuals:    2.35179
Standard Error of the Fit   :    0.38339
R-Squared Value             :    0.97216
Adjusted R-Squared Value    :    0.96868
---------------------------------------------------------------
Constant:   -3.31321    Std Err:    0.75312    t-Score:    -4.3993
Coef 0 (G):    3.97754    Std Err:    0.24808    t-Score:   16.03313
Coef 1 (X):   -0.11972    Std Err:    0.26913    t-Score:   -0.44484
---------------------------------------------------------------
Extended Statistics:

Further information can be generated after the regression. By selecting "Generate Column Data" from the "Perform Fit" menu, the estimated dependent variable data will be output as a new variable in the Data window. The resulting covariance matrix can be viewed by selecting "View Covariance Matrix..." from the "Supporting Data Menu." The residuals from the regression can also be output to the Data window as a new variable by selecting "Output Residuals to Column" from the "Supporting Data Menu."

Method

The Least Squares Fit uses the LAPACK DGELS function to solve the over-specified problem directly. From the LAPACK comments, this function performs the following:

minimize || B - A*X ||

In the above equation, the A matrix contains all independent data, B represents the dependent data, and X represents the coefficients being determined.

To determine the covariance matrix, the least squares process is broken down into components. The transpose of the independent data matrix is multiplied by the independent data matrix, and the variance of the residuals is multiplied by the inverse of this resulting matrix product (although this procedure changes for White's Robust Variance Estimates). The resulting equation is:

cov = var()*(A'A)

All computations are performed using the appropriate LAPACK calls. The variances of each coefficient are taken from the covariance matrix's diagonal.