Residual

Ordinary least squares treats the fitted line as the one that makes the total squared vertical distance from the observed points as small as possible. Setting the derivatives of that sum with respect to the intercept and the slope to zero produces two conditions: the residuals must sum to zero, and the residuals must be uncorrelated with the independent variable in the sample. Solving them gives the slope as the sample covariance between the independent and dependent variables divided by the variance of the independent variable, and the intercept as the mean of the dependent variable less the slope times the mean of the independent variable.

A consequence worth internalising is that the fitted line always passes through the point defined by the two sample means. Squaring, rather than taking absolute values, is what makes the solution closed-form and what makes large misses disproportionately costly to the fit.

Every summary measure of regression quality is built from residuals. The sum of squared residuals is the unexplained variation; total variation is the sum of squared deviations of the dependent variable from its own mean; the difference is the explained variation. The coefficient of determination is one minus the ratio of unexplained to total variation, so a model whose residuals are small relative to the spread of the dependent variable scores high.

The standard error of the estimate is the square root of the sum of squared residuals divided by the degrees of freedom, which for a simple regression is the number of observations less two. It is expressed in the units of the dependent variable and answers a practical question: how far, typically, does a prediction miss? It also feeds the standard errors of the estimated coefficients and therefore every t-statistic and confidence interval the regression produces.

Analysts inspect residuals before trusting a coefficient. Plotted against the independent variable or against time, residuals should show no discernible pattern. A funnel shape signals heteroskedasticity, common when regressing levels of financial variables across firms of very different size; the coefficient estimates remain unbiased but their standard errors are wrong, so significance tests mislead. A wave-like pattern over time indicates serial correlation, endemic in time-series regressions of returns, spreads or macro series, and it typically understates standard errors on positively autocorrelated data. Curvature suggests the functional form is wrong rather than the data being noisy.

Residuals also carry economic content in their own right. In event studies the abnormal return is a residual from a benchmark model, and the volatility of residuals from a factor regression is the basis for tracking error and for the non-systematic risk component of a portfolio.

Expect to compute a single residual by inserting a value of the independent variable into the estimated equation and subtracting the fitted value from the observed one, taking care with the sign convention: observed minus predicted, so a positive residual means the model under-predicted. Expect also to move between the components of the analysis of variance table, deriving the coefficient of determination or the standard error of the estimate when given two of the three sums of squares and the sample size.

Beyond arithmetic, be prepared to interpret a residual plot and name the assumption it violates, to state the consequence for inference rather than for the point estimates, and to keep the conceptual line clear between the unobservable population disturbance and its observable sample counterpart.