Simple Linear Regression

The slope estimate has a compact interpretation: it is the sample covariance between the independent and dependent variables divided by the sample variance of the independent variable. Equivalently, it equals the correlation coefficient multiplied by the ratio of the standard deviation of Y to the standard deviation of X. This is why a strong correlation does not by itself imply a steep slope; the scaling of the two variables matters just as much as the strength of association.

Once the slope is known, the intercept follows mechanically: it is the mean of Y minus the slope times the mean of X. That relationship guarantees the fitted line passes through the point defined by the two sample means, and it forces the residuals to sum to zero. Candidates should be comfortable moving between correlation, standard deviations and the slope without a regression output in front of them.

Total variation in the dependent variable, measured as the sum of squared deviations of each observation from the mean of Y, splits cleanly into two pieces. The explained portion is the sum of squared deviations of the fitted values from the mean of Y; the unexplained portion is the sum of squared residuals. R-squared is the explained sum of squares divided by the total, bounded between zero and one in a regression with an intercept.

The standard error of the estimate summarises typical residual size in the units of Y. It is the square root of the sum of squared residuals divided by degrees of freedom, which in simple linear regression equals the number of observations minus two, since two parameters have been estimated. A small standard error of the estimate signals tight clustering around the line; R-squared, being unit-free, measures relative rather than absolute fit.

Inference on the slope uses a t-statistic: the estimated slope minus the hypothesised value, divided by the standard error of the slope coefficient, compared against a critical value with degrees of freedom equal to observations minus two. The usual null is a slope of zero, but the same machinery tests economically meaningful hypotheses, such as whether a beta differs from one. The ANOVA table delivers an F-statistic, the mean regression sum of squares over the mean squared error; in simple regression it is simply the square of the slope t-statistic and tests the identical hypothesis.

Forecasting requires the standard error of the forecast, which exceeds the standard error of the estimate because it accounts for uncertainty in the estimated coefficients. Prediction intervals therefore widen as the chosen value of X moves further from the sample mean of X, and extrapolation far beyond the observed range is unreliable.

Valid inference rests on four conditions: the underlying relationship is linear in the parameters, the error variance is constant across observations, errors are uncorrelated with one another, and errors are normally distributed. Heteroskedasticity and serial correlation leave the coefficient estimates unbiased but corrupt the standard errors, so t-statistics and F-statistics become unreliable. Residual plots against fitted values or against time are the standard diagnostic; a funnel shape suggests non-constant variance, a wave pattern suggests autocorrelation.

In practice, simple linear regression underpins beta estimation, hedge ratio construction, expense-to-revenue modelling in financial statement analysis, and yield spread relationships in fixed income. A candidate should be able to read a regression output, extract the coefficients and their standard errors, compute R-squared from an ANOVA table, form a confidence interval for a forecast, and state precisely what the residuals represent.