Hypothesis Testing

A test begins by partitioning the parameter space into two mutually exclusive statements. The null carries the equality and is the hypothesis actually tested; the alternative captures what the analyst suspects and is accepted only if the data are sufficiently inconsistent with the null. Convention places the claim the researcher hopes to support in the alternative, so that rejecting the null requires evidence rather than mere absence of contradiction.

The test statistic is a standardized distance: the sample estimate minus the value assumed under the null, divided by the standard error of that estimate. Because the numerator is measured in units of sampling variability, the result can be read against a known sampling distribution. The decision rule then takes one of two equivalent forms. Either the statistic is compared with a critical value that bounds the rejection region for the chosen significance level, or the p-value, the probability of observing a statistic at least as extreme as the one obtained if the null were true, is compared with that significance level. Both routes deliver the same conclusion.

The choice of statistic follows from the parameter tested and the assumptions the analyst is prepared to make. Tests on a single population mean use a t-statistic when the population variance is unknown, which is the usual case; the degrees of freedom equal the sample size minus one. A z-statistic applies when the population variance is known, and is acceptable as an approximation for large samples from a non-normal population under the central limit theorem. Differences in means across two independent samples use a t-test, with the pooled-variance version assuming equal variances and the Welch-type adjustment when they differ; paired observations require a test on the mean difference rather than a difference in means.

Variance questions use different families. A test of a single population variance against a hypothesized value relies on the chi-square distribution, and a test of the equality of two variances uses the F-distribution formed as the ratio of the two sample variances. Correlation is tested with a t-statistic built from the sample correlation and the sample size. Where distributional assumptions are untenable, nonparametric alternatives such as the Spearman rank correlation test apply.

The significance level is not a passive setting; it is the analyst's chosen probability of a Type I error, rejecting a null that is in fact true. Tightening it to one percent from five percent shrinks the rejection region and reduces false positives, but mechanically raises the probability of a Type II error, failing to reject a false null. Power, defined as one minus the probability of a Type II error, is the test's ability to detect a genuine effect. For a fixed significance level, power rises with sample size, with the magnitude of the true departure from the null, and with the precision of the estimator.

This trade-off has direct consequences in investment work. Short return histories produce standard errors large enough that economically meaningful skill or spread differentials fail to reach conventional significance. The correct reading is low power, not evidence of no effect. Equally, very large samples can render trivial deviations statistically significant, which is why statistical significance and economic significance must be assessed separately, after transaction costs, taxes and capacity constraints.

Hypothesis testing underpins most of the inferential machinery elsewhere in the curriculum. Regression output is a battery of tests: each slope coefficient carries a t-statistic against a null of zero, the F-statistic tests joint significance of the slopes, and diagnostic procedures for serial correlation and heteroskedasticity are themselves formal tests. Time-series work tests unit roots before a model can be trusted; portfolio analysis tests whether two strategies' mean returns or return variances differ.

At Level I the practical requirements are specific. Formulate the null and alternative correctly for a stated claim, identify the appropriate statistic and its degrees of freedom, compute the standardized value, locate the critical value in a table for a one- or two-tailed test, state the decision, and express the conclusion in the language of rejecting or failing to reject rather than proving. Candidates should also be able to move fluently between the critical-value and p-value approaches and to relate a two-tailed test to the corresponding confidence interval, which excludes the hypothesized value precisely when the test rejects.