The Gauss-Markov Theorem
1900
- Carl Friedrich Gauss
- Andrey Markov
This theorem states that in a linear regression model where the errors have zero mean, are uncorrelated, and have constant variance (homoscedasticity), the ordinary least squares (OLS) estimator is the Best Linear Unbiased Estimator (BLUE). ‘Best’ means it has the minimum variance among all linear unbiased estimators of the regression coefficients, making it the most precise.
The Gauss-Markov theorem is a central result in the theory of linear regression that gives OLS its strong theoretical appeal. It guarantees that if a specific set of assumptions holds, no other linear and unbiased estimator will be more efficient than OLS. Let’s break down the BLUE acronym. ‘Linear’ means the estimator for the coefficients is a linear combination of the observed dependent variable values. ‘Unbiased’ means that on average, the estimator will yield the true population parameter; its expected value is the true value, \(E(\hat{\beta}) = \beta\). ‘Best’ signifies that the OLS estimator has the minimum variance in its sampling distribution compared to any other linear unbiased estimator.
The core assumptions, known as the Gauss-Markov assumptions, are: 1. The model is linear in parameters. 2. The errors have a conditional mean of zero (\(E(\varepsilon | X) = 0\)). 3. The independent variables are not perfectly collinear. 4. The errors are homoscedastic (have constant variance, \(Var(\varepsilon | X) = \sigma^2\)) and are not autocorrelated (\(Cov(\varepsilon_i, \varepsilon_j | X) = 0\) for \(i \neq j\)).
Crucially, the theorem does not require the errors to be normally distributed. The assumption of normality is added later when one wants to perform exact finite-sample hypothesis tests (like t-tests and F-tests) on the coefficients. When the Gauss-Markov assumptions are violated (e.g., in the presence of heteroscedasticity or autocorrelation), OLS is no longer BLUE, and alternative estimators like Generalized Least Squares (GLS) may be more efficient.
UNESCO Nomenclature: 1209
– Statistics
Precursors
- Method of least squares (Gauss)
- Probability theory (concepts of expectation and variance)
- Linear algebra and matrix theory
- Early work on the theory of estimation
Applications
- providing the theoretical justification for using OLS in many practical scenarios
- serving as a foundation for statistical inference (confidence intervals, hypothesis tests) in linear models
- acting as a theoretical benchmark for comparing the efficiency of other, more complex estimators
Potential Innovations Ideas
Due to scrapping bot traffic, currently more than 40k per day, this content is reserved to community members.
> Login < or > Register < (100% free) to access this, so as all other restricted content and tools.
Related to: gauss-markov theorem, BLUE, best linear unbiased estimator, OLS, homoscedasticity, uncorrelated errors, minimum variance, statistical inference, linear model assumptions, econometrics.