Ordinary least squares treats every observation as equally informative. When some observations are noisier than others — heteroskedasticity, Var(ε_i) = σ_i² varying with i — that even-handedness is wasteful. OLS stays unbiased and consistent, but it is no longer efficient (not BLUE) and its textbook standard errors are wrong.
Weighted least squares re-weights the fit:
β̂_WLS = argmin_β Σ_i w_i (y_i − x_i'β)² = (X'WX)⁻¹ X'Wy, W = diag(w_1, …, w_n)With w_i = 1/σ_i², the observations you can trust more (smaller variance) pull harder on the fit. Mechanically it is just OLS on the transformed data √w_i·(y_i, x_i), and it is the diagonal special case of generalized least squares (GLS). Common weight choices:
| situation | weight |
|---|---|
a mean of n_i underlying units |
w_i = n_i (variance ∝ 1/n_i) |
| a known standard error per point (e.g. a second stage run on estimates) | w_i = 1/SE_i² |
variance a known function of x |
w_i = 1/σ²(x_i) |
| variance unknown | estimate it first (FGLS), then weight |
Done right, WLS restores efficiency and gives correct inference under heteroskedasticity. But it is routinely asked to do things it cannot:
- It does not fix bias. WLS still requires the mean model to be correct. Omitted variables, simultaneity, endogeneity — reweighting touches none of them. A weighted regression of a sorted outcome on a self-selected regressor is exactly as non-causal as the unweighted one. Efficiency is not identification.
- It does not rescue a generated dependent variable. If
y_iis itself an estimate from a prior step, inverse-variance weighting correctly handles the differing precision — but it does not repair the downward bias in standard errors that comes from ignoring first-stage estimation uncertainty. That needs a separate correction (a bootstrap, or an explicit errors-in-variables treatment). - Wrong weights can make things worse. Badly misspecified weights can leave WLS less efficient than plain OLS. When the weights are estimated rather than known, pair WLS with heteroskedasticity-robust (HC) standard errors as a guard.
The clean mental model: heteroskedasticity is a variance problem, and WLS is a variance tool. Endogeneity is a mean / identification problem, and no choice of weights will solve it. Reach for WLS to make honest estimates more precise — never to make a biased estimate causal.