Multiple Linear Regression
Multiple linear regression extends simple regression to model the response as a linear function of several predictors, using matrix algebra for a compact and general formulation.
The Matrix Formulation
Definition
The multiple linear regression model is where is the response vector, is the design matrix (with rows ), is the parameter vector, and .
Definition
The OLS estimator minimizes and is given by provided is invertible (i.e., has full column rank ). The fitted values are , where is the hat matrix.
Properties
ExampleDistribution of the OLS estimator
Under the normal model : where is the residual vector and .
Moreover, and are independent.
Inference
RemarkTesting individual coefficients
To test (whether predictor contributes after accounting for all other predictors), use under . The overall F-test uses .