- simple linear regression: model a response Y as a straight line in one predictor X, the simplest case of linear regression
Y=β0+β1X+ϵ
- $\beta_0$ intercept: predicted $Y$ at $X = 0$
- $\beta_1$ slope: expected change in $Y$ per one unit of $X$
- example: predicting material strength from temperature, $\beta_1 = 2$ means each extra 1°C adds 2 units of strength on average
- $\epsilon$ error: noise and everything the line leaves out
- with $k$ predictors it's multiple regression, $Y = \beta_0 + \beta_1 X_1 + \dots + \beta_k X_k + \epsilon$, each slope read with the others held fixed
β^1=∑(xi−xˉ)2∑(xi−xˉ)(yi−yˉ),β^0=yˉ−β^1xˉ
- ANOVA table: split the variation SST=SSR+SSE and test the model as a whole
| source | SS | df | MS | F |
|---|
| regression | SSR | k | MSR=kSSR | F=MSEMSR |
| residual | SSE | n−(k+1) | MSE=n−(k+1)SSE | |
| total | SST | n−1 | | |
- $H_0$: every slope is 0, $X$ says nothing about $Y$ ( [[Null Hypothesis]] ); $H_1$: at least one slope isn't ( [[Hypothesis Testing]] )
- with one predictor ( $k = 1$ ) the $F$ test is the $t$ test on $\hat{\beta}_1$, $F = t^2$
- checking the fit
- R2=SSR/SST=1−SSE/SST: share of the variation in Y the line explains, near 1 is a tight fit
- adjusted R2 penalizes extra predictors, only matters in multiple regression
- each coefficient gets its own p-value, below 0.05 is the usual “significant”
- assumptions: linear relationship, independent observations, plus constant variance and roughly normal errors for the tests to hold