a loss function ( cost, objective ) is one scalar saying how wrong the model is, which gradient descent pushes down by moving every parameter against ∂L/∂θ
the loss is the definition of the task for a neural network; most standard ones are a negative log likelihood under some noise model
scale matters: if targets or features live on very different ranges, the big ones dominate the loss, so normalize first
- squaring punishes big errors hard, so it's sensitive to outliers
- mean absolute error $|y - \hat{y}|$ is the robust version, it predicts the median instead of the mean