Screenshot 2024-09-17 at 10.22.28 AM.png

  • normalization ( feature scaling ) puts numeric features on a common scale so none wins just because its units are bigger, a standard machine learning preprocessing step
    • example: house size in square feet ( ~1000s ) next to bedroom count ( 1 to 5 ); unscaled, every distance and gradient is about square feet
    • for gradient descent, unequal scales stretch the loss surface into a long thin valley and steps zigzag; scaled, it’s rounder and converges faster
  • min-max, to ; simple, but one outlier squashes everything else
  • z-score ( standardization ), mean 0 and standard deviation 1; the usual default, but still pulled by outliers since and are
    • robust scaling ( median and IQR ) when outliers are bad
  • max-abs, to ; keeps sign and zeros ( good for sparse data ), outlier sensitive
Python