Scaling
18 min
Features on wildly different scales (income in dollars: 20,000-500,000; age in years: 18-90) cause problems for many models — gradient descent (from the Optimization course) takes wildly different step sizes to matter across such different ranges, and distance-based methods let the larger-magnitude feature dominate purely because of its units, not its actual importance.
Min-max scaling rescales every value into [0, 1]:
import pandas as pd
ages = pd.Series([18, 35, 90])
scaled = (ages - ages.min()) / (ages.max() - ages.min())
print(scaled.tolist()) # [0.0, 0.236..., 1.0]
Min-max scaling is sensitive to outliers -- one extreme value stretches the whole [0,1] range, squeezing every other value toward one end.
Standardization (z-score scaling) instead centers values around a mean of 0 with a standard deviation of 1 — less sensitive to outliers than min-max scaling, and the standard choice for algorithms that assume roughly normally-distributed input:
values = pd.Series([2, 4, 4, 4, 5, 5, 7, 9])
standardized = (values - values.mean()) / values.std()
print([round(v, 2) for v in standardized])
This is exactly the same z-score formula from the Statistics course's Hypothesis Testing module — feature scaling and hypothesis testing are using the identical underlying idea: "how many standard deviations away from the mean is this value," just applied to a different purpose (model input vs. statistical significance).
Write `min_max_scale(series)`: rescale a pandas Series to the range [0, 1] via `(x - min) / (max - min)`. Return a list of values, each rounded to 4 decimals.
Write `standardize(series)`: z-score standardize a pandas Series via `(x - mean) / std` (use `.std()`'s default). Return a list of values, each rounded to 4 decimals.
Why does feature scaling matter for algorithms like gradient descent or k-nearest-neighbors, but not for a decision tree?
You can rescale a feature via min-max scaling (bounded [0,1]) or standardization (mean 0, std 1), and know which model types actually need this step.