Cross-Validation
18 min
A single train/test split gives you ONE performance estimate — but that
estimate depends on which specific examples happened to land in the test
set. K-fold cross-validation splits the data into k equal-sized
folds, then repeats training/testing k times, each time using a
DIFFERENT fold as the test set:
def kfold_split_indices(n, k):
fold_size = n // k
folds = []
for i in range(k):
start = i * fold_size
end = start + fold_size if i < k - 1 else n
folds.append(list(range(start, end)))
return folds
print(kfold_split_indices(10, 5))
# [[0, 1], [2, 3], [4, 5], [6, 7], [8, 9]]
Real cross-validation implementations shuffle the data before splitting into folds (this simplified version doesn't, for determinism) -- without shuffling, any pattern in how the data happens to be ORDERED could bias individual folds.
Each fold's score gives an independent-ish estimate of performance — averaging them (and looking at their SPREAD, not just the average) tells you far more than one score would:
def cross_val_scores(y_true, y_pred, k):
folds = kfold_split_indices(len(y_true), k)
scores = []
for fold in folds:
correct = sum(1 for i in fold if y_true[i] == y_pred[i])
scores.append(correct / len(fold))
return scores
scores = cross_val_scores(y_true, y_pred, k=5)
print(f"mean: {sum(scores)/len(scores):.3f}, range: {min(scores):.3f}-{max(scores):.3f}")
If the fold scores are all close together, you can trust the average number. If they vary wildly (0.95 on one fold, 0.60 on another), that's a signal the model's performance is unstable — worth investigating BEFORE trusting a single "88% accuracy" headline number from just one split.
Write `kfold_split_indices(n, k)`: split indices `0..n-1` into `k` contiguous folds as evenly as possible (the last fold absorbs any remainder). Return a list of `k` lists of indices.
Given `kfold_split_indices` below, write `cross_val_scores(y_true, y_pred, k)`: split into `k` folds, and return a list of the ACCURACY computed within each fold separately, each rounded to 4 decimals.
What's the main purpose of k-fold cross-validation?
You can split data into k folds and compute per-fold performance, understanding why cross-validation gives a more trustworthy estimate than a single train/test split.