Retraining Triggers
16 min
Retraining a model isn't free — it costs compute, time, and risk (a
newly retrained model could be worse). Automated retraining pipelines
usually check several independent signals rather than retraining
blindly on a calendar schedule: has the input data DRIFTED (from
monitoring)? Has live performance actually DROPPED? Is the model
simply STALE (too long since its last update, regardless of any other
signal)? Any one of these can justify a retrain.
def should_retrain(drift_score, performance_drop, days_since_training, drift_threshold=0.5, perf_threshold=0.05, max_days=30):
reasons = []
if drift_score > drift_threshold:
reasons.append("drift")
if performance_drop > perf_threshold:
reasons.append("performance_drop")
if days_since_training > max_days:
reasons.append("stale")
return reasons
Returning ALL matching reasons (not just the first one, or a single boolean) matters in practice -- an on-call engineer investigating an alert wants to know it's BOTH drifting AND stale, not just that 'something' triggered, since that changes how urgently and how they'd respond.
Write `should_retrain(drift_score, performance_drop, days_since_training, drift_threshold=0.5, perf_threshold=0.05, max_days=30)`: return a list of every reason retraining is warranted — `'drift'` if `drift_score > drift_threshold`, `'performance_drop'` if `performance_drop > perf_threshold`, `'stale'` if `days_since_training > max_days`. Return an empty list if none apply.
Why check MULTIPLE independent signals (drift, performance drop, staleness) instead of retraining on a fixed schedule alone?
You can combine multiple independent monitoring signals into a retraining decision, and explain why reactive multi-signal triggers beat a fixed retraining schedule.