Concept Lab · Week 6 · Machine Learning
Bias–variance tradeoff
Raise model complexity and watch train error fall while test error turns back up.
The idea: Bias–variance, overfitting and underfitting
Expected test error decomposes into bias² (error from wrong assumptions: a line fit to a curve), variance (error from sensitivity to the particular training sample) and irreducible noise.
As model complexity grows, training error keeps falling but test error falls, bottoms out, then rises: the model starts fitting noise. That U-shape is the central picture of classical ML. Slide the complexity in the simulation and watch it happen.
Remedies for high variance: more data, regularisation, simpler models, bagging, early stopping. For high bias: richer features, more flexible models, boosting. Learning curves (error vs training-set size) tell you which problem you have.
Next simulation: Decision tree splits