What is overfitting, and what are three techniques to prevent it?

technical-conceptual · Junior level · data-ml

What the interviewer is really asking

Check foundational knowledge of model generalisation and practical mitigation strategies.

What to say

• Overfitting occurs when a model learns noise or details specific to the training set, causing poor performance on unseen data.
• Three common remedies: (1) regularisation (L1/L2), (2) cross-validation to detect it early, (3) reducing model complexity or using dropout in neural networks.
• Getting more training data and applying early stopping are also effective.

What to avoid

• Saying overfitting only applies to neural networks — it affects any sufficiently complex model.
• Proposing to fix overfitting by simply removing features randomly without analysis.
• Conflating overfitting with data leakage.

Example answers

Strong: When training a decision tree on a small dataset I noticed training accuracy was 98% and test accuracy was 61%. I added max_depth=5 and min_samples_leaf=10 constraints, which brought test accuracy up to 83%.

Weak: Overfitting means the model is confused by noisy labels, so I'd just clean the data and retrain.