Data & Machine Learning
Model Evaluation
9 practice questions. Free questions open a full answer guide; the rest unlock with Pro.
- Why do we split data into training, validation, and test sets, and what is each one used for?
- Your classifier reports 97% accuracy but stakeholders say it's useless for catching fraud. What's going on, and how would you evaluate it properly?Go Pro
- You're evaluating a fraud classifier where only about 1% of transactions are fraudulent. The team is reporting 99% accuracy. What's wrong with leading on that metric, and which metrics would you actually use to judge the model?Go Pro
- Walk me through the precision-recall tradeoff and how you'd pick a decision threshold for a binary classifier in a real product.Go Pro
- Once you've trained a binary classifier that outputs probabilities, how do you choose the decision threshold for production, and when do the raw probabilities themselves need attention before you can trust that threshold?Go Pro
- Explain the difference between precision and recall. When would you prioritise one over the other?Go Pro
- You've built a recommendation model and its offline ranking metrics look great, but a colleague warns that offline numbers won't predict the online lift. How do you evaluate a model whose own decisions shape the data it's scored on, and reason about why offline and online results diverge?Go Pro
- How do you decide which evaluation metric to use when presenting a model to a business stakeholder?Go Pro
- A stakeholder asks you to add five new features to the model next sprint. How do you evaluate whether to include them?Go Pro
Want questions matched to your role? Paste a job title, job description, or CV for a personalized set, or go Pro to unlock the full bank.