Main Concept
Evaluating model fit assesses how well a model generalizes, which means its ability to make accurate predictions or inferences on new, unseen data, rather than simply performing well on the data it was trained on.
Context
When we evaluate a model, we want it to perform well on new data or new questions it has never encountered before we put it into production
Key Aspects
We can have three different situations when we measure models fit:
- Overfitting: The model performs well on training data BUT poorly on new data, the mode “memorize” instead of learning
- Under fiting: The model performs poorly on BOTH training and new data, the model didn’t learn enough. Could be a problem of having a model too simple or poor data features.
- Good fit (Balanced): The model performs well on training AND new data, the model learned the right patterns.
Examples
- Model performs well in testing but poorly in production” → overfitting
- “Model performs poorly even on training data” → underfitting
Exam Domains
This appears in Domain 1, Task Statement 1.1 — “bias, fairness, fit” as basic AI terms — and Domain 4, Task Statement 4.1 — “understand effects of bias and variance (for example, overfitting, underfitting).”
Related Concepts:
Links:
References