Concept drift

A change in the relationship between inputs and the outcomes a model is expected to predict.

Concept drift is different from a change in input frequency alone. Monitoring should distinguish a changed task definition, a changed operating environment and degraded model performance. Otherwise, a team may retrain a model when the real problem is an outdated label or rubric.

In practice

A revised definition of successful placement changes which outcomes count as success for the same visual evidence.