Model drift

A change in a model's performance or behavior over time relative to the task it is meant to perform.

Changes in data, the task or the system can make earlier evaluation results less representative. A fixed model does not need to change its weights for its performance to deteriorate. Monitor against suitable reference judgments and investigate the cause before deciding to replace or update it.

In practice

A judge that performed well on clear tabletop clips becomes less reliable on new camera views with frequent occlusion.