Human escalation

Routing an item to human reviewers when automated review cannot resolve it under the agreed rules.

Escalation should specify the evidence reviewers receive, the rubric they apply and how their judgments reach a final result. It is different from random spot checking: escalation responds to a particular unresolved case. The relevant human consensus requirements should be agreed for the dataset.

In practice

Judges disagree about whether a release is visible, so experts review the sequence and resolve it against the rubric.