Quality score

A measure of how well an item or dataset meets defined quality criteria.

A score is meaningful only with its rubric, scale and aggregation rules. It may describe completeness, label correctness or another dimension. Keep it separate from a consensus score, which describes agreement among reviewers. A high average can hide failure on a required condition.

In practice

An item receives a low quality score with high reviewer agreement because all reviewers identify the same labeling error.