Platform Guides

Quality Assurance

Measure annotation quality with QA metrics, agreement scores, gold tasks, and alerts.

Overview

Quality assurance combines several signals: QA metrics summarize accuracy over time; inter-annotator agreement (IAA) measures how consistently annotators label the same item; gold tasks are known-answer items mixed into the queue; and quality alerts fire when a metric crosses a threshold.

QA metrics

GET/api/v1/qa-metricsList QA metrics, filterable by project and date range.

Inter-annotator agreement

GET/api/v1/inter-annotator-agreementRead agreement scores for a task or project.
GET/api/v1/annotator-reliabilityPer-annotator reliability scores.

Gold tasks & confidence

Gold tasks calibrate annotators against a known answer; confidence scoring estimates how trustworthy each submitted annotation is.

GET/api/v1/gold-tasksList gold (known-answer) tasks.
POST/api/v1/gold-tasksCreate a gold task.
GET/api/v1/confidence-scoringRetrieve confidence scores for annotations.

Quality alerts & sampling

GET/api/v1/quality-alertsList quality alerts.
POST/api/v1/samplingConfigure review sampling rules.

See the full schema and parameters in the Label Backend API reference.