Learn · Context management & reliability · CCAR-F task 5.5

Human review and calibration

Route outputs to people by confidence and risk, then check that stated confidence matches real accuracy with stratified samples.

Key ideas

• Route by confidence and risk: high-confidence, low-risk outputs pass; low confidence or high impact go to a person.
• Code-checked validation failures always go to review, whatever the stated confidence.
• Calibrate: have humans label a sample and compare stated confidence with actual accuracy. Set thresholds from that data.
• Use stratified sampling across document types and confidence bands, so rare but risky categories get reviewed.
• Give reviewers the source, the extracted value and the reason for flagging, so review is fast.

Worked example: Confidence routing, then weekly calibration

Each week humans label a stratified sample (e.g. 20 items from every document type × confidence band). If items marked 0.9 are only 70% accurate, raise the threshold.

Exam traps

• Trusting a stated confidence score without measuring it against human-labelled outcomes.
• Random-only audit samples that rarely touch rare, high-risk categories.
• Letting high confidence auto-approve a high-impact item; impact should route to review on its own.
• Sending reviewers a bare flag without the source and the reason, which makes review slow.

Official docs

• Define success criteria and build evaluations

Practise CCAR-F questions · Next lesson: Provenance and uncertainty