Call Center Outsourced evidence brief · Desk review · Published
Disposition-code agreement across reviewers
A protocol for measuring whether two reviewers assign the same disposition code to the same call record.
On this page
Key stats
- 2 independent reviewers
- 1 codebook
- 0 performance rankings
Key takeaways
- Measure agreement by code, not only as one overall percentage.
- Keep reviewers independent until both classifications are recorded.
- Treat repeated disagreement as a codebook signal before blaming reviewers.
Question and scope
This protocol asks how often independent reviewers agree on the disposition code for the same call. The population is closed calls sampled from a declared window. The unit is one call with its record, note, code, and two independent classifications.
Codebook version
Record the codebook version and the definitions used by both reviewers. If the codebook changes during the review, keep the earlier classification and apply the new rule only to later cases.
Frame and sampling
Build the frame from the call system. Include ordinary calls, transfers, complaints, and abandoned calls. Use consecutive or systematic sampling and report exclusions and unclassifiable cases.
Reviewer independence
Each reviewer classifies the call before seeing the other's result. Record the code, the confidence, and the reason. Do not discuss individual cases until both classifications are recorded.
Measures
Report exact agreement, agreement within a code family, and disagreement by code. Include the share of calls with a missing note. A high agreement rate on easy calls can hide disagreement on sensitive ones.
Calibration
Where reviewers disagree, return to the codebook. If the definition is ambiguous, record the ambiguity rather than forcing a consensus. A codebook repair is a better outcome than a majority vote.
Privacy and access
Use stable call keys and omit caller identifiers. Keep recordings and notes in the approved system with named access. Report any redactions.
Limits and next step
Agreement does not prove correctness; two reviewers can share the same misreading. The next step is a codebook clarification and a second sample.
Put this into a support lane
Bring one queue, its routing log, and the manager who owns the promise.
Review the queueRelated operating guides
FAQs
Does high agreement prove the code is correct?
No. It shows the reviewers applied the codebook consistently. Correctness still depends on the codebook and the recorded facts.
How should ambiguous calls be counted?
As a separate unclassifiable state with the reason. Do not force them into a code.
What if the codebook changes mid-review?
Keep the earlier classification and apply the new rule only to later cases.