On this page
- Evidence snapshot
- Define the codebook and the sampling unit
- Compare the code with the call evidence
- Data and decision boundary
- Separate coding from policy judgment
- Use reason codes for disagreements
- Calibrate reviewers on a protected set
- Feed findings back into the codebook
- Copy-ready call and handoff lines
- Questions managers ask
- Keep planning
- Sources
Disposition codes drive reporting, staffing, and follow-up. When the code drifts from the call, the report looks precise while the operation loses the ability to explain itself. This guide defines a sampling routine that checks code-to-call agreement, preserves the original note, and routes policy questions to the owner.
Evidence snapshot
Define the codebook and the sampling unit
Use the current codebook and record its version. The sampling unit is one closed interaction with a disposition code, a call record, an agent note, and a reviewer. Freeze the window and the inclusion rules before drawing the sample.
Include ordinary calls, transfers, abandoned calls, callbacks, and complaints. Exclude a category only when the codebook says it is out of scope, and report the exclusion. A sample of easy calls can hide the exact ambiguity the review is meant to find.
Compare the code with the call evidence
For each sampled call, compare the code with the call record, the agent note, the transfer history, and the customer outcome. Record agreement, conflict, or unclassifiable. Do not listen to a recording when a written record is enough; protect caller privacy and reviewer time.
When the note and the code disagree, preserve both and describe the difference. Do not edit the note to match the code. The disagreement is the finding, and it may point to a codebook problem rather than an agent problem.
Data and decision boundary
Use this table as a starting point, then match each row to the client's tools and call guide. The manager column stays outside the team member's normal authority.
| Data or request | Team member can | Manager keeps |
|---|
Separate coding from policy judgment
A reviewer can say that the code does not match the recorded facts. A reviewer should not decide whether a refund was warranted, whether a complaint was valid, or whether a manager should have approved an exception. Those are policy decisions that belong to the operations owner.
Write the boundary into the sampling brief. If a call raises a policy question, mark it for owner review and keep the code assessment separate. A neutral finding is easier to act on than a borrowed conclusion.
Use reason codes for disagreements
Define a short list of disagreement reasons: wrong category, missing note, conflicting note, transfer not recorded, customer outcome unclear, and codebook ambiguous. Use the same list across reviewers so the pattern is comparable.
Report the distribution by code, queue, agent group, and reviewer. A high disagreement rate in one code may mean the definition is unclear. A high rate for one reviewer may mean the reviewer is applying a different standard; check calibration before drawing a conclusion.
Calibrate reviewers on a protected set
Give two reviewers the same protected sample, the same codebook, and the same definitions. Compare their classifications before the full review. Discuss disagreements against the codebook, not against personal preference.
If the reviewers cannot agree, route the ambiguity to the codebook owner. Do not settle it by majority vote. A codebook that requires private knowledge will keep producing disagreement no matter how many samples are drawn.
Feed findings back into the codebook
Close the sample with a short list of repairs: clarify a definition, split an overloaded code, add a required note, or remove a code that no longer routes work. Assign an owner and a review date.
Re-sample after the repair and compare the disagreement rate. A lower rate is useful only if the code still maps to the real work. The aim is a codebook that a new reviewer can apply, not a report that looks clean.
Copy-ready call and handoff lines
Questions managers ask
Does a disposition code audit judge agent performance?
No. It checks whether the code matches the recorded call facts and whether the codebook is usable. Performance decisions belong to the operations owner.
What if the agent note is missing?
Mark it missing and classify the case as unclassifiable if the code cannot be checked. Do not infer the note from the code.
How many calls should be sampled?
Use a declared sample size and window that fit the queue volume and risk. Report the eligible count, exclusions, and unclassifiable cases with the result.
When should the codebook change?
When repeated disagreements point to an unclear definition, an overloaded code, or a missing category. Assign an owner and re-sample after the change.
Sources
- NIST Cybersecurity Framework 2.0NIST, 2024. Governance, identity, access, and recovery outcomes for call-center controls.Source 1
- NIST Privacy FrameworkNIST, 2020. Privacy-risk framing for caller data, recordings, and retention.Source 2
- CISA Cybersecurity Performance GoalsCISA, 2024. Practical identity, logging, response, and recovery goals.Source 3
- PCI Security Standards CouncilPCI SSC, 2024. Payment-card handling boundaries for phone-based customer service.Source 4
- FTC Telemarketing Sales RuleFTC, 2024. Consumer-contact rules relevant to outbound calling controls.Source 5
- U.S. Department of LaborDOL, 2024. Workforce and recordkeeping context for staffing models.Source 6
- U.S. Small Business AdministrationSBA, 2024. Role design and management basics for small employers.Source 7
- U.S. GAO Assessing Data ReliabilityGAO, 2020. Provenance, completeness, and fitness-for-use checks.Source 8