Call Center Outsourced evidence brief · Desk review · Published
What does a quality calibration record need to survive a client audit?
The evidence a quality calibration record needs: the sample, the rubric, the disagreements, the decision, and the follow-up after the session.

On this page
- Key stats
- Key takeaways
- Question and scope
- What the evidence supports
- Define the calibration unit
- Select the sample before the session
- Freeze the rubric version
- Score independently first
- Record the disagreement, not only the consensus
- Name the decision and the owner
- Check the scoring scale
- Measure agreement in a usable way
- Test the record with a new evaluator
- Keep the call evidence
- Handle a disagreement with the client
- Review the follow-up actions
- Sample across teams and sites
- Protect the evaluator's independence
- Report the result to the client
- Review after a rubric change
- Check the retention of the record
- Limits of the evidence
- Manager review questions
- Conclusion
- Put this into a support lane
- Related operating guides
- FAQs
- Sources
Key stats
- 2.67 million U.S. customer service representative jobs in 2025 (BLS)
- $21.53 median hourly wage in May 2025 (BLS)
- 289,500 projected annual openings, 2025 to 2035 (BLS)
Key takeaways
- Keep independent scores, the rubric version, and the selected sample with the session record.
- Record disagreements by criterion and evaluator; do not erase them when the group agrees.
- Name a decision and an owner for each disagreement and check the follow-up at the next session.
- Test the record with a new evaluator and link every score to the call evidence.
Question and scope
This brief asks what a quality calibration record should contain to survive a client audit. The population is one queue and one review period, and the unit is a calibration session that compares evaluator scores on the same calls. The question is whether the record shows a controlled method rather than a meeting with a summary. The analysis covers sampling, scoring, disagreement, and follow-up, not the design of the quality rubric itself. Record the queue, the period, and the evaluators before sampling.
What the evidence supports
Quality-management and contact-center standards call for defined criteria, trained evaluators, and a documented review process. Federal data-quality guidance calls for documenting sources, testing reliability, and stating limitations. None of these sources supplies a universal calibration form. The operating inference is to keep the sample, the rubric version, the individual scores, the disagreement, and the resulting decision in one record. The record should let a reviewer follow one call from selection to the final decision.
Define the calibration unit
A calibration session should compare evaluators on the same call and the same rubric version. The unit is an evaluator-score for one call, not the session average. Record the call identifier, the evaluator, the score, and the date. A session average can hide a pattern where one evaluator is consistently stricter on the same criteria. A session average can hide a pattern where one evaluator is stricter on the same criterion.
Select the sample before the session
Choose calls that represent the common reasons, the difficult cases, and at least one call with a disputed outcome. Do not let an evaluator pick only the calls that make a point. Record the selection rule and the population it came from. A sample chosen after the discussion cannot test agreement. A sample chosen after the discussion cannot test whether evaluators agree before they talk.
Freeze the rubric version
A rubric that changes during the session makes the scores incomparable. Record the version, the criteria, and the scoring scale used for each call. If the rubric changes as a result of the session, mark the change and the date. Do not edit the old scores to fit the new wording. If the wording changes, mark the change and the date instead of editing the old scores.
Score independently first
Each evaluator should score the call before the group discussion. Record the individual scores and the comments separately. A score collected after the discussion measures conformity, not agreement. Keep the independent sheet with the final record so a reviewer can see the starting point. The independent sheet is the evidence that the discussion changed a score, not the record of the session.
Record the disagreement, not only the consensus
A calibration record is most useful when it shows where the evaluators disagreed. Record the criterion, the scores, and the reason each evaluator gave. A record that shows only the final agreed score loses the information that would improve the rubric. Do not erase the original scores. Erasing the original scores loses the information that would improve the rubric and the training.
Name the decision and the owner
Every disagreement should end in a decision: the rubric is clear, the rubric needs a change, or the case needs a policy answer. Record the decision, the owner, and the date. A session that ends with a general agreement to be more consistent has not produced a control. The decision should be specific enough to apply to the next call.
Check the scoring scale
A five-point scale can hide a pattern if evaluators use only the top and bottom. Compare the distribution of scores by criterion and by evaluator. A criterion with no middle scores may be poorly worded or rarely observed. Record the distribution beside the agreement result. A criterion with no middle scores may be poorly worded or rarely observed in the sample.
Measure agreement in a usable way
Choose a method that fits the scale and the sample size, and state it in the record. Do not report a single agreement percentage without the method and the sample. For a small sample, describe the disagreement in counts and examples rather than a precise statistic. The client should be able to follow the calculation. For a small sample, report the disagreement in counts and examples rather than a precise statistic.
Test the record with a new evaluator
Give the record to an evaluator who did not attend the session. Ask them to reconstruct the call, the scores, and the decision. If they cannot, the record is missing a link. This test is more useful than a sign-off page because it shows whether the evidence is usable. If the new evaluator cannot reconstruct the call, the record is missing a link.
Keep the call evidence
The calibration record should link to the recording, the transcript, and the case note used in the session. If the call is deleted under the retention rule, record that fact and the date. Do not keep only a summary of the call. A later auditor should be able to check the score against the evidence. If the call was deleted under the retention rule, record that fact and the date.
Handle a disagreement with the client
The client may score the same call differently or interpret the rubric in its own way. Record the client's position, the queue's position, and the decision. Do not change the score without a documented reason. A calibration record that hides a client disagreement cannot support the next review. A record that hides a client disagreement cannot support the next review.
Review the follow-up actions
At the next session, check whether the previous decisions were applied. Look for a rubric change, a coaching note, or a policy answer. A decision with no follow-up is a meeting note, not a control. Record the status of each action and the owner. A decision with no follow-up is a meeting note, not a control the client can audit.
Sample across teams and sites
A calibration record from one supervisor may not represent another site. Include evaluators from each team that scores the same queue, especially when the client compares sites. Record the site and the team with each score. A local agreement can coexist with a cross-site disagreement. A local agreement can coexist with a cross-site disagreement, so report both in the client summary.
Protect the evaluator's independence
An evaluator should be able to record a score without pressure to match a supervisor. Keep the independent sheet and the discussion notes separate. Do not attribute a disagreement to an individual in the client report unless the client needs it for a specific review. The goal is a better rubric, not a blame record. The goal is a better rubric, not a blame record for the individual evaluator.
Report the result to the client
Give the client the sample, the rubric version, the agreement result, the disagreements, and the actions. Use examples rather than a single percentage. State what the session could not resolve. The report should let the client see whether the quality score is reliable enough for the decision it supports. The report should let the client see whether the score is reliable enough for the decision it supports.
Review after a rubric change
A new rubric needs a new calibration before it is used for performance. Test the wording with the same calls where possible, and record the scores under both versions. A change that looks clearer on paper may create a new disagreement in practice. Update the training and the quality checklist together. Update the training and the quality checklist on the same day as the rubric change.
Check the retention of the record
Keep the calibration records, the rubric versions, and the linked evidence for the period the client and the law require. Limit who can edit a score and review the access log. A record that can be changed without a trace is weak evidence in an audit. A score that can be changed without a trace is weak evidence in an audit.
Limits of the evidence
This brief cannot prove that a quality score is valid for every purpose, because validity depends on the rubric, the sample, and the decision. A calibration record can show a controlled comparison, but it cannot remove evaluator judgment. The sample may miss a rare call type. Use the result to improve the method and to state the limits honestly. Use the result to improve the method and to state the limits honestly.
Manager review questions
Managers should ask which calls were selected, which rubric version was used, and whether the scores were collected independently. They should also ask what decision came from the largest disagreement and whether it was applied. The remaining question is which criterion still produces a split. Record the answers with the period and the sample. They should also ask which criterion still produces a split and what change will be tested next.
Conclusion
A quality calibration record survives a client audit when it preserves the sample, the rubric version, the independent scores, the disagreement, the decision, and the follow-up. A consensus summary alone cannot show whether the method is reliable. Test the record with a new evaluator and link it to the call evidence. The record is useful when a reviewer can reconstruct the session and see what changed.
Put this into a support lane
Bring the current queue rules, the shift handoff, and one week of cases so the review can test a real promise.
Review the queue designRelated operating guides
FAQs
Should the calibration record show only the agreed score?
No. The independent scores and the disagreement show whether the rubric is clear and where evaluators need training.
How many calls should a calibration session review?
Enough to cover the common reasons and the difficult cases within the available time. Record the selection rule and the sample size so the result is not overstated.