Research question and scope

Published September 8, 2026.

This audit asks whether reviewed cases differ systematically from the queue population they are meant to describe. It evaluates one sampling process during a declared period. It does not determine a universal sample size or certify the quality program.

Methodology

Freeze an eligible population and the associated review sample. Compare channel, issue family, outcome, handle-time band, transfer status, reopen status, language need, agent tenure band, and time of day. Document automatic exclusions, missing records, reviewer substitutions, and cases unavailable at review time. Draw a small independent random verification sample from the eligible population and compare its composition with the normal review set.

Measures and analysis

Publish inclusion rates for each declared group with counts and denominators. Show targeted risk reviews apart from random or systematic samples. Examine whether abandoned, transferred, very short, and very long interactions are underrepresented. Measure reviewer agreement on a duplicate subset. Differences identify possible coverage error; they do not prove that omitted cases have worse quality.

Limitations and inference limits

Operational fields may be incomplete, and quality tools may sample only supported channels. Small groups create unstable rates and privacy risk. Agent-level comparisons are inappropriate when case mix and sample probability differ. Results describe coverage of the audited process, not customer satisfaction or causal effects.

Sources

  1. Office of Management and Budget Statistical Standards
  2. NIST Engineering Statistics Handbook
  3. US Government Accountability Office, Assessing Data Reliability
  4. NIST Privacy Framework