Eilidh Penhale’s complaint about a £6.00 overdraft fee was handled well by most measures. The fee was found, the complaint upheld, £6.06 refunded with interest, the final response sent well inside eight weeks, the root cause tagged. A reviewer re-reading the file would probably have passed it. InsightService’s QA check didn’t: the complaint arrived on 30 October, the acknowledgement was due on 4 November, and it went out on 21 November.
In short
- Re-reading a file finds what a reviewer happens to notice. Recalculating it finds what was actually wrong: a late leg of the clock, redress that doesn’t add up, a missing second approver.
- Eight checks, run by code on every sampled case. Acknowledgement and final response in time, vulnerability addressed, root cause, outcome, referral-rights wording, redress recalculated and approved, four eyes where required.
- The sample is random where it can be and certain where it must be. A seeded share of closed cases (20% by default, illustrative) plus every repeat complainant, vulnerable customer and breached clock.
- A person records the result. The QA reviewer agent prepares; a reviewer who didn’t close the case passes or fails it, and a fail needs a reason.
What complaint QA is for
Quality assurance on complaints exists to answer two questions: were customers treated fairly, and is the firm’s complaint handling working as its policy says? The FCA’s rules expect complaints to be investigated competently, diligently and impartially, assessed fairly and consistently, and put right with fair redress (DISP 1.4.1R), and the Consumer Duty expects firms to monitor the outcomes their customers get. Sampling closed cases is how most firms test both.
The usual method is a reviewer, a checklist and a sample. It works, slowly, and it has a blind spot: the reviewer checks what is easy to check by reading. Was there a letter? Was the tone right? Harder things, such as whether the acknowledgement date was inside the policy, whether the interest was calculated correctly, or whether the person who approved the redress was the one who proposed it, take time to verify by hand, so they are verified less often.
Eight checks, recalculated
InsightService’s QA checklist is code. For each sampled closed case it computes:
| Check | How it is decided | Not applicable when |
|---|---|---|
| Acknowledged within the policy timescale | The acknowledgement leg of the clock was met: sent on or before its due date | The case is a query |
| Final response within the policy timescale | The final-response leg was met | The case is a query |
| Vulnerability recorded and addressed | A note of how the case was adapted exists | No indicators on the case |
| Root cause tagged | At least one root cause | A query (fails for a complaint) |
| Outcome recorded | Upheld, partially upheld or not upheld | A query (fails for a complaint) |
| Final letter carries the referral-rights wording | A sent final response mentions the ombudsman | The case is a query |
| Redress matches the calculation and was approved | Recalculated from the stored items, rate, date, goodwill and threshold; same total and approved | No redress due (fails if upheld with fees in question and none offered) |
| Four eyes where required | Above the threshold, the approver was neither the proposer nor the submitter | No remediation |
The score on each review is the share of applicable checks that passed. The deadlines, the rate and the
threshold the checks use are the same illustrative, configurable policy values the case was handled
under (SERVICE_ACK_DAYS, SERVICE_FINAL_WEEKS,
SERVICE_REMEDIATION_RATE, SERVICE_APPROVAL_THRESHOLD), so QA tests the case
against the policy the firm actually set.
Each check is computed from the case; the result is a person’s.
Choosing the sample
A sample has to be fair and it has to be hard to game. InsightService draws it from closed cases not yet
sampled. Each case gets a fixed pseudo-random number from its identifier, so the draw is repeatable and
nobody can choose which files are reviewed; cases under the sample rate
(SERVICE_QA_SAMPLE_RATE, 0.2 by default, illustrative) are taken. Some cases are taken
whatever the draw:
- Repeat complainants, because a customer complaining again is a sign the last answer didn’t land;
- vulnerable customers, because the cost of getting it wrong is highest;
- breached clocks, because a late final response is exactly what QA should explain.
The sample: certain for the risky cases, repeatable and random for the rest.
What the demo’s sample found
The demo has four closed cases, and the QA reviewer sampled all four. Two of them, the complaints from Eilidh Penhale and Leo Somerton about the 3 October overdraft fee run, scored 86%: every applicable check passed except the acknowledgement. Leo Somerton’s chat on 20 October should have been acknowledged by 25 October; Eilidh Penhale’s call on 30 October by 4 November. Both acknowledgements went out on 21 November, with the final response, which itself was well inside its deadline. Ama Begum’s duplicate-fee complaint passed every check. The fourth, a lost-card query, had no applicable checks: no clock, no outcome, no redress.
Why this is a useful finding
Nothing about these cases looked wrong on the surface. The customers got their money back and a letter in good time. The finding is about the process: acknowledgements were not going out promptly. One late acknowledgement is a slip; the same failure across a sample is a question for the team lead about how the queue is worked.
The same mechanism checks redress. On each of these cases the QA reviewer recalculated the remediation, £6.06 for each overdraft complaint and £12.02 for the duplicate fees, from the stored items, and found it matched what was approved. Each total was within the threshold, so the four-eyes check passed without a second approver. Had a remediation above the threshold been approved by the person who proposed it, that check would have failed, even though the product would not have allowed it to happen in the first place: QA verifies the record, not the intention.
The agent prepares; a reviewer decides
Sampling and running the checklist is the job of the QA reviewer, one of the eight agents, run from the QA sampling page. It samples, checks, and prepares one decision: pass the reviews in which every check passed. Reviews with a failure are listed with what failed, for a person to look at.
Closed cases not yet sampled: the seeded share plus every repeat, vulnerable and breached case.
Eight checks per case, each with its evidence; a score per review.
One prepared decision listing the reviews where every check passed. Capped at L2: it asks first.
Read each failed check and its evidence; pass or fail the case. A fail needs a reason.
The reviewer can’t be the person who closed the case.
Patterns across failures go to coaching, the queue or the policy.
Accepting the agent’s prepared decision records a pass for each clean review in the reviewer’s name, with the note that it was accepted from the QA reviewer. The agent itself can never record a QA result: “A person records the QA result.” And a case’s review must be someone else’s than the person who closed it, so the handler can’t mark their own work.
A reviewer reads the file and ticks a list. Dates and amounts are checked when there is time. The review says what the reviewer thought.
The clock legs, the redress and the approver are recomputed for every sampled case. The reviewer spends their time on judgement: was the explanation fair, was the tone right, was the vulnerable customer really helped?
Using what QA finds
A QA programme earns its cost when its findings change something. Because each check is the same on every case, failures can be read as a pattern rather than as individual mistakes. Two late acknowledgements on two cases handled by the same team in the same fortnight point at the queue: were new complaints being acknowledged as they arrived, or only when someone got to them? A run of redress failures would point at the calculation or its inputs. A run of vulnerability failures would point at training.
The product helps with the first step and deliberately stops there. The case controls already stop the most serious failures at the close: a complaint can’t close without an outcome, a sent final response, a root cause, a recorded adaptation for a vulnerable customer and a decided remediation. QA then measures what the controls can’t prevent, such as lateness, and the team lead decides what to do about it. If the answer is a stricter policy, such as a shorter acknowledgement timescale, that is a configuration change the firm makes once, and every later QA round tests against it.
What the checks can’t tell you
Code is good at dates, sums and names. It is not good at fairness. The referral-rights check confirms the ombudsman is mentioned, not that the wording is the firm’s approved text. The vulnerability check confirms a note exists, not that the adaptation was right for the customer. The outcome check confirms one was recorded, not that it was the correct one. Those are the questions a reviewer is for, and the checklist is designed to give them the time to ask them.
For the head of complaints
QA evidence is recalculated, not asserted: when the regulator asks how you test redress consistency, the answer is every sampled case, every time.
The sample includes every vulnerable customer, repeat complainant and breached clock by design.
For the service operations lead
Failures point at process, not only at people: the late acknowledgements in the demo are a queue problem.
Reviewers spend their time on the judgement calls, with the arithmetic done.
For the board
A score per review and a trail of who reviewed what give the board a view of handling quality that doesn’t depend on who did the sampling.
A good QA check finds the problem nobody noticed because everything else looked fine.
Recompute the deadlines, the redress and the approvals on every sampled case, sample the risky ones every time, and leave the judgement to a reviewer who didn’t do the work.