Quality management (QM)

Versión en español: Gestión de calidad.

Quality management lets a supervisor score a recorded call against a versioned scoring form, lets the evaluated agent dispute the score, and rolls the current scores up into per-agent and per-campaign reports. This page is the guide; the exact request and response schemas of every endpoint are in the generated Quality management reference. A first pass can also be made by the platform itself: see Automatic (AI) evaluations. Spanish version of that section: Evaluación de calidad con IA.

QM is off by default

Writes need the tenant flag qm_enabled, which is off by default and is not self-service. While it is off, every write answers 403 with error.code qm_not_enabled and writes nothing. Reads keep working with the flag off: an evaluation is evidence, and the evaluated agent keeps seeing it. A flag that cannot be read counts as off.

Who may do what

Action	Endpoint	Who
Read forms, versions	GET /v1/qm/forms, /v1/qm/forms/{id}, /v1/qm/forms/{id}/versions[/{version}]	any key or login with tenant scope
Create, edit, archive a form	POST /v1/qm/forms, PATCH / DELETE /v1/qm/forms/{id}	an account_admin dashboard login, flag on
Evaluate a recording	POST /v1/recordings/{id}/evaluations	a supervisor or admin seat, or an account_admin login without a seat; flag on
Read a recording's evaluations	GET /v1/recordings/{id}/evaluations[/{evaluation_id}]	dd_ keys and logins that may evaluate; an agent seat or member gets 403 role_forbidden
Resolve a dispute	POST /v1/qm/evaluations/{id}/dispute/resolution	an eligible evaluator who is not the evaluated agent; flag on
Reports	GET /v1/qm/reports/agents, /v1/qm/reports/campaigns	dd_ keys and logins that may evaluate
Read own evaluations, open a dispute	GET /v1/me/evaluations[/{id}], POST /v1/me/evaluations/{id}/dispute	the evaluated agent's own ddu_ login with an active seat

A dd_ integration key is a service, not a person: it can read but every write is 403 role_forbidden. A ddw_ widget token is refused on the writes and on the /v1/me/evaluations routes. See Authentication.

1. Define a scoring form

A form is an ordered list of 1 to 100 criteria plus a pass_threshold (0 to 100). Each criterion has a key (unique within the form), a label, a kind (yes_no or scale) and a weight (1 to 100). A scale criterion needs max (2 to 10). A yes_no criterion may be critical: answering it "no" fails the whole evaluation. na_allowed lets the evaluator mark a criterion not applicable.

curl -s -X POST "$API/v1/qm/forms" \
  -H "Authorization: Bearer $ADMIN_LOGIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Collections — outbound call",
    "pass_threshold": 80,
    "criteria": [
      { "key": "identification", "label": "Identifies the agency and the agent", "kind": "yes_no", "weight": 2 },
      { "key": "disclosure", "label": "Reads the full disclosure", "kind": "yes_no", "weight": 5, "critical": true },
      { "key": "empathy", "label": "Empathy with the debtor", "kind": "scale", "max": 5, "weight": 3, "na_allowed": true }
    ]
  }'

The answer is 201 with the form at version 1. A live form name is unique per tenant (409), and a tenant holds at most 200 live forms (422).

Editing never rewrites history. PATCH /v1/qm/forms/{id} with criteria, pass_threshold or both appends version n+1 and leaves version n untouched, because an evaluation may have cited it. Send base_version to guard against two people editing at once: if it is not the current version you get 409 and nothing is written. DELETE archives the form (idempotent, 204); archived forms are never deleted, cannot be edited, and free their name. GET /v1/qm/forms?include_archived=true lists them.

2. Evaluate a recording

curl -s -X POST "$API/v1/recordings/$RECORDING_ID/evaluations" \
  -H "Authorization: Bearer $SUPERVISOR_LOGIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "form_id": "'"$FORM_ID"'",
    "form_version": 1,
    "answers": { "identification": true, "disclosure": true, "empathy": 4 },
    "comment": "Good opening; close with the recording notice."
  }'

• Answers: true/false for yes_no, an integer from 0 to max for scale, the string "na" where allowed. One answer per criterion.
• The server computes the score; you never send it. It is returned as score_bp in basis points (8750 is 87.5 %), with passed and auto_failed (a failed critical criterion).
• The recording must be available. agent_id, call_attempt_id and campaign_id come from the recording, not from the body.
• The evaluation row is frozen. To correct it, send the same body with supersedes_id set to the current evaluation of that recording and form: a new row is appended and the old one is kept. Without supersedes_id, a recording holds one evaluation per form (409 once there is one).
• You cannot evaluate your own recording: 403 self_evaluation_forbidden.

3. The agent disputes

The evaluated agent reads their evaluations at GET /v1/me/evaluations. The seat comes from the token; any parameter that names an agent (agent_id, seat_id, user_id, sip_extension) is refused with 400. The agent sees the score, the answers, the evaluator's comment and the dispute state, but not who evaluated. dispute_until is the end of the dispute window while a dispute can still be opened.

curl -s -X POST "$API/v1/me/evaluations/$EVALUATION_ID/dispute" \
  -H "Authorization: Bearer $AGENT_LOGIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "reason": "criterion_misjudged", "note": "I read the disclosure at 1:10." }'

reason is wrong_call, criterion_misjudged or other. There is one dispute per evaluation, opened inside the tenant's qm_dispute_window_days window (7 days by default; 1 to 90). While a dispute is open the evaluation keeps counting in the reports. A dispute is refused with 409 when the evaluation was replaced, already has a dispute, is itself the result of a resolved dispute, or the window has closed.

4. Resolve the dispute

An eligible evaluator, who cannot be the evaluated agent, closes it:

# keep the score
curl -s -X POST "$API/v1/qm/evaluations/$EVALUATION_ID/dispute/resolution" \
  -H "Authorization: Bearer $SUPERVISOR_LOGIN_TOKEN" -H "Content-Type: application/json" \
  -d '{ "decision": "upheld", "note": "Reviewed with the call evidence." }'

# replace the score
curl -s -X POST "$API/v1/qm/evaluations/$EVALUATION_ID/dispute/resolution" \
  -H "Authorization: Bearer $SUPERVISOR_LOGIN_TOKEN" -H "Content-Type: application/json" \
  -d '{ "decision": "adjusted", "answers": { "identification": true, "disclosure": true, "empathy": 4 }, "comment": "Adjusted after review." }'

adjusted requires answers (same shape as when evaluating, scored against the same form version) and writes the replacement evaluation and the resolution event in one transaction. upheld takes neither answers nor comment. The answer is 201 with the dispute_event; its adjusted_evaluation_id points to the new evaluation.

While a dispute is open, correcting that evaluation through POST …/evaluations with supersedes_id is refused with 409 evaluation_under_dispute: resolve it as adjusted instead. Once resolved upheld, a correction is accepted again.

5. Reports

curl -sG "$API/v1/qm/reports/agents" \
  -H "Authorization: Bearer $SUPERVISOR_LOGIN_TOKEN" \
  --data-urlencode "from=2026-10-01T00:00:00Z" \
  --data-urlencode "to=2026-10-08T00:00:00Z"

/v1/qm/reports/campaigns is identical with campaign_id instead of agent_id. from and to are required, from must precede to, and the window may not exceed 31 days (400 otherwise). It selects recordings that started in [from, to). Figures are derived on every call from the current evaluations only (a correction replaces the row it supersedes). One row per agent (or campaign), form and evaluator_kind; two forms are different scales and are never averaged together, and neither are a person's evaluations and the automatic ones (see Automatic (AI) evaluations).

Field	Meaning
evaluations	current evaluations counted
avg_score_bp	mean score in basis points; null when there is nothing to average (never 0)
pass_rate_bp	passed over evaluations in basis points; null likewise
passed, auto_failed	passed count; evaluations voided by a critical criterion
adjusted	current evaluations that replaced a disputed one
open_disputes	current evaluations whose dispute is open

Reports carry counts and basis points only: no comments, phone numbers or other free text.

Automatic (AI) evaluations

Since core v0.106.0 (core#1973) the platform can score a recorded call on its own, so calls no supervisor has time to listen to still get a first pass. It is a separate class of evaluation: evaluator_kind is automatic (a person's is human). Field names and enums come from the vendored openapi/dialer-v1.yaml (info.version: 0.106.0); where the spec is silent the statement cites a core source file at tag v0.106.0.

What it evaluates, and what it does not

• What. Each criterion of a scoring form, against the redacted transcript of the call. The model only returns an answer per criterion; it never returns a score. Core computes score_bp, passed and auto_failed with the same scoring as for a person (QmEvaluation in the spec: "The score is computed by core in both cases, never by the model"). The weight and critical of the criteria are not sent to the model. lib/dialer/qm/auto_qa.ex.
• Which form. The oldest live form of the tenant, at its current version. There is no campaign-to-form mapping: every eligible recording is evaluated against that one form. lib/dialer/qm/auto_qa.ex.
• What it does not do. The result is a suggestion for the evaluator and a number in a report; it never changes a call's disposition: the job only writes an evaluation row. It is not written if the model's answers do not cover every criterion or do not fit the criterion (no "not applicable" is invented). It is never written over a person: a recording that already has an evaluation for that form gets no automatic one. Recordings of calls with an AI voice agent (ai_used) are excluded. lib/dialer/qm/auto_qa.ex.
• A person always wins. The first human evaluation of a recording whose current evaluation is automatic replaces it; the caller does not need to send supersedes_id. The automatic row is kept (the table is append-only). lib/dialer/qm/evaluations.ex.
• The agent can dispute it exactly like a human one (section 3). The dispute is never resolved by itself: an eligible evaluator resolves it, and if they adjusted it their human evaluation replaces the automatic one. The agent's own view (QmAgentEvaluation) carries no evaluator_kind, so the agent is not told which class it was.

Who gets it

All of these must hold, or no evaluation is made and nothing is spent:

Condition	Detail	Source
Plan	Tier scale or enterprise (the :qa entitlement). Any other tier, including an unknown one, is not entitled.	lib/dialer/post_call/entitlement.ex
Tenant flag	qm_enabled on for the tenant (see the warning at the top of this page). A migration turns it on by segment for scale and enterprise; a per-tenant override, written only by Dialer Digital, wins over the segment in both directions.	lib/dialer/qm/auto_qa.ex, priv/repo/migrations/20261111030002_seed_qm_enabled_segments.exs
Recording	available, audio not purged, with an agent, not ai_used, with a saved transcript, and no earlier evaluation of that recording and form.	lib/dialer/qm/auto_qa.ex
Provider	A paid provider configured for analysis in the cell. The local provider does not serve this task and the job ends without spending.	lib/dialer/qm/auto_qa.ex

The job is queued when the transcript of the recording is saved (Post-call intelligence describes eligibility for the transcript itself).

The spend brake (fail-closed)

The evaluation goes through the same spend brake as transcription and analysis, with function: "qa". The job asks for a reservation before calling the model and settles it with the real token usage afterwards. Anything other than a granted reservation ends the job without spending and without retrying; a ledger that cannot be read fails the attempt without spending. The model output is capped at 2,048 tokens. See Spend brake for the cap, the denial reasons and the fail-closed rules; they apply unchanged. For an API client the visible effect of a denial is that no automatic evaluation appears.

How they show in reports and lists

evaluator_kind is a field of QmEvaluation (human or automatic; evaluator_user_id is null for automatic). The two reports group by agent (or campaign), form and evaluator_kind, so a person's score and the automatic pass are different rows and are never averaged together, and they take an optional evaluator_kind=human|automatic filter (any other value is 400):

curl -sG "$API/v1/qm/reports/agents" \
  -H "Authorization: Bearer $SUPERVISOR_LOGIN_TOKEN" \
  --data-urlencode "from=2026-10-01T00:00:00Z" \
  --data-urlencode "to=2026-10-08T00:00:00Z" \
  --data-urlencode "evaluator_kind=automatic"

In the dashboard (v0.51.0), an automatic evaluation is labelled "Automatic" in the recording's QM panel (apps/web/src/routes/recordings/QmEvaluationPanel.tsx), and the quality page's report shows the class of each row and can be filtered by it (apps/web/src/routes/quality/QualityPage.tsx).

Errors

Errors use the standard error envelope. The ones specific to QM:

error.code	Status	When
qm_not_enabled	403	a write while the tenant's qm_enabled flag is off
role_forbidden	403	a dd_ key, a member, or a seat that may not evaluate
self_evaluation_forbidden	403	evaluating, or resolving a dispute on, your own recording
evaluation_under_dispute	409	correcting an evaluation whose dispute is still open

What is covered end to end

The platform's browser-driven integration test (quality-management.spec.ts) exercises this flow against a real core and dashboard, with qm_enabled set for the test tenant: with the flag off a form write is refused with qm_not_enabled; a dd_ key is refused with role_forbidden; an account_admin creates a form and evaluates a recording in the dashboard; a supervisor evaluating their own recording is refused with self_evaluation_forbidden; the evaluated agent opens a dispute from their dashboard; the dispute is resolved through this API as adjusted; and the agent report then shows one evaluation at 100 %, one adjusted and no open disputes.