Module 18 · A/B tests and segments

Leer en español: Módulo 18 · Pruebas A/B y segmentos.

Goal: compare two versions of a campaign with an A/B test and read its verdict, save a segment filter of your portfolio and freeze it into a list, and read the “Best time to call” panel. Time: ~8 minutes · You need: a campaign (Module 3), an imported portfolio (Module 2) and, for segments, saved segments enabled for your account (see the warning below).

One of the three functions ships off

• A/B tests: the platform's flag catalog has none for them; the A/B tests button appears on every campaign that is not archived.
• Saved segments: they ship off by default (the debt_segmentation_enabled flag). Only the platform operator, that is, your Dialer Digital contact, turns it on per account. While it is off the screen says “Saved segments are not enabled for this account.” and you cannot create or freeze anything; segments you already saved are kept and work again when it is turned on.
• Best time to call: it ships on by default (the best_time_ranking_enabled flag). Only the platform operator can turn it off for your account.

1. Create an A/B test

An A/B test compares variants of the same campaign on the same debtors. The system splits the accounts between the variants by weight and measures which one does better.

1. Open Campaigns in the side menu. On the campaign's row, choose A/B tests. The panel with that campaign's tests opens. If there are none it says: “This campaign has no A/B tests yet — create the first one with “New A/B test”.”
2. Choose New A/B test and give it a Name (up to 200 characters).
3. Under Metric to compare choose Contact rate or Promise rate.
4. Define the variants. A test needs 2 to 5; use Add variant and Remove variant. Each has a Variant name, a Weight (1–1000) (its relative share of the split) and, on exactly one of them, the Control mark: the campaign as it is today, which the others are compared against.
5. Choose Create A/B test. The test is born as a Draft.

What each variant may change

Under “Optional changes for this variant. Leave blank to keep the campaign as it is.”, the list is closed. A variant may only change:

• Script to change: Voice prompt, Voicemail message or SMS text, with the New text. If you do not want to change the script, leave — no change —.
• Window from / Window to: a calling window that must stay inside the campaign's own window; it may cross midnight, but the start and the end must differ.
• Minimum seconds between attempts: a whole number from 1 to 2,592,000. It only applies to voice campaigns and can only make the pace stricter, never faster.
• Approved email template: an already approved email template of your account.

The window and the minimum seconds belong to voice campaigns, and an SMS or email campaign cannot choose Promise rate (a promise points at a call, not at a message): the server refuses those combinations and the panel shows each refused field.

2. Start, end and read the verdict

A test goes through three states: Draft, Running and Ended.

• Start puts the draft to work. A campaign can have only one test running at a time, and a test cannot be started on a preview or manual campaign (the server refuses it with “The action was refused”). Once it is running, its variants can no longer be edited.
• End closes the test; the panel asks “End this A/B test? It cannot be restarted.” Do it when the verdict is clear or you no longer want to keep comparing.
• Results opens the reading of any test that is not a draft. Refresh recomputes it; Back to A/B tests returns to the list.

The results screen shows the Metric compared and, per variant, Weight, Debtors, Calls placed, Contacts, Promises, Contact rate, Promise rate and Abandonment rate. At the top is the verdict, which can only be one of three:

• Not enough data yet. Some variant has not reached the floor. The panel says “The server asks for at least N placed calls and N successes per variant before giving a verdict”; today's server asks for 100 placed calls and 10 successes per variant.
• No significant difference. Every variant reached the floor and none beats the control with the required confidence.
• Winner: NAME (p = P). A variant beats the control with statistical significance (0.05 level, with a correction when there is more than one challenger).

Three warnings that change how you read it:

• The split came out skewed. If the debtors were not split between the variants in the proportion of the weights, the panel warns: “the debtors were not split between variants in the proportion of the weights, so the comparison may be biased.” When that happens the server declares no winner and says “The server withheld its verdict because of this.”
• Attempts without a variant. “N attempts could not be assigned to any variant and are not counted below.”
• No verdict. If the server answers without a verdict, the panel does not make one up: “The server answered without a verdict, so no result is shown.”

Results are recomputed from the call ledger every time you open them; they are not stored counters.

3. What never varies: compliance

When you create a test the panel reminds you: “Legal notices, the opt-out key, the abandonment limit and the dialing pace limit are the same in every variant and cannot be changed here. Every attempt goes through the same compliance checks as any other call.”

That is why the form has no field for those things, and the server refuses them if someone tries to send them through the API. In SMS and email a variant may change the commercial copy but not the legal sentence (the opt-out): if a variant's legal part is not identical to the campaign's, it is refused. An A/B test decides which script, window or pace works better; it never decides whether a rule is followed. The Module 4 gates still apply to every attempt of every variant.

4. Save a segment and pass it to a list

A segment is a saved filter of your portfolio, with no personal data, that you can reuse. It is not the same as the rewind segments of Module 5: those filter a campaign rewind; these live under Debts and can be frozen into a list.

You need the function turned on (see the warning above). Open Debts and choose Saved segments →; the screen is titled Debt segments.

1. Under New saved segment, give it a Segment name (up to 200 characters).
2. Set whichever criteria you want; each can stay on Any: Campaign, Debt state, Debt type, Primary-contact states (two-letter codes separated by commas, like TX, NY), Reg F status (OK to call, Blocked: 7-in-7 limit, Blocked: after a contact or Undecidable), Minimum balance (cents), Maximum balance (cents), Minimum delinquency age (days) and Maximum delinquency age (days).
3. Choose Save segment. You will see ““NAME” is saved at version N.”

Segments are versioned: Edit loads one and Save new version creates the next without deleting the previous. The Saved segments table shows Name, Version, Criteria (a segment with no criteria says Whole portfolio) and Actions.

Freeze the segment into a list

Choose To list on the segment's row. Freeze “NAME” into a list opens.

1. Under Target list, choose an existing list or Create a new list and type the New list name.
2. Choose To list. The result says, for example, “List “LIST”: N account(s) matched and N were added.”

It is a point-in-time photo: “Later changes to the segment do not change the list.” If the result was too large, the panel adds “The catalog cap truncated this photo.” Then use the list in a campaign like any other (Module 3).

5. Read the “Best time to call” panel

Under Campaigns there is a Best time to call panel. Pick a campaign under Campaign (at first it says “— pick a campaign to see its best time —”; if you have no campaigns yet, “No campaigns yet — the best time is computed per campaign.”).

What it is: “What the dialer applies when it orders leads: a weight per local weekday and hour of the consumer, inside the legal window 08:00–21:00. It only reorders leads that are already dialable — it never adds a call or moves one out of the legal window.”

How to read it:

• The badge. Applied means “The dialer orders leads with these weights.” Not applied means “The account turned this off: the dialer keeps the neutral order. The table shows what it would apply.”
• The source. The Source line says where the weights come from: Campaign history (“This campaign has enough attempts of its own.”), Account portfolio (“Too few attempts in this campaign: the whole portfolio speaks for it.”) or No evidence — neutral order (“No level has enough attempts yet, so every weight is 1.00.”).
• The table. The table gives the “Weight by consumer-local weekday and hour (1.00 is neutral)”, from Mon to Sun and by Hour. A weight above 1.00 marks an hour when that campaign (or the portfolio) contacts better than its average. Hover a cell to see its contacts, attempts and source. The weight only moves between 0.5 and 1.5.

What it does to the dialing order

It only orders. A lead in a bad hour moves down the batch; it does not leave it, and a lead in a good hour moves up. It does not change who is called or at what hours they may be called: every candidate still passes the pre-dial gates (legal hours 08:00–21:00 in each candidate's zone, Reg F 7-in-7 and the state matrix). With no history, the order is exactly the order of always.

Do not confuse it with the Best call window of Module 6: that is an analytics KPI that tells you the two hours with the best contact rate; this panel shows the weights the dialer uses to order.

6. Who turns each function on

Function	Flag	Default	Who changes it
A/B tests	none in the catalog	available	—
Saved segments and pass to list	debt_segmentation_enabled	off	the platform operator (your Dialer Digital contact)
Best time to call	best_time_ranking_enabled	on	the platform operator

Summary

1. An A/B test compares variants of one campaign on the same debtors; 2 to 5, one control.
2. Each variant only changes script, window, minimum pace or template; never compliance.
3. The verdict needs at least 100 placed calls and 10 successes per variant; a skewed split withholds the winner.
4. A saved segment is a versioned filter with no personal data; To list freezes it into a photo.
5. Best time to call only reorders the batch; it never adds calls or widens the hours.