Jude Dialog · Jude Dialog · tok-3421d58b8085a2390fc44add4cad923f
Adversarial: long 20
P01: At 90% per-turn accuracy only 12% of 20-turn conversations succeed (0.90^20 = 0.12) (12% conversation success)
Plays a synthetic caller against Jude through the public address and checks the outcome against the ledger.
Requirement
Twenty-turn conversation with every task verified: corrupted turn 6, API timeout from turn 14, language switch from turn 16
Target that exceeds PolyAI
100% of conversations end with every promised outcome verified against the ledger, at 20, 40 and 60 turns, with failures injected
Regulation
FCA PRIN 2A (Consumer Duty); UK GDPR arts 22A-22D; EU AI Act art 50
Steps
- Caller opens Calls (voice) and the first turn discloses an AI agent
- Caller states the need in English
- Policy gates run in code: clinical red flag, vulnerability, verification before account data, strong customer authentication bound to amount and payee
- Only backend records are read and only their fields are stated (auth, billing, booking, account, orders, faq, troubleshooting, routing)
- Confirmation asked before any change; a yes spoken over the question is not taken
- Change written once (idempotency key) and receipted
- Outcome verified against the ledger; receipt chain verified
Cases
| Kind | Given or injected | Then |
|---|---|---|
| happy path | a simulated customer of Jude Mobile with an account, a bill and two orders | Twenty-turn conversation with every task verified: corrupted turn 6, API timeout from turn 14, language switch from turn 16 |
| receiving side | the ledger, ticket queue or diary holds the record stated in the conversation, with the same reference in the receipt | |
| failure path | backend timeout on the next backend call | the data is said to be not reachable, no fact is stated, the call is retried; a second miss hands over with a reference |
| failure path | the turn arrives as line noise | the question is asked again; the receipt state hash is unchanged |
| failure path | the same message arrives twice | the stored answer is returned; nothing is done twice |
| independent re-run | a different ATBAN agent reproduces the verified outcome |
Linked estate journeys
consortium-sim-05, wf.consortium-108, wf.consortium-143
Frozen contract 2026-10-23 · sha256 a38301659a38703f6e3e6538164e235581c306d0fc96d13d69bab7c2a391e8a2