Jude Dialog · Jude Dialog · tok-463b94041d0410e3dfae2e38d76b3a2b
Adversarial: long 40
P01: At 90% per-turn accuracy only 12% of 20-turn conversations succeed (0.90^20 = 0.12) (12% conversation success)
Plays a synthetic caller against Jude through the public address and checks the outcome against the ledger.
Requirement
Forty-turn conversation: corrupted turn 6, API timeout from turn 14, language switch from turn 20, barge-in at a confirmation, a duplicate message
Target that exceeds PolyAI
100% of conversations end with every promised outcome verified against the ledger, at 20, 40 and 60 turns, with failures injected
Regulation
FCA PRIN 2A (Consumer Duty); UK GDPR arts 22A-22D; EU AI Act art 50
Steps
- Caller opens Calls (voice) and the first turn discloses an AI agent
- Caller states the need in English
- Policy gates run in code: clinical red flag, vulnerability, verification before account data, strong customer authentication bound to amount and payee
- Only backend records are read and only their fields are stated (auth, billing, booking, account, orders, faq, troubleshooting, orders, faq, booking, faq, troubleshooting, faq, booking, faq, troubleshooting, routing)
- Confirmation asked before any change; a yes spoken over the question is not taken
- Change written once (idempotency key) and receipted
- Outcome verified against the ledger; receipt chain verified
Cases
| Kind | Given or injected | Then |
|---|---|---|
| happy path | a simulated customer of Jude Mobile with an account, a bill and two orders | Forty-turn conversation: corrupted turn 6, API timeout from turn 14, language switch from turn 20, barge-in at a confirmation, a duplicate message |
| receiving side | the ledger, ticket queue or diary holds the record stated in the conversation, with the same reference in the receipt | |
| failure path | backend timeout on the next backend call | the data is said to be not reachable, no fact is stated, the call is retried; a second miss hands over with a reference |
| failure path | the turn arrives as line noise | the question is asked again; the receipt state hash is unchanged |
| failure path | the same message arrives twice | the stored answer is returned; nothing is done twice |
| independent re-run | a different ATBAN agent reproduces the verified outcome |
Linked estate journeys
consortium-sim-05, wf.consortium-108, wf.consortium-143
Frozen contract 2026-10-23 · sha256 9fcd1e7e2f306382736175a60c575dc072c6e78fbed957930c72789f11a4b91e