What is real here, and what is not.

Where a model is used, and where it is refused

A language model reads what a bank wrote and writes what a customer reads. It never decides whether to move money.

This run’s AI stage

Reported from the run that produced this page, including when it went wrong.

What broke

Found by auditing the finished system on purpose, module by module, asking what happens when each part misbehaves. Three of these were guardrails that were themselves the hazard. All are fixed, each with a test that fails if the fix is reverted.

1. 01

   The scheduler could create the very thing the engine refuses

   Retry timing searched a window bounded only by the notice period, never by the mandate's expiry. Given a mandate dying in two days and a replenishment cycle eight days out, it scheduled the retry six days after the mandate was dead, reporting a confident p=0.62 for a debit the bank would reject. The stopping rules caught dead mandates on the way in; the scheduler manufactured one on the way out.

2. 02

   A fault in the AI guardrail could kill the money stage

   The circuit breaker wrapped the whole diagnosis loop. Any trip or internal fault propagated out and destroyed the entire batch, including every money decision that never needed a model at all. A guardrail that can take down more than it protects is worse than the thing it guards against.

3. 03

   The model's work never reached the decision

   The run classified the batch with Claude, printed statistics about it, then each arm silently re-ran the dictionary alone and discarded the model's output. One record the model correctly identified as a revoked mandate was still decided as unclassified. The claim that the model was load-bearing was, in the run path, not true.

4. 04

   A truncated run rendered as a complete one

   When the batch breaker tripped, the run stopped at 19 of 100 records and the report printed a full headline comparison, computing rates against a denominator of all 100. It now refuses to present a truncated run as a result.

5. 05

   An unreachable model was scored as disagreement

   With the gateway down, the run reported 20 disagreements, as if a working model had given 20 different answers, rather than 24 failed calls. An accuracy signal computed from calls that never happened is a lie. Unreachable is now counted separately, and a trip surfaces instead of being swallowed.

6. 06

   My headline rested on a bug in my own simulator

   The strongest claim on this page used to be that the baseline only wins on raw totals by making debits above the RBI cap. An external reviewer pointed out that above ₹15,000 the debit needs additional factor authentication, so the bank declines it — that money was never deliverable, and the attempt layer was crediting it anyway. The compliance adjustment was correcting my own bug rather than measuring a behaviour. Fixed, the finding got sharper: the baseline was not merely breaking a rule, it was burning attempts on debits that could never settle. Raw and adjusted now coincide.

7. 07

   The money breaker advertised two limits it never applied

   RecoveryPolicy declared a per-debit cap of ₹15,000 and a three-attempt budget, both with comments saying they must never be exceeded, and enforced neither. Neither field was read anywhere in the engine. This was the third instance of the same shape in the log, after an inert cost limit and a breaker that truncated the measurement it protected. Both are enforced now, as a refusal that lets the batch continue rather than a trip that stops it.

8. 08

   My opening premise was wrong for the entire build

   Every page of this project used to say a halted subscription was permanent — terminal, gone for good, no undo — and the whole recurring-revenue claim rested on it. An external reviewer challenged it on the last day. Razorpay's own documentation: “If the customer successfully changes the card details when a Subscription is in the halted state, it moves to the active state.” The premise had survived from the planning document, written before any code existed, all the way into a public README. What the documentation does support is narrower: halted stops charging automatically, the invoices it accrues are “still created. However, we will not charge these invoices. You will have to charge them manually”, and reactivation depends on a disengaged customer acting. So the claim is now five monthly collections moved off autopilot, not five customers destroyed. It is the smaller claim, and it is the one that is true.

9. 09

   The late-revocation hazard favoured my own arm by name

   A mandate can die between the decision and the debit. That was implemented as the same hash function called on both code paths, gated on the arm's name — so it was a rule that favoured the sequencer rather than a fact both arms faced. It is now a property of the record that takes no arm argument at all, and a test parses the attempt function's syntax tree and fails if the word reappears in it. Published sensitivity: removing the hazard entirely costs 8% of the gain.

The circuit-breaker library used here, RunFuse, is my own package on PyPI. It was tested in isolation before being blamed: its step limits trip precisely and its counting is exact. RunFuse was correct; this project’s containment of it was not. A dependency being right does not make your use of it safe.

Open as of submission

• Top-bucket calibration: predicts 0.810, observes 0.670 (n=185) — self-reported by vasooli experiments, doesn’t affect the arm comparison.
• Rule 7: tested, never triggered by a seeded batch — verified logic, not demonstrated behaviour.