My Router Recommended the Cheap Model 89% of the Time. The Expensive One Ran Anyway

01I quoted a cost number for five weeks. It measured nothing.

My AI coding system runs a triage step before every model call. It sizes the task and writes down three recommendations. Which tier of model should run it. Whether one attempt is enough. Whether the job needs a verifier or a human approval before it counts as done. Three answers, and every one of them is about intent rather than outcome.

Across 3,717 logged decisions, pinned to 4 August at 22:48Z because the file appends while I write, it sized 89% of tasks to the cheapest model and 11% to the flagship.

I quoted that 89% for five weeks as evidence the system was cheap to run. It is not evidence of that. It is not evidence of anything about cost, and the reason is one missing field.

The triage log records the recommendation. It has no column for which model actually ran. It never had one. The component was not built to know.

02Two logs I had never read together

The execution log is a different file. It records what each task actually did, and it stamps every row with the triage verdict that was live for that turn. That stamp makes the join trivial, and I had never run it.

So I ran it. Of the ledger rows carrying a cheap-tier verdict, 816 could be traced to a model. The flagship ran 796 of them. The cheap model ran two. Ten name no model at all.

The advice was followed 0.2% of the time. Two runs. Out of 816.

This is not a bug in the triage step. Its own source says what it is, in a comment, and its last line is exit zero: it appends a sentence to the worker's instructions and steps aside. A classifier with a logging habit. Enforcement lives somewhere else, and on this path there was no enforcement at all.

The failure is mine, and it is a reading failure. I had read the source. I still quoted the output as a spend figure.

03The same number, counted two ways, published under one label

Checking the first mistake turned up a smaller one underneath it. That one is the more common shape.

The health figure for this design is not the cheap share at all. It is the guarded fraction: the share of decisions carrying a mandatory verification step or an approval rail. A cost chart can look excellent while the guards quietly fall off, and only the guarded fraction shows that.

In the first 2,064 decisions, 602 carried a guard. That is 29%, and I had paired it with the cheap tier because the cheap tier was what I was arguing about. Wrong pairing. The 602 counts guarded rows at any tier, flagship included. Restricted to the cheap tier, which is how I had described it, the count is 481, or 23%.

Both numbers are correct. They measure different things and I gave them the same label. Pinned to 4 August, the current figure is 861 of 3,717, or 23.2%, against 481 of 2,064, or 23.3%, in the earlier window. Flat, which is the answer you want from a guard rail.

04Root cause: an advisory component read as an enforcing one

Both errors have the same root. A component that advises and then exits produces a log of intentions, and an intention log reads exactly like an outcome log. Same shape, same timestamps, same field names. Nothing in the file announces which kind it is.

Ask one question of any routing ratio before you believe it: which file did it come from. If the answer is the router's own log, it is a forecast. If it is the execution log, it is a measurement. Only one of those can be quoted as a cost, and almost everybody quotes the wrong one.

The design itself is not the problem, and it is not mine. Cheap-first routing has been published for years as a model cascade; FrugalGPT is the most visible version, where a cheap model answers first and a scoring step decides whether to escalate. What I added is the guard column next to the tier, which is the part that tells you whether the arrangement is still safe.

05The fix: retire the target, then join the files

The triage step used to carry a target: block 30 to 50% of tasks from the expensive path. On 2026-07-28 I retired it in writing, because the component had stopped emitting the single tier label that target was measuring. Once the label was gone the old number was not comparable to anything the live system produces, and a target you cannot compute is worse than no target.

What replaced it is qualitative where it should be and quantitative where it must be. Cheap by default. Expensive reserved for architecture, multi-file work and deep investigation. Workflow only for scope you cannot enumerate up front. One number is tracked, and it is the guarded fraction.

The structural fix is the join. Two logs, one for what was decided and one for what was executed, read side by side. That is the only view in which an advisory component and an executing one can be caught disagreeing. Mine had been disagreeing for five weeks, in a file I already had.

06What this buys, and what it costs

The alternative design is to send everything to the best model. It feels safe. It measures nothing. You pay for reputation, collect no verification record, and never find out which tasks were trivial all along.

A cheap worker with a compulsory checker beats an expensive worker with a reputation. The checker leaves a record; the reputation leaves an invoice.

What does not survive is my evidence for it. The 89% was never evidence. The real evidence is the guarded fraction holding at 23% across two windows, and the join that shows where the work actually landed.

07How every number here was derived

These commands read live state, so re-running them will give different numbers from the ones quoted here, usually larger. That is the point of printing the commands rather than only the conclusions.

# Both files append while you read them, so every count
# below is pinned to one instant.
CUT=2026-08-04T22:48:00Z

# What the triage step RECOMMENDED (3,717 decisions):
jq -r --arg c $CUT 'select(.ts<=$c).tier' \
  ~/.sgnk/state/complexity-gate-log.jsonl | sort | uniq -c
#   3311 floor
#    406 strong

jq -r --arg c $CUT 'select(.ts<=$c).orchestrate' \
  ~/.sgnk/state/complexity-gate-log.jsonl | sort | uniq -c
#   3547 single
#    170 workflow

# Guarded fraction, both denominators. These are DIFFERENT numbers.
# The original window is the first 2,064 rows, ending 2026-07-28T19:53:54Z.
# Guarded at ANY tier:
head -2064 ~/.sgnk/state/complexity-gate-log.jsonl \
  | jq -c 'select(.rule2_gated or .needs_verify)' | wc -l    # 602

# Guarded AND cheap-tier, which is what the sentence claimed:
head -2064 ~/.sgnk/state/complexity-gate-log.jsonl \
  | jq -c 'select((.rule2_gated or .needs_verify) and .tier=="floor")' | wc -l   # 481

# THE JOIN: what the router advised against what actually ran.
jq -r 'select(.gate_tier=="floor") | .model' \
  ~/.sgnk/traces/*.jsonl | sort | uniq -c | sort -rn
#   796 opus
#    10 (null)
#     2 sonnet