What's inside
- Why I am posting this at all
- What the system is, in one paragraph
- 3,193 rows, and 140 of them carry a verdict
- The verdict field has no author
- One row in the routing journal is evidence
- The denominator moved and the numerator did not
- Three things worth checking in your own logs
- What I am changing
- How every number here was derived
For 38 days I have logged every task I run through Claude Code into a JSONL file, one line per task. As of today that file set holds 3,193 lines. 140 of them carry a verdict on whether the task actually worked, and a separate routing journal holds 3,494 rows of which exactly one is usable as evidence that the routing part is learning anything. This post is the honest reading of those three numbers, plus the one schema mistake underneath them that you can check for in your own logs in about ten minutes. Every number here has the command that produces it at the bottom of the page, so you do not have to take my word for any of it.
§ 01Why I am posting this at all.
I have had a LinkedIn account for years and have posted almost nothing on it. The honest reason is not modesty and it is not time. It is that most of what I would have posted would have been claims I could not check. I could have written that my setup makes me faster, or that the routing layer learns from what it does, and I would have believed both. I had no way to show that either one was true.
So I spent the time building the checking part instead. A hook that writes one line per task, a second file that records what the router decided, and a set of gates that refuse to pass when a number cannot be reproduced. It was a while before any of it was worth reading.
This is the first post out of that. The plan is plain: post the readings as they come, including the ones that do not flatter me, and when a number refuses to move, say so rather than quietly changing what I measure. This post is mostly one of those.
§ 02What the system is, in one paragraph.
A shell hook writes one line into ~/.sgnk/traces/YYYY-MM-DD.jsonl every time I
run a task through Claude Code. The line carries a timestamp, a session id, a correlation
id, the skill that ran, the model, the tier, the reasoning effort, input and output tokens,
the latency, and a handful of outcome fields. A second file,
~/.sgnk/state/routing-journal.jsonl, records what the routing layer decided
before the task ran: which model it would pick, at what confidence, and in which mode. That
is the whole apparatus. I am describing it plainly because the finding in this post is not
about anything clever. It is the kind of mistake you make in the first hour of designing a
schema and then do not notice for five weeks.
§ 033,193 rows, and 140 of them carry a verdict.
Across 38 daily files, the first dated 27 June and the last dated today, the ledger holds 3,193 rows carrying a correlation id. Thirty-eight files for thirty-eight calendar days, so there are no missing days in the middle.
Of those 3,193 rows, 140 carry a non-null accepted field: 118 true and 22
false. That is 4.4 percent. The other 95.6 percent record what happened without recording
whether it was any good.
Two things about that number before it gets used for anything. The first is that 4.4 percent is not automatically bad. A verdict on every task would mean I sat and judged three thousand tasks, which I did not and would not. The second is that the file is append-only and was still being written while I wrote this, so 3,193 is a reading taken on 3 August 2026, not a constant. Both denominators in this post grow. Each one has its command in § 09, so you can take your own reading rather than trusting mine.
§ 04The verdict field has no author.
accepted is a bare boolean. It says true or false and nothing else.
I went looking for the field that records who set it, because that is the field that turns a
boolean into evidence. There is no user field. There is no source field beside it. There is
nothing in the row, and nothing anywhere else in the schema, that separates a verdict I made
by reading the output from a verdict written by a gate script running unattended at two in
the morning. I dumped every key that appears anywhere across all 38 files and read the list
by eye rather than grepping for what I expected to find, because a grep for the field you
hope exists is a good way to find it. The only provenance-shaped keys in the whole schema
are assertion_source and gate_source, and neither of them refers to
accepted.
So the sentence I wanted to write, that 140 tasks carry a human verdict, is a claim the file cannot support. The provable sentence is narrower: 140 rows carry a verdict at all. Anything past that is me remembering, and remembering is the thing the log was built to replace.
Here is the part that stung. One column over, the same file gets it right.
assertion_pass is paired with assertion_source, and every one of
the 201 rows carrying assertion_source reads machine-gate. The
machine's opinion has a documented author. Mine does not, in the same row, in the same file,
written by the same person. I gave the machine's opinion a provenance field and never gave
my own one.
§ 05One row in the routing journal is evidence.
The routing journal is the other half of the system, and it is where the harder number lives. It holds 3,494 rows as of today. Of those, 32 carry a real outcome: 25 success and 7 failure. Everything else is a decision with no recorded result.
Then the join fails. Of those 32 rows, the skill field reads
unknown on 28 of them and is null on 3. It is populated on exactly one.
| Field | Value |
|---|---|
| skill | sgnk-supabase-migration |
| timestamp | 2026-07-17T08:19:11Z |
| model | claude-opus-4-8 |
| reasoning effort | high |
| confidence | 0.7009 |
| outcome | success |
The reason an unnamed row is useless is not a tidiness complaint. The routing layer is meant
to learn a per-skill policy: which model and which effort level to pick for which kind of
work. Every one of those policies is keyed on the skill. A row that says a task succeeded
but not what the task was cannot credit anything, because you cannot credit an arm you
cannot identify. Twenty-eight successful or failed tasks with unknown in that
column teach the policy exactly as much as zero of them would.
There is a harder detail underneath. 981 rows in the journal carry
decision_mode: advisory, which means the router computed a recommendation on
those rows and then did not act on it. That is a whole post on its own and I will write it
separately. For this one it is enough to say the loop is open at both ends: the decisions
were advisory, and the outcomes cannot be attributed.
§ 06The denominator moved and the numerator did not.
The one row is dated 17 July. Anchoring the comparison on its own timestamp removes any arbitrary cut date, so here is the split.
Up to and including that row, the journal holds 1,027 rows. Since it, 2,467. And of those 2,467, the number that carry an outcome and name a skill is zero.
So the journal is now about 3.4 times the size it was at the moment the one usable row landed, and the count of usable rows is still one. I am not going to round that to "almost exactly tripled" and move on, because the ratio has drifted past the point where a round word is honest. The two numbers plainly: 1,027 then, 3,494 now.
The diagnostic is worth stating on its own because it generalises past my setup. When the denominator grows and the numerator stands still, you do not have a data problem. You have a wiring problem, and collecting more data will not fix it. I ran the loop for seventeen days after that row on the quiet assumption that volume would eventually produce a second one. It did not, and on the evidence above it was never going to.
§ 07Three things worth checking in your own logs.
None of this is specific to my harness. If you keep any log that records a judgement, these three checks take an afternoon between them and two of them take minutes.
1. Put the author next to every judgement.
Any field that records an opinion needs a sibling field recording who or what formed it, and the vocabulary should be fixed and small: human, gate, replay, import. A boolean with no author reads as evidence for as long as you remember the circumstances, and then it quietly stops being evidence without changing in the file. You will not notice on the day you write it. You will notice on the day you need it, which is the day it is too late to backfill honestly.
2. Write down the question before you write the logger.
One sentence, in advance: the single question this log exists to answer. Then check that a single row can answer it. Mine could not. The question was "which model and effort should this skill use", and answering it needed a join across two files plus a field that was never written on either side. I found that out five weeks in, from the data, rather than five minutes in, from the sentence.
3. Watch the ratio, not the count.
A count of rows only ever goes up, so a dashboard built on counts always looks like progress. The ratio of usable rows to total rows is the one that tells you something: mine went from 1 in 1,027 to 1 in 3,494 while I was not looking, which is to say it fell by a factor of about three and a half while the graph went up. Set the alert on the ratio and it fires. Watch the count and you will feel productive right up to the moment you check.
§ 08What I am changing.
Two changes, both small, neither of them clever. The first is a
verdict_source field beside accepted, with a fixed vocabulary. The
140 rows that already exist get backfilled as unknown rather than guessed. I
could reconstruct a plausible author for most of them from the timestamps, and I am not
going to, because a reconstructed provenance field is the same mistake in better
handwriting.
The second is a required skill on any routing row that carries an outcome. If
the router cannot name what it routed, the row should fail to write rather than write
unknown, because a row that says unknown looks like data and counts
like data and is not data. Neither change recovers the last five weeks. That is the cost of
the original mistake, and it is not refundable.
§ 09How every number here was derived.
Every figure above, with the command that produces it. Run them in this order and you will reproduce the whole post, or find that the denominators have grown, which is the expected outcome and is also the point. Reading taken 2026-08-03
The as-of stamp
# the reading in this post was taken on 2026-08-03.
# IST is UTC plus 5:30, so 12:01Z is 17:31 IST the same day.
date -u +%Y-%m-%dT%H:%M:%SZ
38 days, 38 files, no gaps
# expect 38
ls -1 ~/.sgnk/traces/*.jsonl | wc -l
# expect 2026-06-27.jsonl
ls -1 ~/.sgnk/traces/*.jsonl | head -1
# expect 38: calendar days from 27 June to 3 August inclusive
python3 -c "import datetime;print((datetime.date(2026,8,3)-datetime.date(2026,6,27)).days+1)"
3,193 rows, 140 verdicts, 4.4 percent
# 3,193 rows carrying a correlation_id
cat ~/.sgnk/traces/*.jsonl | jq -r 'select(.correlation_id!=null)|1' | wc -l
# 140 rows carry a non-null accepted: 118 true, 22 false
cat ~/.sgnk/traces/*.jsonl | jq -r 'select(.accepted!=null)|.accepted' | sort | uniq -c
# 4.4 percent. Recompute if the row count moved.
python3 -c "print(round(140/3193*100,1))"
The field with no author
# every key that appears anywhere in the ledger.
# read this list by eye. This is the load-bearing claim of the post,
# and a grep for the field you hope exists is a good way to find it.
cat ~/.sgnk/traces/*.jsonl | jq -r 'keys[]' | sort -u
# 201 rows carry assertion_source, and every one of them reads machine-gate
cat ~/.sgnk/traces/*.jsonl | jq -r 'select(.assertion_source!=null)|.assertion_source' \
| sort | uniq -c
The routing journal
# 3,494 rows
wc -l < ~/.sgnk/state/routing-journal.jsonl
# 32 rows carry a real outcome: 25 success, 7 failure
jq -r '.outcome // "absent"' ~/.sgnk/state/routing-journal.jsonl | sort | uniq -c
# of those 32: 28 unknown, 3 null, 1 named
jq -r 'select(.outcome=="success" or .outcome=="failure")|.skill' \
~/.sgnk/state/routing-journal.jsonl | sort | uniq -c
# the one row itself, in full
jq -c 'select(.skill=="sgnk-supabase-migration")' ~/.sgnk/state/routing-journal.jsonl
# 981 rows carry decision_mode: advisory
jq -r 'select(.decision_mode!=null)|.decision_mode' \
~/.sgnk/state/routing-journal.jsonl | sort | uniq -c
Before and after the one row
T=2026-07-17T08:19:11Z
J=~/.sgnk/state/routing-journal.jsonl
# 1,027 rows up to and including the one row
jq -r --arg t "$T" 'select(.ts<=$t)|1' $J | wc -l
# 2,467 rows since
jq -r --arg t "$T" 'select(.ts>$t)|1' $J | wc -l
# 0 of those 2,467 carry an outcome and name a skill
jq -r --arg t "$T" 'select(.ts>$t)
|select(.outcome=="success" or .outcome=="failure")
|select(.skill!=null and .skill!="unknown")|1' $J | wc -l
# 1,027 + 2,467 = 3,494, and 3,494 / 1,027 = 3.4
python3 -c "print(1027+2467, round(3494/1027,2))"
One caveat on all of the above, and it is the honest one to end on. These are append-only files. If you run these commands tomorrow the denominators will be larger than the ones printed here, and if I am unlucky, the numerator will still be one.
Thank you for reading. If you run the first check on your own logs and find a judgement field with no author, I would genuinely like to hear what it was and what it cost you.
Sources. Everything on this page comes from two files on my own machine, read on 3 August 2026 and not modified:
~/.sgnk/traces/*.jsonl, 38 daily files, 2026-06-27 to 2026-08-03.~/.sgnk/state/routing-journal.jsonl, a single append-only file.