01What is an AI coding agent, and what can it actually do?
sgnk is a one-person studio where a language model writes a large share of the production code. Over the last 54 days I built the machinery that decides whether its output can be trusted, and this series is that machinery, one part at a time.
It is worth being precise about what kind of software this is, because the word assistant does it no favours. It is not a chatbot and not autocomplete. It reads files, runs shell commands, calls APIs, and changes things that are hard to change back. Once software can act rather than only answer, the interesting question stops being how good the output sounds and becomes what it is allowed to do, and what happens when it is wrong.
Five parts do that work, and the figure above is the map. Every post in the series takes one of them, starts from the same map, and shows what it got wrong before it worked. You are not expected to have read the others.
02Why is a rule written into a prompt not a guardrail?
A rule you write into a prompt is advice. It lands in the same context window as the question, the conversation so far, and whatever got fetched along the way, and the whole pile is weighed together to guess what comes next. Usually the rule wins. Sometimes it loses, and when it loses there is no error, no warning and no log line, because from the inside nothing went wrong.
So over 54 days I moved the rules that matter out of the prompt and into programs that sit on the tool boundary. This is what that looks like in practice, with the config, the pattern lists and the ledger rows printed off the running system rather than described.
03Why can you not just test the model harder?
The instinct is to test the model harder. It does not work, and the reason is worth being precise about. Every evaluation you run samples a distribution and trusts the sample to stand for the whole. That assumption is what an interview makes about a person, and it is roughly sound for people because behaviour is roughly stable.
Here it is not stable. The same model that handles a multi-file refactor cleanly will invent a configuration value ten minutes later, and both outputs arrive in the same register at the same speed. There is no signal in the tone. The distribution moves between model versions, between task types, and inside a single response.
That does not make evaluation useless, it makes it insufficient. The question I stopped asking is whether the model is good. The question I ask now is what the surrounding system refuses to let it do.
04What is the difference between advice and a control?
The distinction the rest of this rests on: an instruction inside the context is advice, and a program on the boundary is a control. They are not two strengths of the same thing.
A control is an ordinary program that runs before the action does and can refuse it. It holds no opinion, and there is nothing in it to persuade, because it is not made of language. The guard script in my setup states this in its own header, citing the learned rule that produced it: a textual prohibition is advisory, never the gate. That rule exists because subagents once committed to git despite an explicit written prohibition in every prompt. The fix for an instruction being ignored is not a better-worded instruction.
# By the system's own LR#48 (a textual prohibition# is advisory, never the gate)
05How do you add a guardrail to an AI agent?
Mechanically this is unremarkable, which is the appeal. A control here is a shell script registered against a tool in a config file. Before the tool runs, the harness runs the script. If it exits 2, the tool call never happens, and the model does not get to retry more persuasively, because it was never asked.
The matcher names the tool to intercept and the command is the program that gets to refuse. Six of these are registered against tool calls: the shell, file edits, skills, agent spawning, and two on anything reaching outside the machine. Twenty-eight hooks are wired in total across eight points in a session, but most only observe and inject context. The distinction that matters is which ones can return a refusal.
{"matcher": "Bash","hooks": [{"type": "command","command": "bash ~/.sgnk/bin/sgnk-bash-guard.sh","timeout": 5}]}
06What happens to one task, from prompt to record?
Drawn in order, a single task passes through five components, and it is easier to see what each is for once you see when it acts.
The gate sizes the job before a model is chosen, because the cheapest correct path wins. It cannot refuse anything: it is fail-open by design, on the reasoning that a triage hint which can wedge a turn is worse than no triage. The router recommends which model gets it, climbing a fixed ladder from the cheap tier upward rather than jumping to the flagship. The rails sit on tool calls and are the only components that can refuse. The judge grades finished work pass or fail with a critique rather than a score out of ten. The ledger appends one line and never edits it.
The bar I wrote for a judge is that its grades do not count until it agrees with my own marking at a kappa of 0.7. Three per-skill judges clear it on sets I labelled myself: 0.724 on 46 pairs, 0.7097 on 45, and 1.0 on 35. The gate that sizes every job does not. Its one calibration scored 0.1 against the same bar, on nine comparable pairs, raw agreement three of nine, recorded verdict FAIL, with a note that the labelling method could only have inflated the score. So the bar is real and met in places, and the component that touches every single task is the one that fails it.
Three of those five contain no language model at all. That is deliberate. Persuasion is the model's native medium, so the components that constrain it are the ones with nothing to say back.
07Should a safety guard block, or only warn?
The shell guard runs two pattern lists. A wide one of 39 alternatives covering data loss, irreversible git operations and metered resources, and a narrow one of 21 covering the genuinely unrecoverable: force pushes, SQL table and database removal, cloud resource deletion. The narrow list exits 2 regardless of mode, though three doors sit above it: a bypass variable, a one-shot human approval token, and a recursion guard for nested runs. The wide list, by default, does not block at all. It logs the hit, injects the reason into context, and returns 0.
That is a deliberate shadow period, not an oversight. A gate that blocks everything gets bypassed inside a week, and a bypassed gate protects nothing, so a broad pattern runs in the open first and earns the right to refuse once it proves well-targeted. It has logged 524 warnings since 11 July.
It also matches the text of a command rather than the intent behind it, which is worth knowing before trusting any number it produces. Writing this page tripped it twice: once on a command that read its own deny-list, and once as a hard block on a log message where a colon was followed by the word dropped, which is how a SQL table removal looks to a regular expression. A pattern gate counts matches, not attempts. For the same reason I am not quoting a total for the always-block log, because most of today's entries in it are audit agents and I reading the guard's own source.
git push .*--forceDROP (TABLE|DATABASE|SCHEMA) | TRUNCATE | dropdbvercel (rm|remove|rollback)aws .* (delete|terminate|remove|rm)
08How do you approve a destructive operation safely?
Eight standing rules sit underneath all of this, written on Thursday 11 June 2026, the morning after the system came close to deleting a client's production database. They apply to every task regardless of size.
The heaviest governs anything irreversible, and it does not say be careful. Be careful is unfalsifiable and therefore unenforceable. It specifies a sequence instead: name the operation in plain language, state the blast radius, state what was checked to confirm safety, ask, stop, wait for an explicit yes, then execute and verify the resulting state.
Ok, continue, a thumbs up, and silence are each written into the rule as not-approval. Every one of them is listed because every one of them had been tried. A prior approval does not carry to the next operation either.
09What should an AI agent's audit log record?
State the model can rewrite is not evidence, so the record lives outside it: one line appended per finished task, thirty fields wide, in a file nothing edits. There are 3,297 rows across 39 files, from 27 June to the day this published, which is 38 days. Rows are not tasks: 2,865 correlation ids are distinct, and 248 rows are a scheduled internal job with no model attached at all.
The row on the page is real, from the session that produced the deck, and it is there because it is unflattering. The gate sized the job to the floor tier. The router recommended the cheap model. The expensive model ran anyway, because decision_mode reads advisory and nothing enforces the recommendation. And decision_real_n reads 0, meaning that recommendation had no live observations behind it at all.
The honest coverage figure belongs here too. Of those rows, 142 carry an accept or reject verdict, and only 47 of those are mine. The other 90 were written by a machine grader and 5 have no recorded source. The rest are unmarked. A learning loop needs labelled outcomes, and this one is running on a thin diet of them, which is a fact about the system rather than a fact about the ledger.
"gate_tier": "floor","decided_model": "sonnet","decision_mode": "advisory","decision_real_n": 0,"model": "claude-opus-5",
10How do you stop an agent repeating the same mistake?
The eighth rule governs the rulebook itself. Every correction, mine or one the system catches in its own work, becomes a numbered line in a block that may only be added to, in the form never or always do X, because Y, verified on this date.
The block stands at 68, numbered one to sixty-eight with no gaps and no duplicates, 54 days after the first eight were written. Not one per day, since several landed in batches after a bad afternoon. It is read at the start of every session and can only be appended to; changes to the standing instructions go through review like any other edit to a contract.
The part I did not expect is that the original eight have not changed. The rules governing the work turned out to be nearly stable, and everything that accumulated on top is scar tissue from specific incidents.
66. Run every harness assertion under BOTH bashand zsh before trusting its verdict.67. A tool that writes must verify the writeLANDED before reporting success.68. A test whose subject is a rare fault provesnothing until it reproduces the fault.
11What does the machinery around the model actually buy?
None of it makes the model better. That is the point. The model improves or drifts on someone else's release schedule and I control none of it. The surrounding system is the part I control, and it accumulates: an incident becomes a rule, a rule that matters enough becomes a program, and the programs survive the model being swapped out from under them.
The remaining components each get their own post, along with the incidents that produced them: the alarm that was false and pointed at a real fire, the scoreboard that quietly erased wins, and the backups that failed silently for weeks.
12How every number here was derived
Both source files are append-only, so re-running these will give larger denominators than the ones quoted.
# the eight standing rules, and the day they were written
grep -c '^## RULE [0-9]' ~/.claude/CLAUDE.md # -> 8
date -j -f %Y-%m-%d 2026-06-11 +%A # -> Thursday
# the append-only block: highest number, and that it has no gaps or duplicates
python3 -c \"import re;ns=sorted(int(m[1]) for m in re.finditer(r'^(\\d+)\\. \\*\\*', open('$HOME/.claude/CLAUDE.md').read(), re.M));print(max(ns), len(set(ns)), [i for i in range(1,max(ns)+1) if i not in ns])\"
# -> 68 68 [] (max, unique count, missing numbers)
# NOTE: a bare grep for a leading number also matches RULE 8's own two numbered
# triggers, which is why a naive count reads 70 while the max reads 68.
# controls registered against tool calls, and hooks wired in total
python3 -c \"import json,os;d=json.load(open(os.path.expanduser('~/.claude/settings.json')))['hooks'];print(len(d['PreToolUse']),'PreToolUse |',sum(len(h['hooks']) for a in d.values() for h in a),'total across',len(d),'events')\"
# -> 6 PreToolUse | 28 total across 8 events
# the two pattern lists: wide (warns by default) and narrow (always exits 2)
python3 -c \"import re,os;s=open(os.path.expanduser('~/.sgnk/bin/sgnk-bash-guard.sh')).read();print(len(re.search(r'^DENY=.(.*).\\$',s,re.M).group(1).split('|')),'wide |',len(re.search(r'^DENY_CATASTROPHIC=.(.*).\\$',s,re.M).group(1).split('|')),'narrow')\"
# -> 39 wide | 21 narrow
# guard warnings logged
wc -l < ~/.sgnk/state/bash-guard-hits.log # -> 524
# trace rows. COUNT PER FILE, never with cat: a file with no trailing newline
# gets its last record fused into the next file's first, which both undercounts
# the lines and corrupts two records at every seam.
python3 -c \"import glob,json,os;n=sum(1 for f in glob.glob(os.path.expanduser('~/.sgnk/traces/*.jsonl')) for l in open(f) if l.strip() and json.loads(l));print(n)\"
# -> 3297 (cat | wc -l reports 3317, and cat | json.loads parses only 3265)
# how many of those rows carry a human verdict
python3 -c \"import glob,json,os,collections;c=collections.Counter(json.loads(l).get('accepted') for f in glob.glob(os.path.expanduser('~/.sgnk/traces/*.jsonl')) for l in open(f) if l.strip());print(c[True]+c[False],'of',sum(c.values()))\"
# -> 47 human of 142 verdicts, of 3297 rows