My AI Security Scanner Reported a Leak in a File That Does Not Exist

01My security scanner named a file that has never existed

I run an AI coding system. One part of it is a security auditor, a subagent I send to sweep a repository for problems. It came back with this:

Leaked personal access tokens in testing/workspaces.json.

A personal access token is a password for software: a long string that lets a program act as you. So this is not a report you sit on. It is also not one you forward. The rule in this workspace is that no factual claim gets repeated before a live check, so I looked for the file first.

There is no testing/workspaces.json. Not moved, not renamed, not deleted in some recent commit. The path was invented.

02What a personal access token is, and why automation needs one

Worth defining first. The whole incident turns on it.

A personal access token is a long random string that stands in for your password when a program talks to a service. GitHub's own documentation calls it an alternative to using passwords for authentication when using the API or the command line. Anyone holding one is you, as far as that service can tell.

So why not just click around the website, or use the official command-line tool? Because neither works unattended. A dashboard needs a human at a keyboard. A command-line tool like the GitHub CLI is fine while you are sitting there to finish a browser login. But the moment a script runs on a schedule, in a build, or inside an AI agent's shell, there is nobody to click Approve. A token is the only identity a program can present on its own.

A token is not a shortcut past good practice. It is the only non-interactive authentication that exists, which settles the question of whether to use one and leaves only the question that matters: how you hold it.

03When a dashboard is enough, and when it is not

The real choice is not token versus nicer workflow. It is interactive versus unattended, and nothing else.

If you are deploying by hand once a week, use the dashboard. If you are running a command in your own terminal, use the vendor's login flow, which stores a credential for you and refreshes it. GitHub explicitly recommends the GitHub CLI or Git Credential Manager for command-line work, the built-in token for its own Actions, and a GitHub App for long-lived integrations.

A raw token is what is left when none of those apply: a cron job, a container, a script an agent runs on your behalf. That is a far narrower set of cases than most token drawers admit. Audit yours by asking which token exists because you actually needed one.

04The fix: search for the pattern, not the filename

Two exits open when a report names a file that is not there, and both are mistakes. Call it a hallucination and you throw away the half of the alarm still standing. Soften it to a possible leak and you turn a checkable claim into one nobody can settle.

So the question changed shape. Not is the report right about this file, but instead: are token-shaped strings anywhere on this machine that should not hold them?

That search came back positive, in the one place that writes itself without being asked. Every session my system runs is appended to disk as JSONL, one line per event. Paste a secret into a chat, or let a tool echo one back into its own output, and it lands there in the clear.

This is documented behaviour, not a bug. Anthropic's data-usage page states both halves: known API key and token patterns are redacted before a shared session is uploaded, and the copy that stays on your disk is stored in plaintext. A public request to scrub the local files was filed against the claude-code repository and closed as not planned.

05Counting: how 47 alarming files became 2 real problems

Counting is what separates a finding from a headline, and it runs on your machine as easily as mine.

The pattern is gh, then one of p, o, u, s or r, then an underscore, then 20 or more characters. That is the shape of a GitHub token. It matched 47 of 15,110 transcript files here, re-derived on 2026-08-04 at 22:54Z.

Forty-seven files sounds like a bad afternoon. It is not 47 credentials. Those files hold 163 occurrences of only 20 distinct strings, because a transcript repeats itself constantly. Of the 20, seven run to the 40 characters a classic GitHub token is. Length is a filter and not a verdict, but it is the cheapest filter available.

Read those seven by eye and five are placeholders somebody typed by hand: ghp_ followed by A repeated, ghp_ABCDEFGHIJ, ghp_1234567890. Two are real. One is an environment variable a tool echoed into its own output. One was pasted straight into a chat.

Two, not 163, and not 47. A grep hands you a scare. What survives reading is a task list, and a task list of two is something a person finishes on a Tuesday.

06Every vendor already told you not to do this

Here is the part that made me stop and reread. The storage rules are not vague, and they are not new. They are written on the page where you create the token.

Cloudflare, on the screen that issues one: the token secret is only shown once, and do not store the secret in plaintext where others can access it. Supabase, on secret keys: never expose them publicly, avoid transmission over chat or email. GitHub: never hardcode authentication credentials into your code. Vercel returns the value once and says it can never be retrieved again.

Now read those against what an AI coding tool does by default. It writes every session to disk in plaintext. A session is a chat. So a tool that helps you work breaks two of those three rules automatically, for every session, without anyone deciding to.

That is the finding, and it reaches well past my one machine. The advice was always correct. What changed is that a new category of software now writes a plaintext chat log of everything you type, and nobody has gone back to update the advice to say so.

That is the finding, and it reaches well past my one machine. The advice was always correct. A new category of software now writes a plaintext chat log of everything you type, and nobody has updated the advice to say so.

07How to create and hold a token so a leak costs little

You cannot make a token unleakable. You can make a leaked one nearly worthless, and that is the whole game.

Prefer a fine-grained token over a classic one. GitHub recommends this whenever possible, and the reason is blast radius. A fine-grained token is limited to resources owned by a single user or organisation, can be limited to specific repositories, and carries specific permissions instead of broad scopes. A classic token with repo scope reaches every repository you can see.

Set the smallest scope that works and the shortest expiry you can stand. GitHub's guidance is minimum permissions and an expiration for the minimum time you need it; the default is 30 days and unused tokens are removed after a year. Cloudflare lets you add a time-to-live and restrict by client IP. Vercel's create-token call takes an expiresAt and a projectId that scopes the token to one project. The knobs exist on every platform. Almost nobody turns them.

Keep the value out of files that live in a repository. Never hardcode it. If a .env file is unavoidable, GitHub's own instruction is to encrypt it and never push it to any repository.

The pattern I use is one file outside every project, sourced in the same command that needs it, never printed. That last clause is doing more work than it looks.

08Environment variables are oversold, and my own data proves it

Environment variables are the standard advice and they are good advice, but they are usually oversold, and my own data says so.

Of the two real credentials on this machine, one was pasted straight into a chat. The other was an environment variable that a tool echoed into its own output. It was being held correctly and it still ended up in a plaintext log, because a variable stops being a variable the moment something prints it.

So an environment variable protects against the file being committed, shared, or read off disk by something browsing the repository. It does not protect against a shell that echoes commands, a debug print, a crash dump, an error message that includes the environment, or an AI transcript that records the terminal output.

So an environment variable protects against the file being committed, shared, or read off disk by something browsing the repository. It does not protect against a shell that echoes commands, a debug print, a crash dump, an error message carrying the environment, or an AI transcript recording the terminal output. Storage is a probability. Expiry is a ceiling.

09What changed, and what I refused to pretend changed

The structural fix for the alarm is one line in a rulebook, not code. My system reads a standing rules file at the start of every session. The incident is preserved there as rule 2, dated 2026-06-27: verify a subagent's security or factual claim live before repeating it as fact, and attribute anything unverified.

The rule is deliberately symmetrical. It bans repeating an alarm unverified, and it bans dismissing one unverified. Both failures were live options that day, and only one of them is the famous one.

One thing this did not fix. Neither of the two real credentials was rotated by any rule in the rulebook. Rotation was never the rulebook's job, and writing a rule about noticing does not rotate anything.

Verify a subagent/audit's security or factual
claim live before repeating it as fact, and
attribute anything unverified.
 
loaded: every session, automatically
cost:   paid once, the day it was written

10How to check your own machine, in four steps

This reproduces. Your numbers will differ; the shape will not.

Run the count before you run the panic. The distance between the first number and the last one is the whole point.

11Four standing orders for any worker you cannot interview

Never repeat an alarm as fact before a live check, however specific it sounds. Specificity is not evidence. A language model produces detail at the same speed it produces truth.

Never dismiss an alarm because its pointer is wrong. Wrong address, real fire is a common shape.

Widen from the named location to the underlying pattern. That is the move that found the real thing here.

Assume anything you type into an AI tool is written to disk in plaintext, because it is, and scope your credentials so that being right about that costs you very little.

The fire in room four was real. There was just no room four.

12How every number here was derived

These commands read live state, so re-running them will give different numbers from the ones quoted here, usually larger. That is the point of printing the commands rather than only the conclusions.

# The vendor's own account of both paths: redacted on upload, plaintext on disk
open https://code.claude.com/docs/en/data-usage

# The request to scrub the file that stays, and what was decided about it
curl -s https://api.github.com/repos/anthropics/claude-code/issues/50014 \
  | grep -E '"(title|state|state_reason|created_at|closed_at)"'

# The storage rules, from the pages that issue the tokens
open https://docs.github.com/en/rest/authentication/keeping-your-api-credentials-secure
open https://developers.cloudflare.com/fundamentals/api/get-started/create-token/
open https://supabase.com/docs/guides/api/api-keys

# The rule, verbatim, from the standing rulebook
grep -n -A 4 "Verify a subagent/audit" ~/.claude/CLAUDE.md

# The original live check: the claimed path returns nothing
find . -path "*testing/workspaces.json"

# STEP 1  how many files the pattern touches (the scary number)
grep -rlE "gh[pousr]_[A-Za-z0-9]{20,}" ~/.claude/projects --include "*.jsonl" | wc -l

# STEP 2  how many DISTINCT strings, not how many hits (the real number)
grep -rhoE "gh[pousr]_[A-Za-z0-9]{20,}" ~/.claude/projects --include "*.jsonl" \
  | sort -u | wc -l

# STEP 3  filter to token length; a classic GitHub token is 40 characters
grep -rhoE "gh[pousr]_[A-Za-z0-9]{20,}" ~/.claude/projects --include "*.jsonl" \
  | sort -u | awk 'length($0) >= 40'

# STEP 4  which of your own tokens are fine-grained, and when do they expire
gh api /user/tokens 2>/dev/null || open https://github.com/settings/tokens