Movement is observed, never claimed

The artifact this time is a run report from efferent’s software factory: implement, snapshot the workspace, run the gate pipeline, feed the findings back, try again. The spec was a port of a small project to Zig. The report says attempt one touched 0 files. Attempt two: 0 files. Attempt three: 0 files.

The transcript for those same attempts shows the coder rewriting main.zig. Repeatedly. Work was happening; the record said nothing was.

Same run, later: every deterministic gate green — 13/13 — and the final judge gate rejects with the confident assertion that the zig/ directory does not exist. It existed. Two attempts burned re-proving it.

Two false statements out of one run, and nobody lied. Each channel reported exactly what it could see; what it could see had quietly stopped being where the truth lived. This is the post-mortem: a two-bug detective story, then the three mechanisms that came out of it.

Bug one: the record keeper watched the wrong door#

Where did “files touched” come from? The implementor returns a receipt per attempt, and the receipt’s filesTouched is built from tool calls — the harness executed that write_file, so it knows the write happened. Airtight, right up until the coder stops using the door with the sensor on it.

A heredoc —

cat > main.zig << 'EOF'
const std = @import("std");
EOF

— is one opaque string to tool-call capture. It’s a bash invocation; the write happens inside the shell, past the capture’s event horizon. This coder did most of its zig work through the shell, so the receipt channel wasn’t degraded — it was structurally blind. Three attempts of genuine progress, recorded as three attempts of nothing.

And the record wasn’t decorative. The loop’s stall breaker — the mechanism that ends a run that has stopped moving — keyed on the receipt. A coder in the middle of productive work was one repeated verdict away from being declared immobile and shut down. The lying channel had a kill switch wired to it.

Bug two: the judge drowned in someone else’s files#

The judge gate runs last: only work that survived every deterministic gate spends judge tokens, and it rules on what those gates can’t — intent, honesty. To rule, it gets evidence: a list of the workspace’s files plus the contents of recent source files, both bounded, because context is finite.

The evidence assembly had two quiet assumptions. The file list was alphabetical, capped at 400. And “source file” was an extension allowlist — ts, py, rs, go, that neighborhood.

Now the setup line. An earlier zig run had failed on an environment split — the coder self-provisioned a zig toolchain, built, self-verified green, and all 13 gates exited 127 because they spawned with the harness’s env, not the workspace’s. The post-mortem fix taught the coder to provision toolchains into the workspace, at .local/bin, so coder and gates share one reality. Correct fix. In the re-forge, the coder did exactly that — and the judge’s evidence walk found ~15,000 .local/zig/… stdlib paths sitting in the workspace.

Alphabetical. Capped at 400. . sorts before every letter. The entire visible list was toolchain; zig/ never made the cut. The judge did what any reader of a list does — treated absence as nonexistence — and rejected: the deliverable doesn’t exist. While 13/13 deterministic gates said it did.

The kicker: even if main.zig had made the list, the judge could not have read it. The contents section was gated on the extension allowlist, and .zig/.zon weren’t in it. The judge was asked to rule on a zig port while structurally unable to see zig.

The shape of both bugs#

A harness accumulates claim channels. The receipt claims what was written. The evidence list claims, by omission, what exists. Every retry implicitly claims that trying again could produce something new. None of these channels is malicious, and that’s the point — each was accurate the day it was built and drifted the day behavior moved. The model didn’t game them; the world did.

So the fixes share one design rule: where a claim gates a decision, back it with an observation — and where the observation is bounded, state what the bound means.

Fix 1: ask the workspace, not the reporter#

The forge loop now takes a fingerprint seam: a per-file stat signature (size:mtimeMs) of the whole workspace, taken immediately before and after each implement call:

const before = yield* options.fingerprint
const receipt = yield* implementor.implement({ spec, attempt, feedback, workspaceDir })
const after = yield* options.fingerprint
// OBSERVED work: the workspace's own word, unioned with the receipt's claim.
const filesTouched = [
  ...new Set([...diffFingerprints(before, after), ...receipt.filesTouched]),
].sort()

diffFingerprints is every path whose presence or signature moved — adds, edits, and deletes alike. The placement carries the design:

• Around the implement call only. Gate-side writes — build outputs, test artifacts — land between one attempt’s after and the next attempt’s before, so they never register as implementor movement.
• Hidden directories are pruned from the fingerprint. Toolchains (.local again), build caches (.zig-cache), harness state — infrastructure churn from the coder’s own shell builds must not read as “the implementor changed something”. Hidden files still count; .gitignore is a real edit.
• Union with the receipt. The pruning means a legitimate tool write into a hidden dir would vanish from the diff — the receipt’s claim keeps it visible. Each channel covers the other’s blind spot.

A heredoc moves the stat; the diff sees it. The implemented: N file(s) touched line in the TUI now tells the truth, and the claude -p implementor — which never could track its own files — gets real records for free. Movement is observed, never claimed.

Fix 2: the breaker demands proof in triplicate#

The stall breaker was born in that first zig run’s post-mortem — attempts 2 and 3 had burned against an environment failure the coder could not move, reproducing byte-identical findings. Ending such a run early is right; ending a working run early is the worst outcome a harness has. So the breaker fires only on confirmed immobility:

const stalled =
  !report.ok &&
  filesTouched.length === 0 &&                                      // this attempt: no movement
  previous.filesTouched.length === 0 &&                             // last attempt: no movement
  reportFingerprint(report) === reportFingerprint(previous.report) && // three byte-identical
  reportFingerprint(previous.report) === reportFingerprint(prePrevious.report) // verdicts

(reportFingerprint is the failing findings, sorted and stringified — the verdict’s identity, compared byte-for-byte. In the real file previous/prePrevious are checked for existence first.)

Two no-op attempts, three identical reports. One no-op repeat is deliberately tolerated — models pause, and the lesson-derivation pass needs a same-finding repeat before it may call a failure recurrent. A second identical no-op buys no information and never will, so the run stops and names it: rejected: stalled. A name, not a shrug — the remaining budget survives, and the artifact says why the run ended instead of just running out. And because every attempt persists into the artifact as it finishes, a run killed mid-flight leaves forensics rather than nothing — the findings that fed each attempt are all there, typed and readable.

Fix 3: teach the judge what absence proves#

The evidence assembly got three corrections and one sentence of epistemics:

• Newest-first, everywhere. The implementor’s work is by definition the newest churn in the workspace, so the deliverable can never fall off a bounded list again.
• Hidden directories pruned; hidden files kept — the same infrastructure/deliverable line the fingerprint draws.
• The allowlist widened — zig, zon, c, h, cpp, sh, sql, and friends are readable evidence now.

And the sentence, appended when the list clips:

const marker = unlisted > 0
  ? ` — ${unlisted} more exist but are NOT shown; absence from this list proves nothing`
  : ""

That marker is my favorite line in either commit. The list can’t be unbounded, so the bound is disclosed and interpreted: an LLM handed FILES: and 400 paths will treat it as a census unless told otherwise. The judge asserting “zig/ does not exist” wasn’t hallucinating — it was drawing the natural inference from an evidence format that implied completeness it didn’t have.

Changing what a judge sees is changing the judge, so the prompt version got bumped and the calibration battery re-run against its live baseline: strict agreement 1.000, false-block 0.00, false-pass 0.00. Evidence fixes regress silently otherwise.

What the fingerprint still can’t see#

Honesty section, because an oracle has bounds too.

It’s a stat, not a hash. touch main.zig counts as movement; a same-size write with a restored mtime doesn’t. The oracle proves immobility, never progress — a coder rebuilding in circles, churning visible build output every attempt, will never trip the breaker. Attempt and budget caps stay the backstop for futility.

Both channels share one blind spot. A shell write inside a hidden directory — or inside the walk’s skip set, say a heredoc patching something in node_modules — is invisible to the diff (pruned) and to the receipt (no tool call). A coder doing all its work there would read as immobile. Today that’s theoretical; it wasn’t obvious that heredocs would be either.

Byte-identical is strict. One timestamp or temp path inside a finding message and three reports never match — a genuine stall goes unnamed and burns to the attempt cap. That asymmetry is chosen: a false stalled kills real work, a missed one only costs budget.

The union’s receipt half is still a claim. A black-box implementor could veto stalled forever by claiming a file it never wrote. Tolerable, because receipts come from the harness’s own tool executor, not model prose — but observed ∪ claimed is only as honest as the claimed half is small.

The skip set is a living document. The judge commit itself added __pycache__ — .pyc churn from running a Python suite had padded the observed diff. Every ecosystem eventually donates a churn directory.

The rule#

A claim channel is fine as a courtesy. The moment a decision keys on it — a kill switch, a verdict, a report a human reads — back it with an observation of the world, and write the observation’s bounds into the message. The forge now runs on three of those: a workspace fingerprint for movement, byte-identical report comparison for novelty, a newest-first list that says what its own truncation doesn’t mean. None of them ask the model to be more honest. They stop requiring honesty in the first place.