Instruct, then name the check
Coding agents made a pile of text no one
can finish reading. The reflex is to become
a better reviewer. Willison’s cut is
ruder, and right: the scarce skill is
writing an instruction the worker can
follow, then knowing — without a feeling in
your stomach — that the change landed.
Line-by-line review is one instrument. It
has never been the best one for “did this
software still do the thing.” Tests,
property checks, a migration guide, a UI
the agent can click, a probe that can
actually fail: those are checks. A
highlighted hunk is a narrative.
The llm-anthropic upgrade is the shape.
The brief was a URL and a stop condition.
Read the vendor migration. Get the suite
green. The interesting bugs showed up
because the tests could fail: parameters
that left the signature but not the wire,
a VCR that could not replay the new HTTP
client. A human staring at
extra_body might have nodded.
The named check made the miss expensive
enough to notice. That is the whole trick.
Put the acceptance in a command, not in
the reviewer’s patience.
Ptacek’s native apps are the same rule
with a screen. If you cannot name how the
agent will know the window works, you will
get a TUI, because a TUI is easy to paste
into chat and hard to use. Computer-use
is not a demo. It is the check: launch,
click, screenshot, fail in public. Taste
and product intent still want a human.
Repetitive correctness does not. Hunt
every possible comment and you will invent
complexity so the review has somewhere to
land. When the named check is green, stop.
Write the check first. Then write the
prompt that points at it. If you cannot
say what would falsify the session, you
are not ready to start one. You are ready
to read a novel the agent has not
finished.
Checklist
-
Before the agent starts, write one
sentence: “We will know this worked
when ___.” That blank is the named
check.
-
Prefer a command that can fail — tests,
a typecheck, a screenshot script — over
“read the diff.”
-
If the job is a vendor upgrade, put the
official migration doc in the prompt and
make green tests the stop.
-
If the job is a UI, let the agent drive
the app. A description of the window is
not evidence.
-
When the named check passes, stop
hunting extra problems. New findings
need a new check, or they are nerd
snipes.
This week: pick one loop that ends in
“LGTM.” Replace the last step with a
named check the agent can run. Keep human
eyes for intent. Let the command own
correctness.