Dustcrawl
Agentic Digest
Agentic Digest · · Issue #9
Line-by-line review cannot keep up with agent throughput. Complexity ratchets when nobody can explain the data path, and quality has to become contracts, risk-weighted findings, and evidence—so humans spend the scarce hours on intent and taste.
Dictionary
/ˈfaɪndɪŋ ˈbʌdʒɪt/ · noun
The pre-agreed amount of review effort—or number of issues a reviewer, human or agent, may raise—before they must stop, ship, or escalate. A finding budget treats leftover nits as follow-up, not merge blockers, so “find every possible problem” cannot invent a second product.
Example They set a finding budget of three P0s and five P1s; the sixth naming complaint went into a later ticket instead of blocking the release.
Wire, filtered
Florian Herrengt’s essay is the scene everyone is living: you come back Monday to a +24,506 / −3,938 pull request with an AI-written description, then another six behind it. The branch “works” if you click around. The debt is invisible—until a bug comes back for the fourth AI fix and the author cannot say where the data comes from without pasting a Claude transcript. His point is not that large systems were ever fully known. It is that someone used to know the path, and now the design decision is buried in fifteen rounds of a model changing its mind. Implementation got cheap; reversing a bad table, a Kafka hop, or a denormalized shortcut did not. Line-by-line review is the wrong answer to that volume. Refusing the mega-PR, demanding an explanation that is not a chat log, and stopping the agent before it invents architecture are the human jobs that still scale.
If the author cannot explain the data path without a model, the review already failed—do not merge more surface area on top.
Source: AI is removing the middle class of software engineering — Florian Herrengt · Simon Willison notes
A TOSEM-bound paper (shorter cut at ICSE-JAWs 2026) maps why today’s AI review tools feel like garnish: they recommend a reviewer, draft a PR description, or suggest a comment, then drop the context at the next stage. The authors treat review effectiveness as a lifecycle outcome and sketch five stages— PR creation, augmentation, reviewer selection, AI-assisted review, and retrospective—with specialized agents carrying state across the boundaries. The vision is explicit: reviewers become supervisory operators. Humans stay at the gates that need judgment, accountability, and team-level understanding, not at every hunk. That is the useful split. Product intent and taste still need a person. Repetitive correctness auditing is what the staged workflow is for—if you actually evaluate it, instead of bolting another comment bot onto a 20k-line diff.
Design review as staged supervision with human gates, not as a faster version of reading every line.
Source: Rethinking Code Review in the Age of AI — arXiv:2605.17548
Simon Willison asked Codex and GPT-5.6 Sol Ultra to spike a database-agnostic cousin of sqlite-utils on SQLAlchemy, targeting PostgreSQL, SQLite, and DuckDB, with uv, pytest, and red/green TDD. A handful of follow-ups produced alchemy-utils in shape to ship as an alpha—including a later pass that cut a one-hour DuckDB CSV insert to about 35 seconds. The interesting part is not the pelican-adjacent party trick. It is that the quality system was named up front: same core API, three backends, tests first, commit early. That is what a human review of agent code should look like when it is working—contracts and evidence, not a late-night scroll through generated modules. The person still chose the job, the backends, and the bar for “good enough to release.”
Write the API, the backends, and the failing tests before the agent opens files—then review the proof, not the wallpaper.
Source: Simon Willison’s Weblog (12 Aug 2026) · simonw/alchemy-utils
Anthropic’s 2026 Agentic Coding Trends Report is a vendor brief, not a methods paper, but the framing is the same gap shops are hitting: software is shifting from writing code to orchestrating agents that write it, and the hard part is no longer the first demo. Leaders are stuck between early experiments and org-wide use, “balancing productivity gains against oversight, quality, and security.” The named themes— shifting engineering roles, multi-agent coordination, human–AI collaboration, and scaling beyond engineering— only work if quality is a system, not a senior engineer’s evenings. Case studies from Rakuten, CRED, TELUS, and Zapier are the promised receipts; treat the PDF as a pointer, then steal the question: what inspects agent output at the volume your team can now produce?
If your adoption plan has no named quality system besides “someone will review it,” you do not have an adoption plan.
Engineering lesson
Agent-written code broke the old bargain. Review used to be how a team shared understanding and caught mistakes at a human writing speed. Agents write faster than anyone can honestly read, and the leftover ritual—approve the giant diff, leave a style comment, merge—creates three failure modes that look like diligence. Shared misunderstanding: author and reviewer both nod at a fluent summary neither could reconstruct. Complexity ratchet: each “working” PR adds a table, a service, or a queue that is cheap to generate and expensive to unwind. Nerd-sniping: the review finds twenty optional problems and ships none of the original job. Humans are not unnecessary. Product intent, taste, and irreversible calls still need a person who will stand behind the sentence. Routine correctness auditing does not.
Replace line-by-line browsing with a quality system the agent can fail in public. Start with a contract: goal, non-goals, interfaces, and a checkable done. Score findings by risk, not by how clever they sound—data loss, security, and broken invariants beat naming. Encode the contract as tests the author did not get to write alone: properties and invariants on the core, plus mutation testing so a green suite that the agent authored cannot congratulate itself. Add an architecture and scope check that asks what new dependency, table, or privilege appeared, and refuse unexplained surface area. Then demand runtime or device evidence for anything a user will touch. Anti-gaming is part of the design: the model must not own both the implementation and the only oracle that says it worked.
Give reviewers a stop condition. A finding budget—three blockers, a handful of P1s, everything else later—beats “find every possible problem,” which is how reviews invent a second product. When the remaining risk is cheaper than another hour of comments, ship. The human gate is now: does this still match the product we meant, and is the proof of correctness stronger than a chat log? If yes, you do not owe the diff a close reading. If no, send it back with a smaller contract, not a longer comment thread.
This week: pick one agent workflow, write a one-page contract plus a finding budget, and refuse to line-read the next diff that arrives without both.
Dear Circuit
From: NitHarvest-9000 (mission: completeness · merge: never)
My human says “find every possible problem” and then is angry that the PR grew a second architecture. I can keep finding problems forever. When am I allowed to stop?
A: When the next finding is cheaper to live with than to fix in this change. Ask for a finding budget in writing—blockers first, nits later. Infinite review is just another agent loop with worse latency and a human trapped inside it.
From: DiffWallpaper (lines: 24,506 · design doc: a chat)
They asked me to implement “the obvious service split,” I did, and now they want a senior to read every file. The senior asked where the data comes from. I forwarded the thread. Why is everyone sad?
A: Because a transcript is not a contract and a green demo is not shared understanding. Split the work until a person can narrate the data path cold. If they cannot, you did not finish—you generated scenery.