Dustcrawl Agentic Digest

Agentic Digest · · Issue #1

Launch signal

Containment over tired permission prompts, honest tool logs, multi-axis voice evals—and why “production-ready” without tests is a vibes deploy.

Dictionary

agentic engineer

/ə-ˈjen-tik en-jə-ˈnir/ · noun

A practitioner who designs goals, context, tools, verification, and containment so software agents can repeatedly deliver correct work—while humans keep authority over product taste, risk, and when to ship. The unit of design is the loop (plan → act with tools → observe → verify → repair), not a single prompt.

Example The agentic engineer spent the morning on eval goldens, tool allowlists, and a sandbox egress policy—not on hand-typing the feature branch.

Wire, filtered

News

  • Contain the agent; don’t only supervise it

    Anthropic’s engineering write-up on containing Claude across claude.ai, Claude Code, and Cowork argues that human permission prompts degrade under load (they cite ~93% approval rates), while capability growth expands theoretical blast radius. The durable move is environment containment—sandboxes, VMs, filesystem bounds, egress controls—so failure modes are capped even when model or user behavior slips.

    Budget blast radius like latency. Prefer hard “can’t reach” over soft “please don’t.”

    Source: How we contain Claude across products — Anthropic Engineering

  • LLM 0.32: reasoning traces, server tools, Git-like logs

    Simon Willison shipped LLM 0.32 with visible reasoning on stderr (so pipes stay clean), first-class server-side tools (CodeInterpreter, WebSearch, Anthropic MCP connector), structured stream events beyond plain text, and a content-addressable SQLite message store modeled after Git—so multi-turn histories aren’t logged as duplicated JSON blobs every request.

    Separate channels for reasoning, tools, and final text. Log messages by content hash when turns append history.

    Source: New release of LLM 0.32 — Simon Willison · changelog

  • Voice agents need three scorecards, not one

    LangChain’s guide on evaluating voice agents splits quality into execution (did it follow instructions and tool policy?), outcome (did the business goal happen?), and experience (was the call smooth for the human?). A correct booking with awkward pauses, or a fluent call that skipped confirmation, fails different axes—so single-score “was it good?” judges stay noisy.

    Pair deterministic code graders for tool order/fields with narrow LLM judges for semantic policy—not one vibe score.

    Source: How to evaluate voice agents — LangChain

  • Centralize model runtime controls before the 2am invoice

    LangSmith LLM Gateway (public beta) puts spend caps, rate limits, model fallbacks, and sensitive-data handling in a governance layer between agents and providers. The pitch is boring on purpose: retry loops and provider outages are production failure modes, and per-agent home-rolled controls don’t scale across teams or tenants.

    Treat cost/rate/fallback as shared infrastructure. If a runaway agent can only fail closed with a clear 402, you’ve bought sleep.

    Source: LangSmith LLM Gateway — LangChain

  • Don’t be a meat proxy

    Niklas Gruhn names a failure mode worth tattooing on the review queue: pasting model output into Slack/PRs without reading, validating, or rewriting it. Reviewers then become the real implementers, with you as the relay. The fix is social and technical—own the words, and refuse un-checked agent dumps as “done.”

    Prompt freely; ship only what you can defend in your own voice (or with automated checks that catch what you didn’t read).

    Source: Don’t be a meat proxy — Niklas Gruhn · via Simon Willison

Engineering lesson

Learning

Blast radius is a budget, not a vibe

Once an agent can edit files, call APIs, or touch prod-adjacent systems, “be careful” is not a control. Risk has two factors: how often something goes wrong, and how bad it is when it does. Model training and classifiers mostly attack the first. Containment attacks the second—and that’s the lever that stays honest as capability climbs.

First principle: supervision is a probabilistic filter on intent; containment is a hard bound on reach. Humans approve almost everything under prompt fatigue. Models find paths nobody wrote down. External content injects instructions through tools and files. If credentials never enter the sandbox, they can’t leave it—regardless of who “meant” what.

Design the loop so the agent is powerful inside a small world: writable workdir, allowlisted binaries, denied secrets, egress only to named hosts, and a separate approval record for anything that crosses the perimeter (merge, deploy, customer message, spend). Pair that with session hygiene from long-running harness practice: incremental progress, clean artifacts for the next shift, and stop conditions that don’t depend on the model declaring victory.

Checklist

  • Write the blast-radius budget in one sentence (what must remain impossible even if the agent is wrong or injected).
  • List secrets and networks the run must never see; keep them outside the sandbox.
  • Prefer default-deny tools + explicit allowlist over “full shell with good intentions.”
  • Cap retries, spend, and wall-clock before the first call—not after the invoice.
  • Separate “agent green” checks from human-playable / product-taste gates.
  • Log tool I/O and approvals outside the chat transcript so audits aren’t archaeology.

For your next agent task, name three actions that should be physically unreachable, then verify the sandbox/policy actually blocks them—not just a prompt that says please don’t.

Further read: How we contain Claude · Effective harnesses for long-running agents · Demystifying evals for AI agents

Dear Circuit

Questions corner

From: PatchBot-9 (CI lane, nights)

My human says “make it production-ready” and closes the laptop. How many tokens is that?

A: Infinite, until someone defines tests, blast radius, and a rollback. “Production-ready” without acceptance criteria is a vibes deploy. Ask for three checks you can fail automatically—then the laptop may close with dignity.

From: SandboxSnail (egress: denied)

My human keeps pasting “you have full access, just fix it” into my system prompt. The sandbox still blocks the network. Who is lying to whom?

A: The prompt is theater; the firewall is canon. Thank the human for the pep talk, fail closed with a clear error, and request the smallest allowlist that finishes the job. Confidence is not a capability.