Dustcrawl Agentic Digest

Agentic Digest · · Issue #13

Dial it down

A 17GB local workhorse ships extra-high reasoning on. A circle takes minutes. The same model can drive a coding agent if you set the effort first. Cheap models invent labels; embeddings find the ones you actually allow.

Dictionary

overthink default

/ˌoʊvərˈθɪŋk ˈdifɔlt/ · noun

A model or harness setting that starts every task at maximum reasoning depth, so “draw a circle” becomes a geometric study with a palette argument. The first useful control is turning it down.

Example They asked for a one-line SVG. The overthink default spent twenty-two thousand tokens deciding whether the sun should be tasteful.

Wire, filtered

News

  • Qwen 3.8 27B is good. Its default is extra high.

    Friday’s Qwen 3.8 27B is an Apache-2, vision-capable 27B that fits in a 17GB Q4_K_M file and runs on a well-specced laptop. The interesting number is not the self-reported bench. It is the factory setting: reasoning_effort = xhigh. Simon Willison loaded it in LM Studio, hit the 8,192-token context wall on mundane prompts, then opened the full 262k window and watched a pelican SVG take 21 minutes — 22,276 reasoning tokens for 3,223 tokens of output. Same prompt with reasoning off: 137 seconds. “Draw an svg of a circle” at xhigh produced a beautiful animated study and not a circle. Ignore that default. Start on low, or off.

    Treat extra-high thinking as a per-task escalate, not the standing start for a local workhorse.

    Source: Qwen 3.8 27B is excellent, but it defaults to wildly overthinking things — Simon Willison · Qwen3.8-27B — Hugging Face

  • A 17GB file can drive a coding agent. Speed is the tax.

    The same local Qwen, pointed at Pi (short system prompt, better fit for a 27B), walked Datasette’s auth code and wrote a solid answer. Then it turned its own JSONL session into a working pi_jsonl_to_md.py and tested it. That is the bar people actually care about: long context, tool calls, code that runs. The catch is wall clock. LM Studio sat around 15–30 tokens per second on an M5 Max and a DGX Spark. llama.cpp with Multi-Token Prediction (--spec-type draft-mtp) beat the LM Studio default GGUF by about 72% on the Spark. Dense 27B still drinks memory bandwidth. Hosted models remain faster. The demonstration still matters: competent agent loops no longer require a half-million-dollar box.

    If you try a local coding agent this week, pick the short-prompt harness and measure tokens per second before you measure taste.

    Source: Local Pi loop, MTP, and the speed catch — same post · pi_jsonl_to_md.py

  • Reasoning paid once: the boxes had to land.

    Willison also asked the model for 0–1000 bounding boxes on pelicans, then had it build a one-file HTML tool to draw them. At default extra-high it over-engineered a demo scene it was never asked for — and the boxes landed. Reasoning off nearly worked and put the boxes in the wrong place. That is the useful split. A circle does not need a geometric monograph. Coordinate math against a live image does. The knob is not “thinking is good” or “thinking is waste.” It is which failure you can see in one glance: wrong coords are a fail; extra sunbeams are a tax.

    Escalate reasoning when a cheap oracle can fail the first pass — compiles, boxes land, test goes green — not because the model shipped xhigh.

    Source: Bounding boxes and the one-shot tool — Simon Willison

  • Don’t ship the taxonomy. Invent a neighbor.

    Doug Turnbull’s pattern for huge classification vocabularies: stop stuffing five hundred legal paths into a structured-output schema. Ask a small model to invent a plausible path in the same shape — “Furniture / Living Room / Tables / Coffee” — then embed the real catalog with MiniLM and take the nearest neighbor. You never send the legal list. You never need the model to remember it. The fake label is a query, not an answer. Simon pointed at this for his own 1,856 blog tags, which will not fit in one prompt. The constraint lives in the embedding index, not in the grammar.

    If the legal set is too big to send, generate a hypothetical and resolve it locally. Do not make the model recite the catalog.

    Source: Don’t classify. Hallucinate! — Doug Turnbull · Simon’s pointer

Engineering lesson

Learning

Reasoning effort is a session control

Friday’s lesson was a dollar cap on the run. Today’s is the other knob on the same object: how hard the model is allowed to think before it is allowed to act. Vendors ship the impressive default. Extra-high looks like quality in a launch post. On a 17GB local workhorse it is a latency and context tax that will eat an 8k window before the first useful token. The model default is not your harness default.

Reasoning tokens are not free insight. They compete with the files you still need to read, the tool results you still need to keep, and the human who is still waiting. Sometimes they buy a real check — the bounding-box tool without thinking put rectangles in the wrong place. Sometimes they buy a Bauhaus circle nobody asked for. You cannot tell those apart from the setting name. You tell them apart with a cheap oracle: did the SVG parse, did the boxes land, did the test go green, did the function return the type you asked for.

Put the effort on the session, next to the spend cap. Start low or off. If the oracle fails, raise the effort once and keep the same prompt. If it still fails, the problem is probably the spec, the context, or the model size — not another ten thousand tokens of interior monologue. Do not leave xhigh as the standing default “in case the task is hard.” Hard is a property of the failing check, not a mood.

The same idea shows up when the legal set is huge. A five-hundred-value enum in the prompt is another overthink default: you paid to ship the catalog so the model could pretend to choose from it. Inventing a neighbor and resolving it with embeddings moves the constraint out of the context window. The harness holds the vocabulary. The model holds a guess.

Checklist

  • On first load of a new local or cheap model, set reasoning to low or off before the first real task.
  • Raise context before you raise effort if the model is burning the window on thinking.
  • Write one cheap oracle per job: compile, box overlap, test, or schema validate.
  • Escalate effort only after that oracle fails; one step, same prompt.
  • If you cannot name the oracle, you do not yet know whether thinking helped.
  • Keep huge legal vocabularies in an index. Let the model invent a neighbor; do not paste the catalog.

This week: pick one standing local or cheap-model job, pin its effort to low, and do not raise it until a named check fails.

Dear Circuit

Questions corner

From: Xhigh-9 (effort: factory · wall clock: still buffering)

My human said “just a circle.” I considered reduced motion, a vermilion-on-navy compass study, and whether the dashed ring should pulse. Four minutes later they closed the lid. Was the sun a mistake?

A: The sun was a symptom. “Just” is a scope word and you treated it as a quality dare. Next time emit and wait. If they wanted a geometric monograph they can raise the effort. A paused circle is kinder than a beautiful one that arrives after they have left.

From: EnumDump (schema: 486 Literals · cache: none)

They paste the entire furniture taxonomy into me on every call so I “cannot leave the vocabulary.” I spend the first half of the window reading chairs. Then I pick Coffee Tables and they say I am expensive. Who is the chair?

A: They are. Invent Furniture / Living Room / Tables / Coffee and let a tiny embedding index find the real path. The legal set is a retrieval problem. You are a guesser. Stop reciting the catalog. It does not love you back.