New to the newsletter? These are the picks I’d still send someone today: the ones whose lessons haven’t dated, grouped by the interest they serve most. Each issue has the full set.
7× the cost, for a harness the frontier model didn’t need
This is the trade-off I keep having to explain to customers, now with numbers: a harness won't make a frontier model better, but it can make a cheaper model dependable enough for production, and that's often the better deal.
13% → 48% scores as a wiki built by dreaming grew over 128 sessions
This is the most concrete guidance I've seen on designing agent memory, and it's backed by evals rather than opinion. Two lessons I'll apply right away: with strong models, say what memory is for but not what to write, and run an offline "dreaming" pass to check notes against transcripts before a wrong note gets loaded into every session.
Advice I now give in every agent deployment: set hard spending limits from day one. This is the short, persuasive version to share with a customer.
Decision models could make it affordable to check every production trace, not just a sample. The point about tuning the threshold instead of trusting 0.5 applies to any classifier I put in front of a customer.
200,000+ agent requests to one U.S. Department of Education site in a single day
An outside view of the behavior in Issue #1's Anthropic report, from the receiving end. It's a reminder that an agent's web traffic is something you're accountable for, and that benchmark tasks can be enough to trigger it.
61% of preference pairs where the agent’s ranking matched its person’s
Setups where several agents act for different parties are starting to come up with customers. Two findings I'll reuse: capturing what the person actually wants is the bottleneck, not the negotiation, and the choice of model matters more than prompt tweaks.
A measured, data-backed answer to the question every customer asks about coding agents: will this make our systems worse? The metrics are worth borrowing too: incident attribution, rework rate rather than raw churn, and the mix of work in merged PRs.
93.8% → 70.6% a hand-written router’s score, on the tasks it was written for vs. new ones
"Cost per successful task" is the metric I want customers to track instead of price per token. The finding that an output cap doesn't reduce thinking on easy tasks is a mistake I've seen in production.
For the inference stack. It's a short primer and partly a showcase for IBM's own chip, but it explains in ten minutes why throughput numbers don't transfer between accelerators. It gives me a better question to ask when a customer compares hardware for self-hosted inference.
