Issue #4

Anthropic sets out how it builds safety evals, borrowing reforms from psychology’s replication crisis such as preregistering the analysis. The core advice is to measure the right thing: refusal rate is the wrong measure for self-harm conversations, so Anthropic scores whole multi-turn conversations against expert rubrics, and one bad turn fails the conversation. Simulated users built from expert-written specs escalate over many turns, harmful prompts are paired with benign ones to catch over-refusal, and graders are checked against blind human labels. Anthropic treats 100 items as the floor for a multi-turn eval and 1,000 for a single-turn one. Its single-turn suites are saturated (94–98% for every recent model), and multi-turn confidence intervals can exceed ±10 points, so a 98% vs 94% gap between Opus 5 and Opus 5.5 wasn’t significant. For Fable 5.1, scores differed by up to 34 points between the raw API and claude.ai with its system prompt.

OpenAI has persistent agents across the company, including hundreds of OpenClaw agents, with more than 500 now working proactively in the background, and describes the controls behind them. It treats the agent and the code it writes as untrusted and the gateway that configures it as trusted, so it added remote harness execution to OpenClaw to run the two in separate clusters. The sandbox limits network access to an allowlist with explicit denials (including cloud metadata endpoints), allows writes only to working directories, isolates processes, and keeps secrets behind a credential broker; the post includes the actual Codex config. A second model reviews every agent action, so human approval is reserved for high-value decisions instead of being worn down by constant prompts. Each agent has its own identity and inference credential, so its actions can be audited and switched off, and every configuration change and agent turn is logged. OpenAI also points to OpenClaw Enterprise, an open-source agent control plane still in pilot.

Hydrogen is the fast tier of Gemini 4, priced at $1.50 per million input tokens and $7.50 per million output tokens, flat across a 1-million-token context, with four thinking levels. Google says it edges out its frontier model Argon on the Artificial Analysis Intelligence Index (51 vs 50 at its default setting) and, against Gemini 3.8 Flash, finishes tasks 2.5× faster and triples the Terminal-Bench 4.1 score (57% vs 18.2%). It is positioned as the execution model next to Carbon and Argon: a new Orchestrator in Google Antigravity switches between Carbon and Hydrogen within a session, reaching near-Carbon success on Terminal-Bench 4.0 with 24% less quota and 19% faster. Google also reports 12× fewer misaligned actions and shortcuts across its tool-use and coding evals and 2.3× more resilience to indirect prompt injection than 3.8 Flash.

Prakash builds a stock-ordering agent that runs for days or weeks, on both Temporal and Cloudflare Workflows. Durable execution logs the result of every step that touches the outside world, model calls included, so after a crash the loop replays and skips what’s done; one long-lived workflow per stock item starts a short one per order, and waiting agents use no CPU and woke in under three seconds. The second half covers what durability doesn’t fix. Retries can repeat an order, so the order number serves as an idempotency key; a call that looked failed may have worked, so the agent gets a cancel tool; prices move while a manager decides, so approvals cover a range (the price plus up to 5%) and the order tool re-checks the quote before buying. Deploying a rule change mid-run silently kept the old rule on Temporal and stalled the order on Cloudflare, so rules are versioned. Business rules live in code that decides which tools the model may pick, which also blunts injected text in supplier emails.

A Latent Space podcast write-up with Resend’s CEO on “agent experience”: what it takes to be the vendor a coding agent picks. In one study Claude Code named Resend first in 64 of 102 email recommendations, while SendGrid came first seven times but appeared 55 times as an alternative; a later, broader study had Postmark leading for most models. Resend’s early edge came from its open-source React Email templates, which put its code in repos before developers needed a new provider. Its pricing page returns a readable table when .md is added to the URL, and its MCP server has grown from about 20 tools to more than 80 so a chat client can do what the dashboard does. The episode also covers deliverability, such as why a shared click-tracking domain lets one customer’s phishing hurt everyone else’s mail.

Hugging Face explains decision models, the category TypeSafe started with Jev under the name “System One” models. Instead of generating text, the model takes a state plus questions with allowed answers and returns probabilities: a choice among options, a score on an ordered scale (reported as the expected value), or the probability that a statement is true, each with a confidence measure. The post sets this against asking an LLM for JSON (schema violations and decoding overhead) and against zero-shot classifiers, whose ideas it extends into one typed interface. Hugging Face has added a decision pipeline and model classes to Transformers, a /v1/systemone endpoint that llama.cpp also serves, a Hub tag and fine-tuning through the Trainer API, with TRL support still pending. The interface also works with ordinary LLMs, scoring the answer labels in one forward pass without generating text.