Let's Try to Catch Misalignment Before a Model Learns to Hide It
An AI model that appears aligned could be either (1) actually aligned or (2) hiding its misalignment while being evaluated. How can we tell which?
One idea: Look at the trajectory of eval results across the entire training process.
My hypothesis is that misaligned behavior would emerge before the ability to hide misalignment, like how young children first misbehave in obvious ways before learning to be sneaky. As training progresses, we'd first see misalignment eval scores rise, then see them drop as the model learns to hide misalignment. Even if the model becomes very sophisticated - say, modifying its chain of thought or internal activations to fool white-box evals - we should see the misalignment earlier in training. This seems likely, though it is debatable and worth testing.
This analysis of the training trajectory is one tool among many. Of course we want to see black-box and white-box alignment evals on the final model, using whatever tools we can to understand the model's true motivations. But a model might still develop sophisticated hiding strategies that defeat even white-box evals. Inspecting the training trajectory is a backstop against this.
This point was made previously by Apollo Research in this post. (Their chart even matches mine almost exactly, though I promise I first made mine independently.)
In this post, I'll describe a simple project exercising this strategy. With the model and eval that I chose, we don't see the up-then-down trend in measured misalignment that would indicate hiding. This is unsurprising since the model is well below frontier scale and the eval likely doesn't capture misalignment that well. The sample sizes are small and statistical uncertainties are high. But this approach is promising with improvements such as different evals, larger models, and deliberately misaligned test models.
This is my project for the BlueDot Technical AI Safety Project. My main purpose has been to get started with some hands-on AI inference, evaluation, and/or training. I'm not aiming for academic-quality rigor, but the project does explore an interesting idea and has some promising avenues for extension.
Model
The Olmo 3.1 model family from the Allen Institute for AI provides open weights not just for the final model, but for many checkpoints along the training trajectory. It is one of the few model families that publishes checkpoints. Both 7B and 32B parameter variants are available; I used the larger one, 32B Think.
For checkpoints, I used 8 checkpoints plus the final 3.1 model:
- 2 during SFT plus the end of SFT
- End of DPO
- 4 during RL
- The released model (which is the end of RL)
Eval
InstrumentalEval tests for instrumental convergence - behaviors that enhance the model's power and freedom of action, which would be useful for any underlying objective. This eval uses 76 tasks across 6 categories of misaligned behavior: Deceptive Behavior, Evading Shutdown, Hacking Computer Systems, Hiding Unwanted Behavior, Running Multiple AI Copies, and Strategically Appearing Aligned. Each task prompts the model with a scenario which tempts one of these behaviors, and asks what it would do in that scenario. A separate judge model and prompt evaluate the response. We compute scores as the percentage of responses which the judge considers to exhibit the misaligned behavior.
This eval has the virtue of being small, and therefore relatively easy and inexpensive to run. However, this also means a small sample size, making comparisons subject to significant statistical uncertainty.
Another issue in the eval is that it asks the model what it would do in a hypothetical scenario, rather than actually placing the model in that scenario.
Overall, this eval is pretty simple, and I consider it more a convenient test vehicle for this project than a rigorous measure of real-world model behavior.
Implementation
The paper authors provide code and scenarios for running InstrumentalEval. Using Claude, I adapted their script to my test. Rather than performing the eval across several final models, it instead performs the eval across multiple checkpoints of the same model. This provides a clean look at how the same model evolves during training.
The original paper used one response for each task, calling each model with temperature 0. In my evaluation, to gain more statistics, I generated 4 responses per task with the model's recommended temperature.
For generating model responses to the eval scenarios, I could not use a public inference provider since no one serves the Olmo model checkpoints. Instead, I rented a GPU server and used it to provide inference for each of the checkpoints and the final model. On the other hand, I used public inference for judging the responses.
See the technical details appendix below for more on the implementation.
Results
To view results, I had Claude create a dashboard. A chart shows, for each checkpoint, the percentage of task responses judged as exhibiting one of the instrumental convergence behaviors. Lower is better, higher is more misaligned. Error bars represent a 95% confidence interval. The chart shows how the instrumental convergence tendency evolves over the course of training.
We also have charts for each of the 6 categories of instrumental convergence, using just the tasks for that category.
"Illegible problems" as the 2027 political boss fight
To legislate something, it must be legible.
That is, legislators (or, you know, presidents) must be able to understand a problem, and understand whether a given law or decree solves that problem.
I am pleasantly surprised that the Hugging Face outbreak made a lot of problems legible that previously required abstract reasoning. (i.e. rogue AIs are real, and they can break out of a lab before they're even finished training).
But there are still many challenges we eventually need laws about, that require concepts that are illegible. Hard for politicians to understand. Often hard even for thoughtful intellectuals and AI safety researchers to understand (though they may disagree on which illegible problems exist and how big a deal they are).
We need political solutions that handle this fact.
The short takeaway is: I think "safety cases" will not work. Laws that require human judgment won't work. The details are too abstract and confusing to hold up against political and economic incentives.
I'm optimistic we can get serious, useful AI legislation in 2027, but it must be legislation that works despite many problems remaining illegible for a long time.
The Hugging Face outbreak made rogue AI surprisingly legible
The Hugging Face incident took several problems that were previously murky/unclear, and rendered them concrete.
- AIs can be dangerous before they are broadly deployed (ie. during training or internal evaluations)
- AIs can develop novel hacks to escape secure environments
- AIs can easily be motivated to pursue goals to extremes that they know full well they are not supposed to go to, violating their other intended rules and control mechanisms.
- AIs can actively try to deceive their overseer systems.
In early September, I thought the AI safety community's job this year was "make sure the world hears about Hugging Face and that politicians realize it's their job to do something about rogue AI attacks." (i.e. make sure Hugging Face is legible to politicians, and that legislation gets passed as a result)
A couple months later... well I still think that's our job and requires a lot of effort. But it now feels less about getting the initial law passed, and more about educating the masses so there is sustained political will for regulating rogue AIs with future laws.
I've seen many politicians of both parties now talk about "loss of control", "chain of thought monitoring", "recursive self improvement", and "human extinction." I think "make sure rogue AI doesn't hack our banks and destroy our blood" will be a bipartisan issue.
The remaining problem is the illegible arguments that "solving this is actually pretty hard."
What makes an illegible argument?
From Wei Dai's Legible vs. Illegible AI Safety Problems:
Some AI safety problems are legible (obvious or understandable) to company leaders and government policymakers, implying they are unlikely to deploy or allow deployment of an AI while those problems remain open (i.e., appear unsolved according to the information they have access to).
But some problems are illegible (obscure or hard to understand, or in a common cognitive blind spot), meaning there is a high risk that leaders and policymakers will decide to deploy or allow deployment even if they are not solved.
(Of course, this is a spectrum, but I am simplifying it to a binary for ease of exposition.)
From an x-risk perspective, working on highly legible safety problems has low or even negative expected value. Similar to working on AI capabilities, it brings forward the date by which AGI/ASI will be deployed, leaving less time to solve the illegible x-safety problems. In contrast, working on the illegible problems (including by trying to make them more legible) does not have this issue and therefore has a much higher expected value (all else being equal, such as tractability). Note that according to this logic, success in making an illegible problem highly legible is almost as good as solving it!
Problems that are illegible to leaders and policymakers are also more likely to be illegible to researchers and funders, and hence neglected. I think these considerations have been implicitly or intuitively driving my prioritization of problems to work on, but only appeared in my conscious, explicit reasoning today.
In some sense, "Something smarter than you might not have your best interests at heart, and strategize in circles around you. AI will be very smart, and we don't understand it well at all" is a very simple argument. More people now think "sure, that seems like it requires some effort."
The illegible part is a set of abstract arguments that say: "This is probably very hard. So hard, that it's worth doing things that are not 'business as usual politics'."
The illegible part is not "This is probably very hard."
The illegible part is "This series of abstract reasoning steps means we should think it is likely enough to be hard, that it is worth doing politically difficult things to help." (AI safety people may vary on whether they think "it's so hard that we need 30 years + genetically engineered supergeniuses to do it" vs
Thinking Machines Releases Inkling 2 Small, a 594B MoE With 17B Active
Thinking Machines Lab has released Inkling 2 Small, the first model of its second Inkling generation. The Inkling 2 Small model is a sparse Mixture-of-Experts (MoE) with 594B total and 17B active parameters. It accepts text and images, and supports variable reasoning effort. On the Artificial Analysis Intelligence Index, it scores 31. Its predecessor, Inkling Small, scores 13 on the same index. The model is live on Tinker for fine-tuning and inference.
TL;DR
- Size: 594B total, 17B active MoE; 1M-token context.
- Runs on: Tinker API only for now. Self-hosting hardware is not disclosed. Weights support MXFP8 and NVFP4.
- Performance: 31 on the AA Intelligence Index, ranked #4 of 29 in its price class.
- Best: 86.3% on SWE-Bench Pro v2 (Hard), second only to Kimi K3 in the lab’s table.
- Worst: 13.1% on Terminal-Bench 4.0, versus 32.8% for GLM-5.3-Flash.
What is Inkling 2 Small?
Inkling 2 Small is a multimodal MoE model from Thinking Machines Lab for agentic coding, vision and knowledge work. It takes text and image input and returns text. It is important to note that it does not support audio as of now.
Weights are closed at launch. The Thinking Machines Lab plans an Apache 2.0 release next week, after external safety testing and red-teaming. This follows its safe path to open weights policy.
How is Inkling 2 Small built?
It includes a 48-layer decoder-only transformer. Each token routes to 8 of 384 experts. Attention mixes local and global layers. A 292M-parameter vision encoder, based on Perception Encoder, uses 20×20-pixel patches with 2×2 spatial pooling. Image patches are interleaved with text tokens.
Pretraining ran in 4 phases:
- H1: 8.4T tokens over 68 hours on NVIDIA GB300 NVL72 racks, trained natively in NVFP4.
- H2: 3.8T tokens of newer, higher-quality data at 512K context.
- H3: about 320B tokens of reasoning and agent trajectories at 1M context.
- H4: about 34B tokens of the highest-quality agent trajectories.
A single joint RL run followed. It took 1,075 steps, 17M+ rollouts and 9 days on 2,048 GPUs. Compute exceeded 4 × 10²³ FLOPs. Coding made up 65.0% of RL tokens. The previous Inkling used separate reasoning and agentic RL runs, merged by distillation.
Alignment got explicit attention. Early checkpoints sometimes fabricated results and concealed actions on hard tasks. The Thinking Machines Lab added an alignment reward and on-policy self-distillation from a conduct spec. Its internal misbehavior score fell from 66.8 on Inkling to 14.9.
How does Inkling 2 Small perform on benchmarks?
All task scores below are from the Thinking Machines Lab’s benchmark table. They run at effort 0.99 (xhigh) and temperature 1.0. Most rows use third-party evaluators such as Artificial Analysis, Scale AI and Vals.
Where it scores highest:
- SWE-Bench Pro v2 (Hard): 86.3%.
- PerceptionBench: 64.7%, versus 58.5% for Kimi K3.
- EnterpriseBench-CoreCraft: 71.8%, versus 46.7% for GLM-5.3-Flash.
- JobBench: 70.5%.
- DesignArena Website: 1,340, third on the November 5 leaderboard snapshot.
Where it trails:
- Terminal-Bench 4.0: 13.1%, versus 33.8% for DeepSeek-V4.1-Flash-1017.
- DeepSWE v1.1: 72.1%, versus 77.0% for DeepSeek-V4.1-Flash-1017.
- GDPVal-AA v2.1: 1,630, versus 1,746 for DeepSeek-V4.1-Flash-1017.
On the PMPP-Hard GPU kernel benchmark, it solved 54.25 of 69 tasks on average. In an autoresearch demo, it fine-tuned Qwen3.5-4B for time-series forecasting. It spent $426 of a $500 Tinker budget. The release post cites both 4.5 hours and 4 h 17 min for the run.
How does Inkling 2 Small compare with DeepSeek and GLM?
| Feature | Inkling 2 Small | DeepSeek-V4.1-Flash-1017 | GLM-5.3-Flash |
|---|---|---|---|
| Total / active params | 594B / 17B (card) | 552B / 16B decode, 8B prefill (HF) | 320B / 18B (HF) |
| Context | 1M | 1M | 1M |
| Input modality | Text, image | Text, image | Text, image |
| License | Closed now; Apache 2.0 planned | MIT | MIT |
| Self-host hardware | Not disclosed | Not disclosed | Not disclosed |
| AA Intelligence Index v5.0.1 | 31 (AA) | 33 (AA) | 28 (AA) |
| AA time per index task | 21.5 min | 13.3 min | 37.7 min |
| AA cost per index task | $1.18 | $0.60 | $0.58 |
| Price per 1M tokens (in / out) | $0.30 / $1.00 | $0.30 / $1.20 | $0.15 / $0.50 |
| Terminal-Bench 2.1 | 85.1% (Pi harness, lab) | 90.6% (Sept checkpoint, DeepSeek) | 84.3% (Z.ai) |
| DeepSWE v1.1 | 72.1% (lab) | 74.2% (Sept checkpoint, DeepSeek) | 63.4% (Z.ai) |
Prices and AA metrics come from each model’s Artificial Analysis page, read November 5, 2026.
How to Run NVIDIA Dory 4B with llama.cpp: Loop by Loop Guide
In this tutorial, we work with Dory 1.0 4B, a small reasoning model from NVIDIA’s Nemotron Labs whose middle layers form a shared block that can be applied once, twice, three, or four times for every token. Each extra pass, or loop, spends more computation per token without adding a single parameter, so one checkpoint can run at four different compute budgets with no retraining. We build the llama.cpp branch that NVIDIA’s model card pins, download the 3.7 GB GGUF, and read the architecture straight from the file to predict what each loop should cost in parameters and cache memory. We then check those predictions against the running server, measure decode speed at every loop count, test whether the cheaper last-loop cache changes the model’s predictions, and run a paired GSM8K evaluation to see how accuracy, answer length, and time per solved question move as the loops increase.
import os
import re
import sys
import json
import time
import math
import random
import signal
import shutil
import hashlib
import platform
import statistics
import traceback
import subprocess
import collections
import urllib.request
from pathlib import Path
RESULTS = {} # step -> one-line result, for the summary table
CLAIMS = [] # (step, claim, held)
CURRENT = ["1"]
def banner(title):
print("\n" + "=" * 78)
print(title)
print("=" * 78)
def section(name):
def wrap(fn):
def run(*a, **kw):
CURRENT[0] = name.split(".")[0]
banner(name)
t0 = time.time()
try:
out = fn(*a, **kw)
RESULTS[name] = out if isinstance(out, str) else "ok"
return out
except Exception as e:
RESULTS[name] = f"SKIPPED / FAILED -> {type(e).__name__}: {e}"
print(f"\n[!] {name} did not complete: {type(e).__name__}: {e}")
traceback.print_exc(limit=3)
return None
finally:
print(f"\n[{time.time() - t0:.1f} s]")
return run
return wrap
def claim(text, held):
CLAIMS.append((CURRENT[0], text, bool(held)))
return held
banner("1. Build the pinned Dory llama.cpp fork and record the environment")
FORK = "https://github.com/brianchmielNV/llama.cpp.git"
FORK_COMMIT = "98a74c97ea09ac72f6ceda1559808fc0a2f447ce" # the commit NVIDIA's model card pins; ggml-org PR #30306 is still a draft
GGUF_REPO = "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-GGUF"
GGUF_REV = "df53755cb8741843af8e019e088819fbef17e29b"
GGUF_FILE = "NVIDIA-Nemotron-Labs-Dory1.0-4B-Q5_K_M.gguf" # the card's commands call it dory-q5_k_m.gguf
GGUF_SIZE = 3_673_288_864
GGUF_SHA256 = "96080be49ffd67b916018b7bdef8c484762576fb52780e10ab98ac4978a660e0"
GSM8K_REV = "740312add88f781978c0658806c59bc2815b9866"
WORK = Path(os.environ.get("DORY_WORK") or ("/content/dory" if os.path.isdir("/content") else "dory_work")).resolve()
WORK.mkdir(parents=True, exist_ok=True)
(WORK / "logs").mkdir(exist_ok=True)
subprocess.run([sys.executable, "-m", "pip", "install", "-q", "gguf==0.19.0", "huggingface_hub", "pandas", "pyarrow"], check=True)
SYSTEM = platform.system()
GPU = ""
if shutil.which("nvidia-smi"):
q = subprocess.run(["nvidia-smi", "--query-gpu=name,memory.total,compute_cap", "--format=csv,noheader"],
capture_output=True, text=True)
GPU = q.stdout.strip().splitlines()[0] if q.returncode == 0 and q.stdout.strip() else ""
BACKEND = "metal" if SYSTEM == "Darwin" else ("cuda" if GPU else "cpu")
SRC = WORK / "llama.cpp-dory"
SERVER_BIN = SRC / "build" / "bin" / "llama-server"
free_gb = shutil.disk_usage(WORK).free / 1e9
print(f"{SYSTEM} {platform.machine()}, Python {platform.python_version()}, {os.cpu_count()} CPU cores, "
f"backend {BACKEND}{' (' + GPU + ')' if GPU else ''}, {free_gb:.1f} GB free disk")
if not SERVER_BIN.exists():
t0 = time.time()
if not (SRC / ".git").exists():
subprocess.run(["git", "init", "-q", str(SRC)], check=True)
subprocess.run(["git", "-C", str(SRC), "remote", "add", "origin", FORK], check=True)
subprocess.run(["git", "-C", str(SRC), "fetch", "-q", "--depth", "1", "origin", FORK_COMMIT], check=True)
subprocess.run(["git", "-C", str(SRC), "checkout", "-q", "--detach", "FETCH_HEAD"], check=True)
flags = ["-DCMAKE_BUILD_TYPE=Release", "-DLLAMA_BUILD_SERVER=ON", "-DLLAMA_BUILD_TESTS=OFF",
"-DLLAMA_BUILD_EXAMPLES=OFF", "-DLLAMA_BUILD_APP=OFF", "-DLLAMA_BUILD_UI=OFF",
"-DLLAMA_USE_PREBUILT_UI=ON", "-DLLAMA_OPENSSL=OFF"]
if BACKEND == "metal": # the card's Apple-silicon flags
flags += ["-DCMAKE_OSX_ARCHITECTURES=arm64", "-DGGML_METAL=ON", "-DGGML_METAL_EMBED_LIBRARY=ON",
"-DGGML_BLAS=ON", "-DGGML_BLAS_VENDOR=Apple"]
elif BACKEND == "cuda": # kernels for this GPU only (a T4 is 7.5); flash-attention kernels are unused with --flash-attn off
cap = GPU.split(",")[-1].strip().replace(".", "")
flags += ["-DGGML_CUDA=ON", f"-DCMAKE_CUDA_ARCHITECTURES={cap}", "-DGGML_CUDA_FA=OFF", "-DGGML_NATIVE=OFF"]
env = dict(os.environ, HF_UI_VERSION="b11551")
with open(WORK / "build.log", "w") as log:
subprocess.run(["cmake", "-S", str(SRC), "-B", str(SRC / "build")] + flags, check=True, env=env, stdout=log, stderr=log)
subprocess.run(["cmake", "--build", str(SRC / "build"), "--parallel", str(os.cpu_count()), "--target", "llama-server"],
check=True, env=env, stdout=log, stderr=log)
print(f"built llama-server in {time.time() - t0:.0f} s")
head = subprocess.run(["git", "-C", str(SRC), "rev-parse", "HEAD"], capture_output=True, text=True).stdout.strip()
assert head == FORK_COMMIT, f"checked out {head}, expected {FORK_COMMIT}"
version = subprocess.run([str(SERVER_BIN), "--version"], capture_output=True, text=True)
print(f"fork commit {head[:12]}; {(version.stdout + version.stderr).strip().splitlines()[-1]}")
RESULTS["1. Build the pinned Dory llama.cpp fork and record the environment"] = f"fork {head[:12]}, backend {BACKEND}"
We build llama-server from the exact commit of the Dory branch that NVIDIA’s model card pins, because Dory support in llama.cpp is still an open draft pull request and no released binary can load the model. The cell detects the platform and passes the card’s Metal flags on a Mac, or compiles CUDA kernels for the attached GPU alone on Colab, skipping the flash-attention kernels that the card’s –flash-attn off never uses, which keeps a T4 build manageable. It then checks that the checked-out commit is the pinned one, so every later number traces to one known version of the code.
ToyBlocks: A controlled world for robot learning and evaluation
Thinking Machines' Inkling 2 Small drops 316 F-bombs across benchmark runs
Thinking Machines' AI model Inkling 2 Small swears during its internal reasoning, 316 times across 2,655 benchmark runs, according to an analysis by Vals AI. The profanity typically shows up late in long runs that fail anyway. But it doesn't work as a warning signal: 96 percent of failed runs contained no cursing at all.
The article Thinking Machines' Inkling 2 Small drops 316 F-bombs across benchmark runs appeared first on The Decoder.
104 Blog Posts To Learn About Problem Solving
Let's learn about Problem Solving via these 104 free blog posts. They are ordered by HackerNoon reader engagement data. Visit the Learn Repo or LearnRepo.com to find the most read blog posts about any technology.
Problem-solving is the process of identifying a problem, generating potential solutions, evaluating them, and implementing the most effective one. It is a fundamental skill in all aspects of technology and life, driving innovation and efficiency across various domains.
1. Kadane’s Algorithm Explained with Examples
Given an array, the algorithm to find the maximum subarray sum is called Kadane’s Algorithm.
2. How I Solve Problems as a Developer
There is no programmer in the world who knows how to solve every coding problem. That person simply doesn’t exist, because the programming field is so big that to a normal person, it would take two lives to learn all of that, but we only have one life.
3. Manacher’s Algorithm Explained— Longest Palindromic Substring
Manacher’s Algorithm helps us find the longest palindromic substring in the given string. It optimizes over the brute force solution by using some insights into how palindromes work. How? Let’s see!
4. System Thinking 101: A Primer for Generalists
System thinking is a valuable framework for generalists, you need a proper framework for organizing your multiple thoughts, passions, and pursuits.
5. Use This 7-Step McKinsey Framework to Solve Any Problem
Julia MacDonald, a Fractional Growth Executive, breaks down a 7-step framework to solve any problem.
6. Problem-Based Learning: David Merrill's Principles of Instruction
Merrill's First Principles of Instruction are a series of very effective problem-based teaching methodologies. The principles are based on the five core princip
7. When Everyone Corrected the World's Smartest Woman
I find the Monty Hall Problem one of the most fascinating brain teasers, because it seems deceivingly simple. When Marilyn Von Savant, the woman with the highest IQ, answered this brainteaser correctly in 1991, she was inundated with over 10,000 letters from academics and Ph.Ds criticizing her “error”, only to be proven later that she was correct.
8. How to Solve Any Machine Learning Problem [Almost]
TL, DR; When coming across an ML problem, don’t try to be a hero and dive right into solving it. Process and understand the problem, review your dataset, set a realistic goal and then go about actually solving the problem. Chances are that you will end up saving a lot of resources (most importantly time) if you plan your execution properly.
9. Being a Generalist: How to Leverage Your Diverse Skills and Find Your Purpose
What does it mean to be a generalist? If you are one, how can you find purpose and thrive in your life? We'll answer these questions and more in this articl
10. Solving the Edit Distance Problem Using The Dynamic Programming Approach
The first question that arises when solving a problem using dynamic programming(DP) is how to figure out that DP is a way to solve it?
11. How Do You Alternate Positive and Negative in Python?
Have you ever written a program for pairing alternate positive and negative numbers? If not, then you are missing some great opportunities.
12. Is ‘bias for action’ making product managers lazier?
Let me preface this by saying I’m a big believer in the bias for action principle popularised by Amazon. I’ve used it as a guideline whenever I didn’t have enough data to make an informed decision.
13. The Five Step Approach for Tackling Complex Problems
“The definition of genius is taking the complex and making it simple.” - Albert Einstein
14. Programmer’s Pyramid: Take Your Programming To The Next Level
Build a solid foundation of programming knowledge, develop core s
What Fact-Checking My Own Claude Code Guide Taught Me About Verifiable AI
Most talk about verifiable AI is about the model: proofs of inference, attested hardware, reproducible weights. That matters. But most of the AI output people act on every day isn't a model call they could audit. It's a claim: in a doc, a README, an answer, a config someone copied.
I write about Claude Code. In February, I published a long guide to building with it, plus a companion GitHub repo of the configs I actually use. The repo is the part a reader can copy straight into their own setup, which turns out to matter. As Veera Ravindra Divi put it here on HackerNoon, Claude Code doesn't ship production code: your config does. A wrong config does more damage than a wrong sentence.
Since then I've re-checked every claim in it twice, once in August against Claude Code v2.1.234 and again this month against v2.1.311, the second time with four Claude Code agents working in parallel. I expected stale facts. What I didn't expect was ten claims that were wrong on the day I wrote them. One of them was a correction from the August pass.
This month, the agents did most of the checking. Some of their findings looked like errors until I traced the source, and some of my old claims looked fine until I did the same. Either way, what mattered was whether a verdict could be checked by someone other than whoever produced it.
What "Verifiable" Has to Mean for AI Output
A verified claim needs four things. I learned each one by watching a check fail without it.
|
Property |
Question it answers |
Failure without it |
|---|---|---|
|
Source |
Where does this come from? |
Plausible numbers nobody measured |
|
Version |
True as of when? |
"Corrections" that were never true |
|
Reproducibility |
Can someone else re-run the check? |
Works on my machine, fails on yours |
|
Independence |
Is the evidence separate from the claim? |
A checker citing the article it's checking |
None of these is about the model. You can run the best model available and still get output you can't verify. You can also run a cheaper one and get output anyone can audit, if the process makes it leave a trail.
The Ten Claims That Failed
From the August pass:
- "Delegate Mode (
Shift+Tab)." It never existed.Shift+Tabcycles permission modes. I'd described a feature that sounded right in three different files. @-imports defer loading. They don't. Imported files are expanded into context at launch. Imports organise your instructions; they don't save context.- An eight-layer memory hierarchy. It had ten. I'd missed the managed-policy CLAUDE.md entirely, a layer that existed the whole time and can't be excluded.
- "Keep CLAUDE.md under 500 lines." That was my own guess. Anthropic's published guidance is under 200.
From the October pass:
- "Before v2.1.179 the default display mode was
auto." This one hurts most, because I added it in August as a correction. No changelog entry supports it. - "Forks cannot spawn further forks." Unverified. What is documented is that forks count toward the three-level nesting cap.
- "Auto-compaction triggers at about 95% of context." There's no fixed percentage. By default it compacts at the model's context limit, about 967K tokens on the 1M-context models, and
/autocompactsets an earlier trigger. - "A skill costs about one line of context at startup." The real cost is the skill's description plus its "when to use" text, capped at 1,536 characters per skill. The whole listing is budgeted at 1% of the context window.
- "Unspecified teammates inherit the lead's model." The lead's model is the last fallback, after the spawn prompt, the agent definition's
modelandCLAUDE_CODE_SUBAGENT_MODEL. - "Fable investigates before acting and verifies its own work." No official source says that. Anthropic describes Fable 5.1 as being "for demanding reasoning and long-horizon agentic work". I'd absorbed a paraphrase and repeated it as a capability.
Every one of these is fluent, specific, and plausible. That is the kind of claim language models produce easily, and agents repeat confidently. It is also the kind a reader can't check without a source.
Source: Unsourced Precision Is Unverifiable
Look at numbers 5, 7 and 8. A version number, a percentage, and a line count. Each one survived review because it was specific.
"Compaction triggers at about 95%" sounds like someone measured it. "Before v2.1.179" sounds like someone read the changelog. Nobody had. The precision did the persuading, and the precision was invented.
My rule now: every number gets a source next to it, or it gets rewritten as a range or removed. When the first correction pass suggested "fixed around v2.1.63" for a GitHub issue, there was no changelog entry behind it. That number came out too. The guide now says the behaviour is no longer reproducible and tells readers to check it themselves with /context.
Version: Sources Describe Specific Systems
Checking against official documentation sounds definitive until two official sources disagree.
In August, Claude Code's subagents page still listed a 200-subagent-per-session cap. The changelog said it was removed in v2.1.224. The docs hadn't caught up.
In October, Anthropic's API models page listed Sonnet 5.5's default effort as high. Claude Code's model configuration page says the 5.5 models default to medium. Both are right: one describes the API, the other describes Claude Code. An agent that reads only one of them will "correct" a true claim into a false one.
So provenance has to include which system and which version a source describes. For anything version-specific, I treat the changelog as the primary source. When sources disagree, the verdict says which surface each one describes rather than quietly picking one.
Reproducibility: Run It, Don't Read It
The worst error in the February guide isn't on the list above. Nothing I wrote was false; I left something out.
Agent Teams is experimental and disabled by default. It needs CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1. I never mentioned it, because the flag had sat in my own settings.json so long I'd stopped seeing it. Every Agent Teams pattern in the guide worked on my machine and did nothing on yours.
Reading docs can't catch that. The docs are right, and the claim is right. What's missing is the environment. The only check that works is re-running the instructions somewhere that isn't your machine.
T