The oversight problem consists of describing, eliciting, and understanding behavior. Prior work in black-box behavior investigation and interpretability seek to construct compact, faithful, and informative theories that explain model behavior. We propose training investigator agents and subject models to emit explicit theories, and using these theories as common currency for oversight across models. We hope to create a complementary mode of oversight to weight-space and model-specific approaches by studying the transfer properties of theories across model families.  This post outlines our positioning and initial research agenda built on top of a few recent publications.

Oversight as Theory-Building

Transluce is working on oversight-as-inference, training oversight foundation models on experiments on a particular subject model: “Any oversight question can be formalized as Bayesian inference over the outputs of a Pythonic world model.” The models are posttrained to explain behaviors in natural language. Their method is promising for overseeing a system in depth: expressing interventions as a primitive set such as get_activations, patch, fine_tune, decode, and sample_model is a powerful toolbox, and we are excited to see oversight assistants that can use them fluently.  However, it needs to be redone for every subject model, which seems unfeasible across rapid releases in many different families, unless the adaptation can be done in a very compute-efficient manner. 

We propose to approach oversight by training both the investigator agents and subject models to output good theories, or working beliefs about the environment they are operating in. The question is: can a theory learned from one set of models help predict or investigate a new model, in a way that is more efficient than subject-specific training?  Unlike raw CoT, theories operate in belief space. Correct theories should aid the models in solving tasks instrumentally as well as being more amenable to oversight; belief space also isolates the components relevant to behavior (what quirk of the environment caused the behavior matters less than what it did to the agent’s beliefs in relation to the environment). 

Oversight efficacy is really research judgment on beliefs, and like scientific models, this judgment is best elicited by doing (aka post-training). Obviously, we cannot post-train every subject model, but we can use their theories to bootstrap the end-product investigator. We can filter the subject model’s theories based on whether they successfully predict subject behavior on held-out states,  and use them to warm-start the investigator, which is then trained with verifiable rewards  for predicting and steering behavior in response to environmental interventions.

Similar to how a Rationalist makes beliefs pay rent, we hypothesize that models strong at generating good theories are better general learners. Current models fall short: they mode collapse, lack creativity, cannot set research agendas, and are bad at designing experiments to discover rules in unseen environments, as shown in our recent work WitnessBench. By improving models’ theory-building abilities, we provide both a cognitive scaffold for solving general scientific tasks as well as a medium for oversight. 

We think theories have several benefits:

1. They circumvent the model-specific limitation of oversight assistants by moving the common representation for behavior from weight space or Python code into token representations that condition generation at inference time, similar to how reasoning models moved intermediate computation into context. This reduces the difficulty of creating a behavior representation transferrable across different models. 
2. Explicit theories invite iteration via ontology-building. We want to build a shared public repository of explanations and forms of model failures, and theories are the underlying currency.
3. On the training data front, we can use theories to do data prediction in line with alignment forecasting, or attribute and propose interventions like in our recent investigation of the Cohere Qwen3.8-Flash-Next incident.

The closest existing method we are aware of is Counterfactual Simulation Training for Chain-of-Thought Faithfulness. Our theory differs in that it captures the environment instead of specific inputs, and their weak transfer results between types of counterfactuals may point to the need for explicit theories beyond CoT. 

Research Agenda Sketch

The first step is to create a formal representation for theories. We prototype this in a cellular automata environment by asking models to infer the update rules. In this case, the model represents its theories about update rules as Lean formalizations of them. The actions are fully verifiable, and the model sees only part of the board during discovery, so we have full observability into how the model struggles to arrive at the theory. Correct inference requires a Lean proof of equivalence between the model’s update rule and the actual update rule; we start here and work up to fuzzier representations such as natural language.

To train for theory-building capabilities, we also need to make environments a first-class intervention space. We take