I'm currently evaluating grant applications for the Corrigibility Research Fund. For the most part, my methodology is the same as it was in September. But the applications I get keep falling into the same familiar patterns, so I thought I'd write a little about those patterns, and why I don't want to fund them.

The most superficial pattern of note is SLOP. So much slop. You don't even know.

Slop is more than just whether an application is AI generated. We're now in an era where people across the world, including those who don't really speak English, can contribute to work that would otherwise be restricted to people of a single language or nationality or place, and I like that. When I see an application that discloses its use of AI and gives a reasonable justification for it, I try to look past the stylistic tics and focus purely on whether the underlying application is promising.^[1]

Still, we're not yet in the domain where AIs can write good grant applications without some human assistance, so a high Pangram score is definitely evidence that an application is junk. Another remarkably common sign of slop was messed up formatting or other indications that people just copy-pasted from a text file without even proofreading. If you can't be bothered to read your own application, I'm not sure why I should be. By contrast, the best applications are clear, short, and dense, cutting straight to the way in which the applicant wants to be valuable to the research effort. Don't define corrigibility for me in the intro unless you have reason to think that I don't understand my own area of specialization.^[2]

Moving past these shallow concerns, most applications cluster together into archetypes that I apparently didn't warn against well enough in my earlier writing. An individual application will sometimes fall into multiple of these, but any one of them is usually grounds for rejection. So, without further ado, here are five kinds of grants that I don't want to fund...

1. “A governance layer for agents (that I'm running)”

Far and away, the most common archetype for "corrigibility work" is a proposal to build some kind of control mechanism for agents. Even though I warned that I'm not interested in this kind of work in my methodology post (and when I announced the fund), and even with the infamy around this being one of the oldest tropes in AI safety, people love to try and keep AI in a box.

These kinds of applications are almost always LLM written. AI power-users get burned when they find that their personal agent swarm sometimes does things that they don't like, and so they task their agents with solving the problem. The agents suggest oversight, but the human doesn't want to slow things down or spend the energy actually checking everything. The result is "a governance layer that deterministically checks cryptographically signed provenance tokens" or something similar.^[3] Sometimes the human is honestly convinced that their control schema is worthy of corrigibility funding, and other times it's a transparent attempt to get me to fund their swarm (and thus "get empirical data about real-world agent security").

I don't want to fund this because...

• control is already well-funded (especially prosaic control, which is actively researched by frontier labs)
• it's not actually corrigibility/alignment research
• it won't work for superintelligences.

2. “The (maybe corrigibility adjacent??) project I'm already working on”

This is one of the more common patterns for academics, but it also applies to others. Some research team is inspired to do X or is already doing X, and creates a grant application for their project. They look around for funding sources, see the Corrigibility Research Fund, and think "Maybe I can make my thing look vaguely like corrigibility!"

Your research project might be cool. Maybe you're setting up automated translation pipelines in the developing countries. Maybe you're working on climate change politics or robust power grids or how to compensate journalists in the age of AI. But, sorry, I don't want to fund things unless they actually make headway on the open problems of corrigibility.

A variant of this is the unabashedly commercial entrepreneur who has decided to pursue funding from as many sources as possible (or, more typically, has told their AI agent to do so). Bonus points go to the two independent(!) applicants this round whose work was centered on Chinese divination.

3. “I'll test whether AIs ever misbehave”

Look, science is hard. What if we just, y'know, ran some AIs through some scenarios and showed that they sometimes act in ways that aren't corrigible? Yeah. That sounds doable.

And indeed... it has already been done. Many times. And the problem persists.

The best applications in this category put some new spin o