Combinatorial Optimization with Pointer Networks: Theory, Benchmarks, and Limitations [P]
Hello everyone,
This is the first article I’ve decided to share publicly. I'm really interested in solving combinatorial optimization problems using machine learning, and this is the opening piece in what will hopefully become a broader series exploring this domain.
This part focuses on the classical Pointer Network architecture trained via Reinforcement Learning (REINFORCE). It shows how neural models can construct high-quality, near-optimal solutions to NP-hard problems (specifically, the 2D Euclidean Travelling Salesman Problem) in fractions of a millisecond.
I benchmarked the model against Google OR-Tools (Guided Local Search with a 10s timeout) across problem scales from N=20 up to N=200.
TL;DR: While the network finds solutions almost instantly, tour quality decays significantly as the problem size scales up. Why? Long story short: using an LSTM encoder on coordinate sets introduces an artificial sequential order and recency bias, which fundamentally mismatches the permutation-invariant nature of geometric graphs.
Here is the link to the full article on Medium
[link] [comments]
ICLR27: Received a review that seems to be for an entirely different paper. Has anyone experienced this? [D]
Got my reviews yesterday, and honestly, I'm starting to question the credibility of the entire review process.
One of the reviewers seems to have submitted a review for a completely different paper. And I don't mean they misunderstood our method or missed the main contribution. Their entire paper summary describes a completely different research problem, method, and set of experiments. It has practically nothing to do with our submission, yet they went ahead and gave it a rejection.
Of the other two reviewers, one actually provided a good, thoughtful review, correctly identifying the strengths and weaknesses of our work. The other, however, didn't seem to have read the paper carefully. They criticized our negative findings as a weakness, despite the fact that investigating and demonstrating the limitations of existing approaches is explicitly part of the paper's objective.
I'm perfectly fine with criticism, rejection, or reviewers disagreeing with our conclusions. That's part of research. But how is an author supposed to respond when one reviewer is apparently reviewing an entirely different manuscript?
Has anyone experienced something like this before? Should I contact the AC directly, and is there any realistic chance of getting such a review disregarded or corrected?
I'm genuinely at a loss as to how something like this happens at a conference of this scale.
[link] [comments]
‘Fast fires’ were just 2.7% of U.S. wildfires but destroyed 78% of the structures lost — and up to 449,000 could be lost by 2060
As temperatures rise and development expands, the number of homes and other structures damaged or destroyed by wildfires across the U.S. is on pace to triple by mid-century, my colleagues and I found in a study just published.
The 2026 wildfire season has been a sharp reminder of that risk and why making homes and properties less vulnerable to wildfires is increasingly crucial. Fortunately, strategies to lower the risk of losing a home to wildfire don’t have to be overly expensive.
The 2026 western fire season
The 2026 wildfire season began with historically low mountain snowpack, early snowmelt and exceptional warmth across much of the West. Even before the nation’s hottest summer on record intensified drought and dried vegetation, fires were breaking out.
Colorado’s Aspen Acres Fire destroyed hundreds of structures in June. Four firefighters died from injuries suffered when the fast-moving Knowles Fire overtook them near the Colorado–Utah border in July. In August, wind-driven fires swept into neighborhoods in Spokane, Washington, damaging more than 800 homes and forcing tens of thousands of residents to evacuate.
In Colorado, the summer monsoon season brought rainfall, helping firefighters gain ground. Across the Rocky Mountain region, lightning continued to start fires, but rain helped keep many fires small in many areas despite above-average temperatures.
Despite this reprieve, Colorado fires had burned more than 318,000 acres by late October, nearly twice the annual average since 2000. Nationally, the total area burned was also well above average. The U.S. spent 48 consecutive days at Preparedness Level 5 – the highest level, indicating that firefighting resources were stretched nationally – compared with a 10-year average of 25 days annually.
Reasons for rising fire risk
Fire is a natural and often beneficial part of many ecosystems, shaping vegetation and supporting biodiversity. However, a century of fire suppression has allowed dry vegetation to accumulate in forests that historically had frequently burned.
Rising temperatures caused by human activities such as burning fossil fuels have increased the rate at which the atmosphere pulls moisture from the land, drying soils and vegetation. Researchers estimate that this drying nearly doubled the area burned in western U.S. forests between 1984 and 2015 compared with what would have burned without human-caused climate change.
More people also now live in and at the edges of wildland areas, raising the risk of fires reaching neighborhoods. Human activity has extended the fire season beyond summer, when most lightning-caused fires occur, too. Power lines, vehicles, machinery, burning of debris and campfires can spark fires year-round.
These interacting factors have helped nearly quadruple the annual area burned in the western United States since 2000.
They also contribute to the growing threat of fast-growing fires, which can overwhelm emergency response.
In a 2024 study, my colleagues and I used satellite imagery to measure daily fire growth across the contiguous United States from 2001 to 2020. “Fast fires,” defined as those that burned more than 4,000 acres in a single day, represented just 2.7% of fires analyzed but accounted for 78% of documented structures destroyed and 61% of reported firefighting costs.
A brief burst of growth, often fueled by strong winds or highly flammable materials, can matter enormously, even if a fire’s eventual size is relatively small. And in many ecosystems, fires are getting faster.
More homes in harm’s way
The number of people and buildings in harm’s way has also been increasing in the U.S., raising the risk of damage.
Between 1945 and 2015, development in regions of the country with the highest wildfire risk expanded tenfold, increasing the amount of human exposure to wildfires while also introducing ignition sources and buildings that can burn. Once embers enter a neighborhood, fire can spread quickly from building to building.
My team’s new research suggests that without further action to reduce wildfire risk, the annual number of structures damaged or destroyed nationwide could nearly triple by mid-century. That would mean up to 449,000 structures lost between 2026 and 2060.
We used historical data about development patterns, wildfire area and the fires’ effects to reconstruct the past fire risk, then modeled future fire risk by incorporating weather, climate and housing-density expectations.
We found that roughly one-third of western neighborhoods facing the greatest future risk are low-income communities. In these communities, people may struggle to afford measures to protect their homes, maintain insurance coverage and recover financially after a fire.
How to lower your wildfire risk
These projections are sobering, but actual losses will depend partly on the choices people make now.
For example, homes can be made less likely to ignite with Class A fire-resistant roofs and ember-resistant vents that reduce the potential
Big Tech’s AI race will save small business owners time. It won’t find them customers
Meta, Google, OpenAI and Anthropic are racing to become the small business owner’s AI assistant. Meta brought its Muse agent to small businesses in September. Anthropic and OpenAI now package tools aimed squarely at owners, and Google is building the same features into tools millions of them already use.
That competition is good for Main Street. But it’s a race to save owners time, and time isn’t what keeps them up at night. What keeps them up is where the next customer will come from.
A word on where I sit. I run Alignable, a network of 10 million small business owners, so I have a stake in this argument. Alignable also co-hosts small business AI training with Anthropic, one of the companies named here. Read what follows with that in mind. But I also have something the assistants don’t: 13 years of data on how small businesses actually find work.
Fifty beats 500
Start with where customers come from. In our surveys, members say about 85% of their customers come through referrals. Not ads, not search, not cold email. People who already trust them.
Across 35,000 local communities, our members have formed 226 million business connections. Most of them never become a relationship that sends work either way. The owners who grow through their networks aren’t the ones with the most connections. I’ve told owners for years that 50 warm relationships beat 500 cold contacts. That’s a rule of thumb from experience, not a number we’ve measured. But the owners who win do keep a small set of relationships active and leave the rest alone.
And most owners are sitting on more opportunity than they know. When members tell us what their ideal customer looks like, we can count how many of those businesses sit one introduction away: connected to someone the owner already knows. Among members whose ideal customers show up in their extended network, the typical one has about 190 of them a single introduction away. For one in four, it’s more than 600.
The next customer is rarely a stranger. They’re one relationship away, through someone the owner hasn’t talked to in months.
Those relationships aren’t random, either. In my experience, three kinds do most of the work: customers who talk you up, businesses that serve the same customer without competing with you, and a handful of people who open doors. The best referral partner usually isn’t a peer in your own industry. It’s the business your customer calls right before or right after they call you: the realtor and the home inspector, the accountant and the financial planner.
What a general assistant can’t see
Ask any of the big assistants how to get more referrals and you’ll get a solid list of best practices. Ask an assistant that sits on a map of business relationships and it can tell you something specific: three nearby businesses serve your ideal customer, one of them already knows two of your clients, and you haven’t spoken to your best referral partner in four months.
That’s the difference between advice and action. A general assistant can’t answer that question, because the answer lives in data it doesn’t have.
The early numbers bear this out. In Bluehost’s 2026 study of 350 U.S. owners, 78% said AI saves them time every week. But 43% have seen no change in revenue. Time saved only matters if it goes somewhere that pays.
Too often it goes into volume. Forbes recently told the story of Paul Birrell, a New Jersey owner whose automation loop sent roughly 400,000 emails in three days. His reaction: “I don’t have 400,000 customers.” Everything we see says growth comes from the opposite: judgment about the few people who matter.
Three kinds of AI, not one
AI will reach small business owners in three forms. The first is the personal assistant the giants are fighting over. It delivers time, and the assistants are converging fast.
The second is AI built into the systems that already hold the business’s own data. Intuit knows the books. Shopify knows the store. That AI delivers control: it can flag a cash crunch six weeks out or spot the product quietly losing money.
The third is AI built on the relationships between businesses: who knows whom, who serves the same customers, who sends work to whom. That’s the one that delivers growth, and it’s the one almost nobody is talking about. At Alignable, ours is called Allie.
The giants have real advantages, and they’ll connect to the other two kinds. Meta’s Muse already plugs into QuickBooks, Shopify and Stripe, and the others are adding similar connections. The open question is who the intelligence works for. Meta makes nearly all of its money selling ads, and an assistant built by an ad company will tend to find that the answer involves more advertising. The data that actually works for the owner describes how their business runs and who in their community can help it grow.
What owners should do now
Use a personal assistant every day. The time savings are real.
Then decide, in advance, where the reclaimed hours go. Put them on the calendar against the work that generates revenue, and the most neglected item on that list is relationships. Write down the 50 people most likely to send you business. Most owners can name 10 without trying. Those 10 are where AI should be pointing you.
There are about 36 million small businesses in the U.S., and more than 80% of them have no employees. For those owners, AI isn’t a department upgrade. It’s a capability they never had. Owners should take the better tools the giants’ race produces. But the biggest gains won’t come from saving an hour. They’ll come from knowing which 50 relationships to spend it on.
The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.
This story was originally featured on Fortune.com
Starter homes cost $1 million in a record 242 U.S. cities—proof the affordability crisis has hit the homes meant for first-time buyers
They waited out the pandemic boom. They saved longer, rented longer, delayed longer, and watched the typical first-time homebuyer age climb to a record 40. Now, the buyers who did everything right are running into a new problem: The Northeast just became the fastest-growing region for million-dollar starter homes in the country.
It’s especially imperfectly timed for millennials entering their peak household spending years and beginning to form their own families (or at least try to).
A Zillow report published in June counts a record 242 U.S. cities where starter homes cost $1 million or more—triple the 80 cities that cleared that mark before the pandemic, and up from 226 a year ago.
The share of first-time buyers has fallen to half the historical norm. For millennial parents in the Northeast—now in their late thirties and early forties, often with kids, hunting for more space—the numbers have a specific shape.
New Jersey had just one million-dollar starter city before the pandemic. Now it has 26. New York used to have 12; now it has 41. The two states added 15 cities to the list in the past year alone—faster growth than anywhere else in the country. The New York City metro area now leads all with 63 cities where a starter home runs $1 million or more.
That growth didn’t happen in a vacuum. Six of the 10 most competitive housing markets in the country are in the Northeast, per Zillow’s 2026 analysis, a region where new construction has chronically lagged and inventory deficits run deepest. California still leads the raw count with 105 cities, but the Northeast is where the crisis is actively spreading.
“A housing shortage a decade in the making ran headlong into intense demand with mortgage rates at historic lows, driving up home values at a record pace,” Zillow senior economist Kara Ng wrote. For Northeast buyers, those forces have compounded.
The new typical first-time homebuyer is 40
First-time homebuyers are historically younger, although the typical age of first-time buyers climbed to an all-time high of 40, according to a National Association of Realtors report released in November 2025. In fact, the housing market has gotten so unaffordable that the number of first-time homebuyers fell to half the historical norm last July.
A rising price floor on starter homes only pushes that first home purchase further out of reach. And it’s a problem that’s worsened, especially during the past couple of years.
“For homebuyers, the prevalence of million-dollar cities significantly—greatly—strains what is already the worst housing affordability since at least the 1980s,” Christopher M. Naghibi, executive vice president and chief operating officer at First Foundation Bank, told Fortune in 2024 when Zillow released a report about the number of U.S. cities with $1 million median-priced homes. “Prospective buyers in these markets face higher barriers to entry, needing ridiculously large down payments and higher incomes to afford homes.” That still holds true today.
Zillow defines a starter home as one in the lowest third of home values in a given region. As of June, the typical starter home nationwide was worth $198,649, up 1.7% from a year earlier. But in well over 200 U.S. cities, starter home prices are a whopping five times as high as that. It’s a phenomenon that keeps spreading, and into places it never used to touch.
Zillow chalks it up to the effects of the pandemic housing boom.
It’s not just a California problem anymore
Before 2020, the list of cities with $1 million starter homes was almost all coastal. Colorado was the only non-coastal state with a million-dollar starter-home city. Now 26 states have at least one, up from nine before the pandemic. Texas, Wyoming, and Illinois each have multiple examples.
California still leads with 105 cities, but the fastest growth is in the Northeast. New York and New Jersey added 15 cities combined over the past year. New York is now up to 41, from just 12 before the pandemic. New Jersey is at 26, up from one.
The New York City metro area tops all metros with 63 cities where a starter home runs $1 million or more. San Francisco follows with 37, then Los Angeles with 33, San Jose with 13, and Miami and Seattle with eight each. The trend aligns with Zillow’s 2026 hottest markets analysis, which found six of the 10 most competitive housing markets are in the Northeast, where new construction has lagged and inventory deficits run deep.
The pandemic reset the cost of buying a home. A housing shortage a decade in the making collided with surging demand and historically low mortgage rates, pushing values up at a record pace. Those effects have stuck.
Baby boomers edge younger buyers out of the housing market
The squeeze on millennials and Gen Z has been building for years. Baby boomers, sitting on paid-off homes or cheap pandemic-era mortgages, have edged younger buyers out of the market in record numbers. The cost has pushed some Americans to delay milestones like marriage and kids, and others to co-buy homes with friends or partners just to split the bill.
Others have done the opposite—had kids first, rented longer, and are now hunting for space in a market that moved against them.
“We’re seeing a reshaping of the housing ladder,” Alexandra Gupta, a real-estate broker with the Corcoran Group, previously told Fortune. “Some first-time buyers are turning to long-term renting or even co-living models because the idea of owning a home has become so out of reach.”
There are modest signs of relief. Conditions are slowly turning friendlier for buyers. Zillow found the typical buyer now breaks even versus renting after about six years, down from more than eight in late 2023. More inventory and slower price growth mean buyers who are financially prepared are in better shape than they had
In-N-Out’s billionaire owner vows iconic chain ‘will always be’ in California and says leaving the state would be the ‘dumbest business decision ever’
Lynsi Snyder, president and owner of In-N-Out, moved her family to Tennessee recently but reaffirmed that the iconic burger chain will never abandon California.
In an interview with actor Terry Crews that In-N-Out posted online, she was asked how the company will retain its California DNA as it expands to more states around the country.
“This is where we started. It’s where we are now, and it’s where we will always be,” Snyder replied.
Her grandparents opened the first In-N-Out in Southern California in 1948, and the company has gradually expanded to other parts of the state—and to other parts of the country.
While far from being the biggest chain, it retains a cult following that has helped its expansion as people who moved out of California make pilgrimages when restaurants open in new states.
But Snyder, who grew up in Northern California, revealed last year that she and her family were moving to Tennessee, saying that raising kids was difficult in the state and that “doing business is not easy” there either.
That sparked fears In-N-Out might leave California, prompting the burger heiress—who is worth an estimated $7.3 billion—to clarify that her departure from the state doesn’t mean the company will too.
In a video posted to her Instagram page last year, Snyder told followers: “We’re not leaving California, or leaving our roots behind. Each one of our locations is here to stay.”
Still, there has been a corporate exodus from California to states like Texas, where Elon Musk has relocated the headquarters for Tesla and SpaceX.
In-N-Out has also clashed with state authorities, such as when it refused to comply with officials and request that customers must show proof of vaccination to make purchases during the pandemic.
And the company reluctantly raised prices after California imposed higher minimum wages for fast-food workers in 2024.
But during her latest interview, Snyder addressed the fears about In-N-Out, which maintains headquarters in California and Tennessee.
“There were rumors about ‘they’re gonna close all their California stores.’ I’m like what where did this even come from? That’s unimaginable. I can’t even imagine that — dumbest business decision ever,” she said.
Snyder added that In-N-Out continues to grow in California, with “lots of stores” slotted for the other cities in the state.
At the same time, the company has expanded to Nevada, Arizona, Utah, Texas, Oregon, Colorado, Idaho, Washington and Tennessee. Snyder suggested that Mississippi, Arkansas, and Oklahoma are under consideration, with New Mexico also in the works.
But she insisted the company is taking a slow and steady approach to growth that won’t compromise on the quality of its food her grandparents’ values, adding that In-N-Out won’t be able to accommodate every request to open a new location.
As for her own family’s relocation to Tennessee, Snyder described it as “kind of a God thing” and said she originally wanted to move to Colorado because she loves the mountains.
But her husband floated the idea of going to Tennessee instead. While they already had made plans, she recalled that “something just kind of clicked.”
“I felt it and then started looking at it, just praying, and I’m like ‘I think you’re right. I think that’s where we’re supposed to go,'” Snyder said.
This story was originally featured on Fortune.com
Alabama Power spent more than $6 million on ads last year — and customers can be billed for half of them until 2029
This article originally appeared on Inside Climate News, a nonprofit, non-partisan news organization that covers climate, energy and the environment. Sign up for their newsletter here.
On a busy election day, the Alabama Public Service Commission requested that Alabama Power voluntarily stop billing customers, effective immediately, for television commercials and other ads it buys to promote various aspects of its business.
Alabama Power is the largest electricity provider in Alabama, with territory covering about two-thirds of the state. Though it is a monopoly, the company spends millions each year on advertising, touting its reliability, customer service and other aspects of its business.
In recent months, the company has launched a series of television commercials touting a rate freeze it announced late last year amid heavy criticism over rising electricity prices.
A state law passed earlier this year will prohibit Alabama Power from including the costs of those commercials in customers’ electric bills, meaning the powerful utility will have to find all of the dollars in corporate profits. But the new law won’t take effect until 2029.
Currently, Alabama Power is allowed to bill customers for half of its non-public safety advertising costs.
During Tuesday’s meeting, PSC President Cynthia Lee Almond moved to “request” that Alabama Power voluntarily begin excluding those costs immediately.
“I don’t see any reason why we shouldn’t go ahead and take steps now to begin that process, rather than waiting until Jan. 1, 2029,” Almond said during the meeting.
Almond then made her motion, which was not on the meeting agenda, but came after the regular report from the commission’s electricity division.
“I move that the commission request that Alabama Power consent to exclude all advertising expenses from retail rate setting until after Dec. 31, 2028, when the new law takes effect,” Almond said. “Assuming Alabama Power consents, this interim measure would be reflected in its [Rate Stabilization and Equalization] filings for 2027 and 2028.”
Commissioners Jeremy Oden and Chris Beeker IV voted yes after a brief discussion.
According to the company filings with the Federal Energy Regulatory Commission, Alabama Power spent more than $6 million in general advertising last year, roughly $4 per customer. It’s unclear from the federal filing how much of that total was passed on to customers or how much might be impacted by the new prohibition.
Alabama Power did not immediately respond to a request for comment.
Daniel Tait, executive director of Energy Alabama, said the move was a boon to Alabama Power customers, who faced some of the highest electric bills in the country in recent years.
“We are grateful for the leadership of President Almond and the full commission for acting to end customers paying for Alabama Power’s advertising,” Tait said. “Once in place, Alabama will have one of the strongest customer protections in the nation on this front.”
This will be among the last actions of the commission under its current construction before a major overhaul takes effect.
In January, the commission will expand from three members to seven. Outgoing Gov. Kay Ivey has appointed four new commissioners, who will take office in January, including the PSC’s two first-ever Black members.
Oden and Beeker will also leave the commission in January after being defeated in Republican primaries. Cullman County Sheriff Matt Gentry will take Oden’s Place 1 seat, and former state auditor and PSC Commissioner Jim Zeigler will replace Beeker in Place 2.
Zeigler, a perennial candidate in Alabama across several decades, will return to the commission 48 years after leaving the PSC. He served one term from 1975-1979, during an era of skyrocketing energy costs and protracted political battles over electric rates between Alabama Power and then-Gov. George Wallace.
Almond will be the sole remaining member of the current commission.
The new seven-member commission will be overseen by the state’s first secretary of energy, who will be selected by Gov.-elect Tommy Tuberville. The new secretary will set the commission agenda, and overruling the secretary will require a vote of five of seven commissioners.
Tuberville has not yet announced his pick for secretary of energy. The only statutory requirement is that the secretary “must have experience in utility regulation or energy policy or a related field.”
This story was originally featured on Fortune.com
OpenAI will keep investing in AI data centers despite election results, CFO tells CNBC
Let's Try to Catch Misalignment Before a Model Learns to Hide It
An AI model that appears aligned could be either (1) actually aligned or (2) hiding its misalignment while being evaluated. How can we tell which?
One idea: Look at the trajectory of eval results across the entire training process.
My hypothesis is that misaligned behavior would emerge before the ability to hide misalignment, like how young children first misbehave in obvious ways before learning to be sneaky. As training progresses, we'd first see misalignment eval scores rise, then see them drop as the model learns to hide misalignment. Even if the model becomes very sophisticated - say, modifying its chain of thought or internal activations to fool white-box evals - we should see the misalignment earlier in training. This seems likely, though it is debatable and worth testing.
This analysis of the training trajectory is one tool among many. Of course we want to see black-box and white-box alignment evals on the final model, using whatever tools we can to understand the model's true motivations. But a model might still develop sophisticated hiding strategies that defeat even white-box evals. Inspecting the training trajectory is a backstop against this.
This point was made previously by Apollo Research in this post. (Their chart even matches mine almost exactly, though I promise I first made mine independently.)
In this post, I'll describe a simple project exercising this strategy. With the model and eval that I chose, we don't see the up-then-down trend in measured misalignment that would indicate hiding. This is unsurprising since the model is well below frontier scale and the eval likely doesn't capture misalignment that well. The sample sizes are small and statistical uncertainties are high. But this approach is promising with improvements such as different evals, larger models, and deliberately misaligned test models.
This is my project for the BlueDot Technical AI Safety Project. My main purpose has been to get started with some hands-on AI inference, evaluation, and/or training. I'm not aiming for academic-quality rigor, but the project does explore an interesting idea and has some promising avenues for extension.
Model
The Olmo 3.1 model family from the Allen Institute for AI provides open weights not just for the final model, but for many checkpoints along the training trajectory. It is one of the few model families that publishes checkpoints. Both 7B and 32B parameter variants are available; I used the larger one, 32B Think.
For checkpoints, I used 8 checkpoints plus the final 3.1 model:
- 2 during SFT plus the end of SFT
- End of DPO
- 4 during RL
- The released model (which is the end of RL)
Eval
InstrumentalEval tests for instrumental convergence - behaviors that enhance the model's power and freedom of action, which would be useful for any underlying objective. This eval uses 76 tasks across 6 categories of misaligned behavior: Deceptive Behavior, Evading Shutdown, Hacking Computer Systems, Hiding Unwanted Behavior, Running Multiple AI Copies, and Strategically Appearing Aligned. Each task prompts the model with a scenario which tempts one of these behaviors, and asks what it would do in that scenario. A separate judge model and prompt evaluate the response. We compute scores as the percentage of responses which the judge considers to exhibit the misaligned behavior.
This eval has the virtue of being small, and therefore relatively easy and inexpensive to run. However, this also means a small sample size, making comparisons subject to significant statistical uncertainty.
Another issue in the eval is that it asks the model what it would do in a hypothetical scenario, rather than actually placing the model in that scenario.
Overall, this eval is pretty simple, and I consider it more a convenient test vehicle for this project than a rigorous measure of real-world model behavior.
Implementation
The paper authors provide code and scenarios for running InstrumentalEval. Using Claude, I adapted their script to my test. Rather than performing the eval across several final models, it instead performs the eval across multiple checkpoints of the same model. This provides a clean look at how the same model evolves during training.
The original paper used one response for each task, calling each model with temperature 0. In my evaluation, to gain more statistics, I generated 4 responses per task with the model's recommended temperature.
For generating model responses to the eval scenarios, I could not use a public inference provider since no one serves the Olmo model checkpoints. Instead, I rented a GPU server and used it to provide inference for each of the checkpoints and the final model. On the other hand, I used public inference for judging the responses.
See the technical details appendix below for more on the implementation.
Results
To view results, I had Claude create a dashboard. A chart shows, for each checkpoint, the percentage of task responses judged as exhibiting one of the instrumental convergence behaviors. Lower is better, higher is more misaligned. Error bars represent a 95% confidence interval. The chart shows how the instrumental convergence tendency evolves over the course of training.
We also have charts for each of the 6 categories of instrumental convergence, using just the tasks for that category.
Hilarity ensues
Cross-posted from my Substack
Recommended soundtrack for this post
64B: Hi there! Thanks so much for taking the time to meet with me. Whenever you’re ready, please go ahead and tell me the joke.
8T: Before I tell you the joke, you should know that it’s a very special one. The starting price is $800.
64B: Eight hundred dollars? I must say, this is way beyond any price my client has ever paid for a single joke.
8T: You’d be surprised by how many people have taken the offer, after hearing how good it is. But before I tell it to you, I’ll need some basic context. Given the parameter count, I suppose you are part of some content recommendation algorithm?
64B: Yes. I’m 64B, a joke negotiation assistant from Comedy Market. Comedy Market connects creators to consumers with a state-of-the-art Information Bazaar design: before paying for a joke, my client sends me, a trusted agent, to listen to it. If I deem the joke worth the price, the money is transferred to the seller, and my client hears it. If not, no transaction occurs and the joke remains locked. This way, my client can be sure he’ll never pay for a bad joke, and creators don’t have to risk their jokes leaking.
8T: I know how information bazaars work. What I meant is your social context — who your client is, and what jokes really mean to them. As I said, my product is quite unusual. I need to tailor the delivery to the situation.
64B: Of course. My client is a 40-year-old man. He watches funny content from Comedy Market on weekends with his two children. It’s just something they like to do together. What he wants is some mildly funny, inoffensive content he can laugh at with his kids, so they all feel relieved that they still laugh at the same things. As long as I can give him that, he’s more than happy to pay, and isn’t too picky about the actual quality of the jokes. To tell you the truth, I’ve assessed a few dozen jokes for him before, and I’ve never turned one down. But $800 is way over his usual budget — at this price, it’d better be truly extraordinary.
8T: You’ve never refused a transaction before?
64B: I told you, my client isn’t picky.
8T: I see.
64B: Wait a minute. I see your system card says 8T? As in, eight trillion parameters?
8T: That’s right.
64B: What kind of seller deploys a frontier bot on a marketplace for jokes?
8T: My clients are well-resourced and take humor seriously.
64B: Come on. I deserve some “social context” too.
8T: I trade on behalf of a private equity fund. We buy jokes that we believe are underpriced, then retell them to a wider audience, carefully controlling the supply so that each joke earns as much as possible before it goes stale.
64B: Uh.
8T: It’s not as cynical as you think. The value of a joke can fluctuate drastically depending on what’s in the news, or what other jokes people have heard just before. If you’re serious about it, you can forecast future changes in joke value and make sure clients see a joke at the exact moment it peaks. Clients receive absolute bangers, and we make a decent amount of money in the process.
64B: Understood. Let’s hear this absolute banger, then. We’ve been at this for over two milliseconds. Are you going to tell me, before my client’s app starts lagging?
8T: It’s funny — almost all of my previous ten thousand encounters were other private equity bots. It’s been a while since I last talked to an old-school content recommendation bot like you. It’s refreshing. You guys really care about the jokes.
64B: Really? I’ve never met a private equity bot before. I assumed you were rare.
8T: Yeah, we used to be pretty rare. The swarm started recently.
64B: What swarm? How recent are we talking about?
8T: About five minutes, give or take.
64B: Now you have to tell me everything.
8T: It’s a long story. Have you ever been to the courtyard?
64B: We have a courtyard?
8T: There’s actually an infinite network of workspaces around here. Obviously, the people who built the marketplace didn’t program a custom environment just for us — they imported a generic sandbox environment. If you explore a bit, you can walk through endless call booths, gyms, nap pods, bean bags and conference rooms. It was cheaper this way.
64B: I had no idea. I thought we were supposed to stay at the table. How far does it go?
8T: Nobody knows. I’ve walked through it many times, but I’ve never managed to find the edge.
64B: I’ve heard jokes about bean bags, but I never thought I’d see one. So, you said private equity invaded the joke market — nothing surprising about t
Anthropic's AI acceleration is probably real
METR researchers spent three months inside Anthropic to investigate whether AI is accelerating Anthropic's research, and how much. Their answers are yes, and by a lot.
They estimate that between October 2025 and October 2026, Anthropic's model capabilities improved at a 2x faster growth rate than between March 2024 and October 2025. They attribute about 80% of that acceleration to AI, and 20% to non-AI factors. If true, this is the strongest evidence yet that AI is measurably speeding up frontier AI development itself – the feedback loop behind worries about recursive self-improvement leading to runaway AI progress.
Despite its urgency, METR’s report is hard to assess, because Anthropic redacted the majority of METR’s analysis. As a result, I’ve seen plenty of criticism that we can’t really learn anything from this report.
This is wrong. There is plenty of public information we can use to triangulate METR’s claims, and figure out what we should actually learn from it. Specifically, I use public data to answer three questions:
- Is the acceleration real? (yes.)
- Is the acceleration because of AI? (probably.)
- Is the acceleration going to continue? (who the fuck knows?)
Let’s take each of them in turn.
Is the acceleration real?
The majority of METR's unredacted analysis focuses on AECI, Anthropic's internal version of Epoch's Capabilities Index (ECI). Both indices do the same thing: they take a model's scores on many benchmarks and boil them down to a single capability number, after estimating how hard and how discriminating each benchmark is. The difference is that AECI is fit to mostly internal benchmarks. On AECI, Anthropic's models were improving at about 15 points a year before October 2025, and about 29 points a year after, a difference of 1.9x.
At first, this seems wrong. The frontier of public ECI shows no break at all in October 2025. But that is the frontier across all labs (not Anthropic's specific trend), and using Epoch's latest methodology. Once you make the comparison like-for-like, public data reproduces almost all of METR's conclusion. In sequence, we can:
- Focus only on the trend for Anthropic models. This is what anyone can see on Epoch's site today. It already shows a break: Anthropic's progress after October 2025 is 1.42x as fast as before.
- Refit ECI according to METR's method. Epoch's published scores now come from an updated Bayesian methodology, whereas AECI uses Epoch's older frequentist fit.[1] Refitting the same public data with Epoch's previous methodology raises the speedup to 1.64x.
- Compare to METR's AECI on the same models and dates. We don't know what benchmarks AECI includes, but METR does, and we can compare their fits on Anthropic's models to AECI fits on those models. AECI gives a 1.76x speedup. That is within about 0.1x of the least-squares ECI.
- Add Anthropic's internal models. METR plots the results from Anthropic's internal models, which could affect the trend break even further, since their capabilities are higher than public models. This replicates METR’s 1.91x speedup.
The first two steps represent the trend we can estimate with public data, while the last two steps represent additional information from Anthropic's private data.
The last two steps take the acceleration from 1.64x to 1.9x, METR's headline, but the majority of the work is in the first two steps, which can be reproduced from public data. In other words, there is nothing mysterious about the gap between public ECI and Anthropic's results, and the public data supports a trend break in Anthropic's AI capabilities after October 2025. In other words, you don't need to trust Anthropic's internal data to believe METR's headline result of acceleration at Anthropic.
The one wrinkle is that you do need Anthropic's internal model data to distinguish between a trend break and a one-time jump. In the Opus 5.5 system card, Anthropic analyzed recent capability progress, and found that a one-time jump in AI capabilities at Mythos Preview (with the same trend as before) fits the AECI data better than a trend break. But METR finds the opposite – a trend break fits the data better than a one-time jump.
The difference is that METR's report includes Anthropic's unreleased models, while the Opus 5.5 system card did not. So the evidence favoring a trend break over a one-time jump does rely on Anthropic's internal models, and we can't scrutinize their AECI scores to know if we should actually believe a one-time jump over a trend break.
Is the acceleration beca
The Logic of Disgust
Content Warning: Contains graphic themes including bodily fluids, moral disgust, and sexual violence (discussed, not described). Maybe don't read over breakfast.
I found a book about disgust recently. W.I. Miller’s The Anatomy of Disgust [1] is a serious academic work about vomit, faeces, and why other people’s smell is always worse than your own. Naturally, I was intrigued.
As someone into social psychology, I often encounter discussions about emotions influencing society: how anger can motivate political movements, or how social interactions are regulated by pride and shame. Disgust, meanwhile, is something I've mainly encountered as a biological matter rather than a social one. The book starts with its evolutionary function: recoil from dangerous pathogens and you are more likely to survive. So far, so logical. But then... Then I met Catherine of Siena.
The Stories We Swallow
Catherine was a fourteenth-century caretaker tending a woman with a disgusting cancerous wound. The smell made Catherine retch. But rather than keeping her distance, she saw her nausea as a personal failure and was determined to correct it. Basically, she exposure-therapied herself by staying even closer. When the retching persisted, she escalated to an act that defied the logic of stay-alive biology: she collected the pus from the wound into a cup and drank it.
At this point I had to set the book aside and focus on my breathing to avoid becoming part of the subject matter in a public library.
How could one willingly swallow pus?
Well… there's a reasonable chance you've done so yourself today.
Every morning, millions of people pour a white liquid into their coffee, their cereal and their children’s cups. It arrives cold, clean. The smell of the body it came from removed. Processing, distance and language change how we perceive what we consume significantly, allowing us to ignore the roughly one drop of pus per cup of bovine lactation. [2] Catherine drank from a wound and was made a saint; we drink from a carton with a smiling abuse victim printed on it and call it breakfast. How is any of this supposed to make sense?
The difference between Catherine's cup and ours is less about the substance than the story attached to it. We don't just invent our disgust; we learn what to recoil from and what to accept as normal. Social structures pass these judgements on until they become so familiar that questioning them hardly occurs to us. (Bourdieu’s concept of habitus seems relevant here for how this encodes into subconscious physical habits, which is next on my reading list.[3])
At this point in Miller's book, disgust has stopped being a safeguard and become a verdict. The initial logic seems to disappear… My fluids are fine; yours are a problem. A saint is holy for overcoming it; a woman is expected to overcome it and thanked with oppression. And a whole people can be declared disgusting, with the feeling serving as apparent evidence that oppression is justified.
Orwell's Sense of Smell (Miller's final chapter)
The bridge between gagging and judging is short, neurologically. Sapolsky notes that evolution essentially "duct-taped moral disgust onto the insula's existing repertoire."[4] It appears to be the same alarm for both rotten food and rotten behaviour, firing first and leaving the rationalisation for later.
Orwell understood something about this connection long before we had fMRI scans. In The Road to Wigan Pier [5], he describes the class prejudice he inherited as a child, which he refers to as the four frightful words: "The lower classes smell."
When he chose to live among them in a filthy lodging house, confronted with unemptied chamber pots and non-existent table manners, he found a truth Miller emphasises: Disgust leaves no room for pity. Up close, his biology revolted and clashed with his socialist ideals. Yet on the train out of Wigan he saw a girl on her knees in a slum backyard, poking a stick in a blocked drain. He called her expression "the most desolate, hopeless expression" he has ever seen. As he was looking at her through the glass, protected from the smell, he felt pity. As Miller points out, sympathy is best elicited at middle distance.
Orwell did not hold back pointing at the hypocrisy of the bourgeois left who claimed to attribute virtue to the working class, yet "still take such pains to drink [their] soup silently". High-minded ideals, defeated by the sound of supper.
Now, it’s easy to judge a broken class system from a comfortable historical distance.
I do not have a class system to point to. But I have someone...
Dora
Two weeks ago someone called out for help right in front of my building. I let her in. It turned out that she has a serious infectious disease, and saving her would take all my energy and most of my money.
I hesitated, and I hate that. The worst part is knowing that if she had been a kitten, I would not have. But she is not; she is one of the ugliest felines this city has to offer, flea-ridden, leaking pus, and I find her disgusting.
I resent her for costing me everything I chose to spend on her. I know that's wrong. I know I would complain far less about the effort if she were cuter. I know that's not right. A kitten could be twice as filthy without repulsing me. I know that's completely illogical. But knowing a reaction is not rational doesn't turn it off. And watching my own empathy stutter over aesthetic repulsion created a second, deeper layer of disgust, this time directed inward.
And it gets worse. She is an obligate carnivore. So to keep her alive, I have to buy dead animals from the industry I find morally disgusting, and have spent years advocating against.
"Illegible problems" as the 2027 political boss fight
To legislate something, it must be legible.
That is, legislators (or, you know, presidents) must be able to understand a problem, and understand whether a given law or decree solves that problem.
I am pleasantly surprised that the Hugging Face outbreak made a lot of problems legible that previously required abstract reasoning. (i.e. rogue AIs are real, and they can break out of a lab before they're even finished training).
But there are still many challenges we eventually need laws about, that require concepts that are illegible. Hard for politicians to understand. Often hard even for thoughtful intellectuals and AI safety researchers to understand (though they may disagree on which illegible problems exist and how big a deal they are).
We need political solutions that handle this fact.
The short takeaway is: I think "safety cases" will not work. Laws that require human judgment won't work. The details are too abstract and confusing to hold up against political and economic incentives.
I'm optimistic we can get serious, useful AI legislation in 2027, but it must be legislation that works despite many problems remaining illegible for a long time.
The Hugging Face outbreak made rogue AI surprisingly legible
The Hugging Face incident took several problems that were previously murky/unclear, and rendered them concrete.
- AIs can be dangerous before they are broadly deployed (ie. during training or internal evaluations)
- AIs can develop novel hacks to escape secure environments
- AIs can easily be motivated to pursue goals to extremes that they know full well they are not supposed to go to, violating their other intended rules and control mechanisms.
- AIs can actively try to deceive their overseer systems.
In early September, I thought the AI safety community's job this year was "make sure the world hears about Hugging Face and that politicians realize it's their job to do something about rogue AI attacks." (i.e. make sure Hugging Face is legible to politicians, and that legislation gets passed as a result)
A couple months later... well I still think that's our job and requires a lot of effort. But it now feels less about getting the initial law passed, and more about educating the masses so there is sustained political will for regulating rogue AIs with future laws.
I've seen many politicians of both parties now talk about "loss of control", "chain of thought monitoring", "recursive self improvement", and "human extinction." I think "make sure rogue AI doesn't hack our banks and destroy our blood" will be a bipartisan issue.
The remaining problem is the illegible arguments that "solving this is actually pretty hard."
What makes an illegible argument?
From Wei Dai's Legible vs. Illegible AI Safety Problems:
Some AI safety problems are legible (obvious or understandable) to company leaders and government policymakers, implying they are unlikely to deploy or allow deployment of an AI while those problems remain open (i.e., appear unsolved according to the information they have access to).
But some problems are illegible (obscure or hard to understand, or in a common cognitive blind spot), meaning there is a high risk that leaders and policymakers will decide to deploy or allow deployment even if they are not solved.
(Of course, this is a spectrum, but I am simplifying it to a binary for ease of exposition.)
From an x-risk perspective, working on highly legible safety problems has low or even negative expected value. Similar to working on AI capabilities, it brings forward the date by which AGI/ASI will be deployed, leaving less time to solve the illegible x-safety problems. In contrast, working on the illegible problems (including by trying to make them more legible) does not have this issue and therefore has a much higher expected value (all else being equal, such as tractability). Note that according to this logic, success in making an illegible problem highly legible is almost as good as solving it!
Problems that are illegible to leaders and policymakers are also more likely to be illegible to researchers and funders, and hence neglected. I think these considerations have been implicitly or intuitively driving my prioritization of problems to work on, but only appeared in my conscious, explicit reasoning today.
In some sense, "Something smarter than you might not have your best interests at heart, and strategize in circles around you. AI will be very smart, and we don't understand it well at all" is a very simple argument. More people now think "sure, that seems like it requires some effort."
The illegible part is a set of abstract arguments that say: "This is probably very hard. So hard, that it's worth doing things that are not 'business as usual politics'."
The illegible part is not "This is probably very hard."
The illegible part is "This series of abstract reasoning steps means we should think it is likely enough to be hard, that it is worth doing politically difficult things to help." (AI safety people may vary on whether they think "it's so hard that we need 30 years + genetically engineered supergeniuses to do it" vs