🤖 AI 资讯

Hello and welcome to Eye on AI. In this edition:

• Anthropic and OpenAI both release new, cheaper AI models.
• U.S. President Donald Trump creates an “AI Force.”
• China and the U.S. agree to discuss an AI incident hotline.
• Microsoft executive called OpenAI training on publishers’ copyrighted works “the largest theft of labor in history.”
• AI scientists are making good progress.
• And the AI safety researchers are not all right. 

Before we get to today’s AI news—please consider joining me at the inaugural Fortune AIQ Summit at the New York Stock Exchange on Oct. 1: Spend the afternoon with senior executives from companies on the Fortune AIQ 75 list and explore how you can scale your AI experimentation and translate investments into measurable business value. I will be leading discussions alongside co-hosts, Fortune Editor-in-Chief Alyson Shontell and Live Media Editorial Director Andrew Nusca. Apply here to attend.

Ok, so today’s newsletter is a bit of a potpourri.

All eyes this week, will be on the talks between U.S. President Donald Trump and China’s President Xi Jinping in Washington. We know AI governance is on the agenda of that meeting, which takes place Thursday, but that’s about all we know. My colleague Emily Forlini wrote a piece last week on why the Trump-Xi meeting is unlikely to result in any kind of international agreement to slow the pace of AI development or create an agreed framework for controlling the technology. It’s worth a read.

That said, there was an inkling that these two AI superpowers might, in fact, be able to agree to a few basics. Treasury Secretary Scott Bessent emerged Sunday from meetings with a Chinese delegation led by Vice Premier He Lifeng that took place at the headquarters of JP Morgan in New York and announced that the two sides had agreed to hold further talks about setting up a hotline to notify one another of AI incidents that created national security concerns. What exactly this means in practice is unclear. But as Bessent told reporters, “moving from opaque to more transparency between the number one and number two AI powers in the world is very important.”

Such hotlines have historically helped ease tensions between rivals. At the very least, they might prevent some sort of accidental incident caused by AI from tripping over into armed conflict, or even nuclear war. There are already signs that such AI-triggered accidents are possible: just this weekend, CNN reported that earlier this year the U.S. military almost attempted to seize a Chinese ship in the Middle East that an AI-generated intelligence report had suggested was carrying nuclear weapons components to Iran. The intelligence had been generated by an AI model that fused secret U.S. intelligence with open-source data. The only problem is that the model’s conclusion was an AI “hallucination”—and the error was caught only after the U.S. had launched aircraft carrying armed personnel who were preparing to intercept the Chinese vessel. Had the error not been spotted in time, it could have led to a diplomatic incident—or far worse.

But while a hotline might prevent this kind of incident from spiraling into war between the U.S. and China, it is less clear whether it would do anything to help the world avoid or contain a “loss of control” incident involving an advanced AI system that goes rogue.

Phoning a friend ain’t gonna help

For instance, what if one country (or companies based there) creates an AI that goes rogue and starts hacking banks around the globe? And what if that AI copies itself on servers around the world, making it difficult to shut down without shutting down large parts of the internet? While notifying the other country about this is nice—it might help them take some action to secure their financial infrastructure before too much damage is done—it isn’t clear exactly what the country receiving the heads up is supposed to do. As AI safety researchers keep warning, the world hasn’t figured out a good way to guarantee that AI models adhere to human intentions and values. And neither the U.S. or China has enacted any rules requiring AI models to have some sort of “kill switch.” Nor is it even clear that an effective kill switch can even be built.

So sure, this is a promising, baby step towards some sort of AI governance agreement between the U.S. and China. But there’s also a long history of hotlines failing to evolve into any kind of lasting diplomatic resolution. Right now, the Washington-Beijing AI hotline is a lot like agreeing to build the “Bat Sign” before Batman exists. You can flash it into the sky, but no caped crusader is coming to save us.

Why the hack of OpenAI should worry every company

Another big piece of AI news from last week was the revelation, first reported in the Wall Street Journal, that a small team of white hat hackers had used Anthropic’s Claude Opus 5 model to hack into the community-message platform Discourse and from there to compromise the ChatGPT account of an OpenAI employee. Once they had access to that account, they were able to use it to also access and alter software sitting on a repository where OpenAI stored a lot of its sensitive code.

Coming amid the raging debate about the best way to prevent “rogue AI” incidents, many cybersecurity experts jumped on the incident to make the case that the real issue is not so much that AI is increasingly uncontrollable, but that leading AI companies have horribly lax security. It’s not just OpenAI. Anthropic has also had embarrassing security lapses too. It was ironic, many critics pointed out, that both companies are using the threat of AI-powered cyber attacks as part of a marketing pitch for customers to use their most advanced (and expensive) AI models to secure their networks before the bad guys get to them, but neither seems to have yet done a very good job of doing that themselves.

The incident also highlighted a couple of uncomfortable truths. One is that AI agents running inside companies are a great target for hackers. Making these AI agents useful often means giving them access to lots of other tools and data sources, many of which contain sensitive corporate information or control key business processes. If hackers can gain access to and take command of these agents, they can do a lot of damage very quickly.

To cybersecurity experts, the answer is to lock these AI agents down, and to operate based on “zero trust” principals. Treat every AI agent as a potential insider threat. Give the agents access only to the data they need to complete a task—preferably with using a “just-in-time, just enough a