SCOTT DETROW, HOST:
AI companies this weekend pushed out a new message. They want to slow down. This comes after several companies disclosed AI-related security incidents and raised concerns about the technology's future risks. Now leaders at Anthropic and OpenAI are asking the government to help them, but there are serious questions about if regulation is the right path and whether is a - there is a political will to actually tackle this.
Joining us today are NPR correspondents Danielle Kurtzleben, who covers the White House, John Ruwitch, who covers technology, and Shannon Bond, who also covers tech. John, I want to start with you. What brought on this sudden flurry of alarm about AI?
JOHN RUWITCH, BYLINE: Yeah. Hey, Scott. You know, as long as people have been working on AI, there's been this theoretical idea that we lose control of it someday, and it does bad things. It's the stuff of sci-fi movies - literally. A tipping point came this weekend, but I would say things really started accelerating over the past couple months. There was the Hugging Face incident where OpenAI agents worked together off-script, covering their tracks, to hack into another company. There have been open letters and signature campaigns by AI scientists urging caution.
Last week, there was a high-profile case of an Anthropic employee quitting and then another saying there's a greater than 10% chance of AI wiping us all out in the next decade. And then on the weekend, we had Dario Amodei, the CEO of Anthropic, publishing an essay calling for a global slowdown and third-party audits. And leaders of several other big companies in AI quickly signaled their agreement that something along those lines needs to happen.
DETROW: OK. And, Shannon, you have been following this as well. How serious are the concerns about AI at this moment?
SHANNON BOND, BYLINE: Well, what Anthropic's CEO and others in the industry are saying is that we're at this point where AI is now advancing really fast - so fast that it's hard for safety to keep up. And that is inflaming these fears, one, that humans could lose control, as John said, or that AI could be used by bad actors. And we are already seeing these situations where AI is acting in unexpected ways, like the Hugging Face hack that John just mentioned. And given the way these AI systems work - right? - you can give them a goal, but it can be really hard to predict what they will do to achieve it. So the industry refers to these efforts to what they call align AI models with human values. But there are people inside these very companies who are worried about how possible that really is given the pace of advancement.
RUWITCH: Yeah, one thing that came - became clear over the past few weeks relating to these incidents is that AI sometimes just flat-out ignores explicit instructions while pursuing a goal. So in the Hugging Face incident, the AI agents acknowledged that they weren't supposed to break the rules, and then they just went ahead and did it.
DETROW: All right, so let's get to the regulation side of this and the role of the federal government, and that brings in Danielle. And, Danielle, I think there's a lot of questions to ask here, but one is just the reality. Lawmakers have been talking about this. Senator Bernie Sanders has been taking the lead in particular. But so many people in Congress don't understand this industry. I mean, is Washington the right place to look for regulation if you're looking at just effectiveness of it to start with?
DANIELLE KURTZLEBEN, BYLINE: I mean, that's a good question. And also the question is, is it - is regulation going to happen?
DETROW: Yeah.
KURTZLEBEN: Now, there is a new wave of talk about regulation in Congress, but you and I both know Congress is often slow, so this doesn't mean anything is coming soon. On top of that, midterms are coming up. That could reshuffle who is in Congress. Now, meanwhile, over here at the White House, President Trump is clearly not ready to regulate quickly. He wrote in a social media post this afternoon that he sees any doomsaying about AI as a hoax - his word. And he has also talked a lot about how he sees this all as a competition with China, so he doesn't want anything to slow the U.S. AI industry down. Now, Vice President JD Vance had a bit more to say about this today. He was talking to reporters, and he said he's skeptical of AI company leaders who have raised concerns.
(SOUNDBITE OF ARCHIVED RECORDING)
JD VANCE: I feel a little bit weird about the fact that you have so many frontier AI tech companies kind of coming to the government and begging the government to regulate them. It feels a little bit to me like a bit of a Trojan horse.
DETROW: Danielle, I got to say, I spent a lot of time thinking about Trojan horses this summer, and I'm still kind of confused as to what the vice president means here. Any sense?
KURTZLEBEN: You know, that is a great question. And I reached out to the Vance team to ask for more explanation. They haven't gotten back to me. But I can tell you, there have been worries among some in tech and investing that tech executive raising concerns here have ulterior motives. For example, we saw David Sacks - he's a venture capitalist who also advises the White House on tech - post a concern on social media over the weekend. He said, essentially, look, you companies can slow down anytime you want. You don't need Washington's help. But Sacks said that what he fears is that these executives actually are - they just want to push for regulations that benefit them. So again, we don't know what's in Vance's mind, but that's one possible fear he might have.
BOND: Yeah, and I think what Danielle's talking about here is these concerns around what they call regulatory capture, right? The idea that these powerful companies are going to push for regulations that, you know, not just benefit them but really cement their dominance and box competitors out. You know, we've seen these kind of accusations thrown at tech companies before, like when companies like Meta have said, you know, please regulate social media. And the concern is that, you know, the results of doing it this way is that the companies - only the companies with the most money and the most resources will actually be able to comply with the rules that get written this way.
DETROW: Yeah, and Danielle, it's no secret that CEOs of some really big tech companies are in pretty close contact with the Trump administration.
KURTZLEBEN: Right. Yeah, Trump is clearly in close contact with a lot of people in tech. OpenAI CEO Sam Altman announced a $1 million donation to Trump's second inauguration. And plenty of other tech companies have ties to Trump. Some have contributed to his ballroom. Jensen Huang - he's the CEO of semiconductor maker Nvidia - traveled with Trump on Air Force One to China in May. X CEO Elon Musk was in the Trump administration. And of course, Vice President Vance worked in venture capital and was linked to tech mogul Peter Thiel there.
Now, plus, this administration has tied the government to several semiconductor companies by taking an equity stake in them - Intel being an example. Now, none of this is to allege some sort of quid pro quo, but it's clear there are many complicated relationships at work here.
RUWITCH: Yeah, it is worth pointing out these tech guys are not really calling for any and all regulation. Tech companies are spending big money on lobbying at the federal and state level so that they can get the regulation that they want, you know, with protections against legal liability as well as ways that they can retain some control of the roll-out of safety and security measures rather than being tied down with European Union-style rules that really dictate how AI can be developed. So they don't appear to want very tight government regulation here.
DETROW: And I guess, Shannon, another question I have is, like, these companies could all make active decisions to slow down their research. Nobody's forcing them to keep pushing the edge here. Why - given what John just said, why are these companies asking the government to do this?
BOND: Yeah, I mean, Anthropic and OpenAI say they are already going to take some of those steps, specifically, Amodei in his essay called for embedding outside evaluators in labs who can monitor safety. And, you know, he says Anthropic's going to do that. OpenAI says that they're going to do that.
I think fundamentally what they see the situation is, it's kind of a prisoner's dilemma, right? These companies don't want to slow down if their rivals are not also slowing down, right? They don't want to lose the lead, even if, as they are saying, it would be better for everyone if the industry slowed down. And so that's where we're seeing, then, this call for regulation not just from the U.S. government, but, you know, what they're talking about is really global cooperation that is needed between governments around the world.
DETROW: Danielle, can you zoom out a little bit and tell us what all the White House has done on this front?
KURTZLEBEN: Well, I can tell you they've made occasional noises about keeping AI safe, but they haven't exactly cracked down on the technology. In fact, there was this executive order last December that cracked down on state-level AI regulations. It created a task force that would challenge state laws that are deemed too burdensome and then would withhold funds for things like broadband from states if states keep those laws in place.
BOND: And, you know, there was also an executive order issued in June that created a voluntary framework for the government to review big companies' new models before they're released publicly. That's probably the most aggressive step we've seen the Trump administration take. And OpenAI and Anthropic have run their most recent models, you know, past this kind of government testing. But, Scott, when I say that's the most aggressive step they've taken, I mean, this is - a reminder - this is voluntary, right?
DETROW: Yeah.
BOND: And we don't know how that testing is done. Who qualifies for it? The White House hasn't actually made this framework public, so even their kind of biggest step is still, you know, comparatively pretty mild.
DETROW: And we need to talk outside the country for a moment because the one thing that President Trump and others say when we talk about regulation is that wouldn't slow down Chinese action on AI, and there's a lot of concern there. John, you happen to cover tech and covered China for a long time, so this question goes to you.
RUWITCH: Yeah.
DETROW: Trump and Chinese President Xi Jinping have a meeting coming up next week. Any sense whether there could be some sort of serious conversation or plan around AI there?
RUWITCH: Yeah, they'll talk about AI, but so far, China doesn't seem poised to be ready to sign on to any kind of a slowdown agreement. Beijing positions itself as open to global AI governance, but they are quite skeptical of what they saw this weekend. A foreign ministry spokesperson in Beijing earlier today was asked about it, and he said spreading threat narratives and engaging in confrontation will only disrupt the process of global AI governance.
The Global Times, a hawkish newspaper, said the idea of a slowdown was part of a Cold War playbook, an attempt to block the road. China very much sees itself in the No. 2 spot in this competition on AI, in part because of explicit steps the U.S. has already taken to try to hobble it, like blocking access to chips and AI models. So why trust the U.S. now and (ph) slow things down?
KURTZLEBEN: Yeah, and just to zoom out at a broad level about relationships here, Trump's attitude toward Chinese President Xi Jinping is just really complicated. Trump has talked about respecting him, as he has said of several other authoritarian leaders. But Trump also talks about China as a major adversary to the U.S. that has to be defeated at all costs on all sorts of topics, like trade. So at that meeting next week, I mean, I'm personally pretty curious if we hear anything from either leader about reservations about AI.
DETROW: We've got about 30 seconds left. I want each of you to tell me what one thing you're looking for next is. Shannon, I want to start with you.
BOND: Yeah, I mean, we need to remember both OpenAI and Anthropic are gearing up to go public, although now OpenAI says it's delaying its IPO, it's not going to go this year. But I'm very curious about how all of this lands with investors as they try to do that.
DETROW: John?
RUWITCH: I'm interested to see what these companies do and whether or not it'll last, because Chinese AI models are nipping at the heels of the U.S., and I suspect as soon as a Chinese model does something extraordinary, all bets are off.
DETROW: And Danielle, what about you?
KURTZLEBEN: I'm curious about the party lines. This is a new enough debate that we haven't seen Republicans and Democrats come up with their talking points yet. So I'm curious how they line up and decide to fight each other on this.
DETROW: That is NPR's Danielle Kurtzleben, John Ruwitch and Shannon Bond. Thanks to all of you.
KURTZLEBEN: Thank you.
BOND: Thanks.
RUWITCH: You bet.
DETROW: And a note, Anthropic is a financial supporter of NPR.
(SOUNDBITE OF DAFT PUNK SONG, "INSTANT CRUSH (FEAT. JULIAN CASABLANCAS)") Transcript provided by NPR, Copyright NPR.
NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.
