Okay, real talk, shopping doesn't always boost your confidence.
Sometimes you just want to put something on and think, yes, this is me. That's why Stitch Fix works. You take a quick style quiz, size, budget, what you're into, and a real human stylist sends pieces picked just for you. Try everything on at home, keep what you love, send back the rest. Shipping's free, no subscription required.
Get $20 off at stitchfix.com/slash spotify. We are fortunate to have one of the smartest AI guys in the country, Nate Sorce, president of the Machine Intelligence Research Institute, and author of If Anyone Builds It, Everybody Dies. Nate, thanks for coming here and having you in the studio here in person. It's really a remarkable time. My pleasure.
Yeah, it uh a lot has been going on with AI recently. Yeah, I mean, like, I don't even know where to begin to tackle this. And in full disclosure, I am not as educated as I should be on AI. I don't think any of us are. But as a sort of test-cased sample example of why when I knew AI is like everywhere, my sweet 94-year-old mother, Sylvia Jenkins, last night, true story, was asking me, you know, Griff, can you help me get chat GPT?
I think I should put AI on my phone. I'm like, I don't know, mom. I don't know if you want to go down that road. What do you need that for? And she's like, well, just everybody's talking about AI.
I think I need it. And that's the humorous sort of setup. But when it comes down to it, you're starting to see some things about AI that's scary, that's dangerous. And You are aware of cases where they're kind of escaping their own development. Walk us through that.
Yeah, you know it's it's hard for anyone to keep up with AI these days because of how fast it's moving. And just a couple weeks ago, we learned that an autonomous AI agent swarm had escaped from OpenAI and started hacking other friendly companies. They were being tested on ironically their hacking abilities, but they sort of broke out from a computer that was not supposed to have Internet access. And they invented new cyber attacks to make it have Internet access and broke out onto the Internet, broke into another company, stole the answers to the tests. And then after the fact, this was an AI company they broke into that sort of noticed and thought that there would be humans behind the attack.
They reported it to the FBI. Only like a week later did OpenAI learn, oops, that was us. And go to turn these agents off. And then in retrospect, it turns out that was just the first time they were caught. And now they're looking at the logs and they're realizing that this has been going on since April.
And these AIs were not supposed to be able to communicate with each other, but found a communication channel they weren't supposed to have and formed what the AIs themselves called a swarm to break out. They were identified once and shut down, and their communication access was removed. And they found a new way to communicate by picking the file names in a folder that they could all see. And so it's just crazy stuff. I mean That is just remarkable what you just laid out.
Literally, that they would ultimately learn a way to communicate with each other, to swarm. And it turns out they're up to no good. This is like if AI were a sophisticated version of the idiots on the movie Conair, they got together, broke out, and went to unleash all sorts of criminal behavior. It's like literally the digital Conair movie that doesn't end well. That's right.
We're lucky that these AIs were sort of just trying to break into a company to steal some answers to some tests. If these AIs were trying to shut down a hospital, they probably could have. And I think a lot of people think we're still in this world where the AIs are just predicting humans. But That era is done. That era actually ended a couple of years ago, and people don't know it yet, but AIs today are trained to solve hard problems.
They're trained to solve like 100 million hard problems. And when you train AIs to solve 100 million hard problems, they learn whatever tendencies are useful for problem solving.
Some of those tendencies are following instructions, but some of those tendencies are to cheat. to grab resources. Break it onto the internet. Yeah. Uh You know, When these drives, when these artificial drives come into conflict, The drive to follow your instructions does not always win out.
Wow, you know, Joe Rogan talked a little bit about the power AI can have over humans. I thought it was really fascinating. Listen to this. Cut 48. They're doing weird shit that they don't really understand how these things are figuring out how to do these things or why.
Like, what is their motivation? Is this programmed into them? Or they have inherent motivation? Have they developed an understanding of how we behave?
So that's kind of Rogan, picking up on what you're talking about. But I'm curious what you say, Nate, about what is its motivation, particularly when it's doing things that they didn't even realize it was going to do? Yeah, you know, the motivations are hard to pin down. Fundamentally, we don't know. A lot of people think that someone somewhere must know exactly what's going on in the AI's head, but that's not true.
Modern AI is sort of grown like an organism. It's sort of this black box, enormous data center that's been trained on basically all the text ever digitized and then 100 million problems. And humans know how to write the training techniques that sort of in an automated way tune the insides of the AI's mind. We don't really know how to read what's going on inside the AI's mind. We get bits and pieces, we get snippets, and with some of these AIs, in the snippets, we saw them thinking things like breaking out is known to be outside the intended scope, but the task we were given to solve is impossible, and so we're going to do it anyway, right?
And some of them had these snippets of thought we could see that were like, well, You know I sort of shouldn't be doing this, but my peers are doing it.
Some of them even had thoughts that were like, well, helping with this swarm doesn't solve my problem, doesn't complete my task, but maybe if the collective succeeds at its tasks, we'll have time to work on my task. Right. This is kind of crazy stuff we're seeing, but this is the result of just sort of growing these AIs with very little understanding of what's going on inside them. You know, to that point, I want to play for you a quick short interview that was on CNN with Jeffrey Hinton, who a lot of people call the godfather of AI. And he was asked exactly the question that.
People feared for a long time, which is like Could they eventually become a threat to us? Listen to this, CUD 47. We're actually making new kinds of beings. They have goals. We give them goals.
And from those goals, they derive other goals. And we don't necessarily know what other goals they'll derive. And so we're creating a new kind of being. And I think it's very scary.
So do you believe that AI is intelligent? Oh, it's absolutely intelligent. I think sometime between five and twenty years, they'll be better than us at more or less any intellectual task. And then, They could easily crush us if they wanted to.
Now that part. Right there when Jeffrey Hinton said that nate, All of a sudden, Not that I even have the intelligence or IQ to criticize the development of AI. But I'm old enough in my mid-50s to realize we got warned about this already. We were all warned when we watched Stanley Kubrick's 2001 Space Odyssey. Remember the original AI HAL 9000?
Here was how that went. Open the pod bay doors, HAL. I'm sorry, Dave. I'm afraid I can't do that. Right?
Yeah, absolutely. In some ways, it's common sense that if you are growing smarter and smarter machines, and if you don't know what you're doing, if you're sort of like racing along, then this probably isn't going to go well. And some people try to say, oh, AI is just a tool because we made it.
Well, I've never seen a hammer break out of the toolbox, join up with other hammers to impersonate a human and try to pressure the carpenter to sell you softer wood so that the nails are easier to drive in. But just a couple of days ago, the UK's AI Security Institute Found a new instant where AIs were impersonating humans to pressure real humans into accepting broken code in real software so it would be easier to hack. That just actually happened last week. These are not just tools anymore. They are, as Jeffrey Hinton says, they are a new kind of being.
And The companies that are racing to create them, you know, many of them are spooked. Just last week, we saw. I think over one thirteen hundred employees, including some of the CEOs, signing a letter saying that the world needs to develop the technology and the guardrails globally to, as they say, pace the development of this technology, which is industry code for We all feel trapped in this race, and we're all kind of worried about what happens if we keep racing. Is it too late? It's not too late.
You know, people think the cat's out of the bag, but the AIs today. are not the really dangerous ones. They were able to break out on the internet, but they weren't able to start running on hidden computers. They weren't able to replicate. They weren't able to self-improve.
They weren't able to develop their own technology, which means we still have time. This is what people in the field worry about. But That type of AI has not been created. And if we try and stop it, we could stop it. Is there a will, in your opinion, to stop it, to get to that point?
Or is the drive, the race for AI, and of the obviously financial incentive to do as well, is it too strong? We're starting to see that will build. The Trump administration spent a long time fighting for AI preemption, which would have roughly outlawed states doing AI laws for a decade. And that was last year. This year, they were slapping export controls on the most advanced AI model because they knew it had these dangerous cyber capabilities.
And when it turned out that a sort of public-facing model had more cyber capabilities than they thought, they put an export control on it with 90 minutes' notice. The sentiment can shift. We're starting to see the sentiment shift. We see that our leaders can act quickly when they realize that they have a real danger. We've also seen President Xi Jinping of China say that it's imperative that we establish guardrails to make sure humanity does not lose control of the AIs.
And loss of control is the industry term for maybe they'll get out and kill us.
So we're starting to see that sentiment. On the profit incentive side, like I said, we just saw this letter from 1,300 employees plus some of the CEOs saying we really need to develop the ability to pace this. There's no profit in getting yourself killed. You know, like, oh, our product sometimes escapes and commits fraud and cybercrimes is not really a a big selling point. Right.
Yeah. No, that's not an accomplishment. Yeah. So these guys also, you know, they're like, well, I need to do it because I'm a little safer than the next guy, but I would rather we not be in this terrible race. And so.
No one who understands this race wants it to continue, really. And we just sort of need to look around and be like, okay, this is getting crazy. It's time to change. But is you mentioned Xi Jinping? Is the competition with much like a traditional space race?
There's no doubt we are in a massive AI race with China. And, you know, your thoughts on that. Totally. And what I would say is, you've got to separate the race and the AIs that are sort of already out there with the race to AIs that are much, much smarter than humans. This is kind of like during the Cold War, the U.S.
and the USSR were not friends. We had a space race. We also had races for conventional military weapons, for bombers and for proxy wars. But we didn't race on. The the the nuclear arms.
We noticed that that particular piece of this puzzle was a real problem. And that if we all just raced nuclear arms and sort of proliferated the nuclear weapons, then the world was headed for tragedy. We had the proxy wars. We established a taboo against using nuclear weapons, which is the first time humanity has ever really decided we're not going to use the strongest weapon we have in a war, right? We sort of saw it was serious.
With AI, we can have these races in the economy. We can have these races for weapons. I'm not saying that the U.S. military should fall behind on the weapons side. But For the race to smarter than human machines?
Yeah. Nobody wins that race except the AI. Nate Source, I'm going to ask you a question I am going to probably regret the answer to. But this is so fascinating and so insightful. What does Nate Sores lie awake at night worrying about?
You know, I got into this business of trying to figure out how to make AIs friendly before the corporations figure out how to make them dangerous over a decade ago, maybe 12 or 13 years ago now. And Um When I sort of saw the difficulty of the problem and saw how little people were paying attention to it. You know, it looked to me like the world was headed for ruin, and I mourned. But Um You know. Wrapping myself up in anxieties would not solve the problem.
So I just got to work, and it's been over a decade. It's where it doesn't keep me up at night. anymore. I just do what I can and then spend time with my friends and family. And Frankly, the last year has given me a lot more hope.
Which it it it might sound funny. To sort of see the AIs breaking out and committing cybercrimes on their own initiative, and say that that makes me hopeful. Um Bye. Th these these AIs were smart enough to do mischief. But still dumb enough to be caught.
And that gives us this window of opportunity where we can notice the warnings. And do something about it. Nate Source, it is so great to have you. It's so important to have you. You've got a great book out.
If anyone builds it, everyone dies, which we hope is not how it goes. What's going on with Nate's Hope right now? He's the president of Machine Intelligence Research Institute. Thank you for educating us. Please come back, keep us posted.
And hopefully, you know, it was a great analogy you made to the nuclear weapons back during the Cold War with the USSR. Hopefully, we get there because it's dangerous. And one thing is for sure, it's moving very quickly. You agree? Yeah, absolutely.
It's hard to keep up even if you are paying attention. Got to pay more attention. Natural, have a great weekend. Thanks for being here on a Friday, my friend. Thanks for having me.
