The Godfather of A.I. Has Some Regrets (2023)

This transcript was created using speech recognition software. While it has been reviewed by human transcribers, it may contain errors. Please review the episode audio before quoting from this transcript and email transcripts@nytimes.com with any questions.

From “The New York Times,” I’m Sabrina Tavernise. And this is “The Daily.”

[THEME MUSIC]

As the world begins to experiment with the power of artificial intelligence, a debate has begun about how to contain its risks. One of the sharpest and most urgent warnings has come from the man who helped invent the technology. Today my colleague Cade Metz speaks to Geoffrey Hinton, who many consider to be the godfather of AI.

It’s Tuesday, May 30.

Cade, welcome to the show.

Glad to be here.

So a few weeks ago you interviewed Geoffrey Hinton, a man who many people as the godfather of AI. And aside from the obvious fact that AI is really taking over all conversations at all times, why talk to Geoff now?

I’ve known Geoff a long time. I wrote a book about the 50-year rise of the ideas that are now driving chat bots like ChatGPT and Google Bard. And you could argue that he is the most important person to the rise of AI over the past 50 years. And amidst all this that’s happening with these chat bots, he sent me an email and said, I’m leaving Google and I want to talk to you, and that he wants to discuss where this technology is going, including some serious concerns.

Who better to talk to than the Godfather of AI?

Exactly. So, naturally, I got on a plane and I went to Toronto —

— to sit down at his dinner table and discuss.

Geoff is a 75-year-old Cambridge-educated British man who now lives in Toronto. He’s been there since the late ‘80s. He’s a professor at the university.

So how does Geoff become the Godfather of AI? Where does his story start?

It starts in high school. He grew up the son of an academic, but he always tells a story about a friend describing a theory of how the brain works.

This friend talked about the way the brain stores memories, and that he felt it stored these memories like a hologram. A hologram isn’t stored in a single spot. It’s divided into tiny pieces and then spread across a piece of film. And this friend felt that the brain stored memories in the same way — that it broke these memories into pieces and stored them across the network of neurons in the brain.

It’s quite beautiful, actually.

It is.

That sparked Geoff’s interest. And from there on, he spent his life in pursuit of trying to understand how the brain worked.

So how does Geoff start to answer the question of how the brain works?

So he goes to Cambridge and he studies physiology, looking for answers from his professors. Can you tell me how the brain works? And his physiology professors can’t tell him.

And no one can tell him how the brain works.

We still don’t the details of how the neurons in our brains communicate with one another as we think and learn.

He then moves into a relatively new field called artificial intelligence.

[MUSIC PLAYING]

The field of artificial intelligence was created in the late ‘50s by a small group of scientists in the United States. Their aim was to create a machine that could do anything the human brain could do. And in the beginning, many of them thought they could build machines that operated like the network of neurons in the brain — what they called artificial neural networks.

But 10 years into this work, progress was so slow that they assumed it was too difficult to build a machine that operated like the neurons in the brain. And they gave up on the idea.

So they embraced a very different way of thinking about artificial intelligence. They embraced something they called symbolic AI.

You would take everything that you and I know about the world and put them into a list of rules — things like you can’t be in two places at the same time or when you hold a coffee cup you hold the open end up. The idea was that you would list all these rules step by step, line of code by line of code, and then feed that into a machine. And then that would give it the power that you and I have in our own brains.

So, essentially, tell the computer every rule that governs reality, and the computer makes the decisions based on all of those rules.

Right.

But then Geoff Hinton comes along, in 1972, as a graduate student in Edinburgh. And he says, wait, wait, wait. That is never going to happen.

[CHUCKLES]: That’s a lot of rules.

You will never have the time, and the patience, and the person power to write all those rules and feed them into a machine. I don’t care how long you take, he says, it is not going to happen. And, by the way, the human brain doesn’t work like that. That’s not how we learn.

So he returns to the old idea of a neural network that was discarded earlier by other AI researchers. And he says, that is the way that we should build machines that think. We have them learn from the world like humans learn.

So instead of feeding the computer a bunch of rules, like the other guys were doing, you actually feed it a bunch of information. And the idea was that the computer would gradually sort out how to make sense of it all, like a human brain.

You would give it examples of what is happening in the world. And it would analyze those examples, and look for patterns in what happens in the world and learn from those patterns.

But Geoff is taking up an idea that had been largely discarded by the majority of the AI community. Did he have any evidence that his approach was actually going to work.

His only evidence was that, basically, this is how the human brain worked.

And, at the time, many of his colleagues thought he was silly for even trying.

OK, so what happened next?

So after graduate school, Geoff moves to the United States. He’s a post-doc at a university in California. And he starts to work on an algorithm, a piece of math that can realize his idea.

And what exactly does this algorithm do?

Geoff essentially builds an algorithm in the image of the human brain. Remember, the brain is a network of neurons that trade signals. That’s how we learn. That’s how we see. That’s how we hear. What Geoff did that was so revolutionary was he recreated that system in a computer. He created a network of digital neurons that traded information, much like the neurons in the brain.

So that question he set out to answer all those years ago — how do brains work — he answered it, only for computers, not for humans.

Right. He built a system that allowed computers to learn on their own. In the ‘80s, this type of system could learn in small ways. It couldn’t learn in the complex ways that could really change our world. But fast forward a good three decades. Geoff and two of his students built a system that really opened up the eyes of a lot of people to what this type of technology was capable of.

He and two of his students at the University of Toronto built a system that could identify objects in photos. The classic example is a cat. What they did was take thousands of cat photos and feed them into a neural network and in analyzing those photos, the system learned how to identify a cat.

It identified patterns in those photos that define what a cat looks like — the edge of a whisker, the curve of a tail. And over time, by analyzing all those photos, the system could learn to recognize a cat in a photo it had never seen before. They could do this not only with cats but with other objects — flowers, cars. They built a system that could identify objects with an accuracy that no one thought was possible.

So it’s basically image recognition. Right? It’s presumably why my phone can sort pictures of my family and deliver whole albums of pictures just of my husband or just of my dog, and photographs of a hug or a beach.

Right. So in 2012, all Geoff and his students did was publish a research paper describing this technology, showing what it could do.

That set off a race for this technology in the tech industry.

There was a literal auction for Geoff and his two students and their services.

Google was part of the auction, Microsoft — another giant of the tech world — Baidu, often called the Google of China.

Over two days, they bid for the services of Geoff and his two students to the point where Google paid $44 million, essentially, for these three people who had never worked in the tech industry.

So what does Geoff do at Google after this bidding war for his services?

He works on increasingly powerful neural networks. And you see this technology move into all sorts of products not only at Google, but across the industry.

This is what drives Siri and other digital assistants. When you speak commands into your cell phone, it’s able to recognize what you say because of a neural network. When you use Google Translate, it uses a neural network to do that. There are all sorts of things that we use today that use neural networks to operate.

So we see Geoff’s idea really transforming the world, powering things that we use all the time in our daily lives without even thinking about it.

Absolutely. But this idea, at Google and at other places, is also applied in situations that make Geoff a little uneasy. The prime example is what’s called Project Maven. Google went to work for the Department of Defense, and it applied this idea to an effort to identify objects in drone footage.

Hmm.

If you can identify objects in drone footage, you can build a targeting system. If you pair that technology with a weapon, you have an autonomous weapon. That raised the concerns of people across Google at the time.

Geoff never wanted his work applied to military use. He raised these concerns with Sergey Brin, one of the founders of Google, and Google eventually pulled out of the project. And Geoff continued to work at the company.

But around the same time, the industry started to work on a new application for the technology that eventually made him even more concerned. It began applying neural networks to what we now call chat bots.

Essentially, companies like Google started feeding massive amounts of text into neural networks, including Wikipedia articles, chat logs, digital books. These systems started to learn how to put language together in the way you and I put this language together.

The auto completion on my email, for example.

Absolutely, but taken up to an enormous scale.

As they fed more and more digital text into these systems, they learned to write like a human. This is what has resulted in chat bots like ChatGPT and Bard.

And what gave Geoff pause about all of this? Why was he so concerned?

Well, he still feels like these systems are not as powerful as the human brain. And they’re not.

But in other ways, he realizes they’re far more powerful.

More powerful how, exactly?

Geoff thinks about it like this.

You and I have a brain that can learn a certain amount of information. And after I learn that information, I can convey that to you. But that’s a slow process.

With these neural networks, Geoff points out, you can piece them together. A small network that can learn a little bit of information can be connected to all sorts of other neural networks that have learned from other parts of the internet. And those can be connected to still other neural networks that learn from additional parts.

They can all learn in tandem, and they can trade what they have learned with each other in an instant.

That’s what allows them to learn from the entire internet. You and I cannot do that individually, and we can’t do it collectively. Even if each of us learns a piece of the internet, we can’t trade what we have learned so easily with each other. But machines can. Machines can operate in ways that humans cannot.

[OMINOUS MUSIC]

So what does all this add up to for Geoff?

Well, in a sense, he sees this as a culmination of his 50 years of work. He always assumed that if you threw more data at these systems they would learn more and more. He didn’t think they would learn this much this quickly and become this powerful.

We’ll be right back.

OK. So what exactly is Geoff afraid of when he realizes that AI has this turbocharge capability?

There’s a wide range of things that he’s concerned about. At the small end of the scale are things like hallucinations and bias. Scientists talk about these systems hallucinating, meaning they make stuff up. If you ask a chat bot for a fact, it doesn’t always tell you the truth. And it can respond in ways that are biased against women and people of color.

But, as Geoff says, those issues are just a byproduct of the way chat bots mimic human behavior. We can fabulate. We can be biased. And he believes all that will soon be ironed out.

Where he starts to say that these systems get scary are, first and foremost, with the problem of disinformation.

These are systems that allow organizations, nation states, other bad actors, to spread disinformation at a scale and an efficiency that was not possible in the past.

They can also produce photorealistic images and videos.

Deepfakes.

Right.

He, like a lot of people, is worried that the internet will soon be flooded with fake text, fake images, and fake videos, to the point where we won’t be able to trust anything we see online. So that’s the short-term concern. Then there’s a concern in the medium-term, and that’s job loss. Today these systems tend to complement human workers. But he’s worried that, as these systems get more and more powerful, they will actually start replacing jobs in large numbers.

And what are some examples?

None too surprisingly, Geoff — a computer scientist — points to the example of computer programmers. These are systems that can write computer programs on their own.

Those programs are not perfect today. Programmers tend to use what they produce and incorporate the code into larger programs. But as time goes on, these systems will get better, and better, and better at doing a job that humans do today.

And you’re talking about jobs that aren’t really seen as being vulnerable because of tech up until this point. Right?

Exactly. The thinking for years was that artificial intelligence would replace blue collar jobs — that robots, physical robots, would do manufacturing jobs and sorting jobs in warehouses. But what we’re seeing is the rise of technology that can replace white collar workers, people that do office work.

Mm-hmm.

So that’s the medium-term. Then there are more long-term concerns. And let’s remember that, as these systems get more and more powerful, Geoff is increasingly concerned about how this technology will be used on the battlefield.

In an off-handed way he refers to this as robot soldiers.

Like, actually soldiers that are robots?

Yes, actually soldiers that are robots.

What he talks about is potentially this technology lowering the bar to entry for war — that it becomes easier for nation states to wage war.

So it’s kind of like drones. The people doing the killing are sitting in an office with a remote control, really far away from the people doing the dying.

No, it’s actually a step beyond that. It’s not people controlling the machines. It’s the machines making decisions on their own, increasingly. That is what Geoff is concerned about.

His concern is that as we give machines certain goals — as we ask them to do things for us — that in service of trying to reach those goals they will do things we don’t expect them to do.

So he’s worried about unintended consequences.

Unintended consequences. And this is where we start to venture into the realm of science fiction.

For decades, we’ve watched this play out in books and movies.

If anyone has seen Stanley Kubrick’s great film “2001”—

Mm-hmm.

We’ve watched the HAL 9000 spin outside the control of the people who created it.

That is a scenario, believe it or not, that Geoff is concerned about. And he is not alone.

Basically, robots taking over.

Exactly.

Whether these technologies are deployed on the battlefield or in an office or in a computer data center, Geoff is worried about humans ceding more and more control to these systems.

So this sounds pretty far fetched, honestly. But, OK, let’s play it out as if it wasn’t. What would be that doomsday scenario? Paint the picture for me.

Think about it in simple terms. If you ask a system to make money for you — which people, by the way, are already starting to do — can you use ChatGPT to make money on the stock market? As people do that, think of all the ways that you can make money. And think of all the ways that could go wrong. That is what he’s talking about.

Remember, these are machines. Machines are psychopaths. They don’t have emotions. They don’t have a moral compass. They do what you ask them to do. Make us money? OK, we’ll make you money. Perhaps you break into a computer system in order to steal that money.

If you own oil futures in Central Africa, perhaps you foment a revolution to increase the price of those futures to make money from it. Those are the kind of scenarios that Geoff and many other people I’ve talked to relate. What I should say at this point, though, is that this is hypothetical as we stand today. A system like ChatGPT is not going to destroy humanity.

[LAUGHS]:

Full stop.

Good.

And if you bring this up with a lot of experts in the field, they get angry that you even bring it up. And they point out that this is not possible today. And I really pushed Geoff on this.

So he’s basically arguing that this is a Pandora’s box, that it’s been opened, and that because people are people they’re going to want to use what’s inside of it. But I guess I’m wondering, I mean, much like you’re reflecting here — how much weight should we give to his warnings? Yes, he has a certain level of authority, Godfather of AI and all of that. But he has been surprised by its evolution in the past, and he might not be right.

Right. There are reasons to trust Geoff and there are reasons not to trust him. About five years ago, he predicted that all radiologists would be obsolete by now, and that is not the case. You cannot take everything he says at face value. I want to underscore that.

But you’ve got to remember, this is someone who lives in the future. He’s been living in the future since he was in his mid-20s. He saw then where these systems would go, and he was right. Now, once again, he’s looking into the future to see where these systems are headed. And he fears they’re headed to places that we don’t want them to go.

Cade, what steps does he suggest we take to make sure that these doomsday scenarios never happen?

Well, he doesn’t believe that people will just stop developing the technology.

This technology is being built by some of the biggest companies on Earth — public companies who are designed to make money. They are now in competition.

And he sees this continuing not just with companies, but with governments in other parts of the world.

So, in a way, it’s kind of like nuclear weapons. Right? We knew that they would destroy the world, yet we mounted an arms race to get them anyway.

Absolutely. He uses that analogy. Others in the field use that analogy. This is a powerful technology.

He wants to make sure we get the balance right between using this technology for good and using it for ill.

So, Cade, you’ve laid out a pretty complicated puzzle here. On the one hand, there’s this technology that works a lot differently, and perhaps a lot better, than one of its key inventors anticipated. But on the other hand, it’s a technology that’s also left this inventor and others worried about the future because of those very surprising and sudden evolutions. Did you ask Geoff if, looking back, he would have done anything differently?

I asked him that question multiple times.

It’s a complicated situation for him to be in.

He paraphrased the British philosopher Bertrand Russell —

— saying that you can make a wise decision that still turns out to be unfortunate. And that’s basically how he feels.

It’s reminding me, Cade, of Andrei Sakharov who was, of course, the Soviet scientist who invented the hydrogen bomb, and witnessed his invention, and became horrified, and spent the rest of his life trying to fight against it. Do you see him that way?

I do.

[MUSIC PLAYING]

He’s someone who has helped build a powerful technology. And now he is extremely concerned about the consequences. Even if you think the doomsday scenario is ridiculous or implausible, there are so many other possible outcomes that Geoff points to. And that is reason enough to be concerned.

Cade, thank you for coming on the show.

Glad to be here.

We’ll be right back.

[MUSIC PLAYING] Here’s what else you should know today. After a marathon set of crisis talks, President Biden and House Speaker Kevin McCarthy reached an agreement on Saturday Night to lift the government’s debt limit for two years — enough to get it past the next presidential election. The agreement still needs to pass Congress. And both McCarthy and Democratic leaders spent the rest of the weekend making an all-out sales pitch to members of their own parties. The House plans to consider the agreement on Wednesday, less than a week before the June 5 deadline, when the government will no longer be able to pay its bills.

And in Turkey on Sunday, President Recep Tayyip Erdogan beat back the greatest political challenge of his career, securing victory in a presidential runoff that granted him five more years in power. Erdogan, a Mercurial leader who has vexed his Western allies while tightening his grip on the Turkish state, will deepen his conservative imprint on Turkish society in what will be, at the end of this term, a quarter century in power.

Today’s episode was produced by Stella Tan, Rikki Novetsky, and Luke Vander Ploeg, with help from Mary Wilson. It was edited by Michael Benoist, with help from Anita Badejo and Lisa Chow, contains original music by Marion Lozano, Dan Powell, Rowan Niemisto, and Elisheba Ittoop, and was engineered by Chris Wood. Our theme music is by Jim Brunberg and Ben Landsverk of Wonderly.

That’s it for “The Daily.” I’m Sabrina Tavernise. See you tomorrow.