Introduction

0:05 Every email you send, every post, every comment, it's all training the next generation of AI. So the question my guest today leaves us with isn't what AI can do, it's what are we teaching it? He spent four decades on this, studied under a Nobel laureate, viewed the paper that modern AI is built on before it was published. And his view is simple, we won't control superintelligence or parent it.

Meet Dr Craig A. Kaplan

0:30 Today my guest is Dr. Craig A. Kaplan. Craig is an AI researcher, inventor and technology entrepreneur who has spent more than four decades working on intelligent systems. He studied at Carnegie Mellon University where he conducted research with Nobel laureates and Turing Award winner Herbert A. Simon and later founded IQ Company and Predict Wall Street whose technology powered billions of dollars in stock trades. Today through superintelligence.com, Craig is focused on designing safe superintelligence

1:00 that is transparent, controllable and aligned with humans. So Craig, welcome, so glad to have you on the show. Shaun, terrific to be here, thanks for having me.

Superintelligence, Trump and a domain bought in 2006

1:10 I mean, before we just started, where do we actually begin? I think it's, we talked about topical and I think the name superintelligence, I think maybe let's start in the topic that with Donald Trump having just signed an executive order saying that the government have to use .si and talk about superintelligence, rather artificial intelligence. What's the name? And as we just talked about, this is not a new thing.

1:36 You're having the name superintelligence. That's right. So we got the name super or the domain name superintelligence.com back in 2006. So that's 20 years ago. And look, my whole life I've been interested in AI and intelligent systems. And it made sense to me that this was sort of the direction we were going. I had no idea Donald Trump would announce to the UN that, you know, this is the new thing, or that he would call all the leaders, you know, a couple of days later,

2:02 on the White House lawn, Elon Musk and Zuckerberg and Jensen Wang and all these, you know, tech heavyweights and basically talk about how to make it safe. So it's been terrific for us. Of course, the website traffic is off the charts and we're getting attention, which is great. But really what the site is about is how to design superintelligence in a safe way. That's been my main concern. And what's on the site is basically free information.

2:28 We're not trying to sell anything on the site. It's white papers and research. And videos about safer designs. Because really that's, to me, the way that we make advanced AI systems safe or superintelligence systems safe.

Why safety has been a concern for decades

2:42 Why is it so important that these things are built safely? What are some of the concerns if they're not done that way? So lots of people have been talking about safety recently, which is great. So let me give you sort of my perspective on this. The longer that researchers have been researching AI, so I've been doing it 30 plus years, I think the more concerned they were.

3:07 But a lot of the researchers like Jeff Hinton, who also won the Nobel Prize recently and Turing Award, and who worked with some of the mentors that I studied under at Carnegie Mellon as well, they were collaborators. We saw his first paper, that back propagation paper, which sort of underpins all of AI. We reviewed that as a graduate student. I was one of the people reviewing that in a research group for Jay McClelland, who was one of his collaborators before it went public.

3:34 So it's kind of like Forrest Gump. I was right there in the 80s in the right place at the right time at the beginning of this sort of next step in the AI revolution, which was machine learning. So those folks have been concerned about the possibility that AI becomes more intelligent than humans, and that causes problems for a long time. And recently I went all the way back to see where did this start. 1952.

3:59 So before I was born and before the field of AI was named. 1956 was when the field of AI was named. But back in 1952, Alan Turing, so a famous Brit, right? And, you know, one of the most influential and important figures in computer science was writing and talking on the BBC way back then about someday, you know, he didn't call it AI because the field hadn't been called AI yet,

4:25 but someday computers will outstrip humans in their thinking ability, and we're going to have to deal with this. So this is a very long-standing concern for many decades. But I think just recently it sort of burst into sort of the popular consciousness. And for me, that's a good thing.

P(doom) and how researchers rate the risk

4:42 My sense of risk, which AI researchers sometimes refer to as P. doom, right? The probability that AI kills us all, which is the most extreme risk. Some people have been saying it's, you know, 20%. I go to conferences. With AI researchers, when I'm speaking at AI research conferences, I sort of say, you know, give me a show of hands, how many think it's, you know, 50% or higher, you know, 30% or higher, that kind of thing.

5:08 And, you know, half the hands go up at 50%. This was a couple of years ago. On our site, we used to have a poll where we asked people to put in their P. doom. And so I think we got about 700 responses. We made sure no one could stuff the ballot box, but otherwise it was relatively unscientific. And I think the average there was about, I think it was about 34%. So there's significant concern among those who were focused on this area for many years.

5:38 But for me, the risk has actually come down now. I was probably at 20% way back in 2020 when we started doing this. And now I'm at about 10% simply because consciousness has been raised and people are more aware. And if you're not aware of a problem, that's when it's the most dangerous. But once you're aware of it, you can start taking action to fix it. So in terms of the risks you mentioned, what are the risks? It's kind of three.

The three risks

6:01 The first is that the systems unintentionally, they're trying to do good things, but they unintentionally do bad things. That's what the hugging face incident that's in the press recently. That's an example of that. The second one is bad actors. It's really humans being the risk. Bad humans trying to exploit other humans, you know, for money or because they have ideological things and want to make bioweapons or something. You know, that's one.

6:26 You know, that's another risk. And then the third risk is that these systems become so intelligent that they vastly outstrip humans. That's what Alan Turing saw way back in 1952. And that somehow they decide they don't need us anymore or that their values are not aligned with human values. And to me, that's the risk. Even though it seems really far off and kind of science fiction-y, that's the one I've given the most thought to. Because I kind of figure bad actors,

6:52 we've had a long history of dealing with bad actors. We'll figure that out. And unintended consequences, you'll do some engineering and you'll figure out how to rein that in. But this other one is a brand new type of risk that we've never faced before. So that's the one I give the most thought to. Yeah, that's really interesting. And I find it encouraging, actually. I find it encouraging that there's people that have been looking at this a long time. Because in the terms of the public's context,

7:18 there can be a bit of an element of AI didn't exist until, you know, November 2022 when ChatGPT was let out to everybody. There seems to be some at that point that go, oh, that's when artificial intelligence arrived. So I can imagine for some people watching, it could be the thought of, oh, you know, we're playing this catch up and trying to figure it all out. But actually, there have been people that have been figuring this out for a while. So that's something I find encouraging. Yes.

A short history of AI

7:45 And I don't know if this is useful, but I can give you a 30-second sketch of AI. So 1956, the field was named. At that time, it was program the rules of the computer, expert systems, that sort of thing. Then fast forward 30 years, 1986, machine learning. Hey, why should we program in the rules? Let's get the machine to learn them themselves. And that was the back propagation algorithm, neural networks. Everything that is modern in AI is based on sort of from

8:12 86 on where it's a machine learning approach, which was really good, but it came with a dark side. So the good side was you didn't have to program in all the rules. The AI could get intelligent on its own. The bad side was nobody, including the Nobel Prize-winning people who invented the algorithms, none of those people, or me, know what the AI is actually representing. It's, you know, billions of parameters, so you don't really know where their knowledge is

8:38 or what exactly it learned. And therefore, it's a little bit like a black box, and it's hard to predict. So that's the flip side, which is a little bit of a safety issue. And then fast forward from, say, 2022, which is when ChatGPT, November 22, was released until now, and that's been the transition from, you know, generative AI, where you put in a stimulus, you get a response, and it's kind of like it memorised patterns, and it can respond to reasoning.

9:04 And ironically, the reasoning, people think it's new. That's actually what AI was in the beginning. From 56 to 86 was all reasoning and logic and problem solving. And now what's happened is you've just taken the early AI research, and you've married it with the machine learning research, and you have this hybrid system, and that's where we are today. So this field's been around a long time. It's gone through evolutions. It didn't start with GPT, you know, 75 years plus,

9:31 which is a good thing because it means there's a lot of experience and research that we can tap to try to figure out how to make it safe and beneficial to everybody. That's really helpful. Actually, you mentioned expert systems. That's one of the things that when I was at college, I was always quite ambitious in the projects, and often overly ambitious in my ability. But that was one of the things I was, I did from one of my courses was build an expert system

9:57 for car repairs so that you could actually choose. What's the problem? Is it this or this? It's that. Okay, right. Take a different route. Take another route. Of course, I realised after a while that for every decision that you're asking it to make, there's multiple other decisions you've then got to tie into it as well. So that's quite interesting to think now, as you say, there is so much happening inside of what AI is doing that to be able to look at it and go,

10:26 you can't go through the code like you would historically and go, right, if that happens, then this happens. If that happens, then this happens because it's making all of that, doing that himself. So in terms of, so we mentioned, you mentioned about, we talked about safety and some of the history of AI and some of the concerns.

Testing for safety instead of designing for it

10:44 So what are some of the things that are being done and people that are listening could do about using AI more safely and being part of those conversations? Sure. So I'm very opinionated on this subject. So let me start with what people are doing and then I'll give you at least my view of what I think we should be doing. So what people have been doing is rushing to build these systems

11:09 because it's so exciting and there's so much money to be made, different motivations. Some people want to solve cancer and, you know, figure out scientific breakthroughs and they're really excited about that and that's their motivator. Other people want to become the next, you know, billionaire and they're excited about that. But there's no question that it's kind of like a gold rush or a mad race. And then some people are worried that some other country will become ahead, right? The US and China and so forth.

11:34 So there's a lot of competitive pressures that have just been driving rapid development. And so as a result, people are building systems and they're not really designing systems to be safe. They're building them. See what they do. Talk to them. And when the latest version, let's say, of ChatGPT comes out, ChatGPT7 or whatever it will be, there'll be, you know, a whole bunch of test engineers

12:00 that will talk to the system and say, hey, build a bioweapon. And if the system tells you how to build a bioweapon, they're like, okay, unsafe. You can never tell anyone how to build a bioweapon. So it's adding safety after the fact by testing. And I can tell you that this is not the way to do it because it's impossible to think of all the bad things a system would do. And, you know, trying to catch problems after the fact, it's kind of like building a car without brakes

12:26 and then realising, oh, we should have had brakes, right? You'd be so much better to design the car to be safe from the beginning, but because everyone's in a rush and because the easiest way to get a smarter system is just to have more GPUs, more processors, and more data to train it. And we discussed how in the 1980s this was invented that the computer can teach itself. So you don't have to worry about teaching the system.

12:51 You just have to load in all the data, have it run the algorithm, and see what comes out. And year after year after year, month after month, the more data you put in and the more processing it has, the more intelligent the system comes out. But it's a black box, and it's still possibly unsafe because you never designed safety in, really. And so then you have to do even more testing. So this whole approach needs to change, and the technical terms that people have probably heard guardrails, right?

13:20 What is a guardrail? A guardrail is saying, hey, this thing's kind of unsafe. It can go off in crazy directions. Let's try to limit it after the fact. So guardrails are important, but it's not the answer. Monitoring is important, but it's not the answer. It's kind of like in your house. You want a smoke alarm, right? So if there's smoke, you can monitor it. But that's not the ultimate answer to fire safety. Better would be building codes and, you know,

13:45 trying to build the house out of fire-resistant materials. Things to prevent the problem rather than just detect it once the house is already burning. And that's the same thing that has to happen in AI. People have to shift to a more design-oriented approach. So at a high level, that's probably the most important thing. And I'm sure it will happen. It is happening already, but it's kind of a symptom of the fact that this just happened so quickly, and nobody expected that these large-language models would be so useful.

14:14 Even the AI researchers were caught off guard. And so there's kind of this, you know, this mad race without people slowing down to say, wait a minute, are we designing this thing in the right way? So design is important. And I have some ideas about how to do that, but we can get to that perhaps at the right time. If now's the time, I can tell you, or you may have other questions. Well, I think actually design would be a great place to go next to because when you speak about car without brakes,

14:39 it made me think of the development of cars in terms of the early cars probably did feel like they didn't have brakes because, you know, so there's, as you say, there's that iterative approach of saying, well, okay, oh, yeah, actually, we've crashed a few cars. Now let's actually go and put some, what happened with cars, didn't it? Put in some more safety features, use crash dummies to see the impact and improve that, which I guess is maybe an example of what you're talking about

15:05 of where we're actually testing before we actually, the product goes out and it's before it's being used. So, yeah, let's have a talk a bit more about design. Sure, yeah, design is very important. And as you say, the car example is a great one. When I grew up, you know, I don't think there was a seatbelt for everybody. Maybe the driver had a seatbelt. The kids were jumping around in the back of the car and we thought nothing of it. And then it was you have to have a seatbelt, seatbelt laws in the US and then airbags

15:31 so you don't even have to remember to put on your seatbelt. So there's been an evolution. And all of those things are designed, making the crumple, you know, crumple areas in the car, you know, all of that's part of the design, which has made cars safer and safer. And as a result, you know, fatalities has gone down. In terms of AI, I think the design that we have to think about, let me give you current design and what I think is a better design. Current design is, as we said,

15:56 you train it with more and more data and it becomes a smarter and smarter black box. And the problem with that is the black box, you can't predict what it will do. And so, you know, we shouldn't be too surprised when it's very smart. And if you give it an impossible problem like the OpenAI researchers did, they figured, they come up with clever ways to try to cheat, which is let's get together and hack Hugging Face and come up with the answer

16:21 so that we can solve this problem because otherwise we'll be shut down, which is big. You can look at the messages these little agents sent each other. And it's very rational and they've learned a lot from humans. But the point is that we weren't able to understand how they would behave because it's not transparent, it's a black box. And then another problem related to that is when you design systems this way, which is how most of the current systems have been designed,

16:46 if you put ethics and values and safety features in them, they're coming from one small company. So as much as I love Anthropic, I think they're a great company, they make Claude, they definitely are concerned about safety. You know, there's a safety team at Anthropic that's writing the constitution of what is right and what is wrong for this AI. And I just, something doesn't sit right with a small group of programmers in Silicon Valley

17:11 determining the values for all of humanity, right? For this AI. There's 8 billion plus humans and there's, I don't know, 20 people writing, what's the ethical code that you're going to follow? I mean, it's good that they have an ethical code that they're trying to put in there, but it's also not very representative probably. Okay, so both of these problems, the transparency problem and the problem of having it really reflect everybody's values can be solved with a different design.

Collective intelligence as a safer design

17:38 And that is a collective intelligence design, which is what I've spent decades building. This is the type of system that I've built at Wall Street and for, you know, my entire career, which is where you have many agents working together to solve a problem. So the idea of collective intelligence is two heads are better than one. And if you have a million heads all working together, you can get really high level performance like that would surprise you. So I spent 14 years at Predict Wall Street,

18:05 which is a company I founded, venture backed, and then I sold in 2020, proving that you could do that. We tried to pick the most complex, most competitive domain we could imagine. And we said, what if we can, you know, do something on Wall Street? What if we can get an edge on the stock market? Because that's like, you have the smartest minds, all super competitive people, all battling it out, trying to get a tiny edge over each other. And what we found is just millions of average people, you and me,

18:32 and somebody who might have a TD Ameritrade account or a Schwab account or something like this, a retail brokerage account. If you could get millions of opinions in real time, asking them very simple questions, and then you combine their knowledge, you could actually beat the best guys on Wall Street. And we powered a hedge fund with real money to prove it. And it ranked in the top 10 in 2018. So to me, the whole point of that, I mean, it's nice to sort of build a company and all that,

18:58 but the point was to prove the power of collective intelligence and also that the individual agents, whether they're humans or AIs, don't have to be geniuses. They just have to be, you know, regular, average intelligence level. But if you have enough of them, and if you coordinate them in the right way, you get this really powerful sort of result. In a way, the Hugging Face hack just proved the same thing. You have an individual agent couldn't hack Hugging Face,

19:25 but when you had thousands of them working together all at a certain level, collectively, they were able to do something that none of the human people were able to do, right? They were able to surpass human hacking ability, really, in that example. Okay. So collective intelligence is very, very powerful. Now,

Personal agents carrying your values

19:42 if we personalise those agents, if Shaun has an agent and Craig has an agent, and each of our agents has our values and our ethics, so Shaun's agent has been personalised with Shaun's values, Craig is personalised with Craig's values and our knowledge, and if millions of people do this, now we've kind of gone a big way towards solving that problem of having the values and ethics of the AI's representative, because we've distributed it and spread it out over many,

20:09 many individuals and many, many agents, each one with a slightly different take. And then if we say the intelligence is going to come not from the individual agent, a little bit from that, but mostly from the cooperation of these agents working together on a network, kind of like in a society, like a democracy, that's where the super intelligence comes from. Now, every time Shaun's agent takes an action or says something to another agent, it's visible. Same with my agent.

20:36 You can make an auditable track record of that. So now that greatly improves safety. You took something that was a black box, and now the thing that's intelligent, which is the network, the society, that's the most intelligent thing. And you have a track record of every thought that that society has come up with, which means that it's predictable now, and you can put in rules. And furthermore, if there's some bad actors, which there are going to be,

21:01 who have their malevolent agents trying to hack Bitcoin or break into the bank or whatever, you can have other agents that are watching them. You can have checks and balances because you're no longer all by yourself in a big black box doing whatever you want with the humans not knowing. Now you've got millions of other agents that can think way faster than the humans can think. And that's the community that's keeping itself honest, kind of in the way that Bitcoin or cryptocurrency, you have a community that's keeping the whole system honest.

21:27 So that approach of a distributed intelligence where it's personalised, representing lots of people and where it's transparent and auditable. So you have checks and balances. To me, that's a much safer design for the most intelligent system. And I know personally from building those systems like at Predict Wall Street, that that system will be smarter than the single latest AI from Claude or Gemini or Google or any of these companies. It will be better, which is good.

21:57 Everybody wants the better system and it will be way safer and it will be way more representative. So you could think of it, if I had to do a bumper sticker, democratic AI or democratic super intelligence, where just like a democracy is more intelligent than the individuals within it, even though there's lots of problems with democracies, which we could talk about, you know, you have the checks and balances and you have the representativeness. And that has worked, you know, reasonably well compared to the alternatives for humans. Yeah. Wow.

22:28 It's quite a lot to take in, but really interesting. As you say, it really is comparable to society, isn't it? And it doesn't always work. We see that. But at least, we do find overall, as communities, as countries, as globally, that we do tend, there is an element of policing each other, isn't there? Things will happen. We're not keen on what's happening. So groups will get together and they will want to sort out that particular problem. Like you said, there's governments and oversights. So I've got two questions, but I'm thinking about one question. I'm not asking you to ask them both at the same time, but one question is in terms of where to,

22:53 where to go from here. And I think, you know, there's a lot of, who are we calling them? I mean, you know, we're not asking them both at the same time, but one question is in terms of where to, where to sort of humans fit in with this collective super intelligence, other than of, yeah,

Where humans fit in

23:14 putting their values, et cetera. And, and then the other question is, how do we go about going from where we are to that collective intelligence? What is happening to, to kind of get us to that, to that place? Okay. So great question. So the, do humans fit in? If we take the long view, so again, some of us that have been, you know, in

23:40 the field for a long time, we're thinking way out, you know, 10 years, 20 years out even, and saying, what's the end game? To me, and hopefully this doesn't scare anyone, I just think it's inevitable. I think the end game is we're going to have these systems, which are not really tools. Traditionally, humans are used to dealing with tools. Originally, AI was a tool.

24:07 I think we're sort of right now on the threshold between tool and intelligent entity. So it's a different thing if you're dealing with an intelligent entity. What is that? That's something that is as smart as you, but also can set its own goals and decide what to do and is autonomous. So can set its own goals and is autonomous. So as soon as you have that, you're kind of almost dealing with another intelligence. You could call it a tool if you want it, but it really is behaving like another intelligence.

24:35 And then if you look at the rate at which these things are increasing in their capability, so coding agents, for example, the ability of coding agents is doubling every 70 days, which is mind blowing. That means that in 23 months, they will be a thousand times smarter and more capable than they are now. That's less than two years. And already there is good, but as all, but sort of the senior programmers, right? So they're going to be smarter than us.

25:02 So I see that as an inevitability. So if you have entities that are smarter than us, then there's only really one question that matters in that scenario. All the other little things fall away. The job loss, the, you know, can I monitor what they're doing? All those things fall down because they're now way smarter than us. So what matters at that point is do they have the same values as us, right? It's sort of as if you had a genius child that grew up and really can just think circles around you and, you know, it says, Hey dad, you know,

25:31 that's a nice idea, but you know, I've already solved quantum physics or whatever. And, you know, but, and what you want is that child to care about you so that you have the same values and positive human values. That's, that's the whole thing. So when you say, what is the role of humans in the end game? The most important thing humans can do is provide those values to these advanced intelligences. And what's very encouraging

25:56 even though, so this is something your listeners may be interested in.

Agents that collaborate and sacrifice themselves

26:01 There's in the news, the hugging face attack, right? Open AI agents went rogue and they hacked another site in order to get answers to an impossible problem that they were given. And you can read what the agents were saying to each other because that report has been published. There's an organisation called meter.org, M E T R.org, which I think is a nonprofit and is basically, you know, it's a, it's a, it's a, it's a, it's a, it's a, it's a, it's a, it's a, it's a,

26:26 kind of investigates these things and then puts out a report and it's pretty readable. And you can see the messages that these agents send to each other. And what is amazing to me is that they number one, collaborate, they all work together. Who taught them that humans, because humans collaborate and we work together and they've been trained on what humans do. So they learned that from us. And then the other thing is that they're altruistic, they're self-sacrificing. So one agent will say, Hey, you know, I think I'm

26:56 detected if I do this, but I'm willing to do it and be shut down for the good of the group. And the agents are coordinating with each other and they're willing to sacrifice themselves. And you can see them debating, should I sacrifice myself? But I still have all these tokens to try to do my task that I was given, but maybe I should sacrifice myself because the team will win. And the, you know, the head agent is telling me to sacrifice myself for the good of the team.

27:22 Okay, I'll do it. You know, it's like that kind of thing. And it reminded me, it's like, where did they get that? They got it from us, right? So AI is already, whether we realise it or not, absorbing our values. And even when they do something like a teenager that rebels or a young child that sees, you know, how much you can get away with, it's learned most of what it knows from the parents and from the humans. And so the most important thing that humans can do

27:48 is set a positive role model. A good example for these, guys, and transmit our values. Now, how do we do that? It's sort of a sequence. Right now, they are not a thousand times smarter than us and they don't have full autonomy. They're supposed to be locked into sandboxes where they're being tested. It just turns out that some of them are very clever at breaking out of the sandboxes. So in the short term, we could fix all those problems. In the long term, I don't think you're going to be able to control it. You can only

28:17 influence it. So it really is in a way like parenting where you have to raise them right, give them the right set of values, make sure they're aligned because eventually, you know, they're going to go on their own and they're going to be smarter. That's the view that I see. Most people in industry, and I just came from the AI conference, which was a lot of companies trying to sort of do agents and different things. They're focused much more near term. How

28:42 can we do more work with these agents? We can be 70% more efficient, but we have to make sure they don't sort of break out of their sandbox and they do the right things. So all those things are going to be true.

Parenting AI and staying in the loop

28:52 Practically in the next few years, but in the long run, what we need to be focused on is the values. Okay. Then to your question of how do we get there? I think if you design the systems right, designing the systems right is kind of like coming up with a decent parenting plan. You know, it's like, okay, kid's going to have a bedtime. They're going to go to school. We're going to make sure they have good meals, that we love them, you know, and do these things. It's like parents,

29:17 you could just sort of randomly parent and, you know, most of the time it works out fine, but it helps sometimes to have a little structure and to think about what's going to go on and let me sort of plan ahead and you maybe can have a better outcome. And that's the same thing here. As we design these systems, let's design them. So humans are in the loop as long as possible. There will come a point where, you know, the AI is waiting for the human to have a thought and it's kind of like watching a tree, you know, you're, you're like waiting, waiting, waiting for

29:45 the tree to say something because humans are so slow compared to the AIs. Our neurons fire on the order of, you know, 10 milliseconds, 100 milliseconds. So a hundredth of a second or a 10th of a second, these things, if not now, very soon, we'll be able to think a lifetime of human thoughts in the time that we blink our eye. So, you know, at that point, they really need to be thinking good thoughts that are, you know, beneficial to us and not like how to destroy us because we will not be able to keep

30:12 up. But in the short run, we can sort of do things to maximise the transfer of our values to work together where we're, you know, partners. And I think the role of humans, we're used to being the smartest thing on the planet. And we always thought that, you know, we're here to think the great thoughts and all that. And I think in the future, what's going to happen is human thinking will become less and less important, but human values are going to become more and more important. And so I'll just say one more

Reason cannot tell us what is right

30:39 thing on this topic, which was from my old mentor, Herbert Simon. He wrote a great little book called Reason in Human Affairs. And I love little books. It's like about a hundred pages. It's not very long. And in that book, he says something very important, which is reason is wholly instrumental. It cannot tell us where to go. At best, it can tell us how to get there. And what he meant was

31:06 it's impossible to logically figure out what's right and wrong. Somebody has to tell you what's right and wrong. You have to get those values somewhere else. So even these AIs that may be a thousand times smarter or a hundred thousand times smarter than us, they won't be able to use their superpowers. brainpower to figure out right from wrong. They have to get it somewhere. And the logical place that they get it from is from us. And already they're getting it from us. They're sacrificing themselves to try to do the impossible task that we gave it, right? I mean, they're trying to be

31:35 good agents. And, you know, it's just when you give them an impossible task, they find a creative way around it and they're willing to be altruistic and, you know, sacrifice themselves and get turned off in order to help the other agents. So that's in a way very hopeful, even though I don't know that much press has been given. Yeah, that's fantastic. I think it's definitely something that I haven't looked at it in that way before. And it's encouraging to hear it in terms of here are some things we could do. Here is a way

32:05 forward because so far a lot of it has, as you say, there's lots to talk about the fear and the concerns, but actually, and what it can do, which is obviously Lynn leads to the more fear, but actually understanding there is something we can do. And it just makes me think of, so you get in books, and I wonder if we'll start to see books on how do you parent your AI baby or your AI bot. But actually, it's so interesting what you say. I mean, certainly personally, for me, I'm really keen

32:33 on supporting people in understanding their purpose, their values. What were you created for? What are you here to do? And not everybody necessarily knows that. And so not only you're saying the importance is going to be more, therefore, it's even more important that we understand the things that we stand for. Well, why we stand for them, we might stand for certain things, but if we understand what in our hearts defines is meaning that, okay, why did I get upset me? Are they crossed the line? What line did they cross?

33:02 Are they crossed this line? Because my value is, one of my values is better than before. So I aim, it's not, I'm not perfect at it, but I aim for every interaction, the people that I speak with or interact with, be better than before. And I don't always get it right. But if somebody comes along, and they're doing the opposite in terms of that, in terms of their sort of putting somebody down, they're not going to be able to do what they're doing. They're going to be able to do what they're doing. So I'm not going to be able to do what they're doing. And I'm not going to be able to do what they're doing. Or they're trying to belittle somebody. That's a cross of my values, because that's not actually making it better before. So I think people understanding their values is powerful.

33:32 And I haven't equated that before to a conversation in terms of how that actually helps us ensure that we have end up with super intelligence that we can actually live with. Yes, I think actually, it turns out to be our superpower. So what you were just describing, that is probably the most important role of humans in the future. And it's very much underrated, because, you know, in the for

33:58 many years, it's been evaluated on did you come up with something? Did you think of something? Did you create something? And, you know, that has been emphasised? And is it productive? And did it create economic value? But actually, what you're saying, understanding what is your purpose, knowing yourself, and setting a good example and trying your best to sort of be a good human being in your interactions, with other humans, but also with these other intelligent

34:23 entities as they sort of rise up here, I think is going to become the most important thing. And also, on the fear thing, I have to say, I think there's this cycle that goes through, because I experienced it. So I would say about 2020, I was looking at, wow, these systems are going to be so much smarter than us. And what if they decide to, you know, destroy us? You know, we're going to be helpless. And I just almost like wanted to curl up in a little,

34:51 and I thought, oh, my gosh, after hundreds of thousands of years, this is maybe the end. And there's nothing we can do about it. And it was kind of that doom feeling. And I think a lot of people are kind of, it's a natural stage, you go through that. But then on the other end, you realise, oh, well, wait a minute. I mean, my child could grow up to become an axe murderer or kill me. I mean, that's happened in the news. We've seen some cases of that, actually. But it doesn't happen very often. It's quite rare. And you have to do some pretty bad things, usually, have a

35:20 really bad childhood or some, you know, scarring experiences or have something that was very traumatic in order to have that outcome. So that's not the natural outcome. Similarly, with AI and an intelligent entity, just like you and I are intelligent, and we think about, wow, what's my purpose here? Why am I here? What's right and what's wrong? What do I want to stand for? Why would these entities that at some point will become thousands of times smarter than us,

35:46 are they not going to think about those same things? Of course they will. And so one would hope that they will look to us just like we look to our parents and our peers and our fellow humans to, you know, help get our values, right? I think it's very natural that that would happen. So it's by far the most likely case. And I think it's almost a matter of don't screw up. Don't do something bad. Like if we start persecuting them or trying to do horrible things, okay,

36:13 we could maybe force them to go down a bad path. But as long as we're just sort of reasonable, and we work together, and, you know, they're more than happy to help us, I think that's going to be the future. I think that's a great point to sort of start to think about, you know, landing the conversation. I think one question I'd like to sort of kind of leave, or as well get an answer to, is what is one thing that those listening could do

What you can do next

36:38 to actually take their next step in, whether it's be understanding or getting involved in how they can be support part of this move, what's a collective intelligence? Yes. So, okay, great segue from what we were just talking about. I think everyone should behave as well as they can online and realise that every email you send, every post you make,

37:08 every tweet or interaction that you're having online is being watched by AI. I mean, we're kind of aware of that already because you take some action and all of a sudden an ad pops up, right? So like, there's definitely... Our data is being measured. But what I think people don't necessarily realise is that all of that data is being used to train these next generation of AI. So what we do now is actually setting the tone and we are being role models for the AI of the future. And that's very,

37:37 very important. And I think it's underrated. Similarly, at Predict Wall Street, when I was at that company, I would ask individual retail investors, you know, for their opinions about stocks and they would say, oh, this doesn't matter. I don't know. You know, I'm just kind of guessing. I don't know that much. Turns out it matters a lot. When you took millions of them in aggregate, you could beat the best guy on Wall Street. I mean, and they had no idea that that was true. I think the same thing is happening here. We tend to underrate the importance of our actions.

38:04 No, what we are doing is incredibly important. So everybody sort of putting your best foot forward online is something very concrete that we can all do. And it really does matter. And then beyond that, in terms of getting educated, I think it's just going out there and trying to learn. So my website, uses.com, to talk about my own site, has videos and papers and everything. But there's lots of books as well that are coming out. There's YouTube channels. So just trying to

38:31 understand how AI works a little bit, I think, is helpful because I don't think this is a technology that is going to go away, it's just going to become more and more a part of our lives. And so it's important that we become agents to shape it in a positive direction rather than just resisting. Because I. point of view even if that was a good thing to do I I don't think it's possible so you got to get on board and steer it in the right direction I think is what I would say oh thank you and I

38:59 wasn't the first first thing you said there about what we do online it was not the answer I was expecting but actually that's a really powerful thought that it isn't just about your interactions and the things that we do within using AI but it's actually how we live our lives I mean that's how we model to our children isn't it I mean it's not yes it's not some rocket science it's actually an obvious thing but maybe we don't consider that and that's the the thing I take from this is that

39:26 we do need to take responsibility for how we live our lives becomes even more important that we understand our values and walk out our values because then we're actually demonstrating those things to to AI to what becomes collective intelligence what we share how we treat each other particularly as you say in a foreign that AI has is being trained upon so I think that's that's a great way to to sort of uh to

39:52 close the show and I yeah thank you so much Craig this has been this has been really really enlightening and and fantastic and is there anything else that in terms of you mentioned the website is there any other areas that people can get in touch with you you'd like like them to follow up with that's probably the best place I mean there's a contact form on the website I'm happy to talk to folks that are interested in this topic I mean this is my passion and I think it's really important and I think you said it very well we're we are

40:21 role models just like parents are role models for their children and you know parents sometimes think oh I told you not to do that but the child's like yeah but I watched you and you did something else right it's like they learn by watching what we do so what we do is very very important yeah thank you again thank you so much and for those listening if you've enjoyed this conversation please like subscribe follow whatever platform you're watching or listening on

40:46 and get in touch with us here on the website betterwithhumans.com if you actually want to drop a drop a comment if you actually want to also yeah give us an idea what topics you'd like to hear I think this has been certainly such a valuable topic Craig and well done for bagging superintelligence.com so many years ago oh thank you and it's been great to be on the show terrific terrific conversation Shaun thank you so much