1 If AI is possible, how should we prepare for it?

In this section you will explore the idea that we should accept that AI technology (as described in the Introduction) is possible, and that we should prepare for the use of this technology in a way that maximises the benefits while avoiding the dangers it could bring. Such concerns have been voiced by leading figures in the field of AI, in particular, Stuart Russell.

Figure 2

Show description|Hide description

Illustration of two heads in profile, facing in opposite directions. The head looking to the left is designed to look like a circuit board. The head looking to the right is shaded black. The area where the heads overlap is white.

Figure 2

Russell co-authored with Peter Norvig perhaps the most widely used textbook in AI, and both have done pioneering work in the field. More recently, Russell has called for a rethink in how AI is being adopted. Russell asks us to suppose for a moment, that we did in fact succeed in building AI that had sufficient intelligence to carry out a wide range of tasks normally undertaken by humans. Two questions follow from this:

• What are the potential benefits and dangers of such a scenario?
• How should we prepare for this?

The next activity examines Russell’s ideas in more detail.

Activity 1

Timing: 15 minutes

Part 1

Let’s start by watching Stuart Russell’s proposal in this video.

Download this video clip.Video player: lg003_2022b_vwr005_320x176.mp4

Show transcript|Hide transcript

Transcript

STUART RUSSELL:

So the first question is what is a good decision anyway? An economist will tell you that it means maximising your expected over the whole future utility. And this applies to everything from lottery tickets to Davos meetings to building radio telescopes.

On maximising, AI has made a great deal of progress. 18 years ago, Deep Blue beat Garry Kasparov at chess. Just last week, the game of poker was solved perfectly – bluffing and all – and humans could no longer compete. And right now, the DeepMind system is playing 29 different video games, super humanly well, that it learned entirely from scratch, just by watching the screen. Imagine if a newborn baby did that.

On expectations, these depend on perception and learning, again, a huge amount of progress. The Watson system extracting information from text. Cars watching the world as they go by. Learning algorithms that classify images and write descriptions. Even a system that discovers the concept of a cat entirely for itself just by looking at millions of images of everything under the sun.

Now, a lot of this progress comes from mathematical ideas. Here are just a few of the equations from my undergraduate course. And there will be a test if Linda allows some time later on.

There’s also a lot of progress that comes from commercial investments. So every one of these areas, a 1 per cent improvement is worth billions of dollars. So we may see in the future domestic robots, for example. Search engines that read and understand every page on the web. Even a machine that will discover the missing sock, perhaps in the very distant future.

So the point of AI is that everything civilisation has to offer is the product of our intelligence. So if we can amplify that, then there is no limit to where the human race can go. But I actually want to point to a problem. And that comes in the utility part of the equation. So imagine, for example, that you ask your robot to maybe make yourself some paper clips that you might need. And your robot’s very, very clever. It takes you very literally, and pretty soon the entire world is 6 feet deep in paper clips.

So this is The Sorcerer’s Apprentice and King Midas, all rolled into one. Now, technically, what happens is that if you ask a machine to optimise, and you leave out part of your preferences, the machine will set those elements to an extreme value. For example, if you say, ‘Google Car, quick, take me to Zurich airport’, it will max out the speedometer. And then you say, oh, I didn’t mean break the speed limit. Well, it’ll still put its foot on the gas. And then, when it gets to the airport, slam on the brakes.

So this is the problem of value alignment. And if you combine misalignment of values with a super intelligent machine that’s very capable, then you have a really serious problem for the human race. So the point is that machines can and will make better decisions than humans but only if their values are aligned with those of the human race.

Now, my distinguished colleagues may argue that super-intelligent AI will never happen. Let me take you back to September 11, 1933. Lord Rutherford, the world’s leading nuclear physicist, said that atomic energy was moonshine. Could never happen. The next morning, Leo Szilard invented the nuclear chain reaction, the next morning.

So we have to be careful. Let’s look at nuclear fusion in particular. Long ago, they invented a method of generating unlimited amounts of energy, long ago. It’s called the hydrogen bomb. So now, fusion concentrates on containment. And AI has to do the same thing. If you want unlimited intelligence, you have to solve value alignment.

So one way of doing this is called inverse reinforcement learning. What that means is, for example, a machine sees somebody making coffee in the morning. And then figures out the purpose, the underlying utility function, that explains this behaviour. Namely, that having coffee is a good idea, as long as it’s not too much of it.

Now, it’s not quite as simple as that, as I’m sure you all see. Humans differ in their values. Cultures differ in their values. None of us behaves perfectly. But there is a huge amount of information that the machine can access about human actions. Every television programme, every book, every novel, every movie, every newspaper article is about human actions and, in particular, about our attitudes to those actions.

So the rational thing for a machine to do is to engage in an extended conversation with the human race about its values before it can take any action that affects the real world. So my claim is that in the future, we will be able to design super intelligent machines that do exactly what they’re supposed to do, which is to support greater realisation of human values. And I think this is maybe the most important conversation that we can have over the next 50 years. Thank you.

End transcript

Download

 

Interactive feature not available in single page view (see it in standard view).

While watching this video, think about Russell’s opinion of the progress of recent AI technology.

Afterwards, click on ‘Save and Reveal Discussion’.

To use this interactive functionality a free OU account is required. Sign in or register.

Interactive feature not available in single page view (see it in standard view).

Discussion

Russell’s view on the continuing progress of AI is optimistic, in that Russell is quite confident that AI will one day achieve human level performance in a range of areas. Russell points out that to some extent, it already has, albeit in quite limited domains such as highly constrained game worlds, including examples of IBM’s Watson beating champions of the Jeopardy! game show, or DeepMind’s AlphaGo beating human world champions of Go. Russell’s main point here is that, given these achievements, and the expectation of further advances, it is prudent to consider how to make AI safer, if we are to properly prepare for when (not if) human-level performing AI systems are deployed throughout the world.

Part 2

Do you agree with Russell’s main argument that rather than wondering if AI will reach human-level performance, the safest course of action is to simply accept this will happen? If we accept this eventuality, should we begin now to work out how to best deal with this eventuality?

After you have had a chance to come up with your own answers to these questions, click on ‘Save and Reveal Discussion’.

To use this interactive functionality a free OU account is required. Sign in or register.

Interactive feature not available in single page view (see it in standard view).

Discussion

Russell’s position can be queried on a number of fronts. The most obvious question is whether Russell is being overly optimistic about the capability of AI. Those supporting a position such as Russell’s might argue, however, that this is missing the point he is making, and that it is simply sensible to try to avoid a scenario that includes AI technology that is both powerful and dangerous. With the right measure of risk, and ways of mitigating this, we can have a better outcome of powerful AI that is far less harmful.

However, there is another possible response to Russell’s suggestions, which would mean taking them to a natural, yet sensible conclusion: if we cannot guarantee the safety of the technologies being considered, then perhaps we should not pursue them until their safety has improved.

A key idea from the discussion so far in this section, is that the social and political context within which AI technology is used, may in fact lead to such technology exacerbating existing biases and inequities, often having greater impact on more marginalised members of society. Recall from Week 1, that the use of the COMPAS algorithm in the U.S.A. has led to increased inequality in the justice system, due to existing historical and social disparities in this system.

Activity 2

Timing: 30 minutes

It is worth balancing Russell’s perceived optimism about the capability for AI to approximate human-level performance with consideration of the limitations of AI.

The following talk by Janelle Shane presents some very sobering examples that highlight the fact that there is still quite a way to go before AI systems reach human-level performance in general domains (i.e. outside specialised and constrained domains such as games).

Watch this video, and answer the following questions. Use the response boxes to record your answers.

Download this video clip.Video player: lg003_2022b_vwr006_320x176.mp4

Show transcript|Hide transcript

Transcript

JANELLE SHANE:

So, artificial intelligence is known for disrupting all kinds of industries. What about ice cream? What kind of mind-blowing new flavours could we generate with the power of an advanced artificial intelligence?

So I teamed up with a group of coders from Keeling Middle School to find out the answer to this question. And they collected over 1,600 existing ice cream flavours, and together we fed them to an algorithm to see what it would generate. And here are some of the flavours that the AI came up with.

These flavours are not delicious as we might have hoped they would be, so the question is, what happened? What went wrong? Is the AI trying to kill us, or is it trying to do what we asked, and there was a problem?

So in movies, once something goes wrong with AI, it’s usually because the AI has decided that it doesn’t want to obey the humans anymore, and it’s got its own goals, thank you very much. So in real life, though, the AI that we actually have is not nearly smart enough for that. It has the approximate computing power of an earthworm, or maybe, at most, a single honeybee, and actually, probably, maybe less.

We’re constantly learning new things about brains that make it clear how much our AIs don’t measure up to real brains. So today’s AI can do a task like identify a pedestrian in a picture, but it doesn’t have a concept of what the pedestrian is beyond it’s a collection of lines and textures and things. It doesn’t know what a human actually is.

So will today’s AI do what we ask it to do? It will if it can, but it might not do what we actually want. So let’s say that you are trying to get an AI to take this collection of robot parts and assemble them into some kind of robot to get from point A to point B. Now, if you were going to try and solve this problem by writing a traditional-style computer programme, you would give the programme step-by-step instructions on how to take these parts, how to assemble them into a robot with legs, and then how to use those legs to walk to Point B.

But when you’re using AI to solve the problem, it goes differently. You don’t tell it how to solve the problem. You just give it the goal, and it has to figure out for itself, via trial and error, how to reach that goal.

And it turns out that the way that AI tends to solve this particular problem is by doing this. It assembles itself into a tower, and then falls over and lands at Point B, and technically this solves the problem. Technically it got to Point B.

The danger of AI is not that it’s going to rebel against us. It’s that it’s going to do exactly what we ask it to do. So then the trick of working with AI becomes, how do we set up the problem so that it actually does what we want?

So this little robot here is being controlled by an AI. The AI came up with a design for the robot legs, and then figured out how to use them to get past all these obstacles. But when David Haas set up this experiment, he had to set it up with very, very strict limits on how big the AI was allowed to make the legs, because otherwise.

And technically it got to the end of that obstacle course. So you can see how hard it is to get AI to do something as simple as just walk. So seeing the AI do this, you may say, OK, no fair. You can’t just be a tall tower and fall over. You have to actually use legs to walk. And it turns out that doesn’t always work either.

So this AI’s job was to move fast. They didn’t tell it that it had to run facing forward, or that it couldn’t use its arms. So this is what you get when you train AI to move fast, is you get things like somersaulting, and silly walks. It’s really common. So is twitching along the floor in a heap.

So in my opinion, you know what should have been a whole lot weirder? Is the Terminator robots. Hacking the matrix is another thing that AI will do if you give it a chance. So if you train an AI in a simulation, it will learn how to do things like hack into the simulation’s math errors and harvest them for energy, or to figure out how to move faster by glitching repeatedly into the floor.

When you’re working with AI, it’s less like working with another human, and a lot more like working with some kind of weird force of nature. And it’s really easy to accidentally give AI the wrong problem to solve, and often we don’t realise that until something has actually gone wrong. So here’s an experiment I did where I wanted the AI to copy paint colours - to invent new paint colours given lists like the ones here on the left. And here’s what the AI actually came up with.

So technically it did what I asked it to. So I thought I was asking it for nice paint colour names, but what I was actually asking it to do was just imitate the kinds of letter combinations that it had seen in the original. And I didn’t tell it anything about what words mean, or that there are maybe some words that it should avoid using in these paint colours. So its entire world is the data that I gave it. Like with the ice cream flavours, it doesn’t know about anything else.

So it is through the data that we often accidentally tell AI to do the wrong thing. So this is a fish called a tench, and there was a group of researchers who trained an AI to identify this tench in pictures. But then when they asked it what part of the picture that it was actually using to identify the fish, here’s what it highlighted.

Yes, those are human fingers. Why would it be looking for human fingers if it’s trying to identify a fish? Well, it turns out that the tench is a trophy fish, and so in a lot of the pictures that the AI had seen of this fish during training, the fish looked like this. And it didn’t know that the fingers aren’t part of the fish.

So you see why it is so hard to design an AI that actually can understand what it’s looking at. And this is why designing the image recognition in self-driving cars is so hard, and why so many self-driving car failures are because the AI got confused. I want to talk about an example from 2016.

There was a fatal accident when somebody was using Tesla’s autopilot AI, but instead of using it on the highway like it was designed for, they used it on city streets. And what happened was a truck drove out in front of the car, and the car failed to brake.

Now, the AI definitely was trained to recognise trucks in pictures, but what it looks like happened is the AI was trained to recognise trucks on highway driving, where you would expect to see trucks from behind. Trucks from the side is not supposed to happen on a highway. And so when the AI saw this truck, it looks like the AI recognised it as most likely to be a road sign, and therefore safe to drive underneath.

Here’s an AI mess up from a different field. Amazon recently had to give up on a resume-sorting algorithm they were working on when they discovered that the algorithm had learned to discriminate against women. What happened is they had trained it on example resumes of people who they had hired in the past, and from these examples, the AI learned to avoid the resumes of people who had gone to women’s colleges, or who had the word "women" somewhere in their resume - as in women’s soccer team, or Society of Women Engineers.

The AI didn’t know that it wasn’t supposed to copy this particular thing that it had seen the humans do. And technically, it did what they asked it to do. They just accidentally asked it to do the wrong thing.

And this happens all the time with AI. AI can be really destructive and not know it. So the AIs that recommend new content in Facebook and YouTube, they’re optimised to increase the number of clicks and views. And unfortunately, one way that they have found of doing this is to recommend content of conspiracy theories, or bigotry.

The AIs themselves don’t have any concept of what this content actually is, and they don’t have any concept of what the consequences might be of recommending this content. So when we’re working with AI, it’s up to us to avoid problems. And avoiding things going wrong, that may come down to the age-old problem of communication, where we, as humans, have to learn how to communicate with AI.

We have to learn what AI is capable of doing, and what it’s not, and to understand that with its tiny, little worm brain, AI doesn’t really understand what we’re trying to ask it to do. So in other words, we have to be prepared to work with an AI that’s not the super competent, all-knowing AI of science fiction. We have to be prepared to work with an AI that’s the one that we actually have in the present day. And present day AI is plenty weird enough. Thank you.

End transcript

Download

 

Interactive feature not available in single page view (see it in standard view).

Once you have added your response, click on ‘Save and Reveal Discussion’.

Part 1

Regarding the ice-cream flavours algorithm: What does it mean to say the AI involved ‘is not nearly smart enough’?

To use this interactive functionality a free OU account is required. Sign in or register.

Interactive feature not available in single page view (see it in standard view).

Discussion

The ice-cream flavours algorithm did not understand the meaning of the labels for flavours, and so it was unaware of how the resulting combinations of flavours would taste, and therefore whether humans would like them.

Part 2

Is it enough that an AI does what we ask it to do? What else do we need to make sure of?

To use this interactive functionality a free OU account is required. Sign in or register.

Interactive feature not available in single page view (see it in standard view).

Discussion

An AI may be able to do what it is asked, but because it can’t fully understand what we are asking it, it will not likely do what we want it to do. In the example of AI being asked to assemble a robot, Shane makes the important point that there is a real danger that AI can reach goals we set it, while not actually achieve the outcome we wanted. Shane points out that when formulating a problem for an AI to solve, we need to set the limits of the problem very carefully (think about the example of the AI designing a robot to walk over gaps in a simulated environment, and how the designer needed to carefully specify the limits on how big the robot could be).

Part 3

More seriously, just over halfway through the talk, Shane also talks about a fatal accident in 2016 involving an autopilot AI in a Tesla car. How was the failure in this case a matter of the AI doing what it was asked, but not what was wanted?

To use this interactive functionality a free OU account is required. Sign in or register.

Interactive feature not available in single page view (see it in standard view).

Discussion

The Tesla autopilot AI was being used on city streets, rather than on the highway for which it was designed, and it failed to recognise a truck driving across the car’s path. The failure seemed to be due to the AI autopilot being presented with a side view of the truck, whereas for normal highway driving trucks are seen from behind. Apparently, the AI autopilot mis-recognised the side view of the truck, including the gap underneath the truck between its front and rear wheels, as an overhead street sign, and tried to drive underneath the truck. In this case, it was doing what it had been designed to do, i.e. drive underneath street signs, but it had failed to understand the difference between a sign and a truck.

Part 4

Another example which is relevant to this course is the case of AI technology making discriminatory decisions about the resumes of people applying for jobs at Amazon. Specifically, this technology had learned to discriminate against candidates who were identified as women. Again, how was the failure in this case a matter of the AI doing what it was asked, but not what was wanted?

To use this interactive functionality a free OU account is required. Sign in or register.

Interactive feature not available in single page view (see it in standard view).

Discussion

The Amazon resume reviewing system was trained on a biased training set, which was a historical dataset from Amazon that had a much greater representation of men than women in it. Using this dataset, the AI learned to discriminate against women, rather than hire the best candidate for the job regardless of identified gender. Note that this is an obvious example of where AI technology working within a context of past discriminatory practices can itself exacerbate such discrimination.

The rest of this week will proceed under the assumption that ‘AI is possible’. Given this assumption, it is natural to ask what sorts of issues and challenges with AI technology do we need to be aware of? To this end, in the remaining sections of this week, you will work through a range of information and activities addressing the following series of questions:

• What about diversity, fairness, and equitability in AI?
• What about sustainability in AI?
• What about the use of AI for social good?