Gareth Mitchell: Hello, it’s Gareth Mitchell here, and I'm the presenter of Click Radio on the BBC World Service. Thank you very much for downloading this special podcast produced in association with the Open University. It forms part of our series on openness that we've been running over the last few months. We've been talking about diversity and the extent to which the stuff online is personalised; in other words are we living in a filter bubble? So we’re going to be discussing that in the light of an experiment that we conducted especially for Click a few weeks ago. I'm joined by Tony Hirst from the Open University. Just introduce yourself, Tony.
Tony Hirst: Hello Gareth. I'm Tony Hirst. I'm a lecturer in the Department of Communication & Systems at the OU.
Gareth Mitchell: So we’re going to be talking through some of the results from the experiments, of which more in a moment. In fact the person who’s been poring through the results is Nalida Poll, who is joining us on the Click team. So you’ve been having, I don’t know if I can call it fun, Nali, but you’ve been going through the results anyway.
Nalida Poll: Hello Gareth, hello Tony, thanks for inviting me to be here. I've actually been having fun going through the results, this is the kind of thing I like to do, believe it or not, so yes, thank you.
Gareth Mitchell: Right, well we’ll hear more about your research as we go along, but first let’s just go back a few weeks to our programme on diversity, and we interviewed Eli Pariser, who is the author of this book called the Filter Bubble that you may well have heard about. And he had a number of concerns about the way that results are ranked when we go onto search engines like, well Google especially. This is what we had to say.
Eli Pariser: My worry is that in order for Google to actually be a useful tool, people need to understand how it’s working, and people don’t use Google as a tool which is personalised, they don’t expect it to deliver a subjective version of the world, they expect it to deliver sort of the, I mean Google has this whole mythology around page rank, this algorithm that collects the truth from all of these different websites and democratically arrives at the right answer, and that’s not really how the search engine works anymore. So I think if you had control and you said in this case I want a personalised search, can you please filter out all of the stuff I disagree with, that would be one thing. In this case, and I think this is the real paternalism, Google’s making that choice for you and saying we’re not even going to show you this stuff because clearly you don’t seem like a person who’s interested.
Gareth Mitchell: Eli Pariser setting the scene, Tony, and his worry that we are, you know, we literally are living in this filter bubble, before we get into our own experiment, do you share any of his concerns, Tony?
Tony Hirst: Well I think that the web is a big place and Google, ever since it started, hasn’t necessarily been returning one, two, truth. I don’t think there is any notion of a Google ground truth. There definitely isn’t now and I don’t think there was historically. The Google algorithm ranks pages across the world, but whenever you’ve been going to visit Google from different country websites then you’ve been getting different results. As Google indexes different websites then results can change, so results can change on the page, you can see things differently over time. So whilst there is a concern that we’re getting a view from Google, I don’t know that it’s ever necessarily been a single view that everyone has experienced for the same search results across the world.
Gareth Mitchell: Okay, well we’ve been looking at the contemporary situation and this is where our fantastic Click listeners have been helping us. Now what we asked people to do a few weeks ago was to go onto Google and type in a word, and we wanted this to be obviously the same word to make the experiment consistent. So after quite a lot of thought amongst the team, we chose the word platform. So we thought it would be interesting to see what would happen if you typed platforminto Google in
Nalida Poll: So we have 29 countries as of now, the results are still coming in, going to have to stop them somehow. We have over 150 participants, mostly from the UK, the US, Australia, Canada, New Zealand, I'm from Chile so I got a few, at least 12 responses from my fellow countrymen, and yeah, from every single continent: we have South America, North America, Africa; we have responses from Uganda, Kenya, Oceania, quite a few from New Zealand; Asia, Nepal, from every single continent.
Gareth Mitchell: Right, so all regions, very well covered. As I say, I mean we were just blown away by the number of results that came in, which is why we had to ask for your help to go through them all. So we’re talking about a total of several hundred individual screen grabs. Now, if Eli’s hypothesis is correct, you'd expect to see very large differences in what people found when they typed in platform across these different countries. Was that the case then, Nali?
Nalida Poll: No, that was not the case. I found a consistent pattern, basically the same websites showed up over and over again, especially within country, within country basically everyone had the same seven to eight websites in the first five, for the first five hits basically, and most of the differences I found were between countries.
Gareth Mitchell: Right, okay, so you'd find for instance, you know, people who typed in platform in the UK generally got say the top 5 returns on the word platform were about the same, and the same might apply across France or Chile or wherever. And one might assume because the kinds of people who were taking part in this experiment are the kinds of people who listen to Click, they're interested in technology, and one reason that we chose the word platform is because it can have a meaning to do with trains, which might not necessarily have anything to do with our listeners, like a train platform, but obviously a computing platform, so we thought it would be an interesting word to try. So how prominent for instance was the, you know, the return for Wikipedia? You know, Wikipedia has an entry about computing platforms, did that figure in a lot of the top 5s?
Nalida Poll: Interestingly, not a single response was about trains, at all. Over 150 people, nothing about trains of any kind. Yes, Wikipedia was universally retrieved basically, all of the countries retrieved either computing platform Wikipedia or just platform Wikipedia, mostly in English but also in other languages, depending on the country.
Gareth Mitchell: So within these different countries, I know there are lots of entries, but just give us a sense of the kinds of things that came up in say the top 5 across all the different countries that you looked at.
Nalida Poll: So for the
Gareth Mitchell: Right, so their kind of search optimisation team, they’ll be getting Christmas bonuses this year for getting that at the top of the Google searches for platform, okay.
Nalida Poll: Yes, well deserved.
Gareth Mitchell: Right, so carry on, what were the other things you found?
Nalida Poll: In the
Gareth Mitchell: Okay, so quite a range across the different countries, but not such a range within the different countries. How much of an insight is that for you then, Tony?
Tony Hirst: The dominance of the technology platform was in the
Gareth Mitchell: And as for, say, personalisation, because in some ways I was surprised that the top 5s were so consistent within countries. But then I suppose the ranking within that top 5, it does change slightly.
Tony Hirst: Right, listening to Eli, I think we might well get the impression that when he talks of personalisation we might expect radically different results on the first page of the Google search listings. I think in actuality it’s a bit more subtle than that. The ranking could apply over pages, many, many pages deep, but most of us don’t go to the 10th, 11th, 12th page of Google search results, we tend to stick to the front page, and even on the front page when we do click on results, I think we go down the page in order, we scan down the results and we click the first one we find that hits what we’re searching for. So that means if you’ve got just a slight change in the ordering of pages on the first page, particularly if you’ve got something that might bubble up from a rank of 8th on the page, which might be below the bottom of your screen, up to 3rd or 4th on the screen, where you're far more likely to see it, then just that simple change in ordering, even though the results on the first page are the same, might have a huge difference in which results you click on.
Gareth Mitchell: All right, because I might have looked at that and said well that doesn’t seem to be too much of a difference, but in a way you're saying we can't underestimate that, you know, especially a shift from eight to three, that does make a difference to what people click on, and what’s important for Google, because Google wants to, its business model relies on relevance, so it wants to put a load of results in front of us that we are most likely to click on.
Tony Hirst: Exactly, yes, so Google runs experiments all the time and they know that the ordering of links on the page hugely influences the extent to which you follow those links. So by making just very, very slight changes, making them more relevant, making you more likely to click them, they're serving us the user, as they would see it.
Gareth Mitchell: Yeah, now one thing that we asked our listeners to do was to switch the personalisation settings on Google, you know, if they were logged into Google, into Gmail, to switch personalisation on or off, or indeed to log into Gmail, do the search and then log out, just to see what difference that made. People said it didn’t seem to make that much of a difference, and I just wonder if maybe that’s our choice of word,platform, that, you know, apart from some of the cases that Nali mentioned, is in many parts of the world quite a neutral word. Again, did that make, was that a surprise to you?
Tony Hirst: I think in part it depends on how heavy a user of Google you are. Google’s personalisation, Google gives different search results by country, it also gives different search results for individuals within a country, and the extent to which Google can do that depends on how much they know about you. So when you go to Google, Google will set a cookie on your machine so that it can track searches from that particular browser and it can build up a history of what searches you’ve used and what links you’ve clicked on, and try to tune the search results on that basis. If you're logged into the Google account, then Google can collect far more information about you. You do have personalisation settings, so you can limit the extent to which Google collects data, but at the extreme it can keep a record of all your searches that you’ve made on Google while you’ve been logged in. It can look at the links you’ve clicked on and the pages you’ve viewed from a Google search results page.
So if you're a heavy user of Google when you're logged in then Google’s got a big history about you that you can use to tune your results. And if you're searching a lot in a particular, a very specific area, then Google’s got more information to draw on when it’s personalising the results. So I'm a heavy user of Google when I'm logged into my computer, I search for a lot for technology and I get a lot of very, very technology-related searches on all manner of things. I also get results to things on my own blog, which Google knows that I write, and I also get links that people in my social circle, which is the people I follow on Twitter, which Google can track. So Google can annotate my search results listings with little badges showing which of my friends have recommended a link or shared a link, or even written that blog post or written that article. So my results in particular areas are very, very specific.
Gareth Mitchell: Okay. Another result that you found, Nali, was at one, you know, just a few particular correspondents who got in touch with us with interesting stories, for instance there was an Android phone user who tried lots of different ways of getting results for platform. So just tell us just briefly a little bit about that particular user and what you found.
Nalida Poll: So this user from
Gareth Mitchell: Wow, okay, so a bit like Tony’s example it sounds like what he has in common with Tony, very into technology, uses a lot of these devices, a lot of these platforms, and his returns were pretty personalised. Tony then, coming back to Eli Pariser’s original hypothesis and his concern, his sense that Google, and it’s not the only one, Facebook as well, put us in the filter bubble, they tell us essentially things that we want to know. And he uses a lot of political examples, so he’ll say for instance if you're into a particular political ideology, you will always get results that relate to your ideology because that’s what Google knows you're going to click on and therefore boost its click view rating and its advertising revenue and so on. How much do you buy that hypothesis? Big question!
Tony Hirst: It is a big question. I think we have only so much time to pay attention to things that are of interest to us or are important to us. If we were getting search results that were completely irrelevant to our needs for answering particular questions then we wouldn’t be following those results. We wouldn’t be using Google; we would be using a different search engine. I think the range of opinions that are available to us on the web is huge. It’s far bigger than were available to us before the time of the web, and I think even within a set of Google results, then the Google algorithms do try to provide variety. So if you run a particular search term and there is a particular website that’s got a lot of content about that term, then Google doesn’t fill the front page with just links to that website, it will collapse them all down to a single link and if you want to follow more pages within that website you will be given an option to explore deeper into those results. So there is variety in the page.
I mean Google doesn’t want to give you exactly the same sort of result on every link on the front page of results, because if that result, where you’ve got different flavours of exactly the same answer, is not the result you want, you're going to be disaffected by that search. So it’s in Google’s interest to provide you with variety on the page, whilst still being relevant so that if the first result or the second result isn’t the one you want then the third or fourth might well be, so I think it’s in Google’s interest to try and keep people engaged. And Google want people to spend time looking at the Google results page where they have adverts. So if they can give results that are of interest to you, while not necessarily directly reinforcing your opinion, if they're of interest to you, you may well go back to that results page and then click on one of those links. So it’s grist to the mill for Google, the more it can find out about you by exploring your interests and around the edges of your interests, and we all, we want answers to particular questions, but we’re interested in seeing it when we’re searching, we’re interested in seeing alternatives as well.
Gareth Mitchell: Okay, well we’re going to leave it there, so Tony Hirst thank you very much indeed, and also to Nali for pouring through all those results I mean, and we will stress that this is not a scientific analysis, we’re not trying to pretend it is, but I think as you heard there in the discussion pulled out some interesting points. So thank you very much to all our fantastic Click listeners all over the world who've taken their time to get involved in that project. I mean goodness me it’s a bit of a faff taking a screen grab of your computer and posting it up on Facebook, and you did it and we really, really appreciate it.
Now if you are downloading this podcast it means you are already on the Open University website where Tony and his colleagues have posted a whole load of stuff around our openness season, so whilst you're here, have a fish around, there’s some great stuff on here, Tony’s been blogging some fascinating stuff as we've been going along. We have extended versions of many of our interviews and discussions, one that really sticks in my mind is Andrew Keen versus Jeff Jarvis, that certainly gets going, so download that, have a look around. And also the final edition of our openness season goes out on the radio on July 19th, it will be downloadable as a podcast as well, and we’re going to be recording that in front of a live audience at the Open University, so definitely join us for that. And have a good look at this website whilst you're here, and it’s been a lot of fun doing this season with the Open University as well, fascinating stuff. Thank you very much, I'm Gareth Mitchell, the producer has been Helena Selby, bye bye.
Eli Pariser recently discussed his book The Filter Bubble on BBC Click. The book's central argument is that web users are cossetted in a bubble of information presented to them by filtered web search.
The programme decided to put that to the test by asking its listeners to Google search for the word 'platform' wherever they were, on whatever platform or browser.
Though not strictly a scientific survey, the results suggest that Google searches are less filtered than Pariser's book suggests.
Gareth discusses the nuances of these findings in a special podcast with expert Tony Hirst of The Open Unversity who have co-produced a season on the theme of 'openness'.
- Find out more about the Openness In A Digital Age Season
- Visit the BBC Click website
- Where to start studying computing and ICT with The Open University?