2 Total Management: What hope for leadership?

Management goes hand in hand with technology, which is deployed to automate decision-making, managing situations to make them routine.

Maximise

Figure 5 Use of technology in management

Show description|Hide description

This is a diagram with a box containing ‘Technology in management allows for:’. Lines lead from this box towards two others. The first of these boxes contains the text ‘a) the mechanisation of work tasks so that production speeds up and output increases’, and the second box contains ‘b) the digitisation of data, meaning that evaluation of work processes can become more efficient and evidence-based’.

Figure 5 Use of technology in management

In the workplace, technology enables greater division of labour, or no human labour at all, as people are replaced by machines. Such developments remove human discretion from systems, meaning that ever greater proportions of our lives are automated: most obviously the number of tasks one should perform at work, and in what order they should be completed. This tendency for ever greater use of automated technology for decision-making is known as ‘Total Management’, under which ‘workers (and indeed managers) are increasingly subjected to automated work processes, with performance and judgments about aptitude assessed by algorithms rather than people’ (Grint and Smolović Jones, 2022, p.4; Smolović Jones and Hollis, 2022).

On one level, digital technology and automation promise liberation. More routine, dirty and unpleasant work can be outsourced to machines. Why would we want to clean toilets ourselves if a robot could do so for us? In theory, outsourcing to machines in this way would give people more freedom to pursue their passions.

However, the intensification of digital technology and the ways in which it manages our lives can normalise and lock in prejudice and injustice. In a fundamental sense, algorithms are very human. Being programmed by humans, they start off informed by human values and priorities. Ideally, they would then be set to work at enacting our values in ways that maximise freedom, care, health and happiness. However, we know that humans can be prejudiced, and such tendencies can be unconscious as well as conscious. What algorithms do, at their worst, is lock in assumptions, attitudes and values of a particular time and group of people and normalise them into decision-making.

The word ‘automation’ therefore works on two levels. The first is physical, meaning on the level of the technology itself, which we regard as increasingly normal in directing our lives. Smartphones, for example, have become extensions of our bodies, always with us and always connecting us. The second level is ideological, which works by normalising certain values and assumptions through technology (Berardi, 2019). Racism, as well as other forms of inequality, can become routinised and locked in to everyday decisions and behaviour.

You will now explore the dangers of automated racism – as well as the possibilities for leadership in challenging it – through an activity.

Activity 2 Racist algorithms?

Timing: Allow around 30 minutes

Watch the following video, where ‘poet of code’ and founder of the Algorithmic Justice League, Joy Buolamwini, talks about the dangers of algorithms.

As you watch, try to think of an example in your life where algorithms have created and/or normalised an injustice. If not, you might have seen an instance in the news. You could do some internet research, or ask some friends and colleagues what they think. Plenty of examples exist, for example, in the realms of policing, education and business. After you do this, try to think about the role Black leadership could play in the future in changing how societies and organisations approach algorithms in their activities.

Download this video clip.Video player: Video 1

Show transcript | Hide transcript

Transcript: Video 1 Joy Buolamwini – how I’m fighting bias in algorithms

[APPLAUSE]

JOY BUOLAMWINI:

Hello. I’m Joy. A poet of code on a mission to stop an unseen force that’s rising. A force that I called the coded gaze. My term for algorithmic bias.

Algorithmic bias like human bias results in unfairness. However, algorithms, like viruses, can spread bias on a massive scale at a rapid pace. Algorithmic bias can also lead to exclusionary experiences and discriminatory practices. Let me show you what I mean.

Hi, camera. I’ve got a face. Can you see my face? No glasses face.

You can see her face. What about my face? Well, I’ve got a mask. Can you see my mask?

So how did this happen? Why am I sitting in front of a computer in a white mask trying to be detected by a cheap webcam? Well, when I’m not fighting the coded gaze as a poet of code, I’m a graduate student at the MIT Media Lab. And there, I have the opportunity to work on all sorts of whimsical projects, including the Aspire Mirror, a project I did so I could project digital masks onto my reflection.

So in the morning, if I wanted to feel powerful, I could put on a lion. If I wanted to be uplifted, I might have a quote. So I used generic facial recognition software to build the system, but found that it was really hard to test it unless if I wore a white mask.

Unfortunately, I’ve run into this issue before. When I was an undergraduate at Georgia Tech studying computer science, I used to work on social robots. And one of my tasks was to get a robot to play peekaboo, a simple turn taking game where partners cover their face and then uncover it saying peekaboo.

The problem is peekaboo doesn’t really work if I can’t see you. And my robot couldn’t see me. But I borrowed my roommate’s face to get the project done. Submitted the assignment, and figured, you know what, somebody else will solve this problem.

Not too long after, I was in Hong Kong for an entrepreneurship competition. The organisers decided to take participants on a tour of local startups. One of the startups had a social robot, and they decided to do a demo.

The demo worked on everybody until it got to me, and you can probably guess it. It couldn’t detect my face. I asked the developers what was going on, and it turned out we had used the same generic facial recognition software.

Halfway around the world, I learned that algorithmic bias can travel as quickly as it takes to download some files off of the internet. So what’s going on? Why isn’t my face being detected? Well, we have to look at how we give machines sight. Computer vision uses machine learning techniques to do facial recognition.

So how this works is you create a training set with examples of faces. This is a face. This is a face. This is not a face. And over time you can teach a computer how to recognise other faces.

However, if the training sets aren’t really that diverse, any face that deviates too much from the established norm will be harder to detect, which is what was happening to me. But don’t worry, there’s some good news, training sets don’t just materialise out of nowhere. We actually can create them. So there’s an opportunity to create full spectrum training sets that reflect a richer portrait of humanity.

Now you’ve seen in my examples, social robots was how I found out about exclusion with algorithmic bias. But algorithmic bias can also lead to discriminatory practices. Across the US, police departments are starting to use facial recognition software in their crime fighting arsenal. Georgetown Law published a report showing that one in two adults in the US, that’s 117 million people, have their faces in facial recognition networks. Police departments can currently look at these networks unregulated using algorithms that have not been audited for accuracy, yet we know facial recognition is not fail proof, and labeling faces consistently remains a challenge.

You might have seen this on Facebook. My friends and I laugh all the time when we see other people mislabeled in our photos. But misidentifying a suspected criminal is no laughing matter, nor is breaching civil liberties. Machine learning is being used for facial recognition, but it’s also extending beyond the realm of computer vision.

In her book Weapons of Math Destruction, data scientist Cathy O’Neil talks about the rising new WMDs: widespread, mysterious, and destructive algorithms that are increasingly being used to make decisions that impact more aspects of our lives.

So, who gets hired or fired? Do you get that loan? Do you get insurance? Are you admitted into the college that you wanted to get into? Do you and I pay the same price for the same product purchased on the same platform?

Law enforcement is also starting to use machine learning for predictive policing. Some judges use machine generated risk scores to determine how long an individual is going to spend in prison. So we really have to think about these decisions. Are they fair? And we’ve seen that algorithmic bias doesn’t necessarily always lead to fair outcomes.

So what can we do about it? Well, we can start thinking about how we create more inclusive code and employ inclusive coding practices. It really starts with people. So, who codes matters. Are we creating full spectrum teams with diverse individuals who can check each other’s blind spots?

On the technical side, how we code matters. Are we factoring in fairness as we’re developing systems? And finally, why we code matters. We’ve used tools of computational creation to unlock immense wealth. We now have the opportunity to unlock even greater equality if we make social change a priority and not an afterthought.

And so these are the three tenets that will make up the incoding movement. Who codes matters. How we code matters. And why we code matters.

So to go towards incoding, we can start thinking about building platforms that can identify bias by collecting people’s experiences like the ones I’ve shared, but also auditing existing software. We can also start to create more inclusive training sets. Imagine a ‘selfies for inclusion’ campaign where you and I can help developers test and create more inclusive training sets. And we can also start thinking more conscientiously about the social impact of the technology that we’re developing.

To get the incoding movement started, I’ve launched the Algorithmic Justice League, where anyone who cares about fairness can help fight the coded gaze. On codedgaze.com, you can report bias, request audits, become a tester, and join the ongoing conversation #codedgaze.

So I invite you to join me in creating a world where technology works for all of us, not just some of us. A world where we value inclusion and centre social change. Thank you.

[APPLAUSE]

But I have-- I have one question. Will you join me in the fight?

[LAUGHTER]

[APPLAUSE]

End transcript: Video 1 Joy Buolamwini – how I’m fighting bias in algorithms

Download

Video 1 Joy Buolamwini – how I’m fighting bias in algorithms

Interactive feature not available in single page view (see it in standard view).

To use this interactive functionality a free OU account is required. Sign in or register.

Interactive feature not available in single page view (see it in standard view).

Discussion

In England in 2020, there was mass anger as the government used an algorithm to determine the A level results of school pupils who were unable to sit their exams because of the COVID-19 pandemic. The result was that ‘students from disadvantaged backgrounds were more likely to be downgraded’ from the predicted grades submitted by their teachers, whereas ‘private schools received more than twice as many A and A* grades as comprehensives’ (Duncan et al., 2020). The effects of the algorithm were disproportionately more damaging for racial and ethnic minority students, who were more likely to be studying at comprehensive schools in more deprived areas.

Black leadership could have a powerful influence in shaping the values that inform the design of algorithmic code. Automation and artificial intelligence hold the capacity to enrich everyone’s lives. Ensuring that they are used for the ends of racial equity and justice, however, will be a matter of ongoing struggle.

Management can feel safe and even comforting. It tells us that the world can be tamed, and indeed it often does just that. However, when management becomes too dominant it starts to become Total Management, with serious ethical consequences. There is a role for leadership in disrupting organisations and societies that become too reliant on management.

You will now move on to explore the other practice often confused with leadership – command.