A fatal bug? Did computers cause the Air France disaster?

Airbus 330 flying overhead.
 

On June 1st, an Air France Airbus A330 on a routine flight from Rio de Janeiro to Paris crashed into the Atlantic Ocean. 228 people died in the worst air accident in French aviation history. The disaster was all the more shocking because one of the world's most reputable airlines had lost one of the most reliable airliners ever built. Until the crash of Air France 447, some 600 A330s had flown for sixteen years without a single fatality.  The aircraft crashed in an area of the Atlantic up to 3 kilometres deep leaving little evidence apart from a small amount of floating wreckage and some bodies. The crucial flight recorders (often called the black boxes) now lie on the ocean floor and have not been recovered.

One month later, France's air accident authority, the Bureau d'Enquêtes et d'Analyses pour la Sécurité de l'Aviation Civile, released an interim report based on what little was known; the aircraft had hit the water intact at high speed in a steep dive and showed no sign of fire or explosion. This interim report stated:

"At this stage of the investigation, the only established facts are:

• the presence near the airplane’s planned route over the Atlantic of significant convective cells typical of the equatorial regions;
• based on the analysis of the automatic messages broadcast by the plane, there are inconsistencies between the various speeds measured."

Over a five minute period, the aircraft's computers began to report a series of equipment failures that began in the vital airspeed sensors which are necessary to keep the aircraft in stable flight.

Over a five minute period, the aircraft's computers began to report a series of equipment failures that began in the vital airspeed sensors which are necessary to keep the aircraft in stable flight.

Our knowledge of the last few minutes of the AF447 comes from automated messages radioed back to Air France's maintenance facilities using a system known as the Aircraft Communication Addressing and Reporting System (ACARS). Over a five minute period, the aircraft's computers began to report a series of equipment failures that began in the vital airspeed sensors which are necessary to keep the aircraft in stable flight. At the time, AF447 was flying through a series of intense tropical thunderstorms; it would have flown through lightning and extreme turbulence and may have also encountered freezing conditions. In themselves, these should not have caused the loss of a modern airliner. A number of other aircraft safely threaded through the same storms that night without serious incident.

In the absence of a clear cause, some reporters and bloggers have begun to blame the disaster on the use by Airbus of computerised, "fly-by-wire" technology. It has been suggested that the computers on the aircraft, if they did not actually cause the accident, may have made it impossible for the crew to avoid disaster.

How aircraft are manoeuvred
So, before we have a look at why aircraft use computers and what they do, perhaps a small diversion is in order. Airplanes manoeuvre using a combination of "control surfaces" - sometimes (incorrectly) called flaps - located on the wings and tail. You've probably seen these devices working during take-off and landing. The outer parts of the wings contain the ailerons controlling the amount of roll (or banking) used to turn the aircraft on to another heading. The horizontal surfaces in the tail are called elevators and are used to change the pitch - the nose-up or nose down attitude of the aircraft when it changes height. The vertical surface on the tail is known as the rudder and is also used to turn the aircraft, this time without the sometimes disconcerting tilt of banking. The aircraft wings also contain the flaps which are used during take-off and landing to provide additional lift or drag.

The control surfaces are driven from the cockpit. In very small aircraft this can be achieved using manual linkages not too different from the brake cables found on bicycles. When the pilot moves the joystick, it directly pulls or slackens a cable, the other end of which is attached to a control surface. However, as planes become larger and faster, the amount of force needed to move the ever-larger control surfaces becomes greater and greater, until it is not physically possible to move them at all.

During the 1950s and 1960s aircraft designers increasingly switched to hydraulic linkages similar to those found in cars. In these more modern aircraft, movements of the joystick were transferred to the control surfaces through pressurised hydraulic fluid. Pilots did not need to be especially strong, the hydraulics did all the work. The weakness of hydraulic systems is that the plane needs to be threaded with pipes which must be regularly inspected for defects; a leak could result in disaster. To reduce the risk of any one system failing, the hydraulic system was duplicated - each control surface could be moved by any one of three (sometimes four) independent hydraulic circuits; the hydraulics were said to be multiply redundant. There are only a very few cases where all of an aircraft's hydraulics have failed in-flight, and the technology continues to be used on many modern aircraft.

The weakness of hydraulics is that they are heavy and maintenance intensive. If reliance on them could be reduced, or dispensed with entirely, aircraft could carry a more useful payload and spend longer in the air - both of which make them more profitable. Fly-by-wire is the solution to this; the long, complex hydraulic links between the joystick and the control surfaces are replaced by sensors and electrical cabling. When the joystick is moved, sensors read the changes and send electrical signals to hydraulic pumps located near the control surfaces. These pumps then move the surfaces as if they were directly linked to the joystick. Fly-by-wire technology was developed in the UK and US during the 1960s for military aircraft and received its first commercial use inside the Anglo-French Concorde in 1969, but it was not especially well known until Airbus chose the technology for the A320, unveiled in 1987.

he A320 revolution
Airbus had been founded for political motives with the intention of combining the expertise of various European airspace manufacturers to build a rival to the American airline industry, dominated by Boeing and MacDonell Douglas (now part of Boeing). Although Europe, and especially Britain, had led the world in developing airliner technology throughout the 1950s and 1960s, it had been the Americans who had gone on to dominate the World market for airliners. Airbus' first airliner, the A300, had become a successful twin-engined plane but had used relatively conventional technologies; the A320 would be a huge leap into the future. It was designed to compete both with the world's best-selling airliner, the Boeing 737, and also to replace the older, thirstier, noisier 3-engined Boeing 727.

The A320 was a revolutionary aircraft, not only including fly-by-wire technology, but also being one of the first airliners to be built using substantial amounts of composite materials such as carbon fibre. Its cockpit was equally novel; there would only be two flight crew - the engineer was no longer needed, their role being taken by a highly automated "glass cockpit" that replaced switches and dials with computer screens. Aggressively marketed, the economical A320 family of jets has sold nearly 4000 aircraft, making it the second most successful airliner in the world, and is likely to be built for many years yet. The success of the A320 allowed Airbus to plan even more ambitious aircraft including the twin-engined A330, the four-engined A340 and the enormous A380 double-decked super Jumbo which entered service in late 2007. This family of aircraft has allowed Airbus to rival, and sometimes supplant, Boeing as the world's largest manufacturer of airliners - much to that company's disgust.

Interior of Airbus A340 cockpit.
[image by Storm Crypt,
some rights reserved]

As well as emphasising the comfort, reliability and economy of their aircraft, Airbus have been keen to stress their exceptional safety, made possible by computer technology. Airbus took a decision that computer technology could be used to protect the aircraft from any action by the pilots that could damage or destroy it. The safe operation of an aircraft is constrained by a "flight envelope" which describes factors such as the maximum and minimum speeds, the tightest turn it can make and so on. If an aircraft exceeds its flight envelope it can result in injury to the passengers, damage to the airframe or a complete structural failure. The flight envelope is not a simple, static object; rather it changes on a number of factors such as the altitude. In theory, a computer can ensure that the aircraft remains safely inside the envelope at all times - the aircraft is said to have "flight envelope protection". The consequence of flight envelope protection is profound; the pilot no longer has absolute control of the aircraft; the computer will veto any action that would take the aircraft outside of the flight envelope.

But, before protection can be guaranteed, it is crucial that the computers are completely reliable and accurate.

Reliable computers
The Airbus contains five main computers divided into two main roles. Three of the computers are designated the primary flight control computers and are in day-to-day control of the plane; reading the pilot's instructions, monitoring the aircraft's position, speed and attitude; making the necessary calculations to keep the aircraft safe, and sending commands to the engines and control surfaces. These are backed up by a pair of secondary flight control computers which are constantly monitoring the aircraft, but only act if one or all of the primary flight control computers become unavailable. These computers are distributed around the fuselage so that an impact or hull breach should not disable more than one machine. Likewise, multiple cables link the computers - cutting one, or some of them, will not disable the entire system

In normal use, the computers each read the data from the pilot and sensors built into the aircraft and individually calculate the appropriate response. At preset intervals the responses from each computer are compared. If the result from one computer differs from the other two, it is automatically disqualified from further operation and a backup computer is switched in to make further decisions. Likewise, if one of the computers fails to respond in time for one of these votes, it is disconnected and a replacement called in. In fact, the aircraft can be safely flown and landed using only one computer, so there is massive redundancy built into the computer systems.

The designers of the Airbus computers went to enormous trouble trying to imagine all of the possible problems that could occur. Their first problem was the certainty that computer hardware and software is almost never completely free of bugs that could cause a program to crash and the to computer become unavailable. Therefore the primary and secondary flight computers not only come from different companies, but they must contain different components - so a hardware failure should not spread between the two computer systems. This diversity is replicated inside the software; with the primary and secondary computers each running different programs coded in different languages. These programs were developed by teams with exceptional records of producing high-quality software, using special software tools that should capture bugs long before the programs are ever used in real life.

Airbus's designers then went on to consider what would happen if the aircraft hit trouble - such as some of the vital sensors became unavailable. Just like Isaac Asimov's robots, Airbus aircraft are governed by three Laws.

The designers of the Airbus computers went to enormous trouble trying to imagine all of the possible problems that could occur.

The first is called Normal Law and applies when the aircraft and its systems are healthy. The flight control computers interpret the commands from the joystick and guarantee that the aircraft remains safely within the flight envelope; they also ensure that passengers remain comfortable by reducing the rate of changes in direction or altitude.

If some of the sensors fail, the hydraulics become unreliable or more than two computers are unavailable, the computers switch to Alternate Law. Here some of the protections are removed or relaxed, the aircraft can make more extreme manoeuvres but cannot exceed its flight envelope. This might sound counter-intuitive, you may be thinking this is the sort of circumstance where the pilots need more help from the computers; but Airbus' thinking was that, if the sensors or computers could no longer be trusted to read or interpret data correctly, then it was time to pass more control to the expertise of the pilots.

Further failures would force the aircraft into Direct Law. At this point the aircraft can no longer offer flight envelope protection and the Airbus must be flown like an older generation aircraft.

In the event of a catastrophic failure resulting in the total loss of power, the Airbus has a further mechanical backup mode which could be used to make an emergency landing, but would most likely be used for a few minutes whilst the flight crew tried to recover power. This is extremely unlikely to happen as the aircraft would have to lose both engines, the auxiliary power unit in the tail, have flat batteries and not be able to deploy the ram air turbine (a wind generator which can be swung out from the underside of the aircraft).

Wheels of Boeing777.
 

Flight envelope protection became a huge difference in philosophy between Airbus and its rival, Boeing. The American company was reluctant to remove ultimate control from the human and could cite a number of instances where an aircraft was only saved by exceeding the flight envelope. In 1985 a China Airlines Boeing 747 flying between Taiwan and the United States suffered a relatively minor engine failure over the Pacific. The crew did not follow the proper procedures for restarting the engine and the aircraft eventually tipped into a vertical dive. Disaster was only avoided when the pilot forced the nose up using the elevators. The aircraft vastly exceeded its envelope and suffered severe damage to its control surfaces and undercarriage but it was able to land safely with only two injuries. Airbus countered that such incidents were exceptionally rare and, besides, flight envelope protection would have ensured the aircraft never entered the dive in the first place.

Did the computers have anything to do with the loss of AF447?
The ACARS data sent back to Air France during the last few minutes show that the airspeed sensors mounted on the aircraft were registering as faulty. Following incidents on other Air France A330 and A340 airliners, the company had entered into discussions with Airbus, who had determined that certain sensor designs were prone to becoming clogged with ice or water and recommended that they be replaced as part of scheduled maintenance. Although the aircraft had not received the improved sensors, it had been declared safe to fly, but it is entirely possible that the airspeed sensors had developed a fault. As soon as the computers realised the airspeed readings from the sensors could not be trusted, they switched to Alternate Law, disengaged the autopilot and switched off the automated thrust systems. The computers would continue to keep the aircraft within the flight envelope, but the crew would be in charge of steering and maintaining the correct airspeed. The very last minute of the ACARS data suggests that the problems had continued to spread through the computerised systems responsible for maintaining the aircraft's speed and orientation. The very last message warned that the Airbus had entered a steep descent. Crucially, the data does not suggest that the computers had ever entered Direct Mode or indeed failed all together. The evidence is that the computers were battling to keep the aircraft in the air until disaster was unavoidable - they were working.

Previously, in 2008, an A330 belonging to the Australian operator Qantas experienced an in-flight emergency when one of the computers used to collate sensor data developed a serious fault which resulted in unexpected violent pitching and false stall and overspeed warnings. Fortunately, the computer was deactivated, but not before 115 people on board were injured. Airbus revised their instructions to pilots on how to deal with such an incident which proved useful less than three months later when a second Qantas A330 flying in the same area encountered a similar fault with the same computer in a different aircraft; fortunately, this time, no one was injured. Airbus and the computer's manufacturer are still trying to ascertain the exact cause of the problems but pilots have blamed radio interference from a powerful naval transmitter in Western Australia. Could a similar problem have befallen AF447? It is possible, but Airbus point out that the doomed aircraft used different computer hardware and software from the Qantas jets and it is extremely unlikely a similar bug could exist in both sets of equipment.

It is not impossible, but increasingly unlikely, that AF447's flight recorders will be recovered from the floor of the Atlantic Ocean. If they are found, air accident investigators will be able to examine the operation of the airliner's computers and sensors on a second-by-second basis and listen to the words of the flight crew. If they are not located, then we might never know precisely what happened on the flight. Instead, Airbus and the French authorities will have to make a reasoned judgement on what might have occurred and make recommendations to avoid their recurrence. Even before any report, Air France has replaced all of the airspeed sensors on its A330 and A340 aircraft.

The most likely explanation for the loss of AF447 lies with the failure of those airspeed sensors. If an airliner loses too much airspeed it loses the lift necessary to keep it in the air; it is said to have entered an aerodynamic stall. Stalling can also be brought about by sudden rises in the temperature of the air and by banking the wings. Pilots are trained both to recognise the potential for stalls and to recover from, them. But perhaps the crew of AF447 were overwhelmed by a series of events that began with what should have been a routine sensor failure. As they responded to the imposition of Alternate Law and their new responsibilities for maintaining the aircraft's speed, they would also have been quieting the various alerts appearing on their screens and fighting the storm. This would not have been the first time humans were unable to keep up with a computer in an emergency; the operators of the Three Mile Island nuclear power station in the United States were overwhelmed by so many alarms that they failed to identify a relatively minor problem that could have been easily fixed before it became a near disaster. Even now, Airbus will be examining how air crew are alerted to problems and determining if these might make circumstances worse rather than better.

...flying is still statistically safer than the drive to the airport.

Although much ink and vitriol has been spilled by supporters and detractors of Airbus' highly automated airliners; the accident records for aircraft with flight envelope protection are quite clear. Whilst highly automated aircraft  show improved performance and reliability and economics, they are neither more nor less likely to be involved in an accident. So perhaps it is the economic benefits that drive this technology. Even Boeing, so long a sceptic over fly-by-wire and envelope protection, is adopting it for the Boeing 777 and 787 Dreamliner airliners.

The statistics are also clear; modern aircraft are much safer than those of previous generations and flying is still statistically safer than the drive to the airport.

Find out more

Follow the unravelling of other disaster stories with forensic engineering:
Collapse at Kinzua
Silver Bridge
Tay Bridge
Concorde

Images

The images used in this blog are copyright. All are from flickr.com under the following creative commons licenses:

Airbus A330 flying overhead by Abdallahh - Attribution
Interior of Airbus A340 cockpit by Storm Crypt - Attribution/Non-Commercial/No Derivative Works
Wheels of Boeing777 by Diorama Sky - Attribution/Non-Commercial/No Derivative Works