Automated systems in aircraft reduce some risks at the cost of increasing other risks. Incorrect or inconsistent applications of automation to complex human-machine systems can have unexpected and even deadly consequences.

NASA Ames Research facility has done a lot of thinking about the proper ways to design these systems. Dr. Charles Billings, in particular, has published a number of excellent papers on the subject. Here’s an excerpt from his 1995 paper, “Human-Centered Aviation Automation: Principles and Guidelines”, where he asks (and answers) a fundamental design question:

If the human operator cannot effectively oversee and retain management authority over his tools, he has lost authority over the entire operation. Will this be a tenable situation?

I believe it comes down to a matter of trust. Will we provide pilots with full authority, train them carefully, and trust them to do “the right thing”, whatever it is in particular circumstances? Or will we circumscribe pilot authority by making it impossible to damage the airplane, and in the process perhaps make it impossible to use its ultimate capabilities if they really need them…? My bias, based on a number of cases in which pilots have been able to recover from extreme emergencies, and other cases in which they did not recover but could have had they used all available resources, is that command authority should be limited only for the most compelling reasons, and only after extensive consultation with both test and line pilots or controllers at “the sharp end” of the system.

Boeing and Airbus, the world’s largest manufacturers of transport aircraft, seem to draw the “compelling reasons” line in different places. Under the Airbus computers’ “Normal Law” operating mode, the pilots cannot command inputs that would cause the airplane to enter an dangerous condition (for instance: they cannot stall the plane by increasing the angle of attack without adding thrust; the computer will prevent a stall from happening). Whereas Boeing’s approach is to make dangerous conditions increasingly difficult to cause (for instance: the Boeing’s control column will provide increased resistance against a pilot who is about to stall the airplane, making it physically more difficult for the pilot to cause this condition, but still allowing the possibility).

This continues to be an area of active study and discussion throughout the aviation community, and it has broader application as we interact more often with complex machine-controlled systems. Many pilots decry the apparent loss of airmanship due to the increase in cockpit automation.