Risk

Best to Cut Your Losses Early: Lessons from the Failure of Quibi

October 22, 2020 By Nagesh Belludi Leave a Comment

Streaming startup Quibi is shutting down barely six months after going live. The Wall Street Journal reports,

Founder Jeffrey Katzenberg and Chief Executive Meg Whitman decided to shut down the company in an effort to return as much capital to investors as possible instead of trying to prolong the life of the company and risk losing more money.

Best to Cut Your Losses Early Quibi (short for “quick bite”) was a late entrant into a crowded marketplace, and its short-form serial format aimed at short attention spans failed to get traction with teenagers and young adults amid the pandemic. The Week was puzzled by Quibi’s decision to not allow people to watch it on their television:

Among the early criticism directed at Quibi was the fact that it was mobile only, and users couldn’t watch the app’s original shows on their TVs. This was especially problematic at a time when many people were no longer commuting to work due to the COVID-19 pandemic and were, therefore, not in need of short content to watch on the go.

At heart, Quibi was just another fast food joint with the same menu as the rest. The Guardian wondered if anyone would give Quibi the time of day:

Quibi’s content felt less revolutionary than underbaked, slapdash concepts sledgehammering the viewer with abrupt hits of celebrity. The overarching theme was of celebrity names without thinking through what they would be doing that is interesting or novel. It offered little marginal benefit to the free celebrity fare on Instagram, Twitter, YouTube, or TikTok. Why pay for Quibi, when “if you want snackable Chrissy Teigen content, her social media provides that for you without this sort of hackneyed, first-thought courtroom set-up.”

Quibi’s only bona fide USP was its potential to piggybank on Katzenberg’s deep connections in the Hollywood establishment for content.

Idea for Impact: Investing money, energy, and time into something that’s not working is dreadful to admit, but it’s essential to come to terms with things that don’t go as planned, and your high hopes are dashed. Don’t hold on to an idea that doesn’t pay off soon enough. Best to cut your losses early—you’ll have the least sunk costs and the fewest emotional attachments.

The Waterline Principle: How Much Risk Can You Tolerate?

October 15, 2020 By Nagesh Belludi Leave a Comment

American engineer and entrepreneur Wilbert L. “Bill” Gore (1912–86) was the founder (with wife Genevieve (Vieve)) of W. L. Gore & Associates, the maker of such innovative products as Gore-Tex fabrics, Elixir guitar strings, and a variety of medical products.

Gore’s open and creative workplace emphasized autonomy, fairness, commitment, and experimentation. He instituted a mental model for risk-tolerance called the “Waterline Principle.”

Gore compared the level of allowable risk to the waterline on a boat.

Sanction risks above the waterline since they wouldn’t sink the boat—you have ample autonomy above the waterline. If a decision goes bad and produces a hole in the side of the boat above the waterline, you can fix the hole, learn from the experience, and carry on.
Risks that fell below the waterline, in contrast, can blow a hole that can sink the boat. Below-the-waterline risks need prior approval from the “captain.” Your team can be prepared for such risks, investigate potential solutions, or buy appropriate insurance coverage.

How Much Risk Can You Tolerate - Mental Model 'The Waterline Principle'

Commenting about Bill Gore and his Waterline Principle, business consultant Jim Collins noted in his How the Mighty Fall (2009,)

When making risky bets and decisions in the face of ambiguous or conflicting data, ask three questions:

What’s the upside, if events turn out well?

What’s the downside, if events go very badly?

Can you live with the downside? Truly?

The Waterline Principle encourages prudent experimentation and conscientious risk-taking by lowering the risk waterline.

Idea for Impact: Risk analysis and risk reduction should be one of the primary goals of any intellectual process. Invite your team to identify risks that can sink the boat and those that can cause survivable damages.

An Olympian History of Humanity // Book Summary of Yuval Noah Harari’s ‘Sapiens’

September 10, 2020 By Nagesh Belludi Leave a Comment

Israeli historian and philosopher Yuval Noah Harari’s bestselling 464-page Sapiens: A Brief History of Humankind (2015) retells the 13.5 billion years-long odyssey of human evolution from the Big Bang to the near-future. Harari accounts for how Homo sapiens (the ‘wise man’) overcame the most extraordinary odds and numerous arbitrary inevitabilities to dominate the world the way we do at present.

Harari’s narratives span the cognitive revolution (70,000 years ago,) agricultural revolution (11,000 years,) scientific revolution (500 years,) industrial revolution (250 years,) and information revolution (50 years.) The first of these epochs, the cognitive revolution, coupled with a genetic mutation, was the real game-changer: Homo sapiens didn’t evolve efficiently from stooping apes to standing individuals. There were previously no less than six distinct homines, of which Homo sapiens came out top.

Sapiens argues that what made Homo sapiens special was our ability to develop networks and communities and tell stories, i.e., to organize and build large, connected communities around “shared fictions” or narratives—religion, nationalism, capitalism, trade groups, social institutions, for example. It was only through such intangible beliefs—not biological realities—that Homo sapiens were able to get the better of the physical world.

Homo sapiens’ talent for abstraction set us apart

Yuval Noah Harari's Sapiens: Human Evolution from the Big Bang to the Near-future

Language made it easier to dwell upon abstract matters and flexibly cooperate in ever-larger numbers. Harari’s examples cite how Homo sapiens—from our ancestors all the way up to today—are so willing to create and believe in such conceptual paradigms that have been the key to our success and the key to our problems.

Large numbers of strangers can cooperate successfully by believing in common myths. Any large-scale human cooperation—whether a modern state, a medieval church, an ancient city or an archaic tribe—is rooted in common myths that exist only in people’s collective imagination.

Harari’s inquiry is extensive. His scholarship is rigorous, and his interpretation creative. Yes, most of the book restates familiar facts and theories. Harari does an excellent job synthesizing a lot of information. What makes Sapiens exceptional is it gives culture a starring role in the human drama—something that many in science and sociology are hesitant to do, instead preferring to depict culture as transient, nebulous, and “soft.”

Yuval Noah Harari's Sapiens: A Brief History of Humankind Harari builds on some provocative ideas about Homo sapiens, but sets out his anthropological interpretations with vim and vigor:

The emergence of agriculture—especially livestock farming—is “the greatest crime in history … The domestication of animals was founded on a series of brutal practices that only became crueler with the passing of the centuries.” [Harari has said that he became a committed vegan while writing Sapiens.]
Organized religion is predictably contemptible, “You could never convince a monkey to give you a banana by promising him limitless bananas after death in monkey heaven.” The emergence of religion “was one of the most important revolutions in history, and made a vital contribution to the unification of humankind.” But the notion of supernatural being is increasingly inconsequential as humans are acquired divine abilities and relying increasingly upon ourselves for creating life forms and averting death and destruction. Then, “Is there anything more dangerous than dissatisfied and irresponsible gods who don’t know what they want?”
Consumer capitalism is a dreadful prison. “For better or worse, in sickness and in health, the modern economy has been growing like a hormone-soused teenager. It eats up everything it can find and puts on inches faster than you can count.”

Recommendation: Read Harari’s astonishing history of the species, from insignificant apes to rulers of the world

Yuval Noah Harari’s Sapiens: A Brief History of Humankind (2015) is a must-read. It is a brilliantly executed examination of who we are and of our behaviors. Notwithstanding the seeming overstatements and the occasional drift to sensationalism, Sapiens is extremely interesting and thought-provoking. It is written elegantly, in a clear and engaging style, with a skeptic’s eye and irreverent—and sometimes-sarcastic—sensibility.

We are more powerful than ever before…Worse still, humans seem to be more irresponsible than ever. Self-made gods with only the laws of physics to keep us company, we are accountable to no one.

Harari is implacably cold and literal, abstaining from political correctness and pro-Western predispositions. Sapiens concludes with spine-tingling predictions about the future. Perhaps as a cliffhanger to his subsequent Homo Deus: A Brief History of Tomorrow (2016,) Harari contends that we’re the primary destructive force.

Homo sapiens are sowing the seeds for our own destruction. The forthcoming biotechnological revolution, Harari speculates, may signal the end of sapiens. Bioengineered “amortal cyborgs” may replace us. These post-human organic and inorganic organisms won’t necessarily be immortal but, absent an accident, can live forever. Homo not so sapiens?

How to Embrace Uncertainty and Leave Room for Doubt

September 7, 2020 By Nagesh Belludi Leave a Comment

The value of sound decision-making is to be mainly sought from embracing uncertainty.

As the Presocratic philosopher Xenophanes proclaimed, “All we have is but a woven web of guesses.”

Physicist Richard Feynman on how doubt informs critical thinking and learning The physicist Richard P. Feynman often talked about how doubt informs critical thinking and learning. In a 1964 lecture on “What Is and What Should Be the Role of Scientific Culture in Modern Society,” published in The Pleasure of Finding Things Out (1999,) Feynman warned,

A scientist is never certain. We all know that. We know that all our statements are approximate statements with different degrees of certainty; that when a statement is made, the question is not whether it is true or false but rather how likely it is to be true or false. … Scientific knowledge is a body of statements of varying degrees of certainty — some most unsure, some nearly sure, none absolutely certain.

Science produces ignorance, and ignorance produces more science, reminded Feynman in a 1963 lecture on “The Uncertainty of Science” published in The Meaning of It All (1999,)

To solve any problem that has never been solved before, you have to leave the door to the unknown ajar. You have to permit the possibility that you do not have it exactly right. Otherwise, if you have made up your mind already, you might not solve it.

When the scientist tells you he does not know the answer, he is an ignorant man. When he tells you he has a hunch about how it is going to work, he is uncertain about it. When he is pretty sure of how it is going to work, and he tells you, “This is the way it’s going to work, I’ll bet,” he still is in some doubt. And it is of paramount importance, in order to make progress, that we recognize this ignorance and this doubt. Because we have the doubt, we then propose looking in new directions for new ideas. The rate of the development of science is not the rate at which you make observations alone but, much more important, the rate at which you create new things to test.

The Czechoslovakia-born Israeli American scientist Itzhak Bentov formulated the so-called “Bentov’s Law,” reiterating that science produces ignorance both deliberately and unintentionally. In Stalking the Wild Pendulum: On the Mechanics of Consciousness (1977,) Bentov wrote,

One’s level of ignorance increases exponentially with accumulated knowledge. When one acquires a bit of new information, there are many new questions that are generated by it, and each new piece of information breeds five-ten new questions. These questions pile up at a much faster rate than does accumulated knowledge. Therefore, the more one knows, the greater his level of ignorance.

How to Embrace Uncertainty and Leave Room for Doubt

Idea for Impact: If you can’t tolerate uncertainty and ambiguity, you may as well embrace a fanatical ideology.

Learning the boundaries of your knowledge—the shortcomings, caveats, hedges, and the standard deviations toward everything you think you know—hones decision-making.

In other words, to get to the right answers, you first have to ask the right questions. So the first thing is to ponder about is what questions to ask and how to ask them. What are the things you don’t know, and how can you reach out into these areas that may be new to you to uncover somethings about the world and yourself?

Once you discover the answers, you’ll realize that approximate statements and varying degrees of certainty will require you to think probabilistically. Your inquiry shouldn’t be, “Will I be right, or will I be wrong?” but rather “What is the probability of this scenario versus that, and how does this judgment impact my choices?”

Leave room for doubt, even in your highest conviction ideas. If not, you’ll risk becoming smug and self-satisfied.

The Power of Negative Thinking

May 21, 2020 By Nagesh Belludi Leave a Comment

Stoic philosophy recommends a practice called premeditatio malorum (“the premeditation of evils,”) i.e. intentionally visualizing the worst-case scenario in your mind’s eye.

The first point is to acknowledge that misfortunes and difficulties could, rather than certainly will, come about. The second is to envisage your most constructive response should the worst-case scenario transpire. For instance, if you’d lose your job due to coronavirus, what resources could you rely on, and how could you handle the consequences?

The direct benefit of premeditatio malorum is in taming your anxiety: when you soberly conjure up how bad things could go, you typically reckon that you could indeed cope. You’ll not dwell in the negative thoughts. Even the worst possible scenario couldn’t be so terrible after all.

Another surprising benefit of negative visualization is in raising your awareness that you could lose your relationships, possessions, routines, blessings, and everything else that you currently enjoy—but perhaps take for granted. This increases your gratitude for having them now.

Premeditatio Malorum in Stoicism - The Power of Negative Thinking

This Stoic exercise has an equivalent in Buddhist meditation-based mindfulness practices that encourage nonjudgmental awareness of unpleasant sensations (the vedanā.)

Your emotions, sensations, and events are in flux. They arise and pass. You’re merely to regard yourself as the observer of these thoughts and feelings, but you’re not to identify with them. You are not your thoughts … you are not your feelings. The Buddhist teacher Jack Kornfield writes in The Wise Heart: A Guide to the Universal Teachings of Buddhist Psychology (2015,)

Thoughts and opinions arise but they think themselves and disappear, “like bubbles on the Ganges,” says the Buddha. When we do not cling to them, they lose their hold on us. In the light of awareness, the constructed self of our identification relaxes. And what is seen is just the process of life, not self nor other, but life unfolding as part of the whole.

Idea for Impact: Could you benefit from reflecting on how you think of potential negative events?

An awareness of the possible—and the self-determining attitude—can be quite liberating. Premeditatio malorum is a surprisingly useful technique, if only with a scary name.

“What then should each of us say as each hardship befalls us? It was for this that I was exercising, It was for this that I was training,” as Epictetus philosophized in Discourses (3.10.7–8.)

The Biggest Disaster and Its Aftermath // Book Summary of Serhii Plokhy’s ‘Chernobyl: History of a Tragedy’

May 11, 2020 By Nagesh Belludi Leave a Comment

I visited the Chernobyl Exclusion Zone last year. This 2,600 sq km (1,000 sq mi) region spanning Ukraine and Belarus is the ghastly site of the greatest peacetime nuclear disaster in history. Yes, it’s safe enough to visit—with precautions, of course. [Read travel writer Cameron Hewitt’s worthwhile trip-report.]

Chernobyl is a gripping testimony to the perils of hubris and a poignant monument to the untold misery it imposed upon swathes of people.

To round out my learning from the trip, I recently read Chernobyl: History of a Tragedy (2019,) Harvard historian Serhii Plokhy’s haunting account of the nuclear disaster.

Abandoned City of Pripyat Near the Chernobyl Facility

An Accident That Was Waiting to Happen

At 1:21 A.M. on 26-April-1987, an experimental safety test at Unit 4 of the Vladimir Ilyich Lenin Nuclear Power Plant complex in Chernobyl went dreadfully wrong. The test instigated a power surge. The reactor exploded and burst open, spewing a plume of radioactive elements into the atmosphere.

The discharge amounted to some 400 times more radioactive material than from the Hiroshima atomic bomb. Deputy Chief Engineer Anatoly Dyatlov, who was in charge of the calamitous test, called the ensuing meltdown “a picture worthy of the pen of the great Dante.” Sixty percent of the radioactive fallout came to settle in Belarus. Winds carried radioactive elements all the way to Scandinavia.

Right away, hundreds of firefighters and security forces consigned themselves to stabilize the reactor and stop the fires from spreading to the other reactors. In so doing, they exposed themselves to fatal doses of radiation, spending the rest of their lives grappling with serious health problems.

The world first learned of the accident when abnormal radiation levels were detected at one of Sweden’s nuclear facilities some 52 hours after the accident. It took the Soviet regime three days to acknowledge the meltdown publicly, “There has been an accident at the Chernobyl atomic-electricity station.” Soviet leader Mikhail Gorbachev addressed the nation 18 days after the accident, “The first time we have encountered in reality such a sinister force of nuclear energy that has escaped control.”

A Soviet Dream Town Then, a Graveyard of Dreams Now

Abandoned Contaminated School In Chernobyl Exclusion Zone The ghost town of Pripyat, a purpose-built workers’ settlement a mile from the nuclear plant, seized my mind’s eye. It was one of the Soviet Union’s most desirable communities, and 50,000 people lived there when the accident happened. Today, it’s a post-apocalyptic time warp—full of all kinds of dilapidated civic structures that once showcased the ideal Soviet lifestyle.

Pripyat was evacuated entirely on the afternoon of the disaster. Left to rot, the town has been completely overtaken by nature. A Ferris wheel—completed two weeks before the explosion, but never used—has become an enduring symbol of the inflictions. So have unforgettable images of deserted houses engulfed by forest, loveable stray dogs in dire need of medical attention, and a day-nursery strewn with workbooks and playthings.

Abandoned Pripyat Amusement Park: Graveyard of Dreams

A Human Tragedy: Disaster, Response, Fallout

Chernobyl: History of a Tragedy (2019) is a masterful retelling of the episode and its aftermath. Author Serhii Plokhy, who leads the Harvard Ukrainian Research Institute, grew up 500 kilometers south of Chernobyl. He later discovered that his thyroid had been inflamed by radiation.

Plokhy offers deeply sympathetic portrayals of the plant’s managers and engineers, the first-responders who risked their lives to contain the damage, and the civilians in the affected areas of Ukraine and Belarus.

New Safe Containment of the Chernobyl Nuclear Power Plant Drawing upon the victims’ first-hand accounts as well as official records made available only after Ukraine’s 2013–14 Euromaidan revolution, Plokhy meticulously reconstructs the making of the tragedy—from the plant’s hasty construction to the assembly of the “New Safe Containment” structure installed in 2019.

The cleanup of the radioactive fallout could continue for decades. Robotic cranes will work in intense radiation and dismantle the internal structures and dispose of radioactive remnants from the reactors. The damage from the disaster may last for centuries—the half-life of the plutonium-239 isotope, one constituent of the explosion, is 24,000 years.

Design Flaws, Not All Operator Errors

Plokhy shows how Chernobyl personified the Soviet system’s socio-economic failings. Chernobyl was a disaster waiting to happen—an absolute storm of design flaws and human error.

The Chernobyl nuclear plant was hailed as a jewel in the crown of the Soviet Union’s technological achievement and the lynchpin of an ambitious nuclear power program. The RBMK (high power channel-type reactor) was flaunted as more powerful and cheaper than other prevalent nuclear power plant designs.

Inside Chernobyl's Radioactive Ruins - Engulfed by Forest Anatoliy Alexandrov, the principal designer of the RBMK reactor and head of the Soviet Academy of Sciences, reportedly claimed that the RBMK was reliable enough to be installed on the Red Square. The communist czars skimped on protective containment structures in a great hurry to commission the Chernobyl reactors.

Commissars at the Ministry of Medium Machine Building, the secretive agency in charge of the Soviet nuclear program, knew all too well of the fatal flaws in the design and the construction of the RBMK reactors. Viktor Briukhanov, the Chernobyl plant’s director, had complained, “God forbid that we suffer any serious mishap—I’m afraid that not only Ukraine but the Union as a whole would not be able to deal with such a disaster.”

Yet, the powers-that-were assumed that clever-enough reactor operators could make up for the design’s shortcomings. Little wonder, then, that the Soviets ultimately attributed the accident to “awkward and silly” mistakes by operators who failed to activate the emergency systems during the safety test.

Memorial to the Victims of the Chernobyl Disaster

The Fallings of the Soviet System’s Internal Workings

Chernobyl: History of a Tragedy dwells on Soviet leadership and the ubiquitous disconnects and the vast dysfunctions in the Soviet state’s affairs.

Chernobyl is a metaphor for the failing Soviet system and its reflexive secrecy, central decision-making, and disregard for candor. The KGB worked systematically to minimize news of the disaster’s impact. KGB operatives censored the news of the lethal radioactive dust (calling it “just a harmless steam discharge,”) shepherded the tribunal hearings, and downplayed the political outcomes of the disaster.

Even the hundreds of thousands of Ukrainians, Belarusians, and Russians evacuated from the thirty-kilometer zone weren’t given full details of the tragedy for weeks. In the days following the accident, the Communist Party’s apparatus, well aware of the risks of radiation, did not curtail children’s participation in Kyiv’s May Day celebrations and parades.

Author Plokhy’s most insightful chapters discuss the historic political fallout of the disaster. Moscow downplayed the design flaws in the reactor and made scapegoats of a handful of the plant’s engineers and operators—just three men received 10-year prison sentences in 1987. One of the three, Deputy Chief Engineer Anatoly Dyatlov (whom I quoted above referring to Dante,) was granted amnesty after only three years. He died five years later from a heart failure caused by radiation sickness.

Chernobyl’s outstanding narrative feature is the interpretation of the disaster in the framework of the fate of the Soviet Union. Plokhy explains how Chernobyl was a decisive trigger to the unraveling of the Soviet Union. Chernobyl served as an unqualified catalyst for Gorbachev’s policy of glasnost (“openness.”) Too, it fanned the flames of the nationalist movements in the soon-to-break-away republics of Ukraine, Belarus, and the Baltics.

Recommendation: Read This Captivating Account of a Great Human Tragedy

Serhii Plokhy’s Chernobyl: History of a Tragedy (2019) is a must-read record of human fallacies and hubris. It’s a poignant narrative of the courage and helplessness of the thousands of firefighters, police officers, doctors, nurses, military personnel, and the communities who risked their lives to mitigate the aftermath of the disaster, investigate, and “liquidate” the site. On top, Chernobyl is an edifying thesis on how the disaster accelerated the decline and the downfall of the Soviet Union.

It’s Probably Not as Bad as You Think: The 20-40-60 Rule

May 5, 2020 By Nagesh Belludi Leave a Comment

The 20-40-60 Rule, believed to be written by humorist Will Rogers for his movie Life Begins at 40 (1935,) states,

When you are 20, you care about what everybody thinks of you.
When you are 40, you don’t care about what people think of you,
and when you are 60, you actually understand that people were too busy thinking about themselves.

In essence, don’t agonize about what other people are thinking about you. They’re perhaps busy worrying over what you’re thinking about them.

The 20-40-60 Rule became popular when venture capitalist Heidi Roizen cited it (incorrectly attributing it to the actress Shirley MacLaine) at a 2014 lecture at Stanford. First Round Capital’s Review has noted,

People have enormous capacity to beat themselves up over the smallest foibles—saying the wrong thing in a meeting, introducing someone using the wrong name. Weeks can be lost, important relationships avoided, productivity wasted, all because we’re afraid others are judging us. “If you find this happening to you, remember, no one is thinking about you as hard as you are thinking about yourself. So don’t let it all worry you so much.”

Idea for Impact: Don’t Beat Yourself Up Over Your Mistakes

Chances are, people around you aren’t nearly as critical of you as you are of yourself. No one’s going to remember or care about your mistakes, and neither should you.

A Superb Example of Crisis Leadership in Action

May 4, 2020 By Nagesh Belludi Leave a Comment

It is in a crisis that leaders show their mettle. The New York Times notes,

The master class on how to respond [to a crisis] belongs to Jacinda Ardern, the 39-year-old prime minister of New Zealand. On March 21, when New Zealand still had only 52 confirmed cases, she told her fellow citizens what guidelines the government would follow in ramping up its response. Her message was clear: “These decisions will place the most significant restrictions on New Zealanders’ movements in modern history. But it is our best chance to slow the virus and to save lives.” And it was compassionate: “Please be strong, be kind and united against Covid-19.”

Leaders Show Their Mettle During a Crisis Our political leaders’ responses to the current COVID-19 crisis are particularly instructive about how leaders should act in a crisis:

Lead from the front. Initiate quick, bold, and responsible action, even when it carries political risk. Don’t be overcome by panic.
Think the crisis through. Weigh your options carefully, and then make the call confidently. Stay focused. Don’t let stress impede your problem-solving capabilities.
Avert an information vacuum. Any gap in the available information will be filled by guesswork and speculation.
Provide an accurate picture of what’s going on. Be transparent and honest right from the beginning. Acknowledging the gravity of the situation and being clear about how you’re going to collectively address the crisis leaves your constituencies with a sense of confidence in your message.
Choose your words carefully. Don’t create a false sense of security. Avoid making throwaway comments that might be misconstrued.
Communicate often. Fine-tune your message. Update your analysis and reaffirm your assurance of support. Keeping everyone in the loop diffuses fears and uncertainties.
Empower employees to be part of the solution. Invite and respond to employees’ feedback and concerns. They’ll need to know they’re being heard.

Idea for Impact: When a crisis hits, constituencies fall back on their leaders for information, answers, confidence, and direction. Set the appropriate tone for the organizational response by being supportive, factual, transparent, open-minded, calm, and decisive.

Five Where Only One is Needed: How Airbus Avoids Single Points of Failure

April 6, 2020 By Nagesh Belludi Leave a Comment

In my case study of the Boeing 737 MAX aircraft’s anti-stall mechanism, I examined how relying on data from only one Angle-of-Attack (AoA) sensor caused two accidents and the aircraft’s consequent grounding.

A single point of failure is a system component, which, upon failure, renders the entire system unavailable, dysfunctional, or unreliable. In other words, if a bunch of things relies on one component within your system, and that component breaks, you are counting the time to a catastrophe.

Case Study: How Airbus Builds Multiple Redundancies to Minimize Single Points of Failure

As the Boeing 737 MAX disaster has emphasized, single points of failure in products, services, and processes may spell disaster for organizations that have not adequately identified and mitigated these critical risks. Reducing single points of failure requires a thorough knowledge of the vital systems and processes that an organization relies on to be successful.

How Airbus Avoids Single Points of Failure Since the dawn of flying, reliance on one sensor has been anathema.

The Airbus A380 aircraft, for example, features 100,000 different wires—that’s 470 km of cables weighing some 5700 kg. Airbus’s wiring includes double or triple redundancy to mitigate the risk of single points of failure caused by defect wiring (e.g., corrosion, chafing of isolation or loose contact) or cut wires (e.g., through particles intruding aircraft structure as in case of an engine burst.)

The Airbus fly-by-wire flight control system has quadruplex redundancy i.e., it has five flight control computers where only one computer is needed to fly the aircraft. Consequently, an Airbus aircraft can afford to lose four of these computers and still be flyable. Of the five flight control computers, three are primary computers and two are secondary (backup) computers. The primary and the secondary flight control computers use different processors, are designed and supplied by different vendors, feature different chips from different manufacturers, and have different software systems developed by different teams using different programming languages. All this redundancy reduces the probability of common hardware- and software-errors that could lead to system failure.

Redundancy is Expensive but Indispensable

The multiple redundant flight control computers continuously keep track of each other’s output. If one computer produces deviant results for some reason, the flight control system as a whole excludes the results from that aberrant computer in determining the appropriate actions for the flight controls.

How Airbus Avoids Single Points of Failure: Multiple Redundant Flight Control Computers By replicating critical sensors, computers, and actuators, Airbus provides for a “graceful degradation” state, where essential facilities remain available, allowing the pilot to fly and land the plane. If an Airbus loses all engine power, a ram air turbine can power the aircraft’s most critical systems, allowing the pilot to glide and land the plane (as happened with Air Transat Flight 236.)

Idea for Impact: Build redundancy to prevent system failure from the breakdown of a single component

When you devise a highly reliable system, identify potential single points of failure, and investigate how these risks and failure modes can be mitigated.

For every component of a product or a service you work on, identify single points of failure by asking, “If this component fails, does the rest of the system still work, and, more importantly, does it still do the function it is supposed to do?”

Add redundancy to the system so that failure of any component does not mean failure of the entire system.

If you can’t build redundancy into a system due to some physical or operational complexity, establish frequent inspections and maintenance to keep the system reliable.

Postscript: In people-management, make sure that no one person has sole custody of some critical institutional knowledge, creativity, reputation, or experience that makes him indispensable to the organization’s business continuity and its future performance. If he/she should leave, the organization suffers the loss of that valued standing and expertise. See my article about this notion of key-person dependency risk, the threat posed by an organization, or a team’s over-reliance on one or a few individuals.

How A Single Point of Failure Became The Boeing 737 MAX’s Achilles Heel

January 7, 2020 By Nagesh Belludi Leave a Comment

Two thousand nineteen was one of the most turbulent years in Boeing’s history. Its 737 MACS (pardon the pun) troubles went from bad to worse to staggering when aviation regulators around the world grounded the aircraft and a steady trickle of disclosures increasingly exposed software problems and corners being cut.

The flaw in this aircraft, its anti-stall mechanism that relied on data from a single sensor, offers a particularly instructive case study of the notion of single point of failure.

One Fault Could Cause an Entire System to Stop Operating

How a Single Point of Failure, Reliance on one Angle-of-Attack Sensor, Created the Boeing 737 MAX Disaster A single point of failure of a system is an element whose failure can result in the failure of the entire system. (A system may have multiple single points of failure.)

Single points of failures are eliminated by adding redundancy—by doubling the critical components or simply backing them up, so that failure of any such element does not initiate a failure of the entire system.

Boeing Mischaracterized Its Anti-Stall System as Less-than-Catastrophic in Its Safety Analysis

The two 737 MAX crashes (with Lion Air and Ethiopian Airlines) originate from a late-change that Boeing made in a trim system called the Maneuvering Characteristics Augmentation System (MCAS.)

Without the pilot’s input, the MCAS could automatically nudge the aircraft’s nose downwards if it detects that the aircraft is pointing up at a dangerous angle, for instance, at high thrust during take-off.

Reliance on One Sensor is an Anathema in Aviation

The MCAS was previously “approved” by the Federal Aviation Administration (FAA.) Nevertheless, Boeing made some design changes after the FAA approval without checking with the FAA again. The late-changes were made to improve MCAS’s response during low-speed aerodynamic stalls.

The MCAS system relied on data from just one Angle-of-Attack (AoA) sensor. With no backup, if this single sensor were to malfunction, erroneous input from that sensor would trigger a corrective nosedive just after take-off. This catastrophe is precisely what happened during the two aircraft crashes.

The AoA sensor thus became a single point of failure. Despite the existence of two angle-of-attack sensors on the nose of the aircraft, the MCAS system not only used data from either one of the sensors but also did not expect concurrence between the two sensors to infer that the aircraft was stalling. Further, Lion Air did not pay up to equip its aircraft with a warning light that could have alerted the crew to a disagreement between the AoA sensors.

Boeing Missed Safety Risks in the Design of the MAX’s Flight-Control System

Reliance on one sensor’s data is an egregious violation of a long-standing engineering principle about eliminating single points of failure. Some aircraft use three duplicate systems for flight control: if one of the three malfunctions, if two systems agree, and the third does not, the flight control software ignores the odd one out.

Angle-of-Attack Sensor on Boeing 737 MAX Aircraft - Single Point of Failure that Contributed to Two Crashes If the dependence on one sensor was not enough, Boeing, blinded by time- and price-pressure to stay competitive with its European rival Airbus, intentionally chose to do away with any reference to MCAS in pilot manuals to spare pilot training for its airline-customers. Indeed, Boeing did not even disclose the existence of the MCAS on the aircraft.

Boeing allows pilots to switch the trim system off to override the automated anti-stall system, but the pilots of the ill-fated Lion Air and Ethiopian Airlines flights failed to do so.

Idea for Impact: Redundancy is the Sine Qua Non of Reliable Systems

In preparation for airworthiness recertification for the 737 MAX, Boeing has corrected the MCAS blunder by having its trim software compare inputs from two AoA sensors, alerting the pilots if the sensors’ readings disagree, and limiting MCAS’s authority.

One key takeaway from the MCAS disaster is this: when you devise a highly reliable system, identify all single points of failure, and investigate how these risks and failure modes can be mitigated. Examine if every component of a product or a service you work on is a single point of failure by asking, “If this component fails, does the rest of the system still work, and, more importantly, does it still do the function it is supposed to do?”

« Previous Page

Risk

Wondering what to read next?

Wondering what to read next?

Homo sapiens’ talent for abstraction set us apart

Recommendation: Read Harari’s astonishing history of the species, from insignificant apes to rulers of the world

Wondering what to read next?

The value of sound decision-making is to be mainly sought from embracing uncertainty.

Idea for Impact: If you can’t tolerate uncertainty and ambiguity, you may as well embrace a fanatical ideology.

Wondering what to read next?

Wondering what to read next?

An Accident That Was Waiting to Happen

A Soviet Dream Town Then, a Graveyard of Dreams Now

A Human Tragedy: Disaster, Response, Fallout

Design Flaws, Not All Operator Errors

The Fallings of the Soviet System’s Internal Workings

Recommendation: Read This Captivating Account of a Great Human Tragedy

Wondering what to read next?

Idea for Impact: Don’t Beat Yourself Up Over Your Mistakes

Wondering what to read next?

Wondering what to read next?

Case Study: How Airbus Builds Multiple Redundancies to Minimize Single Points of Failure

Redundancy is Expensive but Indispensable

Idea for Impact: Build redundancy to prevent system failure from the breakdown of a single component

Wondering what to read next?

One Fault Could Cause an Entire System to Stop Operating

Boeing Mischaracterized Its Anti-Stall System as Less-than-Catastrophic in Its Safety Analysis

Reliance on One Sensor is an Anathema in Aviation

Boeing Missed Safety Risks in the Design of the MAX’s Flight-Control System

Idea for Impact: Redundancy is the Sine Qua Non of Reliable Systems

Wondering what to read next?