Numbers Games: Summary of The Tyranny of Metrics by Jerry Muller

The Tyranny of Metrics (2018) by Jerry Muller, a historian at The Catholic University of America, is a poignant reflection on our society’s obsession with measurement. Muller’s eloquent arguments shed light on the dual nature of metrics—they can be valuable diagnostic tools, yet their misuse as the sole measure of success and tied to rewards poses significant problems. Drawing upon many empirical examples across various fields, Muller skillfully reveals the inherent pitfalls of our reliance on metrics.

Consider the initial allure of measuring and publicly disclosing the success rates of surgeons performing specific procedures. At first glance, this transparency appears beneficial, empowering patients to make informed decisions. However, a disheartening trend emerged once these performance scorecards entered the public domain. Surgeons, fearing a decline in their reported success rates, started avoiding the most complex cases. Shockingly, even cardiac surgeons refused to operate on critically ill patients, jeopardizing lives to protect their perceived success.

Muller further elucidates the case of hospital emergency rooms, where the pursuit of improving the metric for timely patient admission became paramount. In a tragic turn of events, the desire for statistical accolades overshadowed the urgent needs of the suffering. Ambulances formed a distressing queue outside the facility as the metric was manipulated, leaving genuine care and compassion languishing in the background.

In 2009, when Medicare implemented public reporting and penalties for hospitals with higher-than-average 30-day readmission rates, hospitals resorted to manipulating the metric. They cleverly distorted the numbers by categorizing many readmitted patients as outpatient services, concealing them and evading penalties.

Education, too, falls victim to the obsession with metrics. The relentless focus on using metrics to influence teacher retention or determine school closures has given rise to a phenomenon known as “teaching to the test.” Educators find themselves trapped, compelled to prioritize teaching subjects aligned with standardized exams, such as math and English while neglecting crucial skills like critical reading or crafting extended essays. Pursuing metric-driven success inadvertently sacrifices holistic education on the altar of narrow measurement.

During the Vietnam War, the US Defense Secretary introduced the “body count” metric. This idea, advocated by US Defense Secretary Robert McNamara, suggested that a higher number of enemy casualties indicated greater success and brought the US closer to victory. However, an unintended consequence emerged when the body count became an informal measure for ranking military units and determining promotions. In this dangerous pursuit of numbers, the metric lost touch with reality, often inflated to fulfill the desire for perceived success. Counting bodies became a precarious military objective in and of itself, overshadowing the true essence of the conflict.

Muller’s perspective does not advocate completely disregarding metrics as a management tool. Instead, he emphasizes the importance of utilizing meaningful and comprehensive metrics that contribute to informed decision-making. He distinguishes between measurable aspects and measurements that hold true significance.

To achieve this, Muller discourages starting with the metric itself. Instead of asking, “What metrics should we use?” he suggests a more practical approach: “What are we trying to accomplish?” This approach involves establishing goals and metrics that evaluate achieving desired outcomes and customer satisfaction (effectiveness) while efficiently utilizing available resources.

In a compelling concluding chapter, Muller encapsulates his central thesis with a resounding declaration: “Measurement is not an alternative to judgment; measurement demands judgment.” This statement emphasizes the need to make informed decisions regarding whether to measure, what to measure, how to interpret the significance of measurements, whether to assign rewards or penalties based on results and who should have access to the measurements.

Recommendation: Skim Tyranny of Metrics. This tome serves as an authoritative guide for comprehending the profound influence of numerical indicators on the very foundation of modern society. It should be considered essential reading for anyone seeking to understand why organizations often operate below their full potential.