Designed for No One: The Hidden Danger of AveragesProfessor David Puelz on the statistical case against sameness.“Averages are abstractions, variation is reality.” A mentor from graduate school introduced me to this idea, and I never realized how often I’d encounter its many incarnations. In a world bursting with complexity, a simple summary in the form of an average is the ultimate temptation. In this article, I describe how an average is at best an information-starved summary, and at worst it can lead to critical miscalculations. The United States Air Force after World War II experienced several plane accidents. Engineers ruled out equipment failure, and the pilots were expertly trained. The Air Force struggled to determine what was happening.¹ During the investigation, a young and contrarian lieutenant named Gilbert Daniels decided to take ten body measurements from 4,063 members of the Air Force’s flying personnel, including their height, arm length, chest circumference, and reach. He then asked the simple question: How many of these men were close to average on all ten? Daniels was generous about what counted as average, allowing a broad middle band on each measure. The answer was zero. Even on just three of the ten measures, fewer than four men in a hundred qualified.² The Air Force had measured thousands of real men, calculated their dimensions with perfect accuracy, and used those numbers to design cockpits for pilots that would never sit at the controls. The problem was never that the Air Force had calculated the wrong average. The problem was that it had mistaken the average for a man. Now to finance. Suppose an investment gains fifty percent one month and loses fifty percent the next. You invest one hundred dollars, it grows to $150 and then drops by half to $75. The average return over those two months is zero, but you lost twenty-five percent of your money. What happened? Because wealth compounds, variation in returns is a drag on it. This variation is known as risk. While a simple average says you broke even, it is deceiving you when compounding and risk sneak through the back door and sink your bank account. Finally, a turn to public health. When I taught at The University of Texas at Austin (UT) during the COVID-19 pandemic, my students and I did something simple. We graded the forecasts. UT had built a model to guide Austin’s staged COVID alert system, but it made systematic mistakes. It overpredicted hospitalizations, deaths, and ICU capacity, and it overpredicted exactly when it mattered most, i.e., during the surges.³ Overprediction means the models forecasted alarmingly large numbers of hospitalizations, deaths, and ICU visits when the realized numbers came in much lower and manageable. Why? Part of the reason was model design. It assumed entire populations divide into tidy “compartments” of disease status and meet, mix, and spread COVID-19 at an average rate. The critical problem: Pandemics don’t happen to averages. People are not well mixed, and those differences matter. The virus’s fatality risk, or the chance of dying from COVID-19 once infected, spanned nearly four orders of magnitude with age, from about two thousandths of one percent for a child to roughly fifteen percent for someone in their mid eighties. The single average risk the country argued about was a figure almost no one actually faced. We debated the average patient the way the Air Force had built cockpits for the average pilot. The average patient did not exist either. The silver lining in these examples is realizing where the true information lives. Variation in cockpit design–like an adjustable car seat–decreased the Air Force’s plane crashes. Variation in investment returns characterizes how quickly and asymmetrically your wealth rises and falls. Variation in populations across borders and nations governs how pandemics spread. Averages abstract away from variation critical to understanding complex systems. This revelation represents the statistical case for President Carvalho’s “defense of inequality” during our second convocation. It is not a defense of privilege. It is a defense of difference. We demand academic excellence only, and we thrive on the variation derived from not asking our applicants and students to conform. A UATX education cultivates differences rather than erasing them. Other elite institutions push for conformity and anonymity–they worship the average. They are building a cockpit that fits no one and treating a patient who will never exist. Here is what I have come to believe. The average is what you reach for when you have given up on understanding the structure of a thing. It is a confession dressed as a summary. Heterogeneity, the plain fact that people and firms and risks differ in ways that matter, is not noise you failed to average away. It is the signal. Statistics gives us the tools, evidence gives us the clues, and science is an ongoing process of learning which differences are erroneous and which ones matter. The Air Force figured that out in 1952 and built a seat that moves. The university where I teach made the same wager about people, that free minds and real excellence grow from making room for difference, not sanding it down to the mean. Most of the world is still bolting us to a man who does not exist. Build the seat that moves. Then let people sit where they actually are. David Puelz teaches at the University of Austin. He works on computational methods for applied data analysis in finance, medicine, and public policy. *The views expressed in this essay are those of the author and do not necessarily reflect the views of the University of Austin (UATX), its faculty, staff, leadership, or trustees. Public Events This FallAll events are on campus in downtown Austin and free to the public unless otherwise noted.
|