I read this yesterday. I think it applies here. From the Scibabe moment of science Facebook page.
Today’s Moment of Science… Survivorship bias & phantom bullet holes.
We like to think that we’re rational thinkers, able to spot logical fallacies out in the wilderness. But what happens when the cause of your logical fallacy is information that’s long gone? This is a kind of selection bias known as survivorship bias, and it’s annoyingly common.
“Old houses were better constructed.” Or you only see the sturdiest of the old ones today because nobody made an effort to save the 14th century shacks.
“Music from back in the day is standing the test of time.” There’s caterwauling that deserves its fate in the bowels of music history from every era, I’m afraid.
“You can become a billionaire without going to college if you just work hard and read books, look at this tech bro who I think is irl Tony Stark.”
Truly, I enjoy an inspiring story about not having student loan debt as much as the next millennial going through an existential crisis. But every story of someone who got rich or famous tells the quieter story of far more people who wanted it just as badly, tried, and failed.
So, airplanes.
Abraham Wald was a Hungarian Jew who fled Europe for the US in the late 1930s because of that whole ‘Hitler was a murderous cockface’ thing. He had his PhD in mathematics and went to work for the Statistical Research Group (SRG) at Columbia University. The SRG has been described as a collection of the “most extraordinary group of statisticians ever organized.”
Planes kept coming back from the war with bullet holes in a certain pattern, if they came back at all. Damage clustered around the wing tips and tail. Weight needed to be kept low while giving the plane more protection, and the areas that had sustained a lot of fire seemed like the obvious places to armor up. Right?
But Wald suggested that planes were likely to be hit fairly evenly, not that the enemy fire was clustering around these spots tactically. He considered the data he didn’t have: the planes that never came home. Where were they being hit?
If he was right that planes took fire everywhere, then the planes should be armored heavily where the surviving planes hadn’t taken damage. The engine and the cockpit came back seemingly unscathed every time in otherwise bullet ridden planes. Those areas were subsequently fortified.
It’s a good lesson in survivorship bias. And saved untold lives.
Abraham Wald’s work contributed considerably to the burgeoning field of operational research. Because of his immigration status, he reportedly wasn’t able to get a security clearance or even look at the final reports he contributed to. He died in a plane crash in 1950 on the way to India for a lecture tour at just 48 years old.
This has been your Moment of Science, never sure I have all the data.
To get the MOS delivered to your inbox every weekday with rocket scientist orgies, NASA funded dolphin handjobs, and friggin... Australia... head to patreon.com/scibabe.
March 3, 2022, 12:26 p.m. - Andy Eunson
I read this yesterday. I think it applies here. From the Scibabe moment of science Facebook page. Today’s Moment of Science… Survivorship bias & phantom bullet holes. We like to think that we’re rational thinkers, able to spot logical fallacies out in the wilderness. But what happens when the cause of your logical fallacy is information that’s long gone? This is a kind of selection bias known as survivorship bias, and it’s annoyingly common. “Old houses were better constructed.” Or you only see the sturdiest of the old ones today because nobody made an effort to save the 14th century shacks. “Music from back in the day is standing the test of time.” There’s caterwauling that deserves its fate in the bowels of music history from every era, I’m afraid. “You can become a billionaire without going to college if you just work hard and read books, look at this tech bro who I think is irl Tony Stark.” Truly, I enjoy an inspiring story about not having student loan debt as much as the next millennial going through an existential crisis. But every story of someone who got rich or famous tells the quieter story of far more people who wanted it just as badly, tried, and failed. So, airplanes. Abraham Wald was a Hungarian Jew who fled Europe for the US in the late 1930s because of that whole ‘Hitler was a murderous cockface’ thing. He had his PhD in mathematics and went to work for the Statistical Research Group (SRG) at Columbia University. The SRG has been described as a collection of the “most extraordinary group of statisticians ever organized.” Planes kept coming back from the war with bullet holes in a certain pattern, if they came back at all. Damage clustered around the wing tips and tail. Weight needed to be kept low while giving the plane more protection, and the areas that had sustained a lot of fire seemed like the obvious places to armor up. Right? But Wald suggested that planes were likely to be hit fairly evenly, not that the enemy fire was clustering around these spots tactically. He considered the data he didn’t have: the planes that never came home. Where were they being hit? If he was right that planes took fire everywhere, then the planes should be armored heavily where the surviving planes hadn’t taken damage. The engine and the cockpit came back seemingly unscathed every time in otherwise bullet ridden planes. Those areas were subsequently fortified. It’s a good lesson in survivorship bias. And saved untold lives. Abraham Wald’s work contributed considerably to the burgeoning field of operational research. Because of his immigration status, he reportedly wasn’t able to get a security clearance or even look at the final reports he contributed to. He died in a plane crash in 1950 on the way to India for a lecture tour at just 48 years old. This has been your Moment of Science, never sure I have all the data. To get the MOS delivered to your inbox every weekday with rocket scientist orgies, NASA funded dolphin handjobs, and friggin... Australia... head to patreon.com/scibabe.