Looking for the Silent Data
It’s easy to overlook the small details that tell the true story.
One such case was during World War II when the Allies were examining damaged bomber planes that returned from sorties. Looking at where damage was taken, it was assumed these were the areas that needed the most protection. However, statistician Abraham Wald correctly noted that their findings were only on the planes that survived a few hits and returned.
What needed to be reinforced was the areas where there was no damage — typically the engines — as these were the vulnerable parts where one hit was all it took. Those planes never returned to be studied and their data became silent.
This is called Survivorship Bias — a focus on the people or materials that made it through a selection process while ignoring those that did not.
Survivorship bias exists in a surprising amount of forms in our life without us ever realizing it. Ever hear someone complain that things (vehicles, appliances, music, etc) are not made like they used to be? That’s because we’re only seeing the survivors, not the failed models that have long since been forgotten.
So for every wealthy start-up founder, there are 100 other entrepreneurs who end up with only a cluttered garage.
DAVID COWAN OF BESSEMER VENTURE PARTNERS
Being able to recognize survivorship bias is part of thinking a bit more critically of what we’re presented instead of taking it at face value. Not being afraid to ask a few more questions and get to the real answers, whatever they may be.
So, next time you hear a fantastic story of how someone stumbled their way to success, maybe remind yourself that correlation does not prove causality and to have a deeper look at the data.