One of the greatest examples of Survivorship Bias came from WW2.
During WWII, countries had to solve many mathematical and strategic tasks to succeed in winning and surviving war. One of those problematic assignments was to find ways of improving military aircraft so they would be more resistant to enemy fire. While statisticians struggled to find the best way to protect the planes, one man named Abraham Wald had a genius idea that is still implemented to this day.
The Navy produced a diagram that depicted all of the locations where Bombers were usually damaged during combat missions. After receiving this diagram, Engineers had the task of reviewing where additional armour was required. But before they could do this, Wald shook the establishment by disagreeing with the statisticians method of analysis.
Engineers were asked to evaluate and determine which sections of the Bombers needed to be up-armored to minimize the damage from enemy fire and lower the numbers of Bombers being shot down.
According to Wald, the statisticians were only looking at the planes that came back, meaning that the damage was not critical. Wald pointed out that they should do the exact opposite of what the Navy was planning to do. According to him, they should understand that the undamaged areas on the diagram were the reason that the aircraft were able to make it back.
There is a tendency to focus on the survivors rather than whatever you would call a non-survivor depending on the situation. After any event that leaves behind survivors, the non-survivors are often destroyed or removed. If the failure becomes invisible, then naturally, we will focus on the success.
This logical error is still rather popular in modern society. For example, when we talk about developing new business, we often list similar business models that already succeeded. Then we point out the difficulties that these businesses endured to avoid the same mistakes. What this type of thinking makes us forget is that there are tons of similar business models that have failed. We should consider the reasons of why they are not in the market anymore, instead of trying to fix and improve models that were already successful. With this in mind, if we look back at what Wald was offering the Navy, it is to consider that aircraft with crucial damage did not make it back home and that they are only looking at the survivors.
How does this relate to customer & client feedback you might ask?
When you only gather feedback from customers / clients that view you as a great company to work with (survivors), you are in turn giving yourself a pat on the back. All very well if you feel like inflating your ego, but this is the same as Survivorship Bias. What you really need to do, is seek feedback from those who no longer use your services (non – survivors), and really find out why they never came back. It is only then you will get an accurate picture of your company that will help you grow.
Interested In finding out how we can undertake this feedback service for you?
Get in touch at firstname.lastname@example.org.