The dangerous errors in business analysis are not arithmetic mistakes. They are biases - situations where every number is correct and the conclusion is still wrong, because of what the data quietly excludes.
1. Survivorship bias
What it is: analysing only what survived. Studying your current customers to understand loyalty, or your retained staff to understand engagement.
Why it misleads: the group that left holds the answer you need. Your loyalty analysis is a study of people who did not have the problem.
The catch question: who is not in this dataset because they left?
2. Selection and self-selection bias
What it is: the people in your data chose to be there. Survey respondents, reviewers, webinar attendees.
Why it misleads: those with strong opinions respond at much higher rates, so your data is bimodal and unrepresentative. Online reviews are the extreme case - delighted and furious, almost nobody in between.
The catch question: what was the response rate, and how might responders differ from the rest?
3. Confirmation bias
What it is: looking for evidence supporting the view you already hold, and scrutinising contrary evidence more harshly.
Why it misleads: it is invisible from inside. You will feel like you are being rigorous.
The catch question: what did we decide in advance would change our mind?
4. Simpson's paradox
What it is: a trend that appears in aggregate data and reverses in every subgroup - or vice versa.
Why it misleads: it is genuinely counter-intuitive and appears frequently in real business data, most often when group sizes differ substantially.
The catch question: does this pattern hold when I segment the data?
5. Regression to the mean
What it is: extreme results naturally drift back towards average, with or without intervention.
Why it misleads: you intervene with your worst-performing region, it improves, and you credit the intervention. It would probably have improved anyway. This one is responsible for an enormous amount of unwarranted confidence in management interventions.
The catch question: what would have happened if we had done nothing?
6. Look-elsewhere effect (multiple comparisons)
What it is: testing many things and reporting whichever came out significant.
Why it misleads: test twenty segments at the 5% threshold and you expect one false positive by chance alone.
The catch question: how many things did we look at before finding this?
7. Base rate neglect
What it is: ignoring how common something is to begin with.
Why it misleads: a screening test that is "95% accurate" for a condition affecting 1% of people produces mostly false alarms. The same logic applies to fraud detection, churn prediction and candidate screening tools.
The catch question: how common is this in the population to start with?
8. Recency and salience bias
What it is: the vivid recent case dominating your sense of the whole.
Why it misleads: one furious client outweighs 300 quietly satisfied ones in the room.
The catch question: what does the full period show?
The habit that catches most of them
Before accepting a conclusion, ask the three universal questions: who is missing, does it hold when segmented, and what would have happened anyway? See also the statistical mistakes that come from technique rather than bias.
The Statistics for Business course covers bias and study design alongside the methods. Enquire today.