None of these mistakes require statistical sophistication to avoid. They require someone in the room willing to ask an awkward question. Here are the ten that cost the most money.
1. Treating noise as a trend
The most expensive error in business. A metric moves, a meeting is called, an initiative is launched - and the movement was ordinary week-to-week variation. The fix: know the normal range of every metric you report on before you interpret any change in it.
2. Quoting an average with no spread
"Average resolution time is four hours" hides the fact that one ticket in twenty takes three days - and those are the ones generating complaints. The fix: always report a percentile alongside the mean. The 90th percentile is usually where the business problem lives.
3. Survivorship bias
Analysing only your current customers, retained staff or successful projects. Your loyalty data excludes everyone who left, which is precisely the group you needed to understand. The fix: ask who is missing from every dataset before analysing it.
4. Confusing correlation with causation
Stores with more staff have higher sales; therefore add staff. Or: busy stores were given more staff. The fix: when a correlation is about to drive spending, test it deliberately with a controlled comparison.
5. Percentages of tiny bases
"Enquiries from that campaign convert at 50%" - two out of four. The fix: always publish the denominator. Always.
6. Comparing counts when you need rates
More complaints this year is meaningless if you served 40% more customers. The fix: normalise. Per customer, per order, per thousand transactions.
7. Cherry-picking the comparison period
Growth looks excellent measured from the worst month last year. The fix: fix your comparison convention in advance and stick to it, and show the full series rather than two points.
8. Stopping a test when it looks good
If you check a test daily and stop the moment it turns significant, you will generate false positives reliably. The fix: decide the duration and sample size before starting, then leave it alone. This is p-hacking, whether or not it is intentional.
9. Reading a survey that only enthusiasts answered
Response rates below about 20% make non-response bias the dominant feature of your results. Satisfied and dissatisfied customers respond at different rates. The fix: report the response rate prominently and treat the results as indicative, not representative.
10. Confusing statistical significance with importance
A significant 0.1% improvement may not be worth implementing. The fix: always report the effect size and its confidence interval alongside significance, and ask whether the smallest plausible value in that interval would still justify the work.
The one habit that prevents most of these
Before accepting any number, ask three questions: what is the denominator, who is missing, and how much does this normally vary? That takes fifteen seconds and catches the majority of the errors above.
Building the reflex across a team
These mistakes persist in organisations where nobody feels qualified to challenge a chart. Fixing that is a training problem - the Statistics for Business course at London School of Business UK is designed to give managers exactly that confidence. Enquire today.