Lies and Damned Lies in One Statistic
Evolution of a Misleading Infographic
Posted by Charlie Recksieck
on 2021-10-14
This might seem like a low-stakes story, but there's a disturbing truth in this about how bad information and particularly bad statistics can calcify and seem true when the little lie is repeated enough times.
Twain or Disraeli?
Mark Twain famously wrote this about statistics:
"Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: ’There are three kinds of lies: lies, damned lies, and statistics.' "
Funny enough that even in this famous quote questioning the veracity of things, Mark Twain himself is attributing a statement to British Prime Minister Benjamin Disraeli, even though there is no evidence of Disraeli saying this. But still.
This expression basically means that the quoting of a statistic, even when that figure may be true, can be used in a context to intentionally (or unintentionally) mislead.
What I Saw Originally
Somewhere in my Twitter feed in July, I saw somebody post this infographic of one simple statistic.
You can read it for yourself, but you see that they are postulating that only 2.5% of companies "successfully" complete their projects.
Those of us who have dealt with (or worked in) corporate America might feel that this incredibly negative view of workplace efficiency seems true. But read it again. That number can't possibly be right.
That's what I bumped on when I first read it. As you can see from the image, that this graphic was made by "The Scalers", even though I saw this posted by somebody besides them. As it turns out, there were 5 degrees of separation from the original statement until me seeing this goosed-up misleading declaration. Below is the path, which is a simple game of telephone.
1 - The Real HR.com Statistic
I was able to trace the original source through various citations and references notes over the years. The original point and true statement was from a 2004 report from the people at HR.com
"Only 2.5% of Global Businesses Achieve 100% Project Success."
That is a very different statement. Even ignoring the keyword "Global" in there, the bigger conditional operative in the sentence is "100%". A tiny amount of companies or organizations achieve 100% success.
In the HR.com report, 100% success was defined as meaning all projects were reported as "on time, within budget, to scope and with the right business benefits".
2 - Price-Waterhouse
The big amplifier for this statistic or damned lie came in 2014 when Price, Waterhouse & Cooper included a take on the factoid in a 2014 presentation.
They listed the number of "companies that successfully complete 100% of their projects" as 2.5%.
Although they don't cite the HR.com report directly, it's the same 2.5% but a change in wording from "100% project success" to "complete 100% of their projects." Right away the bar is lowered. But it not only comes from respected Price Waterhouse Cooper, but it has the ring of something shocking you can say in a business context that people will agree with you on.
3 - The Scalers
Now we move on to the people who make the infographic. It came from a company called "The Scalers" who specialize in offshore outsourcing and management. One can imagine that they are trying to emphasize a point that it's incredibly rare for any company to do things right, so your company could certainly use their help with an outsourcing solution.
In a 2019 report from The Scalers, they say "FACT - A PwC study of over 10,640 projects found that only 2.5% projects were completed successfully. The rest either failed to meet some of their original targets, missed the original budget or deadlines. (Source: Gallup)"
The Scalers report cited in that URL above gets it right. But look at the difference between "only 2.5% projects were completed successfully" vs. the "Only 2.5% of companies successfully complete their project.
4 - Whoever Posted This Graphic On Twitter
This post of the infographic as I read it in 2021, had even less context. It was used as a shocking-but-feels-true "fact" highlighting that companies, project management teams and individual workers just don't get the job done.
That's the one that I came across. The words "all" or "100%" was missing. Saying that "2.5% of all companies successfully complete their projects" is a far, far cry from saying "2.5% of companies successfully complete all of their projects."
One word makes a hell of a difference.
Why Am I Going To This In Such Depth
Over 800 words in, I think I've forgotten my exact point why I felt so hoodwinked about this. But in a media age where "truthiness" and presenting your cocktail party arguments with third-hand information filtered through a game of telephone ... well, personally I think this is all killing us.
As somebody once said, "Facts matter". I can't tell you the first person to have said that. That would require me to go down a whole other rabbit hole.
I guess I'm saying let's all be careful of what statistics we use instead of numbers we pull out of our, um, hats start to seem like the truth just because none of us ever verify them.