I’ve just come to understand one of the core differences between IT and medicine: most medics know how to arrive at a conclusion based on evidence but most people working in Business and IT don’t. As a result far too many decisions in Business and IT are based on something that looks like evidence, sounds like evidence, but actually isn’t evidence. And I don’t know about you, but I find that slightly sickening.
What’s my evidence for this view? (Ha!) It’s an internal document I’ve just been reading which says:
A conservative estimate of £10m p.a. colleague efficiency savings would be realised through deploying SuperSystem to 25,000 users across the organisation.
It includes no references. There’s no way for me to come to my own conclusion. No links to any validatable and verifiable studies showing what savings have been achieved elsewhere. Da nada. Nothing. As a reader, I have to trust the writer’s judgement. The older I get, the less I like that.
It’s not just a matter of being given the evidence though. It’s a matter of being able to evaluate it. Consider this from a campaigning website:
Dihydrogen Monoxide (DHMO) is a colorless and odorless chemical compound, also referred to by some as Dihydrogen Oxide, Hydrogen Hydroxide, Hydronium Hydroxide, or simply Hydric acid. Its basis is the highly reactive hydroxyl radical, a species shown to mutate DNA, denature proteins, disrupt cell membranes, and chemically alter critical neurotransmitters. The atomic components of DHMO are found in a number of caustic, explosive and poisonous compounds such as Sulfuric Acid, Nitroglycerine and Ethyl Alcohol.
Now if you’ve an iota of common sense you’d want it banned, wouldn’t you? Ha! Fooled ya (unless you were paying attention in chemistry lessons of course). It’s water. The DHMO site is there to show how easy it is to baffle with pseudo-scientific bull.
In Business and IT this is done all the time. I’ve done it myself. I’ve found some statistics that seem to support my case and bunged them on to a slide with a footnote saying “Source: Gartner” or wherever. It’s an empty ritual: no-one makes any pretence of reading the study or critically evaluationg Gartner’s data and conclusions. We go through the ritual so we can all pretend we’re being objective and rigorous, but in fact we’re not.
This flaccid acceptance of rubbish is widespread. Some time ago I was in a meeting where the discussion went as follows:
||Our target is that 80% of balls will be blue
||So that means they’re pie-bald and 80% of the surface of each ball will be blue?
||Er, 80% of the balls will be blue
||Oh I see – you mean that for every 100 balls, 80 of them will be entirely blue?
||(Snarling) … It’s just a turn of phrase…
It was all I could do not to snarl back “No it isn’t a turn of phrase – it’s a number” but the other people in the meeting weren’t interested. Which only goes to prove my point, really.
Imprecision, innumeracy, lack of scepticism, lack of critical thinking, call it what you will: it lures us in to blarney and over-optimism. Yes, it’s hard to be rigorous and objective because in Business and IT by and large we don’t have standardised tools like double blind controlled trials to base our decisons on. There are exceptions of course, like the the rules-based numeracy required of accountants and the occasionally rigorous testing of direct marketing campaigns. But neither of those apply to the world of IT.
Wherever we work, we cannot give up the responsibility of using our brains and for seeking out and using the best analysis and measurement tools we can find. The problem is too many people charged with making decisons lack the habit of critical thinking which you need to cut through this sort of fluff. It’s a cultural issue too: why was I the only person in the meeting who was outraged that a proverb was presented as a metric? Because I don’t work for an engineering company, for a start. Finally, the fact that it’s difficult is no excuse for giving up entirely and relying on “street smarts” “judgement” “instinct” or “gut”. It was gut instinct that led medics to grind up spiders with dung and rub them in to people’s eyes to cure blindness.