The doctor-patient relationship is based on specific decisions. Alice is sick. So what’s on her mind when she visits the hospital? Firstly, she wants a diagnosis. What is wrong with her? Then she wants to know the best treatment to reduce her symptoms and return her to health. Perhaps she hopes for a cure or at least to be well enough to return to work. All these things apply to Alice personally. Alice is not interested in the best treatment for the average person with a similar condition. The typical treatment could be penicillin, which might kill Alice because of her allergy.
The Statistics Don’t Work
Evidence-Based Medicine (EBM) is based on a statistical fallacy. The ecological fallacy is a formal fallacy fundamental to statistical decision-making. It states
You can’t apply group statistics to the individual.
Ordinary people know this intuitively. Let’s take an example. The average professional musician earns perhaps $75,000 a season and has up to 1,000 dedicated fans. Actually, I have no idea what typical musicians make. But whatever the numbers, these incomes didn’t apply to Elvis Presley, the Beatles, or ABBA. You can’t use group statistics to predict an individual. It just doesn’t work, and this should be mind-numbingly obvious. Here is a BMJ article describing the problem, “Ecological studies make large-scale comparisons between groups of people.” Moreover, these “studies are prone to the ecological fallacy”. Notably, EBM aims for medicine based on large-scale comparisons between groups.
I used to build artificially intelligent systems, such as neural networks. Machine intelligence and associated systems use targeted information and don’t use group statistics. Similarly, the decision-making systems that help you search online are not based on such statistics. For example, a database search does not try to find Bob’s record using the average for people named Bob. Instead, decision science uses techniques focused on the individual.
What Are P-values?
Doctors often use p-values to compare two groups, such as treatment and controls in a clinical trial.
Investopedia suggests the following definition:
“In statistics, the p-value is the probability of obtaining results at least as extreme as the observed results of a statistical hypothesis test, assuming that the null hypothesis is correct.”
Don’t bother trying to understand the thing. It’s a kludge. Science writer Christie Aschwanden described it well. Take Professor Steve Goodman of Stanford University, who spent his career on this junk. He could state the definition, “but I cannot tell you what it means, and almost nobody can.”
You can explain the p-value correctly or make it understandable. Forget about doing both.
Statistics professor Regina Nuzzo explains that “people get it wrong, and this is why statisticians are upset and scientists are confused.”
The reason for this confusion is that Robert Fisher came up with the p-value in the pre-computer age. Its main benefit is it agrees with the scientific method described by Karl Popper. Fisher was at Rothamsted Experimental Station, where his main concern was comparing agricultural field experiments: crops and similar. These were comparing large groups (e.g. fields). Thousands of plants in groups where the individual grain had no importance to a farm’s productivity. A farmer cares about tons of products rather than a single grain of wheat. In commercial medicine, you are that single grain.
Unfortunately, EBM moved these methods into medicine and failed to realise they were inappropriate. Medicine’s problem is individual patients. Treat every individual well; on average, you get great results. But treat your individuals according to the group statistics, and you have a lot of sick people. Moreover, for the confused EBM doctor, it is not apparent why.
Asking The Right Question
Returning to Alice visiting the hospital. Her needs and questions concern her as an individual. She is not interested in the best treatment for the population on average. She wants to know the best treatment for her unique biology. Unfortunately, EBM does not provide this information. We need to start again with proper decision science and people concerned about the patient rather than profits.
Why do clinical trials not ask:
What’s the chance this will help an individual patient?
Rather they ask what’s the p-value difference between some groups, which does not address the central medical question. Again, we need direct, helpful information rather than social statistics and the ecological fallacy.