Dr Archie Cochrane was a medical officer and prisoner in a german WW2 transit camp. The food was scarce and limited, leading to an outbreak of swollen ankles (oedema). Only a week later, Cochrane was dealing with oedema above the knees. Things were getting worse.
The Trial
Archie Cochrane remembered the wet beriberi caused by a deficiency of vitamin B1 (thiamine). Deciding to investigate, he bought some black market yeast, a good B vitamin source. Cochrane selected 20 young prisoners with oedema above the knees for his clinical trial. Surprisingly, he explained to them James Lind’s experiment on vitamin C, a classic in medical history. Lind’s trial was a good choice. It was a classic, easy to understand, and solved a similar problem to the one Cochrane was facing.
Emptying two wards, he placed even-numbered prisoners in one and odd-numbered prisoners in another. This selection was an effective way of randomising them into two groups. Putting them in separate wards was practical but not perfect as they were in slightly different environments. Also, he didn’t blind his study. Both the clinic staff and the patients knew what was happening. Nonetheless, Cochrane must have realised this was a reasonable way to do his trial at the time.
He gave the treated soldiers two spoonfuls of yeast each day and the control group a vitamin C tablet. The vitamin C presumably was meant to be an inactive placebo with no effect. However, vitamin C does have an impact on the patient. The misuse of the placebo is a general medical research problem.
There are no accepted rules for placebo controls. In this case, Cochrane had enough about him to specify what was in the placebo when he reported the trial. In general, doctors can choose anything as a so-called placebo and not state what they use. So, for example, when trialling a headache remedy, they might sweeten the placebo with aspartame which can induce migraine. An active placebo is an old trick unscrupulous researchers use to bias the results. The ruse will remain hidden since they don’t need to list the placebo ingredients.
Far from being the gold standard, the World Medical Association’s Helsinki Declaration states that placebo-controlled trials are unethical. Commercial medicine uses placebos to benefit bad drugs, and researchers should only use them in extreme circumstances. Something to think about the next time you hear the phrase, but it wasn’t a placebo-controlled trial. Why the continued hype over placebo-controlled trials? It helps the drug companies bottom line.
Since his clinic had minimal resources, Cochrane measured fluid intake and frequency of peeing. However, Cochrane says there was no way to measure the volume, although that does not seem like it would have been too difficult.
For two days, the two groups symptoms remained unchanged. We might expect this lack of benefit because the yeast supplement would have taken time to work. There was some improvement on day three, which Cochrane described as “hopeful”. However, the fourth day was “conclusive”. Nine of the ten treated patients felt better, but none in the placebo group. This result convinced Archie Cochrane that the yeast was working.
Archie was right to find it convincing. Let’s assume a probability of 0.06 (6 to 100 odds) for a patient in the control group to get better. This gives a 0.53 (53 in 100) chance of getting no improved subjects and a 0.34 (34 in 100) chance of getting a single patient improving (using the Wolfram Alpha binomial calculator). This estimate is reasonable and conservative, considering we have no control improvements. With this value, we have a probability of 9×10-11 (that is 0.00000000009) of getting nine or more improvements in the yeast group. OK, so there are issues with the controls, measurements, and estimates here. In particular, the results were subjective rather than definitive. But there were reasons Cochrane thought this crappy little trial his best ever.
Cochrane then gave his James Lind talk to the German guards and explained the wet beriberi epidemic. They gave him plenty of yeast. So he treated those with oedema above the knee with two teaspoons a day and one spoon for those with ankle oedema. As Cochrane put it, “The fear of a major catastrophe passed, and we began to relax.”
The time course of oedema in the camp
Archie Cochrane had started as a brilliant Cambridge biology graduate and would have had a thorough understanding of experimental methods. However, for most of his life, he worked as a physician and epidemiologist rather than a basic scientist. Cochrane later seemed confused about his small prisoner-of-war experiment. He thought he had used a little bit of science and had a lot of luck.
Since his trial resolved the problem in the prison camp and helped the inmates, he should be proud. Instead, this later response is quite sad. It was his most successful trial. Rather than running it down because of becoming familiar with modern statistical methods, he could have asked a more interesting question. Why was this little trial so successful when later ones were not?
Archie’s Hindsight
Archie later thought that his diagnosis was incorrect. It wasn’t wet beriberi. However, he gave no reason for this reinterpretation. Odema is a symptom of beriberi, and the swelling went away when given B vitamins, leaving the patient feeling better. He admitted the treatment worked but claimed, “I still do not know why”. His later suggestion was that the yeast contained a small amount of protein and corrected the fluid balance.
Oedema is also a symptom of protein deficiency. But the daily requirement for protein is 60-70 grams for an adult, much more than in a spoonful of yeast. Moreover, the oedema resolved quickly. So his second explanation is a bit of a stretch. Looking back with the benefit of his later work is all well and good. But given no reason for his change of mind other than peer pressure, we can question his latter interpretation.
Secondly, Cochrane realised that he had effectively randomised the patients separating odd and evens into two groups. At that time, people seldom randomised their groups. Such selection bias can be an issue, but there are many ways of dealing with it. Archie Cochrane had chosen a reasonable and practical approach for his little experiment.
Finally, he was unhappy with the design of his study. I think he did better than he recalled, considering he was in a prison camp with few resources. Yes, it wasn’t an ideal controlled experiment. In his words, the controls were “somewhat inadequate”. Also, he knew which patients were getting the treatment and which the placebo.
Cochrane says he “can take little credit as the design of the trial was largely fortuitous.” Well, I differ and think this is false modesty. He was a good doctor who helped his patients with a neat experiment. As he later put it, “it was amazing”. I can believe it was Cochrane’s most successful trial, but I find it sad he thought it was also his worst because of the poor design. In addition, he had become confused with the statistical developments in medical research, particularly in his speciality of epidemiology. Sadly, Archie Cochrane had lost his practical touch and good scientific taste.
The Father Of Evidence-Based Medicine?
Archie Cochrane was an outstanding and honest physician. However, EBM people coopting his ideas describe Archie as a medical maverick, which is hardly accurate as his suggestions fell well within the mainstream. Arguing for more scientific medicine is hardly unusual. I expect every scientist who has worked in the area would want similar. His book “Effectiveness and Efficiency” helped inspire the later development of EBM.
Indeed, we now have a Cochrane Collaboration in his honour. This group tried to list and validate medical data by combining several randomised controlled trials in “systematic reviews” or “meta-analyses”. These are often massive collections with large-scale trials recombined in a single study. The Cochrane Collaboration think this is honouring Archie. But, surprisingly, I guess if Archie Cochrane were alive, he would have raged against the Collaboration and their methods.
Randomised Controlled Trials
Cochrane wanted trials with the subjects placed randomly into groups. This randomisation is a low-level method of avoiding bias. On average, the means and standard deviations in the groups will be about the same. But since the selection is random, we expect one in every 20 trials will be significantly different just by chance. So five per cent of significant RCTs are just a fluke and potentially misleading. There is nothing inherently wrong with randomising the groups to remove bias. Still, there are many other techniques, each with its own characteristics.
Take selecting a football or cricket team in the schoolyard. The captains could do it randomly by tossing a coin for the players. However, an often used alternative to multiple coin tossing is having the team captains take turns selecting players. This alternate selection gives an acceptable method that minimises the perceived skill difference. Still, the captain with the first choice has a potential advantage. So the captains often start by tossing a coin for who gets the first choice.
Cochrane, in his worst and best trial, just put sequential patients into teams, one team with the odd-numbered players and the other with the even. While he wasn’t happy with the method, mathematicians and computer scientists might explain that there isn’t an ideal selection method. But tossing a coin is fair, easy to do and understand.
Controlled is the other descriptive prefix for these trials. What Archie meant by control was designing the experiment to reduce noise, bias, and other errors. In many disciplines, scientists do this sort of thing as a routine. It’s expected and taught in first-year college. So what is medicine up to if this is considered such a big thing?
Scientists who build theories get all the glory. But gifted experimentalists demonstrate genius equally. The three that come to mind are Ernest Rutherford, Marie Curie, and Gregor Mendel. I suppose Mendel also developed the theory of genes at the same time, but I think the trio make the point.
Let’s give Archie Cochrane credit. He wasn’t expecting medics to become gifted experimentalists like Rutherford or Curie. He merely suggested some guidelines for reducing error and bias in trials. The last thing he expected was for clinical trials design to become a series of tick boxes. Randomised – tick, large-scale – tick, placebo-controlled – tick, double-blind – tick and so on. Cochrane was for guidelines, numbers, and statistics, which is just first-year college stuff.
What Changed?
I’m doing this in broad brush strokes to get to the main point. Cochrane thought RCTs were cheap and cheerful. Easy and relatively quick. After all, what’s the problem with coin tossing and spending a few minutes making sure you are doing sensible things in your experiment? Archie says, “The technique will nearly always be an RCT.” But he gives the EBM game away when he says, “The first is to call attention to the low cost of such RCTs.” Of course, he is correct; there is no fundamental reason why a good experiment needs to cost more than a bad one.
Cochrane wrote the book “Effectiveness and Efficiency” in 1972. The title portrays his approach well. Back in the day, clinical trials we often reasonably small, say 30 patients in each group, relatively inexpensive and easy for another research group to repeat. Up to and including this time, medicine was moving forward quickly, with breakthroughs and new ideas coming thick and fast. Medicine introduced new antibiotics, body scanners, transplants, and effective drugs. Universities could check new ideas quickly and cheaply.
Up until the 1970s, trials were smaller and more easily replicated. The recent change in the size of clinical trials is dramatic. Martin Bland, a statistician from the University of York, found in 1972 that the average size of trials in the Lancet and British Medical Journal (BMJ) was between 30 and 40. But by 2007, 35 years later, the average figures for the Lancet and BMJ were 3,116 and 3,104. The later studies were about 100 times larger. They would also cost 100 times more with inflation added on top of that. Corporate medicine had priced universities out of the game.
But it didn’t stop there. EBM tightened the screws. It now costs more per patient in a clinical trial—lots more. The costs have skyrocketed. It is no longer practical for a regular doctor or scientist to have an idea and test it. Here is a table of prices published in 2021. The numbers are in millions of dollars!
The power of science is replication. But how can people check these trials are correct? The average family doctor could not afford the couple of million dollars needed for an initial trial. Similarly, a university department does not have this sort of money hanging around. Universities would require substantial grants, and the funding body might not be generous. Funders do not support medical mavericks or repeating experiments and stick to the establishment. Even whole countries might baulk at the risk associated with the $40-100M+ needed to complete the hurdles and meet the new regulations. So essentially, only the large drug companies can afford to do these things.
The drug companies control the system. They love the high costs of RCTs because it keeps you, doctors, universities, and small competitors out. But unfortunately, EBM is not the effectiveness and efficiency that Archie hoped for. Instead, corporate medicine subverted his ideas so the drug companies could make massive profits from a medical monopoly.