Telling good science from bad (a cautionary tale)
Alison Campbell (November 1, 2019)
Recently I came across the claim that cystic fibrosis (CF) can be cured by diet.
This was news to me, given that the mutation that causes CF is well-documented, as are the necessary treatments, and I wasn't aware of any evidence that diet alone would correct the faulty membrane pump involved. So I said so. In response I was told to look into “mineral replacement” and view a Youtube video. A video made by Dr Joel Wallach, apparently “nominated for a Nobel prize in medicine”, or so my informant told me.
Well, that's the logical fallacy known as an “appeal to authority” right there. That is, I was told that Dr Wallach's claims must have something in them, not because of published evidence in support of them, but because of a supposed Nobel nomination.
It's also a somewhat unlikely claim, given that the Nobel committee keeps nominations secret, so I checked a bit more deeply. It turns out that Dr Wallach (a Doctor of Veterinary Medicine & a Doctor of Naturopathy, not an MD or a PhD) was nominated by a group of naturopaths, and that the nomination was not viewed as legitimate by the committee.
Mind you, he could still be correct in his claim that CF is both preventable and curable by taking selenium supplements. If it were true, it would be a game-changer for those with the disorder. Surely we could expect him to have published his results in a way that would allow other physicians and scientists to test & attempt to replicate them? As far as I could tell, using Google Scholar, this hasn't happened.
He has co-authored a paper claiming that that patients from Keshan, China, in an area with soils that are selenium-deficient, show the same pancreatic lesions that are seen in some CF patients. However, if the selenium-CF link were real, you'd think that there'd an epidemic of cystic fibrosis in Keshan. But this doesn't seem to be the case – if it were I'd expect it to have been highlighted in this recent systemic review.
Now, patients with CF apparently can and do suffer from selenium deficiency, but it seems that this is likely a result of the illness (patients have to take a daily cocktail of digestive enzymes as the disease interferes with digestion and absorption of nutrients). There are also cases where parents have attempted to use selenium supplements to treat their children with CF, with sometimes tragic results. And this double-blind trial of selenium supplementation found no impact on CF patients. However, Wallach claims that this and every other disease known unto mankind is due to mineral deficiencies.
He also sells a lot of books and supplements on his webpage (11). Which is an … interesting … place. I did find it a bit unusual that Dr Wallach (12) (DVSc, ND) states that he carried out 3,000 human autopsies in the period 1962-1987, given that autopsies are usually performed in cases of sudden death, and by a pathologist or medical examiner – although the system in the US varies depending on which state you're in (13). (That's along with 17,500 animal autopsies, which works out as 700 per year, which was presumably done alongside his studies for an ND and other activities).
Why did I spend time looking into Wallach's claims? Well, first up, it's always worth examining claims like this, in case there's something in them (though in this case plausibility, or prior probability (14), suggests there was not).
But secondly, I was concerned that I was pointed his way – and in an approving manner – by a science teacher. This bothered me. A lot.
Why? Because Wallach's claims match a lot of the criteria for spotting bad science (see here (15) and here (16), for example). The fact that there doesn't appear to be any good published peer-reviewed evidence to support them is an example. That is, he's pitching directly to the consumer. So is the clear conflict of interest: selling supplements that provide the cure he claims to have discovered. From what I could determine, his ‘study' had no control group, didn't use blinding in its testing, and involved a small sample size. None of this inspires confidence, and these are all things that students should surely be finding out about as they learn about the nature of science.
Yes, of course it's important to be open-minded, but we really should remember those criteria when assessing claims (in the media, on individual's webpages, or on Youtube) about novel discoveries. Sadly, too many people don't do this?
References
(1) https://tinyurl.com/yxljz8q5
(2) https://tinyurl.com/yde6omjp
(3) https://tinyurl.com/y67odv3h
(4) https://tinyurl.com/y5ydecny
(5) http://skepdic.com/wallach.html
(6) https://tinyurl.com/yxuzyqqx
(7) https://tinyurl.com/yxzlfn6z
(8) https://tinyurl.com/y29gnpus
(9) https://tinyurl.com/y2onuwk2
(10) https://tinyurl.com/yxuzuwmy
(11) https://tinyurl.com/y55n7rhh
(12) https://tinyurl.com/y55n7rhh
(13) https://tinyurl.com/y4ba55yy
(14) https://tinyurl.com/yxzlfn6z