Procrustes is alive and well

I was first conscious that I had met Procrustes about 20 years ago, though I did not at that time know his name. At the beginning of a course of instruction on how to examine medical patients the clinical tutor had us don headphones plugged into an amplifier while his stethoscope wandered over the chest of a lady who each year donated her time to the greater glory of Medicine. She had a diseased mitral heart valve and we were invited to identify the "low pitched rumbling diastolic murmur" and "There! Listen carefully!

She has an opening snap." Did anyone have problems? I decided to be uncharacteristically assertive. I could hear nothing but the normal heart sounds. I was sorry (and was secretly ready to declare that the Emperor had no clothes), but even after repeatedly listening I could not detect the murmur or the snap. In the end, as the rest of the class were getting restless, I decided to agree that the Emperor did after all have some clothes on.

Procrustes looked over my shoulder many times in the following years of training and finally I learned his name when reading an excellent little book on clinical examination by Pappworth1 in which he warned the reader against being influenced by this figure from Greek legend. To quote Robert Graves' account, Procrustes "...who lived beside the toad...had two beds in his house, one small and the other large. Offering a night's lodging to travellers, he would lay the short men on the large bed, and rack them out to fit it; the tall men he would lay on the small bed, sawing off as much of their legs as projected beyond it. Some say, however, that he used only one bed, and lengthened or shortened his lodgers according to its measure."2 The crime of Procrustes is often committed in Medicine when a doctor makes a snap diagnosis by instinct, the seat of his pants, pattern recognition or whatever jargon metaphor is current; and then makes her observations fit the preconception. This is easier to do than one might think, since many of the signs in clinical medicine are very subtle. As Pappworth put it (p.81) "...suggestibility is a potent source of error and the experienced doctor can make the less experienced and uncritical hear anything he wishes him to hear...But apparently, with increasing years, and even sometimes with increasing deafness, some cardiologists are hearing many more things than they could 20 years ago."

Even a superficial look at the history of Science shows many examples of Procrustes at work; and when advances have been made, it is often because an investigator has refused to see what he "ought" to see. We are now quite used to the idea that observation should take precedence over dogma, but in the 16th century this was a revolutionary idea often ascribed to Copemicus's De revolutionibus orbium coelestium (On the revolution of heavenly bodies) of 1543. "In substituting for the authority of the ancients the principle of subordination to the facts as the source of all knowledge, Copernicus's work marks an essential turning point in the history of ideas and scientific progress."3 Whether or not this is true (and there is some evidence that Copernicus placed the sun at the centre of the universe simply because he thought such a splendid body deserved the place of honour), it was certainly a time for the overturning of dogma.

William Harvey is usually credited with describing in 1628 how blood circulates rather than ebbs and flows. But like Newton, he stood on the shoulders of others such as Realdus Columbus who pointed out in 1559 in his De re anatomica that Galen erred in believing that blood passed from the right to left ventricle through pores in the intervening wall or septum; "But they err by a long way, for the blood is carried to the lung through the pulmonary artery and in the lung it is refined, and then together with the air it is brought through the pulmonary vein to the heart. This up to now no one has either observed or recorded in writing, although it was most meet to be observed by all."4 Later he does not mince words in saying "Yet truly, there is a race of men stupid, and ignorant, who have neither the wish nor the ability to find anything new. And therefore, whatever a physician with a great name writes they immediately subscribe to it nor will they depart from their beliefs one jot."5

I had little difficulty in finding an anatomical example of this from recent times. Standard textbooks of microanatomy state that Brunner's glands drain directly into the crypts of Lieberkuhn and generations of anatomy demonstrators, including myself no doubt, must have pointed this out to medical students who presumably "saw" that this was so. A fellow demonstrator, Tom Treasure, pointed out that "'...the examination of class anatomy teaching slides contradicted this view."6 There are in fact clearly visible ducts which drain from the glands into the duodenum.

The very eminent have not been free of Procrustes' influence, but they have had the good fortune to be "right" in their views. Gregor Mendel's classic 1866 paper on the genetics of garden peas7 was neglected until its rediscovery in 1900, but until it was re-examined very closely by the great R.A. Fisher in 1936, none of the many thousands of people who must have read it had noticed that Mendel's results were much better than expected. It is suggested that Mendel "knew" what results he ought to get and adjusted the data accordingly, but in doing so got results that were too good to be true; or as Fisher put it: "The discrepancy is strongly significant, and so low a value could scarcely occur by chance once in 2000 trials. There can be no doubt that the data from later years of the experiment have been biased strongly in the direction of agreement with expectation."8 He charitably suggested that Mendel was deceived by an assistant who helpfully adjusted the figures, but Wright points out how easy it is for unconscious systematic bias to creep into sorting and counting data and concludes "Taking everything into account, I am confident, however, that there was no deliberate effort at falsification."9

Emest Rutherford is another example of a great scientist who "knew" what to expect and selected "good" results while neglecting results from experiments when he felt his apparatus was working less well. There was no attempt made to conceal this however and his 1886 paper "On the Passage of Electricity Through Gases Exposed to Réntgen Rays" contains a note: "[only] the observations marked with asterisks were used to calculate the constants in equation (4)",10

Getting the answers right seems to excuse being selective. However, Wilson's biography of Rutherford contains an account of how Rutherford and Chadwick prevented a coworker in their field from the embarrassment of publishing biased results which supported the wrong conclusions. Petersson, a Swede working in Vienna, asked Rutherford in 1924 to help arrange publication of a paper describing a new method of measuring atomic disintegrations. Rutherford and Chadwick felt that something was fishy about the results as they were working in the same field, and delayed publication until Chadwick could visit Vienna in December 1927. He found that "the observers, the counters of the scintillations, were three youngish women...of what Petersson called Slavic descent because he believed (I'm only repeating what he said to me) that...Slavs had better eyes...and that women would be more reliable than men as counters of scintillations..." Chadwick quickly found that the women knew each time what was expected to happen, but when they were "blinded" to what was expected the results were similar to those that Rutherford and Chadwick were obtaining. It was not a question of cheating but "_..they were deluding themselves. They were seeing what they were expected to see."11

Unfortunately, there was no Rutherford or Chadwick to prevent embarrassment to Davenas et al. in 1988 though there is some evidence that the editor of Nature went out of his way to try. Their experiment purported to show that very dilute solutions of antibody (sometimes so dilute in fact that there could be no molecules of antibody at all) could bring about changes in a biological marker system.12 Fierz quickly pointed out that the results seemed to lack the normal experimental spread of errors13 (too good to be true again) and Maddox et al. stated that not only had no attempt been made to exclude systematic observer bias but that data from experiments that did not meet expectation were not included in the published results.14 When bias was excluded, the results could not be reproduced.

In medicine Procrustes can of course influence not only the doctor but the patient. When a person is ill, the patient expects to be "made" better and the doctor may even naively believe that she has been instrumental in bringing this about when in fact the expectation ought to be that the patient will get better anyway nine times out of ten. When the use of a treatment system which, at least on a priori grounds, cannot be expected to work is followed by the patient's improvement, the improvement is seen by the uncritical as validating the system. When the instances are multiplied dozens of times, the practitioner will not feel the need to look more closely as the efficacy of the system is too obviously self-evident. "After all," he may ask, "do not 90% of my patients get better with my treatment (and did not the remainder come to me too late)?" Medicine, much more than any other branch of investigation, uses techniques designed to eliminate Procrustes' evil influence. (See box.)

In its fight against Procrustean influence, or what some more prosaic people call patient and observer bias, medicine uses controlled trials in which patients are randomly allocated on a double blind basis to treatment or control group and the person assessing the results of treatment remains blind to who is in which group until the end of the trial. Statistical analysis is then used to help decide how likely it is that the observed differences could have arisen by chance, If the likelihood is small, it is tentatively concluded that the treatment may actually have brought about the differences, which may or may not be in the direction of improvement. Surprisingly few efforts are made outside the field of medicine systematically to eliminate observer bias and well designed trials in the area of alternative medicine are rather hard to discover. Procrustes continues to lie in wait not just for Theseus, but for us all.

References

1. Pappworth, M.H. A primer of Medicine. London, 1969.
2. Graves, R. Greek Myths, pp.329-30. London, 1958.
3. de la Cotardiere, P. (Ed). Larousse Astronomy, p.14. Hamlyn, 1986.
4. In Whitteridge, G. William Harvey and the Circulation of the Blood, pp.49-50. London, 1971.
5. Ibid. p.54,
6. Treasure, T. The ducts of Brunner's glands. J.Anat. 1978;

127(2):299-304.

7. Mendel, G, Versuche uber Pflanzenhybriden. Verhandlungen Naturforschender Vereines in Brunn, 1886, 10:1.
8. Fisher, R.A, Has Mendel's work been rediscovered? Annals of Science. 1936; 1:530.
9. Wright, S. In Stern, C. and Sherwood, E.R. (Eds.) The Origins of Genetics, p.175. San Francisco, 1966.
10. In Wilson, D. Rutherford—Simple Genius, p.117. London, 1983.
11. Ibid. pp.461-2.
12. Davenas et al. Human basophil degranulation triggered by very dilute antiserum against IgE.-Nature, 1988; 333: 816-818.
13. Fierz, W. Letter. Nature, 1988; 334: 286.
14. Maddox, J. et al. "High dilution" experiments a delusion. Nature, 1988; 334: 287-290.
15. Allgood, S.O. Vitalcillin, a new treatment for terminal grottosis. J.Vital Med. 1988; 3: 32-41.
16. Yes, you're right. Reference 15 and succeeding ones are, like Procrustes, mythical.

Trials and Error

Consider the efforts of Dr Allgood15 in the treatment of grottosis, an often fatal disease. It was suggested that a new drug, Vitalcillin, might cure grottosis and that in view of favourable reports from occasional prescribers of the drug it would be almost criminal to do other than prescribe it to all grottosis victims. Allgood pointed out that since some patients actually recovered you would not know whether Vitalcillin was responsible and that as it had some toxic and potentially fatal side effects, you might-actually make people worse rather than better. It was-decided that the only ethical thing to do was to administer Vitalcillin to only half of a group of grottosis patients so that any improvement in the treated group compared to the untreated or control group must be due to Vitalcillin.

After six weeks, ten out of thirty in the treatment and eighteen out of thirty in the control group had died. Allgood's assistant, Dr Spoyler, was quick to point out that there seemed to have been some bias in selecting patients for treatment as they seemed in the main to be pretty, well-nourished young women whereas those in the control group were for the most part balding, fortyish men.16

The trial was re-designed so that patients were allocated to treatment or control on the spin of a coin, but members of the control group soon noticed that they weren't getting Vitalcillin and that the doctors were spending little time with them, whereas the treatment group complained that the nurses were spending most of their time with the control group so that it was nearly impossible to obtain a bedpan when needed. However, knowing that they were receiving the new wonder drug made them feel much better, despite the severe diarrhoea which it seemed to cause.

A decision was then made to allocate treatment on the spin of a coin and to provide the control group with dummy medication or placebo so that neither group would be biased as to how they felt by the knowledge of whether of not they were receiving Vitalcillin. Allgood's house physician, Dr Kleverbugge, suggested that it would perhaps be better if no one knew until the end of the trial and eventually it was agreed that the hospital pharmacist would allocate patients to treatment or control groups and provide medication for each patient in packets which bore only the patients' names and code numbers. At the end of the trial the code could be broken to discover who had got what.

Allgood reported that the results seemed to favour Vitalcillin though the advantage was by no means as clearcut as had at first been thought: 14 out of 30 controls and 19 out of 28 of the treatment group were still alive at six weeks.17 Kleverbugge had by this time moved on to another job and wrote to the Journal of Vital Medicine that he had just read a book on statistics and that there was little to celebrate, as he had calculated that there was a high probability that the observed differences between the two groups could have arisen by chance.18