Medicine and its related products and regulations

   A better man than I am said:

   "I think the educational and psychological studies I mentioned are examples
of what I would like to call cargo cult science. In the South Seas there is a
cargo cult of people. During the war they saw airplanes land with lots of good
materials, and they want the same thing to happen now. So they’ve arranged to
make things like runways, to put fires along the sides of the runways, to make
a wooden hut for a man to sit in, with two wooden pieces on his head like
headphones and bars of bamboo sticking out like antennas – he’s the controller
– and they wait for the airplanes to land, exactly the way it looked before.
But it doesn’t work. No airplanes land. So I call these things cargo cult
science, because they follow all the apparent precepts and forms of scientific
investigation, but they’re missing something essential, because the planes
don’t land.
   "Now it behooves me, of course, to tell you what they’re missing. But it
would be just about as difficult to explain to the South Sea Islanders how they
have to arrange things so that they get some wealth in their system. It is not
something simple like telling them how to improve the shapes of the earphones.
But there is *one* feature I notice that is generally missing in cargo cult
science. That is the idea that we all hope you have learned in studying science
in school – we never explicitly say what this is, but just hope that you catch
on by all the examples of scientific investigation. It is interesting,
therefore, to bring it out now and speak of it explicitly. It’s a kind of
scientific integrity, a principle of scientific thought that corresponds to a
kind of utter honesty – a kind of leaning over backwards. For example, if
you’re doing an experiment, you should report everything that you think might
make it invalid – not only what you think is right about it: other causes that
could possibly explain your results, and things you thought of that you’ve
eliminated by some other experiment, and how they worked – to make sure the
other fellow can tell they have been eliminated.
   …
   "The easiest way to explain this idea is to contrast it, for example, with
advertising. Last night I heard that Wesson oil doesn’t soak through food.
Well, that’s true. It’s not dishonest, but the thing I’m talking about is not
just a matter of not being dishonest, it’s a matter of scientific integrity,
which is another level. The fact that should be added to that advertising
statement is that *no* oils soak through food, if operated at a certain
temperature. If operated at another temperature, they *all* will – including
Wesson oil. So it’s the implication which has been conveyed, not the fact,
which is true, and the difference is what we have to deal with."
   …
   "The first principle is that you must not fool yourself – and you are the
easiest person to fool. So you have to be very careful about that. After you’ve
not fooled yourself, it’s easy not to fool other scientists. You just have to
be honest in a conventional way after that." – from "Surely You’re Joking, Mr.
Feynman!", Bantam Books, 1985.

   I’ve been the target of a rising tide of attacks for my snide ridicule of
someone who contended that application of Vitamin E to an abrasion would cause
it to heal more quickly. My snideness was not justified (and for that I
apologize) but my attack was. I ended my article with a short paragraph in
which I said that anecdotal evidence is useless and that only double blind
experiments were important.

   Oddly enough, no-one defended the concept of using Vitamin E on abrasions,
they only attacked this comment about anecdotal evidence. I’d like to make a
more serious attempt to justify this position (and also alter it slightly).

   A double blind experiment is one which is designed to eliminate two very
important sources of error:
        1. Observer bias
        2. Multiple degrees of freedom.

   It has been demonstrated repeatedly that two observers, watching exactly the
same set of occurrences but told to expect different things, will report what
they expected to find, even though they contradicted each other. Each will
swear that they have reported honestly, and each will be wrong. (An unbiased
observer can demonstrate that both have misreported.) (One version of this
experiment had people observing rolls of 2 dice, being told that the dice were
dishonest and would show more 6′s or more 1′s. Each did report finding such a
bias, even though the dice were actually honest, and the experiment was filmed
for later analysis which *showed* they were honest.)

   Multiple degrees of freedom are a problem when you compare two occasions,
but there are multiple differences between them. [If I pour water from one
container to another, it remains clear. If I add salt, sugar, pepper and blue
food coloring to the water, it turns blue. Thus from this I conclude that salt
makes water blue.] The problem is amplified when you don’t even know what all
the degrees of freedom are. [My lab assistant added the blue food coloring when
my back was turned. For a more serious example of this, read any article about
"Piltdown Man".]

   The specific case in point is an anecdote about improved healing of a cut by
application of Vitamin E. He reports that healing was speeded, but how was this
measured? (It turns out it was a subjective feeling of "Boy, this sure is
healing fast!") How do we know what other factors may have changed between this
specific case and other times he cut himself? (Maybe his nutrition is better,
and he healed faster because of it. Maybe there are other factors involved as
well. We just don’t know.) More important yet, how do we know it really healed
any faster? It could just be that this was a less serious cut, or that it took
exactly the same length of time as some other cut, but was observed wrongly.

   A serious study of this case would require making 10 or 20 deliberate cuts,
to a controlled depth, two to a person. You then treat one with Vitamin E, and
the other with corn oil. (But beware, it may be that it is the OILINESS of the
Vitamin E which helps, and corn oil may have the same effect. You have to
choose something which you KNOW doesn’t help.) Neither the patient nor the
doctor know which cut got which treatment. You can then observe whether Vitamin
E helped more than corn oil (or whatever you use for a placebo).

   I stated that anecdotal evidence is useless – this was a bit too strong. It
is very premature to use anecdotal evidence as if it were an established fact,
but it does repesent a source of potential investigation for the future.

   However, not all anecdotes are created equal. Suppose someone says "You know
how slowly I heal when I cut myself? Well, a couple of weeks ago I cut myself,
and it was a full moon, so I went out into the woods near my house and took my
clothes off and danced and sang under the moon, and you know? Darned if I
didn’t heal faster than I usually do!"

   Immediately this spurs a series of questions: Does it matter whether he sang
in English, or would Latin work better? What were the details of the dance?
Does it matter whether it was the full moon? If he danced fully clothed, would
that make a difference? But before asking ANY of those
questions, we apply a test of reasonableness, and come to the conclusion that
it is exceedingly unlikely that dancing naked under the moon had anything to do
with of his cut. It is much more likely that dancing under the moon made him
FEEL better, and thus made it SEEM to him as if his cut healed faster even
though it didn’t.

   Whenever one hears an anecdote, one applies a test of reasonableness to it,
rejecting it based on prior knowledge. There is no rule on how to decide
whether it is reasonable (application of bread-mold to a cut makes it heal
faster) or unreasonable (dancing naked under the moon makes a cut heal faster)
– and as such we can sometimes waste a lot of time investigating something
which turns out to be unimportant – and equally we can sometimes pass by
something which is very important. ("Stones from the sky" is surely a classic
example of this.)

   That’s the breaks. No human endeavor is without error, but the scientific
method is intended to be conservative: Better than 10 truths be missed than
that we believe one falsehood. If we’re going to make errors, let’s make
erroneous exclusions rather than erroneous inclusions. Thus it can seem to
those outside that science moves slowly and seems dogmatic and thick headed.

   The body of scientific knowledge is not complete, but what is
within it is as trustworthy as we can make it. Very occasionally, we must
replace a section wholesale, but as time goes on such changes alter
proportionally smaller parts of the whole body, and there are some parts of it
that haven’t changed in millenia. (Euclidean geometry, for one.)

   In this particular case (Vitamin E and cuts) my intuition was as follows:
"The list of vitamins represent 50 or so chemicals distinguished only by the
fact that our bodies need them but cannot produce them internally. They are not
particularly distinguished in any other way from the millions of chemical
compounds running around internally in our bodies. As such, they should be no
more likely to have special biochemical properties than any randomly chosen
list of organic chemicals, and thus reports of special properties are more
likely to be the result of improper observation or improper reporting. Given
hundreds of millions of potential organic compounds available, there is no
reason to give this list of 50 or so any special attention."

   I will try to summarize what I believe to be his position, without being
snide: "There are a large number of conditions for which no reasonable
treatment is available, or for which the treatment can only be dispensed by a
doctor at great expense. I want to control my own body and my own health, and
when something goes wrong I want to be involved in the
…

read more »

Links

• El Pollo Loco Gift Card
• travelocity discount promotion code

This entry was posted on Monday, September 28th, 2009 at 6:24 am and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.