« "Seeing White in Black"--a Fungible Concept | Main | Moving on »

January 31, 2009

Life Outside the Bell Curve, a.k.a. The Limitations of the Evidence Base

The strength and weakness of what we consider traditional American medicine (more accurately referred to as allopathic medicine) is that it focuses on the aggregate rather than the individual, on macro-level patterns rather than micro. The reverse is largely true for homeopathy.

If one stops to really consider the methodology of the analysis that underlies allopathic medicine the significance of this becomes more clear. The researcher first identifies a question that is relevant to a substantial community. It has to be a substantial community because it isn't possible to gather enough data to show statistical significance (if it exists) from a small data set. Therefore one is limited to asking questions that at least provide the potential for collecting statistically significant data.

Supporting this focus on large-scale scientific inquiry is funding, which goes primarily to support finding solutions to conditions that effect large numbers of people, even mildly, rather than those that effect small communities severely. For example, in the USA, you'd have to be sleeping under a rock to not be aware of the tremendous push in the last decade for research in breast cancer. The push has paid off, and breast cancer has shifted from being something terrifying and deadly to one of those where, if you have to get cancer, better to get one you have a fair chance of surviving -- it is considered more of a chronic illness than a deadly disease.

That is now, ten or twenty years ago it was a completely different story. Meanwhile, ovarian cancer has always been one of the really bad diagnoses to get, but because it is comparatively rare (affecting fewer people) it is not as well funded, not as well researched, not as well diagnosed, and has stayed pretty much the same for prevalence and mortality for the past 20 years. (NOTE: This is a drastic oversimplification, and I am aware of that.) If you look at the stats comparing breast cancer to ovarian, even with the dramatic improvements in treatment of breast cancer and little improvement in treating ovarian, breast cancer still kills over twice as many women in the US as ovarian.

SEER: Cancer Mortality Trends, BrCA/OvCa
SEER data: http://seer.cancer.gov/faststats/selections.php

How devastating an illness is doesn't drive funding (which drives both quantity and quality of research) -- it is how many lives it touches. As long as it is still a significant problem, even if we've made huge strides in treatment, it will still drive funding. At some point, this can become a self-perpetuating cycle, the funding drives itself and we keep funding the same things because we have for so long.

Another aspect of what drives research questions is being able to clearly define the question. This is why conditions such as fibromyalgia, that are defined by nonspecific symptoms -- symptoms shared with many other conditions -- tend to be under-researched. The nature of the condition means it is ill-defined and difficult to research.

Bottomline, you don't want to get either a rare disease or one that is vague or ill-defined. God forbid that you be an outlier. You know the whole standard distribution curve analysis, right? Bell curve? 95% of all people fall within 2 standard distributions from the mean. What about those 5% of people who don't fall in the main part of the curve - the outliers? Well, we don't really know what to do with them -- that is not what the research looks at.

Does this mean that rare conditions don't exist? Of course not. Does the presence of a non-specific symptom for which there is no clearcut or inexpensive differential diagnostic process mean that the symptom doesn't exist or that it is purely hypochondriacal or psychosomatic? Of course not. But it does mean that it is sometimes it is, sometimes it isn't, and as a healthcare provider it is often expensive and difficult to tell the difference. Does that mean nothing is wrong? That is isn't REAL? Or that there is no solution? No way to make life even a little easier? Heck no to all of the above. It does mean that the burden of figuring out the problem, solutions, what helps, what doesn't, all tends to lie largely on the person with the problem and their loved ones.

Another complicating situation is when the patient has multiple symptoms. They might be related, or they might not; it might be a syndrome, or it might be a series of otherwise unrelated problems that just happen to be present in the same person. Almost a coincidence. As a general rule, the medical literature and research process don't do well with complicated situations. That makes it hard for the doctors, and it isn't something that it easy to address in training. It is hard to understand the various bits and pieces as part of an overall pattern. It is not unusual to break it down into bits and pieces and look at them one at a time. This to manage the pain, that to control the inflammation, this lifestyle change to build strength and combat fatigue, this dietary change to help manage fatigue at certain times of day.

Often the only difference between being told a problem is in your head and not is the trust between you and your clinician. By this I mean you have a pre-existing relationship with your clinician which has generated trust on both sides. Even then, there is a fair chance they will say, "How much stress have you been under lately?" or "Why don't you just make sure you're eating well, exercising, drink lots of water, and make sure you are sleeping well - none of those late nights, ok? Come back if that doesn't work." Luckily, vague symptoms sometimes respond to vague solutions.

My personal perspective on this dynamic is a little off-center. I am trained in evidence-based healthcare research methodologies and consult in this area. I have a fair understanding of how the evidence stacks up, or doesn't, the significance of insufficient evidence to support an important question, the importance of being able to assess the levels of evidence available and make decisions based on the best available evidence (even if it isn't perfect). At the same time, I have the eccentric joy of being a member of a family just stuffed chock full of outliers. I know quite well what happens when what is wrong with you or your loved one doesn't fit under the curve.

Alright, well, so our healthcare and research systems are not designed to deal with all situations. The evidence is incomplete, and that is part of our process - it is always incomplete, and always will be until we stop learning and asking questions. That means the decisions, services and accommodations based on that research share the same flaws. As long as we understand that, we can adapt the decisions, services, and accommodations; we can include the outliers, those who don't fit under the curve. The danger, the very real danger comes in trusting the research completely without questioning, without acknowledging the inherent flaws and limitations of the evidence base.

Posted by pfa at January 31, 2009 08:51 PM

Comments