Evidence-Based Medicine
After my 3 year family medicine residency, I signed on to do a year-long fellowship in what is now called "Evidence-Based Medicine." Back then we gave it all kinds of names like "quantitative methods in clinical decision-making." The site of my work was through the University of Oklahoma Health Sciences Center Department of Family Medicine, where I had done my residency. It was somewhat of a self-designed fellowship under the tutelage of Stephen Spann, MD, for whom I carry the utmost respect as an early champion of Evidence-Based Medicine. I also did work through the Harvard Schools of Public Health (though I was only actually there for an 8 day course--but it sounds impressive!).
Basically, physicians who are interested in Evidence-Based Medicine (or EBM) are complete skeptics. We are scientists to the CORE. We are Mr. Spock-like in our adherence to logic. We don't believe any claims that are made in medicine unless they are backed by hard-core evidence--that is, solid, well-designed studies of the effectiveness of the claim that are not damaged by many types of bias or poor study design.
If a pharmaceutical manufacturer claims its drug reduces heart attack risk, then it better have the properly-designed, double-blinded, placebo-controlled trials to prove it. It has been estimated (though I am not sure how anyone actually came up with the statistic) that at least 50% of modern, allopathic (standard western) medicine is NOT evidence-based. That is, it does NOT have well-designed studies behind its claims. This actually does not surprise me.
It is extremely hard to do properly-designed studies that don't bring some type of bias into their results. All of the pieces of the puzzle have to be put together just right to have a study or trial "strong" enough to really make their claims hold water. They need of sufficient sample size for the "power" of their statistics to hold. They need the proper type of statistical analysis. They need to control for as many variables as possible. They need to minimize the myriad types of bias that might enter into the study and its interpretation.
In fact, it is so hard to find high-quality studies, that in EBM most interventions that are proposed in modern medicine, when evaluated, are graded on a strict taxonomic "grading scale." For example, some type of medical intervention may be given a grade of "A" if the volume of evidence--the number of well-designed studies--clearly shows the intervention to do what it claims to do. If the volume of evidence is more shaky, it might be given a grade of "C."
Needless to say, I am simplifying to an extreme degree--back when I was working on my fellowship this science was still sort of in its infancy. It took until 2004 for the larger groups of physicians who worked in this arena to come to a consensus on a somewhat universal grading system.
Perhaps one of the most relevant applications of Evidence-Based Medicine guidelines involves Preventive Medicine--recommendations for services that primary care physicians provide that are chosen because they help prevent illness and injury and improve quality and length of life. For example, mammograms and Pap smears and colon cancer screening. We have a lot of evidence that has accumulated over the years as to what preventive services really do save lives and which ones sound good but actually don't save lives.
Rather than spout off about individual services in detail, I'll give you the URL to one of EBM's proudest achievements: the United States Preventive Services Task Force Recommendations.
We in Family Practice use these guidelines every day. Insurance companies (generally) follow these guidelines. For example, most insurance companies (I'm proud to say) don't charge copays for proven preventive services such as colonoscopies.
Another link to one of the most widely-respected Evidence-Based reviews of medical interventions is the Cochrane Collaboration:
Cochrane's name is synonymous with high quality reviews of studies on an extremely wide range of medical subjects. I go there often to research questions of the evidence-based validity of some issue I have heard about or read about or was presented by a patient.
Quantitative Decision-Making:
The other part of my work in EBM involved trying to find better, more accurate ways of making difficult medical decisions. As a brand-new physician, I was still uncomfortable with the uncertainty of many of the decisions we had to make. I (and my research cohort physicians) knew there were mathematical and statistical ways to quantify many medical decisions. Our DREAM scenario would unwind this way:
A seriously ill patient presents to a physician. The physician gets an initial history and then consults a computerized data base of relevent tests or procedures. The data base indicates how those tests actually and quantitatively change the probability of the diagnoses the physician is considering. For example, I'm in the ER, and in front of me is a 46 year old man with chest pain. But it is not "typical" heart attack-type chest pain. I would have a rough estimate in my mind that he had perhaps a 15% probability of actually having a heart attack. By consulting my computerized data base, I could then choose which tests (lab tests, CT coronary arteriography, stress test or heart catherization) by quantifiable means I need to do to either prove he is NOT having a heart attack or prove he is. Or in other words, drops his probability of heart attack below, for example, 3%; in which case I'd be ok not keeping him in the hospital for a heart attack; or pushes the probability over say, 75% in which case I'm calling the cardiologist to take over his care.
Tests are not perfect. There are almost always false positives or false negative results. A rapid strep test, for example, if done on one hundred people who actually have strep, will say 5 of those people are negative for strep. When tested on one hundred people who do not have strep, it will falsely say 2 of them do have strep.
Using this kind of data, we can generate how any given test changes the probability that a patient actually has or does not have a disease. Using these sorts of mathematical probabalistic analyses, we often find some completely non-intuitive results.
An example: A college student wants to be tested for HIV. He is not a high risk patient. His general population likelihood of having HIV is around 0.1% (the general prevalence of HIV in his population). So he gets a screening test that is pretty accurate. The screening test turns up positive. Does that mean he has HIV? No, his probability has moved up from 0.1% to only about 9%. There is still a 91% probability he does NOT have HIV. (He would then actually undergo much more specific testing of a second-step nature to confirm or refute the first test results.)
There is a common assumption among the lay public that if you have a positive test, you have the disease. But as can be seen, this is often a fallacy, and not always intuitive. It really becomes a problem when testing large populations at low risk. That is why the vast number of abnormal mammograms are false positives. But we are willing to scare a lot of women with false positives just so we catch the very rare REAL cancers.
This is why the US Preventive Services Task Force evidence-based recommendations are so valuable. Things we think might make sense in terms of screening tests actually DON'T help, and may make things worse. For example, I still occasionally have a female patient who will ask me, "Why don't you screen for ovarian cancer?" The answer is, when we've done studies using the tests available (ultrasound of the ovaries, Ca-125 tumor marker blood tests), we get too many false positives. So when we calculate how many women would then undergo exploratory surgery to see if they actually have ovarian cancer, and how many women would have complications or die from the surgery or anesthesia--more women would be harmed than would be saved. So we do not recommend routine ultrasounds or Ca-125 blood tests as general population screening for ovarian cancer. (Many physicians DO offer these tests to women with a first-degree family history of ovarian cancer, however--as their overall risk is significantly higher than the general population.)
Then there is the issue of the "cascade effect." For awhile a few years back, I'd get an occasional patient who'd ask me if they could have a "whole body CT scan" just to check if anything might be wrong or early cancers might be growing. I'd have to explain that no study has ever shown that whole-body CT scanning prevents any form of cancer or other disease. It would carry too high a false positive rate. (Not to mention a huge radiation exposure and cost.) We might find some little "thing" or shadow showing up in the liver. But we can't tell exactly what that odd CT shadow is in the liver, so we then do an ultrasound. Well, the ultrasound is not clearly defining a harmless lesion, so the patient then gets sent to a GI specialist who is now obligated to do a liver biopsy with a big needle to make sure the lesion is not cancer. So then, during the liver biopsy procedure, a blood vessel gets nicked (a very rare complication of liver biopsy), and the patient then has to go to emergency surgery to fix the nick. By the way, the liver lesion turned out to be a benign and insignificant growth. This is the Cascade Effect--doing one test, which then finds something else that is a "red herring" or of no clinical significance. Then that "something" needs to be investigated with still more testing, etc, etc.
A seriously ill patient presents to a physician. The physician gets an initial history and then consults a computerized data base of relevent tests or procedures. The data base indicates how those tests actually and quantitatively change the probability of the diagnoses the physician is considering. For example, I'm in the ER, and in front of me is a 46 year old man with chest pain. But it is not "typical" heart attack-type chest pain. I would have a rough estimate in my mind that he had perhaps a 15% probability of actually having a heart attack. By consulting my computerized data base, I could then choose which tests (lab tests, CT coronary arteriography, stress test or heart catherization) by quantifiable means I need to do to either prove he is NOT having a heart attack or prove he is. Or in other words, drops his probability of heart attack below, for example, 3%; in which case I'd be ok not keeping him in the hospital for a heart attack; or pushes the probability over say, 75% in which case I'm calling the cardiologist to take over his care.
Tests are not perfect. There are almost always false positives or false negative results. A rapid strep test, for example, if done on one hundred people who actually have strep, will say 5 of those people are negative for strep. When tested on one hundred people who do not have strep, it will falsely say 2 of them do have strep.
Using this kind of data, we can generate how any given test changes the probability that a patient actually has or does not have a disease. Using these sorts of mathematical probabalistic analyses, we often find some completely non-intuitive results.
An example: A college student wants to be tested for HIV. He is not a high risk patient. His general population likelihood of having HIV is around 0.1% (the general prevalence of HIV in his population). So he gets a screening test that is pretty accurate. The screening test turns up positive. Does that mean he has HIV? No, his probability has moved up from 0.1% to only about 9%. There is still a 91% probability he does NOT have HIV. (He would then actually undergo much more specific testing of a second-step nature to confirm or refute the first test results.)
There is a common assumption among the lay public that if you have a positive test, you have the disease. But as can be seen, this is often a fallacy, and not always intuitive. It really becomes a problem when testing large populations at low risk. That is why the vast number of abnormal mammograms are false positives. But we are willing to scare a lot of women with false positives just so we catch the very rare REAL cancers.
This is why the US Preventive Services Task Force evidence-based recommendations are so valuable. Things we think might make sense in terms of screening tests actually DON'T help, and may make things worse. For example, I still occasionally have a female patient who will ask me, "Why don't you screen for ovarian cancer?" The answer is, when we've done studies using the tests available (ultrasound of the ovaries, Ca-125 tumor marker blood tests), we get too many false positives. So when we calculate how many women would then undergo exploratory surgery to see if they actually have ovarian cancer, and how many women would have complications or die from the surgery or anesthesia--more women would be harmed than would be saved. So we do not recommend routine ultrasounds or Ca-125 blood tests as general population screening for ovarian cancer. (Many physicians DO offer these tests to women with a first-degree family history of ovarian cancer, however--as their overall risk is significantly higher than the general population.)
Then there is the issue of the "cascade effect." For awhile a few years back, I'd get an occasional patient who'd ask me if they could have a "whole body CT scan" just to check if anything might be wrong or early cancers might be growing. I'd have to explain that no study has ever shown that whole-body CT scanning prevents any form of cancer or other disease. It would carry too high a false positive rate. (Not to mention a huge radiation exposure and cost.) We might find some little "thing" or shadow showing up in the liver. But we can't tell exactly what that odd CT shadow is in the liver, so we then do an ultrasound. Well, the ultrasound is not clearly defining a harmless lesion, so the patient then gets sent to a GI specialist who is now obligated to do a liver biopsy with a big needle to make sure the lesion is not cancer. So then, during the liver biopsy procedure, a blood vessel gets nicked (a very rare complication of liver biopsy), and the patient then has to go to emergency surgery to fix the nick. By the way, the liver lesion turned out to be a benign and insignificant growth. This is the Cascade Effect--doing one test, which then finds something else that is a "red herring" or of no clinical significance. Then that "something" needs to be investigated with still more testing, etc, etc.
No comments:
Post a Comment