I was reading about diagnostic tests for sepsis on www.pulmccm.org just now. Excellent site to keep yourself up-to-date on all matters Pulmonary and Critical Care, by the way. It's discussing the Septicyte Rapid and the Intellisep proprietary tests. See link above for the article. These tests are technology looking for an indication and cash flow, as unfortunately so many are. Let me tell you a little secret about the studies cited that are purported to support the diagnostic utility of these tests.
Ignore Any Report of +/- Predictive Value of a Diagnostic Test.
If you know that, and to look instead for the sensitivity and specificity of the test, you can just memorize my rule and move on to your next task of self-edification today. But if you want to understand why, read on.
The predictive value (+ or -) of a test depends on sensitivity and specificity of the test and the "prevalence" of the disease under consideration in the tested population. If you're trying to understand a test, you don't care about the prevalence of disease in that population, because it may not reflect the prevalence in your population of interest. Sensitivity and Specificity are the measures of the test itself, in isolation. So whenever you see positive and negative predictive value reported, you can bet your arse that it's because the either low or high prevalence of disease in the test population made the PPV (positive predictive value) and NPV (negative predictive value) look better than the sensitivity and specificity.
(Recondite but pivotal fact: Bayes' theorem, which integrates sensitivity and specificity and prevalence to get PPV and NPV relies on an assumption of conditional independence, meaning that the sensitivity and specificity don't change with changing prevalence. Usually it is unknown whether that assumption is true in a specific case, but we're stuck with it.)
In order to be a good test, it needs to have high sensitivity or specificity, or both, which leads to good likelihood ratios (which integrate sens/spec for combination with disease "prevalence" in the test population). High sensitivity/specificity is on the order of 90% or greater as you once learned; a test with both over 90% is a pretty good test, if the values can be trusted. If either of them are below the 90% threshold, beware. A worst case scenario is the "rule of 100" that states that if the sensitivity and specificity sum to near 100, that test is useless; that is, it doesn't change the pre-test probability of disease. If you want to play around and test the rule, go over to the calculator on the sidebar of the sister blog, Status Iatrogenicus.
Any "study" of a diagnostic test that fails to state clearly the sensitivity and specificity, reporting instead PPV and NPV, is trying to pull the wool over your eyes until proven otherwise. But even when authors do report sensitivity and specificity, know that they are notoriously optimistic; i.e., inflated. This can come about through the usual legerdemain, especially selection bias in the populations tested, exclusion criteria (as mentioned in the PulmCCM post), lack of a gold standard, publication bias, and outright fraud.
As just one example, troponin (by the way, I hate troponin) is stated by the cardiologists to be not only exquisitely sensitive, but highly specific too. I have been hearing this since medical school in the 1990s. It is not specific, unless you select your population carefully. There is always a trade-off between sensitivity and specificity, and this "High Sensitivity Troponin This and That" test that is ordered wantonly and recklessly suffers for lack of specificity. Such tests are positive in distance runners, as one example. And the reason the cardiologists report such high sensitivity for troponin is because their diagnostic test samples only include people with angina/chest pain and no other obvious non-cardiac diagnosis. They don't include people who just completed a marathon, women with urosepsis, old dudes with emphysema, folks with pneumonia and septic shock. Yet those are just the people who get troponin ordered in the ED as part of a "rainbow" of tests, which then must be considered cautiously by returning to the clinical scenario and the clinical pre-test probability.
Finally, encouraging you to default to sensitivity and specificity serves as a reminder that you must consider the disease prevalence/pre-test probability for your particular patient; you cannot assume that it is the same as in the study test population.
No comments:
Post a Comment