Hope: The Mother of Bias in Research

I realized the other day that underlying every slanted report or overly-optimistic interpretation of a trial's results, every contorted post hoc analysis, every Big Pharma obfuscation, is hope.  And while hope is generally a good, positive emotion, it engenders great bias in the interpretation of medical research.  Consider this NYT article from last month:  "Dashing Hopes, Study Shows Cholesterol Drug Had No Effect on Heart Health."  The title itself reinforces my point, as do several quotes in the article.

“All of us would have put money on it,” said Dr. Peter Libby, a Harvard cardiologist. The drug, he said, “was the great hope.”

 Again, hope is wonderful, but it blinds people to the truth in everyday life and I'm afraid researchers are no more immune to its effects than the laity.  In my estimation, three main categories of hope creep into the evaluation of research and foments bias:

1. Hope for a cure, prevention, or treatment for a disease (on the part of patients, investigators, or both)
2. Hope for career advancement, funding, notoriety, being right (on the part of investigators) and related sunk cost bias
3. Hope for financial gain (usually on the part of Big Pharma and related industrial interests)

Consider prone positioning for ARDS.  For over 20 years, investigators have hoped that prone positioning improves not only oxygenation but also outcomes (mostly mortality).  So is it any wonder that after the most recent trial, in spite of the 4 or 5 previous failed trials, the community enthusiastically declared "success!"  "Prone Positioning works!"  Of course it is no wonder - this has been the hope for decades.

But consider what the most recent trial represents through the lens of replicability:  a failure to replicate previous results showing that prone positioning does not improve mortality.  The recent trial is the outlier.  It is the "false positive" rather than the previous trials being the "false negatives."

This way of interpreting the trials of prone positioning in the aggregate should be an obvious one, and it astonishes me that it took me so long to see the results this way - as a single failure to replicate previously replicable negative results.  But it hearkens to the underlying bias - we view results through the magnifying glass of hope, and it distorts our appraisal of the evidence.

Indeed, I have been accused of being a nihilist because of my views on this blog, which some see as derogating the work of others or an attempt to dash their hopes.  But these critics engage, or wish me to engage in a form of outcome bias - the value of the research lies in the integrity of its design, conduct, analysis, and reporting, not in its results.  One can do superlative research and get negative results, or shoddy research and get positive results.  My goal here is and always has been to judge the research on its merits, regardless of the results or the hopes that impel it.

(Aside:  Cholesterol researchers have a faith or hope in the cholesterol hypothesis - that cholesterol is a causal factor in pathways to cardiovascular outcomes.  Statin data corroborate this, and preliminary PCSK9 inhibitor data do, too.  But how quickly we engage in hopeful confirmation bias!  If cholesterol is a causal factor, it should not matter how you manipulate it - lower the cholesterol, lower cardiovascular events.  The fact that it does appear to matter how you lower it suggests that either there are multiplicity of agent effects (untoward and unknown effects of some agents negate some their beneficial effects in the cholesterol causal pathway) or that cholesterol levels are epiphenomena - markers of the effects of statins and PCSK9 inhibitors on the real, but as yet undelineated causal pathways.  Maybe the fact that we can easily measure cholesterol and that it is associated with outcomes in untreated individuals is a convenient accident of history that led us to trial statins which work in ways that we do not yet understand.)

Brief Updates on Hypothermia, Hyperglycemia, Cholesterol, Blood Pressure Lowering in Stroke and Testosterone

I've read a lot of interesting articles recently, but none that are sufficient fodder for a dedicated post.  So here I will update some themes from previous blog posts with recent articles from NEJM and JAMA that relate to them.

Prehospital Induction of Hypothermia After Cardiac Arrest
In this article in the January 1st issue of JAMA, investigators from King County Washington report the results of a trial which tested the hypothesis that earlier (prehospital) induction of hypothermia, by infusing cold saline, would augment the assumed benefit of hypothermia that is usually initiated in the hospital for patients with ventricular fibrillation.  Please guess what was the effect of this intervention on survival to hospital discharge and neurological outcomes.

You were right.  There was not even a signal, not a trend towards benefit, even though body temperature was lower by 1 degree Celcius and time to target hypothermia temperature in the hospital was one hour shorter.  However, the intervention group experienced re-arrest in the field significantly more often than the control group and had more pulmonary edema and diuretic use.  Readers interested in exploring this topic further are referred to this post on Homeopathic Hypothermia.

Hyperglycemic Control in Pediatric Intensive Care
In this article in the January 9th issue of NEJM, we are visited yet again by the zombie topic that refuses to die.  We keep looking for subgroups or populations that will benefit, and if we find one that appears to, it will be a Type I error, thinks the blogger with Bayesian inclinations.  In this trial, 1369 pediatric patients at 13 centers in England were randomized to tight versus conventional glycemic control.  Consistent with other trials in other populations, there was no benefit in the primary outcome, but tightly "controlled" children had much more and severe hypoglycemia.  The "cost effectiveness" analysis they report is irrelevant.  You can't have "cost effectiveness" of an ineffective therapy.  My, my, how we continue to grope.

The Cardiologist Giveth, then the Cardiologist Taketh Away: Revision of the Cholesterol Guidelines

There has been quite a stir this week with the publication of the newest revision of the ACC/AHA guidelines for the treatment of cholesterol.  The New York Times is awash with articles summarizing or opining on the changes and many of the authors are perspicacious observers:

• 3 Things to Know About the New Cholesterol Guidelines by Harlan Krumholz of Yale
• Don’t Give More Patients Statins by authors from Harvard and UCSF
• Experts Reshape Treatment Guide for Cholesterol
• New Cholesterol Advice Startles Even Some Doctors
• Questions For A New Class of Cholesterol Drugs

As the old Spanish proverb states, "rio revuelto, ganancia de pescadores" - when the river is stirred up, the fishermen benefit.  I will admit that I'm gloating a bit since I consider the new guidelines to be a tacit affirmative nod to several posts on the topic of the cholesterol hypothesis (CH).  (More posts here and here and here, among several others - search for "cholesterol" or "causal pathways" on the Medical Evidence Blog search bar.)

The Cholesterol Hypothesis on the Beam: Dalcetrapib, PCSK9 inhibitors, and "off-target" effects of statins

The last month has witnessed the publication of three lines of research that could tip the balance of the evidence for the cholesterol hypothesis depending how things play out.  Followers of this blog know that I have a healthy degree of skepticism for the cholesterol hypothesis which was emboldened by studies of torcetrapib (blogged here and here) and anacetrapib that have come to light along with the failures of vytorin (ezetimibe; blogged here and here and here) and the addition of niacin to statins to improve cardiovascular outcomes in parallel with improvements in cholesterol numbers.

I think it's finally time to bury the CETP inhibitors. The November 29th NEJM (published online on November 5th) reports the results of the dal-OUTCOMES trial of dalcetrapib in patients with a recent acute coronary syndrome. Almost 16,000 patients were enrolled in this study of high risk patients, providing the study with ample power to detect meaningful improvements in cardiovascular outcomes - but alas, none were detected. The target is HDL, so the LDL hypothesis is not debunked by these data, but I think it is challenged nonetheless.

Luck that Looks Like Logic? Statins (Rosuvastatin), the Cholesterol Hypothesis, and Causal Pathways

The Cholesterol Hypothesis (CH), namely that the association between elevated cholesterol (LDL) and cardiovascular disease and events is a CAUSAL one, and thus that intervening to lower cholesterol prevents these diseases has seduced mainstream medicine for decades. However, much if not most of the evidence for the causality of cholesterol in atherogenesis and its reversal by lowering cholesterol derives from studies of "Statins" or HMG-CoA-reductase inhibitors; indeed the evidence that lowering LDL cholesterol (or raising HDL) through other pathways has salutary effects on cardiovascular outcomes is scant at best as has been chronicled on this blog (see posts on torcetrapib and ezetimibe/Vytorin). Not myself immune to the beguiling allure of the CH, I admit that I take Niacin, in spite of normal HDL levels and scant to no trustworthy evidence that, in addition to raising HDL and lowering LDL, it will have any primary (or secondary or tertiary) preventative effects for me.

In yesterday's NEJM, Glynn et al report the results of analysis of data on a secondary endpoint from the JUPITER trial of Rosuvastatin. (http://content.nejm.org/cgi/content/abstract/360/18/1851 .) The primary aim of the trial was to determine if Rosuvastatin was effective for primary prevention of cardiovascular events in people with normal cholesterol levels and elevated CRP levels. The secondary endpoint described in the article was the occurrence of venothromboembolism during the study period. Because I see no obvious evidence of foul play, and because this study was simply impeccably designed, conducted, and reported, I'm going to hereafter ignore the fact that it was industry sponsored, and that there is probably some motive of "off-label promotion by proxy" (http://medicalevidence.blogspot.com/2008/06/off-label-promotion-by-proxy-how-nejm.html .) here...

Lo and behold: Rosuvastatin lowered venothromboembolism rates. The difficulties posed by ascertainment of this outcome notwithstanding, this trial has convincing evidence of a statistically significant reduction in DVT and PE event rates (which were very low - ~0.2%/100 persons/year) during the four year period of study. And this does not make a whole lot of sense from the standpoint of the CH. There's something more going on. Like an anti-inflammatory property of Statins. Which is very interesting and noteworthy and worthwhile in its own right. But I'm more interested in what kind of light this sheds on the validity of the CH.

Because of my interest in the fraility of the normalization hypothesis/heuristic (the notion that you just measure something and then raise or lower it to the normal range and make things ALL better) I am obviously a reserved skeptic of the Cholesterol Hypothesis, which was bolstered by if not altogether reared by data from trials of statins. And these new data, combined with emerging evidence that statins may have salutary effects on lung inflammation in ARDS and COPD, among perhaps others, make me wonder - was it just pure LUCK rather than a triumph of LOGIC that the first widely tested and marketed drug for cholesterol happened to both reduce cardiovascular endpoints AND lower cholesterol, even though not necessarily as part of the same causal pathway? Is it just "true, true, and unrelated?" Are they the anti-inflammatory properties or some other piece of the complex biochemical effects of these drugs on the body that leads to their clinical benefits? Other examples come to mind: Is blood pressure lowering just an epiphenomenon of another primary ACE-inhibitor effect on heart failure? Because these effects appear to be superficially and intuitively related does not mean that they are an obvious causal pathway.

What if things had happened another way. What if Statins had eluded discovery for another 20-30 years. What if study of the cholesterol hypothesis meanwhile proceeded through evaluation of Cholestyramine, Cholestipol, Niacin, and other drugs, and what if it had been "disconfirmed" by failure of these agents to reduce cardiovascular outcomes? These hypotheticals will be answerable only after more study of Statins and other drugs as well as their mechanisms. The data presented by the Harvard group as well as their other work with CRP are but one leg of a long journey toward elucidation of the biological mechanisms of atherogenesis, coagulation, and downstream clinical events.