Showing posts with label ascertainment bias. Show all posts
Showing posts with label ascertainment bias. Show all posts

Tuesday, January 28, 2020

Bad Science + Zealotry = The Wisconsin Witch Hunts. The Case of John Cox, MD

John Cox, MD
I stumbled upon a very disturbing report on NBC News today of a physician couple in Wisconsin accused of abusing their adopted infant daughter.  It is surreal and horrifying and worth a read - not because these physicians abused their daughter, but because they almost assuredly did not.  One driving force behind the case appears to be a well-meaning and perfervid, if misguided and perfidious, pediatrician at University of Wisconsin who, with her group, coined the term "sentinel injuries" (SI) to describe small injuries such as bruises and oral injuries that they posit portend future larger scale abuse.  It was the finding of SI on the adopted infant in the story that in part led to charges of abuse against the father, Dr. Cox, got his child put in protective services, got him arrested, and threatens his career.  Interested readers can reference the link above for sordid and sundry details of the case.

Before delving into the 2013 study in Pediatrics upon which many contentions about SI rest, we should start with the fundamentals.  First, is it plausible that the thesis is correct, that before serious abuse, minor abuse is detectable by small bruises or oral injuries?  Of course it is, and it sounds like a good narrative.  But being a good plausible narrative does not make it true and it is likewise possible that bruises seen in kids who are and are not abused reflect nothing more than accidental injuries from rolling off a support, something falling or dropping on them, somebody dropping them, a sibling jabbing at them with a toy, and a number of things.  To my knowledge, the authors offer no direct evidence that the SIs they or others report have been directly traced to abuse.  They are doing nothing more than inferring that facial bruising is a precursor to Abusive Head Trauma (AHT), and based on their bibliography they have gone out of their way to promote this notion.

Saturday, September 15, 2007

Idraparinux, the van Gogh investigators, and clinical trials pointillism: connecting the dots shows that Idraparinux increases the risk of death

It eludes me why the NEJM continues to publish specious, industry-sponsored, negative, non-inferiority trials. Perhaps they do it for my entertainment. And this past week, entertained I was indeed.

Idraparinux is yet another drug looking for an indication. Keep looking, Sanofi. Your pipeline problems will not be solved by this one.

First, let me dismiss the second article out of hand: it is not fair to test idraparinux against placebo (for the love of Joseph!) for the secondary prevention of VTE after a recent epidode! (http://content.nejm.org/cgi/content/short/357/11/1105).

It is old news that one can reduce the recurrence of VTE after a recent episode by either using low intensity warfarin (http://content.nejm.org/cgi/content/abstract/348/15/1425) or by extending the duration of warfarin anticoagulation (http://content.nejm.org/cgi/content/abstract/345/3/165). Therefore, the second van Gogh study does not merit further consideration, especially given the higher rate of bleeding in this study.


Now for the first study and its omissions and distortions. It is important to bear in mind that the only outcome that cannot be associated with ascertainment bias (assuming a high follow-up rate) is mortality, AND that the ascertainment of DVT and PE are fraught with numerous difficulties and potential biases.

The Omission: Failure to report in the abstract that Idraparinux use was associated with an increased risk of death in these studies, which was significant in the PE study, and which trended strongly in the DVT study. The authors attempt to explain this away by suggesting that the increased death rate was due to cancer, but of course we are not told how causes of death were ascertained (a notoriously difficult and messy task), and cancer is associated with DVT/PE which is among the final common pathways of death from cancer. This alone, this minor factoid that Idraparinux was associated with an increased risk of death should doom this drug and should be the main headline related to these studies.

Appropriate headline: "Idraparinux increases the risk of death in patients with PE and possibly DVT."

If we combine the deaths in the DVT and PE studies, we see that the 6-month death rates are 3.4% in the placebo group and 4.5% in the idraparinux group, with an overall (chi-square) p-value of 0.035 - significant!

This is especially worrisome from a generalizability perspective - if this drug were approved and the distinction between DVT and PE is blurred in clinical practice as it often is, we would have no way of being confident that we're using it in a DVT patient rather than a PE patient. Who wants such a messy drug?

The Obfuscations and Distortions: Where to begin? First of all, no justification of an Odds Ratio of 2.0 as a delta for non-inferiority is given. Is twice the odds of recurrent DVT/PE insignificant? It is not. This Odds Ratio is too high. Shame.

To give credit where it is due, the investigation at least used a one sided 0.025 alpha for the non-inferiority comparison.

Second, regarding the DVT study, many if not the majority of patients with DVT also have PE, even if it is subclinical. Given that ascertainment of events (other than death) in this study relied on symptoms and was poorly described, that patients with DVT were not routinely tested for PE in the absence of symptoms, and that the risk of death was increased with idraparinux in the PE study, one is led to an obvious hypothesis: that the trend towary an increased risk of death in the DVT study patients who received idraparinux was due to unrecognized PE in some of these patients. The first part of the conclusion in the abstract "in patients with DVT, once weekly SQ idraparinux for 3 or 6 months had an efficacy similar to that of heparin and vitamin K antagonists" obfuscates and conceals this worrisome possibility. Many patients with DVT probably also had undiagnosed PE and might have been more likely to die given the drug's failure to prevent recurrences in the PE study. The increased risk of death in the DVT study might have been simply muted and diluted by the lower frequency of PE in the patients in the DVT study.

Then there is the annoying the inability to reverse the effects of this drug with a very long half-life.

Scientific objectivity and patient safety mandate that this drug not receive further consideration for clinical use. Persistence with the study of this drug will most likely represent "sunk cost bias" on the part of the manufacturer. It's time to cut bait and save patients in the process.


Monday, August 20, 2007

Prophylactic Cranial Irradiation: a matter of blinding, ascertainment, side effects, and preferences

Slotman et al (August 16 issue of NEJM: http://content.nejm.org/cgi/content/short/357/7/664) report a multicenter RCT of prophylactic cranial irradiation for extensive small cell carcinoma of the lung and conclude that it not only reduces symptomatic brain metastases, but also prolongs progression-free and overall survival. This is a well designed and conducted non-industry-sponsored RCT, but several aspects of the trial warrant scrutiny and temper my enthusiasm for this therapy. Among them:

The trial is not blinded (masked is a more sensitive term) from a patient perspective and no effort was made to create a sham irradiation procedure. While unintentional unmasking due to side effects may have limited the effectiveness of a sham procedure, it may not have rendered it entirely ineffective. This issue is of importance because meeting the primary endpoint was contingent on patient symptoms, and a placebo effect may have impacted participants’ reporting of symptoms. Some investigators have gone to great lengths to tease out placebo effects using sham procedures, and the results have been surprising (e.g., knee arthroscopy; see: https://content.nejm.org/cgi/content/abstract/347/2/81?ck=nck).


We are not told if investigators, the patient’s other physicians, radiologists, and statisticians were masked to the treatment assignment. Lack of masking may have led to other differences in patient management, or to differences in the threshold for ordering CT/MRI scans. We are not told about the number of CT/MRI scans in each group. In a nutshell: possible ascertainment bias (see http://www.consort-statement.org/?o=1123).

There are several apparently strong trends in QOL assessments, but we are not told what direction they are in. Significant differences in these scores were unlikely to be found as the deck was stacked when the trial was designed: p<0.01 was required for significance of QOL assessments. While this is justified because of multiple comparisons, it seems unfair to make the significance level for side effects more conservative than that for the primary outcome of interest (think Vioxx here). The significance level required for secondary endpoints (progression-free and overall survival) was not lowered to account for multiple comparisons. Moreover, more than half of QOL assessments were missing by 9 months, so this study is underpowered to detect differences in QOL. It is therefore all the more important to know the direction of the trends that are reported.

The authors appear to “gloss over” the significant side effects associated with this therapy. It made some subjects ill.

If we are willing to accept that overall survival is improved by this therapy (I’m personally circumspect about this for the above reasons) the bottom line for patients will be whether they would prefer on average 5 additional weeks of life with nausea, vomiting weight loss, fatigue, anorexia, and leg weakness to 5 fewer weeks of life without these symptoms. I think I know what choice many will make, and our projection bias may lead us to make inaccurate predictions of their choices (see Lowenstein, Medical Decision Making, Jan/Feb 2005: http://mdm.sagepub.com/cgi/content/citation/25/1/96).

The authors state in the concluding paragraph:

“Prophylactic cranial irradiation should be part of standard care for all patients with small-cell lung cancer who have a response to initial chemotherapy, and it should be part of the standard treatment in future studies involving these patients.”

I think the decision to use this therapy is one that only patients are justified making. At least now we have reasonably good data to help them inform their choice.