Clinical Trialists Should Use Economies of Scale to Maximize Profits of Large RCTs

The lever is a powerful tool

I am writing (very slowly) a review article about ionized calcium in the ICU - should it be measured, and should it be treated?  There are several recent large observational studies that look at the association between calcium and outcomes of  critical illness, but being observational, they do not offer guidance as to whether chasing calcium levels with calcium gluconate or chloride will improve outcomes or whether hypo- or hyper-calcemia is simply a marker of severity of illness (the latter is of course my bet.)

Thinking about calcium levels and causation and repletion, one cannot help but think about all sorts of other levels we check in the ICU - potassium, magnesium, phosphate - and may other things we routinely do but about which we have no real inkling of an idea as to whether we're doing any patients any good.  (Arterial lines are another example.)  Are we just wasting our time with many of the things we do?  This question becomes more urgent as evidence mounts that much of what we do (in the ICU and elsewhere) is useless, wasteful, or downright harmful.  But who or what agency is going to fund a trial of potassium or calcium replacement in the ICU?  It certainly seems unglamorous.   Don't we have other disease-specific priorities that are paramount in importance to such a trial?

I then realized that a good businessman, wanting to maximize the "profit" from a large, randomized controlled trial (and the dollars "invested" in it), would take advantage of economies of scale.  For those who are not business savvy (I do not imply that I am), business costs can be roughly divided into fixed costs and variable costs.  If you have a factory making widgets you have certain  costs such as the rent, advertising, widget making machines.  These costs are "fixed" meaning that they are invariable whether you make 100 widgets or 10,000 widgets.  Variable costs are the costs of materials, electricity, and human resources which must be scaled up as you make more widgets.  In general, the cost of making each widget goes down as the fixed costs are spread out over more widget units.  Additionally, if you can leverage your infrastructure to make wadgets, a product similar to a widget, you likewise increase profits by lowering costs per unit.

PCI versus CABG - Superiority is in the heart of the angina sufferer

In the current issue of the NEJM, Serruys et al describe the results of a multicenter RCT comparing PCI with CABG for severe coronary artery disease: http://content.nejm.org/cgi/content/full/360/10/961. The trial, which was designed by the [profiteering] makers of drug-coated stents, was a non-inferiority trial intended to show the non-inferiority (NOT the equivalence) of PCI (new treatment) to CABG (standard treatment). Alas, the authors appear to misunderstand the design and reporting of non-inferiority trials, and mistakenly declare CABG as superior to PCI as a result of this study. This error will be the subject of a forthcoming letter to the editor of the NEJM.

The findings of the study can be summarized as follows: compared to PCI, CABG led to a 5.6% reduction in the combined endpoint of death from any cause, stroke, myocardial infarction, or repeat vascularization (P=0.002). The caveats regarding non-inferiority trials notwithstanding, there are other reasons to call into question the interpretation that CABG is superior to PCI, and I will enumerate some of these below.

1.) The study used a ONE-SIDED 95% confidence interval - shame, shame, shame. See: http://jama.ama-assn.org/cgi/content/abstract/295/10/1152 .
2.) Table 1 is conspicuous for the absence of cost data. The post-procedural hospital stay was 6 days longer for CABG than PCI, and the procedural time was twice as long - both highly statistically and clinically significant. I recognize that it would be somewhat specious to provide means for cost because it was a multinational study and there would likely be substantial dispersion of cost among countries, but it seems like neglecting the data altogether is a glaring omission of a very important variable if we are to rationally compare these two procedures.
3.) Numbers needed to treat are mentioned in the text for variables such as death and myocardial infarction that were not individually statistically significant. This is misleading. The significance of the composite endpoint does not allow one to infer that the individual components are significant (they were not) and I don't think it's conventional to report NNTs for non-significant outcomes.
4.) Table 2 lists significant deficencies and discrepancies between pharmocological medical management at discharge which are inadequately explained as mentioned by the editorialist.
5.) Table 2 also demonstrates a five-fold increase in amiodarone use and a three-fold increase in warfarin use at discharge among patients in the CABG group. I infer this to represent an increase in the rate of atrial fibrillation in the CABG patients, but because the rates are not reported, I am kept wondering.
6.) Neurocognitive functioning and the incidence of defecits (if measured), known complications of bypass, are not reported.
7.) It is mentioned in the discussion that after consent, more patients randomized to CABG compared to PCI withdrew consent, a tacit admission of the wariness of patients to submit to this more invasive procedure.

In all, what this trial does for me is to remind me to be wary of an overly-simplistic interpretation of complex data and a tendency toward dichotimous thinking - superior versus inferior, good versus bad, etc.

One interpretation of the data is that a 3.4 hour bypass surgery and 9 days in the hospital !MIGHT! save you from an extra 1.7 hour PCI and another 3 days in the hospital on top of your initial committment of 1.7 hours of PCI and 3 days in the hospital if you wind up requiring revascularization, the primary [only] driver of the composite endpoint. And in payment for this dubiously useful exchange, you must submit to a ~2% increase in the risk of stroke, have a cracked chest, risk surgical wound infection (rate of which is also not reported) pay an unknown (but probably large) increased financial cost, risk some probably large increased risk of atrial fibrillation and therefore be discharged on amiodarone and coumadin with their high rates of side effects and drug-drug interactions, while coincidentally risk being discharged on inadequate medical pharmacological management.

Looked at from this perspective, one sees that beauty is truly in the eye of the beholder.