Why Most True Research Findings Are Useless

In his provocative essay in PLOS Medicine over a decade ago, Ioannidis argued that most published research findings are false, owing to a variety of errors such as p-hacking, data dredging, fraud, selective publication, researcher degrees of freedom, and many more.  In my permutation of his essay, I will go a step further and suggest that even if we limit our scrutiny to tentatively true research findings (scientific truth being inherently tentative), most research findings are useless.

My choice of the word "useless" may seem provocative, and even untenable, but it is intended to have an exquisitely specific meaning:  I mean useless in an economic sense of "having zero or negligible net utility", in the tradition of Expected Utility Theory [EUT], for individual decision making.  This does not mean that true findings are useless for the incremental accrual of scientific knowledge and understanding.  True research findings may be very valuable from the perspective of scientific progress, but still useless for individual decision making, whether it is the individual trying to determine what to eat to promote a long healthy life, or the physician trying to decide what to do for a patient in the ICU with delirium.  When evaluating a research finding that is thought to be true, and may at first blush seem important and useful, it is necessary to make a distinction between scientific utility and decisional utility.  Here I will argue that while many "true" research findings may have scientific utility, they have little decisional utility, and thus are "useless".

Tell Them to Go Pound Salt: Ideology and the Campaign to Legislate Dietary Sodium Intake

In the March 28th, 2013 issue of the NEJM, a review of sorts entitled "Salt in Health and Disease - A Delicate Balance" by Kotchen et al can be found.  My interest in this topic stems from my interest in the question of association versus causation, my personal predilection for salt, my observation that I lose a good deal of sodium in outdoor activities in the American Southwest, and my concern for bias in the generation of and especially the implementation of evidence in medicine as public policy.

This is an important topic, especially because sweeping policy changes regarding the sodium content of food are proposed, but it is a nettlesome topic to study, rife with hobgoblins.  First we need a well-defined research question:  does reduction in dietary sodium intake:  a.) reduce blood pressure in hypertensive people?  in all people?  b.) does this reduction in hypertension lead to improved outcomes (hypertension is in some ways a surrogate marker)?  In a utopian world, we would randomize thousands of participants to diets low in sodium and "normal" in sodium, we would measure sodium intake carefully, and we would follow the participants for changes in blood pressure and clinical outcomes for a protracted period.  But alas, this has not been done, and it will not likely be done because of cost and logistics, among other obstacles (including ideology).