Why Most True Research Findings Are Useless

In his provocative essay in PLOS Medicine over a decade ago, Ioannidis argued that most published research findings are false, owing to a variety of errors such as p-hacking, data dredging, fraud, selective publication, researcher degrees of freedom, and many more.  In my permutation of his essay, I will go a step further and suggest that even if we limit our scrutiny to tentatively true research findings (scientific truth being inherently tentative), most research findings are useless.

My choice of the word "useless" may seem provocative, and even untenable, but it is intended to have an exquisitely specific meaning:  I mean useless in an economic sense of "having zero or negligible net utility", in the tradition of Expected Utility Theory [EUT], for individual decision making.  This does not mean that true findings are useless for the incremental accrual of scientific knowledge and understanding.  True research findings may be very valuable from the perspective of scientific progress, but still useless for individual decision making, whether it is the individual trying to determine what to eat to promote a long healthy life, or the physician trying to decide what to do for a patient in the ICU with delirium.  When evaluating a research finding that is thought to be true, and may at first blush seem important and useful, it is necessary to make a distinction between scientific utility and decisional utility.  Here I will argue that while many "true" research findings may have scientific utility, they have little decisional utility, and thus are "useless".

The Cardiologist Giveth, then the Cardiologist Taketh Away: Revision of the Cholesterol Guidelines

There has been quite a stir this week with the publication of the newest revision of the ACC/AHA guidelines for the treatment of cholesterol.  The New York Times is awash with articles summarizing or opining on the changes and many of the authors are perspicacious observers:

• 3 Things to Know About the New Cholesterol Guidelines by Harlan Krumholz of Yale
• Don’t Give More Patients Statins by authors from Harvard and UCSF
• Experts Reshape Treatment Guide for Cholesterol
• New Cholesterol Advice Startles Even Some Doctors
• Questions For A New Class of Cholesterol Drugs

As the old Spanish proverb states, "rio revuelto, ganancia de pescadores" - when the river is stirred up, the fishermen benefit.  I will admit that I'm gloating a bit since I consider the new guidelines to be a tacit affirmative nod to several posts on the topic of the cholesterol hypothesis (CH).  (More posts here and here and here, among several others - search for "cholesterol" or "causal pathways" on the Medical Evidence Blog search bar.)