The Crisis in Nutritional Science

I've mentioned John P.A. Ioannidis on my pages many times before (the first, I think).  He's the author of what’s widely quoted as one of the most downloaded papers in history, “Why Most Published Research Findings are False 2”, in which he presents data that as much as 70% of published science is wrong.

Last year, he extended his purview to probably the richest source of bad science, nutritional epidemiology, in an article in the Journal of the American Medical Association.  Bottom line, this field has got to be fixed because it is just so far from good science that it’s dangerous. Not only is it endangering peoples’ health, it’s ruining confidence in science as a way of finding out how the world works. You can read and download the paper (2 page pdf) here.  The unusual part of getting to this article is that I bounced there from Watts Up With That, a post by frequent guest author Kip Hansen, "Epidemiology, Diet Soda, and Climate Science".  You should RTWT. 

As I always do, some quotes to get you to read it.

In recent updated meta-analyses of prospective cohort studies, almost all foods revealed statistically significant associations with mortality risk.  Substantial deficiencies of key nutrients (e.g., vitamins), extreme over consumption of food, and obesity from excessive calories may indeed increase mortality risk.  However, can small intake differences of specific nutrients, foods, or diet patterns with similar calories causally, markedly, and almost ubiquitously affect survival?

Ioannidis' clear thinking comes across very clearly here.  He's saying that when they looked at epidemiological nutrition studies, almost every food item looked at had “statistically significant associations with mortality risk” or in other words, everything we eat is killing us faster and sooner, or making us live longer, and he asks if that's even possible.  Everything?  Nothing is neutral or has no effect?

Perhaps my favorite paragraph from the whole article (emphasis added):

Assuming the meta-analyzed evidence from cohort studies represents life span–long causal associations, for a baseline life expectancy of 80 years, eating 12 hazelnuts daily (1 oz) would prolong life by 12 years (i.e., 1 year per hazelnut), drinking 3 cups of coffee daily would achieve a similar gain of 12 extra years, and eating a single mandarin orange daily (80 g) would add 5 years of life.  Conversely, consuming 1 egg daily would reduce life expectancy by 6 years, and eating 2 slices of bacon (30 g) daily would shorten life by a decade, an effect worse than smoking.  Could these results possibly be true?

Before you order your yearly 23 pounds of hazelnuts, hold on a minute.  It stretches credulity to think all of those could be true.  One year of extra life for every hazelnut eaten daily?  Or 6 years less life for one egg eaten daily? What happens if you have one hazelnut and and one egg daily?  Do the effects cancel?  Does only one year cancel, so you only die 5 years sooner?  These answer are the result of the way these meta-analyses work; they find spurious correlations.  You might recall an article I did on the King of Junk Food Science (where the adjective "junk" modifies science, not food) and a link to a FiveThirtyEight column where they post funny spurious correlations they found.  In these nutritional studies, they frequently study "all-cause mortality", but the top causes of mortality include accidents (#3) and medical mistakes (usually left out of the rankings, but numerically could be more than accidents, taking #3).  How could they be improving mortality dramatically without affecting those to some degree? 

Individuals consume thousands of chemicals in millions of possible daily combinations. For instance, there are more than 250 000 different foods and even more potentially edible items, with 300 000 edible plants alone. Seemingly similar foods vary in exact chemical signatures (e.g., more than 500 different polyphenols). Much of the literature silently assumes disease risk is modulated by the most abundant substances; for example, carbohydrates or fats. However, relatively uncommon chemicals within food, circumstantial contaminants, serendipitous toxicants, or components that appear only under specific conditions or food preparation methods (e.g., red meat cooking)may be influential. Risk-conferring nutritional combinations may vary by an individual’s genetic background, metabolic profile, age, or environmental exposures. Disentangling the potential influence on health outcomes of a single dietary component from these other variables is challenging, if not impossible.

Dr. Ioannidis concludes nutritional epidemiology is intrinsically unreliable.  It produces results that cannot be considered causal.  I hope/trust that's enough to get you interested in reading the article because I can't do it much justice without well exceeding the limits of TL:DR.  I recommend the version on WUWT rather than the original JAMA paper, if you're only going to read one.  Author Kip Hansen shows the mess that is nutritional epidemiology and then compares the field to climate science, another field with an incredible number of variables that may or may not interact with each other.

Similarly, for climate science, the object of study, the Earth’s climate system is not only exceptionally complex, but also chaotic.  First, we have to understand that, as we see in nutrition science, climate is comprised of hundreds of interacting components, each changing on time scales ranging from seconds to centuries, each being integral influencing and causal factors for the others — all correlated in ways we often (almost always) do not fully understand.  And, as in nutrition science, almost all climate variables are correlated with one another; thus, if one variable is found to be correlated to some weather/climate  outcome, many other variables will also yield significant associations in the huge present-time and historical data sets relating to Earth’s weather and climate.

Thus we find the situation, unacknowledged by most of the climate science field, that [paraphrasing Ioannidis] “Disentangling the potential influence on medium to long range climate outcomes of a single climatic factor, such as atmospheric GHG concentrations,  from these myriad other variables is challenging, if not impossible” based simply on the complexity of the climate itself.

John Ioannidis - Stanford University photo