Fixing Science for the 21st Century and Beyond

Hat tip to Zendo Deb at 357 Magnum for the story in “Einstein, We Have a Problem”.  The link, in turn, is to an online magazine I didn't previously know, The New Atlantis and “Saving Science”, one of a three part series on the Integrity of Science.  I've only been able to read this first article, and really recommend it to anyone concerned about the problems with science, some of which I've written about before.  Let me borrow the same money quote that Zendo Deb uses:

The science world has been buffeted for nearly a decade by growing revelations that major bodies of scientific knowledge, published in peer-reviewed papers, may simply be wrong. Among recent instances: a cancer cell line used as the basis for over a thousand published breast cancer research studies was revealed to be actually a skin cancer cell line; a biotechnology company was able to replicate only six out of fifty-three “landmark” published studies it sought to validate; a test of more than one hundred potential drugs for treating amyotrophic lateral sclerosis in mice was unable to reproduce any of the positive findings that had been reported from previous studies; a compilation of nearly one hundred fifty clinical trials for therapies to block human inflammatory response showed that even though the therapies had supposedly been validated using mouse model experiments, every one of the trials failed in humans; a statistical assessment of the use of functional magnetic resonance imaging (fMRI) to map human brain function indicated that up to 70 percent of the positive findings reported in approximately 40,000 published fMRI studies could be false; and an article assessing the overall quality of basic and preclinical biomedical research estimated that between 75 and 90 percent of all studies are not reproducible.

Not surprisingly he quotes John Ioannidis' famous 2005 paper, “Why Most Published Research Findings are False”, along with some more of his research.  I first wrote about John Ioannidis back in 2013.  His paper may have been the one that opened the floodgates, because the number of papers retracted annually started to blossom in that decade:

The number of retracted scientific publications rose tenfold during the first decade of this century, and although that number still remains in the mere hundreds, the growing number of studies such as those mentioned above suggests that poor quality, unreliable, useless, or invalid science may in fact be the norm in some fields, and the number of scientifically suspect or worthless publications may well be counted in the hundreds of thousands annually.

Ioannidis says the most common fields for bad research papers are health, biomedicine, and psychology, but that may just be a function of where research money is going.

“Saving Science” is a far bigger article than just these few things.  Author Daniel Sarewitz spends time discussing how we got into this mess.  As WWII came to a close, there was an acknowledgement of how much that scientific teams had contributed to the victory and a deliberate effort to keep those teams together.  Vannevar Bush, the MIT engineer called the “General of Physics” by Time Magazine, was the public face behind this push.  He pushed a vision so appealing in its imagery that everyone bought into it.

Scientific progress on a broad front results from the free play of free intellects, working on subjects of their own choice, in the manner dictated by their curiosity for exploration of the unknown.

Through example after example Sarewitz demonstrates that the progress of the late 20th century was virtually never, “free play of free intellects, working on subjects of their own choice”, but instead was almost always science being managed, being driven on specific topics for specific applications.   Scientific knowledge advances most rapidly, and is of most value to society, when it is steered to solve problems — especially those related to technological innovation.  Could it be that the War on Cancer has floundered because there's nobody in charge; nobody driving toward a goal and asking specific people specific questions? 

From how the DOD pushed the development of the transistor, while putting up with poor results at first, to how the “War on Cancer” turned into individual scientists looking for findings that would give them a good paper and a good reputation rather than saving lives, it's a fascinating story.  I have to admit that my initial reaction was that he was calling for more government oversight of science, bigger government, but as I read the article that slipped away, and what I think I see is a call for science to be more applied and not just, well, playing at whatever they find interesting: “the free play of free intellects, working on subjects of their own choice”. 

A topic he didn't address was the politicization of science and how that drives bad science.  In John Ioannidis' landmark 2005 paper he said one of the important factors associated with results being false is politicization: the hotter the research field, the more likely the results are to be false.  Which explains Climate Science in one sentence.

One thing is for sure: as taxpayers, we're paying for a lot of bad science.  For that reason alone, we should care.

Yeah, it's a good half hour read with no distractions in the room and it's worth every minute of it.