Wednesday, August 31, 2016

Fixing Science for the 21st Century and Beyond

Hat tip to Zendo Deb at 357 Magnum for the story in “Einstein, We Have a Problem”.  The link, in turn, is to an online magazine I didn't previously know, The New Atlantis and “Saving Science”, one of a three part series on the Integrity of Science.  I've only been able to read this first article, and really recommend it to anyone concerned about the problems with science, some of which I've written about before.  Let me borrow the same money quote that Zendo Deb uses:
The science world has been buffeted for nearly a decade by growing revelations that major bodies of scientific knowledge, published in peer-reviewed papers, may simply be wrong. Among recent instances: a cancer cell line used as the basis for over a thousand published breast cancer research studies was revealed to be actually a skin cancer cell line; a biotechnology company was able to replicate only six out of fifty-three “landmark” published studies it sought to validate; a test of more than one hundred potential drugs for treating amyotrophic lateral sclerosis in mice was unable to reproduce any of the positive findings that had been reported from previous studies; a compilation of nearly one hundred fifty clinical trials for therapies to block human inflammatory response showed that even though the therapies had supposedly been validated using mouse model experiments, every one of the trials failed in humans; a statistical assessment of the use of functional magnetic resonance imaging (fMRI) to map human brain function indicated that up to 70 percent of the positive findings reported in approximately 40,000 published fMRI studies could be false; and an article assessing the overall quality of basic and preclinical biomedical research estimated that between 75 and 90 percent of all studies are not reproducible.
Not surprisingly he quotes John Ioannidis' famous 2005 paper, “Why Most Published Research Findings are False”, along with some more of his research.  I first wrote about John Ioannidis back in 2013.  His paper may have been the one that opened the floodgates, because the number of papers retracted annually started to blossom in that decade:
The number of retracted scientific publications rose tenfold during the first decade of this century, and although that number still remains in the mere hundreds, the growing number of studies such as those mentioned above suggests that poor quality, unreliable, useless, or invalid science may in fact be the norm in some fields, and the number of scientifically suspect or worthless publications may well be counted in the hundreds of thousands annually.
Ioannidis says the most common fields for bad research papers are health, biomedicine, and psychology, but that may just be a function of where research money is going.

“Saving Science” is a far bigger article than just these few things.  Author Daniel Sarewitz spends time discussing how we got into this mess.  As WWII came to a close, there was an acknowledgement of how much that scientific teams had contributed to the victory and a deliberate effort to keep those teams together.  Vannevar Bush, the MIT engineer called the “General of Physics” by Time Magazine, was the public face behind this push.  He pushed a vision so appealing in its imagery that everyone bought into it.
Scientific progress on a broad front results from the free play of free intellects, working on subjects of their own choice, in the manner dictated by their curiosity for exploration of the unknown.
Through example after example Sarewitz demonstrates that the progress of the late 20th century was virtually never, “free play of free intellects, working on subjects of their own choice”, but instead was almost always science being managed, being driven on specific topics for specific applications.   Scientific knowledge advances most rapidly, and is of most value to society, when it is steered to solve problems — especially those related to technological innovation.  Could it be that the War on Cancer has floundered because there's nobody in charge; nobody driving toward a goal and asking specific people specific questions? 

From how the DOD pushed the development of the transistor, while putting up with poor results at first, to how the “War on Cancer” turned into individual scientists looking for findings that would give them a good paper and a good reputation rather than saving lives, it's a fascinating story.  I have to admit that my initial reaction was that he was calling for more government oversight of science, bigger government, but as I read the article that slipped away, and what I think I see is a call for science to be more applied and not just, well, playing at whatever they find interesting: “the free play of free intellects, working on subjects of their own choice”. 

A topic he didn't address was the politicization of science and how that drives bad science.  In John Ioannidis' landmark 2005 paper he said one of the important factors associated with results being false is politicization: the hotter the research field, the more likely the results are to be false.  Which explains Climate Science in one sentence.

One thing is for sure: as taxpayers, we're paying for a lot of bad science.  For that reason alone, we should care.
Yeah, it's a good half hour read with no distractions in the room and it's worth every minute of it.   

10 comments:

  1. I have reached the conclusion that "science" is every bit as corrupted as every other institution in this country. Whether it is driven by money or their supposed reputation can not be know but they are no more honest than the religions that they heckle. Another example....theprinciplemovie.com/wp-content/uploads/2016/08/Albert-Einstein-The-Earth-Mover.pdf

    ReplyDelete
  2. I strongly recommend reading Richard Feynman's The Meaning of It All: Thoughts of a Citizen-Scientist.

    He was an amazing man. One short passage: "No government has the right to decide on the truth of scientific principles, nor to prescribe in any way the character of the questions investigated. Neither may a government determine the aesthetic values of artistic creations, nor limit the forms of literary or artistic expression. Nor should it pronounce on the validity of economic, historic, religious or philosophical doctrines. Instead it has a duty to its citizens to maintain their freedom, to let those citizens contribute to the further adventure and the development of the human race.”

    Contrast that to the current Climate Change crowd that wants to criminalize those that question their "settled science."

    ReplyDelete
    Replies
    1. I have to confess to being one of the fanboys that idolized Feynman in the 80s. I read several (a half dozen?) of the books that were out about him, have a copy of the 3-volume collection Feynman Lectures in Physics around here, and more. There was at least one Nova hour interview with him that I recall watching, I think it was when he was suffering from his final bout with cancer. In the intervening years, I've read that some of those stories from the '80s were crafted to improve his image, but there's so much right there that even if you discount it 20%, it's still great.

      A perfect example is the quote is "I'd rather have questions that can't be answered than answers that can't be questioned". Sums up the warmists perfectly.

      Delete
    2. I am a fanboy too. I actually know someone that had him as a professor. Loved "Surley You're Joking". Now I am going to drag it out and read it again.

      Delete
    3. I was privileged to hear him speak when I worked at Fermilab.

      The lab's director at the time was R.R. Wilson, a colleague of Feynman's, and the man who convinced him to join the Manhattan Project.

      Charming, and a hilarious speaker. Didn't get to meet him, as the crowd around him was like 10 people deep!

      Delete
    4. Terry - true story: back in (whatever year it was) my wife and I both bought each other a copy of "Surely You're Joking" for Christmas, obviously without knowing what the other was doing.

      Jim - that's a close as I got to meeting (well, at lest shaking hands with) the Apollo 11 crew at the 25th anniversary celebration. What you'd really like is the chance to get to know them a bit better, but that takes weeks or months, and that ain't happening.

      Delete
  3. Ioannidis, and others, fail to mention a key reason that poor science is so easily arranged these days: the woefully inadequate theoretical underpinnings of the statistics deployed in all modern sciences, soft and hard (including physics).

    A p-value never did, and cannot, mean what everybody thinks it means; any "confidence interval" that can actually be calculated, has no formal mathematical meaning whatever; "hypothesis testing" is devoid of intellectual content, and needs to be abandoned forthwith and forever; nothing at all 'has' a probability; a decision is not, and never can be, a probability; and so forth.

    'Modern' statistics is a kludge, at best; it never worked as a theoretical system; only the application of wishful thinking connects its formalisms to anything real; it aids and abets bad, fraudulent, unreproducible research. It, provably, always leads to more confidence in research results than is warranted; and now, both better ways to think about the role that statistics plays in science and decisions, and better statistical methods, more distant from hand-waving and closer to what we might like to know, now exist. The entire 'thing' called 'modern' statistics, needs to be junked.

    Working scientists, engineers, systems analysts, need to re-think what they never really thought about before, but only just assumed, and start over. Their entire way of thinking about statistical analysis stands on sand. The entire thing will crumble for you, once a few grains start to fall out of place. The place to start is here:

    William Briggs, Uncertainty: The Soul of Modeling, Probability & Statistics
    https://www.amazon.com/Uncertainty-Soul-Modeling-Probability-Statistics/dp/3319397559/

    ReplyDelete
    Replies
    1. http://wmbriggs.com/post/19623/ says But one thing is clear, since (as we learned Sunday), our minds (our intellects) are not material, then any theory of evolution involving physical forces /necessarily/ fails to explains the rational nature of human beings.

      Briggs believes human thought is not a computational process which operates using neurons within a human brain? Where does he believe human thinking occurs? On a big virtual server on the cloud in "Heaven", run by the "Lord of Hosts"?

      Statistics is for guessing efficiently about the causes of observed effects. Is Briggs seriously claiming human thought doesn't occur within the standard model of physics?

      Delete
  4. I think some of this problem rests with that old "Publish or Perish" mindset. I had also actually read about several of the problems with some of these so called peer reviewed papers turning out to be faulty, especially the stem cell line. I think that there was a very complete article on it in the Discover magazine, a magazine that funny enough, I find to lean in the direction of basically putting scientists up on a pedestal. They tend to look at anyone who would question their so called settled science as heretics, whether global warming or frogs in the Amazon rain forest. And heaven forbid that an ordinary citizen with intelligence and the ability to think should differ from their ideas. If they have a comment published in the magazine, it is always in the light of a backwoods hillbilly with no education and a predilection to dragging their hands on the ground.

    ReplyDelete