Friday, March 12, 2021

The Difference Between Statistical Significance and If Something Matters

The CDC released a report this past Monday that showed statistical significance in wearing face masks vs. not and the rates of Covid 19 infection growth and the death rate.  I think it's a perfect example of the kind of association or correlation studies we get smacked with all the time.  A change that can be shown to be statistically significant may mean absolutely nothing in real life (IRL). 

Let's start with the high level conclusion, the one they use to sell the results. 
Mandating masks was associated with a decrease in daily COVID-19 case and death growth rates within 20 days of implementation. Allowing on-premises restaurant dining was associated with an increase in daily COVID-19 case growth rates 41–100 days after implementation and an increase in daily death growth rates 61–100 days after implementation.

What are the implications for public health practice?
Mask mandates and restricting any on-premises dining at restaurants can help limit community transmission of COVID-19 and reduce case and death growth rates. These findings can inform public policies to reduce community spread of COVID-19.
Wow!  That means masks matter!  To heck with the years of previous studies from the same CDC, as we talked about last November that said 14 Randomized Controlled Studies did not show a substantial effect from masks or hand washing on Influenza transmission (and presumably other respiratory viruses).  This is new and improved science! 

So let's jump ahead to the math because that's the important part. Bold and underline added. 
During March 1–December 31, 2020, state-issued mask mandates applied in 2,313 (73.6%) of the 3,142 U.S. counties. Mask mandates were associated with a 0.5 percentage point decrease (p = 0.02) in daily COVID-19 case growth rates 1–20 days after implementation and decreases of 1.1, 1.5, 1.7, and 1.8 percentage points 21–40, 41–60, 61–80, and 81–100 days, respectively, after implementation (p<0.01 for all) (Table 1) (Figure). Mask mandates were associated with a 0.7 percentage point decrease (p = 0.03) in daily COVID-19 death growth rates 1–20 days after implementation and decreases of 1.0, 1.4, 1.6, and 1.9 percentage points 21–40, 41–60, 61–80, and 81–100 days, respectively, after implementation (p<0.01 for all).  Daily case and death growth rates before implementation of mask mandates were not statistically different from the reference period.

During the study period, states allowed restaurants to reopen for on-premises dining in 3,076 (97.9%) U.S. counties. Changes in daily COVID-19 case and death growth rates were not statistically significant 1–20 and 21–40 days after restrictions were lifted. Allowing on-premises dining at restaurants was associated with 0.9 (p = 0.02), 1.2 (p<0.01), and 1.1 (p = 0.04) percentage point increases in the case growth rate 41–60, 61–80, and 81–100 days, respectively, after restrictions were lifted (Table 2) (Figure). Allowing on-premises dining at restaurants was associated with 2.2 and 3.0 percentage point increases in the death growth rate 61–80 and 81–100 days, respectively, after restrictions were lifted (p<0.01 for both). Daily death growth rates before restrictions were lifted were not statistically different from those during the reference period, whereas significant differences in daily case growth rates were observed 41–60 days before restrictions were lifted.
Note the underlined last statement in the first paragraph.  Daily case growth and daily death growth rates were not statistically different than the reference period, 1–20 days before implementation of the mask mandate. 

Allow me to put the numbers from the first paragraph in a format I find more understandable: Mask mandates were associated with 0.5, 1.1, 1.5, 1.7, and 1.8 percent decreases in daily COVID-19 case growth rates at 1–20 days, 21–40, 41–60, 61–80, and 81–100 days after implementation.     Ordinarily, in this sort of study, when they present numbers like the first paragraph, those are relative numbers.  We don't know the daily growth rate from this study, but for illustration, let's assume it was growing at 10% per day.  That decrease of 0.5% at 1-20 days is not 10% - 0.5% or 9.5%.  It's 10-(0.5% of 10) or 9.95% growth rate.  You would need a lot of samples to detect that small a difference and they did: they had 2,313 datasets for the first data set and 3,076 for the second.

In plain English, if your chances of getting the infection was 10% without a mask, wearing the mask improved it to 9.95%.  Take the worst daily case growth rate increase of 1.8%.  Converted from relative risk to absolute risk, that's 9.82%.   Just for fun, let's bump the odds of getting infected from 10% to 20%.  Those two numbers turn to 19.9% and 19.64%.

A very reasonable question is whether a relative risk of 0.5 or 1.8% really matters IRL.  Again, that's 1.018 chance vs a 1.000 chance. The relative risk of things we absolutely know are bad, like smoking, aren't 2% worse, they're 2 times worse or more. Relative risks like this could be from anything and not the thing they think they're measuring.  For example, I find no mention of the type of mask; whether an actual N95 mask, a surgical mask, or a cloth gaiter.  Are all masks the same?  I doubt it.  Read my article with the table of spurious correlations that make up much of the junk science we're subjected to daily. 

Their figure from the study. 

The exact number on what your chances are of getting infected are hard to find.  There are tons of factors that must be accounted for: how much virus you're exposed to, the health of the tissues in your respiratory tract, and a broad swath of other metabolic health questions.  A reasonable question might be, "do masks matter?"  Based on this I would have to say yes they do.  They just don't matter much at all. 


  1. Masks may matter, but horsesh*t "studies" like this don't.

    When you study something, you have to control for other variables.
    This happy horsesh*t controls for nothing, then issues results within the margin of error, with a straight face.

    This is like looking at speeding laws and claiming deaths up or down have to do with those laws, or not, when if fact, one has no effing idea what, if any, effect new laws have.

    "We passed Law X.
    Deaths went down 1%.
    Therefore, the laws are what worked.

    Um, NO.
    Did people comply?
    At what rate, compared to before you passed the law.
    Did you enforce the law?
    Is it possible that deaths decreased not because of the law, but because of any number of other variables, like less driving, safer roads, better traffic management, higher auto insurance premiums for accidents, safer cars, more seatbelt usage, or even just blind luck, and that the law in question had actually no effect at all, because you never examined the other variables, and didn't control for them at all???

    That's the level of anti-scientific b.s. being attempted with that asstarded "study", which doesn't control for any one of dozens of other variables, all of which could account completely for what deaths did, without mask laws even getting to the plate at all.

    This is textbook Junk Science, and was obviously such the minute it was mentioned.

    I literally was shooting holes in it, sight unseen, the minute I heard it on the radio, without even looking, and lo and behold, it's as bad or wrose than I thought, because it would have to be.

    They literally just backed up a cow, had it defecate into a sandbag, and labelled it a scientific study, and shipped the sandbag to the media, unadulterated.

    Everyone whose name is on doing this or releasing it should literally be taken out behind a barn, and shot in the head on the spot for crimes against humanity, and gross malignant sociopathy. and then have their bodies fed to hogs, and the entire affair filmed live and released to all media outlets.

    Pour encourager les autres, and because they have it coming.

    1. And BTW, those 14 studies?
      They were studying whether masks never intended to protect the wearer worked to protect the wearer.
      Not one of them studied whether me wearing a snot mask protects other people, which is precisely the point at issue.
      Over half the studies weren't even actual studies, just a group of people looking at other studies, and doing a poll of polls to decide if anything mattered.

      Which is like polling scientists, many of whom weren't even studying the question you're concerned about, to determine at what temperature water freezes, as opposed to simply getting a glass of water and a thermometer, and actually studying the exact question under consideration, which latter effort would be actual science.

      IOW, if the CDC's lips are moving, they're lying.
      Full stop.

  2. Annals of the Royal College of Surgeons of England, 01 Nov 1981

    A study from 1981 stopped using masks in an operating room, and got no increase in infections. Study concluded that masks don't do anything.

    I say: if your glasses are fogging, that means the mask isn't working going out and therefore it's not likely working coming in. I say: the minimum mask to seal against your face is a rubber cartridge type for protecting against solvents, with several straps and a tight fit. Denninger has pointed out for months that the pattern for covid transmission does not match aerosol, it matches face -> hand -> surface -> hand -> face, just like other colds and flus; handwashing works, and masks don't work. I say: the way the public uses masks is counterproductive, because they touch their face more than they would without the mask.

    1. Yes, I've read that study. It's really surprising, but big science discoveries can be.

      Anyone who has worked with hazardous substances and worn actual protective gear, the kind with seals as you describe, realizes the casual masks are useless. The idiots are the ones who think they can get everyone in the country to wear quality masks, with NO training, and the masks will be effective. How about starting here: no man can have a beard because no adequate mask will seal properly on top of a beard.

      As the Anon 1248P comment (can't tell if you're the same anonymous) says, look at how the professionals dress for Ebola. Look at Level IV biosafety containment suits.

      About a year ago, I said, "Eventually, everyone gets it and gets over it (or not, whether the fatality rate is 1% or 3% or 0.2% is still hard to know); the essence is that it's still self-limiting in the long run." I've read that people still test positive for antibodies to the 1917 Spanish Flu, meaning we were exposed to it at some point. Once they emerge, they remain in the ecosystem. Some viruses burn a hole in the population so effectively that a ring can be vaccinated around a hot spot to cut off its spread. Smallpox is one. Ebola might well be another. With something like this, at some point we all get immunity to it somehow, either by getting it, getting vaccinated, or by cross immunity to a similar virus.

    2. 1) The masks were never intended to protect the wearer.
      They are designed to protect everyone else from the wearer, by keeping the virus-laden snot particles on your end, rather than flying up to 30' away when you sneeze, like they do unencumbered.
      If this is news to anyone, kindly hit yourself in the head with a baseball bat as hard as you can, until that truth penetrates.

      2) The masks were never intended to be a total occlusive hermetically sealed device.
      The term for that is called "Level A Encapsulating Suit".
      You could look it up.
      If you thought the idea was to totally seal you inside the mask, see the cranial tune-up suggested in #1, above.

    3. (cont.)
      3) As to the referenced study, in order:
      a) It's 40 years old. So how many studies since then have confirmed that result? My money is on 0, but I'm open on the question.
      b) How many hospitals, including the one in the study, have gone to mask-free because of this finding?
      c) The hospital was recording a very high infection rate only 4 years prior to the survey, and recording a notable decrease every year annually. They were thus a sloppy hospital, and were doing some pretty drastic infection control measures to get their numbers down over 3% in as many years. Including, in all probability, blasting patients with multiple broad-spectrum antibiotics, which approach will kill almost everything, even if you do the surgery in a compost heap. No mention is made of those measures, but statistical common sense dictates that the experiment could have actually increased exposure, and something totally outside the scope of the experiment counter-acted it, beyond the reader's knowledge. This is the essence of junk science: attributing correlation as causation, when you haven't controlled for any other one to one hundred variables.
      d) This is all fine for changing surgical practice, if it's ever validated a few dozen times, but the purpose of public masking is to decrease the dissemination of virus-laden particles from coughs and sneezes amongst people not a foot away, not in a sterile room, and not getting blasted by IV antibiotics for a week after their exposure.
      d) The above leaves me safely calling the study's findings wholly irrelevant to the question of public masking.

    4. (cont.)
      4) This isn't hard. Put out petri dishes at 6'. Have infectious individuals cough and sneeze at them from that distance.
      Culture them.
      Then repeat that experiment, but this time with them wearing surgical masks, and plain cotton snot masks.
      Call us when the infection rate of the petri dishes for those covered is the same or higher than that of those where no masks are worn.
      I could do that study for under $1K, tomorrow, all in, out the door, including tax, but I can tell you what the results would be for absolutely no chareg whatsoever, without bothering to conduct it:
      Masks reduce infections.

      Change my mind, by posting an actually relevant study, that does what I've just described.

      You'd think, after a year, someone would do that, and the fact that no one has leads me to believe no one wants to do it because the results will make them look like a jackass, and they'd be out $1000.

      Just a hunch there.
      But until someone does it, and gets a result that shows no difference, they're simply speaking from ignorance, whether circumstantial, or deliberate.

      Masking in public isn't the whole answer, but unless someone can show it doesn't work, on average, it should continue as part of the solution, along with frequent hand washing/decon, and 6' distancing.

      BTW SiG: The current fatality rates in the U.S. run from 0.5% in AK (with an 8.2% infection rate among the entire state population) to 2.9% in NJ (with an infection rate of 9.3% of the population, coupled with them putting COVID+ patients in the most vulnerable sub-population, as we've heard, which is why NYFS is #2 worst, at 2.8% CFR), as I posted yesterday. The rough nationwide CFR is a shade under 2%, give or take a fraction. With a viral infection penetration rate of less than 10%, almost everywhere. +/-2% of 33M is the current body count, so I wouldn't advocate going for 100% penetration to anyone not wishing for 5M dead, but not everyone is that prudent, and/or more ghoulish, with other people's lives.
      When we've vaccinated more widely, it's an entirely different ballgame, IMHO, and common-sense pushback from people who are non-idiots is why CDC had to walk back their idiotic restrictions on people who've already been vaccinated, within a week of burping them out.

      I repeat, if their lips are moving, the CDC is lying.

    5. 3) As to the referenced study, in order:
      a) It's 40 years old. So how many studies since then have confirmed that result?

      The same number of studies which pointed out the Population Bomb, Peak Oil, Global Warming, Social Security's solvency, and the Federal Reserve's inflation target are all hoaxes. For the same reasons. Oh and the medical malpractice legal industry agreeing that sometimes bad things happen, and not every bad outcome is the result of misbehavior by the health care worker.

      I don't believe masks were ever sold as containing coughs and sneezes, they were sold as containing ordinary exhalation. That objection is moving the goalposts.

      Masking in public isn't the whole answer, but unless someone can show it doesn't work, on average

      It doesn't work, on average. There is too much data now from adjacent US states which had different mask and lockdown policies but same-shaped count curves.

    6. Arguing policies as if it = actual behavior is more junk science, on the same level as assuming another gun law will reduce gun crime.

      You: Dunce cap. Corner stool. Assembly required.

  3. You've see how Doctors Without Borders dress for ebola? Whole body covered, tape closing the seams at wrists and ankles, and someone to undress you so the dirty outside doesn't touch you? That's what it takes to keep aerosol stuff off you. Just imagine your clothing sprayed or smeared head to toe with sticky wet black permanent ink, dirty grease, or copier toner. Then take it off without getting any on you. This could be a homeschool science project, fun for the whole family. College students could do it in mixed groups and end up starting families.

    Meanwhile, you see the mask zombies on the sidewalk touching their face every five minutes.

    1. Unlike Ebola, Kung Flu isn't a disease with a CFR of between 60-95%.
      So that protocol would be silly beyond discussion.
      But you're right, a substantial number of people are stupid about properly following masking guidelines for the PPE they're actually using.

      Now, see if we can guess why even with mask laws, we still have infections.
      (If one takes out "mask" and "infection", and substitutes the words "gun" and violent crimes", respectively, they'll be well on the way to coming up with the answer.)

  4. What we have going now is theatre. Where I worked (now retired) with aircraft composites we had to wear a full Tyvek suit, a separate hood and two battery belt powered hepa filters. Before taking it off you had to go through a vacuum cleaning process and all vacuum cleaners had hepa filters. If you had an itchy nose then tough luck. With glasses you wore an elastic band to keep them on tight. What they have done with Covid is make it a health EMERGENCY which keeps them from having to provide injection safety, effectiveness safety, and testing quality. Covid is a russia, russia, russia extension after the failed ukraine, ukraine, ukraine failure. It made for a fantastic election failure.

  5. Fake science is dangerous, you might think a little about what you post.

    Real PPE works. It works even better when people are trained to use it. It's not difficult.

    I'm at a complete loss to understand why the USA isn't producing a billion or more N95 masks a month and giving them to everybody; and teaching everybody how to use them.

    1. All our masks are produced in China.
      Since you asked.

      The first thing ChiComia did a year ago was seize 3M's N95 mask plant there, and take 100% of all production until they got all the masks they wanted, indefinitely.

      An effort was made to start up production here, but those jobs will be shipped back to China 5 seconds after this passes, and even N95s aren't the whole solution.

  6. As the Anon 1248P comment (can't tell if you're the same anonymous) says

    I'm sure the Central Scrutinizer can see they both came from the same IP, but I'm semi-surprised you don't get that data.

    The idiots are the ones *who think they can get everyone in the country to wear quality masks, with NO training, and the masks will be effective.*

    These people aren't idiots, they're rulers doing "kingsmanship". They're doing communist politics/secular religion, and they're very successful at it. Most people are obeying them, and that's all that matters. Middle class people, who have brains which work nominally, find it very difficult to imagine the mindset of a sociopath or psychopath. Thus the Big Lie technique works. Have you gotten your National Socialist Gates' Worker's Party infrared arm tattoo yet?