I’ve been criticizing epidemiologists–including the ones publishing in JAMA, the Lancet, NEJM, etc., and the famous ones who were making apocalyptic predictions on TV last month–for doing what is clearly bad work. My main complaint is, again, that their estimates about the danger of the virus are based on the wrong data (current infections) collected the wrong way (non-random testing of people who present themselves as sick). We all know better than that. You don’t sample on the dependent variable. You don’t sample in ways that suffer from severe selection bias. If you mostly test people who show up saying they are sick, and 3.4% of them die, it doesn’t tell you how many people have the infection, nor does it tell you what percent of people who have the infection will die.

Now, while many economists and others trained in stats have been saying the same thing, it’s surprising how many untrained people say we should instead defer to epidemiologists. You can see some of their arguments on Facebook and others in the comments to previous posts.

1. “Epidemiology is a science and you aren’t trained in it.”

Problem: There are basic methods with statistics that are invariant across all disciplines that use them. For instance, you must avoid selection bias. They are violating these basic methods.

2. “You can’t criticize their model unless you have a better one”

This criticism confuses the critique of their model with the critique of their data. Yes, I think the models we are seeing are poor, because they don’t handle endogeneity or variance well. But my main criticism is that they are using the wrong data collected the wrong way. Good model + bad data => bad science.

As a parody of 1 and 2 together, consider this dialogue:

Physicist: 2+2=5, plus uncontrolled experiment, therefore my theory of physics.

Non-physicist trained in even more math than the physicist: No, 2+2≠5. And your experiment is bad because you lack proper controls. Your paper is bad.

Physicist: Hey, you aren’t a physicist! And you don’t have a better model of physics, do you?

The physicist’s response here is, frankly, stupid. And you, the reader, know better.

You don’t need to have a better model to show someone else’s model sucks. Think of how most seminar papers in the social sciences go. Someone comes in with a model about how, say, the roll out of trains affected agricultural prices. The economists in the audience can pick it apart even if they don’t have a better model. Others can find errors or problems in data collection even if they aren’t trained in the field.

Further, I ask you to go and read the papers on COVID, especially the early ones which were used to justify current policy. These aren’t highly sophisticated papers using the most current difference-in-differences analyses to estimate effects. They are almost entirely crude statistics with nothing else. Most of them just say: We tested 1000 people who came to the hospital saying they had breathing problems, and 35 of them died. Here is the breakdown of death by age and co-morbidities. That kind of info might well be useful for certain purposes, but it would be blatantly and laughably incompetent to use it to estimate what percent of the public will die if they catch the disease.

If this is representative of the field, the field of epidemiology (which is as much a social science as a natural science) is something like 45 years behind econ and 30 years behind poli sci.

Stop defending bad science, people.

3. “Hey, the WHO never said 3.4% of people would die. They were well aware of the limits of their data.”

Lots of gaslighting going on here.

Yes, if you go digging back through the testimony, you see a small number of people saying, “Oh, sure, 3.4% of people diagnosed with COVID died, but we have little to no idea what the infection fatality rate is because of our poor testing procedures. We’ve been testing for the purpose of helping the sick rather than for getting data to estimate the dangers.” But what you mostly see with the Imperial College people, Fauci, and others, is making mass projections based on the crude case fatality rates calculated from bad data.

I can understand shutting down everything temporarily in an abundance of caution. But states are immediately obligated to collect the right kind of data the right way, so we can get a proper estimate of the real dangers and make decisions competently. They haven’t done so. The past month has seen government failure on a mass scale.

4. “You’re just insisting on good data because you’re a libertarian who doesn’t like it when governments push people around in the name of the common good!”

Is that supposed to be a criticism or a compliment?