Archive

Ineffective Theory

Epistemic Orders of Magnitude

This famous video is a good demonstration of just wide a range of physical sizes objects in our universe span. The same sort of extreme range exists in other contexts: time scales, energies, magnetic field strengths, and so on. These are less amenable to nice videos. Nevertheless, in all these cases, remembering the wide range of available scales is critical for good intuition. You and the Sun are both physical objects made of atoms, but the Sun has \(\sim 10^{29}\) times your volume. For most purposes, you and the Sun are not “the same sort of thing”.

You have a lot of intuition for how to deal with things that are larger than you are. For instance, maybe you have a friend who goes to the gym twice a day and weighs 50% more than you do. Huge! And yet, any intuition you’ve gained from interacting with him is doomed to be completely unrelated to the properties of the Sun.

Sometimes you drive a car. This is an object that’s larger and heavier than you by an order of magnitude or two. Intuition gained from driving a car generalizes reasonably well to many other types of heavy machinery, and even has some relevance to airplanes. But it’s completely unrelated to the Sun.

And so on. It might be true for you to say “my house is large”. It’s also true to say “the Sun is large”. But except in the narrow technical sense of “larger than you are, by any amount”, the words “large” simply don’t have the same meaning in those two statements.

This is all pretty obvious and silly, so let’s move on to thinking about probabilities. Here are some things I believe about tomorrow:

  1. The sun will rise.
  2. The U.S. government will not experience a successful coup d’etat.
  3. It will not snow.

These are all predictions. They are all things I believe with probability greater than 50%. But there the similarities end. Except in the narrow technical sense of being “claims best understood in the framework of probability”, these three items are not remotely the same sort of claim, and it’s a mistake to treat them on the same footing.

Without thinking too much about fundamentals (like the laws of physics), let’s get some crude estimates of probabilities. The earth has existed for a few billions of years, and the Sun has risen every day of every year. Just from this, the probability that the Sun does not rise tomorrow can’t reasonably be said to be larger than \(10^{-12}\). Tighter bounds are available of course, but the point is already clear.

Now for claim 2. What’s the probability that the United States government collapses tomorrow? Well, it’s low, but it’s not \(10^{-12}\) low. Again, a first estimate comes from noting the the government is a few hundred years old, so \(10^{5}\) days. That suggests that the per-day probability of a coup can’t be much larger than \(10^{-5}\). Trying to lower this number is actually quite difficult: looking at other countries will tend to raise it, and if you want to argue about political fundamentals, well, the probability that your model is wrong is much larger than \(10^{-5}\). So let’s stick with that.

Will it snow tomorrow? It’ll be just around freezing for most of the day. The weather says there’s a 1% chance of precipitation. Of course, the probability that the forecast is substantially wrong is greater than 1% (as I know from experience), but in any case it’s of the order of \(10^{-2}\).

If I say “I think it won’t snow tomorrow” and you reply “Yes it will, I can totally smell it in the air” — you’re probably wrong, because I checked the weather and the weather forecast is more reliable than smelling the air. If I say “the government will still be here tomorrow” and get the reply “no, I saw on 4chan that there’s a plot…”, well, you’re wrong, and please can we make a bet? If I say “the Sun will rise tomorrow” and you reply with “nah, it looked a weird color this morning, so it won’t rise tomorrow” — you’re not wrong, you’re insane.

Also note from the three examples above that very small probabilities don’t really exist. Certainly, as you know, 0% is not a real probability (in the real world). But obtaining finite probabilities like \(10^{-10}\) is already an astonishing feat. You may have a lovely argument for why the probability should be that low, but I still need to assign a probability to your argument being wrong, and that probability is almost never going to go below 1%. Pretty much the only way to convincingly give probabilities lower than that is to have a massive pile of data showing that no, really, we tested \(10^{10}\) times, and this thing has just never happened. No modeling uncertainty involved.

This brings me to science, and the price of data. All fields of science make probabilistic claims. Often the “uncertainty” aspect is played down: “We found the Higgs boson!” is better press than “Yesterday we had a 4.98 sigma effect and today it’s 5.01”. Nevertheless, no claim is ever certain, and the uncertainty is always there.

But again, there the similarities end. The famed five-sigma threshold in physics corresponds to a probability of \(\sim 10^{-7}\) that a bump in a histogram is merely a statistical fluke rather than a real physical effect. The fact that proton-proton collisions are cheap allowed there to be a large pile of data backing up the discovery of a \(125\,\mathrm{GeV}\) Higgs, with sophisticated arguments and modeling kept to a minimum. The pile of data is now much larger, of course. Today, it’s more likely that there is a malicious conspiracy of particle physicists covering up the truth than that the Higgs data is the result of random noise.

As you move to fields without abundant cheap data, it becomes impossible to create such low probabilities. For instance, the standard model of cosmology involves a period of rapid expansion known as inflation. What is the probability that this is wrong: that there was never any cosmological inflation? Unclear, but I’d be shocked to hear even the most adamant partisan give a probability under 1% (and more than happy to take related bets). Or we can leave physics entirely: does raising the minimum wage cause unemployment? Confident assertions are a dime a dozen, but any claim of a probability lower than 10% should be looked on with extreme scepticism.

Superficially, the various sciences have a lot in common. The practitioners all have letters after their names, quantitative models are available, models are tested by being compared to some pile of data, and models that persistently fail such tests eventually get thrown out. But that doesn’t mean that intuition about one field carries over to others. It’s important to remember that claims made in particle physics are not the same sort of thing as claims made in, say, cosmology. You can be confident that your flight tomorrow won’t result in a fatal accident (a quick search suggests the probability is \(\sim 10^{-7}\)), and casually dismiss anyone who worries otherwise. The same confidence in most scientific results would be badly misplaced.