This post was going to be a more detailed explanation, but it rapidly became too complicated. So let’s make it short and simple. Probability is about what we know and what we don’t. It is a measure of information and uncertainty.
There’s too much philosophy floating about. We will start with Newton’s Flaming Laser Sword. We want a probability that is practical, easy to understand. and logically consistent.
Chance: The Surprising Source Of Information
An impossible event has a probability of zero. Nothing is less certain than that. Conversely, a probability of one is guaranteed to happen, and nothing can be more certain. Everything in between these limits measures our uncertainty.
The surprise is that information theory, which we use to analyse computer logic, is based on probability. In other words, the information which runs our world is a measure of the gambler’s odds. Moreover, even the scientific method has similar foundations.
As Claude Shannon of Bell Labs stated:
- An event with 100% probability is unsurprising and gives us no information.
- The less probable an event, the more surprising it is, and the more information it provides.
- The non-occurrence of a zero probability event is also unsurprising and tells us nothing.
- If two independent things occur, the information is the sum of the individual events.
Science is based on experiment, a fancy term for trial and error. At its most fundamental quantum level, even physics is based on probability. We expect to find randomness and uncertainty in higher-end disciplines such as biology and psychology. But it is at the core of the hard sciences and our understanding of logic.
Most things we do or understand have an associated level of uncertainty. Correspondingly there is no scientific proof, just ideas that haven’t been superseded yet. Next time you hear medical doctors demanding “proof”, recognise that proof is just plain unscientific. They are instructing you to cease your thought crimes and conform. It is a rehash of the kid in the schoolyard’s demand for you to “prove it”.
Uncertainty
Looking at chance in terms of information helps us to understand more clearly what it means. It reminds us to follow the sources of information rather than just thinking about counting and samples.
Science is the way we understand the world. All science is uncertain. Often what we don’t know is more important than what we know. Remember, what we know can be wrong. As the great physicist Michael Faraday put it, “There’s nothing quite as frightening as someone who knows they are right.”
Think Betting Odds
The way to think about such problems is as a professional gambler. Forget about the so-called significant p-value of 0.05 and other confusing concepts. Instead, consider a 1 in 20 chance—similarly, a highly significant p-value of 0.01 means odds of 1 in 100. Break out and take away the complexity.
Your horse has odds of 100 to 1. If you bet a dollar and the horse wins, the bookie gives you 100 dollars. This means that the bookie thinks your chance of winning is less than 100 to 1, and he will make a profit. Notice the bookie does not care which horse wins or its real odds. He just wants a good chance of getting your money.
You don’t have to be particularly clever to deal with odds and whole numbers. That’s one reason why bookies use them. They want the largest number of gamblers, so make it as easy as possible. Conversely, evidence-based medicine uses statistics like the p-value for obfuscation.