Today’s post is about the issue of model shopping, a dubious and potentially evil process, with the global mean temperature as an illustrative example. A mathematical model is one or more equations or algorithms that summarize some set of data. A recent Occupy Math post, Mathematics, Statistics, and Despair, discussed mathematical models and showed how to think about the issue of someone trying to pull the mathematical wool over your eyes. Today’s post highlights a similar problem, but with a different color of mathematical wool.
A frequent problem with research in the social sciences is model shopping. A researcher or student, having collected data, will run it through a large variety of different statistical analyses (Occupy Math has seen instruction manuals for graduate students on how to do this) until they find one that says their results are significant. The problem with this is that the usual standard for significance is that the statistical model says this result could happen by accident only one time in twenty. If you check 43 models, then the chance that a completely random result will appear significant, to at least one test, is 89%. The practice of going out and shopping around for a model until you find one that “works” means many published results in the social sciences are, mincing no words, without evidentiary support.
The way to avoid accidentally publishing results that are crap is to pick a single, appropriate statistical model before you start your research.
A more down-to-earth example occurred in Occupy Math’s room at Battenfeld Scholarship Hall at the University of Kansas. In Dungeons and Dragons a character starts off with statistics created by six rolls of three six-sided dice. One of Occupy Math’s roommates was idly rolling three dice. After about ten minutes he rolled three sixes – yielding the best possible result, a statistic of 18! He looked up and asked “can I roll up a character starting with this roll?” His argument was that he had legitimately rolled an 18; but he also had stumbled upon a technique for always getting an 18. The effect of rolling until you get an 18 and then starting a character is to change the chances of getting an 18 from “1 in 216” to “completely certain”. This is a way of cooking the numbers that seems almost honest until you think it all the way through.
The technique described above to roll up a great fantasy gaming warrior (strength is rolled first) is an example of cherry picking data.
The graph shown above is from data gathered by satellite about the temperature of the earth’s lower atmosphere. This data was gathered by the award winning meteorologist Roy Spencer. Alert readers will notice that there is a very large spike in the data in 1998. If you look at the arguments of the confused, misguided people that think climate change is a conspiracy to scam funding you will find the phrase “climate change since 1998” fairly often. What this comes down to is:
Picking 1998 as a starting year can make it look like the globe is cooling off. A perfect example of data manipulation.
If you read the web pages, or listen to, people that deny human-caused climate change (there are several flavors), you will find that they are model shoppers to the hilt. An honest argument from data agrees on the data set, uses the whole data set, and tries to find clear, simple descriptions of the effects.
A crazy argument picks bits and pieces based on their ability to support the desired conclusion. To be very clear, starting the model in 1998 is a way of cheating. It is dishonest. Anyone concerned with truth or fact would not do this. To be completely fair, Occupy Math suspects that the majority of the deniers have been led astray by an actual conspiracy and do not understand that they are lying in a fashion that potentially endangers millions of human lives.
Occupy Math is certain that evidence does not follow from conclusions.
The fact that the world has been much warmer in the past is taken by climate deniers as evidence, for example, that the current change is both natural and nothing to worry about. An example is the very warm period called the Paleocene-Eocene Thermal Maximum. This warming was much larger than any of the current predicted outcomes of modern climate change. This warming corresponded with large increases in the amount of carbon dioxide in the atmosphere. This leads to two conclusions: a person trying to understand what is happening deduces that increasing carbon dioxide can raise global temperature. A crazy person deduces that, because it happened via natural processes in the past, it’s not a problem in the present. The fact that there were no large coastal cities at the end of the Paleocene seems to completely escape them. The lack of a complex agricultural supply chain in the Paleocene appears to be beyond their comprehension.
The change in the world wide human population from zero to over six billion since the end of the Paleocene is apparently irrelevant to climate deniers.
What we do about climate change is one of Occupy Math’s hobby horses. Many people support the view that we need to get the level of carbon dioxide back to what it was in the 1950s, problem solved. The problem is that, lack of political will aside, there is no path to this goal that does not involve three critical problems:
- Massive flooding of coastal cities.
- Around a billion climate refugees. Occupy Math thinks the official estimates are low, because they do not factor in the effect of wars that will be triggered by the migrations.
- Big problems with crop failures and starvation.
If we look at the absolute pig’s breakfast made of finding refuge for a measly few million Syrian refugees, imagine several hundred million people that are migrating because they lack fresh water and food, or because their homes are now under water. This is an official estimate, not Occupy Math’s more depressing guess.
Growing enough food for everybody is no longer a technological challenge. It is a huge political challenge. We now have the technology to make cool, efficient grow lights that emit the frequencies plants need for photosynthesis. These lights are a new type of light emitting diode (LED) with emission spectra tuned by quantum dots. This opens the door to skyscraper-shaped farms that don’t set themselves on fire with the waste heat from their grow-lights. This technical coup doesn’t matter if everyone insists they have a right to grow their traditional crops, even when yields collapse. We may also need to eat crickets.
Coastal cities are probably the least of the three big problems. The Dutch are practically rubbing their hands at the prospect of consulting on how to live below sea level with pretty much every country with a coastline. The Dutch are also taking a responsible line about helping the poor survive all this. We are going to have to learn to build redundant zero-failure sea walls. The good news is that these can include tidal power generation and water parks. Possibly seafood farms. Cheap crab instead of crickets! Seaweed salad for all!
We can solve the problems caused by climate change unless we refuse to confront them. Guess whose fault that would be?
Occupy Math is sure people have a right to their opinion. Climate change denial is an opinion that can help kill millions of people, something Occupy Math thinks should be discouraged. The fact that denial of climate change is strongly supported by model shopping and cherry-picking of data strongly suggests that the deniers do not have a leg to stand on. It also goes a long way to explaining Occupy Math’s motto: Math is the Right of all Free People. Do you have interesting examples of “facts” that are completely insane and the methods used to support them? Comment or tweet and Occupy Math will look into it.
I hope to see you here again,
University of Guelph,
Department of Mathematics and Statistics