Following yesterday’s first look at emails from the 2 year effort to get NOAA to release emails from FOIA requests, where we learn that some scientists felt ‘Hit on the head with a hockey stick’ and that “The paleodata always got a lot more attention from the general public than it deserved.”.
There’s a great article today at SPPI today Michael Mann — the ghost of climate past which summarizes the whole hockey stick affair quite well. It draws heavily on a post from May of this year on PJ Media by Rand Simberg titled The Death of the Hockey Stick?
In that post I found this little nugget below from December 2009, one that I apparently missed in the furor immediately following Climategate1. I’m now correcting that oversight.
What is it? It’s the hockey stick recreated in Excel, using proxy data and instrumental data freely available on the web, something you can easily do at home. I figure the more people that explore this themselves using the easy to follow steps, the more people will understand what a statistical abuse the hockey stick is. The source of this tutorial is as surprising to me as it may be to you, and there’s no special software or secret Mannian Excel plugins needed to do the work yourself. I’ll let Rand Simberg explain.
From his May 2012 essay:
Ultimately, in addition to Mann’s claim for the dramatic recent uptick (which we are supposed to presume was a result of the late industrial revolution and equally dramatic increase in carbon dioxide into the atmosphere as a result of the liberation of carbon from burning long-buried fossil fuels), Keith Briffa of the Climate Research Unit (CRU) at the University of East Anglia in England controversially declared, based on Eurasian data, that the well-documented Medieval Warm Period (MWP), from around 950 to 1250 CE — the European Middle Ages — didn’t actually exist.
This claim was important, if not essential, to Mann’s thesis, because his initial formulation only went back to 1400, the beginning of the so-called Little Ice Age. Critics of the theory thus argued immediately upon its presentation that it shouldn’t be surprising that the earth was warming now, given that we are still coming out of it, and that the medieval warming in the absence of late Carolingian SUVs and coal plants argued that the climate naturally cycled, with no need to invoke Demon Carbon. That is to say, to the degree that the hockey stick has a blade in the twentieth century, it would have another a millennium ago.
The theory has continued to take blows over the years since it was first presented. About a decade ago, a paper was published by Willie Soon and Sallie Baliunis claiming that there was good evidence that both the (still extant) MWP and current warming were driven by solar activity rather than carbon emissions. But these initial attacks were beaten back by the climate mafia (as we now know from the leaked emails between Mann and his partners in crime in East Anglia from two and a half years ago). The real damage came when a retired Canadian mining engineer, Steve McIntyre, and a professor at the University of Guelph, Ross McKitrick, started digging into Mann’s methodology, and found flaws in both his statistical analysis and data interpretation, and published a paper describing them in Geophysical Research Letters in 2005. They showed that Mann’s methodology would generate a hockey stick almost independently of the data input, by feeding it spectral noise. Later, Internet satirist (and apparent statistician by day) Iowahawk provided a primer on how to create a hockey stick at home, using a standard spreadsheet program.
Yep, Iowahawk.
I liked this part of Iowahawk’s “how to” primer the best:
Is there anything wrong with this methodology? Not in principle. In fact there’s a lot to recommend it. There’s a strong reason to believe that high resolution proxy variables like tree rings and ice core o-18 are related to temperature. At the very least it’s a more mathematically rigorous approach than the earlier methods for climate reconstruction, which is probably why the hockey stick / AGW conclusion received a lot of endorsements from academic High Society (including the American Statistical Association).
The devil, as they say is in the details. In each of the steps there is some leeway for, shall we say, intervention. The early criticisms of Mann et al.’s analyses were confined to relatively minor points about the presence of autocorrelated errors, linear specification, etc. But a funny thing happened on the way to Copenhagen: a couple of Canadian researchers, McIntyre and McKitrick, found that when they ran simulations of “red noise” random principal components data into Mann’s reconstruction model, 99% of the time it produced the same hockey stick pattern. They attributed this to Mann’s method / time frame for selecting of principal components.
To illustrate the nature of that debate through the spreadsheet, try some of the following tests:
- Run step 3 through step 7, but only use the proxy data up through 1960 instead of 1980.
- Run step 5 through step 7, but only include the first 2 principal components in the regression.
- Run step 3 through step 7, but delete the ice core data from the proxy set.
- Run step 2 through step 7, but pick out a different proxy data set from NOAA.
Or combinations thereof. What you’ll find is that contrary to Mann’s assertion that the hockey stick is “robust,” you’ll find that the reconstructions tend to be sensitive to the data selection. M&M found, for example, that temperature reconstructions for the 1400s were higher or lower than today, depending on whether bristlecone pine tree rings were included in the proxies.
What the leaked emails reveal, among other things, is some of that bit of principal component sausage making. But more disturbing, they reveal that the actual data going into the reconstruction model — the instrumental temperature data and the proxy variables themselves — were rife for manipulation. In the laughable euphemism of Philip Jones, “value added homogenized data.” The data I provided here was the real, value added global temperature and proxy data, because Phil told me so. Trust me!
I urge readers to replicate it yourselves. Knowledge is power, especially first hand knowledge. Here’s all you need to do it. I’ll be happy to publish what you learn.