Berkeley Earth Super Duper Exposed

Over the past week I have spent a lot of my spare time examining the Berkeley Earth temperature data to a greater depth than ever before. What I have found is shocking. It is unbelievable just how bad the dataset they are using truly is. There is no science here. This is fraud at worst, gross incompetence at best. Perhaps the best way to explain what I found is to explain how I went about examining and organizing their data.

The first thing I realized was Excel was just not powerful enough to handle the complexity and size of the task of data analysis. So I imported the information into an Access database. Access is capable of handling far more records than Excel. It is also far more powerful.

The very first thing I did was to examine how complete the records were. I want to look at annual averages. So I ran a query to order all the records by station and by year. Then I looked at how many readings each station had for each year. What I found were thousands of incomplete years. Some had only one month. Obviously, you can't compute an annual average for a year when there are not 12 months recorded. Considering how the temperatures vary in one year, a missing month can skew the average by quite a bit. It is certainly an inaccurate record which can not be used.

I also found dozens of records with more than 12 readings in one year. In fact I found as many as 99 monthly averages recorded for one station in one year. These are obviously duplicate records.

After I extracted from the original dataset only those years with complete records and eliminated all the duplicate records the number of stations dropped from over 4600 to 3127. In other words some 32% of the stations were eliminated because they consistent of incomplete or duplicate data. That is a really high casualty rate.

The final step was to create the program to generate the information I wanted to extract from the set of complete annual data. Based upon a start date and an end date, the program extracts every station with a complete record for each year in the date range, and then reports the annual average temperature of all stations for each year. Secondarily, the program also provides a count of stations.

Below is a screen grab of the program output for 1975 through 1980.

 

So, I now have a database tool which can almost instantly generate a record of temperatures from stations continuously reporting between any two years between 1880 and 2004. This is where my next graph comes in.

Do you see the problem here? Out of 3127 stations in the record only 2 contain a complete record from 1880 to 2004. Only 5 were continuously reporting from 1950 to 2004. That includes the original 2 by the way. There were only 44 stations reporting from 1980 to 2004. There were 380 stations reporting from 2000 to 2004. Yet, in 2004 there were 805 stations reporting.

So it appears the only usable data in the entire 120 MB's of original data is that from just two stations. The rest of it is too fragmentary, incomplete, or just does not cover enough time to be useful. Just two stations, one in Russia and one in Switzerland.

This is all they have. Unbelievable.

Just two more graphs and we will call it a day. I think these are pretty self explanatory.

Below is a graph showing the high and low annual averages for each year from 1900 to 2004. You will notice only the lowest reading vary, and they vary hugely. You are looking at temperatures in the -55° C  (-67° F) range. That would be Antarctica. You are seeing the effects of 12 stations running from 1953 to 1994 for periods ranging from 42 years to 1 year. Do you think having 2, 3 or 12 annual averages at such an extreme might have some noticeable affect on the "global average"? This is an extreme example of how ridiculous this entire business truly is.

Here is the record of stations reporting by year from 1900 through 2004. Enough said, don't you think?

Dog Daze

Here we are, the 22nd of July, and I am in Georgia. This is a miserable time of the year in my home state. Hot. Humid. You don't get that evening temperature drop after the sun goes down. We calls 'em dog days. You would suspect that has something to do with dogs, because dogs don't sweat. Dogs pant. A lot. Historically speaking, it is tough to be a big hairy dog in Georgia around this time of year. In days of old, dogs would be known to invade the root cellar or hide up under a cool front porch or in the crawl space under a house. Some place shady with open dirt, because those places tend to be cooler than most. Dogs also like to roll in mud and swim around in the kind of ponds typically found in cow pastures.

Dogs who have been rolling in mud or swimming in water subject to cow pie contamination are generally speaking not the ideal bed buddy. The last thing you want rummaging around the bed with you on a hot July night is a big, hairy, hot, reeking of wet dog, dog breathing gallons of hot doggie breath on you for extra good measure.

All things considered, dog days is a pretty apt description of this time of year.

However the origins of the term are not here in the wilting summers of the south and the sufferings of hairy dogs. The term comes to us down through the ages from the ancient Greeks. In late July the star Sirius becomes visible, appearing to rise at dawn, just before the sun. Sirius is called the dog star, because it is part of the constellation Canis Major, which the Greeks imagined to be a dog chasing a rabbit. Sirius is the dogs nose. The time of the year where Sirius rises were therefore called dog days.

Of course the Romans conquered Greece, then conquered pretty much everything all the way to Scotland. Later on Shakespeare wrote a bunch of plays featuring Romans and so forth. Mark Twain wrote the Adventures of Huckleberry Finn, which featured disreputable thespians doing Othello don't you know. Somewhere in an amongst all that hoopla and flap doodle the ancient Greeks and that entire astronomy part just kinda dropped off. As far as we are concerned nowadays, Homer is Bart Simpson's Dad. It is what it is.

Having read both Huckleberry Finn and Tom Sawyer, having once watched the movie Romeo and Juliet as part of an English Lit class in high school, and being a true native of the south I am somewhat of an expert on these matters. Plus I have been to the planetarium at the Fernbank Science Center. Twice.

You can trust me on this one.

2016 Was the hottest year evah! Or was it?

According to Berkeley Earth and a bunch of other people, 2016 was the hottest year ever. Again. It seems like every year the ministers of information on all things climate related seem to make the same statement. Today I am going to deconstruct this claim focusing on Berkeley Earth specifically.

The picture below compares my graph of the Berkeley Earth data with their graph. Since my graph goes farther back, I have marked three common points. From this you can see, other than a minor Y axis scaling difference and a different ending date, the two graphs are identical. I have made no alterations to the data of any kind. At the end of this piece I will provide precise information so any who wish to do so can verify my work.

 

 

Now, let's begin deconstructing the data from which both charts above were made.

 

When organizations such as Berkeley Earth presents a graph of data to the public they are asking you to believe their data. The proof, as they say, is in the data. Yet, they are in fact lying to you by omission. They are omitting important details concerning not only where their data comes from but when. The illusion they are creating is you are looking at a continuous, consistent record of temperature over a very long span of time. That is absolutely not the case. From 1950 to 2004 a total of 2789 temperature reporting stations were opened while an additional 2349 were shuttered. That is a very important detail.

In order to properly assess data trends over a long period of time it is absolutely essential to establish consistent, stable measuring points without bias over the entire length of the time period being studied. If you move or change a measuring point or if you add or subtract measuring points you introduce additional variability into the data not created by the system being studied.

Take for example measuring the annual fluctuations of the water level in a large reservoir. To do that you would establish one or more stable locations from which to measure. Such as driving a pylon down into the lake bed until a firm under structure is reached and attaching a measuring device to the stable platform. What you would not do is drive around in a boat every year taking a number of depth readings at random locations. That would give you a lot of very useless data.

My conclusion, which is well warranted, is the complete Berkeley Data set is worthless for trend analysis and forecasting. As I stated above, in order to be useful in that manner a data set must consist of continuous, stable measuring points. The Berkeley data set, taken as a whole, is anything but that. There is, however, data with in the complete set which is valuable.

In 1951 there were 576 stations which came on line. Of those stations 211 maintained continuous, reasonably consistent measurements for 50 years. The graph below is the result of those 50 years of measurements from those stations.

This is a drastically different result from that of the complete data set. Instead of a pronounced warming trend as in the complete data set, we see a slight cooling trend of less than .25° C. Essentially this is no change as the year to year variability is many times greater than the total projected change.

The conclusion from this study is no evidence of any warming. None.

You can replicate my data for this time period or for any time period within the data set. As long as you follow the guidelines for a proper time based study of temperature trends, namely you can only use data from stations reporting for the entire time period being studied, you will get similar results. No warming, slight warming, or slight cooling. What you will not find is any warming trend even remotely resembling the chart put forth by Berkeley Earth. That is fact.

The methodology I have followed using their data is the correct one. The results I have derived are therefore valid. The methodology Berkeley Earth uses is, as I discussed above, very much incorrect and therefore their results are not valid.

Update: I follow a number of climate blogs and engage in conversation with a number of people. One of those blogs is the Deplorable Climate Science Blog by Steve Goddard. A reader on there passed this youtube clip on to me. This clip shows how the number and locations of climate stations have changed over time. During the period where the "experts" say warming has increased dangerously what you see is exactly what I stated. Large numbers of stations in colder climates disappear while stations in hotter parts of the world appear.

https://www.youtube.com/watch?v=58mDaK9bH5o&feature=player_embedded&lang=en&app=desktop


Additional information concerning data and source

My data source. I used the raw, unadjusted data from land based measurements: http://berkeleyearth.org/data/

Below is the break down of the 211 stations used for my study based upon the long term annual average temperature for each station.

Below are the stations numbers I used for my study.

Of Olly, Sassie, Mister Kitty and Life with Cats

We recently lost two of our kitties in the space of one week. Olly and Sassy. Sassy was kind of an unknown quantity to me, she lived in the basement with the kids. She never really had the opportunity to get used to me. I am the new guy around here. Going on two years, come September, of being married and I am still the new guy in some ways. Not so for Olly. To Olly I had become part of the furniture. He was Momma's boy, and always would have been, but he had gotten to love me pretty good. He learned I was not a threat, which was a good thing. He discovered I am actually pretty gentle and I give good scratchies, which could be tolerated in small doses. But then he discovered something Sharon had already discovered. I am one of those people that goes to bed hot but wakes up cold. Sharon is exactly opposite me, she goes to bed cold and wakes up hot.

I have always been that way. Even in the coldest cold of winter time, I have to stick my feet and legs out of the covers. I usually pull the covers off my chest. So I end up with the covers basically over my waist. I seem to just radiate heat, and I have to let that escape. I will wake up wrapped up like bug in a rug, because I eventually cool off sometime in the night. I have no idea when that happens, because I am asleep, you silly ninny.

Well Olly was a skinny minnie and a chilly willie. On cold days he would crawl up under the covers and hang out all day. We would go into the bedroom and see this bump under the covers. That would be Olly. One day Olly decided to lay down on me and that's when he discovered my bed furnace. He would always end up sleeping on Sharon's legs or burrowed under the covers next to her, but he spent a fair amount of time laying in my lap or up against my legs. So I became more than something to be tolerated or ignored. We became buds.

Olly was a good kitty. You had to read him, only give him what he was willing to take, and take only what he was willing to give. Follow the rules and he was truly a loving kitty. The fact it had to be on his terms didn't diminish what it was. When I would get up in the middle of the night with my own particular brand of insomnia and he would leave Mom to come curl up in my lap it was special. It was our moment.

Thinking about young mister Olly caused me to think about other great cats I have known. One of my top cats was this old, battle scarred guy we called Mister Kitty. Mister Kitty was bad. I really can't say this nicely and do it justice, so you'll just have to get over it. Mister Kitty was the king cat of the neighborhood for a long time. If it was a male, he pretty much kicked its ass. Bad. He was just an average size kitty, but he fought like a holy terror. He got cornered in the back yard one afternoon by two red chows. I was raking leaves in the front yard at the time and saw them chase him around the corner of the house. Before I could even make it to that corner I heard first one then the other yelp out in horrible pain. Next thing I know they were running on home, whimpering, with their tails between their legs. When I came to the back door of the house there was Mister Kitty, waiting to be let in, looking at me as if to say "What? I wasn't doing nuthin!" Like I said, he was bad.

Needless to say, he was a tom. Let me put it to you this way. A lady at church who lived a couple of miles away was talking one Sunday morning about this horrible orange  cat that was terrorizing her house because she has a female cat in heat. My mom asked what it looked like. She said "He's a horrible, beat up orange tabby. He just looks evil! He's missing this big patch on his right cheek, looks like something just ripped part of his face off!" That was Mister Kitty. That put his range of amorous nocturnal activities out to a radius of at least two miles.

Lovin' and fightin', that was Mister Kitty. My Dad, who as a general rule didn't care for cats much, liked Mister Kitty. I think he respected him. When Mister Kitty would come back from the wars to rest up, he would go comatose on the sofa. Dad would cover him up with a small blanket and say, "leave him alone, he's tard".

The funny thing about Mister Kitty is that was not his original name. My brother Paul brought him home from somewhere. He was like a wild cat. He wouldn't let anyone near him. Paul had named him Orfink. We thought that was, well, just a stupid name. Orfink, as he was named then, wound up in the basement and he wasn't having none of you. You couldn't get near him. Dad told Paul in no uncertain terms the cat had to go, he was not going to have his basement turned into a litter box. We tried too. Whatever our efforts were, he was not impressed.

It was Mom who tamed the wild kitty. This is what she told us she did. Everyday she would bring him food and water and just talk to him sweetly and kindly. She would say things like "I'm not going to hurt you mister kitty, I have some food" and the like. At first, she would put the food down and leave. Later, she would put it down and wait him out, talking gently and sweetly the while. Eventually she wouldn't put it down until he came out. That was how he was tamed and how he got his real name.

After that, Mister Kitty basically became my cat. He would go into the basement at night right under my bed and call me to come let him in during the middle of the night. He slept in my bed, in my window, or in my sock drawer. When I came home, he would come running. He was my buddy. He never really trusted Paul. I think Paul traumatized him bringing him home.

All that changed when Mom went to bed when her cancer came back. She had beaten it before, and would beat it again this time, but somehow Mister Kitty knew. He left my bed and bedroom and moved in with Mom. She told me later one morning after chemo, when she was so sick and weak she couldn't move, she woke up with Mister Kitty in the bed grooming her and licking her. At first she thought "what is that horrible cat doing!!!" but then she realized it was really soothing. In fact, it made her feel better. Mister Kitty had a big purr box, and though battle scarred and all he was soft and warm. He stayed with her, sleeping in the bed with her, until she got better.

Mister Kitty, to put it bluntly, was a bad ass battle cat and a caterwauling lascivious tom. Yet, somehow, he knew and he remembered. That sweet lady who loved him out of his hidden wild cat phase needed him and he was there for her. I have no idea how he knew but I don't question it. Mister Kitty ended up leaving us as he lived. Fightin' and lovin'. But during that time he put those pursuits aside and loved on my Mom in her need as best he could with everything he had.