Saturday, July 22, 2017

Berkeley Earth Super Duper Exposed

Over the past week I have spent a lot of my spare time examining the Berkeley Earth temperature data to a greater depth than ever before. What I have found is shocking. It is unbelievable just how bad the dataset they are using truly is. There is no science here. This is fraud at worst, gross incompetence at best. Perhaps the best way to explain what I found is to explain how I went about examining and organizing their data.

The first thing I realized was Excel was just not powerful enough to handle the complexity and size of the task of data analysis. So I imported the information into an Access database. Access is capable of handling far more records than Excel. It is also far more powerful.

The very first thing I did was to examine how complete the records were. I want to look at annual averages. So I ran a query to order all the records by station and by year. Then I looked at how many readings each station had for each year. What I found were thousands of incomplete years. Some had only one month. Obviously, you can't compute an annual average for a year when there are not 12 months recorded. Considering how the temperatures vary in one year, a missing month can skew the average by quite a bit. It is certainly an inaccurate record which can not be used.

I also found dozens of records with more than 12 readings in one year. In fact I found as many as 99 monthly averages recorded for one station in one year. These are obviously duplicate records.

After I extracted from the original dataset only those years with complete records and eliminated all the duplicate records the number of stations dropped from over 4600 to 3127. In other words some 32% of the stations were eliminated because they consistent of incomplete or duplicate data. That is a really high casualty rate.

The final step was to create the program to generate the information I wanted to extract from the set of complete annual data. Based upon a start date and an end date, the program extracts every station with a complete record for each year in the date range, and then reports the annual average temperature of all stations for each year. Secondarily, the program also provides a count of stations.

Below is a screen grab of the program output for 1975 through 1980.
 

So, I now have a database tool which can almost instantly generate a record of temperatures from stations continuously reporting between any two years between 1880 and 2004. This is where my next graph comes in.


Do you see the problem here? Out of 3127 stations in the record only 2 contain a complete record from 1880 to 2004. Only 5 were continuously reporting from 1950 to 2004. That includes the original 2 by the way. There were only 44 stations reporting from 1980 to 2004. There were 380 stations reporting from 2000 to 2004. Yet, in 2004 there were 805 stations reporting.

So it appears the only usable data in the entire 120 MB's of original data is that from just two stations. The rest of it is too fragmentary, incomplete, or just does not cover enough time to be useful. Just two stations, one in Russia and one in Switzerland.

This is all they have. Unbelievable.

Just two more graphs and we will call it a day. I think these are pretty self explanatory.

Below is a graph showing the high and low annual averages for each year from 1900 to 2004. You will notice only the lowest reading vary, and they vary hugely. You are looking at temperatures in the -55° C  (-67° F) range. That would be Antarctica. You are seeing the effects of 12 stations running from 1953 to 1994 for periods ranging from 42 years to 1 year. Do you think having 2, 3 or 12 annual averages at such an extreme might have some noticeable affect on the "global average"? This is an extreme example of how ridiculous this entire business truly is.



Here is the record of stations reporting by year from 1900 through 2004. Enough said, don't you think?




Dog Daze

Here we are, the 22nd of July, and I am in Georgia. This is a miserable time of the year in my home state. Hot. Humid. You don't get that evening temperature drop after the sun goes down. We calls 'em dog days. You would suspect that has something to do with dogs, because dogs don't sweat. Dogs pant. A lot. Historically speaking, it is tough to be a big hairy dog in Georgia around this time of year. In days of old, dogs would be known to invade the root cellar or hide up under a cool front porch or in the crawl space under a house. Some place shady with open dirt, because those places tend to be cooler than most. Dogs also like to roll in mud and swim around in the kind of ponds typically found in cow pastures.

Dogs who have been rolling in mud or swimming in water subject to cow pie contamination are generally speaking not the ideal bed buddy. The last thing you want rummaging around the bed with you on a hot July night is a big, hairy, hot, reeking of wet dog, dog breathing gallons of hot doggie breath on you for extra good measure.

All things considered, dog days is a pretty apt description of this time of year.

However the origins of the term are not here in the wilting summers of the south and the sufferings of hairy dogs. The term comes to us down through the ages from the ancient Greeks. In late July the star Sirius becomes visible, appearing to rise at dawn, just before the sun. Sirius is called the dog star, because it is part of the constellation Canis Major, which the Greeks imagined to be a dog chasing a rabbit. Sirius is the dogs nose. The time of the year where Sirius rises were therefore called dog days.

Of course the Romans conquered Greece, then conquered pretty much everything all the way to Scotland. Later on Shakespeare wrote a bunch of plays featuring Romans and so forth. Mark Twain wrote the Adventures of Huckleberry Finn, which featured disreputable thespians doing Othello don't you know. Somewhere in an amongst all that hoopla and flap doodle the ancient Greeks and that entire astronomy part just kinda dropped off. As far as we are concerned nowadays, Homer is Bart Simpson's Dad. It is what it is.

Having read both Huckleberry Finn and Tom Sawyer, having once watched the movie Romeo and Juliet as part of an English Lit class in high school, and being a true native of the south I am somewhat of an expert on these matters. Plus I have been to the planetarium at the Fernbank Science Center. Twice.

You can trust me on this one.

Saturday, July 15, 2017

2016 Was the hottest year evah! Or was it?

According to Berkeley Earth and a bunch of other people, 2016 was the hottest year ever. Again. It seems like every year the ministers of information on all things climate related seem to make the same statement. Today I am going to deconstruct this claim focusing on Berkeley Earth specifically.

The picture below compares my graph of the Berkeley Earth data with their graph. Since my graph goes farther back, I have marked three common points. From this you can see, other than a minor Y axis scaling difference and a different ending date, the two graphs are identical. I have made no alterations to the data of any kind. At the end of this piece I will provide precise information so any who wish to do so can verify my work.

 
 
Now, let's begin deconstructing the data from which both charts above were made.
 
When organizations such as Berkeley Earth presents a graph of data to the public they are asking you to believe their data. The proof, as they say, is in the data. Yet, they are in fact lying to you by omission. They are omitting important details concerning not only where their data comes from but when. The illusion they are creating is you are looking at a continuous, consistent record of temperature over a very long span of time. That is absolutely not the case. From 1950 to 2004 a total of 2789 temperature reporting stations were opened while an additional 2349 were shuttered. That is a very important detail.


In order to properly assess data trends over a long period of time it is absolutely essential to establish consistent, stable measuring points without bias over the entire length of the time period being studied. If you move or change a measuring point or if you add or subtract measuring points you introduce additional variability into the data not created by the system being studied.

Take for example measuring the annual fluctuations of the water level in a large reservoir. To do that you would establish one or more stable locations from which to measure. Such as driving a pylon down into the lake bed until a firm under structure is reached and attaching a measuring device to the stable platform. What you would not do is drive around in a boat every year taking a number of depth readings at random locations. That would give you a lot of very useless data.

My conclusion, which is well warranted, is the complete Berkeley Data set is worthless for trend analysis and forecasting. As I stated above, in order to be useful in that manner a data set must consist of continuous, stable measuring points. The Berkeley data set, taken as a whole, is anything but that. There is, however, data with in the complete set which is valuable.

In 1951 there were 576 stations which came on line. Of those stations 211 maintained continuous, reasonably consistent measurements for 50 years. The graph below is the result of those 50 years of measurements from those stations.


This is a drastically different result from that of the complete data set. Instead of a pronounced warming trend as in the complete data set, we see a slight cooling trend of less than .25° C. Essentially this is no change as the year to year variability is many times greater than the total projected change.

The conclusion from this study is no evidence of any warming. None.

You can replicate my data for this time period or for any time period within the data set. As long as you follow the guidelines for a proper time based study of temperature trends, namely you can only use data from stations reporting for the entire time period being studied, you will get similar results. No warming, slight warming, or slight cooling. What you will not find is any warming trend even remotely resembling the chart put forth by Berkeley Earth. That is fact.

The methodology I have followed using their data is the correct one. The results I have derived are therefore valid. The methodology Berkeley Earth uses is, as I discussed above, very much incorrect and therefore their results are not valid.

Update: I follow a number of climate blogs and engage in conversation with a number of people. One of those blogs is the Deplorable Climate Science Blog by Steve Goddard. A reader on there passed this youtube clip on to me. This clip shows how the number and locations of climate stations have changed over time. During the period where the "experts" say warming has increased dangerously what you see is exactly what I stated. Large numbers of stations in colder climates disappear while stations in hotter parts of the world appear.

https://www.youtube.com/watch?v=58mDaK9bH5o&feature=player_embedded&lang=en&app=desktop


Additional information concerning data and source

My data source. I used the raw, unadjusted data from land based measurements: http://berkeleyearth.org/data/

Below is the break down of the 211 stations used for my study based upon the long term annual average temperature for each station.

Below are the stations numbers I used for my study.



Sunday, July 2, 2017

Of Olly, Sassie, Mister Kitty and Life with Cats



We recently lost two of our kitties in the space of one week. Olly and Sassy. Sassy was kind of an unknown quantity to me, she lived in the basement with the kids. She never really had the opportunity to get used to me. I am the new guy around here. Going on two years, come September, of being married and I am still the new guy in some ways. Not so for Olly. To Olly I had become part of the furniture. He was Momma's boy, and always would have been, but he had gotten to love me pretty good. He learned I was not a threat, which was a good thing. He discovered I am actually pretty gentle and I give good scratchies, which could be tolerated in small doses. But then he discovered something Sharon had already discovered. I am one of those people that goes to bed hot but wakes up cold. Sharon is exactly opposite me, she goes to bed cold and wakes up hot.

I have always been that way. Even in the coldest cold of winter time, I have to stick my feet and legs out of the covers. I usually pull the covers off my chest. So I end up with the covers basically over my waist. I seem to just radiate heat, and I have to let that escape. I will wake up wrapped up like bug in a rug, because I eventually cool off sometime in the night. I have no idea when that happens, because I am asleep, you silly ninny.

Well Olly was a skinny minnie and a chilly willie. On cold days he would crawl up under the covers and hang out all day. We would go into the bedroom and see this bump under the covers. That would be Olly. One day Olly decided to lay down on me and that's when he discovered my bed furnace. He would always end up sleeping on Sharon's legs or burrowed under the covers next to her, but he spent a fair amount of time laying in my lap or up against my legs. So I became more than something to be tolerated or ignored. We became buds.

Olly was a good kitty. You had to read him, only give him what he was willing to take, and take only what he was willing to give. Follow the rules and he was truly a loving kitty. The fact it had to be on his terms didn't diminish what it was. When I would get up in the middle of the night with my own particular brand of insomnia and he would leave Mom to come curl up in my lap it was special. It was our moment.

Thinking about young mister Olly caused me to think about other great cats I have known. One of my top cats was this old, battle scarred guy we called Mister Kitty. Mister Kitty was bad. I really can't say this nicely and do it justice, so you'll just have to get over it. Mister Kitty was the king cat of the neighborhood for a long time. If it was a male, he pretty much kicked its ass. Bad. He was just an average size kitty, but he fought like a holy terror. He got cornered in the back yard one afternoon by two red chows. I was raking leaves in the front yard at the time and saw them chase him around the corner of the house. Before I could even make it to that corner I heard first one then the other yelp out in horrible pain. Next thing I know they were running on home, whimpering, with their tails between their legs. When I came to the back door of the house there was Mister Kitty, waiting to be let in, looking at me as if to say "What? I wasn't doing nuthin!" Like I said, he was bad.

Needless to say, he was a tom. Let me put it to you this way. A lady at church who lived a couple of miles away was talking one Sunday morning about this horrible orange  cat that was terrorizing her house because she has a female cat in heat. My mom asked what it looked like. She said "He's a horrible, beat up orange tabby. He just looks evil! He's missing this big patch on his right cheek, looks like something just ripped part of his face off!" That was Mister Kitty. That put his range of amorous nocturnal activities out to a radius of at least two miles.

Lovin' and fightin', that was Mister Kitty. My Dad, who as a general rule didn't care for cats much, liked Mister Kitty. I think he respected him. When Mister Kitty would come back from the wars to rest up, he would go comatose on the sofa. Dad would cover him up with a small blanket and say, "leave him alone, he's tard".

The funny thing about Mister Kitty is that was not his original name. My brother Paul brought him home from somewhere. He was like a wild cat. He wouldn't let anyone near him. Paul had named him Orfink. We thought that was, well, just a stupid name. Orfink, as he was named then, wound up in the basement and he wasn't having none of you. You couldn't get near him. Dad told Paul in no uncertain terms the cat had to go, he was not going to have his basement turned into a litter box. We tried too. Whatever our efforts were, he was not impressed.

It was Mom who tamed the wild kitty. This is what she told us she did. Everyday she would bring him food and water and just talk to him sweetly and kindly. She would say things like "I'm not going to hurt you mister kitty, I have some food" and the like. At first, she would put the food down and leave. Later, she would put it down and wait him out, talking gently and sweetly the while. Eventually she wouldn't put it down until he came out. That was how he was tamed and how he got his real name.

After that, Mister Kitty basically became my cat. He would go into the basement at night right under my bed and call me to come let him in during the middle of the night. He slept in my bed, in my window, or in my sock drawer. When I came home, he would come running. He was my buddy. He never really trusted Paul. I think Paul traumatized him bringing him home.

All that changed when Mom went to bed when her cancer came back. She had beaten it before, and would beat it again this time, but somehow Mister Kitty knew. He left my bed and bedroom and moved in with Mom. She told me later one morning after chemo, when she was so sick and weak she couldn't move, she woke up with Mister Kitty in the bed grooming her and licking her. At first she thought "what is that horrible cat doing!!!" but then she realized it was really soothing. In fact, it made her feel better. Mister Kitty had a big purr box, and though battle scarred and all he was soft and warm. He stayed with her, sleeping in the bed with her, until she got better.

Mister Kitty, to put it bluntly, was a bad ass battle cat and a caterwauling lascivious tom. Yet, somehow, he knew and he remembered. That sweet lady who loved him out of his hidden wild cat phase needed him and he was there for her. I have no idea how he knew but I don't question it. Mister Kitty ended up leaving us as he lived. Fightin' and lovin'. But during that time he put those pursuits aside and loved on my Mom in her need as best he could with everything he had.






Wednesday, June 21, 2017

West Point Way

Just hanging in West Point at my latest contract job. It seem like it has been raining continuously down here for about three weeks. It's hard being away from home. Harder when the weather is awful.

NO FISHING!!!!! OMG!!!!!

I want to see how this video upload works so let's, shall we?

video

Friday, June 16, 2017

Great Moments in Church Aviation: The one perfect paper airplane

I was born into a family of Baptists and raised as a Baptist for all of my formative years. Whether this was always true or not from the time I was old enough to make the distinction I was raised a Southern Baptist. Not that I really understand the distinction now, but at least I know we are a separate group. Southern Baptists are not really as stodgy an fuddy duddy as you may have heard; they can be a fun loving bunch. They do have their limits. On a fine spring Sunday morning back in 1974 or maybe 1973 a fellow church going Southern Baptist and I pushed those limits.

It those distant days people observed a certain solemnity as concerns Sunday morning worship services. For one thing us men folk wore very uncomfortable clothes and shoes. Probably because being uncomfortable and having fun tend to be mutually exclusive things. Young folks were expected to act with decorum. Wearing slick bottom shoes can ensure a certain measure of decorum when traversing waxed and polished floors. Every Sunday the ushers of the church would hand out these stiff, singly folded, color printed pieces of paper known as the Sunday bulletin. That was their first mistake. Parents often allowed the young men to spend Sunday worship service in the balcony. This was their second mistake.

You see, church bulletins are stiff and thick. They are in fact the perfect material for the making of paper airplanes.

In those days I often sat on the first row of the balcony with a friend who would eventually graduate from high school with me named Blake. Blake and I created a number of airplanes. We would place our creations on the edge of the wall demarcating the end of the balcony. Often, when the congregation would bow their heads and close their eyes in solemn prayer these paper construction would creep over that edge as if by accident.

Now, for the most part, paper airplanes made from church bulletins tend to fly pretty good, but the widely recognized method of folding the paper makes an airplane that tends to corkscrew. Most of our creations either nose dived or just turned and flew back under the balcony where few people ever sat. Southern Baptists tend to see relative pulpit position as a badge of faith. Such churches tend to fill up from the front, which seems to be the exact opposite of normal human behavior but what can you do? So, while we felt gloriously naughty and bold our creations in reality were mostly duds.

Until one fine Sunday morning I threw accepted airplane construction technique out the window and created the one, perfect Church bulletin airplane. I won't divulge my design secrets here. Because....

THERE CAN BE ONLY ONE!

My creation found itself in the position of many other such creations. Perched upon that edge marking a separation between the balcony and all that empty air above the heads of some fifteen hundred solemnly serious Southern Baptists, waiting for a moment of prayer. That moment came when all the ushers moved forward to the front of the church for the offering. We call that passing the plate. With the ushers lined up in front of the pulpit, the preacher enjoined the congregation to bow their heads and offer up a prayer. As soon as every head was bowed and every eye closed, my creation leaped into that great empty expanse.

What happened next very nearly cause Blake and I to rupture an internal organ in a supreme act of laughter suppression. My airplane seemed to soar with a will of its own. It never deviated from its course, turning neither left nor right. Instead it soared straight ahead in a perfectly level, gently descending flight. Unseen save for Blake, Me, and God it slowly floated all the way down over all those bowed heads and pegged the pulpit stand dead center with an audible thunk. From there it landed smack dab in the middle of an offering plate.

Seconds later the prayer came to amen, and the ushers came forward. One of them grabbed the plate with the airplane, seemed to pause in confusion, then promptly stuffed it in his pocket.

By this point me and Blake were fairly hurting. It was all we could do. I remember having my head between my knees trying to hold it in.

Shortly after the offering had been given and received and the service got back into full swing we were still in the grip of laughter when suddenly we found our ears gripped between the thumb and forefinger of one of the ushers, a man named Durham Phagan. Which I must say is one heck of a name. I will never forget what he said to us.

"I have been mistaken before, but this time I don't think I am. I do believe there will be no more paper airplanes falling off the balcony from this point on."

Well, all I can say is he was not mistaken. Our career in Southern Baptist Aviation had come to an end. But what a glorious end. It was the one, perfect, church bulletin paper airplane. I bet even God had to smile over that.

Thursday, June 8, 2017

The Flaw at the Heart of the Global Warming Myth

It is Thursday 4:14 in the afternoon, and we are coming to you live from the Lanett Alabama Econo Lodge. It has been a long week but things are rapidly coming to a close. Today I propose to show you what I believe will be the last thing you will ever need to see about the myth of global warming. Yes, I said myth. Over the past few posts I have shown you how the proof CO2 is driving temperatures up is weak. I have shown you how the number of monitoring stations active in the world has changed. We have examined station data from Australia, to Wake Island, to Scotland and seen no evidence of catastrophic warming of any kind. If you are scratching your head and wondering why none of this adds up I am going to show you. The answer is in the data and in the text files that go with it.
 
 
What does this tell you? It tells you simply they are not reporting the actual temperatures, they are reporting how much the temperatures vary from the 1951 to 1980 average. In fact, according to the text, they reported each month as a deviation from the monthly 1951 to 1980 average. That is how they created a homogenized model of the global average through time.
 
 
 
That is why the plot of their data from the year 1980 shows no seasons. No summer, no winter. Just variations from an average.
 
 
 
This is exactly where they got it wrong. Their model is badly flawed. Remember how I said the number of active stations in the world has grown exponentially during the time their charts show so much rise in temperature? That is the heart of the flaw. In order for their method to work the annual global average can not change because of new stations coming on line or old ones going off line. Meaning if you drop off a station in Antarctica and add 10 stations in the Bahamas you have just caused the global average to go up. It doesn't matter if you move all the data up or down by adding or subtracting 50°, the difference between the land of frost and the land of sunshine remains.
 
The correct way to homogenize or smooth a large collection of records with varying local temperatures, different starting and ending dates, with no way to determine the true, accurate local temperature back in time, and no way to determine the referential accuracy between instrumentation is to homogenize each individual station record to it's own zero reference point. This way you are asking a simple question. How much temperature rise or fall did this particular location see over that amount of time. Then, when you look at a time frame of the record you are seeing a true record of change on average.
 
Let me prove this to you. I have created a spread sheet simulation that mimics four temperature stations. Two of these are random numbers between 40 and 50, one consists of random numbers between 30 and 40, and the last one consists of random numbers between 50 and 60. I then processed the numbers using their method and lastly using my method.
 
 
When I chart the data, regardless of how many times I generate a new set of random numbers, the result looks like this.
 

I created my simulation to generate numbers that varied within temperature bands, but always maintained the exact theoretical averages of 45, 55, and 65. That would be the orange line above. It always, always, reflects the reality contained in the data regardless of how I play with the data. I can add station data, or take it away, but the actual trend of the data remains at or close to zero as long as I didn't change the average in any station over time. Even then, it would take a fairly large change in one of the four created stations to make much of a difference on the chart.  
 
The blue line, however, demonstrates their model is highly sensitive to changes in the number of stations active and the differences in local climate. This is their flaw, and it is a big one. Where the stations were added in that big change from 1977 to 2000 makes a huge impact. That would not be the case if they had constructed their core mathematical model correctly.
 
The evidence I have presented is, in my opinion, conclusive. Their chart of annual global averages shows a likely probability of correlation between temperature and the number of stations on line. Under examination that correlation does not break down as the CO2 correlation did. In fact further examination supports that correlation. 
 
In order for their model to work they would have had to maintained an exact ratio of weather stations to temperature zones, which they did not do. This is why their over all global data does not match any individual records set. If the basic mathematical model for how they engage with the data is so seriously flawed then any conclusions they derive must also be flawed. They are in fact divorced from reality.
 
Honestly, it is hard to imagine any group of high powered scientists not seeing this basic flaw. It is hard to imagine no one even testing the model against real world results to make sure it was right.
 
Anyway, I believe my work here is finished.