Statistical Accuracy of Internet Weather Forecasts

Interesting article linked from Slashdot last Friday, involves weather accuracy as reported online, and includes my employer the National Weather Service (NWS/NOAA).

Brandon Hansen considers the statistical accuracy of popular on-line weather forecast sources and shows who's on target, and on who you probably shouldn't rely. Motivated by a trip to a water park that was spoiled with hail despite a 'clear sky' forecast, he does a nice job of depicting deviations, averages, and overall accuracy in a manner that stats junkies are sure to love.

The entry has a lot of charts and is a little difficult to muddle through (unless you're a statistician) and seems to have a few drawbacks: It only measures over 14 days (not a long enough sample IMHO) and it only measures temperature accuracy, not the precipitation - which is what I thought his whole reason for starting this experiment.

One insightful commenter on the Slashdot page posted:

Kudos to this guy for the work he put into the effort but it is really comparing apples and oranges. A forecast is a time sensitive product. You can't look at the forecast provided on day x from two different sources and compare them unless the forecast was provided at the same time of day.

The National Weather Service collects all the weather data used by forecasters, they also provide the 1st forecast. AccuWeather and others take the National Weather Service forecast then watch the new data (using National Weather Service provided data) to offer a refined forecast a few hours latter. Who do you think is going to be the most accurate the guy who provides the first forecast or the guy who waits for more data and then refines the for cast? AccuWeather's has statistics that show they are more accurate then the National Weather Service but if you used the AccuWeather forecast then waited for the next National Weather Service update I bet National Weather Service would be more accurate.

I am surprised that this guy used the weather.com and not the National Weather Service for the actually temp for all his calculations. (It doesn't matter b/c I am sure weather.com is right from National Weather Service data). He did point out that AccuWeather is the only one who provides forecasts > 10 days in advance.

My preference for weather forecasts is:

National Weather Service
AccuWeather (easy to understand graphics and 2 week forecasts)
The Weather Underground (Years ago they were the 1st to provided free access to hurricane computer models)

Ever since I went to work for NOAA, I've started using their forecast site weather.gov, so far it's been pretty reliable. Not as fancy or customizable as weather.com (no cookies set to recognize your home town on further visits) but it has the raw tools you need.

Speaking of which, I need to go see if they've got any further predictions on the snow we're expecting tonight through tomorrow... maybe I can predict another snow day from work...