In a previous article, I covered 5 Sources of Free Weather Data for your Site, but did not provide any actual code to use the data. Since then, I covered sources #1 and #2. None of these sources had any English-formed forecasts. For that, we have to go to the Zone Forecast Product, but we don’t have the luxury of XML here. There is a healthy bit of RegEx and string manipulation, but you can do it!
Where to Get the Data
The data is available by browsing the http://www.weather.gov/data/ directory. Here, you will find a whole set of products available. To access the ZFP data, you need to browse the http://www.weather.gov/data/XXX/ZFPXXX directory where ‘XXX’ is the code for the local station from where the data is to be retrieved from. For example, if I wanted the ZFP data for KLOT (Lockport, IL), I would call up http://www.weather.gov/data/LOT/ZFPLOT.
There is an exception to this rule, however. If you look at http://www.weather.gov/data/AFC/, they show the ZFP for two areas. I think some of these are like this because one office is forecasting for to areas or something (correct me in the comments if I’m wrong). So that formula above is more of a guideline for finding the right link for your area, but I found it to be 95% accurate. (That’s still an ‘A’, right?)
Understanding the Big Picture
Click on the thumbnail for the explained section of this data. Starting at the top is the header. We don’t use this. It contains the station ID and the type of product this is – we know that already. It also contains the release time. Although that is significant, that same information is in the header of the individual forecasts as well, so I will be discarding that in this example.
The second yellow box is the header for that specific forecast zone. We will be using some data from this. The zone code is the most critical because we need to know for which zone this forecast is for. The only other piece of information in this box that I use is the Release Time, which is when the forecast was released (ironic). This block of information is repeated at the top of every forecast in the file.
Below the second yellow box is the actual forecast data, which is the meat of what we are looking for. Below that is the separator, which is means that the code block for the next forecast zone is next, and so on…
Breaking up the Forecasts
We only want to deal with one forecast zone at a time, so lets break them up. First, I need to get rid of that pesky header at the top of the file. We do that by splitting the data by the double line breaks, and removing that first paragraph from the resulting array. Then, join it back together.
/* remove the file header */
/* $data = {text from http://www.weather.gov/data/LOT/ZFPLOT} */
$data = trim($data);
$data = explode("\n\n", $data);
unset($data[0], $data[1]);
$data = implode("\n\n", $data);
Now, we can split of the zone forecasts. We simply split the text by the separator (‘$$’), and keep the forecasts in an array.
/* separate the forecasts */
/* $data = {result from previous example} */
$data = explode('$$', $data);
/* trim off any extra line breaks */
$data = array_map('trim', $data);
Perfect. Now we have an array or forecasts with their respective header block.
Parsing the Zone Codes
This is the hardest part of understanding the data. We need to know for which geographical area the forecast is for. Most zone forecasts are only for one zone, but it is possible that it may be for more than one. In the above example, the zone code is ILZ014, which is zone 14 of Illinois. Unfortunately, the zone code & purge date line can be as complicated as ILZ001>003-005-IAC045-163-051200-, which is Illinois zone 1, 2, 3, 5 as well as Iowa counties 45 and 163. For the full specs on how to understand this, see the Universal Geographic Code specifications.
Before we can do anything, we need to find the zone codes. To do that, we need to use one heck of a regular expression, found on line 8 of the sample below. This is what I used to verify that I am finding everything correctly.
<?php
/* get the forecast from a local cache */
/* http://www.weather.gov/data/LOT/ZFPLOT */
$data = file_get_contents('ZFPIND.txt');
/* the regular expression to find all the location codes */
$regex = '/(([A-Z]{2})(C|Z){1}([0-9]{3})((>|-)[0-9]{3})*)-/';
/* for this example, we are doing a replacement to show we found everything...
normally, something like preg_match() would be used */
$data = preg_replace($regex, '<span style="background: #ff0; ">' . "\${1}" . '</span>-', $data);
/* do I really need to explain this? */
echo '<pre>' . $data . '</pre>';
/**
* Why do I comment out the PHP closing tag?
* See: http://phpstarter.net/2009/01/omit-the-php-closing-tag/
*/
/* ?> */
We’re not done with this yet. As you can see from the Universal Geographic Code reference page, they like to make shortcuts. For example, they will use INZ021-028>031 instead of INZ021-INZ028-INZ029-INZ030-INZ031. From a programming standpoint, I think we both know which is better to understand. When I put this data in a database, I need to be able to query a specific location and see if there is data for it. So, if I store it as INZ021-028>031, how do I query for zone 29? These ranges will have to be expanded.
I’m not going to go into great detail on how to do this…I will just provide a couple functions to do it. Call parse_zones() like I do at line 69, and it will take care of the rest.
<?php
/**
* The NWS combines does not repeat the state code for multiple zones...not good for our purpose
* All we want to do here is convert ranges like INZ021-028 to INZ021-INZ028
* We will also call the function to expand the ranges here.
* See: http://www.weather.gov/emwin/winugc.htm
*/
function parse_zones($data)
{
/* first, get rid of newlines */
$data = str_replace("\n", '', $data);
/* split up individual states - multiple states may be in the same forecast */
$regex = '/(([A-Z]{2})(C|Z){1}([0-9]{3})((>|-)[0-9]{3})*)-/';
$count = preg_match_all($regex, $data, $matches);
$total_zones = '';
foreach ($matches[0] as $field => $value)
{
/* since the NWS thought it was efficient to not repeat state codes, we have to reverse that */
$state = substr($value, 0, 3);
$zones = substr($value, 3);
/* convert ranges like 014>016 to 014-015-016 */
$zones = expand_ranges($zones);
/* hack off the last dash */
$zones = substr($zones, 0, strlen($zones) - 1);
$zones = $state . str_replace('-', '-'.$state, $zones);
$total_zones .= $zones;
}
$total_zones = explode('-', $total_zones);
return $total_zones;
}
/**
* The NWS combines multiple zones into ranges...not good for our purpose
* All we want to do here is convert ranges like 014>016 to 014-015-016
* See: http://www.weather.gov/emwin/winugc.htm
*/
function expand_ranges($data)
{
$regex = '/(([0-9]{3})(>[0-9]{3}))/';
$count = preg_match_all($regex, $data, $matches);
foreach ($matches[0] as $field => $value)
{
list($start, $end) = explode('>', $value);
$new_value = array();
for ($i = $start; $i <= $end; $i++)
{
$new_value[] = str_pad($i, 3, '0', STR_PAD_LEFT);
}
$data = str_replace($value, implode('-', $new_value), $data);
}
return $data;
}
$zones = parse_zones("INZ021-028>031-035-036-190915-");
header('Content-type: text/plain');
var_dump($zones);
/**
* Why do I comment out the PHP closing tag?
* See: http://phpstarter.net/2009/01/omit-the-php-closing-tag/
*/
/* ?> */
Putting it All Together
So we know how to download the latest forecast data for a specific station, split it up, and get the zones for each one. Now it’s time to put it together and in a format where we can store or display it. In this example, I am adding some more basic PHP to form a large array with the forecast block and zones in each element. From there, you can easily place the data in a database or whatever.
Note: Note that I am using data from KIND and not KLOT in this example, because KIND has forecast blocks that cover multiple zones, showing how parse_zones() work.
<?php
/**
* adds the two functions from the previous example:
* parse_zines()
* parse_ranges()
*/
include('functions.php');
$data = file_get_contents('ZFPIND.txt');
/* remove the file header */
$data = trim($data);
$data = explode("\n\n", $data);
unset($data[0], $data[1]);
$data = implode("\n\n", $data);
/* separate the forecasts */
$data = explode('$$', $data);
/* trim off any extra line breaks */
$data = array_map('trim', $data);
$data_form = array();
foreach ($data as $field => $value)
{
$lines = explode("\n", $value);
$zones = parse_zones($lines[0]);
$blocks = explode("\n\n", $value);
$forecast = $blocks[1];
$data_time = parse_date_time($value);
$data_form[] = array('zones' => $zones, 'date_time' => $date_time, 'forecast' => $forecast);
}
header('Content-type: text/plain');
var_dump($data_form);
/**
* Why do I comment out the PHP closing tag?
* See: http://phpstarter.net/2009/01/omit-the-php-closing-tag/
*/
/* ?> */
After running the example, you will see that we have an array with several numbered elements. Each element is a sub-array containing an array of zones and an element containing the forecast data.
So there you have it. With one request to the NWS website, you can gather all the ZFP data for an entire forecast area. Again, this is useful when you need to gather forecasts for the general area, not just one zone at a time. If you only need to grab one zone at a time, stay tuned because I will be writing soon on how to make use of an easier data source that grabs one zone at a time that makes for easier parsing on your part.
Making it Look Pretty
Another teaser for a future article…Once you find the page containing the data you want to work with, it looks like an undaunted task to turn that data into something presentable. Let me assure you right off the bat that it is doable with a healthy bit of string manipulation and regular expressions. Just for a bit of comparison, you should be able to turn this into this. How do we split up the days and add the cool icons? I will cover that in the next article. To be continued…