florida Florida Freeze and Weather Station Data

[kinzyjr 3,688]

NOAA makes weather records from the Florida stations available to download for free.  Out of the 1,700+ stations, 358 contain at least some temperature data.  If one were to download all of these records, import them into a database, and use some crafty SQL queries to generate sheets for each of the impact freezes it would provide a really good side-by-side resource to compare the microclimates in each region of Florida.

That is exactly what has been done with the 0000_202011040720_F_SQL_v2.xlsx spreadsheet attached to this post.  Each tab in the spreadsheet contains records for one of our unfortunate cold events.  The records come sorted first by the TAG column, which represents one of the areas in the pictures and is defined by a set of latitude and longitude boundaries.  A second sort is by the station name alphabetically.  This gives you a region of weather stations sorted alphabetically that allows you to see the temperatures in the region beside each other.

The lists are able to be filtered or sorted in any way you choose, so if you are only interested in weather stations in a particular region or set of regions, this is easily accomplished.  For easy viewing, the rows for each region alternate in shading.  This is easy to remove or change if you wish.

If you would like to see the boundaries of a region, the weather stations used with a link to their corresponding Weather Underground station, or a listing of the freezes and some commentary lifted from Florida Citrus Mutual’s website, or a description of the station location, the 202007121300_NOAA_WeatherStations_TemperatureOnly.xlsx sheet will have a plethora of this information.

For anyone who likes to look at the various airport weather stations on Wunderground, 202004292350_AirportWeatherStations.xlsx will give you as complete listing of these stations as I could assemble.

Now for a description of the various TAGs:

PEN = Pensacola area

PAN - Panama City Area

TAL - Tallahassee Area

EPN - Eastern Panhandle

CNF - Central North Florida

JAX - Jacksonville area

NEF - Northeast Florida - Dayona + St. Augustine and surrounding area

NWC - Northwest Central Florida

SWC - Southwest Central Florida

NIC - North Inland Central Florida

SIC - South Inland Central Florida

ECF - East Central Florida

SWF - Southwest Florida

SEF - Southeast Florida

MUK - Miami and the Upper Keys

KEY - The South and Western Florida Keys

0000_202011040720_F_SQL_v2.xlsx 202007121300_NOAA_WeatherStations_TemperatureOnly.xlsx 202004292350_AirportWeatherStations.xlsx

[RedRabbit 1,534]

This is awesome @kinzyjr! I look forward to playing around with the data whenever I get some time.

[Reeverse 615]

These are great!! Thanks 

[JJPalmer 305]

This is fantastic - I’ve only searched through SW and SWC Florida, but the data really helps visualize the great microclimate of St. Petersburg. Often the city stays as warm as Naples and Fort Meyers and is consistently the warmest in the SWC designation. Wish there was a station on Anna Maria.  Again, great job on compiling! 

[kinzyjr 3,688]

After seeing @pj_orlando_z9b's breakdown of days below 40F: ( https://www.palmtalk.org/forum/index.php?/topic/66579-florida-freeze-watch-december-2020/&page=6&tab=comments#comment-970076) , I decided I'd generate the results for all of the NOAA weather stations.  Download the attached sheet for the results.

202012281630_DaysUnder40.xlsx

[RedRabbit 1,534]

1 hour ago, kinzyjr said:

After seeing @pj_orlando_z9b's breakdown of days below 40F: ( https://www.palmtalk.org/forum/index.php?/topic/66579-florida-freeze-watch-december-2020/&page=6&tab=comments#comment-970076) , I decided I'd generate the results for all of the NOAA weather stations.  Download the attached sheet for the results.

202012281630_DaysUnder40.xlsx 328.37 kB · 0 downloads

Nice work!

I was thinking this might be a good alternative to standard hardiness zones. Particularly since tropical plants don't do well for extended periods below 40f. 

[Mr. Coconut Palm 1,118]

I wish someone a lot more computer savvy than me would do that for South Texas and the Rio Grande Valley, and make a corresponding Climate Zone Map of the area as a result of it.

John

[kinzyjr 3,688]

1 hour ago, Mr. Coconut Palm said:

I wish someone a lot more computer savvy than me would do that for South Texas and the Rio Grande Valley, and make a corresponding Climate Zone Map of the area as a result of it.

John

If you can get a group of people onboard to help you, downloading the data from NOAA and setting up a database to hold the imports is probably the most important part.  There are over 5,000 stations in Texas.  There are probably at least 1,000 that hold MAX and MIN temperature data. 

Any possibility of enlisting some volunteers from the Palm Society of South Texas, the Houston IPS affiliate, or getting any of the students at the universities to help out?

I can share the tools and the methods I used to import, select, and sanitize the data.  Basically, Microsoft Excel and SQLite database were the predominant tools used.  There's a table that holds all of the stations, using the NOAA label (ex. GHCND:USC00410012) as the primary key (can't be a duplicate).  The temperature data is uploaded with the NOAA label, high, low, and precipitation data.  Since both tables use the NOAA label, you can basically join them together and queries against them.  That's where having a techie onboard comes in handy.

Here are some helpful hints for anyone or any group(s) of people wanting to do the data collection and analytics:

To get the data for a state:

1. Access the NOAA Search Tool currently at this website: https://www.ncdc.noaa.gov/cdo-web/
2. Click on SEARCH TOOL under the DISCOVER DATA BY section
3. Fill out the search using the info in Figure 1 (ex. is for TX) and click Search. 
   1. Note the date range should be from the earliest possible date to the latest to get the most data.
4. When the screen comes up for the state you chose, click on the State name in blue (it's a hyperlink).
   1. See Figure 2
5. You should come to a screen where it gives you a summary of all of the stations in the data set.
6. Scroll down and click the Station List (Figure 3).
   1. Yes, there really are 5308 stations
7. When you find a station that has the data you want, you can click on it and then click ADD TO CART.

Figure 1: Search criteria

 

Figure 2: The whole state with the blue link

Figure 3: Station List

How to tell if a station may have temperature data:

1. Look at Figure 3 above and ask what is different about the one I put the red box around from the rest.
   1. You should notice that all of the others have decimals.
   2. The ones with decimals typically do not have temperature data.
   3. The ones without numbers or with whole numbers typically DO have temperature data.
      1. Examples:
      2. ABERNATHY, TX US doesn't have decimals or even a number.  It likely has data.
      3. LAKELAND 2, FL, US has a number, but it is a whole number.  It likely has data.
      4. ABILENE 3.3 SW, TX US has a decimal.  It likely does not have data.

What information should I capture about each NOAA station?:

1. In a word, everything.  You need the Network ID to use as the NOAA_ID in your database.  I recommend capturing the name, latitude and longitude, elevation, start date and end date.  In my table, I also added special notes, a region tag, and a column where I could add a link to the corresponding weather underground station.

What information should I capture in each weather record?:

1. The minimum you'll need to capture are the weather station, date, high, and low.   I also capture the precipitation in my records.  I typically download everything and then just filter out what I don't want.

How do I import data into SQLite:

1. Online Documentation: https://www.sqlitetutorial.net/sqlite-import-csv/
2. I'd recommend using the Import a CSV file into a table using SQLite Studio section of this documentation.

Some issues you may run into (and ways to deal with them):

1. Bogus data:
   1. Ever see -100F in Texas in June?  Yeah, not here in Florida either.  It's difficult to correct negatives as there could have been negative temperatures in areas of the state; so just multiplying by -1 doesn't always change it to the correct value.  You pretty much have to look for all of the negatives, and decide which ones to correct on a case by case or group by group basis.
   2. Blanks: When you query, you can filter these out with a WHERE condition this: WHERE LOW <> ''.  This keeps them from polluting your AVG and MIN functions when finding record lows or average lows.
   3. Ridiculous Numbers: Did you know it was 3F in South Florida one time?  Yeah, me either.  When running queries, the TAG regions I set up allowed me to set constraints for that region on what the low temperature was allowed to be.  That helped significantly in filtering out obviously spurious data.
2. Missing or Incomplete data:
   1. When you download data for a large metro area, sometimes the reporting station changes several times over a century or so.  Your only option for contiguous data is to merge them together or set date ranges to select from one station if you are getting data from a set of years in a range, but another station with a different date range.  The start and end dates on the NOAA station records can help with this.
3. Data that fails to import:
   1. 99.9% of the time, it's because the file is formatted wrong.  They need to be saved as CSV for the process above.  Sometimes Excel will save a ton of blank rows under the actual data that will fail any constraints set up on the data.
4. Database crashes:
   1. Always know where the database file is stored and make copies of it frequently.  Put the copies somewhere safe.  In my case, I have several jump drives with historical copies as well as having copies on mirrored drives.  Redoing work is not a lot of fun.

This is probably enough for anyone or any group that wants to tackle the endeavor for a different state.  If anyone has questions, post them and I'll try to answer them timely.

[Mr. Coconut Palm 1,118]

18 hours ago, kinzyjr said:

If you can get a group of people onboard to help you, downloading the data from NOAA and setting up a database to hold the imports is probably the most important part.  There are over 5,000 stations in Texas.  There are probably at least 1,000 that hold MAX and MIN temperature data. 

Any possibility of enlisting some volunteers from the Palm Society of South Texas, the Houston IPS affiliate, or getting any of the students at the universities to help out?

I can share the tools and the methods I used to import, select, and sanitize the data.  Basically, Microsoft Excel and SQLite database were the predominant tools used.  There's a table that holds all of the stations, using the NOAA label (ex. GHCND:USC00410012) as the primary key (can't be a duplicate).  The temperature data is uploaded with the NOAA label, high, low, and precipitation data.  Since both tables use the NOAA label, you can basically join them together and queries against them.  That's where having a techie onboard comes in handy.

Here are some helpful hints for anyone or any group(s) of people wanting to do the data collection and analytics:

To get the data for a state:

1. Access the NOAA Search Tool currently at this website: https://www.ncdc.noaa.gov/cdo-web/
2. Click on SEARCH TOOL under the DISCOVER DATA BY section
3. Fill out the search using the info in Figure 1 (ex. is for TX) and click Search. 
   1. Note the date range should be from the earliest possible date to the latest to get the most data.
4. When the screen comes up for the state you chose, click on the State name in blue (it's a hyperlink).
   1. See Figure 2
5. You should come to a screen where it gives you a summary of all of the stations in the data set.
6. Scroll down and click the Station List (Figure 3).
   1. Yes, there really are 5308 stations
7. When you find a station that has the data you want, you can click on it and then click ADD TO CART.

Figure 1: Search criteria

 

Figure 2: The whole state with the blue link

Figure 3: Station List

How to tell if a station may have temperature data:

1. Look at Figure 3 above and ask what is different about the one I put the red box around from the rest.
   1. You should notice that all of the others have decimals.
   2. The ones with decimals typically do not have temperature data.
   3. The ones without numbers or with whole numbers typically DO have temperature data.
      1. Examples:
      2. ABERNATHY, TX US doesn't have decimals or even a number.  It likely has data.
      3. LAKELAND 2, FL, US has a number, but it is a whole number.  It likely has data.
      4. ABILENE 3.3 SW, TX US has a decimal.  It likely does not have data.

What information should I capture about each NOAA station?:

1. In a word, everything.  You need the Network ID to use as the NOAA_ID in your database.  I recommend capturing the name, latitude and longitude, elevation, start date and end date.  In my table, I also added special notes, a region tag, and a column where I could add a link to the corresponding weather underground station.

What information should I capture in each weather record?:

1. The minimum you'll need to capture are the weather station, date, high, and low.   I also capture the precipitation in my records.  I typically download everything and then just filter out what I don't want.

How do I import data into SQLite:

1. Online Documentation: https://www.sqlitetutorial.net/sqlite-import-csv/
2. I'd recommend using the Import a CSV file into a table using SQLite Studio section of this documentation.

Some issues you may run into (and ways to deal with them):

1. Bogus data:
   1. Ever see -100F in Texas in June?  Yeah, not here in Florida either.  It's difficult to correct negatives as there could have been negative temperatures in areas of the state; so just multiplying by -1 doesn't always change it to the correct value.  You pretty much have to look for all of the negatives, and decide which ones to correct on a case by case or group by group basis.
   2. Blanks: When you query, you can filter these out with a WHERE condition this: WHERE LOW <> ''.  This keeps them from polluting your AVG and MIN functions when finding record lows or average lows.
   3. Ridiculous Numbers: Did you know it was 3F in South Florida one time?  Yeah, me either.  When running queries, the TAG regions I set up allowed me to set constraints for that region on what the low temperature was allowed to be.  That helped significantly in filtering out obviously spurious data.
2. Missing or Incomplete data:
   1. When you download data for a large metro area, sometimes the reporting station changes several times over a century or so.  Your only option for contiguous data is to merge them together or set date ranges to select from one station if you are getting data from a set of years in a range, but another station with a different date range.  The start and end dates on the NOAA station records can help with this.
3. Data that fails to import:
   1. 99.9% of the time, it's because the file is formatted wrong.  They need to be saved as CSV for the process above.  Sometimes Excel will save a ton of blank rows under the actual data that will fail any constraints set up on the data.
4. Database crashes:
   1. Always know where the database file is stored and make copies of it frequently.  Put the copies somewhere safe.  In my case, I have several jump drives with historical copies as well as having copies on mirrored drives.  Redoing work is not a lot of fun.

This is probably enough for anyone or any group that wants to tackle the endeavor for a different state.  If anyone has questions, post them and I'll try to answer them timely.

Kinzy,

Thank you.  I will try to find someone over here who can do it.  Your suggestion of getting someone in the Palm Society of South Texas is a good idea if any of them are computer savvy, especially since we are not having meetings now due to the Covid situation.  This would be a good project for someone in the group that would really benefit all of us.

John

[Xenon 1,001]

On 1/2/2021 at 7:29 PM, Mr. Coconut Palm said:

Kinzy,

Thank you.  I will try to find someone over here who can do it.  Your suggestion of getting someone in the Palm Society of South Texas is a good idea if any of them are computer savvy, especially since we are not having meetings now due to the Covid situation.  This would be a good project for someone in the group that would really benefit all of us.

John

Hi John, here is a very rough map of zone 9 and 10 in Texas (based on the numbers only): 

[Austinpalm 112]

13 minutes ago, Xenon said:

Hi John, here is a very rough map of zone 9 and 10 in Texas (based on the numbers only): 

Great map Jonathan!  Thanks for developing it.  Very interesting to see the small area of 10a out in Starr and Zapata counties as well as areas in Galveston/Houston.  

[Mr. Coconut Palm 1,118]

On 1/4/2021 at 12:44 AM, Xenon said:

Hi John, here is a very rough map of zone 9 and 10 in Texas (based on the numbers only): 

Hi Jonathan,

Thanks.   That is great! Great job.  It looks like Zone 10A extends all the way up to the backside of Matagorda Island, which makes sense since I have heard that there are some Red Mangroves growing on the backside (bayside) of Matagorda Island.

John

[Austinpalm 112]

On 1/4/2021 at 12:44 AM, Xenon said:

Hi John, here is a very rough map of zone 9 and 10 in Texas (based on the numbers only): 

Jonathan,

Did you crunch numbers for the entire state and then only show areas that were zone 9 or higher; or did you crunch numbers from just a portion of the state and show the results of that effort?  I ask because I am a little surprised that there is not a 9a UHI in Austin.  I was noticing a few more tropical things surviving in the downtown area before I moved south.  Just curious.  Thanks for the effort!  Am also curious how possible it is to get better resolution for South Texas.  Does the 10b extend south to Boca Chica an possibly inland a bit? Not much development in that area, so getting data is difficult I expect.  The same can be said for many areas of South Texas.  Probably why I like it. 

[Xenon 1,001]

1 hour ago, Austinpalm said:

Jonathan,

Did you crunch numbers for the entire state and then only show areas that were zone 9 or higher; or did you crunch numbers from just a portion of the state and show the results of that effort?  I ask because I am a little surprised that there is not a 9a UHI in Austin.  I was noticing a few more tropical things surviving in the downtown area before I moved south.  Just curious.  Thanks for the effort!  Am also curious how possible it is to get better resolution for South Texas.  Does the 10b extend south to Boca Chica an possibly inland a bit? Not much development in that area, so getting data is difficult I expect.  The same can be said for many areas of South Texas.  Probably why I like it. 

I only looked for where to draw the zone 9 and 10 borders. 

I agree with you about 9a UHI in Austin; I was lazy and mostly used airport data and the stations available from local NWS pages rather than the whole NCDC archive (not a huge difference). Camp Mabry is definitely 9a, southern and southeastern Travis county straddles the 8b/9a border. 

There is very little data available for coastal Cameron county. I think 10b is limited only to SPI and areas with the Laguna Madre directly to the north and northwest like Port Isabel (and probably Brazos Island/Boca Chica though South Bay is shallow even relative to the LM). Bayfront areas without water to the N/NW like Laguna Vista run a bit colder and KPIL sometimes runs even colder than Brownsville being in a rural area despite being fairly close to the coast. Port Mansfield is not very warm either, about on par with Harlingen. 

If I had to pick the "one warmest spot in TX", it would probably be the north end of Brazos Island with the ship channel directly due north and the deepest part of the Laguna Madre (near the causeway) to the northwest. 

Edited January 16 by Xenon

[Jeff985 616]

8 hours ago, Xenon said:

I only looked for where to draw the zone 9 and 10 borders. 

I agree with you about 9a UHI in Austin; I was lazy and mostly used airport data and the stations available from local NWS pages rather than the whole NCDC archive (not a huge difference). Camp Mabry is definitely 9a, southern and southeastern Travis county straddles the 8b/9a border. 

There is very little data available for coastal Cameron county. I think 10b is limited only to SPI and areas with the Laguna Madre directly to the north and northwest like Port Isabel (and probably Brazos Island/Boca Chica though South Bay is shallow even relative to the LM). Bayfront areas without water to the N/NW like Laguna Vista run a bit colder and KPIL sometimes runs even colder than Brownsville being in a rural area despite being fairly close to the coast. Port Mansfield is not very warm either, about on par with Harlingen. 

If I had to pick the "one warmest spot in TX", it would probably be the north end of Brazos Island with the ship channel directly due north and the deepest part of the Laguna Madre (near the causeway) to the northwest. 

That looks pretty accurate. Based on my observations I’d push the zone 10 further up Galveston bay. At least if you’re within a mile of the water I’d say the zone 10 stops at the Fred Hartman bridge. Also the lake front neighborhoods around Clear Lake would be 10a. San Leon, specifically Clifton Beach, has a few weather stations that are often warmer than Galveston. There are some good looking, really tall, Norfolk Island pines in that area. 

[Xenon 1,001]

3 hours ago, Jeff985 said:

That looks pretty accurate. Based on my observations I’d push the zone 10 further up Galveston bay. At least if you’re within a mile of the water I’d say the zone 10 stops at the Fred Hartman bridge. Also the lake front neighborhoods around Clear Lake would be 10a. San Leon, specifically Clifton Beach, has a few weather stations that are often warmer than Galveston. There are some good looking, really tall, Norfolk Island pines in that area. 

Yes, I noted the southern shore of Galveston Bay in the map. Kemah and lakefront Clear Lake is where I would place the cutoff though.  Large Norfolk Pine is actually one of the key things I look for in a "numerical 10a" climate for areas that don't have weather stations. 

[Jeff985 616]

6 hours ago, Xenon said:

Yes, I noted the southern shore of Galveston Bay in the map. Kemah and lakefront Clear Lake is where I would place the cutoff though.  Large Norfolk Pine is actually one of the key things I look for in a "numerical 10a" climate for areas that don't have weather stations. 

There’s a decent looking Norfolk in La Porte's Shady River neighborhood, which is about 1/2 mile inland. It does have some overhead canopy. Google street view images are from 2011 and it wasn’t there then, but it is pre 2018. I’ll try to get a picture the next time I drive through that neighborhood. There was also one on Meadowlawn Street in Shoreacres that was damaged in 2018 but survived. Apparently the owner didn’t like the way it looked after the freeze since it was removed about a year later. 

[Jeff985 616]

On 1/17/2021 at 6:18 AM, Jeff985 said:

There’s a decent looking Norfolk in La Porte's Shady River neighborhood, which is about 1/2 mile inland. It does have some overhead canopy. Google street view images are from 2011 and it wasn’t there then, but it is pre 2018. I’ll try to get a picture the next time I drive through that neighborhood. There was also one on Meadowlawn Street in Shoreacres that was damaged in 2018 but survived. Apparently the owner didn’t like the way it looked after the freeze since it was removed about a year later. 

@XenonHere’s the Shady River La Porte Norfolk. I also found one in Morgan’s Point, but I’m not sure that one was there prior to 2018.