How To Create Garmin Topo Maps - Part 3 - Water Data

Author: Dan Blomberg | Last updated October 16th, 2010 at 07:06pm

Introduction

This part details how to download high resolution water data from the USGS NHD database.  After we download the data we will learn how to process it rapidly via postGIS to get rid of parts we don't want and make it very easy for global mapper or GPSMapEdit to process.

Note: In late 2010 the USGS introduced a new map that allows you to download data by 100k quad and get more segments per download.

Software

Some software is required and some is reccomended for downloading and processing water data:

Downloading The Data (New Map)

  1. Open the excel spreadsheet you downloaded from here.
  2. Go to the new USGS National Map Viewer.
  3. Zoom to the area you want to download (in general) by double clicking on the map; this new map works similar to Google Maps.
  4. Once you are looking at the area where you want to download data click "GIS Toolbox"; it will turn into "Active Tool: Map Navigation"
  5. Now click the USGS tab in the tools bar
  6. Now click the blue arrow (Download Data); the toolbar will convert to something similar to this:
  7. Now click on Index 24k and change it to Index 100k; this should change the grid pattern on the map
  8. Now click on the first 100k quad you want to download (this should be in the order of the excel sheet from step 1 so you know which items you have downloaded.  If the toolbar is in the way just drag it somewhere else on the map.
  9. The quad you click should turn green and have an "A" in it and the overlay toolbar should have opened similar to this:
  10. Click the name of the quad in the overlay toolbar; in my case this is "Flagstaff"
  11. In the box that opens at the quad click "See Available Data"
  12. Put a check box next to "Hydrography", change the data type to "Shapefile" and then click add to cart:
  13. The quad should appear in the cart to the right side of the screen.  Now select the next quad you want to download and repeat steps 9 through 12.  Do this as many times as required to get the hydro data that you need (I normally don't request more than 10-15 quads per order) .
  14. Now click "Checkout"
  15. Enter your email address and wait for the a confirmation email to arrive.  At this point if you didn't get all the data you want continue with steps 9 through 13.
  16. The email with your data links can take up to 48hrs to arrive.
  17. Once the email arrives download the NHD data.  Although multiple links will exist in the email they are all the same file so you only have to download one:
  18. Extract each zip file to its own folder in your nhd folder (for example nhd\NHD######)

You can now continue to "Combining The Data"

Downloading The Data (Old Method)

  1. Open the excel spreadsheet you downloaded from here.
  2. Go to the NHD Geodatabase Viewer
  3. Click "Zoom To State" on the left side.
  4. Select your state and let the viewer reload.
  5. On the right side - click on "Base Map Features"
  6. Now click the check box next to 100k Quad Index.  This will display your 100k Quads
  7. On the right side - click on "Hydrologic Units"
  8. Now make sure the check box is set for Subbasins and click the radio button for subbasins.  This will make subbasins your active layer.  Your right side should look like this:
  9. On the left side click the "Polygon Extract" tool
  10. Drag a box to select the subbasins in your first 100k quad.  A popup will appear at this point and list all the subbasins you've selected. You can only have 10 listed.  If you have more than 10 close the popup and create a new smaller box.
    Note: If a window did not pop up make sure your pop-up blocker is set to allow pop-ups from all usgs.gov addresses (*.usgs.gov).
  11. The top lists the subbasins you have selected.  These will be larger than the 100k quad index; the map will have them highlighted
  12. Select "High Resolution" in the next section section.  In the third section select "Shapefile".  Enter your email address in the final blank.  It should look similar to this:
  13. Click "extract"
  14. When the emails arrive just download the file and extract it to your nhd folder (keep each zip file in their own NHD###### folder).
  15. Repeat these steps until you have selected all the subbasins for you area.  This took 8 emails for the entire state of Mississippi.  Try not to repeat subbasins too many times.

Combining The Data

Each file you received from the NHD will have a seperate folder containing the files we need: NHDArea.shp, NHDFlowline.shp, and NHDWaterbody.shp.  We need to combine all the files we have into a single set of shapefiles that can then be quickly processed.  There are a few ways to do this.

FWTools

FWTools is the fastest and least time consuming way to combine the shapefiles but is slightly more advanced.  It also requires you to have FWTools installed.

  1. Install FWTools and remember its location (normally: C:\Program Files\FWTools2.1.0)
  2. Download the three batch files we will need: Merge NHD Data Batch Files
  3. Extract all three files to your nhd folder.
  4. You will need to edit each one (right click on the file and click Edit)
  5. On the first line change "C:\your_map_directory\nhd" to match your directory, for example "C:\My Maps\nhd"
  6. On the second line change the C:\Program Files\FWTools2.1.0 if you are using a different version than 2.1.0 or installed it to another location.
  7. Save all three files

Now that our basics are setup will will start combining files.  You will repeat this for each type "Waterbody, Area, and Flowline"

  1. Go to the first NHD###### folder and then into the hydrography folder; copy the following files to your nhd folder (the base folder with the three bat files) for each type:
    1. Flowlines - NHDFlowline.dbf, NHDFlowline.prj, NHDFlowline.shp, NHDFlowline.shx
    2. Area - NHDArea.dbf, NHDArea.prj, NHDArea.shp, NHDArea.shx
    3. Waterbody - NHDWaterbody.dbf, NHDWaterbody.prj, NHDWaterbody.shp, NHDWaterbody.shx
  2. Your nhd folder should look similar to this now:
  3. Now open another "My Computer" or "Windows Explorer" and browse to the /nhd folder.
  4. For each NHD###### folder (except the first one) go into it and then take NHDArea.shp and drag it to the other window and onto "mergearea.bat".  Drag each NHDFlowline.shp onto "mergeflowline.bat". Finally, drag each NHDWaterbody.shp onto "mergewaterbody.bat".
  5. Repeat this for all the NHD###### folders you have.

You have now combined all the shapefiles together with FWTools. Continue to Processing The Data.

Global Mapper

  1. Open Global Mapper
  2. For the areas:
    1. Open the first NHDArea.shp file in the first NHD######
    2. Now open all the other NHDArea.shp files in each individual NHD###### folder.
    3. Click File>Export Vector Data>Export Shapefile
    4. Check the "Export Areas" box
    5. Save the file as NHDArea.shp in your main /nhd folder
  3. For the flowlines:
    1. Open the first NHDFlowline.shp file in the first NHD######
    2. Now open all the other NHDFlowline.shp files in each individual NHD###### folder.
    3. Click File>Export Vector Data>Export Shapefile
    4. Check the "Export Lines" box
    5. Save the file as NHDFlowlines.shp in your main /nhd folder
  4. For the waterbodies:
    1. Open the first NHDWaterbody.shp file in the first NHD######
    2. Now open all the other NHDWaterbody.shp files in each individual NHD###### folder.
    3. Click File>Export Vector Data>Export Shapefile
    4. Check the "Export Areas" box
    5. Save the file as NHDWaterbody.shp in your main /nhd folder

You have now combined all the shapefiles together with Global Mapper. Continue to Processing The Data.

GPSMapEdit

Although it is possible to combine the files by opening them all and then saving them as a shapefile I do not reccomend it because it will make processing the data more difficult since you have to type them all currently.  I would use the FWTools method over GPSMapEdit.  If you want to do it with GPSMapEdit just select a random type for each area and line because we will override it when we process the data.

Processing The Data

Although GPSFileDepot has an article detailing how to process the water data already this tutorial will vary slightly because if you have followed the steps so far all the files are premade for you.  We will use PostGIS for the processing.

Putting The Data In The Database

 A few assumptions are made about your PostGIS setup.  We assume you have a postgis database called postgis. We assume you have a postgres username.

  1. Open the command prompt (Start>Run>cmd)
  2. Change to the postgresql bin directory normally: C:\Program Files\PostgreSQL\8.3\bin use this command: cd "C:\Program Files\PostgreSQL\8.3\bin"
  3. We will use the following command to convert the shapefiles to sql files:
    1. Areas - shp2pgsql "C:\your_map_directory\nhd\NHDArea.shp" NHD_Area > nhd_area.sql
      Make sure the part in blue points to your actual map directory.
    2. Flowlines- shp2pgsql "C:\your_map_directory\nhd\NHDFlowline.shp" NHD_Flowline > nhd_flowline.sql
      Make sure the part in blue points to your actual map directory.
    3. Waterbody- shp2pgsql "C:\your_map_directory\nhd\NHDWaterbody.shp" NHD_Waterbody > nhd_waterbody.sql
      Make sure the part in blue points to your actual map directory.
  4. Now that we have a .sql file we need to load it into the database with this commands:
    1. Areas -  psql -d postgis -h localhost -U postgres -p -f nhd_area.sql
      Remove the red -p if your username does not use a password.
    2. Flowlines - psql -d postgis -h localhost -U postgres -p -f nhd_flowline.sql
      Remove the red -p if your username does not use a password.
    3. Waterbody - psql -d postgis -h localhost -U postgres -p -f nhd_waterbody.sql
      Remove the red -p if your username does not use a password.

Now that the data is loaded we will start processing it.  You can do this with pgAdmin III (as shown in this tutorial) but we have provided a zip file with all the queries to make life easier.

Processing The PostGIS Data

  1. With the DOS prompt that is still open and hopefully in the PostgreSQL bin folder we will execute a series of queries.
  2. Download the NHD queries.
  3. Extract all three files to the PostgreSQL bin folder
  4. In the dos prompt run the following commands for each water type:
    1. Areas - psql -d postgis -h localhost -U postgres -p -f area_sql.txt
      Remove the red -p if your username does not use a password.
    2. Flowlines - psql -d postgis -h localhost -U postgres -p -f flowline_sql.txt
      Remove the red -p if your username does not use a password.
    3. Waterbody - psql -d postgis -h localhost -U postgres -p -f waterbody_sql.txt
      Remove the red -p if your username does not use a password.

This has processed all the data and given it appropriate MP_TYPES.

Export Shapefiles From PostGIS

Your command prompt should still be open to the PostgreSQL bin folder at this time.

  1. Modify and execute the following commands.  Make sure the blue points to your nhd directory and remove the -p if your username does not require a password
    1. Areas - pgsql2shp  -f "C:\your_map_directory\nhd\Area.shp" -u postgres -p postgis nhd_area
    2. Flowlines - pgsql2shp  -f "C:\your_map_directory\nhd\Flowline.shp" -u postgres -p postgis nhd_flowline
    3. Waterbody - pgsql2shp  -f "C:\your_map_directory\nhd\Waterbody.shp" -u postgres -p postgis nhd_waterbody
  2. At this time delete nhd_area.sql, nhd_flowline.sql, and nhd_waterbody.sql.
  3. Go to your nhd folder
  4. Rename the following files:
    1. NHDArea.prj --> Area.prj
    2. NHDFlowline.prj --> Flowline.prj
    3. NHDWaterbody.prj --> Waterbody.prj
  5. Delete all NHDArea, NHDFlowline, and NHDWaterbody files.
  6. This should leave you with a set of files similar to this:

Final Data Preparations

Global Mapper

The data is already prepared for use by Global Mapper.  No editing of the shapefiles is needed.  Just make sure you have all three shapefiles in your nhd folder.  At this time you can also delete/archive the old files and the NHD###### folders.

GPSMapEdit

  1. Open your first shapefile with GPSMapEdit
  2. On the screen that pops up asking you to select the "type of object" click the "From Field" tab
  3. Scroll until you find the "MP_TYPE" column.
  4. Click on the header to select the column, it should look similar to this:
  5. Click Next
  6. Click the "name" column for naming.
  7. Check and make sure the datum is NAD83 and click next.
  8. Click finish
  9. Click file> save map as
  10. Name it something useful and save the .mp file in your nhd folder.
  11. Repeat this process for all three water types and you'll be good to go.

What's Next?

Next we will go over getting and processing the transportation data including roads, railroads, highways, and a few "odd" lines such as powerlines and ferry routes.

<-- Previous - Elevation Data | Table Of Contents | Next - Transportation -->