DEM o'Graphics Potpourri 3

WCS, MicroDEM, Leveller, Photoshop & those damned datums

Supplement to 3D Artist issue #45, page 28-29 article by Thomsen-Norre

Editor's note: This page by the author supplements his feature article in 3DA#45 about how to use the Windows programs MicroDEM (free) and Leveller (us$75) along with Photoshop to get vector data of various datums into World Construction Set 5.50+.

C O N T E N T S

7.5-Minute SDTS New Format DEMs

As 3DA#45 was going to press, the 7.5-minute world suddenly became more complicated. The SDTS 7.5-minute DEMs we've all come to know contained a positional error. For most practical purposes, the error was meaningless to most of us, but, in correcting it, a somewhat different SDTS format was born. The Old Format 7.5-minute SDTS DEMs are no longer available on the USGS GeoData site, and USGS has passed the free download responsibility for the New Format SDTS DEMs to GIS Data Depot (GISDD), www.gisdatadepot.com/dem/.

What does this change mean? First, the good news: WCS 5.54 and MicroDEM can handle the New Format SDTS (not all programs can yet). Second, hang on to your Old Format SDTS DEMs and keep them separate; you'll likely find a need for one or another in the future. Third, finding what you want can be a chore as the new files are organized not only by state, but also by county. Fourth, if you thought the Old Format naming convention required something to make it more meaningful, you'll find the naming convention GISDD has assigned the New Format downloads to be completely incomprehensible.

New Format Tips

1. Run, don't walk to the 3D Nature Web site, www.3dnature.com, and download Frank Weed's free DOS command line program, SDTS_Rename. Place it and the DEM's .tar.gz and text files (or a whole series of .tar.gz and .txt files) in the same directory, and execute from a DOS window. In a flash your files will go from being incomprehensibly identified with something like 4632_1456655_DEM_SDTS.TAR.GZ and .TXT to something like 30_1_1m_Newcastle_WY.tar.gz and .txt. Quite an improvement.

2. As noted, GISDD has the New Format DEMs organized by state and county. There is no alphabetical quad listing, as was the case with the USGS site. To ease the pain, I recommend that you bookmark the USGS Geographic Names Information System (GNIS) query page and become familiar with how it works. Better than aspirin. geonames.usgs.gov/pls/gnis/web_query.gnis_web_query_form

3. Yes, WCS 5.54 and MDEM will read the New Format DEMs, but these DEMs are available in horizontal resolutions of 10m or 30m, or both, and their elevations may be in meters, decimeters, or feet. So far I've not experienced difficulty with the elevations in WCS or MDEM; however, if you want to merge in WCS and you find your area is composed of mostly 30m, but one or two are only available as 10m, then hopefully you've saved your Old Format and have a 30m to replace the 10m. WCS will not merge DEMs of different resolutions.

MDEM won't directly merge mixed resolution DEMs either, but, if you don't have an Old Format 30m, you can beat the problem and merge DEMs by using MDEM's DTED export. See the section on merging 10- and 30-meter DEMs at "MicroDEM for Artists."

More About those Damned Datums & Other Things

"DEM o'Graphics Potpourii, Part3: World Construction Set, MicroDEM, Leveller, Photoshop & those damned Datums" in 3D Artist #45 explored creating geographic DEMs with datums other than NAD27 for use in World Construction Set 5.5x. Those techniques still hold, but creative and imaginative programmers do not stand still and Professor Peter Guth of the US Naval Academy has added new capabilities to MicroDEM that can make 3D artists' lives easier and extend their reach. Let's take a look at a few cases:

Transforming UTM DEMs into geographic Latitude/Longitude
MDEM's DTED export will transform single or merged UTM-based DEMs to geographic Lat/Long-based DEMs with a WGS84 datum. The resulting DT1 DEM can then be saved in the native MDEM format with another datum and imported into WCS as outlined in 3DA#45 using Leveller (LEV). If you are interested in a subset area (and subsetting is always recommended), it is necessary (in order to keep things squared-up) to take your larger area and convert it to DTED before subsetting to the native MDEM format and the appropriate datum. [Note: Unfortunately, WCS will not currently import the MDEM DT1 format as it does other DTED files.]

Mixing UTM/geographic Shapefiles of differing datums
What to do if your available vector data is a mix of Lat/Long- and UTM-based Shapefiles with differing datums? The focus in the 3DA#45 how-to is on changing a DEM's datum to match a Shapefile's datum, but, when your Shapefile bag is a mixed one of different projections and datums, that's another kettle-of-fish, as the DEM's datum and the vector file's datum must be the same for an accurate match. Again, it's MDEM to the rescue:
  1. If your Shapefiles are Lat/Long-based, you can do a Shapefile datum shift in Data Manipulation/Resample/Shapefile/Datum Shift to resample the file from one datum to one matching your DEM's datum.
  2. If some of the Shapefiles are UTM-based and in a datum different than the DEM's datum, use first the UTM to Lat/Long option in Resample/Shapefile, and then resample the resulting Lat/Long file to the desired datum with Shapefile/DEM Shift.

When your program won't load Shapefiles
What to do if you want to digitize in MDEM, but your program won't load Shapefiles: MDEM also provides the opportunity to digitize in DXF by selecting the Map Annotation icon in the open DEM/Image window(s). However, to digitize on a georeferenced image, you must load the comparable DEM first and then the image. Your choice of UTM or geographic Lat/Long coordinates can be made in Options/ImportExport, where you check or uncheck the DXF UTM export box.

DEM resolution enhancement
Changing a DEM's resolution using DTED export works best if 7.5 minute DEMs are Level 2. (See "MicroDEM for Artists," particularly the section on merging 10- and 30-meter DEMs.) Enhancement can be useful with Level 1 DEMs, but, if they are among those suffering a banding problem, enhancement will likely make the problem worse. You might want to take a look at using the DTED export resolution enhancement with small scale 1km DEMs. I've found it to be particularly useful with the Globe format small scale (1km) DEMs that MDEM now imports. It's worth taking the time to experiment to see what can be achieved.

The image below is a comparison of a Globe of the Black Hills of Wyoming and South Dakota with its native 30 arc second spacing and enhanced to 15 arc second spacing using MDEM's DTED export option. A vertical exaggeration of 5 was applied for the image capture.

The Black Hills in original 30 arc second spacing (left) and enhanced to 15 arc second using MicroDEM.
[see image at correct size in new window]

Other DEM Data Sets

There are two sets of DEM data you may not be aware of, but should be--the USGS National Elevation Data (NED) at edcnts12.cr.usgs.gov/ned/default.htm and the Global Land One-km Base Elevation (GLOBE) Project data at www.ngdc.noaa.gov/seg/topo/globe.shtml. MDEM will import both data sets, and WCS 5.54 the NED.

NED - National Elevation Database

NED is not organized by quads as the SDTS data is. Instead, it is available as 30-meter user-selected seamless areas. It represents the best 7.5-minute data USGS has available with one qualification--when the area contains 10-meter data, that data is resampled to 30-meters. You'll also happily note using it that the banded grain which afflicted many of the Level 1 SDTS DEMs is seriously reduced, if not gone.

Unlike the SDTS 7.5-minute data and its UTM NAD27 coordinate system, NED is Lat/Long NAD83 based, and is available in three different formats--ARC Grid, Grid Float, and BIL-Integer. MDEM and WCS 5.54 import both the Grid Float and the Integer. Areas less than 10Mb in size can be ordered free for later ftp download. Time from ordering to download notification time, from my experience, is in the neighborhood of a day. Ten megabytes and beyond are fee-based, and the data will be shipped on a CD.

NED Tips
  1. BIL-Integer data files are smaller than the Grid Float, and you'll get more area for your free 9.9999999Mb download. You also have the choice of elevation as meters, decimeters, or centimeters.
  2. When you go to the NED Store, you have two options--"Use the Map Interface" or "Enter Coordinates by Hand." I prefer the latter and, to set things up and make it easy to find the coordinates, I've merged in MDEM the SDTS for my general area of interest (63 quads covering an area somewhat larger than my county boundaries), exported it to Lat/Long NAD84 using the DTED export, and finally resaving in the MDEM format as NAD83. From that reference DEM, I find the coordinates for the area of a specific project by subsetting in the DEM's window and using Calculate/Map Window Corners.
  3. If you are looking for just a specific quad or quads and don't know the their geographic extents, use TopoZone.com's free place name search capability to find the extents under Quad Information: www.topozone.com.

GLOBE Data

Simply put, Globe data is a cooperative, international effort to use the best available data of the Earth's surface from multiple sources in a consistent manner. Like DTED, the data is geographic Lat/Long and WGS84. I prefer it to GTOPO30 and DTED Level 0, and I find it easier to work with Globe's large tiles and subset than it is to download, store, and work with DTED folders for a comparable area. To import it into MDEM, you will also need to download the Geo-Vu header as well as the elevation data. (Note: NIMA's 1x1 degree DTED tile can now be group selected for download at 164.214.2.59/geospatial/digital_products.htm.)

Small-scale data
If you have a frequent need to use GTOPO30, DTED, or Globe data, I recommend that you take a look at obtaining the reasonably priced CDs for theses data sets at members.bellatlantic.net/~pguth/ptbr.htm. Each data set is on a single CD and works with a simple graphical selection process using MDEM. It doesn't take long to have your area of interest selected and saved MDEM native. The only limitation is MDEM's 7201x7201 column and row limit. To give you an idea of what that limitation means, a graphical selection from the Globe CD of the lower 48 states checked in at 7081x3001.

Click here to 'frame up' the full DEM o'Graphics series and also find out WHAT's NEW!
'Frame up' for more + What's New 		> DEM o'Graphics - 3DA#37
> DEM o'Graphics Potpourri, Part 1 - 3DA#39
> DEM o'Graphics Potpourri, Part 2 - 3DA#40
> DEM o'Graphics Potpourri, Part 3 - 3DA#45
> Quick 3D Terrains for Illustrators - 3DA#39
> Run, Don't Walk - 3DA#42 full text

[3DA home page] 3DA#45 [back issues] [subscription form]
Revised: 18 Aug 01
http://www.3dartist.com/3dao/r/thomsnor/tndems3.htm
© Copyright 2001 Columbine, Inc. - All Rights Reserved
3D ARTIST magazine / Columbine, Inc. - P.O. Box 4787, Santa Fe, NM 87502 USA
505/424-8945 voice - 505/424-8946 fax - E-mail: webmaster@3dartist.com
Any mentioned trademarks are the property of their respective owners.
Any prices given are U.S. list (no S&H) in U.S. dollars, subject to change.