MicroDEM for Artists

By Bill Allen

MicroDEM is a freeware mapping program written and published by Prof. Peter Guth, Professor of Oceanography at the U.S. Naval Academy. This page is an independent appreciation of that effort, aimed at helping non-GIS users, especially 3D graphics artists, to get up to speed faster. For more about the program and where to get it, see "What is MicroDEM?" For advanced MicroDEM how-to articles, read the DEM o'Graphics series in 3D Artist magazine.

Contents Grayscale Maps for terrain export Merging 10 & 30m DEMs Pointers & Clarifications Glossary - terms used on this page What is MicroDEM? - where to get it About This Page - what's new

GIS for Mac This page is about MicroDEM, which is available only for MS-Win95+/NT+. References to World Construction Set on this and related pages of course also apply to the WCS Mac version. Merging topos in Photoshop - Mac/Win Tar what? - includes Mac .tar.gz info

Grayscale Maps

MicroDEM offers an easy way to turn DEMs into grayscale files that can be used as heightfield/displacement maps in many 3D applications that can't load some types of DEMs or any DEMs at all.

1. Open your DEM with File/Open DEM, do Modify/Map area/1:1 view, right-click on the view, and use Display Parameter/Elevation. (If your copy of MD doesn't have 1:1 view when looking at DEMs and images using UTM coordinates, it's time to upgrade.)
2. Right-click again, and use Elevation colors/Display colors: Gray scale.
3. You can right-click once more and use Save image as a BMP, but the better course is to use the File menu and either Save map as GEOTIFF or Save map with world file. Both of these result in geo-referencing that is of use to MicroDEM and some destination programs when loading the file.

There can be a couple of problems here, the biggest of which you just discovered if you got an "Out of resources" error. See "Views, unable to get 100% view." The other possible problem is if your application requires 16-bit grayscale and you don't have a way to convert to that from the 24-bit RGB file MicroDEM saves (it's a color BMP, but all the colors are grays).

Note: Ignore File/Grayscale map in the main menu for the purposes discussed here. Also, while, yes, they are all quite gray toned, this section is not referring to DEMs viewed with reflectance (right-click then Display/Parameter/Reflectance) or seen in MicroDEM's OpenGL viewer with Lighting checked. "Grayscale" here refers to a rather visually unattractive stair-stepping of grays according to elevation.

Merging 10 & 30m DEMs

If you are fortunate enough to have some 10m SDTS DEMs for your region of interest, you still may not have a complete set. So what do you do if you want to merge DEMs? You could just work with the lower-res and far more common 30m DEMs. However, once you've seen 10m data, you will want to relegate 30m files to LOD duty. The solution, originated by MicroDEM's author, Prof. Peter Guth, is to convert the 10m and 30m DEMs both to 9m DTED files and then merge them together.

Note: 1:24000-scale, 7.5-min. DEMs using UTM coordinates cannot be merged across UTM zones. In 3D Artist issue #43, Thomsen-Norre details how to get around this restriction by using merging and DTED conversion methods much like shown here below for merging 10 and 30m DEMs. (3DA#43 also has an article about merging 10 and 30m DEMs, derived from this page.)

The example we will use is in one of my own regions of interest, where I had set out to do a fly-around of Mount Ellsworth on the northwest side of Lake Powell, with its peak at almost the center of three 10m DEMs. How to incorporate the corresponding 30m DEM to fill that empty quad? You can follow the steps with your own set of DEMs, or use the same DEMs we did: From the DEMCon CD (look under 100+ Maps for CanyonLands) or from the following four USGS locations, grab all the available 30m Level 2 (30.2...) and 10m DEMs for...

This screen shot shows an example of a USGS location for 1:24000-scale 7.5-min. DEMs available online for free download, in this case for the Ant Knoll quad in southeastern Utah. The folder contains both the typical Level 1 and Level 2 (improved) 30-meter SDTS DEMs, as well as a 10-meter SDTS DEM, which is not common for most of the country. Note that the file names do not reveal what they contain. Although there are two better DEM choices here, it could be that the older, lower-quality 30m Level 1 DEM will merge better with adjacent quads that are available only as Level 1 DEMs.

In MicroDEM, first make sure you have Options/Views/Missing values set to sea level unchecked. Select File/Data Manipulation to bring up the Data Manipulator with its Merge/DEMs command to merge the four 30m DEMs into a new DEM. Then repeat the steps to merge the three 10m DEMs into another new DEM.

The report from a completed merge of four 30m SDTS DEMs in their usual .tar.gz form, saved out in the MicroDEM native .dem format.

In each merge, you will be prompted for file names until you cancel, then prompted for a name for the new DEM. Following that, there will be some screen activity and confirmations, but the merged image will appear only briefly. You can close the Data Manipulator to inspect a file in the main MicroDEM interface, or you can move right on to the next steps.

Now convert each of the new DEMs into a 0.3-arc-sec. DTED file using the Data Manipulator's Export/DEM in DTED format command. You will be prompted for the file name and then it's lat(itude) and long(itude) intervals. Respond in each case with "3," for 3 x 0.1 = 0.3 secs. (If this dialog says only, "Input an integer," you need to upgrade.) MicroDEM will append a .DT1 file name extension to the new file (but make sure the file gets a new name, because, if you accept the name MicroDEM offers you, the program isn't hooked up yet to let you back out gracefully if you realize you've made a mistake).

When working with these DTED files, if prompted for "Options for very large DEM," respond with Open DEM: Read entire DEM. You most likely don't want to do thinning, at least not yet.

Now do one more merge, this time with the two DTEDs. By using a full-coverage 30m merge, you make sure to fill in most of any gaps that may exist between shared 30m and 10m DEM borders (gaps certainly exist for these particular DEMs). And, by putting the originally-30m DTED into the merge first, and putting the originally-10m last, you make sure that the 10m data will take precedence wherever there is an overlap. Let's say that again: Put the best data into the merge last.

You will probably notice that this merge has been saved as a .dem file. This is MicroDEM's own unique native DEM file format, one that is supported so far only by Daylon Leveller and Marvin Landis's mdem2stm utility for use with his LightWave plug-in, DEMScape. Since Leveller also supports World Construction Set's .elev file format, going through Leveller is the current route to get a file like this into WCS, with reference to geographic and elevation extents information noted from MicroDEM. For other programs, such as Bryce, World Builder, and most 3D applications, going to grayscale may be the only route. Regardless of the path, Thomsen-Norre reports that the export will work best if you subset the merged DEM to exclude the areas along the edges without elevation data.

Important! With saving to the MicroDEM .dem format, the datum used for the new file is what's set under Options/Maps/Primary Datum. See "Datum notes."

How big an area can you assemble? There are limits as to how many rows and columns of data MicroDEM can handle, as noted under Help/Program limits. You can pack in more 30m DEMs than 9m or 10m, but you might consider a 15m compromise. Just substitute "5" for "3" in the instructions above when prompted for lat/long intervals. Experiment with different settings to see what works best for you.

The originally 30m and 10m SDTS DEMs viewed in MicroDEM's OpenGL viewer as one big 9m file (no vertical exaggeration). The camera is looking north-northeastward across the center of the merge. Two discontinuity ditches converge on the slopes of Mt. Ellsworth. These are on the border between originally 30m and 10m data. The originally-30m data appears in the lower right quadrant. The results are so good for this kind of viewing that one might be tempted to "puff up" 30m DEMs this way even if no 10m DEMs are available!

Pointers & Clarifications

Getting started: MicroDEM has no user manual, but click on the Help file's Contents tab and read down through the list to get a good introduction to the program and to computer mapping in general. The Help file is extensive and, if you look and think hard enough and work at something, you will probably find an answer to most questions.

Try right-clicking anytime you think you should be able to do something but can't find the command. If something doesn't work right, you may be dealing with data that doesn't match the command or doesn't match other data in use. Look for clues with Analyze/Header and Analyze/File metadata.

When using the latest "debug" version of MicroDEM, you may run into new or changed features. This offers the opportunity to discover great new capabilities, but you may not find them covered in the Help file, which is only as current as your last full install.

I have bumped my noggin many times in learning to use MicroDEM, and have had the benefit of very experienced advice in getting back on course. I would like to share that here with what follows, hitting some topics that may particularly puzzle new users.

General Pointers

Tar what?: We're all used to grabbing archive files (.zip, .sit, .arj) and having to open them before we can use what's inside. Don't do that with USGS SDTS .tar.gz files! These file names can be difficult to manage, but wait until out pops more than a dozen .ddf files with obscure, non-unique naming. Fortunately for our sanity and drive space, MicroDEM works directly and transparently with .tar.gz files (and with USGS 1:250,000 90m .gz files). This is true whether the files retain their original names or were renamed by you, keeping the .tar.gz extension.

For users of some programs (e.g., Bryce, Leveller, WCS) who really do need to open SDTS .tar.gz files, WinZip can do the job on Windows, as can a combination of Stuffit and Untar for Macintosh (40.3K file download), while Unix and Linux handle .tar.gz natively. Some browsers download .tar.gz files changed to _tar.gz, and you may need to change that back before you can access the files.

PKzip, Stuffit Expander, and the WinMe compressed folder feature all can work with most or all zip-like files (.zip, .gzip, .gz, .z), but don't know what to do with .tar files.

See the faqs.org Compression FAQ for more about gzip and TAR files and utilities.

CAUTIONS:
WinZip is strongly warned against by some in the GIS community for use with .tar.gz files. My experience is that it works fine so long as you 1) turn off its Options/Configuration/TAR File Smart CR-LF Conversion setting, 2) make sure the file name extension is .tar.gz, and 3) let it have its way on creating temp files.
Expander is available free for Windows, but users may want to guard against letting it assert itself as the primary program to open .zip files, especially for WinMe, which handles .zip natively as a compressed folder (very nice). If Expander supercedes that function, right-click on some file and select Send To/Compressed Folder. Windows will complain and offer you the opportunity to give it back control.

Get organized: In the 3D Artist issue #39 installment of his "DEM o'Graphics" series, Thomsen-Norre explains an approach to organizing GIS files that involves immediately renaming files upon download with logical, readable names. My own approach uses a logical, readable folder structure where I can keep files with their original names. (That saves the renaming step, lets me keep all original files for a given quad together, and makes it easy to check whether a file I'm looking at online is one I already have.) Either way, what matters is starting and keeping to a discipline of file management. Otherwise, your hard drive can quickly become a shambles of large files that you can't use because you can't easily determine what data they contain.

DEM holes: MicroDEM has three DEM "hole filling" functions. One works on filling missing data as a DEM is being imported, controlled by Options/Import-Export/Auto fill holes on DEM merge. In the main menu, the Edit/Fill DEM Holes function is intended for filling holes in (only) the current view of a DEM. The primary repair tool is accessed by going to File/Data Manipulation/Edit/Fill DEM holes.

Subsetting: MicroDEM has two subsetting (cropping) functions. The subsetting in MicroDEM's main interface is for temporary views, which is not only very handy for looking around but vital for determining what the coordinates will be for permanently subsetting a DEM and for matching that with subsets of DRGs and DOQs. To subset permanently to a new file, use File/Data Manipulation/Subset, then exit the Data Manipulator to load the new cropped file into MicroDEM.

Note 1: In his "DEM o'Graphics" series in 3D Artist magazine, Thomsen-Norre repeatedly emphasizes the use of subsetting as a tool to prepare DEMs for more useful and easier-to-use exports to landscape programs and 3D applications.

Note 2: Your newly subset DEM will be saved with the Primary Datum, not necessarily the original DEM's datum. See the next section, "Datum notes."

Datum notes:

1. Use Analyze/Header to find out what the datum is for the DEM you are viewing.

   When you click on a DEM window's Map information icon, or use Alt-i when a DEM is the active window, everything in the Information popup is about that DEM except the Datum. The datum you see is the Primary Datum that you set (or is still at default as installed) under Options/Maps/Primary Datum. It may or may not be the datum of the DEM being viewed. This is also the case for the datum and coordinate information shown in the message bar at the bottom of the MicroDEM window.

2. When you save a new DEM, the datum used for the new file is what is set under Options/Maps/Primary Datum, not what the datum was for the source DEM(s). This is true for File/Save DEM and for the Data Manipulator's Merge, Subset, and Thin operations.

   Export is a different case. For instance, when a DEM is exported to DTED, it is always made to use the WGS84 datum and lat./long. coordinates. See the Help file's "Export Options" section.

3. As the Help file's "Datum Caution" section warns, changing the Primary Datum affects only files loaded after the change, so you may want to close any open files first.

Views open too big: Do DEMs keep opening too big? Just go into the main menu's Options dialog and change Maps/Default Size to something manageable like 480x640. (Options is where you take care of many other tasks, too, like changing initial grid display settings.)

Rebooting: MicroDEM is constantly evolving and, as such, it is beta software. Having to sometimes close and reboot the program is normal. This is true for something as benign as being unable to bring up the Data Manipulator, as well as for getting the program back to its normal state after receiving a system error message, or for recovering from trying to stop a process. In my experience, MicroDEM crashes gracefully. It almost never causes chaos to the main system that adversely affects other open programs or their data, or that requires or would recommend rebooting Windows itself. (One wishes all non-beta programs were so well behaved.)

Merge vs. Drape

MicroDEM has two kinds of "merges": One kind merges like DEMs or like images (DRG, DOQ, satellite), and the other kind of merge superimposes an image on a DEM. The latter function is called "draping" when used with MicroDEM's OpenGL, Oblique, Perspective, and Flying views. This page uses "draping" when talking about superimposing images on DEMs in any fashion, and "merging" only for combining like images or like DEMs into one new file.

Draping images onto DEMs: MicroDEM's special viewing functions such as OpenGL are straightforward in prompting you for the option of draping an associated image onto a DEM view. But how to superimpose in MicroDEM's main interface? First open a DEM with File/Open DEM. Then open the corresponding image, such as a DOQ with File/Open Image or a DRG with File/Open Scanned map. Right-click on the image (not the DEM), click on Display parameter, then on DEM elev merge. The associated DEM will now appear under the image. To adjust the appearance, right-click and select Merge colors.

Tip: In addition to draping in MicroDEM, try exporting a reflectance view to Photoshop and combining it there with a topo map or other corresponding image. This technique is explained at length in Thomsen-Norre's "Quick 3D Terrains for Illustrators" in 3D Artist issue #39.

Merging images or DEMs: Use File/Data Manipulation (or the In<->Out icon) to bring up the Data Manipulator (if this option has become unavailable, reboot MicroDEM). There, use Merge/DEMs or Merge/Imagery. You will be prompted for the successive files and, at the prompt for one file past the number you want, cancel, and answer the prompt for a file name for the new file. If you are merging DTEDs, consistently answer "Full DEM" to the thinning/subsetting prompt if one appears.

You will see some screen action and confirmations, and then you will find yourself back in the Data Manipulator without any image or DEM. Close the Data Manipulator and use the File menu's Open DEM, Open Image, or Open Scanned map to load your newly-merged creation into MicroDEM.

Also see:
"Merging 10 & 30m DEMs" about how to merge DEMs of different resolution.
"Datum notes" about how your newly merged DEMs are saved with the Primary Datum, not necessarily the original DEM's datum.

Views, unable to get 100% view

First, how to get specific zoomed views: When looking at UTM files, such as 1:24000 DRGs and 10m or 30m DEMs, you can go immediately to 100% view with Modify/Map area/1:1 view. If you don't see this main menu command, either you aren't in a UTM file, or your version of MicroDEM needs updating.

The other way to set an exact view is to use Modify/Map area/Custom image size and type in the width and height. Press the info icon on the active map window (or Alt-i) and note from the Information popup the "<width> by <height>" from the "DEM points" or "Image points on map" line. Then manually enter the numbers into Custom image size (Alt-m-m-c).

That's straightforward with UTM files, but, with latitude/longitude files such as DTEDs, MicroDEM will take one of your dimensions and make its own decision on the other. That's because the data array, unlike your screen pixel array, is not square.

Of course, you also can zoom in on a view using the spyglass zoom-plus icon, or use Modify/Map area/Zoom In or the keyboard shortcut (Alt-m-m-i).

Now the problem, part 1: If you are using Windows 95/98/Me, you may find that you cannot go to a 1:1, 100% view of a DEM or DRG without getting an "Out of system resources" error, no matter how much RAM and VRAM you have. This is a problem basic to old DOS-based Windows, something that appears to have been remedied somewhat but not completely in WinMe. It is reportedly not a problem under WinNT4/2000, however that may not be the case, as some WinNT users have experienced it.

The problem, part 2, is that MicroDEM saves an image at that image's currently displayed resolution. If you can't get an image to a 100% view, then you simply cannot output it to a raster image file at full resolution.

What's the solution? There isn't a sure one, but you can experiment with your video settings. Changing 32-bit to 24- or 16-bit display appears to help, then experiment with Custom image size or zoom to discover what the largest view is that you can reach before crashing MicroDEM, and then live within that constraint. Also, remember that you can subset the view or file, which may be all you need to get the results you want.

[the problem]

Glossary

collar

The white margin around a DRG topo map that includes "marginal information" which can be quite useful, such as the map's date and rotation, and the names of adjoining maps (occasionally wrong on older DRGs).

datum

An Earth-shape (ellipsoid) reference model upon which points on the Earth's surface are calculated. For instance, NAD27 is common for UTM 1:24000 DRGs and 30m and 10m DEMs, though corresponding DOQs may be in a different datum, such as NAD83. DTEDs always use WGS84. Draping data from one datum, such as images or DLGs, onto DEMs in another datum, will result in noticeable misalignments.

DDF

Data Descriptive File - This is the file name extension used for the multiple files stored in a single USGS SDTS .tar.gz archive for a DEM or DLG.

DEM

digital elevation model - This term is used very generally to describe a great variety of terrain files and file formats. There are also a number of mutually incompatible file formats that use the .dem file name extension, including a format native to MicroDEM itself.

DLG

Digital Line Graph - A USGS format, with several variations, for descriptions of individual data types such as transportation (e.g., roads) and hydrography (streams, lake shores).

DOQ

digital orthophoto quadrangle - Aerial photography of the Earth's surface, usually provided in GeoTIFF form, but also seen in other formats. A DOQ for a 7.5-min., 1:24000-scale quad is huge, thus the common use of DOQQs--digital orthophoto quarter quadrangles.

DRG

Digital Raster Graphic - A scanned or digitally-created topographic map just like you may have taken hiking. These usually are indexed-color GeoTIFF files.

DTED

Digital Terrain Elevation Data - A DEM format specified by NIMA.

GeoTIFF

A special expanded version of the TIFF6 standard used for various purposes, especially for DRGs, that provides for embedding topographic information. Embedded geo-referencing removes the need for a companion world file, though one may be provided for compatibility.

GIS

geographic information systems

grayscale

Grayscale "raster" files come in two flavors: 8-bit and 16-bit, where possible value ranges can be from 0 (black) to, respectively, 255 or 65,535 (white).

LOD

level of detail - To decrease memory impact and to speed rendering, a terrain or other object in a 3D scene may be represented by substitutes of different detail according to proximity to the camera - low detail when far away and high detail when seen close.

MGRS

Military Grid Reference System - This is a U.S. military map coordinate system based on the UTM and UPS (Universal Polar Stereographic) grid systems.

NAD27

North American Datum 1927, CONUS - Also called NAS-C, this datum is being superceded by NAD83 and WGS84, but is still very commonly found used for USGS 1:24000 DRGs and DEMs.

NAD83

North American Datum 1983, CONUS - Also called NAR-C.

NIMA

National Imagery & Mapping Agency - This was formerly the DMA, Defense Mapping Agency.

SDTS

Spatial Data Transfer Standard - A USGS format used for data such as 1:24000 DEMs and DLGs. An SDTS "transfer" consists of multiple files using the .ddf file name extension that usually are packed together into a single .tar.gz file.

USGS

U.S. Geological Survey - www.usgs.gov

UTM

Universal Transverse Mercator - A rectangular metric mapping coordinate system used instead of latitude and longitude.

To see a map showing UTM grid zones in MicroDEM, first make sure that you have Options/Menus/Cartography checked, then use File/New Vector map and Cartography/MGRS Grid Zones. Use the Subset (crop) icon in the Map Selection Window to focus on your area of interest.

Another UTM zone map: www.dmap.co.uk/utmworld.htm.

WGS84

World Geodetic System 1984 - A worldwide datum.

world file

A map or image (DRG, DOQ, etc.) companion file that includes basic plain-text information about location and scale (geo-referencing). World files share the image file name but use a different extension. For a TIFF, the companion world file uses the extension, .tfw. World files for JPEG and BMP files use .jpw and .bpw.

MicroDEM sometimes creates and uses an associated additional plain-text "datum file" with extension .bdw for BMPs and .tdw for TIFFs. You also may find .met metadata files that contain even more information.

What is MicroDEM?

MicroDEM is created and published by Dr. Peter Guth, Professor of Oceanography at the U.S. Naval Academy. It is a scientific, military, and educational program for MS-Windows that deals with computer mapping using a variety of digital elevation data and image types. If you want to move realworld geographical data into your 3D graphics work, whether for map illustration or scene creation, MicroDEM is often your first step, and may even be the only avenue for getting that data into the form needed by whatever landscape program or general 3D application you are using.

MicroDEM is only available for Windows 95/98/Me/NT4/2000. The install includes two optional sub-versions, MGT (Marine Geophysics Trainer), and PAPA, a pipeline planning program in early development. Also included now is a library of TrueType military map symbols that wargamers and news map illustrators may find useful. TerraBase II is the full military version of MicroDEM supported by the U.S. Army Engineer School at Fort Leonard Wood.

Underwriting for MicroDEM/TerraBase II has come from many government and military agencies, as you will see in the Help file's "Credits" section. MicroDEM is available to the public for free download without warranty or active support--what you see is what you get, including an extensive Help file but no documentation that holds your hand like comes with commercial software. MicroDEM also can be received on a $20 CD-ROM with the professor's DEMCon DEM conversion utility and a very large variety of DEMs.

• MicroDEM news - www.usna.edu/Users/oceano/pguth/website/microdemdown.htm
• MicroDEM download - www.usna.edu/Users/oceano/pguth/microdem/win32/
  Microdem_debug.exe is the latest beta version. If you don't yet have the full 26Mb install, you need to get that, then come back for the frequently-improved debug version.
• DEMCon & data CDs - members.bellatlantic.net/~pguth/ptbr.htm

About This Page

Thanks to MicroDEM's author, Prof. Peter Guth, for MicroDEM itself, for providing guidance on some of the points explored on this page, and especially for the DEM-to-DTED technique he originated for merging 10m and 30m DEMs. And many thanks to the pseudonymous Thomsen-Norre for so much help with understanding MicroDEM. I urge you to read and re-read his series in 3D Artist. I'm still learning from them.

All the credit for any mistakes or problems on this page goes to this author. Being very busy people, please don't ask for personal support on using MicroDEM, working with DEMs, or other GIS/landscape topics. However, if you find a mistake on this page, please do let me know. Questions will be read, but may not be answered immediately if at all. Any answer most likely will appear for general consumption on this page or the DEM o'Graphics pages.

Page history:
23 Feb. 2001: New workaround for saving images at 1:1 full resolution.
19 Jan. 2001: New supplemental "Merging DRGs Manually" page.
12 Jan. 2001: Note added about merging across UTM zones.
15 Dec. 2000: This page was first publicly posted.

Version: Some aspects of MicroDEM discussed on this page apply only to the latest versions of the program. This page was prepared with the full MicroDEM setup version 12.35.30.4 of 30 Oct. 2000, and is correct for debug versions no older than 12.43.7.2 of 7 Dec. 2000. (Thomsen-Norre's article about merging DEMs across UTM zones article in 3DA#43 requires version 12.45.25.1 or later, which also should work slightly better for merging 10 and 30m DEMs.) More recent setup and debug versions are available at www.usna.edu/Users/oceano/pguth/microdem/win32/.