HowTo:MAPS

From XastirWiki

[edit] About this page

This page is a subset of the information contained in the file README.MAPS of the Xastir distribution. It is not kept rigidly in sync with that file, nor does it contain all of the information in that file.

To find the real README.MAPS, please look in the source directory for Xastir, or in the installation directory ${prefix}/share/doc/xastir where "${prefix}" is usually /usr/local, but could also be /usr if you installed xastir from a binary package.

[edit] Maps for the impatient

If you compiled with ImageMagick and have wget or libcurl installed, you can use the online Tiger/Terraserver maps exclusively. However, having maps on your computer is often faster than transferring images over the Internet, and is not subject to network failures. See the section below on map caching for an exciting feature that works with most online maps to really speed things up!

[edit] Introduction

[edit] What do you need?

Xastir isn't of much use unless you can display data in relation to geodetic information of the Earth. The developers have gone out of their way to provide the ability to use many map formats, as well as ways to make your own maps for use within Xastir. Fortunately, Xastir comes out of the box with the ability to display many map formats. Below is a list of the many types of map formats and the associated library packages needed to view them.

  Map Type:           Libraries Required/Notes:
  ---------------     -------------------------
  DosAPRS             Built-in.
  WinAPRS             Built-in.
  X-APRS              Built-in.
  MacAPRS             Built-in.
  PocketAPRS          Built-in.
  USGS GNIS           Built-in.  Can split into county-sized chunks using xastir/scripts/split_gnis.pl to speed things up.
  Address Lookup      Built-in.
  Weather Alerts      Shapelib
  pre-2003 Tigermaps  Shapelib
  post-2003 Tigermaps Shapelib, pcre
  ESRI Shapefiles     Shapelib
  Image Maps          ImageMagick.  Often need lcms library and others as well (whatever it takes to make ImageMagick
                      happy).  Can also use the XPM library for some image types without installing ImageMagick.
  Internet Maps       ImageMagick plus wget or libcurl.  Often need lcms library and others as well (whatever it
                      takes to make ImageMagick happy).
  UI-View Maps        Convert from .INF to .GEO format using xastir/scripts/inf2geo.pl
  OziExplorer Maps    Convert some maps to .GEO format using xastir/scripts/ozi2geo.pl
  USGS DRG Topo       libtiff, libtiff-devel, libproj, libgeotiff
  USGS SDTS           GDAL/OGR.  Preliminary. 
  MapInfo             GDAL/OGR.  Preliminary.
  Tiger/Line Raw      GDAL/OGR.  Preliminary.
  Microstation DGN    GDAL/OGR.  Preliminary.
  S57                 GDAL/OGR.  Preliminary.

The recommended configuration of Xastir allows to use of many of these map formats.

[edit] U.S. Users

  Minimum:  Shapelib, pcre
  
     Allows use of all built-in map types plus 2003 (and later) Tigermap data, shapefile weather alerts, and local ESRI Shapefile format
     maps, including U.S. satellite/image/topo maps via Internet.

  Medium:  Shapelib, pcre, lcms, ImageMagick, libcurl/wget,
  
     Allows use of all the above plus Internet maps and local image maps.

  Maximum:  Shapelib, pcre, lcms, ImageMagick, libcurl/wget, libtif, libproj, libgeotiff
     
     Allows use of all the above plus USGS topo maps.

  Bleeding edge:  GDAL/OGR
  
     Adds more map types, but they're not fully supported yet.  You can certainly get some use out of them though, like overlaying
     SDTS contour vectors on top of terraserver satellite images.

[edit] Rest of World

  Minimum:  lcms, ImageMagick, libcurl/wget
  
     Allows use of all built-in map types plus local and Internet image maps, including Canadian topo maps via Internet.

  Medium:  lcms, ImageMagick, libcurl/wget, Shapelib, pcre
  
     Allows use of all the above plus ESRI Shapefile maps.

  Maximum:  libproj, libtiff, libgeotiff, GDAL/OGR
  
     Adds more map types.  Some of these may not be useful in your part of the world.  GDAL/OGR stuff is not fully supported yet.

[edit] The documentation

[edit] Map directory hiearchy

You can have any number of maps in the /usr/local/share/xastir/maps directory. You can organize the maps however you like. You can also use symbolic links to link to files/directories on other disks. Map files are loaded in the order that they appear in the chooser unless you adjust the layering priorities in the "Properties" dialog (it's recommended that you use these now instead of a directory hierarchy to choose the layering).

There are many methods for organizing many maps. Create a map hierarchy using something that makes sense to you. The old method of creating transparent/filled or raster/vector directories has been superceded by the new map layering features in the Map Chooser->Properties dialog. It's now suggested that the map layering be done there, and the directory layout designed to make it easiest for the user to select maps. Perhaps a good start would be:

    World/
    Canada/
    Canada/Province/
    USA/
    USA/State/WA/
    USA/State/WA/County/
    ...
    GNIS/
    Overlay/

Please note that the map directories are entirely up to the user now. Use the map layering facilities (Map Chooser->Properties) to determine the order in which they will be drawn, and to determine which will draw filled areas. Note that raster maps will always draw filled, it doesn't matter what the setting suggests. For vector maps, you have a choice of:

   Fill = No       Polygons will never be filled
   Fill = Yes      Polygons will always be filled
   Fill = Auto     Polygon fill is determined by map (or if it's a Shapefile, by any associated dbfawk file).

[edit] Map types by file extension

Vector Format:

   .shp/.shx/.dbf  Shapefile vector map (need all three files)
   .pdb            PocketAPRS vector map
   .map            APRSdos/WinAPRS/MacAPRS vector map
   .gnis           GNIS labels file (actually points instead of vectors)
   .geo            Vector map or Internet vector map.  Note that for http/ftp-based maps Xastir requires an IMAGESIZE line.

Raster Format:

   .tif/.fgd       geoTIFF raster image map
   .geo            Raster image map or Internet raster image map.  Note that for http/ftp-based maps Xastir requires an IMAGESIZE line.

Note: ".geo" is listed twice because it can fit in both categories, depending on the base format of the map file that the .geo file points to. .geo files are handled by the XPM library or by ImageMagick, so most anything that can be handled by those installed libraries can by handled by Xastir.

[edit] Where to find maps

[edit] PocketAPRS vector maps

The WinAPRS 2.51 distribution included the full collection of PocketAPRS vector maps for the USA. The download is about 60MB, from the TAPR ftp site.

Also try ftp://ftp.tapr.org/aprssig/palmstuff/palmmaps/

[edit] Raster Maps

For Canada theres a country wide set of topographic map images available from the "Department of Natural Resources Geomatics Canada". Running /scripts/toporama250k.pl will pull down 430mb of 1:250k map images from that departments site. If thats not enough data, running /scripts/toporama50k.pl will pull down around 10gigs.

For Ireland, EI8IC (http://www.mapability.com) has produced a set of overlay maps that show the County Borders, Main Towns and Roads. John, EI7IG, has created a set of ".geo" files to go with them and they are all available in one tarball from ireland.aprs2.net They are free for non-commercial use.

Also available are a map of the country, with an associated geo file, the South East, it's associated geo and a Weather Radar .geo file.

[edit] Shapefile format maps (Requires shapefile support)

Shapefile format maps are slowly becoming the standard vector format of maps used with Xastir. Xastir 1.3.2 and above by default enable dbfawk support for parsing shapefile metadata, see below.

A nice world map is available at http://aprsworld.net/gisdata/world/ thanks to James Jefferson, TerraSpace, Russia, and the Digital Chart of the World (for Antarctica). Note that this file is currently available only in uncompressed or in tar/bzip2 format. To decompress the latter file, you'd type "bunzip2 filename" and then "tar xvf filename" (or tar xvfj <filename>).

[edit] TIGER/Line shapefiles

Tiger/Line 2006 data converted to shapefile format is available from ftp://aprs.tamu.edu/. Look in the "pub/TIGER_2006_SE/" directory. These maps only work with dbfawk support.

Unpack the zipfiles in your maps directory. I recommend creating a "TIGER_2006_SE" subdirectory or something similar, and unpacking the zipfile there. When unpacked, each state zipfile will produce a directory with the state's postal abbreviation as its name, and two subdirectories, "Polylines" and "Polygons." These directories each contain shapefiles with different information in them --- Polylines are linear features such as roads and streams, Polygons are area features such as parks and airports. You need both, even though they have the same names (the name of the county they cover).

You can also create your own TIGER/Line shapefiles from the raw data at the Census web site, http://www.census.gov/geo/www/tiger/index.html. The process requires two conversions: converting the linear features, and assembling the polygon features. The TIGER/Line data is a topological format, and polygons are not stored in a simple manner; this second step is time consuming.

To convert the polylines: Download the county files you want. Then unzip each one into its own directory (call it "in_directory"). Then cd to in_directory and run the following command:

     ogr2ogr -f "ESRI Shapefile" -t_srs EPSG:4326 foo .

This will create a subdirectory of in_directory called "foo" which will contain a number of shapefiles and auxiliary data. The files called "CompleteChain.*" are the ones you want; they are the polylines corresponding to roads and other linear features. Copy these to your xastir maps directory and rename them to the name of the county they represent. Then install tgr2shp.dbfawk in your share/xastir/config directory and reindex maps. Ta da. See above: tgr2shp.dbfawk is now part of the xastir distribution..."

Note that you'll need dbfawk support compiled into Xastir to make these Tiger/Line maps look nice. For the 2003 TIGER/Line data, Derrick Breshear created scripts to process all of the Tiger/Line raw data into Shapefiles, then they were put on a download site made available by Kevin Glueck, "http://xastir.tamu.edu/". The 2006 Second Edition data were processed by Jason Winningham and made available on the ftp://aprs.tamu.edu/ site by Gerry Creager. Thanks go to Tom, Derrick, Kevin, Jason and Gerry for making these maps so accessible to the rest of us.

The second step to rolling your own TIGER/Line shapefiles is to assemble the polygons. This requires GDAL/OGR configured with python support, and requires that you have the Python language installed. Use the script Xastir_tigerpoly.py included in the "scripts" directory of Xastir's source tree.

As an example, if you have un-zipped the file TGR35001.zip (the TIGER/Line file for Bernalillo County, NM) into a directory called TGR35001, then

     Xastir_tigerpoly.py TGR35001 Bernalillo_County_NM_Polys.shp

would create a shapefile of TIGER polygons (water bodies, political boundaries, parks, wilderness areas) that you can install in your maps directory and read directly using normal shapefile support.

Viewing these shapefiles as filled polygons requires dbfawk support, and the rules for displaying them are contained in the file tgr2shppoly.dbfawk.

Tiger/Line 2000 map data is available from http://www.esri.com/data/download/census2000_tigerline/index.html but requires free registration. A example of these types of files can be found at "http://wetnet.net/~we7u/xastir/maps/shapefile/USA/WA/". You'll need to run "tar xzvf <filename>" on each one of the files from that last site after downloading. Some layers that may be of interest:

  CMSA/MSA Polygons 2000              tgr*msa00.shp
  PMSA Polygons 2000                  tgr*pms00.shp
  Consolidated Cities                 tgr*city.shp
  County - Current                    tgr*cty.shp
  County 2000                         tgr*cty00.shp
  Designated Places 2000              tgr*plc00.shp
  Designated Places Current           tgr*plccu.shp
  Key Geographic Locations            tgr*kgl.shp
  Landmark Points                     tgr*lpt.shp
  Landmark Polygons                   tgr*lpy.shp
  Line Features - Hydrography         tgr*lkH.shp
  Line Features - Landmarks           tgr*lkD.shp
  Line Features - Misc. Transport     tgr*lkC.shp
  Line Features - Non-visible         tgr*lkF.shp
  Line Features - Physical            tgr*lkE.shp
  Line Features - Rails               tgr*lkB.shp
  Line Features - Roads               tgr*lkA.shp
  Line Features - Unknown             tgr*lkX.shp
  School Districts - Elementary       tgr*elm.shp
  School Districts - Middle           tgr*mid.shp
  School Districts - Secondary        tgr*sec.shp
  School Districts - Unified          tgr*uni.shp
  Traffic Analysis Zones              tgr*taz.shp
  Urban Areas                         tgr*urb.shp
  Water Polygons                      tgr*wat.shp
  Alternate Feature Names             tgr*alt.shp
  Address Matching Info               tgr*add2.shp
  ZIP+4 Left and Right Info           tgr*zip.shp
  Key Geographic Location Addresses   tgr*add.shp
  Landmark Polygon Names              tgr*lpy2.shp
  Landmark Polygons Multi-Landmark    tgr*lpy3.shp
  Water Polygons Multi-Names          tgr*wat2.shp

Older Tiger97 maps converted to shapefile download-able from: http://www.mapshots.com

Some other shapefile format maps are available at: http://www.nws.noaa.gov/geodata/

Still other shapefiles are available from esri.com, geographynetwork.com and gisdatadepot.com (from their "free data" link). Lot's of useful stuff at these sites, just look around. You can also check with your County GIS office for shapefiles of your county. You can convert them from State Plane to Lat/Long projection using instructions later in this document.

Canada street maps in Shapefile format are available at http://www.geobase.ca

If you also use GeoTIFF images, get the handy usgs_24kgrid.zip file from http://data.geocomm.com/quadindex/. This file provides a grid of all of the 7.5' maps and their names for the U.S. A nice world map is available at http://aprsworld.net/gisdata/world/ thanks to James Jefferson, TerraSpace, Russia, and the Digital Chart of the World (for Antarctica). Note that this file is currently available only in uncompressed or in tar/bzip2 format. To decompress the latter file, you'd type "bunzip2 filename" and then "tar xvf filename".

http://www.bts.gov has the "National Transportation Atlas" that you can download pieces of (at least for the larger layers) or ask for (free) CD-ROM's to be mailed to you.

[edit] Map Caching

Map caching works in conjunction with the map download routines, which fetch maps from http or ftp addresses. The map caching features use functions found in Berkeley DB 4.0 library, AKA libdb (version 4.0 or better), to create and manage a cache directory and a persistent database of information about the files in the cache directory. This directory defaults to ~/.xastir/map_cache. The files under this directory can be cleaned up or removed by hand at any time - the caching routines will simply download the maps and put new entries into the .db file.

Map files are saved with filenames that include the number of seconds since the Unix epoch. This allows the caching routines to easily determine the age of maps so that older cached maps can be deleted. This is currently (Dec 2004) controlled with a compile time maximum age setting. See map_cache.h for specifics and for brief documentation about all of the map_cache functions.

[edit] Drawing your own maps using Xastir

The final result here will be ESRI Shapefile vector maps that you can then use within Xastir. You'll need Shapelib compiled into Xastir to make use of this feature.

Turn off Object Transmit first (Interface menu).
Place an object or item at the start location of your route.
Click on the "Move" button to active move mode, then move that object around the map. Once you're done you'll have a bunch of straight lines that depict the course.
Now bring up Station Info on that object and click on the Store Track button. If you've got Shapelib compiled in, you'll get a shapefile created with the track you just drew.
After you're done you can delete the object and then turn on the Object Transmit toggle again. You'll end up transmitting the deleted final position of the object for a few hours unless you kill/restart Xastir, but that's ok. Nobody will see it on their maps 'cuz it's a deleted object.
Look in ~/.xastir/tracklogs/ for the Shapefile. Copy it to your maps directory, changing the name on the three files as you like, then create a dbfawk file for it if you want to change how it is drawn in Xastir.

Another way would be to drive the course with a GPS and then download the track via Xastir/GPSman to create a Shapefile map.

Yet another way would be to drive it with an APRS rig and then Store Track as described above. You could also drive with GPS/Xastir (without APRS) to get a very detailed track and then save your own track.

[edit] Drawing SAR search segments in Xastir

Turn on "Draw" mode (togglebutton at the top of the main window). The cursor should change to the Measure/Draw symbol.

Using the middle mouse button, click on each vertice until you are almost done describing a polygon (don't close it yet). If you don't have a middle mouse button, clicking on BOTH mouse buttons at once will do it if you have "emulate third mouse button" enabled in your X-Windows configuration.

Go to Map->Draw CAD Objects->Close Polygon. The polygon will get completed, the area enclosed will be computed, and a dialog will pop up asking you for more information. You do not have to enter any information in this dialog, but a name of some type is helpful in order to be able to identify each segment when modifying/deleting/viewing these segments.

Note that these CAD Objects get saved in a file and reloaded each time you start Xastir, but they cannot currently be written out as a map file and used as a general map.

Also note that these CAD Objects don't get transmitted. They are currently for local display only.

The display of various kinds of data related to these segments can be modified in the Map->Draw CAD Objects menu. Modifying parameters for any segment is done in the Map->Draw CAD Objects->CAD Polygons dialog.

Click on the dashed line in the menus in order to separate that menu from the rest and allow you to keep the menu on the screen. This is very useful when drawing CAD Objects or changing display/filtering options.

Turn off "Draw" mode when you're done. The cursor should return to normal.

[edit] Summary of distance/area/angle calculations

[edit] Distance

Haversine formula (2-parameter atan version) for computing distance between two points, for our general distance routines:

  calc_distance()
  calc_distance_course()
  distance_from_my_station()

Haversine formula (2-parameter atan version) in "Measure" for both the X/Y offsets and the total distance.

Haversine formula is more accurate for shorter distances than spherical trigonometry calculations. Both methods are equivalent for longer distances. Both are Great-Circle calculations (as opposed to Rhumb-line or planar geometry). See the GIS FAQ for info on these methods. Dr. Math website mentions both of these and refers people to the same GIS FAQ.

Also note that the Haversine formula is ill-conditioned if the two points are antipodal (on opposite sides of the Earth). From the GIS FAQ: "but the error, perhaps as large as 2km (1 mi), is in the context of a distance near 20,000km (12,000 mi)." For our purposes that's just fine.

[edit] Area

Area of a spherical rectangle from Dr. Math's website for the "Measure" function. There are errors in their last two derived formulas. They have been notified of it. Our implemented code is correct.

The area calculation for CAD Object Polygons is probably not spherical, but closer to planar. It might take some doing to calculate the area of irregular polygons on a spherical surface. We use Greene's Theorem to compute the area based on the vectors. The lengths of each vector are computed via the Haversine formula, so it's not strictly a planar area calculation either. Somewhere in-between planar and spherical? This should be plenty good enough for areas that are less than 12 miles on a side (that's about where planar geometry starts to diverge from spherical trigonometry), perhaps also for somewhat larger areas than this due to the advantage of computing the vector lengths with the Haversine formula.

[edit] Angle

Great Circle departure angle for the "Measure" function.
Great Circle departure angle for the calc_distance_course() function.
Dead-reckoning angle is essentially a Rhumb-line calculation (constant heading), with a few caveats:

We compute the next point along a constant angle based on the last position transmitted, the time elapsed, and the course desired. This then gets corrupted a bit either by the standard APRS grid that it has to snap to (and the position snapped to for the previous position) and/or by the relatively course angle available for Base-91 compressed positions.

Standard APRS positions give 1-degree angular resolution but 60-foot or so grid points. Base-91 Compressed positions give something like 1 to 3-foot grid points but less angular resolution (+/- 2 degrees. We can't win!

The end result is that we start off like a drunk sailer due to our short transmit times and the above limitations, then the angle smooths out as the time increases between the transmits. You can see this easily at zoom level 1 when you place an object, when you make a change to an object, or move an object (transmit rate goes up for any of these conditions).

Dead-reckoning effectively does a Rhumb-Line calculation as it is trying to do a constant heading. That means it is not the shortest distance between the points. I'm still trying to get my head around that last concept as it is counter-intuitive to someone versed in compass/maps on land and with shorter distances.

A Great-Circle route is the shortest distance between two points on a sphere, whereas a Rhumb-line is what you get if you move along a constant compass heading and is a bit longer.

We probably do not need to go to ellipsoid calculations for Xastir. Spherical calculations should be accurate enough for what we do.

HowTo:MAPS

From XastirWiki

Contents