Example: Detecting and Correcting a Wrong Projection

In this example we explore projection dialogs and a classic coordinate system problem. T  We save a drawing into projected shapefiles and then show on import how a projection can be quickly and easily checked and corrected if it is wrong. The terms projection and coordinate system are used as interchangeable synonyms in Manifold.   The more technical dialogs tend to use the term coordinate system, while in casual use we often say projection

 

This is a long example which combines several different activities.   There are five major parts to this example.

 

 

 

 

 

 

There are many illustrations to cover virtually every small step.  With only a bit of experience Manifold users will not think twice about the small steps but to help beginners all steps are illustrated here.

 

To fit into this documentation, illustrations show a small Manifold desktop, with only a few panes, docked to the right side.  In real life we use a much larger Manifold desktop, and more panes would be turned on, with panes docked to the left or to the right, or undocked, as we prefer.   Right-click a pane's tab to change where it is docked.  Manifold will remember that new arrangement for our next session.

Create example data

We start with our sample drawing that shows provinces in Mexico as areas.

 

 

 

 

To see what coordinate system the drawing uses we choose the Info pane.

 

 

 

 

The Info pane shows the Mexico drawing use the Latitude / Longitude coordinate system.

 

 

 

 

We will now export the drawing into ESRI shapefile format.  Choose File - Export to launch the Export  dialog.

 

 

We choose a File name of MexLatLon to remind us that we are exporting Mexico in Latitude / Longitude projection, and we choose SHP Files in the Save as type box.  Press the Export button.

 

 

If we use Windows Explorer to take a look at the files created we see that five files were created by the export process.   

 

An ESRI "shapefile" is actually a minimum of three files that have .shp, .dbf and .shx extensions.  Although the ESRI shapefile standard mentions no way of saving projection information, an informal standard of using a fourth, .prj, file has arisen over the years.  Use of .prj files has become so widespread that many, but not all, applications will write an accessory .prj files to specify the projection used together with the three shapefiles described by the ESRI standard.  Manifold writes a .prj file as well.

 

In addition,  Manifold creates a fifth file, the .mapmeta file, to save projection information in a highly detailed form for the use of Manifold applications. See the Accessory Files Created topic and the Shapefiles Strangely Out of Shape topic.

 

If shapefiles are written with a .prj file that follows accepted conventions, then those shapefiles will be easy to import into almost any GIS package that reads shapefiles and understands .prj conventions.   Unfortunately, not all applications write .prj files and there are millions of shapefiles floating around in the world's spatial data archives that were created without .prj files.   This topic is all about how to handle such files.   

 

Since this example is intended to show what happens when working with shapefiles that do not have a .prj attached, we will perform a quick spatial lobotomy by highlighting both the .prj file and the  .mapmeta file and then pressing the Delete key.  

 

 

Now we have a triplet of shapefiles, exactly as they might be encountered in some random download from the web with not a hint of what projection they may use.   

 

 Because we created them by exporting a drawing of Mexico in Latitude / Longitude projection, we know these shapefiles use Latitude / Longitude projection.  But without a .prj file or, for Manifold users, the .mapmeta file, there is no way for an application to know that automatically.

Change Projection of the Mexico Drawing

We will now change the projection of the Mexico drawing to some other projection.    We will then export it again to a shapefile so that we will have a second shapefile for Mexico, but using a different coordinate system than the MexLatLon shapefiles.

 

To change the projection of the Mexico drawing, in the Info pane we click on the coordinate system picker button in the main menu. and choose Reproject Component.

 

 

 

 

That launches the Reproject Component dialog.   

 

 

 

 

We click that dialog's coordinate picker button and in the resulting menu we choose the Pseudo-Mercator projection from the favorites list the menu offers.  This is the same projection that Manifold, ESRI's ArcGIS Pro, Google, Microsoft and virtually all web servers use as a default projection.

 

Two favorites are built into Manifold and we can easily add more if we like.   See the Example: Adding a Favorite Coordinate System to learn how to add a coordinate system to the Favorites system.  

 

 

 

 

That sets the Pseudo-Mercator coordinate system as the intended New system.   We press Update Component to re-project the drawing into Pseudo-Mercator.

 

 

 

 

The result is not exactly what we expected, as Mexico disappears. That happens sometimes when switching between significantly different coordinate systems, as the viewport for the drawing window may need to be reset for the new projection.  We press Zoom to Fit  to get it back.   We couild also Ctrl-click the lower Mexico layer tab to zoom to fit that layer.

 

 

 

 

That's better.   Some projections will show a visibly different display when a drawing is re-projected, but at the latitude of Mexico there is not much visual difference between a Latitude / Longitude projection of Mexico and the Pseudo-Mercator projection.

Export to a Shapefile

Once again we will export the Mexico drawing to a shapefile.  This time, what we are exporting is in a different projection, the Pseudo-Mercator projection.

 

 

 

 

As before, we choose File - Export in the main menu.

 

 

In the Export dialog we choose a FileName of MexPsMerc, intending that to remind us later on this file contains Mexico in Pseudo-Mercator projection.   We choose SHP Files as the type and press Export.

 

 

In Windows Explorer we see that five additional files have been created.   As before, we delete the .prj and the .mapmeta files to remove all projection intelligence from the shapefiles.

 

We now have two different sets of shapefiles that contain the Mexico drawing.  One of them has Mexico in Latitude / Longitude projection and the other set has Mexico in Pseudo-Mercator projection.   Neither of the shapefiles has any information stating what projection is used, other than, of course, the names we chose to use to remind ourselves which is which.

Create a Map with a Bing Satellite Base Layer

We will now create a map with a reference layer that has a "known good" projection, using the Bing satellite web server.   

 

 

 

 

We right-click onto an empty portion of the Project pane...

 

 

...and choose Create - New Map.     We could also have chosen File - Create - New Map.  

 

 

For the Base layer in the new map we choose Bing Maps Satellite from the pull down list in the Base layer box.    All of the image servers in our Favorites list will be available in that list.    The default choice offered for a Coord system is Pseudo-Mercator, or whatever other coordinate system we have set as the system default in the Tools - Options dialog.   We will leave that as is, since Pseudo-Mercator is a good choice when using image servers as base layers.  

 

The New Map dialog will list all of the components in the project that we can add to the map as layers.  In this case, that is just one component, the Mexico drawing.   We do not check the box to add it as a layer.   We want to create our new map with just one layer, the Bing satellite layer.

 

Press Create Map.

 

 

 

 

A new map appears in the project pane.  The New Map dialog has also created a Bing Maps Satellite data source in the map, to power the base layer the new map uses.

 

 

 

 

We pop open the new map.   It has only one layer, the Bing satellite layer.   

 

Tech Tip:  Bing, like most image servers, uses the same Pseudo-Mercator projection that the map window does.  If the layers a map contains use a different projection than the map, the map is happy to reproject them on the fly into whatever coordinate system the map uses.   That happens so fast in Manifold that we wouldn't notice the reprojection.  However when reprojecting a coordinate system like Pseudo-Mercator into a significantly different system like Latitude / Longitude, the warping caused by the reprojection can distort the appearance of labels or icons in image servers like Bing Streets or Google Streets.  It is therefore a good idea to use Pseudo-Mercator projection for maps that will include image server layers.

 

 

 

 

We zoom into a region that shows the Southern portion of the North American continent, about where we reckon Mexico is located.  

Import Shapefiles

We now will make use of the example shapefiles we created earlier.   We cannot import into the Bing data source (Microsoft will not allow us to write data into their servers), so first we click anywhere in the lower portion of the Project pane, outside of the Bing data source hierarchy.  That moves the focus into the main part of the project, so anything we import will be placed there.

 

 

 

 

Choose File - Import to launch the Import dialog.

 

 

To reduce visual clutter in the Import dialog  we choose the SHP Files (*.shp) setting to show only the .shp files.   We highlight both .shp files to load both of their names into the File name, box, and then we click Import.   Manifold knows shapefiles are really at least three files, and will import all necessary data from the .shp as well as the .dbf and other files.

 

 

 

 

New drawings and their tables appear in the Project pane.   

A Lucky Import

We will now display the new MexPsMerc drawing by double-clicking on it to open it in a drawing window.

 

 

 

 

It appears to look OK.   There is nothing about the display which immediately would tell us whether the initial coordinate system for this drawing was correctly assigned.  To see what coordinate system was assigned we look in the Info pane.

 

 

 

 

The Info pane reports that the projection for this drawing has not yet been assigned.  When Pseudo Mercator appears in red text, that advises us that Pseudo Mercator has been assigned only as a placeholder, since the originating data source did not specify what coordinate system to use.  

 

Experienced GIS operators at this point will know the above is a matter of dumb luck, because they know very well that shapefiles by themselves provide no coordinate system information at all.  There is nothing whatsoever about the MexPsMerc.shp file that tells any software what coordinate system the data within utilizes, let alone that the data is computed for the Pseudo Mercator coordinate system.   

 

How did Manifold know that the contents of the MexPsMerc.shp file were in Pseudo Mercator coordinate system?  

 

Manifold did not know what coordinate system was in the file.   When Manifold imports a file that comes with no coordinate system information Manifold simply assigns the default coordinate system, which happens to be Pseudo Mercator coordinate system, as a placeholder, displaying the placeholder in red text to indicate it is just a placeholder.   When importing the MexPsMerc.shp file Manifold received no coordinate system info from the shapefile so Manifold assigned the default Pseudo Mercator coordinate system.   In this case by pure coincidence that happens to be the right coordinate system.

 

 

We created the shapefile that was imported, so we know in this rare case coincidence has worked out.  We may as well take this opportunity to verify this import as OK.   In the Info pane we click the coordinate system picker button, choose Assign Initial Coordinate System, the only choice, and then we choose the Pseudo-Mercator coordinate system from the list of favorites.

 

 

 

 

The coordinate system read-out in the Info pane switches to using black text to show the coordinate system has been assigned and is no longer just a provisional placeholder.   

 

 

 

 

We can confirm the initial coordinate system has been correctly assigned by dragging and dropping the MexPsMerc drawing into the map.    In the illustration above we have closed the other open drawings to reduce clutter.

 

 

 

 

Like magic, the MexPsMerc drawing appears exactly where Mexico should be.

An Unlucky Import

So now that we know that dumb luck is good to have on our side when importing shapefiles, we will see if our luck holds with the other shapefile, which contains Mexico in Latitude / Longitude coordinate system.  

 

We begin by right clicking on the MexPsMerc layer tab in the map and choosing Delete from map.  This will remove the layer from the map so that we can clearly see what happens when we drag and drop the MexLatLon layer into the map.  That does not delete the drawing from the project: it just deletes it from appearing as a layer in this particular map.

 

 

 

 

First, we double-click on the MexLatLon drawing to open it in a drawing window.

 

 

 

 

Visually, it seems to be the right shape.  An alert GIS operator would notice that the status bar reports an absurd geographic location for Mexico, just off the coast of Africa, when hovering the mouse over the drawing window.  An experienced GIS operator might also notice that scale indicates Mexico is about the size of our swimming pool, but many of us might not notice such clearly wrong values.   

 

Tech Tip:  This is how toxic data gets imported into workflow.  The drawing seen above has the wrong coordinate system assigned, but when opened in its own drawing window it looks perfectly OK.  Only when it appears as a layer together with correct data will it be seen to be in error.    If we do not catch the error now we risk using this drawing to create other drawings, proliferating the error into our workflow and into our archives.  Ultimately, if we publish the data we might even end up contributing to the world's growing inventory of junk data.  This topic shows us how to avoid doing that.

 

We will see what coordinate system has been assigned to this drawing by seeing what the Info pane reports.

 

 

 

 

As before, the Info pane reports that the projection for this drawing has not yet been assigned, reporting Pseudo Mercator in red text to indicate Pseudo Mercator has been assigned only as a placeholder.

 

 We know that the initial coordinate system assigned as a provisional placeholder is wrong, because the drawing we exported into that shapefile used Latitude / Longitude coordinate system.  Because we removed the .prj and the .mapmeta files that reported the coordinate system used by the shapefile there is no way for any GIS software to know what that shapefile contains.

 

We know in this particular case what the drawing is supposed to use as a coordinate system because we created the drawing.   But suppose we did not create it?   Suppose we downloaded the drawing from some random web site or suppose we created it a long time ago and forgot what coordinate system we used?   In that case we might look at what seems to be a perfectly good display in the drawing window, check the current coordinate system in the dialog and decide that the drawing was correctly imported using Pseudo Mercator, ignoring the red text that warns us the provisional initial assignment of coordinate system may be incorrect.

 

An easy way to see if the initial coordinate system has been correctly assigned is to compare the drawing to a known good layer.  We will do that by adding it to our map with the Bing layer.

 

 

 

 

We drag and drop the MexLatLon drawing into the map.   

 

 

 

 

Nothing happens.   Nothing appears because placeholder coordinate system assigned to the drawing is the wrong coordinate system.   When Manifold interprets the data within the drawing based on the provisional, placeholder initial coordinate system, Pseudo Mercator, the data does not appear in the right place.

 

 

 

 

In fact, the drawing appears as a nearly microscopic dot off the coast of Africa at the (0,0) intersection of the Prime Meridian (zero longitude) with the Equator (zero latitude).  If we zoomed far, far in we would see a tiny rendition of Mexico a few meters wide by a few meters tall.    Pseudo Mercator coordinate system uses meters as degrees of measure so the coordinate numbers in a Latitude / Longitude coordinate system, which are geographic degrees result a drawing only a few meters in dimension, with all of the meter numbers within a few meters of the (0,0) intersection.

 

Lucky for us, it is easy to assign the correct initial coordinate system if we know what it is supposed to be.    We will assign the Latitude / Longitude coordinate system as the initial coordinate system.

 

 

 

 

In the Info pane we click the coordinate system picker button, choose Assign Initial Coordinate System, the only choice for a coordinate system that is a provisional placeholder, and then we choose the Latitude / Longitude coordinate system from the list of favorites.

 

 

We know that this particular drawing uses the Latitude / Longitude coordinate system, which just happens to be one of the two default Favorites built into Manifold.    If the drawing used some other coordinate system we could choose More... to launch the full Coordinate System dialog that would allow choosing any of thousands of different coordinate systems or specifying a custom coordinate system.

 

After we click the Latitude / Longitude choice, the tiny dot off the coast of Africa disappears.  We Ctrl-click the MexLatLon tab to Zoom to Fit that layer, a quick way of panning and zooming the display to the proper location of Mexico.

 

 

 

 

Like magic, Mexico appears where it is supposed to be now that the correct initial coordinate system has been assigned to the drawing.  The Info pane now reports Latitude / Longitude in black text as the assigned coordinate system.  We have also taken a moment to Style the drawing with colors, to create a more visually appealing display.

 

Notes

Synonyms -  The terms projection and coordinate system are used as synonyms by Manifold.  Cartographers seem to favor the term projection while programmers seem to prefer coordinate system.  This documentation uses the two terms interchangeably.

 

Detective work - In this case because we created the shapefiles we knew that MexLatLon used Latitude / Longitude  coordinate system, so we could immediately and confidently assign that as the correct initial coordinate system.   But suppose we got the shapefile from some random web site and we didn't know what coordinate system it is supposed to use?  In that case we would usually first try Latitude / Longitude anyway as a guess.   If that did not work we would hunt through whatever website provided the shapefile to see if somewhere on the site we could find an explanatory note or some sort of metadata document that says what coordinate system is used.  

See Also

Info Pane: Component

 

Assign Initial Coordinate System

 

Reproject Component

 

Coordinate System

 

Style

 

Example: Convert a 0 to 360 Degree Projection - We often encounter data, both images and drawings, using latitude and longitude degrees that appears to be in Latitude / Longitude projection but which has longitude values from 0 degrees to 360 degrees and latitude values from 0 degrees to 180 degrees, instead of the usual arrangement of -180 degrees to 180 degrees for longitude centered on the Prime Meridian, and -90 degrees to 90 degrees for latitude centered on the Equator.  This example shows how to utilize such data by assigning the correct projection.

 

Example: Import Shapefile and Create a Map - Step by step process to import a shapefile and to create a map.

 

Example: Import a Shapefile - ESRI shapefiles are a very popular format for publishing GIS and other spatial data.  Unfortunately, shapefiles often will not specify what projection should be used.  This example shows how to deal with that quickly and easily.

 

Example: An Imageserver Tutorial - An extensive tutorial showing step by step how to add new data sources that are image servers, how to show them as layers in a map, how to create a new drawing that matches the projection of the map and how to trace over what is seen in an image server layer to create an area object in the drawing.

 

Example: Assign Initial Coordinate System - Use the Info pane to manually assign an initial coordinate system when importing from a format that does not specify the coordinate system.

 

Example: Change Projection of an Image - Use the Reproject Component command to change the projection of an image, raster data showing terrain elevations in a region of Florida, from Latitude / Longitude to Orthographic centered on Florida.

 

Example: Adding a Favorite Coordinate System - Step by step example showing how to add a frequently used coordinate system to the Favorites system.

 

Shapefiles Strangely Out of Shape

 

Accessory Files Created