Example: Style Applied to an Image Server Image

Because the Style pane simply changes the way an image is displayed and not the data, it can operate on read-only data served by various web servers such as WMS REST servers.    In this example we look at every detail of creating a data source using an image server and then manipulating the appearance of the display with Style.

 

Important:  URLs for servers like the Maryland government server used in this example often will change.   The links used in this example may have changed.  If links quoted in this example have changed, please use Google or some other search engine to find the current links.

 

We will connect to a WMS server operated by the state of Maryland in the United States.   The current URL to a list of folders is http://lidar.geodata.md.gov/imap/rest/services

 

The server provides LiDAR data in various forms, including as terrain elevation, for various counties and cities in Maryland.  We have navigated to the page which provides elevation data for Charles County in Maryland.

 

 

Sites which publish data via web servers will often have a page for each data set published.  The page gives metadata and other information and also usually provides links to connection URLs to get to the data in various formats.   We plan on using a WMS connection capability so we will right click onto the WMS link and then choose Copy link address.    

 

The page used for this example:  http://lidar.geodata.md.gov/imap/rest/services/Charles/MD_charles_dem_ft/ImageServer

 

The WMS link used for this example: http://lidar.geodata.md.gov/imap/services/Charles/MD_charles_dem_ft/ImageServer/WMSServer?request=GetCapabilities&service=WMS

 

Browser aficionados will recognize we are using Opera as our browser, the new version with automatic ad blocking and built-in ability to browse via VPN to make it more difficult for privacy-stealing scum to capture and resell us as a commodity.

 

In Manifold we choose File - Create - New Data Source.   The dropdown menu provides a list of favorites to choose from as well as a More... option.  

 

 

Choose More... to launch the New Data Source dialog.

 

 

In the New Data Source dialog as the Type we choose Web Server: wms, we provide a reasonably descriptive name for this data source (instead of the default, not terribly mnemonic "Data Source") and in the Source box we Paste the text that we copied as the link address.  Press Create Data Source.

 

The new data source appears in our project.  When we expand the data source we see there is an image that is dynamically provided by the web server using WMS protocol.

When we open the image after a pause it fills in rapidly even if we access it from a different continent, such as Europe.   Cool!  It displays elevations for all of Charles County, Maryland.  

Enabling Styling for a Read-Only Server

Patience, please - Bear with us for this next section: we will learn a very important technique that is far easier and quicker to do in real life than either reading about it here step-by-step or doing it the first time.  

 

Imagery and data we get from the web server in Maryland is wonderful, but the web server is read-only.    That is true of all web-based or read-only data sources provided images, including all image servers, tile servers, WMS servers, REST servers and so on.  We would like to style that imagery so we can get beyond black and white and do more with Manifold.   To do that, we need to learn some simple techniques that allow us to style data that comes into our projects from read-only sources.

Temporary Styling

If all we want to do is to style the image provided by the web server on a temporary basis for the duration of our current Manifold, session, we can go right ahead and style the image seen above using Style, so long as we have checked the Cache data box when creating the data source.  That is the default setting, so normally we can temporarily style images provided by web serves.   After applying the style we have to do a View - Refresh and the new styling will appear.   However, since the web server in Maryland is read-only, the styling that is applied is cached within the Manifold project on a purely temporary basis.  Once we close the project whether or not we saved the project to a .map the styling disappears.

Persistent Styling

Usually when we go to the effort of styling an image we want to save that styling information when we save the project to a .map, so the next time we open that .map file the styling is still there.   To do that, we will create a local image that takes data from the web server but which can be styled locally.   We will exploit Manifold's ability to create multiple images that take their data from the same table, for example, as demonstrated in the Style: Palettes topic.   

 

All we need do is make a local copy of the image we are using from the web server, point it at the web server's table and then we can work with the local copy.   All data still comes in from the web server, but because the image which is providing the visualization of the data is local and stored in the project, the style properties we apply to that image will also be stored locally within the project.

 

We now will demonstrate how to do that step-by-step.  As usual, the following procedure covers every step so it takes much longer to look through than to actually do.

The image produced by the data source just created is called 0 Image (it was named that by the WMS server) and the table for the image's data is called simply 0.  

We click on 0 Image to highlight it and then we press Ctrl-C to copy the image.   We could, of course, press the Copy button in the Project pane's toolbar but most power users will simply do a Ctrl-C.   Copy and paste are such common functions that almost all power users have Ctrl-C and Ctrl-V hard wired into their brains and muscle memory.

We click anywhere in the lower part of the Project, that is, outside of the data source hierarchy, and press Ctrl-V to paste.

 

 

That creates a local copy of the 0 Image image from the data source.  Manifold automatically adjusts the properties of the image so it continues to pull data from the MD lidar wms charles data source table.

 

we can now begin use the local copy of the 0 Image just like it was the image directly from the server.   Our local copy stores no data: it simply takes data from the 0 table that is provided by the server.   As we pan and zoom within the local image Manifold will automatically get data from the server just as if the remote image were open.

 

However, now because we are working with a local image for which infrastructure data is stored on a table under control of our project and stored within our project, any style changes will be retained when we Save the project.  The next time we Open the project those style changes will be remembered and will still apply to the local image.

 

 

When we open the local image the title bar says nothing about the server, because we have opened a local component.  It is, indeed the same as the image from the server because it is showing exactly the same data that is brought in from the server.

Styling the Image

Now that we have created a 0 Image in the local part of our project, we will use the Style pane to applying styling to that image.   The data on which Style operates will come dynamically from the web server, but local styling will be applied and saved in our project.

 

 

We will zoom into the region marked by the magenta arrow.

 

 

We see above the raw image as served by the web server.   When using web servers the server will automatically provide whatever level of tiled imagery is appropriate for the viewing rectangle and zoom level.    Zooming into a view as seen above provides far greater details than the more summary tiles in the top level of the pyramid of details seen when the entire data set is in view.  

Apply Hill Shading

The contrast is not terrible as a default view, but we can improve the display by using shading and coloring with a palette.   Shading requires a single-channel raster, where the single channel represents terrain elevations.   That is easy to arrange.

 

To do so we will use the Style pane.

 

 

The Style pane opens using default channel assignment, showing all channels even if the same value is used for all.

 

The image is served as full color image tiles by the WMS server as a grayscale image, where all channels have the same number.   Because the tiles are full color tiles (even though the color for each pixel is a grayscale with the same number used for R, G and B) that is what Manifold presents by default in the Style pane.  But even though there are three (actually four, counting the unused Alpha channel...) channels provided by the WMS server, really, the data is simply just one channel that represents elevation data in feet.   That same, single number is provided in Channel 0, Channel 1 and Channel 2.    It is really just one-channel data pretending to be three channels.

 

Instead of using single number for R, G and B, we want to use it as a single channel to drive a palette image and to allow hill shading.

 

 

We press the pull-down menu in the channel usage menu.

 

 

From the pull-down menu we choose Channel 0.   Since the grayscale value is the same for all channels it doesn't really matter what channel we choose.  We also want to use that single channel to provide a grayscale visualization.  To do that we set the number of Breaks to 2 and then press the Tally box.   

 

 

We double-click into the lower color box and change it to white color.   If not already set by default from our last use of the dialog, in the pull down menu for Fill we choose interpolate.  Press Update Style to apply the changes.   The visual display is shown as an interpolated grayscale palette and the data is ready to use for shading.

 

We can now click the Options tab.

 

 

In the Options tab we check the Use shading box.  The Z scale value specifies the ratio of elevation to coordinate system units, allowing us to set the relative heights to emphasize or to reduce the visual impact of shading.  We change the Z scale to 100, a value which previous trial and error has taught us provides good visual results with the particular data set we are using.   Finally we press Update Options.

 

 

The display is hill shaded, using pixel values from whatever tiles the WMS server in Maryland has sent us for this view and zoom level.   

Apply a Colorful Palette

We can improve the appearance of the display by also applying a palette.

 

 

A palette will look better with more than 2 intervals.   We will try using five intervals.

 

We enter 5 in the Breaks box, we use a method of equal intervals and then we press the Tally button.

 

 

Five intervals appear with the default black color set for each.   To apply a color palette to the color wells, we press the Apply Palette button.    In the Classic submenu collection of palettes we will choose the Altitude palette, a reasonable palette for default use for terrain.  

 

 

The palette is loaded into the interval color values. We press Update Style.   

 

 

The result is a much more understandable rendering of the elevation data for Charles County.  We can zoom into the plateau in the middle of the right edge of the window to take a closer look.

 

 

As we zoom further in the WMS server will serve us tiles with greater detail.   Manifold will hill shade them and color them as well.

 

 

As we pan around in the data at higher zoom levels we can see this is a phenomenally detailed data set.   We can see how incredibly detailed it is by zooming out in a series of steps.

 

 

We zoom out one step with each press of the zoom out button or by using the mouse wheel.   As we zoom out, the WMS server provides less detailed tiles at each zoom level.  Those too are shaded and colored on the fly by Manifold, using our selected palette.

 

 

As we zoom further out we can see new features appearing.   At different levels of zoom the display may be slightly less sharp or more sharp as Manifold interpolates between the levels of zoom provided by the web server to match the specific zoom we command.

 

 

As we keep zooming out, details are smoothed out more and more by the tiles the WMS server provides.

 

 

At all times Manifold works with whatever data the WMS server sends.

 

 

Zoomed out to where the entire data set is visible we see that the data is so big that the beautifully detailed scene we saw, at the location marked by the arrow,  occupies less than a pixel or two at this zoom level.

Lessons

There are two main lessons to learn from this example: the first is that because Style applies to the view and is not a modification of the data, it can be applied to read-only image data such as that served on-the-fly from web servers.   The second lesson is that the world's web servers are full of astonishingly detailed data provided online within many terabytes of data sets available for our use.  

 

For example, using just this single web server in Maryland, from the other side of the planet we can not only explore the detailed topography of a bridge crossing over a small tributary of the Potomac River, we can connect to the Baltimore City elevation data to view and use the topographic details of Fort McHenry.   We visit the page at:

 

https://lidar.geodata.md.gov/imap/rest/services/BaltimoreCity/MD_baltimorecity_dem_ft/ImageServer

 

And we create a data source from the link

 

https://lidar.geodata.md.gov/imap/services/BaltimoreCity/MD_baltimorecity_dem_ft/ImageServer/WMSServer?request=GetCapabilities&service=WMS

 

Using the procedure given above, we copy and paste the 0 Image from the Baltimore City WMS data source into a new, local image we rename to be 0 Image Baltimore.  We then apply the same palette and same hillshading, and we zoom into Fort McHenry in Baltimore harbor.

 

 

Note that from our ability to hill shade it and to color it with a palette we are not working with a "dead" picture served to us but we have the real data at hand in the tiles within the table from which Manifold creates the display.  We can query that pixel data to use as we like.

 

Having such stupendously detailed data at our fingertips opens up many possibilities, not the least of which is exploring the planet to find archaeological sites, lost cities, roads between past civilizations, mineral resources or to manage agriculture, forests and other industries.   LiDAR has already revolutionized the acquisition of incredibly detailed terrain elevation data for many uses around the world.  Manifold's ability to acquire such data from almost any source and to manipulate that data almost as if it were local is the next phase of that revolution.

 

 

We can use Style to apply a palette, as seen above, zoomed into the same region we worked with earlier in this topic.  We have used the Color Brewer CB Accent palette with 12 breaks.

 

 

We can apply Shading as well, zooming in a bit more.

 

 

If we would like to change the palette we can do so as described in this topic and in the Style: Palettes topic.  For example, in the illustration above we have reversed the palette to reverse color ordering.

 

 

By changing the Fill value to Closest higher value we can force each height level to use a discrete color from the palette, as seen above.

 

 

As we Zoom In new data is automatically brought in from the web server and the style settings we have specified are applied.

 

 

As we Zoom Out we can see what the overall coloring is with this palette.    To emphasize hill shading contrast, for the image above we have increased the Z scale value to 500.    Changing the Z scale value can increase or decrease the amount of apparent shading.

Multiple Copies of the Image with Multiple Styles

Suppose we would like to keep this Style arrangement but we would also like to experiment with different styling, but without increasing the size of the project?   

 

That is easy to do since we can make as many copies of this image as we like and style each one differently without increasing the size of the stored .map project file.   Since all the images will take their data from the server, where the data remains stored, adding extra images will not increase the size of the .map project at all.    If we have cache turned on the cache will grow, of course, but that is the result of whatever browsing we do and is not increased by the number of images since all images taking their data from the same web server table will all use that same cache.

 

 

We click on the local 0 Image, press Ctrl-C to copy it and then right away press Ctrl-V to paste it.

 

 

That creates a local copy named 0 Image 2 of the local 0 Image.  The copy is automatically named 0 Image 2 because it is within the same hierarchy.    

 

 

If we open 0 Image 2 we see that it is indeed an exact copy of 0 Image, with exactly the same style properties and taking its data also from the 0 table within the remote data source.

 

But the new 0 Image 2 is a new image unconnected to the original 0 Image and it can be styled differently.   With the focus on the 0 Image 2 window we use the Style pane.

 

 

The style pane shows the style properties in use.   In the intervals list we click the first row to highlight it, we scroll down to the end of the palette and shift-click the last row to highlight both it and all rows in between and then we right-click into the highlighted rows to open the context menu allowing us to adjust all highlighted rows.  We choose Reverse to reverse the ordering of colors in the palette.

 

 

With the color ordering reversed we press the Update Style button to apply the new palette to 0 Image 2.

 

 

The new palette provides a different feel to the display.

 

 

As we Zoom In data will be brought in from the web server as required for 0 Image 2. Just like the 0 Image from which it was copied and pasted, 0 Image 2 takes its data from the web server data source's table 0.   In the image above we have reduced Z scale to 100 for a less contrasty hill shading effect.

 

At the same time, the original 0 Image is still open and can be independently styled, zoomed and so on.

 

 

We can have both images open, with 0 Image seen above, at the same time to compare the styling of each.   When we Save the project the style properties for both images will be saved with them.  The next time we Open the project the images will be styled exactly the same way.   The illustration above shows 0 Image zoomed in to a different location, using a Z scale of 500 for higher hill shading contrast.

 

 

In the display above we have changed the Z scale to 200.

 

 

In the display above we have changed the Z scale to 100.    We can explore different views with different settings to choose the one we prefer.

 

We can use Locations to save a location and then open the second image to the same view.    The scene above uses a Z scale of 200.

 

All the while, there is no extra storage required for the second image since both images take their data from the remote web server.  The data remains in storage in Maryland.  

 

 

We could have one hundred images using different palettes take from the same web server with no additional storage required. The illustration above shows a third image copied and pasted, using yet another palette, the CB Spectral palette.

 

 

The image above shows a fourth image, colored using the CB Pastel palette.

Commercial Web Servers

The same techniques apply to images we bring in from data sources created from commercial web servers, like Bing or Google.

 

 

In the project above we have added a New Data Source called Google Streets that is a Manifold image server using the Google Maps Street Map image server.    In the illustration above we have copied the Google Maps Street Map Image from within the Google Streets data source and have pasted it within our local project hierarchy.

 

 An important nuance:  we normally create maps with image server base layers by using the New Map dialog, as shown in the Maps topic.   The base layer in such cases is taken directly from image inside the image server data source.    To use the technique in this topic, we should first delete that layer from the map, and then make a copy of image server layer within the project, as shown in the above illustration.  We can then drag and drop that local copy into the map.

 

 

We can now use the local Google Maps Street Map Image just like we would the image within the data source hierarchy.    We can open it as seen above, zoomed into some area of interest, and then use the Style pane.

 

 

By default,  Channel 2 drives the R output, Channel 1 drives the G output and Channel 0 drives the B output. Channel 3 drives the A or Alpha channel.  We will make a quick change by choosing RGBA for the one-click channel arranger button.

 

 

We can then click the button to re-arrange the channels so that Channel 0 drives the R output, Channel 1 drives the G output and Channel 2 drives the B output. Channel 3 continues to drive the A or Alpha channel.   Press Update Style to apply that arrangement to the image.

 

 

The image immediately changes to show the new style.    

 

As before, as we Zoom In to the image Manifold will automatically fetch data from the Google server to display.

 

 

That data will automatically be colored using the new Style.

 

 

We can zoom further in.  If we wanted to, we could have changed Style to have made other changes.

 

 

At any time we can change Style, for example, by re-arranging the channel assignments.

 

 

Because all coloring is done on the fly there is zero overhead in terms of storage space.   The illustration above uses Channel 1 to drive the R output, Channel 0 to drive the G output and Channel 2 to drive the B output.  Channel 3 continues to drive the A or Alpha channel.

 

The image below uses a Channel 0 as a single channel for all outputs, 3 Breaks, a Fill setting of closest higher value,  the CB Spectral palette and Shading applied with a Z scale of 500.  

 

 

Shading is an effect that makes no sense with most image servers, but sometimes when used with restraint it can give a useful 3D effect to world maps.  Below we return (using the back arrow in Manifold's main menu) to the zoomed in view of Paris and apply a Z scale of 0.4

 

 

Finally, we restore the Style to the original BGR channel to output arrangement.  

 

 

Since the image is local, any changes we make to Style will be retained once we Save the project.

 

Notes

Ever-evolving URLs and hidden resources - The direct link to the LiDAR services page for Maryland that was used at the beginning of this topic is http://lidar.geodata.md.gov/imap/rest/services  However, Maryland seems to change that link frequently and the actual link to the LiDAR servers is very difficult to find in their website.  Instead, the site promotes various websites hosted by ESRI that offer data to those who get a login account.   Those are not the droids we are looking for.  To use REST servers directly we need to get access to those REST web servers, not to a web mapping application that itself has access to the data but serves only images to consumers.  Lucky for us, Manifold has published a .map project called ArcGIS_REST_Servers.map in the Examples page on the Manifold website.  

 

Blurriness at different zoom levels - Depending on whether we cache or do not cache, and also depending upon the web server, some zoom levels may show blurriness compared to others.  Consider the display when zoomed into "terraces" in part of the terrain.

 

 

In the image above, the terraces appear to have sharp edges.   If we change the zoom to zoom out one click of a wheel mouse we may see some blurriness to the "terraces" caused by quantized LiDAR height measurements.

 

 

What is going on is that the WMS server does not provide tile data for all zoom levels.  Instead, data between zoom levels is interpolated.  If the cache option is turned on, Manifold will do the interpolation.  If the cache option is turned off, the server will do the interpolation.   Having cache on results in faster display, but it may result in blurriness between zoom levels issued by the server.   Having cache off may or may not result in blurriness depending on how the server does interpolation in between zoom levels.

 

See Also

Images

 

Images and Channels

 

Palette Images

 

Style

 

Style: Images

 

Style: Channels and Outputs Tutorial

 

Style: Autocontrast

 

Style: Palettes

 

Example: Spectacular Images and Data from Web Servers - A must see topic providing a gallery of views illustrating how Manifold can use web servers such as image servers and other free resources to provide a seemingly endless selection of spectacular background maps, satellite images and GIS data with nearly zero effort.

 

Example: An Imageserver Tutorial - An extensive tutorial showing step by step how to add new data sources that are image servers, how to show them as layers in a map, how to create a new drawing that matches the projection of the map and how to trace over what is seen in an image server layer to create an area object in the drawing.

 

Example: How Images use Tiles from Tables - An example showing how an image is made up from data stored in a table in tiles.

 

Example: Create Two Images From One Table - More than one image can show data from the same table, including from the same tile field.

 

Example: An Image using Computed Fields in a Table - How an image can be created from tiles where the data for the tiles is taken from a field that is computed on the fly.

 

Example: Change the Contrast of an Image - In this example we use the Style pane to change the contrast of an image.

 

Example: Autocontrast and Hill Shading Images using Style - This example shows how the Style pane can hill shade an image using the values of pixels as heights and generating shadows as if the Sun were located at the specified azimuth and altitude.   This capability is used most frequently with raster images to give an impression of three dimensionality in cases where the values of pixels represent terrain elevations.

 

Example: Set Image Transparency using Alpha - The A row in the Style pane allows us to specify what transparency we want to apply to the image, either by applying the same value for A for all pixels or by using one of the other channels to also control the A value.  An example includes applying transparency to an image server image.