Example: Spectacular Images and Data from Web Servers

Few Manifold capabilities provide so much effect for so little effort as the use of web servers to automatically provide imagery or data for a Manifold project.    As we pan or zoom within the display window Manifold automatically fetches the necessary tiles from the web server to build the view we want.  With a few mouse clicks we can create a background map that could take hours to create from scratch.

 

See File - Create - New Data Source for instructions on connecting to many different types of web servers, including custom web server connection strings.

 

This topic is a companion and introductory topic to the Example: An Imageserver Tutorial topic which shows step by step how to add data sources that are image servers and how to use them.

 

Image servers instantly can provide background maps in virtually any style desired without the effort of formatting and they can provide a satellite view in astonishing resolution of almost any place on Earth.  The resolution of free satellite imagery is so good  that in many locations we can easily see individual people and determine the types of vehicles in use.   It is difficult to exaggerate the tremendous value of image servers for so little effort as made possible by Manifold.

 

As a client Manifold can connect to all typical technologies used for serving GIS data over the web as well as to many exotic technologies.  The result is the ability to connect to many thousands, if not millions, of sources online that provide access to a cornucopia of data.  This topic provides a gallery of the spectacular imagery and data we can obtain at will, for free, by using Manifold's ability to connect to a very wide range of web-based data sources.

A Shortcut:  Use New Map

There are two ways to add web served layers to a project:

 

 

 

Using the New Map dialog is how many users add image servers to a project, because usually we want to use an image server as a base layer in a map anyway, and most often we want to use one of the image servers in the Favorites list.

 

See the Example: An Imageserver Tutorial topic for a fast shortcut example.

Adding a Web Served Layer

To add any web served image layer, choose File - Create - New Data Source.  

 

 

Data sources in our Favorite Data Sources list appear at the top of the menu, including a default list of five image server layers.  We can choose any of them with a single click.  

 

If one of the Favorites in the initial list is not the desired choice, we choose More... to launch the New Data Source dialog.  That allows us to choose any of a seemingly endless variety of data sources:

 

 

Use the New Data Source dialog to configure a new image server or other web served layer.

 

To create an image server data source:

 

  1. Choose File - Create - New Data Source, or use the Project pane context menu.

  2. Choose one of the Favorites in the list, or press More... to launch the New Data Source dialog.

  3. In the Type box choose Web Server: imageserver, or whatever is the desired web served technology (like WMS).

  4. In the box just below the Type box choose the desired image server, for example, Bing Maps Street Map.   

  5. Image servers have pre-built URLs in the Source box.  When choosing a web technology like WMS, we can enter the URL for the desired server in the Source box.

  6. Specify a Name for the data source if something other than the default is desired.

  7. Press OK.

  8. The new data source appears in the project.  Click on the + icon to expand it.

  9. Double-click on the image within to open it and see what the image server provides.

 

See File - Create - New Data Source for instructions on connecting to many different types of web servers, including custom web server connection strings.

Satellite Views from Image Servers

Image servers are web servers using a protocol of serving images as tiles.  They are often used to provide images that show satellite photographic views or images representing digital maps.

 

Although the images used here are larger than average for documentation, they are small compared to the truly astonishing imagery we can create by spreading a Manifold window across an entire monitor or across multiple monitors.

 

Ancient spectacles:  Below, a view of the Colosseum in Rome.  The oval form is clearly visible from overhead.  This particular view is zoomed out from the highest resolution view available since zooming in to full resolution does not show the entire arena.  See a closer view of the Colosseum in the Example: An Imageserver Tutorial topic.

 

 

Modern spectacles:  Below we see a zoomed in view of the "Hollywood" sign in the hills above Los Angeles, California.   The sign is seen from orbit by a satellite and served by Microsoft's Bing imageserver site.   The smaller bushes are about half a meter, about a foot and a half, wide.  We can "read" the sign because a low sun angle causes the vertical letters to cast shadows in the form of the letters.

 

 

A technology icon: Below we see the Googleplex, the solar-pane covered headquarters of Google in Mountain View, California, as seen from orbit in satellite view by Google's competitor, Microsoft's Bing imageserver site.   Resolution like this, where individual people can be seen walking the paths, was exceptional for non-military satellite imagery only a few years ago but now it is becoming routine for larger and larger portions of the Earth.

 

 

Ancient secrets: the Sphinx at Giza, Egypt, as seen in resolution that covers most third world areas.  Vehicles are easily seen but individual people cannot be clearly seen.  Image servers have transformed Earth sciences, allowing almost anyone from anywhere to go hunting for lost cities and relics of past civilizations in deserts that would be impractical to visit in person.

 

 

Modern secrets: a US B2 bomber surrounded by guards and armored vehicles at Whiteman Air Force Base, Missouri.  Civilian satellites now generate so many petabytes of imagery as they ceaselessly orbit the world that national militaries cannot keep up with the flow of high resolution imagery available on the web.  This particular image was served by Google and acquired by a Manifold user in Europe zooming in from space to view highly sensitive military installations in the US. 

 

There is such a flood of detail available through image servers using Manifold that anyone in the world from almost any location with an Internet connection and Manifold could easily review the armed forces infrastructure of any country, right down to knowing which nuclear missile silos in the middle of nowhere deep in the US are being maintained and which have been neglected.  

 

 

Natural wonders: A scene showing Horseshoe Falls, Niagara Falls, Canada with the border between the US and Canada running through the somewhat more greenish swath of water in the upper right.  The use of image servers is revolutionizing environmental stewardship.

 

 

Modern wonders: Aircraft at Charles de Gaulle Airport, Paris.   Planning the infrastructure that now connects the world has never been easier.

 

 

Tourist delights: The Eiffel Tower as seen from an orbiting satellite far overhead.  The resolution is so good that individual tourists can be seen on the ground and on the upper observation decks.

 

 

Map Views from Web Servers

We take it for granted that we can see digital maps over our telephones, tablets and in-car navigation devices, but the range of cartography available to Manifold users is far greater.  Instead of the one or two providers most people use, such as Google Maps, on their telephones or tablets when travelling, Manifold can work with hundreds of different image servers and tile servers providing cartographic renderings in a vast array of different styles.   Many of these are free, many are regional or not worldwide or specific to a particular country or language and many require a subscription.   But even sticking only to free choices the results are astonishingly broad and deep.

 

We begin with some views of Europe, showing the same region but populated with a display provided by different servers.

 

 

Microsoft's Bing street map provides a typical web map display.

 

 

For a completely different background map, we can use the Canvas server's "dark" or "light" maps.  These are useful when showing layers above of point locations where we do not want the background map to be distracting.

 

 

Many servers provide map layers of physical features.

 

 

ESRI provides a world map of physical features using a REST server.

 

 

For oceanographic work we might prefer a display that minimizes features on land and emphasizes oceanic bathymetry.

 

 

Numerous open source servers provide completely royalty-free and open displays, such as the Open Street Maps (OSM) display of terrain elevation data using Migurski's formatting of terrain features.

 

We now will zoom far into the display, first into Germany and then into Munich right into the central square, the Marienplatz to see a variety of different map layers available from different web servers.

 

 

OSM layers are completely free.  Above we see the OSM base map, which provides a massive amount of information when zoomed into local areas such as the downtown of a city.

 

 

Various organizations and even individual hobbyists have customized different aspects of OSM to provide different displays based on special interests.  The above shows an OSM imageserver display aimed at users of public transportation.  It shows the main public transport links in downtown Munich.

 

 

WikiMapia provides a completely free map using simpler styling than OSM with less-overwhelming amounts of information.

 

 

A commercial map, also free, from Yandex, a popular choice in Eastern Europe.

 

 

The classic Google street map formatting and data.

 

Other Web Servers

There are hundreds of thousands of web servers that are free to use from which Manifold can pull web served layers using the New Data Source dialog.  These provide scientific data, aerial or satellite photography, demographic data, base maps, or endless varieties of other types of data.

 

 

A WMS server showing world-wide composite satellite imagery taken from the Sentinel 2 satellite, with photos selected to create a composite image.  

 

 

A WMS server showing VIIRS satellite sensor showing infrared detection of populated places in the Arabian peninsula and nearby regions.

 

 

A WMS server showing MODIS satellite images: vegetation activity (growth) during last eight days in the third week of January, 2020.  Darker green indicates more growth.

 

 

Demographic data from a WMTS server: Mortality risk from drought.  Red indicates higher risk.

 

 

Real-time GEOS satellite imagery in infrared, showing major weather systems.   World countries borders overlaid for easier interpretation.

 

 

MODIS satellite data: Snow extent in North America for the last eight days on a date in January, 2020.

 

 

MODIS satellite data: Snow extent in Eurasia for the last eight days on a date in January, 2020.  Record-breaking warm weather has resulted in anomalously little snow in the European plain of Russia.

 

 

Real estate parcels and building footprints in Chartres, France, from the IGN ArcGIS REST web server.  Check the property lines before buying a home in France!

 

 

A custom server layer showing OpenRailwayMap data in the vicinity of Chartres, France,  overlaid on a Bing streets background for context.  Make sure your new house in France is near a tram, subway, or rail station, for easy visits to Paris and worldwide airport connections.   See the Example: Connect to a Custom OpenRailwayMap Server topic.

 

Data from hundreds of satellite sensors and a vast array of government data is now published online, completely free, as web-served layers that Manifold can utilize.  Many government agencies worldwide publish data as web served layers, including data such as real estate parcels and many other valuable layers.

LIDAR imagery

LiDAR is a technology that uses lasers to scan terrain from aerial platforms such as aircraft or drones.  It generates massive amounts of data with very high precision.   The following images show data from the same server used in the Example: Style Applied to an Image Server Image topic to show how to use a WMS web server and to apply Style and palettes.  The data is LiDAR data for Charles County in the state of Maryland in the United States served by the state of Maryland.

 

 

The image above, styled with a palette that is typical for terrain elevation displays, shows the incredible detail available in LiDAR data.   The image below shows the same data with the Style dialog used to apply a palette that breaks terrain heights into distinct groupings.

 

 

The truly cool thing about Manifold's ability to utilizes a wide range of web server, tile server, imageserver and similar technologies is that we can create map layers almost instantly with nearly zero effort that serve almost any interest.   We can also acquire data from such servers, for example, vector data for drawings, and we can utilize satellite photos of almost any location on Earth either as the fundamental objects of our interest, as backgrounds or as sources from which we can create custom maps and drawings.

 

Notes

How can we find image servers like those in this topic?  - Many are built into Manifold and appear in the list when we choose Web Server: imageserver as the Type in the New Data Source dialog as illustrated in the Example: An Imageserver Tutorial topic.  Others may appear as a default Source string when we choose different types of web servers in the Type box, for example, Web Server: osm.  For others we must provide a Source string after we choose the Type of web server, for example, for a Web Server: wms or Web Server: arcgisrest type, as illustrated in the Example: Style Applied to an Image Server Image topic.   As mentioned in that topic, we can use the usual Internet search engines to find web sites with links that provide source strings to connect to various web servers we can use.    Every day there are more and more.    

 

Collections of Web Servers - Check the Product Downloads page for pre-packaged project files in Release 9 / Viewer .map format that Manifold publishes which contain collections of dozens of popular web servers data sources.   Two favorites (visit the Product Downloads page for the latest links, in case links below have changed):

 

 

 

Important:  Some web servers, notably TMS servers, use embedded command tokens.  Use Manifold's custom setting to connect to those.   See the File - Create - New Data Source topic for how to do that.

 

Imageserver or Image Server? - Manifold uses two styles, with and without a space character, using the two styles as synonyms without any special meaning attached to one or the other.   For historical reasons going back to prior Manifold releases the term imageserver tends to be used in programmatic contexts.   For discussions in ordinary text the term image server tends to be used.

 

Is a WMS server an Image Server?  -  There are many web technologies for serving tiled images over the web to form layers.   These include popular technologies like WMS, WMTS, TMS and so on, that Manifold can utilize.  In a sense, all those servers provide images so many web sites describe themselves as "image servers" no matter what technology they use to serve image tiles.   In Manifold dialogs the term image server is used more specifically, for a particular type of server used by companies like Bing and Google where the web server provides a single layer.   Other types of web serves that provide images are referred to by the technology they use, such as WMS server, or TMS server.   

 

See Also

File - Create - New Data Source - an absolutely key topic for connecting to many thousands of different web servers.

 

Web Servers

 

ArcGIS REST Servers

 

Custom Servers

 

OSM Servers

 

TMS Servers

 

WFS Servers

 

WMS Servers

 

WMTS Servers

 

Example: An Imageserver Tutorial - An extensive tutorial showing step by step how to add new data sources that are image servers, how to show them as layers in a map, how to create a new drawing that matches the projection of the map and how to trace over what is seen in an imageserver layer to create an area object in the drawing.

 

Example: Vector Layers from an ArcGIS REST Feature Server - Visit an ESRI web site, copy a URL, and then use that URL to connect to an ArcGIS REST web server that shows petroleum fields in Kansas, getting the data as a vector drawing layer. Style the layer as if it were local.  ESRI refers to ArcGIS REST servers that provide vector data as feature servers.  

 

Example: Connect to a WFS Server for State Government Data - Gathering our courage, we connect to a WFS server that provides 1200 vector layers, run by the state of Massachusetts.   We open a layer showing airports and then scrape the vector data into our own local storage.  

 

Example: Connect to a WMS Server for National Map Layers - Visit the National Map services web page, copy a URL for a shaded relief layer from USGS, and then use Style to enhance that shaded relief data for combination with other layers and really spectacular effects.

 

Example: Connect to a Custom Server for Cadastral Data - We connect to a custom image server that provides cadastral information originally from the French national cartographic agency, IGN.  We create a map and use the Style pane to re-style the web served image on the fly into a more usable form.

 

Example: Connect to a Custom OpenRailwayMap Server - We connect to a custom server that provides an OpenRailwayMap view of railroads worldwide, showing railway, tram, and subway infrastructure based on OpenStreetMap data.  Our first try at creating a data source does not work.  After consulting the Log Window we try again with a slight adjustment and our second try works.

 

Example: Connect to an OSM Vector Server - We connect to an OSM Server that provides a vector layer containing points and lines in the OpenStreetMap database.  We then show how to scrape (copy) data from the OpenStreetMap server into local storage.  We extract building footprints from the local copy.

 

Example: Raster Layers from an ArcGIS REST Image Server - Visit an ESRI web site, copy a URL, and then use that URL to connect to an ArcGIS REST web server that provide a raster layer showing a mosaic of aerial photographs near Portland, Oregon.

 

Example: Create a New Data Source from a Manifold Image Server - Manifold image server modules are snippets of code which use the Manifold Image Server Interface (ISI) to automatically fetch image tiles from popular image servers like Virtual Earth, Wikimapia, Yahoo!, Google Maps, Yandex and many others. Image servers can provide street maps, overhead satellite imagery, combinations of streets and satellite imagery and other data as well.  Using Manifold Image Servers is one of the most popular Manifold features.

 

Example: Create a New Data Source from a MAP File - Create a new data source from an existing Manifold .map project file.   This is the classic way to nest projects, one calling another, to create libraries of data and projects.   Access to nested projects has effectively zero performance loss and the links within projects take up effectively zero space so we can create huge constellations of data at our fingertips.

 

Example: Create a Data Source within an Existing Data Source - When a data source is writable, for example, if the data source is a Manifold .map file, we can work within that data source as if it were at the top level of our project.   For example, we can create a new data source that is nested within the existing data source.   This example shows how.

 

Example: Trace an Area in a Map over an Image Background - In a map with a drawing layer above an image layer (served dynamically by an imageserver), create an area object in the drawing by tracing over the outlines of something seen in the image layer below

 

Example: Style Applied to an Image Server Image - Because the Style dialog simply changes the way an image is displayed and not the data, it can operate on read-only data served by various web servers such as WMS REST servers.    In this example we look at every detail of creating a data source using a WMS REST  server and then manipulating the appearance of the display with Style.  We will connect to a WMS server that provides LiDAR data in various forms, including as terrain elevation.