Getting Startedi_workflow01.png

The Manifold work cycle:

 

 

Data means tables, images, drawings, maps, data sources, queries, scripts, and more.  These are called components.  

Launch

Manifold System is built on Radian technology for the fastest GIS ever.   Nothing is faster than opening a Radian format Manifold .map file. It opens instantly.  Watch the YouTube videos to see the eye-popping speed of the new .map format.

 

 

 

 

For new projects, the fastest way to work is to import data into Manifold and then save everything in a Manifold .map project.  Manifold .map files open instantly, even when they are huge.  Migrate the data you use into Manifold .map storage from other formats.  Export to legacy formats for interoperability, but enjoy the unbeatable speed of Manifold .map projects for your own work.

i_workflow02.pngAdd

 

 

 

 

 

 

il_mouse_buttons.png

Viewil_toolbar_nav.png

 

 

 

 

 

 

 

il_wheel_mouse.png

Work

 

 

 

 

 

 

 

 

Save and Exitil_lidar_md_sm.png

 

 

 

 

 

 

 

Learn

 

 

 

 

 

 

 

A great way to learn SQL with Manifold is to ask Manifold to automatically write SQL for you and then to copy and adapt what Manifold creates.  See the SQL Example: Learning to Union Areas in SQL from Edit Query topic for an example.  Use the web and read books on SQL.  It's easy to learn and incredibly powerful.

 

Learn scripting to do even more.  V8 JavaScript, a really fast scripting engine, is built into Manifold so it is always-on and easy to use.  Manifold supports ten other scripting languages if you prefer something else.

 

Projections / Coordinate Systems

The terms projection and coordinate system are used as synonyms by Manifold.    See the coordinate system used by whatever layer or window has the focus in the Contents pane.

Imports and Projections

When importing data, Manifold will use whatever coordinate system the source says should be used for that data.   Most modern formats used in GIS will automatically specify the coordinate system to be used.

 

Some formats, usually older formats with older data or CAD formats not designed for spatial data, do not specify what coordinate system should be used for the data.  When Manifold imports data from a format that does not specify the projection to use, Manifold will assign the default Pseudo-Mercator coordinate system to the imported components as a provisional placeholder.  To warn us that initial assignment is just a placeholder Manifold will use red text in the Contents pane.   That default assignment will almost certainly be wrong so immediately after import we must specify the correct initial coordinate system to use.

 

To specify the correct projection, launch Assign Initial Coordinate System from the Contents pane and specify what projection should be used.  That can be difficult to determine in the case of older data.  When downloading data from the web or other sources, always look for any accompanying notes or metadata information that says what projection is used for the data.

 

If we have no idea what projection is used and the file is older data, a good guess would be Latitude / Longitude.   For example, shapefiles that do not have a .prj file attached to specify what projection they use are often in Latitude / Longitude projection.

 

For a complete, step-by-step example, see the Example: Assign Initial Coordinate System topic.

Default Projection

Manifold applies the default projection listed in the Tools - Options dialog when creating new components.  The factory default projection used by Manifold is the WGS 84 / Pseudo-Mercator (EPSG:3857) coordinate system that almost all web servers use.  Newer GIS packages, like ESRI's latest ArcGIS offerings, also use Pseudo-Mercator as a default.  

 

When we create a new, blank map, it too begins life using the Pseudo-Mercator projection.   The map then automatically takes on whatever projection is used by the component that is the first layer added to the map.  

Changing the Default Projection

Older GIS packages often use Latitude / Longitude as a default.   We can change the default projection in the Tools - Options dialog if we like, but there are pluses and minuses to switching to Latitude / Longitude instead of Pseudo-Mercator.

 

The modern way to do map backgrounds in almost all modern GIS packages is to use web servers to automatically generate the background map layer.  By a margin of about a million to one such web servers use Pseudo-Mercator as their projection  so it makes sense to use that same projection as a default for new components and maps in Manifold too.   Many newer GIS packages also use the same WGS 84 / Pseudo-Mercator (EPSG:3857) projection by default, for example, ESRI's latest ArcGIS family products.

 

Another big reason to use Pseudo-Mercator as a default is that the units of measurement are sane, linear units: meters.   That makes it much more difficult for beginners to massively goof up measurements, as routinely happens when people use Latitude / Longitude without realizing that the units of measurement are degrees, which change size depending on latitude.

Projections Manifold Knows

Manifold understands and can work with over 6500 coordinate systems, including all coordinate systems and transformations specified by the vast EPSG system.   When importing from or linking to a data source that conveys coordinate system information Manifold will automatically use the coordinate system specified for the data, including any projections specific to the data source, such as SRID designations of coordinate systems used by a particular data source.   See the Projections topic to learn how to work with projections in Manifold.

 

Some file formats utilize accessory files such as "world" files, or .PRJ files to convey coordinate information for formats, such as shapefiles, which do not automatically convey coordinate system information.   When reading formats Manifold will automatically read coordinate system information from accompany files, if they exist, in the following order: "world" files, .PRJ files, .XML files (Manifold System 8.00 accessory file) and .MAPMETA files (Manifold accessory files).  In the event of conflicting coordinate definitions the last read file wins since the read order is in order of more rigorous definitions.

 

Manifold maintains coordinate system information in JSON format to enable easy interoperability with third party code.  When exporting data to a file Manifold will write a .MAPMETA accessory file saving the JSON coordinate system information in plain text.  When exporting to shapefiles, Manifold will automatically write a .prj file with projection information as well.  

Importing Files

Importing or linking a file will create Manifold infrastructure for what is coming in.  For example, import an image from some file format like JPEG or TIFF and Manifold will create two components:  a table component for the data plus an image component that displays the data in the table.   Import an ESRI shapefile and Manifold will create a table for the data plus a drawing component that displays that data.  The variety of data Manifold can import includes virtually every format and data source in existence.   See the Big List of Formats and Data Sources topic for the huge list.     

 

dlg_file_import.png

 

Importing a .csv file with one table in it will create a table.  Importing or linking a file database like an .mdb could create hundreds of tables, queries and other components at once.  Connecting to an enterprise class DBMS might involve thousands of items, neatly organized within the hierarchical structure of the DBMS as revealed by Manifold.

 

Importing a table from an .mdb, .db, .html, .xls, or .wkx file:

 

  1. Launch 32-bit Manifold

  2. Choose File - Import.

  3. Browse into the folder where the file is located.

  4. Click on the filename.

  5. Press the Open button.

 

 

Importing a table from other file formats:

 

  1. Launch either 64-bit or 32-bit Manifold

  2. Choose File - Import.

  3. Browse into the folder where the file is located.

  4. Click on the filename.

  5. Press the Open button.

 

Manifold uses Microsoft drivers to connect to Microsoft .mdb, .db, .html, .xls, and .wkx file formats.  Because of Microsoft limitations on using 32-bit Microsoft software in 64-bit Windows systems, to connect reliably to those Microsoft file formats in 64-bit Windows systems we should use 32-bit Manifold.   Every Manifold license provides both 32-bit and (if our Windows system is 64-bit) 64-bit Manifold, so when desired we can always launch Manifold in 32-bit mode to enable use of 32-bit Microsoft drivers.

 

See the discussion in the MDB Files in 64-bit Windows essay.

 

Importing a drawing, image or other component:

 

  1. Launch either 64-bit or 32-bit Manifold

  2. Choose File - Import.

  3. Browse into the folder where the file is located.

  4. Click on the filename.

  5. Press the Open button.

  6. If necessary, launch Assign Initial Coordinate System from the Contents pane to specify the coordinate system used.

 

Modern GIS data formats will specify the projection used by the data.   When a format provides projection information Manifold automatically will utilize that information to correctly set the projection used by the data.

 

When importing GIS data from a file format which does not provide projection information, when we open that drawing or image Manifold will warn us that was imported from a spatially-clueless format by showing the coordinate system read-out in red text in the Contents pane.  

il_assign_initial_coord_sys_01.png

That tells us we must launch Assign Initial Coordinate System from the Contents pane to assign the initial coordinate system manually.

 

Example: Importing a Shapefile

ESRI shapefile format uses 1970's technology but it is still the most widely used GIS format for interchange.  A "shapefile" is an ensemble of at least three files, ending in .shp, .shx and .dbf, and, if we are lucky, a fourth file ending in .prj.

 

If a shapefile has a .prj to specify the projection, Manifold will read the .prj and the shapefile will be imported using the correct projection automatically.  If a shapefile does not include a .prj file, we must specify the projection the shapefile should use.  If we fail to do that the shapefile might look OK but it will be junk data that will waste our time later on.

 

Importing a shapefile:

 

  1. Choose File - Import.

  2. Browse into the folder where the shapefile is located.

  3. Click on the filename that ends in .shp  

  4. Press the Open button.

  5. If the shapefile does not include a .prj file, we must launch Assign Initial Coordinate System from the Contents pane to   specify the coordinate system the shapefile is supposed to use.

 

For a step-by-step illustrated example, see the Example: Import a Shapefile topic.  See the SHP, Shapefiles topic for additional discussion.

 

See the Example: Detecting and Correcting a Wrong Projection  topic for what happens when a shapefile is imported with the wrong projection.

 

More Examples

This documentation includes very many examples.   Begin with the big list in the Examples topic.

Notes

Three Letter Extensions - Most file names in Windows end in what is called a three letter extension, which is usually three letters at the end of the file name following a dot . character.    The three letter extension is one way Windows at times (but not always) keeps track of what a file is supposed to be.    Unfortunately, by default Windows hides the three letter extensions of files and instead tries to associate files with icons of whatever program is normally associated with that file.  This is confusing when working with the many file formats that Manifold and similar products utilize.  

 

Therefore, please turn off the hiding of extensions by Windows.  A typical way to do so in most versions of Windows would be from Windows Explorer, choose Tools - Folder options, press the View tab and then in the Advanced Settings pane ensure that the Hide extensions for known file types is unchecked.  Press the Apply to Folders and then press OK.  You will then be able to see extensions such as .map and others.  See also the Essay on three letter extensions and why the default hiding of them by Windows is such a bad thing.

 

Why import and then work?   Consider image formats: there are hundreds of different image formats, all with their own limitations.   Instead of forcing us to learn a hundred different ways to work with images, taking into account the limitations of each format, Manifold does a conversion when bringing an image into Manifold so we only need to learn about  images in Manifold.   To export that image out to some other format Manifold will do a conversion outbound.  When an image is imported into Manifold, it is no longer a JPEG or PNG or TIFF image.  It is a Manifold image.   If we want to save it as a TIFF Manifold will convert it into a TIFF on the way out.

 

Why create a table to import some small JPEG?  Because everything is stored in a table in Manifold, even small images from a JPEG.  That means nothing is hidden in some secret handshake form.   All data is there in a table where the SQL you already know allows you to get your hands on the data and do what you want with it.  Don't know SQL yet?   Shame on you.  You'll learn quickly enough (it's easy) and until you learn you can use the Transform dialog templates without knowing any SQL.

 

About .map files - Manifold stores data in .map format project files.  A Manifold .map file can be interchanged by all Manifold products based on the Radian engine. ".map" stands for Manifold Project, the file database format used for all Manifold database and GIS products.   In a time when disk space is endless and cheap and human time is limited and expensive, .map files are designed to be as fast as possible and not as small as possible.  Saving a big table or image in a .map file will usually result in a bigger file on disk than other formats but almost always the result will be far faster operation.

 

 

 

MAP files and MXB files -  For everyday work we want our projects to open instantly, save instantly and operate at eye-popping speed.  When exchanging data or saving archives for long-term, rarely used storage we want our projects to be as compressed and as small as possible.   To serve both needs Manifold has two formats for project files:  super-fast MAP and super-compressed MXB.

 

Manifold .map format provides phenomenal speed and capacity for everyday work.  Manifold .map projects pop open instantly, save instantly and can be nested with links within projects to other projects with zero loss of performance.   Manifold .mxb files are highly compressed files that archive a project in the most compact possible form.   They are perfect for creating the smallest possible project file for exchange over Internet or for archival storage.

 

Manifold ODBC driver - The Manifold ODBC driver makes it easy for other applications to read/write Manifold .map files as well.  See, for example, the Example: Create an ODBC Data Source with Windows topic and similar examples.

 

Dataports:  A dataport is a Manifold module that interacts with file formats, databases and other sources of data.   When we import data from a given file format, say, from shapefiles, we are calling the dataport for shapefiles.  

 

tech_tina_sm.pngPlease note: While it may be convenient to leave data within original formats, such as shapefiles as often encountered in GIS, doing so means living within the limitations imposed by those formats.   Older formats may be fine for small amounts of data but when larger amounts of data are involved they can be slower than Manifold .map files and they often have limitations such as no ability to mix vector types, limits on data types, limits on size or other limits.  Using .MAPCACHE can deliver impressive speed even with older formats but even so using Manifold .map format is almost always faster, usually much faster, for larger files and .map format does not suffer from the content limitations of older formats.

 

Copying and pasting between sessions - When copying and pasting between different Manifold sessions remember to copy all parts of a component.   An image component, for example, is just a display module to show the contents of the associated table which stores the tiles for that image.   To copy an image from one Manifold session to another we must copy and paste both the image as well as the table for that image.

 

Docked and Undocked -  Manifold allows us to undock panes and windows and to move them around our Windows desktop as we like.  We don't have to keep undocked Manifold windows and panes within the boundaries of the main Manifold desktop.   

 

Reserved names - All names starting with mfd_ (case not significant) are reserved for use by the system.  Names for fields, indexes, constraints, tables, components other than tables, properties... everything. Do not name anything beginning with mfd_ or MFD_ or in any upper or lower case combination of those characters.

 

Mnemonic names - It helps to name data sources to know their origin at a glance.   In the example recommended above we use Books MDB as the name of a data source so at a glance we are reminded the data source is an .mdb file having to do with books.   

 

i_datasource_tooltip01.png

 

But if we forget, hovering the mouse cursor over a data source displays a tooltip that shows the origin.  If the data source has information in the Description property the description will be shown in that tooltip instead.

 

Compacting Data Sources - If a data source can be compacted, right-clicking on that data source in the Project pane will show a Compact context menu command.  If the data source does not support compacting, the command will not appear.  In addition to the context menu Compact command that can be used manually, the Manifold object model includes means to compact a data source, to check whether a data source supports compacting, to check whether a data source has data to compact or has unsaved changes.

 

Saving Data Sources - The Project pane tracks any unsaved changes for data sources that support saving, including the root .map file itself.   Data sources with unsaved changes will be displayed using a different icon that shows an asterisk * in the lower right corner of the icon.   Making changes via a script to a data source that supports saving may require a few seconds before the icon is updated.   If a data source does not support saving the icon will not change and the Project pane context menu command for that data source will not include a Save command.

 

Manual data entry - The idea of manually creating components and manually inputting data is not so far-fetched as it may seem.  For example, we can create points in a drawing by clicking on locations in a drawing in a map using a lower layer (a satellite image or map from a web server) as a guide.    Whether we are marking trees, fire hydrants or archeological relics it is often possible to rapidly click and create hundreds of points or areas or other features per day.   Over a surprisingly short period of time many thousands of records can be accumulated as the result of manual data inputs.

 

Table Nomenclature - When writing about tables we will often use the words row and record interchangeably and also the words column and field interchangeably.  

 

CPU Parallelization - Manifold automatically runs parallel for internal Manifold tasks and for transforms and similar operations, using all of the CPU cores available in your system in parallel.   When writing queries manually using the Command Window make sure to add a THREADS SystemCpuCount() command to the query to automatically parallelize the query to use all CPU cores in your system.

 

GPGPU - Manifold automatically uses NVIDIA GPUs for massively parallel computation.    NVIDIA GPUs of Fermi class or more recent which NVIDIA supports with CUDA are required.  See the GPGPU topic for details.

 

Limitations - Manifold can address such large memory that we will run out of physical resources on a machine, like disk space, before we hit Manifold size limits. However, one limitation may be touched by users of very large data on the desktop.  Although there is no practical limit to the size of data Manifold can work with on the desktop a hard limit on the amount of data that can be changed in a single operation is 256 GB.   For example, if we have a single image that is 1 terabyte in size we can display that and manipulate it but if we want to change all of the values in all of the pixels in the image we will have to do that in four operations, each up to 256 GB in size.   The 256 GB limit is the current, somewhat arbitrary setting on temporary caches utilized internally by Manifold.  It could be increased in future updates should user demand require.

 

See Also

Projects and .map Files

 

Importing and Linking

 

User Interface Basics

 

How to Edit a Single File

 

Tables

 

Queries

 

Maps

 

Drawings

 

Images

 

Labels

 

Layouts

 

View - Panes - Project

 

Assign Initial Coordinate System

 

Examples - Do not miss!  Browse through the many examples for step by step tutorials.

 

Example: Project Pane Tutorial - In this example we take an extended tour of the Project pane, engaging in a variety of simple but typical moves that are illustrated step by step.

 

Example: Import a Shapefile - ESRI shapefiles are a very popular format for publishing GIS and other spatial data.  Unfortunately, shapefiles often will not specify what projection should be used.  This example shows how to deal with that quickly and easily.

 

Example: Closing without Saving - An example that shows how File - Close without saving the project can affect local tables and components differently from those saved already into a data source, such as an .mdb file database.

 

Example: Spectacular Images and Data from Web Servers - A must see topic providing a gallery of views illustrating how Manifold can use web servers such as imageservers and other free resources to provide a seemingly endless selection of spectacular background maps, satellite images and GIS data with nearly zero effort.

 

Example: An Imageserver Tutorial - An extensive tutorial showing step by step how to add new data sources that are imageservers, how to show them as layers in a map, how to create a new drawing that matches the projection of the map and how to trace over what is seen in an imageserver layer to create an area object in the drawing.

 

Where Manifold Data is Stored

 

The MAPCACHE File

 

Keyboard and Mouse Quick Reference

 

Performance Tips