The well-known American GIS company ESRI utilizes several different "geodatabase" formats for data, in three main types:
File geodatabases - Stored as files and folders within a file system. These exist in two versions, the old format used by ArcGIS 9 and the new format used by ArcGIS 10 and subsequent. This topic discusses use of the new format file geodatabases.
Personal geodatabases - Stored in Microsoft Access file format and limited to 2 GB.
Enterprise geodatabases - Also referred to as "SDE" style geodatabases and stored within a database server such as Oracle or PostgreSQL.
File geodatabases are recommended by ESRI over personal databases and, in the new format, are becoming one of the most frequent formats used to publish spatial data for use by ESRI applications. Old format file geodatabases are rarely encountered. ESRI provides an SDK/API, a software library with an application programming interface, to connect to new format ESRI file geodatabases for read/write access. Manifold uses the ESRI API to connect to ESRI file geodatabases, to connect to vector (drawings) data and tables. ESRI's API does not allow access to rasters in file geodatabases.
All required ESRI DLLs for new format file geodatabases are included within Manifold installation packages, including portable installations, and are installed automatically when Manifold is installed. Launching Manifold always includes ESRI file geodatabase capability. There is no need to install anything extra to allow Manifold connections to ESRI file geodatabases.
The Manifold GDB Dataport is Read / Write
Many, but not all, applications which utilize ESRI's API to connect to file geodatabases are read/write. Some are read-only. The Manifold GDB dataport is fully read/write and allows Manifold to edit ESRI file geodatabases in place. Manifold includes numerous technical features to facilitate read/write use of ESRI file geodatabases without problems.
The Manifold GDB dataport:
Uses ESRI FileGDB API 1.5, the latest version.
Allows creating, renaming, and deleting components of all types, storing component definitions inside the geodatabase.
Supports storing Manifold MFD_META properties inside the geodatabase. This allows overriding coordinate systems, setting up styles, and other Manifold features within the GDB.
Supports creating Manifold virtual computed fields within a GDB table.
Allows inserting, deleting and updating records in GDB tables.
Supports and utilizes ESRI spatial indexes on vector data.
Supports datetime fields.
Exposes btree indexes on GDB data.
Allows running queries using the ESRI FileGDB engine. As usual, we can switch between GDB queries and Manifold queries via !native and !manifold commands in the Command window.
Refers to components using their query names whenever possible.
Exposes coordinate system data.
Reads multi-patch geometry.
Reads geometry with curves, including circular arcs and ellipsoidal arcs.
Reads and writes geometry with Z values.
Forces index names to be different from field names.
Re-uses API objects when reading data from GDB tables, providing higher performance.
Automatically adjusts variants of produced BTREE indexes for unique / nullable field properties.
Forces use of unique indexes on OID and similar special fields in GDB tables, avoiding problems caused when the API sometimes fails to describe them as such.
Logs errors returned by the API if the geodatabase fails to open.
Caches data for btree indexes on the OID field on demand, for significantly faster searches on those indexes.
Verifies the type of geometry values to match the type specified for the table when writing geometry data to a GDB table. If the type of a new geometry value does not match the type of values in the table, the write fails.
Optimizes memory used during reads.
Export to GDB adds zero Z to geom that do not have it.
Handles renames and deletes of components and changes to table schemas.
Supports refreshing data from the Project pane (right click on the GDB data source and choose Refresh). This is useful for picking up changes to the geodatabase made outside of Manifold.
Supports autogenerated fields, such as OBJECTID.
Supports non-NULL constraints.
Translates coordinate systems with WKID values matching the correspondent EPSG code to EPSG:xxx to let Manifold enable coordinate transforms. Small differences between the definition of the coordinate system used by the database and the definition of the EPSG code used by Manifold are put into the coordinate system overrides.
Attaches EPSG codes to coordinate systems with equivalent WKID codes, to allow using grid-based and other EPSG coordinate transforms when re-projecting components with these systems.
Limitations of the ESRI API:
The ESRI API supports opening file geodatabases created with ArcGIS 10, but not "old format" ArcGIS 9. See the GDB, ESRI File Geodatabase, Old Format topic.
The ESRI API does not support rasters (Raster Dataset, Raster Catalog, Mosaic Datasets and Raster Attributes).
Spatial queries using the File Geodatabase API are limited to the envelope-intersects operator.
The ESRI API is not thread-safe, so Manifold cannot utilize it for parallel computation. For maximum speed and parallel performance, copy GDB data into the Manifold project and do intensive work within Manifold storage.
The ESRI API snaps coordinates of geometry objects to an internal grid, so the final coordinates of the written geometry values within the file geodatabase usually will differ slightly from the coordinates supplied by Manifold data.
File geodatabases cannot handle mixed geometry types in the same table. However, the ESRI API allows mixing geometry types in the same table even thought file geodatabases cannot handle that. For example, a point object can be added into a table that contains areas, with the API not throwing any errors during the write; however, the spatial index then without warning stops returning half of the objects in the table. To avoid such internal inconsistency by the API, Manifold rejects attempts to write mixed geometry types via the API.
For an example of connecting to an ESRI GDB, please see the Example: Connect to an ESRI GDB File Geodatabase topic.
ESRI's file geodatabase format is the new ArcGIS "geodatabase" format that has replaced personal geodatabases and, in the new format, has replaced the now-obsolete "old" format file geodatabase used with ArcGIS 9. Data in a file geodatabase is stored and managed in a huge collection of files, all located within a single folder.
In this topic, we illustrate import of a file geodatabase containing information on a gas utility company in Naperville, Illinois.
Important: When importing a file geodatabase file the tables and queries that appear in the Manifold project are Manifold components with no further connection to the file geodatabase file from which they were imported.
There are so many files within a GDB folder that it makes sense to first set the file type filter to GDB Files, so only the gdb file we want appears. We can then double-click it more easily.
Choose File-Import from the main menu.
In the Import dialog browse to the GDB folder containing the gdb file.
Double-click the gdb file desired.
Everything found in that gdb file geodatabase will be imported into the project.
This particular .gdb database contains numerous drawings and tables, organized within folders, related to a gas utility company in Naperville, Illinois. We can double-click a drawing or a table to open it.
For a more interesting display, we first create a new data source using a Bing satellite image server as shown in the Example: An Imageserver Tutorial topic. We then create a map and drag and drop the Bing layer into the map, and then we drag and drop a drawing into the map.
The drawing we added shows tax parcels in Naperville. We have used the Style pane to color them using different colors.
Important: When linking a file geodatabase file the tables and queries that appear in that data source in the Manifold project stay resident in the file geodatabase .gdb file. They are .gdb components even though they may appear in many respects, for the convenience of the user, to be Manifold components. Personal geodatabase files linked into a Manifold project are fully read/write.
The Save cache button allows setting cache options. Most often when linking to a format like file geodatabase, we will ensure the Save cached box is not checked. Working with a linked file geodatabase will be faster if we check the box, but if we are going to cache data within the project we may as well simply import the file geodatabase and use full Manifold speed. We uncheck the box and then we press Link.
To link a file geodatabase file:
Choose File-Link from the main menu.
In the Link dialog browse to the folder containing the gdb file.
Click the gdb file desired.
Check or uncheck the Save cache box as desired.
Press Link. A linked data source will appear in the project.
Press the + icon next to the data source to expand the data source to see the tables and queries it contains.
That creates a data source called NapervilleGas.gdb that contains all of the drawings and tables in the file geodatabase. We can click on the + icon by the data source cylinder to expand the data source hierarchy.
We can display a drawing in a map above a Bing satellite image server layer, as we did earlier in this topic. In the illustration above we show tax parcels in Naperville, using the Style pane to color them. We can style drawings within a GDB but only for the duration of the Manifold session. If we close the project and then re-open it, the style we have applied to drawings within the GDB will be lost.
ESRI GDB as a format cannot save Manifold style information, so if we would like to save style applied to drawings in a GDB in a persistent way, we can create a drawing in the Manifold project that links back to the data source, but which can be styled because it is resident in the project. This is similar to the technique used in the Example: Style Applied to an Image Server Image topic.
We link the NapervilleGas GDB into our project to create a NapervilleGas.gdb data source (illustration at left above). We expand the resulting data source, click on the TaxParcel Drawing to highlight it, and we press the Copy button on the toolbar (illustration at right above).
We could also simply press Ctrl-C for Copy.
Next, we Right-click into the Manifold .map part of the project pane, that is, outside of the NapervilleGas.gdb data source, and we choose Paste from the context menu.
We could also just click into the .map part of the project to move the focus there and then press Ctrl-V for Paste.
That creates a new drawing in the project called TaxParcel Drawing. Like all drawings, the new TaxParcel Drawing does not actually contain any data. Drawings are simply a component that says what geometry field in what table to visualize.
We can right-click onto the new TaxParcel Drawing and choose Properties to open the properties pane to see that the new drawing consists of just two short items of information: the drawing uses the geometry field called Shape from within the table called [NapervilleGas.gdb]::[TaxParcel], that is, the TaxParcel table that is inside the NapervilleGas.gdb data source.
Drawings are just a link, a shortcut to where the data resides, in this case a table within an external data source, along with some other properties, if desired, like style.
We can double-click open the new TaxParcel Drawing and then use Style to apply formatting to areas.
If we open up the Properties dialog for TaxParcel Drawing we can see that styling the drawing added two style properties, each with information in JSON format that describes the style. The new TaxParcel Drawing is in the Manifold .map portion of the project, so when we save the project the drawing and all the style properties it has will get saved, ready to be used the next time we open the project.
Suppose we would like to utilize the styled tax parcel drawing we have just created in a different Manifold project. We can copy and paste components and data sources between different Manifold projects, so that is easy to do.
We have saved our project as NapervilleGas_gdb.map in Manifold .map project format. Perhaps this is just one of many Manifold projects we have saved on an archival server our workgroup or organization uses.
In the illustration above, we have opened our NapervilleGas_gdb.map in one Manifold session.
We would now like to use the nicely-styled TaxParcel Drawing in a new project. We launch a second session of Manifold, which opens with a new, blank project, seen above.
In the NapervilleGas_gdb project we Ctrl-click on the NapervilleGas.gdb data source and we Ctrl-click on the TaxParcel Drawing, to highlight both of them. Next, we press the Copy button on the Project pane toolbar (or, we simply press Ctrl-C for Copy).
We switch to the new, blank project session of Manifold and we press the Paste button on the Project pane toolbar.
The NapervilleGas.gdb data source and the TaxParcel Drawing appear in the project.
When we double-click open the TaxParcel Drawing we see it is the same drawing, retaining the same style.
Copying and pasting data sources between Manifold sessions works when the PATH to the data source resource is the same from both sessions.
GDB drawings cannot handle a mix of object types. Therefore when exporting to GDB, Manifold will split a drawing that contains a mix of points, lines and areas into three different drawings within the GDB. Likewise, when writing to a GDB that is linked, for example, when copying a drawing from a Manifold project and pasting into a linked GDB data source, Manifold automatically splits a drawing containing geoms of different object types into separate drawings for each type within the GDB
Pass through of GDB to third party applications - When an ESRI GDB file geodatabase is connected as a data source within a Manifold project, that data source then becomes available to any third party application which connects to the Manifold .map project by using the Manifold ODBC driver. This provides the ability to read/write ESRI file geodatabases even to those applications which on their own cannot connect to ESRI file geodatabases. See examples cited below.
ESRI is not Manifold - When connecting to GDB we must be aware we are no longer using Manifold code but instead are using ESRI code. ESRI products are high quality and there are many advantages to using ESRI's own code to connect to intricate ESRI formats such as GDB, since presumably ESRI's code can handle all the many internal and undocumented quirks of the format. One major disadvantage is that ESRI's code in general is not anywhere near as bulletproof as the Radian technology used in Manifold. ESRI code can and does crash, so connections to GDB fall outside of Manifold's reputation for never crashing.
Manifold Viewer - View GDB Faster than ArcGIS Explorer - ESRI's ArcGIS Explorer product is a free viewer that allows ESRI people to view ESRI GDB Geodatabases, along with other data. Unfortunately, ESRI has stopped producing Explorer and has replaced it with ArcGIS Earth, which cannot connect to and display ESRI GDB Geodatabases. The video compares Explorer to Viewer side-by-side so ESRI users can see that if they need to continue viewing GDB Geodatabases for free they can reliably use Viewer for fast, high quality GDB viewing.
GDB, ESRI File Geodatabase, Old Format
Example: Convert an ESRI File Geodatabase into a .map Project - How to convert data in ESRI's current file geodatabase format into a Manifold .map project in a single step. We convert ESRI's example NapervilleGas GDB geodatabase, all 857 files, into a single, unified Manifold .map project.
Example: Convert an ESRI Personal Geodatabase into a .map Project - How to convert data in ESRI's legacy personal geodatabase format into a Manifold .map project in a single step.
Example: Create an ODBC Data Source with Windows - How to create an ODBC data source (a DSN) using dialogs built into Windows 10.
Example: Connect to Manifold from Release 8 - Step by step procedure to connect from Manifold System Release 8 to a Manifold .map file using Manifold's ODBC driver.
Example: Connect to an ESRI GDB File Geodatabase - Connect Manifold to an ESRI GDB file geodatabase, display the contents, make a selection in the GDB and overlay in a map.
Example: Connect to an ESRI GDB usng GDAL/OGR - Instead of using Manifold's built-in ability to connect to modern ESRI GDB file geodatabases, use the Manifold GDAL/OGR dataport to take advantage of the GDAL library's ability to connect to deprecated GDB formats.
Example: Connect Through Manifold ODBC to a Third Party - With Release 8, use an ODBC connection to a Manifold .map to connect through the .map project to a third party, external data source, an ESRI GDB file geodatabase. We use Manifold facilities as an intermediary to give Release 8 capabilities it does not have on its own, to link into data stored within an ESRI file geodatabase.
Example: Connect LibreOffice Through Manifold to an ESRI GDB - A companion example topic to the Example: Connect Through Manifold ODBC to a Third Party topic. Shows how to connect LibreOffice Base, the database part of LIbreOffice, through Manifold to link an ESRI GDB file geodatabase table into LibreOffice.