GPKG

GPKG is the OGC GeoPackage format for storing vector and raster spatial data within an SQLite database container within a single .gpkg file.  As an interchange format, GPKG is a well-implemented, capable format that is clearly superior to older formats such as shapefiles.   There are two main limitations to GPKG:

 

 

 

We can either import data from GPKG into a Manifold .map project or we can leave the data within the GPKG file and link to it  to enable editing or other use of the data "in place" within the GPKG.   Which we choose will depend on the number of objects involved, the speed of GPKG we can tolerate or not tolerate, and the convenience of keeping data within the GPKG for any interchange requirements.

 

Compared to fast databases like PostgreSQL, GPKG is a much slower format and is generally too slow to use with larger numbers of objects.  A 1.3 GB GPKG is painfully slow, too slow to use, while the same data stored in PostgreSQL would be acceptably fast and stored in Manifold / Radian .map format would be pleasantly instantaneous.   

Import

Important: When importing a GPKG file the tables, images and drawings that appear in the Manifold project are Manifold components with no further connection to the GPKG file from which they were imported.  

 

To import from GPKG format:

 

  1. Choose File-Import from the main menu.

  2. In the Import dialog browse to the folder containing data of interest.

  3. Double-click the .gpkg file desired.

  4. Everything found in that .gpkg database will be imported into the project.

 

dlg_import_gpkg.png

 

Double-clicking on the desired .gpkg file in the Import dialog as seen above will import into our project everything found in that .gpkg database.  This particular GPKG file is one of the samples published by OGC.

 

il_import_gpkg01_01.png

 

The .gpkg database contains a drawing and an image.  Those have been imported together with all of the GPKG database infrastructure tables required to organize the DBMS built into the file.   We can double-click on the image or the drawing to open them.

 

il_import_gpkg01_02.png

 

For a more interesting display, we first create a new data source using a Google street maps imageserver as shown in the Example: An Imageserver Tutorial topic.   We then create a map and drag and drop the Google layer into the map, and then we drag and drop the Veg_DC.geom drawing into the map.    

 

GPKG does a good job of storing and providing coordinate system information so the drawing has the correct coordinate system assigned and appears in the right place in a map along with known good layers such as Google.   The drawing will be re-projected on the fly to match the Pseudo-Mercator projection used by the map.   In the illustration above we have used Style to format the colors of the drawing's areas to provide a more interesting display.

 

 

il_import_gpkg01_03.png

 

Zoomed in, the drawing seems to show different vegetation types in Washington, DC, in the United States.

Linking

GPKG may be much slower than fast DBMS like PostgreSQL, but it is fast enough to enable native storage of spatial data when there are not too many objects.  In Manifold we can take advantage of that by linking a GPKG file into our project.   The link creates a data source cylinder that indicates the data is stored outside of the project, in the original GPKG file.  When we expand the data source we can see the data within.  This works well for smaller data sets until GPKG gets too slow.

 

Important: When linking a GPKG file the tables, images and drawings that appear in that data source in the Manifold project stay resident in the GPKG file.   They are GPKG components even though they may appear in many respects, for the convenience of the user, to be Manifold components.

 

To link a GPKG format file:

 

  1. Choose File-Link from the main menu.

  2. In the Link dialog browse to the folder containing data of interest.

  3. Click the .gpkg file desired.

  4. Check or uncheck the Save cache box as desired.

  5. Press Link.  A linked data source will appear in the project.

  6. Press the + icon next to the data source to expand the data source to see the tables, images and drawings it contains.

 

dlg_link_gpkg01.png

 

The Save cache  button allows setting cache options.

 

dlg_link_gpkg02.png

 

Most often when linking to a format like GPKG, we will ensure the Save cached box is not checked.  Working with the GPKG will be faster if we check the box, but if we are going to cache data within the project we may as well simply import the GPKG and use full Manifold speed.   We uncheck the box and then we press Link.

 

il_link_gpkg01_01.png

 

That creates a data source that contains all of the tables and other contents of the .gpkg file's database.  We can double-click on the Veg_DC.geom drawing to open it.

 

il_link_gpkg01_02.png

 

The drawing opens in default format.    To color it with more appealing fill colors for the areas, we launch Style.

 

il_link_gpkg01_03.png

 

We use the fid field to automatically color fill color for the areas using the settings above.   We have pressed the Full Range button to calculate the full range of values before pressing the Tally button to generate 16 breaks for applying the palette chosen, the Color Brewer Spectral palette.  Press Apply.

 

il_link_gpkg01_04.png

 

That colors areas in the drawing using the fid field.  This has zero logical utility: it is just a means of coloring the drawing in a distinctive manner so later on we can see that the styling we have applied is indeed saved within the GPKG file.

 

il_link_gpkg01_05.png

 

 

For a more interesting display, we first create a new data source using a Bing street maps imageserver as shown in the Example: An Imageserver Tutorial topic.   We then create a map and drag and drop the Bing layer into the map, and then we drag and drop the drawing from the data source into the map.    The drawing will be re-projected on the fly to match the Pseudo-Mercator coordinate system used by the map.

 

To continue on with this workflow to save the new styling, see the Example: Link GPKG and Save Style topic.

Installing SQLite and SpatiaLite

GPKG is not a simple file format like .csv or .shp, but instead is a small DBMS system packaged within a file.   GPKG is implemented as an SQLite database extended with SpatiaLite capabilities, so it requires installation of SQLite software together with SpatiaLite software on any computer on which a GPKG file will be used.   If we do not have SQLite and SpatiaLite installed on our computer we cannot use GPKG.

 

There are two ways to get the SQLite and SpatiaLite software required to work with GPKG.  The official way is to visit the official websites for SQLite and SpatiaLite and download what the creators of those products provide.   As a practical matter that path is realistic only for experts.  The unofficial way is to visit the Manifold downloads page and to download the pre-packaged zip files Manifold provides to make the process easier.  Those zip files are highly unofficial files that are provided with no support and no guarantees, but the process is much easier for people who are not programmers or expert users of open source.

For Experts

The official sources for SQLite and SpatiaLite:

 

 

To install SQLite and SpatiaLite on our computer, we visit the above web pages and then we find and download an appropriate set of pre-compiled binaries or, using our C programming skills, we create our own binaries.  We install the right combination of binaries in the right places and, like magic, everything works perfectly.  This is straightforward for experts.

For the Rest of Us

For users who prefer an easier process, Manifold has assembled an informal collection of binaries for popular open source DBMS packages into two collections of files, one for 64-bit operation and one for 32-bit operation.    The collections include binaries required for clients connecting to MySQL and PostgreSQL / PostGIS, as well as binaries for SQLite and SpatiaLite required to work with GPKG files and other SQLite / SpatiaLite databases.

 

Visit the Downloads Page on the Manifold web site and download the two extension-dlls zip files, one ending in x64 and the other ending in x86,  from the section titled DLLs For Popular Open Source DBMS Packages.  Unzip the x64 zip file into the bin64 folder of your Manifold installation.   Unzip the x86 zip file into the bin folder of your Manifold installation.  Done.

 

The Manifold collections of binaries are informal, unofficial and not supported in any way.  They provide an experimental guide to what should be obtained and installed using the official, expert process of working with the open source involved.

Installation Example

We are working with Manifold on a 64-bit Windows 10 system.   We have unzipped the extension-dlls-x64.zip file into the bin64 folder of our Manifold installation.  That creates a folder called extension-dlls-x64 as seen below in File Explorer.

 

il_gpkg_dlls01_01.png

 

That is all we need to do.   Manifold will automatically utilize any .dll files that are located within the bin64 hierarchy.   If we are curious what .dll files have been installed, we can explore the extension-dlls-x64 folder.

 

il_gpkg_dlls01_02.png

 

It contains three folders, one each for MySQL, PostgreSQL / PostGIS and SQLite / SpatiaLite.

 

il_gpkg_dlls01_03.png

Within the sqlite-spatialite-x64 folder we see the .dll files that comprise the SQLite and SpatiaLite software ensemble.   If we get SQLite and SpatiaLite through other means, such as by direct downloads from the official sites, these are the .dll files we must have within our PATH for 64-bit execution.

 

Notes

Why doesn't Manifold just install required DLLs as part of Manifold installation?  Manifold probably will if GeoPackage becomes popular as a standard.  As an interchange format it is much better than older interchange formats like shapefiles.   

 

How were the unofficial collections of DLLs obtained?  These have been downloaded over time by Manifold personnel for their own informal, internal use.  Since Manifold people work almost exclusively in 64-bit settings the x64 .dlls are probably more reliable than the 32-bit, x86 versions.

 

See Also

Example: Modify GPKG Geometry with SQL then Add Drawing- This topic provides a "Hello, World" example that shows a simple, but typical, task involving spatial data.  We will take a country-sized data set in GeoPackage (GPKG) format and change all areas in the data to the boundary lines for those areas and then save those boundary lines as a new table.  We add a spatial index to the table and create a new drawing to visualize the new table.

 

Example: Link GPKG and Save Style - A companion topic to the GPKG topic.   How to link a GPKG, open a drawing, Style it and then save so the stying is retained within the GPKG file.