Where Manifold Data is Stored

Manifold projects use data stored data within the project's .map file and can also work with data stored outside the project's .map file, for example, within other data sources or linked from external databases.     A Manifold project can store all data entirely within the .map file, it can work with data stored entirely outside of the .map file or it can work with a mix of both.   

 

The minimum size of a .map file is just big enough to store the two system tables, mfd_meta and mfd_root, and other minimum housekeeping information that tell Manifold where to find the data that will be used in the project.  The mfd_root table lists the names and types of all components in a project and the mfd_meta table stores metadata for components in the project.

Imported Data is Stored in the Project File

Importing data from a file copies data from the originating file format and stores it within the project as a local component, leaving the original file unchanged and preserving no connection to the originating file.   Until the project is saved the data is maintained in a temporary Manifold format file.  When the project is saved to a .map project file the data will be stored in that .map file.   

 

i_file_import.png

Linked Data Remains Stored in the Originating Source

Linked components such as tables, drawings or images will appear within the project pane hierarchy and can be used as seamlessly as if they were stored within the project, but all linked data remains stored in the original file and is not copied or otherwise moved into storage within Manifold.   It just seems that the data has been brought into the project.

 

i_file_link.png

 

Manifold projects are designed to make it easy to use data that is stored outside the project.    When at all possible Manifold facilities and commands will work exactly the same with data and components, such as tables, images, drawings, etc., whether they are physically stored inside the .map project file or are linked in from an external file or brought into the project from data sources.    

 

Because Manifold will do its best to provide the same look and feel to data whether it is stored within the .map project or brought in from an external source it is sometimes easy to forget when data is not stored within the .map project file.    We should keep in mind where data is located when performing operations such as making copies of .map files or when making or abandoning edits to data.

 

To help us remember when we are using stored externally data Manifold will use a yellow "database" cylinder icon for folders containing items from a data source or linked from a file.   i_yellow_cylinder.png

 

Whenever we see a yellow cylinder we know:

 

  1. That data is not stored inside the project.    Saving the project and then copying the .map file, for example for backup purposes, will not make a physical copy of the data but will only copy a link to where the data is physically located.   

  2. Any edits made to that data (if the file or data source is writable and edits are possible) usually will take effect immediately.  If we make edits and then do not save the project the edits will still have taken effect out in the originating file or data source.   They will not be "undone" because we exited Manifold or closed the project without saving the project.

 

Linking data from a file leaves the data stored in that file while Manifold works with it and displays it.   Likewise, when working with data from a data source that data remains within the data source while Manifold works with it and displays it.   When possible Manifold will utilize local cache to assist performance in both such cases but when linked data or a data source is writable the use of cache does not prevent any edits from immediately taking effect.

 

Local components, that is those which are not linked or which do not come in from a data source, will be located within the project pane hierarchy with no yellow database cylinder in their top level, containing folder.  The data for such components is stored locally within the .map file.   Any edits made to that data will take effect in the temporary, working copy of the project used for display purposes but will not be written to the .map file until we do a File - Save operation.   If we exit Manifold or close the project without saving it the edits will be lost and the original version of the .map file as last saved will remain unchanged.  That behavior is similar to how most applications, like Microsoft Word, work.

 

When we Save a project to a .map file we save all data in that project along with all the various components in it, including linked data and data sources.   However, linked components and data sources will be saved not as copies of the data but only as links, that is pointers specifying the location of the file or the data source where the project should go to get the data.  tech_yoshi_sm.png

 

Important: Based on how we linked the originating file or specified the data source those pointers to where the project should get the data may or may not be preserved if we move our use of the .map file to some other location.    For example, if we used absolute path names to specify the location of a linked file and then we move the .map file to a different computer where such an absolute path name no longer works the links will not be valid.

Examples

Example:  We Link into a project an Access .mdb file called friends.mdb that contains a table showing names, addresses and telephone numbers of contacts.   Although we can now use that table in our project the data continues to be stored in the .mdb file and not in the project .map file.  Any edits we make  to telephone numbers in that table will take immediate effect.   If we make changes to the telephone numbers in that table and then we Exit Manifold or Close the project without saving the project, the next time we open the friends.mdb file either in Access or by linking it into a Manifold project we will see the new, changed telephone numbers.  

 

Example:  Using File - Import  we Import into our Manifold project from an Access .mdb file a table showing names, addresses and telephone numbers of contacts.    The import process takes a copy of the data from the .mdb file and saves it within the temporary files Manifold creates for a project. The table now exists within the Manifold project completely independently of the Access file from which it was imported. Nothing we do in Manifold will make any changes to the .mdb file.    If we make changes to the telephone numbers in the table and then we exit Manifold or close the project without saving the project, everything we've been working on in the project disappears.    

 

The next time we open the .mdb file or get data from it we will see the telephone numbers were not  changed.  The .mdb file that was the original source remains unchanged, which makes sense since it didn't participate in our work at all once Manifold copied the data for the table out of the .mdb file to import into the temporary Manifold workspace.

 

Example:  Same as before, using File - Import  we Import into our Manifold project from an Access .mdb file a table showing names, addresses and telephone numbers of contacts.   We now save the project in Manifold .map format using  the name myfriends.map.   We can copy the myfriends.map file, move it to a different computer and open it with Manifold and the table will be there exactly as we saved it in the project.    When we Import data into a Manifold project and save the .map project file that data is part of the file.  

 

If we make changes to the table, for example, changing telephone numbers, and then we Save the project, the next time we open the myfriends.map file the new telephone numbers will be in the table.   

 

Suppose we now open the myfriends.map project in Manifold, we open the table and we change some of the names.  We now Exit Manifold without saving the project.  No changes are made to the myfriends.map file since we exited Manifold without saving.   The next time we open it we will see the names in the table have not been changed.

 

This is exactly the same as opening a Microsoft Word document, making some changes and then exiting Word or closing the document without saving it.  Any changes made are not saved.    

 

Example: As before, we Link into our project an Access .mdb file called friends.mdb that contains a table showing names, addresses and telephone numbers of contacts. We save the project in Manifold .map format using the name myfriends.map.  

 

We can make a copy of that .map file, move it to a different folder on the same machine and open it and in general the link will still work, because links are stored using full path names.   We can see the path name used for a linked component by right-clicking on the heading for that linked component (the one with the yellow database cylinder icon) and choosing Properties.

 

If the linked friends.mdb file was located on our computer at C:\\myaccessfiles\friends.mdb, when Manifold launches on that same computer and opens the myfriends.map project file it will be able to link to the friends.mdb file from anywhere on that computer.   Of course, if we delete the friends.mdb file or if we move it somewhere else to a different Windows folder, like to C:\\oldfiles\friends.mdb, then the link will no longer work and the linked table will no longer appear in our project when we open the myfriends.map file.

 

Suppose we copy our myfriends.map file and send a copy to someone else for use on a different computer.   When they open it they won't see the linked table because there will be no friends.mdb file on their computer.   If we want our friends to be able to use data on which a project depends it is easiest to just import the data into the .map file so when we send the .map file we know we send everything.

 

If we and our friend both have network access to the same storage we could, of course, keep our friends.mdb file somewhere in network storage where a single, fully-qualified network path name could correctly refer to it either from our computer or our friend's computer.   In that case we could link the .mdb file into our project using the fully-qualified network path name and then have the link work regardless of which computer was used to run Manifold and open that .map file.

 

Organizations will often maintain data warehouses in network storage so users can link to big data in network storage without having to make a lot of local copies.   Links and creating data sources in projects are exactly the way to use such network data warehouses.  However, we should always remember that because accessing data through a network usually is not as fast as accessing data on fast local storage, such as a local SSD, working with data that requires slower access through a network will result in lower performance.

 

See Also

Getting Started

 

How to Edit a Single File

 

The MAPCACHE File