Memory Requirements

Your computer's memory includes Random Access Memory (RAM) and hard disk memory. Manifold System uses both types of memory together with the Windows operating system and any other applications that may be running at the same time. Regardless of the amount of RAM or free hard disk space available, there are certain fundamental limits that cannot be exceeded:

· No single image or surface can exceed 16 EB (exabytes) in size. Since one exabyte is 2^64 bytes, about a trillion gigabytes, as a practical matter there is no size limit for images and surfaces. One will run out of disk space billions of times sooner than the image size limit is approached.

· In 32-bit Manifold editions, for components other than images or surfaces (drawings, scripts, tables, labels, comments), no single component can exceed 4 GB (gigabytes) in size. Projects are not limited in size, but no single component of these types in the project can exceed 4 GB. In 64-bit Manifold editions, no single component can exceed 16 terabytes in size.

The intrinsic limits on images, surfaces and other components are so large that they will likely exceed the processing capacity and disk storage capacity of most machines in common use.

Use x64 Windows and x64 Manifold Editions

This is so important it is worth saying right up front: 32-bit Windows editions are crippled by an inability to use more than a relatively small amount of RAM. RAM has become so inexpensive that there is virtually no excuse not to have several gigabytes, at least 4 GB, of RAM in your computer. However, to use that RAM effectively you must be running x64 Windows and x64 Manifold. x64 Windows is more reliable and runs much faster and enables use of x64 Manifold, which runs much faster than 32-bit editions of Manifold.

Hard Disk Free Space Requirements

There are four demands for hard disk free space made by Manifold:

Project files - There must be enough free space on hard disk for the size of project (either compressed or uncompressed) when it is saved as a .map file. Obviously, if we have only 500MB of free space on disk we will not be able to save a .map file that is 1 gigabyte in size.

TEMP files - When Manifold opens a .map file it creates one or more temporary copies in the Windows TEMP folder to allow abandoning edits (that is, to close a project without saving any changes), to enable operation of Undo and for other purposes. Such temporary files can temporarily require four times the size of the project in uncompressed form. Thus, to work with a 1 gigabyte project in all circumstances we should allow at least four gigabytes of free space on hard disk.

System paging file - Microsoft Windows automatically uses virtual memory by swapping processes out to a hard disk file called the system paging file or pagefile. If we have only 512MB of RAM but want to run applications that require 1.5 gigabytes or RAM, Windows will automatically swap parts of the executing processes back and forth between RAM and the paging file so that at any one instant only 512 MB of RAM is used. There must be enough free space on hard disk for as large a pagefile as is necessary for Manifold and any other applications that are running.

Some users have multiple disk drives or have organized a single disk drive as multiple volumes. Make sure you have enough free space on all disk drive volumes involved if the project files, TEMP files and paging file are on different disk volumes. For example, a user might think there are tens of gigabytes of free space available on an E: disk volume where project files are kept without realizing that there is no free space on the C: disk volume where the Windows TEMP files and paging file might be kept.

Cache files - If you work with linked images from OGC WMS servers or TerraServer, Manifold by default will cache images and parts of images. Caution: Cache files persist forever and, in extreme cases, use up your free disk space unless they are manually deleted. See the recommended actions in the Managing Cache Files topic.

RAM and Virtual Memory Requirements

For performance reasons it makes sense to have as much RAM as possible and to use x64 Windows and x64 Manifold so you can take advantage of that RAM. It is a good idea to have somewhat more RAM than the maximum project size to allow RAM for Windows and for other applications that may be running in background.

To work with large projects, one must have a system pagefile can expand to as large a size as is necessary and the free disk space available to host large pagefiles if need be. In addition, it is critically important to have enough free disk space for all TEMP files that must be created by Manifold.

Geometric computations and many other functions use virtual memory for temporary computational requirements. Such virtual memory will be automatically switched by Windows between RAM and disk but can never exceed 2 GB in 32-bit Windows editions. This requirement for internal computational storage space means that while components and projects can be very large, the temporary storage space limitation of 2 GB may prevent some computational operations from proceeding to completion with very large data sets. For example, when creating buffer zones each buffer zone must be able to fit into Windows virtual memory. Running the Buffers transform operator on a drawing therefore will not work unless the buffers to be created will require substantially less than 2 GB for the buffer zone objects.

Although 2 gigabytes, of course, is enough for an immensely large number of buffer zone objects it may not be enough for, say, creating a buffer zone around every object in a 3 or 4 gigabyte drawing. In such cases, divide up the job into smaller parts and perform each in turn.

Users working with 32-bit Windows should keep in mind that the 2 GB process space will not all be available for a particular task. If you are using a 32-bit edition of Manifold, the Manifold process can only use 2 GB of memory. Some of that memory is spent on system services as well as Manifold services such as the window system. This can leave only about 1 GB of memory for tasks such as Normalize Topology or running the Buffers transform operator.

Memory usage patterns vary greatly between various algorithms used within Manifold, but a lack of memory can be encountered if an algorithm has used up some memory (for example, 500 MB) and wants more (for example, 1000 MB), but the system can not provide it. Thus, a times a lack of memory error can occur even though the amount of memory in actual use is well below the limit.

The way to avoid such difficulties is to utilize 64-bit Windows with x64 Manifold editions and to install plenty of RAM. x64 Manifold editions can use 16 Terabytes of memory, the same as can be addressed in 64-bit Windows.

Saving and Loading Projects

Manifold will often encounter peak demands for memory when opening or saving a project because it is at that time when reserve copies of projects are being made or are being utilized to allow abandonment of imports or exports or edits, or to Undo operations. When operating with limited amounts of free disk space it is possible for disk space requirements to exceed the amount of free space that is available on disk.

Free hard disk space available might change between the time a project is saved and when later it is opened. For example, we may have created a very large project and saved it and then later used up free space on our hard disk so that little or no free space is available. In such cases we might not be able to open large projects that were previously saved because we no longer have sufficient free space on disk for the TEMP files that must be created when the project is opened.

Reducing Memory Usage

When Manifold performs various operations it may need to use memory (either RAM memory or temporary files on disk) to allow Undo or abandonment of the project when exiting without saving changes. Saving the project will free memory that has been occupied by changes made since the last File - Open or File - New operation. Such savings can be very important when working with large amounts of data.

In addition to the suggestions in the Performance Tips topic, reducing the amount of data in tables can reduce memory requirements. At the very least, make sure to convert fixed-length text columns to variable-length text columns. Another space-saving option would be to store data in a linked table (say, within SQL Server) and to map the columns from the linked table to a drawing's table via relations.

A Note on .map File Compression

Saving .map project files in compressed form will save a lot of space on disk for routine storage of .map files. However, it does not save any space in TEMP file usage or other memory requirements because all operational work is done on the uncompressed data. The relatively small size of compressed .map files may catch users unaware as their files expand, possibly by a factor of ten, into operational use.

For example, a compressed .map file may be 200 MB, but it could easily expand by a factor of seven when uncompressed into a 1.4 gigabyte project. A 1.4 gigabyte project could require almost six gigabytes of free space on disk for temp files. If we planned on needing only 800MB of free disk space based on the compressed size of the project, we might think we had plenty of free disk space if we had four gigabytes free space available. However, that would be too little to deal with a project of this size once it is uncompressed and in active use.

Tech Tip

If for any reason (such as an electrical power failure) you crash Windows or crash Manifold by attempting to work with very large projects without enough free disk space you could end up with some large files in your Windows TEMP folder. Delete these before continuing work so that you do not waste disk space on unnecessary TEMP files.