Lotus Domino on AIX memory usage explained

The IBM AIX platform handles memory differently than all other Lotus Domino supported platforms. AIX uses a segmented architecture that has both advantages and disadvantages. The primary advantage is increased performance. Of the disadvantages, a 32-bit program must be cognizant of the segmented architecture to take full advantage of the 32-bit address space. This article describes the possible segment layouts for a 32-bit application on the AIX platform as well as the layout that Lotus Domino uses currently. It also shows what happens to the layout as you use the different Domino memory options.

This article is intended for experienced Lotus Domino system administrators familiar with the AIX platform.

Understanding segment layouts

By default, the 32-bit virtual address space on AIX consists of 16 segments of 256 MB each. For any application that uses the default segment layout, the virtual address space looks like the one shown in figure 1:

• The first segment (0x0) is used for kernal text and data.
• The second segment (0x1) is used for user text.
• The third segment (0x2) is used for user stack and data.
• Segments four through thirteen (0x3 to 0xC) are available for user process if either shmat() or mmap() is called.
• Segment fourteen (0xD) is reserved for shared library text.
• Segment fifteen (0xE) is available for user process.
• The last segment (0xF) is used for pre-process shared library data.

For anyone unfamiliar with the terms used in figure 1, here are definitions:

• Text. Code that is read-only and executable. It can be of three-type kernel code, user code, and shared library code.
• Data. Read/write data area that can be of three type kernel data, user data, and shared library data.

NOTE: Shmat() and mmap() are used in Lotus Domino to get shared memory.

Programs that use the AIX large memory model have the following memory layout shown in figure 2:

• The first segment (0x0) is used for kernal text and data.
• The second segment (0x1) is used for user text.
• The third segment (0x2) is used for user stack and data.
• Segments four through seven (0x3 to 0x6) are reserved by the process heap.
• Segments eight through thirteen (0x7 to 0xC) are available for user process if either shmat() or mmap() is called.
• Segment fourteen (0xD) is reserved for shared library text.
• Segment fifteen (0xE) is available for user process.
• The last segment (0xF) is used for pre-process shared library data.

Current versions of Lotus Domino use the AIX large memory model.

In AIX 5L version 5.2 and later, there is a very large memory model. This is the model that we hope to use in the future with Lotus Domino, but it is not currently used. This model has three forms: the first form is for programs that have a process heap (user data) less than 2.5 GB and greater than 256 MB (see figure 3). The model looks similar to the default AIX segment layout, but segments four through eight (0x3 to 0x7) are available for user heap if dynamic segment allocation (DSA) is used. Also, segments 0xA to 0xE are available for user process if shmat() or mmap is called() by dynamic segment allocation.

The second form is for programs that have a process heap greater than 2.5 GB as shown in figure 4 in which:

• Segment one (0x0) is reserved for kernal text and data.
• Segment two (0x1) is reserved for user text.
• Segment three (0x2) is used for the user stack. However, shared library text and data are also used in this segment (0x2).
• Segments four through sixteen (0x3 to 0xF) are available for the user heap if used by dynamic segment allocation.
• Segments 0xB to 0xF are available for the user process if shmat() or mmap() is called by dynamic segment allocation.

The third form is for programs that have a process heap less than 256 MB (see figure 5). In this form:

• The first segment (0x0) is used for kernal text and data.
• The second segment (0x1) is used for user text.
• The third segment (0x2) is used for user stack and data. However, shared library text and data may also be stored in this segment.
• The rest of the segments (0x3 to 0xF) are available for the user process is shmat() or mmap() is called by dynamic segment allocation.

Back to top

Lotus Domino memory control options

Now that you have been exposed to some basic information about AIX memory models, let's explore the model that Lotus Domino uses and how the Domino memory control options change the virtual address space.

The rest of this article shows you how to change the segment layouts by configuring settings in the Domino server's Notes.ini file. Before you proceed, here is a legend to help you understand the tables of information better.

• Program name. This is an example of a typical Domino application.
• Kernal text and data. This memory is shared.
• User text. This memory is shared for all programs with the same executable file (for example, three update tasks use only one chunk of memory shared between the three tasks.)
• User stack. This segment is unique to each process.
• Process heap. This segment is unique to each process.
• Shared memory. This segment is shared by all Domino processes.
• Shared library text. This segment is shared by all processes on the system.
• Unavailable segment. This segment is currently unusable by Lotus Domino.
• Shared library data. This segment is unique to each process.
• Non-Domino. This segment is shared and used by third-party applications.

Program shared data shared by all Domino applications starts at 256 MB and grows by 256 MB segments as needed until the maximum number of free segments is reached.

This first table shows the results of having no Domino memory settings enabled.

This is the layout of the shared memory:

• NSF Buffer Pool (UBM=3/8 shm) = 750 MB
• Other Domino shared memory = 1250 MB

Configuring the ConstrainedSHMSizeMB setting

The ConstrainedSHMSizeMB setting in the Domino server's Notes.ini file restricts shared memory. There are two ways in which to use this setting. If ConstrainedSHMSizeMB=1, then shared memory is restricted to a default size. In AIX, the default size is 2.25 GB. You can also specify a size in megabytes for this setting, which is what we have done in this article for demonstration. For example, if you change the setting as follows:

you may find the following results listed in this table.

This is the layout of the shared memory:

• NSF Buffer Pool (UBM=3/8 shm) = 654 MB
• Other Domino shared memory = 1099 MB

If you configure ConstrainedSHMSizeMB= 1744 in the Notes.ini file and set the dataseg used to 3 (the number of data segments), you may find the following results in this table.

This is the layout of the shared memory:

• NSF Buffer Pool (UBM=3/8 shm) = 654 MB
• Other Domino shared memory = 1099 MB

In the following table, ConstrainedSHMSizeMB is set to 1024 MB.

This is the layout of the shared memory:

• NSF Buffer Pool (UBM=3/8 shm) = 384 MB
• Other Domino shared memory = 640 MB

Configuring the PercentAvailSysResources setting

The PercentAvailSysResources setting in the Domino server's Notes.ini file lets you control memory allocation on your server. Values for this setting range from 2 percent to 100 percent. In the following table, PercentAvailSysResources is set to 25 percent on a system with 4 GB of memory. In other words, 1 GB of memory has been allotted to the Domino server.

This is the layout of the shared memory:

• NSF Buffer Pool (UBM=3/8 shm) = 384 MB
• Other Domino shared memory = 1664 MB

In this configuration, the setting of the NSF Buffer Pool used 3/8 * (RAM * .25). There is no defined limit to the growth of Domino shared memory other than the operating system imposed limit of segments.

The next table shows the results of setting NSF_BUFFER_POOL_SIZE_MB= 512 and PercentAvailSysResources=25 in the Notes.ini file.

This is the layout of the shared memory:

• NSF Buffer Pool (UBM=3/8 shm) = 512 MB
• Other Domino shared memory = 1536 MB

In this configuration, the setting of the NSF Buffer Pool is directly configured, and the PercentAvailSysResources is ignored. There is no defined limit to the growth of Domino shared memory other than the operating system imposed limit of segments.

Recommendation

We recommend that you do not use any form of ConstrainedSHMSizeMB. In the currently used AIX large memory model, ConstrainedSHMSizeMB should be used only to support third-party applications that require shared memory. If you use it, then Lotus Domino has less virtual address space to use. This is unlike all other Domino supported platforms. On other platforms, using ConstrainedSHMSizeMB to reduce the shared memory automatically increases the amount of private data space for the application.

To illustrate, here's an example: The Domino server is running a very heavy Lotus Domino Web Access HTTP load that requires 1024 MB of private data space. On AIX, you use dataseg –f 4 * in the Domino program directory to enable all Domino program files to use four segments of private memory. On other platforms, you need to use the operating system imposed virtual address space size. Then you subtract the space used for program text and shared library text. This gives you the amount of virtual data space available to Lotus Domino. In this example, you subtract the 1024 MB from the virtual data space available to Lotus Domino and use this for the ConstrainedSHMSizeMB setting.