The Linux OS manages memory using pages, where physical memory is partitioned and accessed using the basic page unit. The default page size is 4096 Bytes in the x86 architecture. The mapping of these virtual memory pages to actual physical memory addresses is bufferred in "Transaction Lookaside Buffers" (TLB) with a fixed amount of entries. When utilizing a large amount of physical memory with the default pages size, the program can consume the TLB and add processing overhead. Hugepages allows the page size to be bigger than the default 4096 Bytes, therefore hugepages can improve memory performance by increasing the maximum size of data and text that can be used in a program and reducing the TLB misses when accessing large data sets. On Cray machines, the uGNI and MPI implementation use huge pages for its internal buffers, so increasing the default page size can also increase the size of message buffers accessible by the high speed network.
When to Use Huge Pages
When to Avoid Using Huge Pages
Applications sometimes consist of many steering programs in addition to the core application. Applying huge page behavior to all processes would not provide any benefit and would consume huge pages that would otherwise benefit the core application. See
How to use hugepages
To use hugepages, load one of the 9 available hugepages modules below before running compilers. Loading the module sets the default huge pagesize to 2M, 8M, 16M, or 64M. No changes are required at compile and link time. The loaded module sets the proper library paths and libhugetlbfs will automatically be linked to the binary.
The following command lists the available huge pages:
----------------------------- /opt/cray/craype/2.5.8/modulefiles ------------------------------
Note, although larger hugepage modules are available, hugepages larger than 64M are not supported on our Cray system.
To use 2MB page size:
It is recommended that the specific module also be added to the corresponding job batch file.
aprun -n XYZ ./mycode