Skip to content

Commit

Permalink
doc: update information on using hugepages
Browse files Browse the repository at this point in the history
[ upstream commit 8397cac ]

Current information regarding hugepage usage is a little out of date.
Update it to include information on in-memory mode, as well as on
default mountpoints provided by systemd.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
  • Loading branch information
anatolyburakov authored and kevintraynor committed Dec 15, 2020
1 parent 05a0668 commit 1c361d2
Showing 1 changed file with 47 additions and 25 deletions.
72 changes: 47 additions & 25 deletions doc/guides/linux_gsg/sys_reqs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -156,8 +156,36 @@ Without hugepages, high TLB miss rates would occur with the standard 4k page siz
Reserving Hugepages for DPDK Use
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The allocation of hugepages should be done at boot time or as soon as possible after system boot
to prevent memory from being fragmented in physical memory.
The reservation of hugepages can be performed at run time.
This is done by echoing the number of hugepages required
to a ``nr_hugepages`` file in the ``/sys/kernel/`` directory
corresponding to a specific page size (in Kilobytes).
For a single-node system, the command to use is as follows
(assuming that 1024 of 2MB pages are required)::

echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

On a NUMA machine, the above command will usually divide the number of hugepages
equally across all NUMA nodes (assuming there is enough memory on all NUMA nodes).
However, pages can also be reserved explicitly on individual NUMA nodes
using a ``nr_hugepages`` file in the ``/sys/devices/`` directory::

echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages

.. note::

Some kernel versions may not allow reserving 1 GB hugepages at run time,
so reserving them at boot time may be the only option.
Please see below for instructions.

**Alternative:**

In the general case, reserving hugepages at run time is perfectly fine,
but in use cases where having lots of physically contiguous memory is required,
it is preferable to reserve hugepages at boot time,
as that will help in preventing physical memory from becoming heavily fragmented.

To reserve hugepages at boot time, a parameter is passed to the Linux kernel on the kernel command line.

For 2 MB pages, just pass the hugepages option to the kernel. For example, to reserve 1024 pages of 2 MB, use::
Expand Down Expand Up @@ -186,35 +214,29 @@ the number of hugepages reserved at boot time is generally divided equally betwe

See the Documentation/admin-guide/kernel-parameters.txt file in your Linux source tree for further details of these and other kernel options.

**Alternative:**

For 2 MB pages, there is also the option of allocating hugepages after the system has booted.
This is done by echoing the number of hugepages required to a nr_hugepages file in the ``/sys/devices/`` directory.
For a single-node system, the command to use is as follows (assuming that 1024 pages are required)::

echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

On a NUMA machine, pages should be allocated explicitly on separate nodes::

echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
Using Hugepages with the DPDK
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. note::
If secondary process support is not required, DPDK is able to use hugepages
without any configuration by using "in-memory" mode.
Please see :doc:`linux_eal_parameters` for more details.

For 1G pages, it is not possible to reserve the hugepage memory after the system has booted.
If secondary process support is required,
mount points for hugepages need to be created.
On modern Linux distributions, a default mount point for hugepages
is provided by the system and is located at ``/dev/hugepages``.
This mount point will use the default hugepage size
set by the kernel parameters as described above.

Using Hugepages with the DPDK
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
However, in order to use hugepage sizes other than the default, it is necessary
to manually create mount points for those hugepage sizes (e.g. 1GB pages).

Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps::
To make the hugepages of size 1GB available for DPDK use,
following steps must be performed::

mkdir /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
mount -t hugetlbfs pagesize=1GB /mnt/huge

The mount point can be made permanent across reboots, by adding the following line to the ``/etc/fstab`` file::

nodev /mnt/huge hugetlbfs defaults 0 0

For 1GB pages, the page size must be specified as a mount option::

nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0
nodev /mnt/huge hugetlbfs pagesize=1GB 0 0

0 comments on commit 1c361d2

Please sign in to comment.