307 lines (228 sloc) 10.1 KB
UNIX ``ulimit`` Settings
.. default-domain:: mongodb
.. contents:: On this page
:backlinks: none
:depth: 1
:class: singlecol
Most UNIX-like operating systems, including Linux and OS X, provide
ways to limit and control the usage of system resources such as
threads, files, and network connections on a per-process and per-user
basis. These "ulimits" prevent single users from using too many system
resources. Sometimes, these limits have low default values that can
cause a number of issues in the course of normal MongoDB operation.
.. note::
Red Hat Enterprise Linux and CentOS 6 place a max process
limitation of 1024 which overrides ``ulimit`` settings. Create a
file named ``/etc/security/limits.d/99-mongodb-nproc.conf`` with
new ``soft nproc`` and ``hard nproc`` values to increase the
process limit. See ``/etc/security/limits.d/90-nproc.conf`` file as
an example.
.. _system-resource-utilization:
Resource Utilization
:program:`mongod` and :program:`mongos` each use threads and file
descriptors to track connections and manage internal operations. This
section outlines the general resource utilization patterns for MongoDB.
Use these figures in combination with the actual information about your
deployment and its use to determine ideal ``ulimit`` settings.
Generally, all :program:`mongod` and :program:`mongos` instances:
- track each incoming connection with a file descriptor *and* a
- track each internal thread or *pthread* as a system process.
- 1 file descriptor for each data file in use by the
:program:`mongod` instance.
- 1 file descriptor for each journal file used by the
:program:`mongod` instance when :setting:`storage.journal.enabled` is ``true``.
- In replica sets, each :program:`mongod` maintains a connection to
all other members of the set.
:program:`mongod` uses background threads for a number of internal
processes, including :ref:`TTL collections <ttl-collections>`,
replication, and replica set health checks, which may require a small
number of additional resources.
.. _mongos-connection-use information:
In addition to the threads and file descriptors for client connections,
:program:`mongos` must maintain connections to all config servers and
all shards, which includes all members of all replica sets.
For :program:`mongos`, consider the following behaviors:
- :program:`mongos` instances maintain a connection pool to each shard
so that the :program:`mongos` can reuse connections and quickly
fulfill requests without needing to create new connections.
- You can limit the number of incoming connections using
the :setting:`net.maxIncomingConnections` run-time option.
By restricting the number of incoming connections you can prevent a
cascade effect where the :program:`mongos` creates too many
connections on the :program:`mongod` instances.
.. include:: /includes/note-max-conns-max.rst
Review and Set Resource Limits
You can use the ``ulimit`` command at the system prompt to check
system limits, as in the following example:
.. code-block:: sh
$ ulimit -a
-t: cpu time (seconds) unlimited
-f: file size (blocks) unlimited
-d: data seg size (kbytes) unlimited
-s: stack size (kbytes) 8192
-c: core file size (blocks) 0
-m: resident set size (kbytes) unlimited
-u: processes 192276
-n: file descriptors 21000
-l: locked-in-memory size (kb) 40000
-v: address space (kb) unlimited
-x: file locks unlimited
-i: pending signals 192276
-q: bytes in POSIX msg queues 819200
-e: max nice 30
-r: max rt priority 65
-N 15: unlimited
``ulimit`` refers to the per-*user* limitations for various
resources. Therefore, if your :program:`mongod` instance executes as a
user that is also running multiple processes, or multiple
:program:`mongod` processes, you might see contention for these
resources. Also, be aware that the ``processes`` value (i.e. ``-u``)
refers to the combined number of distinct processes and sub-process
You can change ``ulimit`` settings by issuing a command in the
following form:
.. code-block:: sh
ulimit -n <value>
There are both "hard" and the "soft" ``ulimit``\ s that affect MongoDB's
performance. The "hard" ``ulimit`` refers to the maximum number of
processes that a user can have active at any time. This is the
ceiling: no non-root process can increase the "hard" ``ulimit``. In
contrast, the "soft" ``ulimit`` is the limit that is actually
enforced for a session or process, but any process can increase it
up to "hard" ``ulimit`` maximum.
A low "soft" ``ulimit`` can cause ``can't create new thread,
closing connection`` errors if the number of connections
grows too high. For this reason, it is extremely important to set
*both* ``ulimit`` values to the recommended values.
``ulimit`` will modify both "hard" and "soft" values unless the
:setting:`-H` or :setting:`-S` modifiers are specified when
modifying limit values.
For many distributions of Linux you can change values by substituting
the ``-n`` option for any possible value in the output of ``ulimit
-a``. On OS X, use the ``launchctl limit`` command. See your
operating system documentation for the precise procedure for changing
system limits on running systems.
After changing the ``ulimit`` settings, you *must* restart the
process to take advantage of the modified settings. You can use the
``/proc`` file system to see the current limitations on a running
Depending on your system's configuration, and default settings, any
change to system limits made using ``ulimit`` may revert following
system a system restart. Check your distribution and operating
system documentation for more information.
.. include:: /includes/note-suse-ulimit.rst
.. _recommended-ulimit-settings:
Recommended ``ulimit`` Settings
Every deployment may have unique requirements and settings; however,
the following thresholds and settings are particularly important for
:program:`mongod` and :program:`mongos` deployments:
- ``-f`` (file size): ``unlimited``
- ``-t`` (cpu time): ``unlimited``
- ``-v`` (virtual memory): ``unlimited`` [#memory-size]_
- ``-n`` (open files): ``64000``
- ``-m`` (memory size): ``unlimited`` [#memory-size]_ [#rss-linux]_
- ``-u`` (processes/threads): ``64000``
Always remember to restart your :program:`mongod` and
:program:`mongos` instances after changing the ``ulimit`` settings to
ensure that the changes take effect.
Linux distributions using Upstart
For Linux distributions that use Upstart, you can specify limits
within service scripts if you start :program:`mongod` and/or
:program:`mongos` instances as Upstart services. You can do this by
using ``limit`` `stanzas <>`_.
Specify the :ref:`recommended-ulimit-settings`, as in the following
.. code-block:: sh
limit fsize unlimited unlimited # (file size)
limit cpu unlimited unlimited # (cpu time)
limit as unlimited unlimited # (virtual memory size)
limit nofile 64000 64000 # (open files)
limit nproc 64000 64000 # (processes/threads)
Each ``limit`` stanza sets the "soft" limit to the first value specified and the "hard"
limit to the second.
After changing ``limit`` stanzas, ensure that the changes take
effect by restarting the application services, using the following
.. code-block:: sh
restart <service name>
Linux distributions using ``systemd``
For Linux distributions that use ``systemd``, you can specify
limits within the ``[Service]`` sections of service scripts if you
start :program:`mongod` and/or
:program:`mongos` instances as ``systemd`` services. You can do this
by using `resource limit directives
Specify the :ref:`recommended-ulimit-settings`, as in the following
.. code-block:: sh
# Other directives omitted
# (file size)
# (cpu time)
# (virtual memory size)
# (open files)
# (processes/threads)
Each ``systemd`` limit directive sets both the "hard" and "soft" limits to the value
After changing ``limit`` stanzas, ensure that the changes take
effect by restarting the application services, using the following
.. code-block:: sh
systemctl restart <service name>
.. _proc-file-system:
``/proc`` File System
.. note::
This section applies only to Linux operating systems.
The ``/proc`` file-system stores the per-process limits in the
file system object located at ``/proc/<pid>/limits``, where ``<pid>``
is the process's :term:`PID` or process identifier. You can use the
following ``bash`` function to return the content of the ``limits``
object for a process or processes with a given name:
.. code-block:: sh
for process in $@; do
process_pids=`ps -C $process -o pid --no-headers | cut -d " " -f 2`
if [ -z $@ ]; then
echo "[no $process running]"
for pid in $process_pids; do
echo "[$process #$pid -- limits]"
cat /proc/$pid/limits
You can copy and paste this function into a current shell session or
load it as part of a script. Call the function with one the following
.. code-block:: sh
return-limits mongod
return-limits mongos
return-limits mongod mongos
.. [#memory-size] If you limit virtual or resident memory size on a
system running MongoDB the operating system will refuse to honor
additional allocation requests.
.. [#rss-linux] The ``-m`` parameter to ``ulimit`` has no effect on
Linux systems with kernel versions more recent than 2.4.30. You may
omit ``-m`` if you wish.