Skip to content

Commit

Permalink
Update the documentation for the LCI parcelport.
Browse files Browse the repository at this point in the history
  • Loading branch information
JiakunYan authored and hkaiser committed Jul 10, 2023
1 parent 793e5a6 commit bf05f03
Showing 1 changed file with 21 additions and 5 deletions.
26 changes: 21 additions & 5 deletions docs/sphinx/manual/using_the_lci_parcelport.rst
Original file line number Diff line number Diff line change
Expand Up @@ -97,13 +97,29 @@ If you are using ``mpirun`` or ``srun``, you can just pass
``--hpx:ini=hpx.parcel.lci.priority=1000``, ``--hpx:ini=hpx.parcel.lci.enable=1``, and
``--hpx:ini=hpx.parcel.bootstrap=lci`` to the |hpx| applications.

If you are running on a Cray machine, you need to pass `--mpi=pmix` or `--mpi=pmi2` to srun
to enable the PMIx or PMI2 support of SLURM since LCI does not support the default Cray PMI.
For example,

.. code-block:: shell-session
$ srun --mpi=pmix [hpx application]
.. _tune_lci_pp:

Performance tuning of the LCI parcelport
========================================

We encourage users to increase the zero-copy serialization threshold of LCI to ``8192`` (bytes) by
either the cmake variable ``-DHPX_WITH_ZERO_COPY_SERIALIZATION_THRESHOLD=8192``
(this will affect all the parcelports) or the command line option
``--hpx:ini=hpx.parcel.lci.zero_copy_serialization_threshold=8192``
(this will only affect the LCI parcelport).
We encourage users to set the following environmental variables
when using the LCI parcelport to get better performance.

.. code-block:: shell-session
$ export LCI_SERVER_MAX_SENDS=1024
$ export LCI_SERVER_MAX_RECVS=4096
$ export LCI_SERVER_NUM_PKTS=65536
$ export LCI_SERVER_MAX_CQES=65536
$ export LCI_PACKET_SIZE=12288
This setting needs roughly 800MB memory per process. The memory consumption mainly
comes from the packets, which can be calculated using `LCI_SERVER_NUM_PKTS x LCI_PACKET_SIZE`.

0 comments on commit bf05f03

Please sign in to comment.