EduHPX_0.0.3

EduHPX 0.0.3

EduHPX shows the parcels of an HPX program and the MPI messages below them on one timeline. Version 0.0.3 adds a 3D view of the collective trees with the nodes of your cluster, the measuring MPI as a download, and a complete guide from the installs to the first trace of your own program.

What is new

3D tree with the nodes of the cluster

A new 3D tree button next to Tree shows a collective as a tree in 3D, laid out like the 3D cluster view of the EduMPI GUI.

• Every node of the cluster is a pale glass cube (the first node bottom left), and the localities that run on it are small numbered cubes inside it.
• Green lines are sends, red lines are receives, thick for data and thin for requests; the Send and Recv buttons switch each kind on and off.
• Colours of the localities, as in the EduMPI GUI: send/recv ratio (green = sent more, red = received more, olive = about the same, white = no communication), max send ratio (white to green), max recv ratio (white to red), and arrival of the data (green first, red last).
• Click a cube to look at it: the camera flies to it, and a box shows what the locality sent and received, when its data arrived, and its parent and children. Scroll out (or Back to overview) to return. Drag turns the scene, wheel and pinch zoom, hover shows a line's size and whether it stays in the node or goes over the network.
• The critical path is shown as a gold band; the other lines fade.
• The run file now records the node of every locality (locality_nodes in its header, written by eduhpx-merge). Older run files still open: the localities are then assumed to be spread evenly over the nodes. All 64 built-in examples were merged again with the node list; their messages are unchanged.
• In both tree views the summary is lighter: the split of the time into waiting, HPX and MPI and the colour legend of the lanes are only shown in the lane views.

The 3D view needs the Qt module Qt Quick 3D and a graphics driver for it. If it is missing, only that view shows a note; everything else works.

Install and trace your own program

• The measuring MPI is a download now: eduhpx_mpi_data-5.0.11rc1-eduhpx1.tar.gz (Open MPI 5.0.11rc1 with the EduMPI hooks, without database) is attached to this release with its SHA-256 checksum.
• scripts/install-mpi.sh checks the checksum, builds the tarball from the unpacked archive, installs it, and runs a small test (4 ranks, the log files are read back).
• Tracing guide: install the measuring MPI, install HPX with the parcel log, build your program against it (with the new sample-app/ as a model), run it with eduhpx-run (also as a Slurm job), open the result, and what to do if something does not work.
• The measuring MPI is cheaper: its event store keeps small records in a few cache lines and reuses its buffers. On the broadcast benchmark (64 KiB, 4 localities on one node) the overhead against stock Open MPI went from +12.4 % to +8.9 % (flat) and from +9.7 % to +7.0 % (hierarchical). The log files and their format are unchanged.
• The README has a new section Overhead: what the measuring MPI and the parcel log cost, and what that means for reading a trace.

HPX: a newer commit, much cheaper hooks

• install-hpx.sh now builds HPX master ec3b59b (the commit with HPX PR 8001, which switches the lock-event tracing off by default) and applies the parcel log patch ported to it (patches/hpx-ec3b59b1.patch). The Tracy compile switch that carries the hooks costs about 4 % instead of 14 to 17 %; with the parcel log on, the stack is about 13 % slower than pure HPX instead of 22 to 26 % (one node, 4 localities, 64 KiB broadcast). Programs have to be built again against this HPX. The old patch stays in the repository, because the examples were made with it.

Viewer

• The steps of one call belong together. A hierarchical all-to-all is a gather, an exchange between the roots and a scatter; all-reduce is a reduce and a broadcast, all-gather a gather and a broadcast, a scan a gather and a scatter. The viewer now names them as steps of one call (All-to-all 2 of 10, step 3 of 3: scatter), joins them under the strip, and has buttons to jump between the steps of the same call.
  This builds on the first outside contribution to EduHPX, pull request #1 by iemAnshuman: keep the phases of a hierarchical all-to-all together in the strip. Thank you! The pull request ordered the phases of the all-to-all; the viewer now does the same for all collectives that are made of several steps (a call is only recognised if its groups follow each other within 0.5 ms, so separate calls with the same number are not joined).
• The README says what HPX's own Tracy integration shows of parcels (tasks, threads and the parcels of each locality, but not the MPI messages below them), and the screenshots were made again from the re-merged examples.

Upgrading

• Run files of 0.0.2 open without change.
• To trace again with this version: install the new measuring MPI (install-mpi.sh) and the new HPX (install-hpx.sh), and build your program against that HPX. The format of the MPI log files did not change.

Known limits

• The measuring chain was tested on one Linux cluster (Slurm, Open MPI with UCX). macOS as a measuring system, other launchers than mpirun, and runs without Slurm were not tested.
• The macOS app is not signed (see the README for the first start).
• Only parcels that go over the MPI parcelport have an MPI message below them.

Files

• Linux: EduHPX_0.0.3-x86_64.AppImage (make it executable, then start it)
• macOS: EduHPX_0.0.3.dmg (if macOS blocks the app: see the README of the repository, section macOS)
• Windows: EduHPX_0.0.3-windows.zip (unpack, start eduhpx.exe)
• EduHPX_0.0.3_examples.zip: the example run files (they are also built into the app: File > Examples)
• eduhpx_mpi_data-5.0.11rc1-eduhpx1.tar.gz and eduhpx_mpi_data-5.0.11rc1-eduhpx1.tar.gz.sha256: the measuring MPI (source), to trace your own program: see the tracing guide