Visuco - A Visual Analytics System for Optimizing Communications in Massively Parallel Applications

About

• Visual analytics software for identifying bottlenecks on prallel communications used MPI and finding better communication options

• This project is a work for the paper below.
  A Visual Analytics System for Optimizing Communications in Massively Parallel Applications
  Takanori Fujiwara, Preeti Malakar, Venkatram Vishwanath, Khairi Reda, Michael E. Papka, and Kwan-Liu Ma
  In Proceedings of IEEE Symposium on Visual Analytics Science and Technology (VAST), 2017.

• Features

  • Enable to visualize large scale communications (8,000 computers nodes and more)
  • Three different type of views to identify communication bottlenecks
  • Alternative better route suggestion

• Dataset

  • Dataset is available from https://github.com/takanori-fujiwara/par-comm-data

Requirements

• Note: Tested on macOS Ventura and Ubuntu 22.0.4 LTS.
• XCode (latest) for macOS. build-essential (latest) for Ubuntu.
• Qt5 (latest)
• (Follows are required if you want to generate new graph data.)
• Python 3
• graph-tool (latest)
• R (latest)

Installation

Mac OS

• Install latest XCode from Mac App store

• After finishing installation of XCode, launch XCode at once to verify XCode.

• Install xcode-select with a command below.

  xcode-select --install

  After this command, follow the instruction from XCode.

• Install Qt. If you use home brew(http://brew.sh) use the commands below

  brew update

  brew install qt5

• Check qmake version.

  qmake -v

  If the version is not qt5, use the command below.

  brew link --force --overwrite qt5

Follows are in case you want to generate new graph data (there are sample outputs in ./sample_analysis_data/)

• Install R and R packages.

  brew install r

  Check R runs the same version installed above.

  which R

  If this does not return /usr/local/bin/R, add export PATH=/usr/local/bin:$PATH in .bash_profile. Also, try to brew unlink r && brew link r.

  R (run R in terminal)

  >>> install.packages(c("sna", "argparse", "qap"))

• If you have not installed python, install python

  brew install python3

  Check python3 runs the same version installed above.

  which python3

  If this does not return /usr/local/bin/python3, add export PATH=/usr/local/bin:$PATH in .bash_profile. Also, try to brew unlink python3 && brew link python3.

• Install graph-tool. See this requirements and installation. instruction.

  • After installing graph-tool, check whether it works or not.

    python3 (run python3 in terminal)

    >>> from graph_tool.all import *

Building the software

• Build the software with the commands below.

  cd ./src/qtCode/visuco

  make clean

  rm visuco.pro.user (if there is a .user file)

  qmake visuco.pro -spec macx-clang

  make

• Launch "visuco.app" generated by the above build process in the same directory.

Ubuntu

• Install g++.

  sudo apt update

  sudo apt install -y build-essential

• Install Qt5 (https://wiki.qt.io/Install_Qt_5_on_Ubuntu).

  sudo apt install -y qtbase5-dev qt5-qmake cmake

• Install fonts and OpenGL.

  sudo apt install -y libfontconfig1

  sudo apt install -y mesa-common-dev

  sudo apt install -y libglu1-mesa-dev

Follows are in case you want to generate new graph data (there are sample outputs in ./sample_analysis_data/)

• Install R and R packages.

  sudo apt install -y r-base

• If you have not installed python3, install python3 and python3-pip

  sudo apt install -y python3.xxx (xxx: python version you want to install) sudo apt install -y python3-pip

• Install graph-tool. See this requirements and installation. instruction.

Building the software

• Build the software with the commands below.

  cd ./src/qtCode/visuco

  make clean

  qmake visuco.pro -r -spec linux-g++-64

  make

• Launch "visuco" application generated by the above build process in the same directory.

---

Initial Settings

• Launch "visuco.app" or "visuco" in the built directory.
• Select "Preferences" in the menu bar and set paths for python and RScript commands.

---

Getting Started

• Select "Generate Analysis Data" from "File" menu.

• Set each setting and press "OK".

  • From "sample_data" directory, select "IMB-MPI1_n32_c1_routes.txt" as a route file, "IMB-MPI1_n32_mapping.txt" as a mapping file, and "IMB-MPI1_n32_c1_hopbytes" as a hopbyte file.
  • Set "1" as a number of cores, "256" as thres # of nodes to start aggregation.
  • Select an output directory where you want.

• Select "Load Analysis Data" from "File" menu.

• Select the directory you generated in the "Generate new analysis data" step.

• After the steps above, you can see the visualized result in each view.

Generate new analysis data including graph information from route files and mappind data

• Select "Generate Analysis Data" from "File" menu.

• Select a route file (TXT) that you want to analyze. A route file must be the same format with an output file from bgqroute (https://github.com/pmalakar/bgqroute).

  For example,

    Hop 1: [6-12] 6 (0 0 1 1 0 0) -> 4 (0 0 1 0 0 0)
    Hop 2: [6-12] 4 (0 0 1 0 0 0) -> 12 (0 1 1 0 0 0)
    ...
  

• Select a matrix mapping file (TXT file with space separation). This is used for mapping the coordinates of ranks to the coordinates in the matrix view.

  The format must be

    1st line: size of each dimensions
    2nd line: Permutation order in the matrix view as you want to use.
    3rd line: Network topology type. Currently, we support only "5dtorus" and "theta". Select one of them.
  

  For example, if you use 5D torus with 4x2x4x8x2 sizes (each number is cooresponding to A,B,C,D,E dimension) and EDCBA order

    4 2 4 8 2
    4 3 2 1 0
    5dtorus
  

• (Optional) Select a hop-byte file (TXT file with space separation). This is used when the system suggests alternative routes or mapping.

  The format must be

    Route Source,Destination,Message Size,Number of Hops
  

  For example,

    4 10 342321 3
  

• Set number of cores according to your environment that used for obtaining the route file.

• Set a threshold for the number of nodes, which is used for graph partitioning and aggregating the node.

• Finally, set an output directory as you want and push "OK". Then, wait for a while until finishing running all scripts. (We have a plan for implementing notification of finishing running.)

---

Load generated new analysis data

• Select "Load Analysis Data" from "File" menu.
• Select the directory you generated in the "Generate new analysis data" step.

---

Interactions

• (Work in progress)

How to Cite

Please, cite:

• Takanori Fujiwara, Preeti Malakar, Venkatram Vishwanath, Khairi Reda, Michael E. Papka, and Kwan-Liu Ma, "A Visual Analytics System for Optimizing Communications in Massively Parallel Applications." In Proceedings of IEEE Symposium on Visual Analytics Science and Technology (VAST), 2017.