Skip to content
This repository has been archived by the owner on Feb 7, 2024. It is now read-only.

Commit

Permalink
Update benchmarking notebook and figures
Browse files Browse the repository at this point in the history
- Fix link to it from introduction.ipynb
- Change figure titles to begin with unique string rather than MOUS code
- Reference updated figure names in notebook for accurate render
  • Loading branch information
amcnicho committed Apr 15, 2021
1 parent e1ede2d commit 67391f9
Show file tree
Hide file tree
Showing 8 changed files with 7 additions and 7 deletions.
12 changes: 6 additions & 6 deletions docs/benchmarking.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X133d_X1a36_chans_10_DATA_repr.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X133d_X1a36_chans_10.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X133d_X1a36.png)\n",
"\n",
Expand All @@ -136,7 +136,7 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X1273_Xc66_chans_16_DATA_repr.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X1273_Xc66_chans_16.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X1273_X2e3.png)\n",
"\n",
Expand All @@ -163,7 +163,7 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X1273_X2e3_chans_45_DATA_repr.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X1273_X2e3_chans_45.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X1273_X2e3.png)\n",
"\n",
Expand All @@ -190,7 +190,7 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X12a3_X3be_chans_48_DATA_repr.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X12a3_X3be_chans_48.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X12a3_X3be.png)\n",
"\n",
Expand Down Expand Up @@ -264,7 +264,7 @@
"The time spent in each function `<plots>`\n",
"\n",
"The communication of data between workers constituted a relatively small proportion of the total runtime, and the distribution of data between workers was relatively uniform, at all horizontal scalings, with some hot spots beginning to present once tens of nodes were involved. This is demonstrated by the following figure, taken from the performance report of a representative test execution:\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X12a3_X3be_threads_256_chans_48_worker_bandwidth.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/bandwidth_A001_X12a3_X3be_threads_256_chans_48.png)\n",
"\n",
"The time overhead associated with graph creation and task scheduling (approximately 100 ms per task for dask) grew as more nodes were introduced until eventually coming to represent a fraction of total execution time comparable to the computation itself, especially in the test cases with smaller data.\n"
]
Expand All @@ -279,7 +279,7 @@
"\n",
"The total runtime curves for tests run on AWS show higher variance. One contributing factor that likely dominated this effect was the use of [preemptible instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html) underlying the compute nodes running the worker configuration. For this same reason, some cloud-based test runs show decreased performance with increased scale. This is due to the preemption of nodes and associated redeployment by kubernetes, which sometimes constituted a large fraction of the total test runtime, as demonstrated by the task stream for the following test case. Note the horizontal white bar (signifying to tasks executed) shortly after graph execution begins, as well as some final tasks being assigned to a new node that came online after a few minutes (represented by the new \"bar\" of 8 rows at top right) in the following figure:\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X1273_Xc66_threads_40_chans_45_task_stream.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/task_stream_A001_X1273_Xc66_threads_40_chans_45.png)\n",
"\n",
"Qualitatively, failure rates were higher during tests of CASA on local HPC infrastructure than they were using dask on the cluster or cloud. The cube refactor shows a noticeable improvement in this area, but still worse than the prototype."
]
Expand Down
2 changes: 1 addition & 1 deletion docs/introduction.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@
"[Visibility](visibilities.ipynb) and [Image](images.ipynb) overview sections to demonstrate the simplicity and scalability of the \n",
"technology choices. Notional examples of [Calibration](calibration.rst), [Flagging](flagging.ipynb) and [Imaging](imaging.rst) \n",
"are provided to illustrate future design and implementation direction. Finally, the most compute intensive areas of CASA imaging are \n",
"implemented and [benchmarked](benchmark.ipynb) to demonstrate the parallel scalability and raw performance now possible from a \n",
"implemented and [benchmarked](benchmarking.ipynb) to demonstrate the parallel scalability and raw performance now possible from a \n",
"pure-Python software stack. \n",
"\n",
"A detailed explanation of technology choices, including the xarray and dask frameworks, the zarr storage format, and the functional \n",
Expand Down

0 comments on commit 67391f9

Please sign in to comment.