Skip to content
This repository has been archived by the owner on Feb 7, 2024. It is now read-only.

Commit

Permalink
fix formatting
Browse files Browse the repository at this point in the history
  • Loading branch information
ryanraba committed Apr 15, 2021
1 parent e99fab8 commit f23225f
Showing 1 changed file with 13 additions and 14 deletions.
27 changes: 13 additions & 14 deletions docs/benchmarking.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
"id": "0YnE3V8l-vy5"
},
"source": [
"# Benchmarking"
"# Benchmarks"
]
},
{
Expand All @@ -31,14 +31,15 @@
"source": [
"## Introduction\n",
"\n",
"As CASA next-gen development proceeds, applications designed to measure a suite of agreed-upon benchmarks may eventually prove useful or necessary. For this proof of concept, performance benchmarking and profiling were carried out on an ad hoc basis throughout development and then, after the functionality under demonstration stabilized, more formally against a small set of data taken to be generally representative of important imaging use cases. This notebook contains no executable code cells, but serves to illustrate the process and results following the most recent round of performance testing.\n",
"As CASA next-gen development proceeds, applications designed to measure a suite of agreed-upon benchmarks may eventually prove useful or necessary. For this proof of concept, performance benchmarking and profiling were carried out on an ad hoc basis throughout development and then, after the functionality under demonstration stabilized, more formally against a set of data taken to be generally representative. This notebook contains no executable code cells, but serves to illustrate the process and results following the most recent round of performance testing.\n",
"\n",
"- Description of test methodology and datasets\n",
"- Per-dataset test results\n",
"- Discussion and analysis of results\n",
"- Reference/publish parquet files with timing data?\n",
" - AWS configuration \n",
"- Earlier single-machine and cloud tests, [memory profiling](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/bench_top.png) **if we decide to include this**\n"
"- Appendices?\n",
" - Reference/publish parquet files with timing data?\n",
" - AWS configuration **possibly better as an update to Installation section of readthedocs**\n",
" - Earlier single-machine and cloud tests, [memory profiling](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/bench_top.png) **if we decide to include this**\n"
]
},
{
Expand Down Expand Up @@ -97,7 +98,7 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![repr_01091](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X133d_X1a36_chans_10.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X133d_X1a36_chans_10.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X133d_X1a36.png)\n",
"\n",
Expand All @@ -124,7 +125,7 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![repr_00271](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X1273_Xc66_chans_16.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X1273_Xc66_chans_16.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X1273_X2e3.png)\n",
"\n",
Expand All @@ -151,7 +152,7 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![repr_00717](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X1273_X2e3_chans_45.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X1273_X2e3_chans_45.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X1273_X2e3.png)\n",
"\n",
Expand All @@ -178,7 +179,7 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![repr_00983](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X12a3_X3be_chans_48.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X12a3_X3be_chans_48.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X12a3_X3be.png)\n",
"\n",
Expand Down Expand Up @@ -252,7 +253,7 @@
"The time spent in each function `<plots>`\n",
"\n",
"The communication of data between workers constituted a relatively small proportion of the total runtime, and the distribution of data between workers was relatively uniform, at all horizontal scalings, with some hot spots beginning to present once tens of nodes were involved. This is demonstrated by the following figure, taken from the performance report of a representative test execution:\n",
"![bandwidth_00983](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/bandwidth_A001_X12a3_X3be_threads_256_chans_48.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/bandwidth_A001_X12a3_X3be_threads_256_chans_48.png)\n",
"\n",
"The time overhead associated with graph creation and task scheduling (approximately 100 ms per task for dask) grew as more nodes were introduced until eventually coming to represent a fraction of total execution time comparable to the computation itself, especially in the test cases with smaller data.\n"
]
Expand All @@ -279,8 +280,8 @@
},
"source": [
"### Configuration of computing resources\n",
"\n"
"Dask profiling data were collected using the `[performance_report](https://distributed.dask.org/en/latest/diagnosing-performance.html#performance-reports)` context manager in tests run both on-premises and in the commercial cloud.\n",
"\n",
"Dask profiling data were collected using the [`performance_report`](https://distributed.dask.org/en/latest/diagnosing-performance.html#performance-reports) function in tests run both on-premises and in the commercial cloud.\n",
"\n",
"Some values of the [distributed configuration](https://distributed.dask.org/en/latest/worker.html) were modified from their defaults:\n",
"```\n",
Expand Down Expand Up @@ -326,8 +327,6 @@
" type: text/x-shellscript\n",
" image: \n",
"```\n",
"**full configuration details are possibly better presented via expandable subset of Installation section of readthedocs**\n"
"\n",
" "
]
},
Expand Down

0 comments on commit f23225f

Please sign in to comment.