Skip to content
This repository has been archived by the owner on Feb 7, 2024. It is now read-only.

Commit

Permalink
Update benchmarking.ipynb
Browse files Browse the repository at this point in the history 
Remove titles from image link markdown
  • Loading branch information
@amcnicho
amcnicho committed Apr 15, 2021
1 parent 3821b3f commit e1ede2d
Showing 1 changed file with 19 additions and 21 deletions.
40 changes: 19 additions & 21 deletions docs/benchmarking.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -109,13 +109,13 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![A001_X133d_X1a36_chans_10_DATA_repr.png](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X133d_X1a36_chans_10_DATA_repr.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X133d_X1a36_chans_10_DATA_repr.png)\n",
"\n",
"![node_A001_X133d_X1a36](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X133d_X1a36.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X133d_X1a36.png)\n",
"\n",
"![cluster_A001_X133d_X1a36](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X133d_X1a36.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X133d_X1a36.png)\n",
"\n",
"![aws_A001_X133d_X1a36](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X133d_X1a36.png)\n"
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X133d_X1a36.png)\n"
]
},
{
Expand All @@ -136,13 +136,13 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![A001_X1273_Xc66_chans_16_DATA_repr.png](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X1273_Xc66_chans_16_DATA_repr.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X1273_Xc66_chans_16_DATA_repr.png)\n",
"\n",
"![node_A001_X1273_Xc66](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X1273_X2e3.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X1273_X2e3.png)\n",
"\n",
"![cluster_A001_X1273_Xc66](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X1273_Xc66.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X1273_Xc66.png)\n",
"\n",
"![aws_A001_X1273_Xc66](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X1273_Xc66.png)"
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X1273_Xc66.png)"
]
},
{
Expand All @@ -163,13 +163,13 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![A001_X1273_X2e3_chans_45_DATA_repr.png](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X1273_X2e3_chans_45_DATA_repr.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X1273_X2e3_chans_45_DATA_repr.png)\n",
"\n",
"![node_A001_X1273_X2e3](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X1273_X2e3.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X1273_X2e3.png)\n",
"\n",
"![cluster_A001_X1273_Xc66](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X1273_X2e3.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X1273_X2e3.png)\n",
"\n",
"![aws_A001_X1273_X2e3](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X1273_X2e3.png)\n"
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X1273_X2e3.png)\n"
]
},
{
Expand All @@ -190,15 +190,13 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![A001_X12a3_X3be_chans_48_DATA_repr.png](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X12a3_X3be_chans_48_DATA_repr.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X12a3_X3be_chans_48_DATA_repr.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X12a3_X3be.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X12a3_X3be.png)\n",
"\n",
"![node_A001_X12a3_X3be](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X12a3_X3be.png)\n",
"\n",
"![cluster_A001_X12a3_X3be](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X12a3_X3be.png)\n",
"\n",
"![aws_A001_X12a3_X3be](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X12a3_X3be.png)\n"
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X12a3_X3be.png)\n"
]
},
{
Expand All @@ -222,7 +220,7 @@
"\n",
"There does not appear to be a performance penalty associated with the adoption of a pure Python framework in comparison to the compiled C++/Fortran reference implementation. This is likely due in large part to the prototype's reliance on the `numba` Just-In-Time (JIT) transpiler and the C foreign function interface relied on by third-party framework packages including `numpy` and `scipy`.\n",
"\n",
"The Fortran gridding code in CASA appears slightly more efficient than the `numba` JIT-compiled Python code in the prototype. However,  the test implementation more efficiently handles chunked data and does not have intermediate steps where data is written to disk, whereas CASA generates TempLattice files to store intermediate files."
"The Fortran gridding code in CASA appears slightly more efficient than the JIT-decorated Python code in the prototype. However, the test implementation more efficiently handles chunked data and does not have intermediate steps where data is written to disk, whereas CASA generates TempLattice files to store intermediate files."
]
},
{
Expand Down Expand Up @@ -266,7 +264,7 @@
"The time spent in each function `<plots>`\n",
"\n",
"The communication of data between workers constituted a relatively small proportion of the total runtime, and the distribution of data between workers was relatively uniform, at all horizontal scalings, with some hot spots beginning to present once tens of nodes were involved. This is demonstrated by the following figure, taken from the performance report of a representative test execution:\n",
"![A001_X12a3_X3be_threads_256_chans_48_worker_bandwidth.png](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X12a3_X3be_threads_256_chans_48_worker_bandwidth.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X12a3_X3be_threads_256_chans_48_worker_bandwidth.png)\n",
"\n",
"The time overhead associated with graph creation and task scheduling (approximately 100 ms per task for dask) grew as more nodes were introduced until eventually coming to represent a fraction of total execution time comparable to the computation itself, especially in the test cases with smaller data.\n"
]
Expand All @@ -281,7 +279,7 @@
"\n",
"The total runtime curves for tests run on AWS show higher variance. One contributing factor that likely dominated this effect was the use of [preemptible instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html) underlying the compute nodes running the worker configuration. For this same reason, some cloud-based test runs show decreased performance with increased scale. This is due to the preemption of nodes and associated redeployment by kubernetes, which sometimes constituted a large fraction of the total test runtime, as demonstrated by the task stream for the following test case. Note the horizontal white bar (signifying to tasks executed) shortly after graph execution begins, as well as some final tasks being assigned to a new node that came online after a few minutes (represented by the new \"bar\" of 8 rows at top right) in the following figure:\n",
"\n",
"![A001_X1273_Xc66_threads_40_chans_45_task_stream](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X1273_Xc66_threads_40_chans_45_task_stream.png)\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/A001_X1273_Xc66_threads_40_chans_45_task_stream.png)\n",
"\n",
"Qualitatively, failure rates were higher during tests of CASA on local HPC infrastructure than they were using dask on the cluster or cloud. The cube refactor shows a noticeable improvement in this area, but still worse than the prototype."
]
Expand Down

0 comments on commit e1ede2d

Please sign in to comment.