Skip to content
This repository has been archived by the owner on Feb 7, 2024. It is now read-only.

Commit

Permalink
add names to image links
Browse files Browse the repository at this point in the history
  • Loading branch information
@ryanraba
ryanraba committed Apr 15, 2021
1 parent f23225f commit 5b3d4d7
Showing 1 changed file with 18 additions and 18 deletions.
36 changes: 18 additions & 18 deletions docs/benchmarking.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -98,13 +98,13 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X133d_X1a36_chans_10.png)\n",
"![data_repr_A001_X133d_X1a36_chans_10](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X133d_X1a36_chans_10.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X133d_X1a36.png)\n",
"![node_A001_X133d_X1a36](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X133d_X1a36.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X133d_X1a36.png)\n",
"![cluster_A001_X133d_X1a36](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X133d_X1a36.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X133d_X1a36.png)\n"
"![aws_A001_X133d_X1a36](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X133d_X1a36.png)\n"
]
},
{
Expand All @@ -125,13 +125,13 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X1273_Xc66_chans_16.png)\n",
"![data_repr_A001_X1273_Xc66_chans_16](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X1273_Xc66_chans_16.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X1273_X2e3.png)\n",
"![node_A001_X1273_X2e3](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X1273_X2e3.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X1273_Xc66.png)\n",
"![cluster_A001_X1273_Xc66](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X1273_Xc66.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X1273_Xc66.png)"
"![aws_A001_X1273_Xc66](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X1273_Xc66.png)"
]
},
{
Expand All @@ -152,13 +152,13 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X1273_X2e3_chans_45.png)\n",
"![data_repr_A001_X1273_X2e3_chans_45](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X1273_X2e3_chans_45.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X1273_X2e3.png)\n",
"![node_A001_X1273_X2e3](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X1273_X2e3.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X1273_X2e3.png)\n",
"![cluster_A001_X1273_X2e3](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X1273_X2e3.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X1273_X2e3.png)\n"
"![aws_A001_X1273_X2e3](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X1273_X2e3.png)\n"
]
},
{
Expand All @@ -179,13 +179,13 @@
"\n",
"Converted visibility data dimensions and uncompressed volume of the DATA array (and chunks, for a given factor) are shown below:\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X12a3_X3be_chans_48.png)\n",
"![data_repr_A001_X12a3_X3be_chans_48](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/data_repr_A001_X12a3_X3be_chans_48.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X12a3_X3be.png)\n",
"![node_A001_X12a3_X3be](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/node_A001_X12a3_X3be.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X12a3_X3be.png)\n",
"![cluster_A001_X12a3_X3be](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/cluster_A001_X12a3_X3be.png)\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X12a3_X3be.png)\n"
"![aws_A001_X12a3_X3be](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/aws_A001_X12a3_X3be.png)\n"
]
},
{
Expand Down Expand Up @@ -253,7 +253,7 @@
"The time spent in each function `<plots>`\n",
"\n",
"The communication of data between workers constituted a relatively small proportion of the total runtime, and the distribution of data between workers was relatively uniform, at all horizontal scalings, with some hot spots beginning to present once tens of nodes were involved. This is demonstrated by the following figure, taken from the performance report of a representative test execution:\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/bandwidth_A001_X12a3_X3be_threads_256_chans_48.png)\n",
"![bandwidth_A001_X12a3_X3be_threads_256_chans_48](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/bandwidth_A001_X12a3_X3be_threads_256_chans_48.png)\n",
"\n",
"The time overhead associated with graph creation and task scheduling (approximately 100 ms per task for dask) grew as more nodes were introduced until eventually coming to represent a fraction of total execution time comparable to the computation itself, especially in the test cases with smaller data.\n"
]
Expand All @@ -268,7 +268,7 @@
"\n",
"The total runtime curves for tests run on AWS show higher variance. One contributing factor that likely dominated this effect was the use of [preemptible instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html) underlying the compute nodes running the worker configuration. For this same reason, some cloud-based test runs show decreased performance with increased scale. This is due to the preemption of nodes and associated redeployment by kubernetes, which sometimes constituted a large fraction of the total test runtime, as demonstrated by the task stream for the following test case. Note the horizontal white bar (signifying to tasks executed) shortly after graph execution begins, as well as some final tasks being assigned to a new node that came online after a few minutes (represented by the new \"bar\" of 8 rows at top right) in the following figure:\n",
"\n",
"![](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/task_stream_A001_X1273_Xc66_threads_40_chans_45.png)\n",
"![task_stream_A001_X1273_Xc66_threads_40_chans_45](https://raw.githubusercontent.com/casangi/cngi_prototype/master/docs/_media/task_stream_A001_X1273_Xc66_threads_40_chans_45.png)\n",
"\n",
"Qualitatively, failure rates were higher during tests of CASA on local HPC infrastructure than they were using dask on the cluster or cloud. The cube refactor shows a noticeable improvement in this area, but still worse than the prototype."
]
Expand Down

0 comments on commit 5b3d4d7

Please sign in to comment.