Support topo-aware IB performance validation #373
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jeseszhang1010
force-pushed
the
topo-aware
branch
2 times, most recently
from
da95b71
to
20671a6
Compare
jeseszhang1010
requested review from
abuccts,
cp5555,
jithinjosepkl,
rafsalas19 and
yzygitzh
jeseszhang1010
force-pushed
the
topo-aware
branch
from
20671a6
to
8b9ed8d
Compare
abuccts
reviewed
Jul 12, 2022
superbench/benchmarks/micro_benchmarks/ib_validation_performance.py
Outdated
Show resolved
Hide resolved
superbench/benchmarks/micro_benchmarks/ib_validation_performance.py
Outdated
Show resolved
Hide resolved
cp5555
reviewed
Jul 12, 2022
superbench/benchmarks/micro_benchmarks/ib_validation_performance.py
Outdated
Show resolved
Hide resolved
cp5555
reviewed
Jul 12, 2022
cp5555
reviewed
Jul 12, 2022
superbench/benchmarks/micro_benchmarks/ib_validation_performance.py
Outdated
Show resolved
Hide resolved
jeseszhang1010
force-pushed
the
topo-aware
branch
2 times, most recently
from
cc0860b
to
e4c1071
Compare
abuccts
approved these changes
Jul 21, 2022
cp5555
approved these changes
Jul 21, 2022
jeseszhang1010
force-pushed
the
topo-aware
branch
2 times, most recently
from
3ec835a
to
cf94e24
Compare
Codecov Report
@@ Coverage Diff @@
## main #373 +/- ##
==========================================
- Coverage 88.89% 88.88% -0.02%
==========================================
Files 82 83 +1
Lines 5044 5191 +147
==========================================
+ Hits 4484 4614 +130
- Misses 560 577 +17
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
Add a new pattern `topo-aware`, so the user can run IB performance test based on VM's topology information. This way, the user can validate the IB performance across VM pairs with different distance as a quick test instead of pair-wise test. To run with topo-aware pattern, user needs to specify three required (and two optional) parameters in YAML config file: --pattern topo-aware --ibstat path to ibstat output --ibnetdiscover path to ibnetdiscover output --min_dist minimum distance of VM pairs (optional, default 2) --max_dist maximum distance of VM pairs (optional, default 6) The newly added topo_aware module then parses the topology information, builds a graph, and generates the VM pairs with the specified distance (# hops). The specified IB test will then be running across these generated VM pairs. Signed-off-by: Jie Zhang <jessezhang1010@gmail.com>
Signed-off-by: Jie Zhang <jessezhang1010@gmail.com>
This commit adds unit test to verify the generated topology aware config file is correct. To do so, four new data files are added in order to invoke gen_topo_aware_config function to generate topology aware config file, then compares it with the expected config file. Signed-off-by: Jie Zhang <jessezhang1010@gmail.com>
jeseszhang1010
force-pushed
the
topo-aware
branch
from
cf94e24
to
13652b7
Compare
Signed-off-by: Jie Zhang <jessezhang1010@gmail.com>
jeseszhang1010
force-pushed
the
topo-aware
branch
from
13652b7
to
d941ed3
Compare
jithinjosepkl
approved these changes
Jul 26, 2022
rafsalas19
pushed a commit
to rafsalas19/superbenchmark
that referenced
this pull request
Jan 26, 2023
An enhancement for topo-aware IB performance validation microsoft#373. This PR will auto-generate a required ibstate file `ib_traffic_topo_aware_ibstat.txt` which is used as input to build a graph.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add a new pattern
topo-aware, so the user can run IB performancetest based on VM's topology information. This way, the user can
validate the IB performance across VM pairs with different distance
as a quick test instead of pair-wise test.
To run with topo-aware pattern, user needs to specify three required
(and two optional) parameters in YAML config file:
--pattern topo-aware
--ibstat path to ibstat output
--ibnetdiscover path to ibnetdiscover output
--min_dist minimum distance of VM pairs (optional, default 2)
--max_dist maximum distance of VM pairs (optional, default 6)
The newly added topo_aware module then parses the topology
information, builds a graph, and generates the VM pairs with
the specified distance (# hops).
The specified IB test will then be running across these generated VM pairs.
Signed-off-by: Jie Zhang jessezhang1010@gmail.com