Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmarks: Add Feature - Add GPU-Burn as microbenchmark #324

Merged
merged 15 commits into from
Mar 16, 2022

Conversation

rafsalas19
Copy link
Contributor

Description
Modifications adding GPU-Burn to SuperBench.

  • added third party submodule
  • modified Makefile to make gpu-burn binary
  • added/modified microbenchmarks to add gpu-burn python scripts
  • modified default and azure_ndv4 configs to add gpu-burn

@rafsalas19 rafsalas19 requested a review from a team as a code owner March 7, 2022 19:07
@cp5555 cp5555 requested review from abuccts and yzygitzh March 7, 2022 22:32
@cp5555 cp5555 added benchmarks SuperBench Benchmarks micro-benchmarks Micro Benchmark Test for SuperBench Benchmarks labels Mar 7, 2022
@cp5555 cp5555 changed the title Gpu burn module Benchmarks: Add Feature - Add GPU-Burn as microbenchmark Mar 7, 2022
@cp5555 cp5555 mentioned this pull request Mar 7, 2022
@yzygitzh
Copy link
Member

yzygitzh commented Mar 8, 2022

@rafsalas19 Thanks a lot for the PR! Overall it looks good to me. I have some general comments:

  1. Please help provide a unit-test (which should be both CUDA/ROCm test) that checks the output parsing logic. test_gpu_copy_bw_performance.py can be a reference.
  2. Please help add an example usage file for gpu-burn microbenchmark. gpu_copy_bw_performance.py can be a reference.
  3. Please help add description for new metrics in micro-benchmarks.md.

Copy link
Member

@abuccts abuccts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls run python3 setup.py lint to check lint issues, python3 setup.py format may help fix part of lint issues automatically.

third_party/Makefile Outdated Show resolved Hide resolved
superbench/benchmarks/micro_benchmarks/gpu_burn_test.py Outdated Show resolved Hide resolved
superbench/benchmarks/micro_benchmarks/gpu_burn_test.py Outdated Show resolved Hide resolved
superbench/benchmarks/micro_benchmarks/gpu_burn_test.py Outdated Show resolved Hide resolved
@rafsalas19 rafsalas19 requested a review from cp5555 as a code owner March 10, 2022 21:18
Copy link
Member

@abuccts abuccts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls run python3 setup.py format and commit changes to pass the pipeline.

tests/benchmarks/micro_benchmarks/test_gpu_burn_test.py Outdated Show resolved Hide resolved
docs/user-tutorial/benchmarks/micro-benchmarks.md Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Mar 14, 2022

Codecov Report

Merging #324 (bc5a79a) into main (84359fd) will decrease coverage by 0.16%.
The diff coverage is 77.77%.

@@            Coverage Diff             @@
##             main     #324      +/-   ##
==========================================
- Coverage   88.78%   88.61%   -0.17%     
==========================================
  Files          77       78       +1     
  Lines        4593     4665      +72     
==========================================
+ Hits         4078     4134      +56     
- Misses        515      531      +16     
Flag Coverage Δ
cpu-unit-test 73.30% <77.77%> (+0.07%) ⬆️
cuda-unit-test 88.55% <77.77%> (-0.17%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...bench/benchmarks/micro_benchmarks/gpu_burn_test.py 77.46% <77.46%> (ø)
superbench/benchmarks/micro_benchmarks/__init__.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 84359fd...bc5a79a. Read the comment docs.

@abuccts abuccts enabled auto-merge (squash) March 15, 2022 14:11
@cp5555 cp5555 requested a review from abuccts March 16, 2022 08:17
@abuccts abuccts merged commit ff51a3c into microsoft:main Mar 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmarks SuperBench Benchmarks micro-benchmarks Micro Benchmark Test for SuperBench Benchmarks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants