Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add performance graphs #198

Merged
merged 4 commits into from
Jun 5, 2023
Merged

Add performance graphs #198

merged 4 commits into from
Jun 5, 2023

Conversation

varunsh-xilinx
Copy link
Member

Summary of Changes

  • Add initial performance graphs to the repository

Motivation

After #197, we want to benchmark the inference server and track changes over time.

Implementation

I added scripts to create graphs from the gathered benchmark data and show them on the website with sphinx-charts under a new page called Performance.

Notes

In the future, running similar kinds of benchmarks could be part of the CI to ensure that changes don't degrade the server performance. However, some open questions need to be answered first:

  1. How long should the test run for to be meaningful? In initial tests, even at the MLPerf limits, there were variations in the results.
  2. Temporary dips in performance may be tolerable e.g. if the cause is known. It shouldn't immediately reject the PR if the performance drops.
  3. How much of a drop is significant enough to merit remedial action?

Signed-off-by: Varun Sharma <varun.sharma@amd.com>
Signed-off-by: Varun Sharma <varun.sharma@amd.com>
Signed-off-by: Varun Sharma <varun.sharma@amd.com>
Signed-off-by: Varun Sharma <varun.sharma@amd.com>
@gbuildx
Copy link
Collaborator

gbuildx commented Jun 5, 2023

Build successful!

@varunsh-xilinx varunsh-xilinx merged commit 8edb56a into main Jun 5, 2023
@varunsh-xilinx varunsh-xilinx deleted the add-graphs branch June 5, 2023 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants