Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Adapt reader benchmarks #5005

Merged
merged 9 commits into from
May 26, 2023
Merged

Conversation

bogdankostic
Copy link
Contributor

@bogdankostic bogdankostic commented May 23, 2023

Related Issues

Related PRs

Proposed Changes:

This PR adapts the reader benchmarking script such that we can simply provide a Pipeline containing a reader and a label file.

How did you test it?

Manual test.

Notes for the reviewer

We use run_batch instead of eval_batch methods as measuring the time of eval isn’t representative for pure inference speed because it creates a significant overhead.

Checklist

  • I have read the contributors guidelines and the code of conduct
  • I have updated the related issue with new insights and changes
  • I added tests that demonstrate the correct behavior of the change
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
  • I documented my code
  • I ran pre-commit hooks and fixed any issue

@coveralls
Copy link
Collaborator

coveralls commented May 23, 2023

Pull Request Test Coverage Report for Build 5088703774

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 39.621%

Totals Coverage Status
Change from base Build 5081399763: 0.0%
Covered Lines: 8876
Relevant Lines: 22402

💛 - Coveralls

@bogdankostic bogdankostic marked this pull request as ready for review May 25, 2023 09:20
@bogdankostic bogdankostic requested a review from a team as a code owner May 25, 2023 09:20
@bogdankostic bogdankostic requested review from ZanSara and removed request for a team May 25, 2023 09:20
Copy link
Member

@vblagoje vblagoje left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

@bogdankostic bogdankostic merged commit 796340e into main May 26, 2023
44 checks passed
@bogdankostic bogdankostic deleted the adapt_reader_benchmarks branch May 26, 2023 09:40
@ZanSara ZanSara removed their request for review May 31, 2023 07:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants