Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[s2s] run_eval.py QOL improvements and cleanup #6746

Merged
merged 5 commits into from
Aug 26, 2020
Merged

[s2s] run_eval.py QOL improvements and cleanup #6746

merged 5 commits into from
Aug 26, 2020

Conversation

sshleifer
Copy link
Contributor

  • default to saving metrics to metrics.json
  • cleanup: delete some dead code.
  • save seconds_per_sample, runtime, num_samples

@codecov
Copy link

codecov bot commented Aug 26, 2020

Codecov Report

Merging #6746 into master will decrease coverage by 0.03%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6746      +/-   ##
==========================================
- Coverage   78.96%   78.93%   -0.04%     
==========================================
  Files         157      157              
  Lines       28486    28486              
==========================================
- Hits        22495    22485      -10     
- Misses       5991     6001      +10
Impacted Files Coverage Δ
src/transformers/generation_tf_utils.py 80.70% <0.00%> (-2.26%) ⬇️
src/transformers/file_utils.py 82.41% <0.00%> (-0.26%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a75c64d...0b9f1cd. Read the comment docs.

@sshleifer sshleifer changed the title [s2s] run_eval QOL improvements [s2s] run_eval.py QOL improvements and cleanup Aug 26, 2020
@sshleifer sshleifer merged commit 61518e2 into huggingface:master Aug 26, 2020
@sshleifer sshleifer deleted the run-eval-easier-to-use branch August 26, 2020 22:59
@sshleifer sshleifer linked an issue Aug 27, 2020 that may be closed by this pull request
[s2s] run_eval saves samples/second #6745
Closed
Zigur pushed a commit to Zigur/transformers that referenced this pull request Oct 26, 2020
@sshleifer @Zigur
[s2s] run_eval.py QOL improvements and cleanup(huggingface#6746)
cf612bd
fabiocapsouza pushed a commit to fabiocapsouza/transformers that referenced this pull request Nov 15, 2020
@sshleifer @fabiocapsouza
[s2s] run_eval.py QOL improvements and cleanup(huggingface#6746)
2b9b623
fabiocapsouza added a commit to fabiocapsouza/transformers that referenced this pull request Nov 15, 2020
@fabiocapsouza
Revert "[s2s] run_eval.py QOL improvements and cleanup(huggingface#6746…
420e859 
…)"

This reverts commit 2b9b623.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[s2s] run_eval saves samples/second
1 participant
@sshleifer