Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add Rouge score #399

Merged

Conversation

karthikrangasai
Copy link
Contributor

@karthikrangasai karthikrangasai commented Jul 23, 2021

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

What does this PR do?

Fixes #51 .

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@codecov
Copy link

codecov bot commented Jul 23, 2021

Codecov Report

Merging #399 (218418f) into master (b1062c9) will decrease coverage by 0.39%.
The diff coverage is 82.82%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #399      +/-   ##
==========================================
- Coverage   96.45%   96.05%   -0.40%     
==========================================
  Files         120      122       +2     
  Lines        3804     3900      +96     
==========================================
+ Hits         3669     3746      +77     
- Misses        135      154      +19     
Flag Coverage Δ
Linux 75.51% <38.38%> (-0.99%) ⬇️
Windows 75.51% <38.38%> (-0.99%) ⬇️
cpu 96.05% <82.82%> (-0.35%) ⬇️
gpu ?
macOS 96.05% <82.82%> (-0.35%) ⬇️
pytest 96.05% <82.82%> (-0.40%) ⬇️
python3.6 95.25% <82.82%> (-0.33%) ⬇️
python3.8 96.05% <82.82%> (-0.35%) ⬇️
python3.9 95.94% <82.82%> (-0.35%) ⬇️
torch1.3.1 95.25% <82.82%> (-0.33%) ⬇️
torch1.4.0 95.33% <82.82%> (-0.33%) ⬇️
torch1.9.0 95.94% <82.82%> (-0.35%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
torchmetrics/text/rouge.py 70.73% <70.73%> (ø)
torchmetrics/functional/text/rouge.py 90.19% <90.19%> (ø)
torchmetrics/__init__.py 100.00% <100.00%> (ø)
torchmetrics/functional/__init__.py 100.00% <100.00%> (ø)
torchmetrics/text/__init__.py 100.00% <100.00%> (ø)
torchmetrics/utilities/imports.py 82.50% <100.00%> (+0.92%) ⬆️
torchmetrics/functional/regression/spearman.py 93.33% <0.00%> (-4.45%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b1062c9...218418f. Read the comment docs.

@Borda Borda changed the title Feature/51 add rouge score add Rouge score Jul 23, 2021
Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we rather use only rouge_score only for testing and have our own implementation?

requirements/test.txt Outdated Show resolved Hide resolved
@SkafteNicki SkafteNicki added enhancement New feature or request New metric labels Jul 23, 2021
@SkafteNicki SkafteNicki added this to In progress in Text via automation Jul 23, 2021
Copy link
Member

@SkafteNicki SkafteNicki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just some initial comments

requirements/test.txt Outdated Show resolved Hide resolved
torchmetrics/functional/text/rouge.py Show resolved Hide resolved
torchmetrics/functional/text/rouge.py Outdated Show resolved Hide resolved
torchmetrics/functional/text/rouge.py Outdated Show resolved Hide resolved
torchmetrics/functional/text/rouge.py Outdated Show resolved Hide resolved
torchmetrics/text/rouge.py Outdated Show resolved Hide resolved
torchmetrics/text/rouge.py Outdated Show resolved Hide resolved
torchmetrics/functional/text/rouge.py Show resolved Hide resolved
@karthikrangasai
Copy link
Contributor Author

karthikrangasai commented Jul 23, 2021

The import errors will be gone after #383 is merged and rouge-score is added as a text dependency because it will install it before the imports are checked for.

@Borda
Copy link
Member

Borda commented Jul 26, 2021

@karthikrangasai how is it going? the mentioned PR has been merged... 🐰

@karthikrangasai
Copy link
Contributor Author

I have updated the dependencies but the pkg-install test fails for some reason. I don't know why it is not able to import rouge_scorer.

Copy link
Member

@SkafteNicki SkafteNicki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, final set of comments :]

requirements/text.txt Show resolved Hide resolved
torchmetrics/text/rouge.py Show resolved Hide resolved
torchmetrics/text/rouge.py Outdated Show resolved Hide resolved
torchmetrics/text/rouge.py Show resolved Hide resolved
torchmetrics/text/rouge.py Outdated Show resolved Hide resolved
torchmetrics/text/rouge.py Outdated Show resolved Hide resolved
@Borda
Copy link
Member

Borda commented Jul 28, 2021

@karthikrangasai mind add typing? (mypy is failing)

CHANGELOG.md Show resolved Hide resolved
docs/source/references/modules.rst Outdated Show resolved Hide resolved
torchmetrics/functional/text/rouge.py Show resolved Hide resolved
torchmetrics/functional/text/rouge.py Outdated Show resolved Hide resolved
torchmetrics/utilities/imports.py Outdated Show resolved Hide resolved
@mergify mergify bot added the ready label Jul 28, 2021
@Borda Borda requested a review from SkafteNicki July 28, 2021 16:57
@mergify mergify bot removed the ready label Jul 28, 2021
@mergify mergify bot added the ready label Jul 29, 2021
Copy link
Member

@awaelchli awaelchli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow that's a very old feature request <3

I mainly looked over the docs, very nice!

tests/text/test_rouge.py Outdated Show resolved Hide resolved
torchmetrics/text/rouge.py Outdated Show resolved Hide resolved
Text automation moved this from In progress to Reviewer approved Jul 29, 2021
requirements/test.txt Show resolved Hide resolved
tests/text/test_rouge.py Show resolved Hide resolved
@SkafteNicki SkafteNicki enabled auto-merge (squash) July 29, 2021 12:07
@SkafteNicki SkafteNicki merged commit bb083f2 into Lightning-AI:master Jul 29, 2021
Text automation moved this from Reviewer approved to Done Jul 29, 2021
@Borda Borda added this to the v0.5 milestone Aug 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Text
Done
Development

Successfully merging this pull request may close these issues.

[Metrics] ROUGE
6 participants