Skip to content
This repository has been archived by the owner on Dec 16, 2022. It is now read-only.

FBetaMeasure metric with one value per key #5638

Merged
merged 15 commits into from Jun 2, 2022

Conversation

eraldoluis
Copy link
Contributor

@eraldoluis eraldoluis commented May 21, 2022

Fixes #5637 .

Changes proposed in this pull request:

  • I have implemented a class called FBetaMeasure2 that returns a dictionary where each key has only one value.
  • This implementation also allows the return of the per-class values and the averages (at the same time).

Before submitting

  • I've read and followed all steps in the Making a pull request
    section of the CONTRIBUTING docs.
  • I've updated or added any relevant docstrings following the syntax described in the
    Writing docstrings section of the CONTRIBUTING docs.
  • If this PR fixes a bug, I've added a test that will fail without my fix.
  • If this PR adds a new feature, I've added tests that sufficiently cover my new functionality.

After submitting

  • All GitHub Actions jobs for my pull request have passed.
  • codecov/patch reports high test coverage (at least 90%).
    You can find this under the "Actions" tab of the pull request once the other checks have finished.

@eraldoluis eraldoluis changed the title Fbeta measure2 FBetaMeasure metric with one value per key May 21, 2022
Copy link
Member

@dirkgr dirkgr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks quite good. Can we give it a more descriptive name than FBetaMeasure2? That's the only real concern I have.

all_metrics[f"{c}-recall"] = r
all_metrics[f"{c}-fscore"] = f

if "macro" in self._averages:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since these calculations don't take any time, can you just do all of them every time? It doesn't hurt anything to have all the averages all the time, does it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Renamed to FBetaVerboseMeasure and removed the average option (always return all three averages).

@eraldoluis
Copy link
Contributor Author

Thank you @dirkgr for reviewing this PR. I made the suggested changes.

@dirkgr dirkgr self-assigned this May 31, 2022
CHANGELOG.md Outdated Show resolved Hide resolved
@dirkgr dirkgr enabled auto-merge (squash) May 31, 2022 20:22
@dirkgr
Copy link
Member

dirkgr commented Jun 1, 2022

It seems to be failing tests. I put an updated version at #5651 that fixes some things, but it looks like the actual test for the new metric is failing.

@eraldoluis
Copy link
Contributor Author

eraldoluis commented Jun 1, 2022

It seems to be failing tests. I put an updated version at #5651 that fixes some things, but it looks like the actual test for the new metric is failing.

Thank you, @dirkgr , for pointing this issue. There were two tests missing adaptation from the older version. They have now passed in my tests.

I have also incorporated your edits from #5651 .

@dirkgr dirkgr enabled auto-merge (squash) June 2, 2022 00:03
@dirkgr dirkgr merged commit a6271a3 into allenai:main Jun 2, 2022
@dirkgr
Copy link
Member

dirkgr commented Jun 2, 2022

Thank you for sticking with it!

@eraldoluis
Copy link
Contributor Author

Thank you for your support!

@eraldoluis eraldoluis deleted the fbeta_measure2 branch June 16, 2022 20:22
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

FBetaMeasure metric with one value per key
2 participants