Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(evaluator): calculate average of metric result across all dataset #1947

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

serious-gist
Copy link

Issue

Currently mmocr does not returns the avergae of all metric results in MultiDatasetEvaluator

Solution

Adding an method which calculates the average of all the metric results.

How to tackle this PR.

Just go through the changes made and test cases that are added.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Sharad Sirsat seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@serious-gist serious-gist force-pushed the feature/return_averaged_metric_over_all_dataset branch 4 times, most recently from 3f37077 to b68dba4 Compare July 5, 2023 20:54
@serious-gist
Copy link
Author

@Harold-lkk I have made changes as per my requirement, please review and comment. Let me know if there is anything else.

@OpenMMLab-Assistant-004

Hi @serious-gist,

We'd like to express our appreciation for your valuable contributions to the mmocr. Your efforts have significantly aided in enhancing the project's quality.
It is our pleasure to invite you to join our community thorugh Discord_Special Interest Group (SIG) channel. This is a great place to share your experiences, discuss ideas, and connect with other like-minded people. To become a part of the SIG channel, send a message to the moderator, OpenMMLab, briefly introduce yourself and mention your open-source contributions in the #introductions channel. Our team will gladly facilitate your entry. We eagerly await your presence. Please follow this link to join us: ​https://discord.gg/UjgXkPWNqA.

If you're on WeChat, we'd also love for you to join our community there. Just add our assistant using the WeChat ID: openmmlabwx. When sending the friend request, remember to include the remark "mmsig + Github ID".

Thanks again for your awesome contribution, and we're excited to have you as part of our community!

@gaotongxiao
Copy link
Collaborator

Hi, thanks for your contribution! Before we are able to merge your PR, please don't forget to sign CLA here :)

image

Also, if you get an error message like "xxx seems not to be a GitHub user", it means that the email you used to commit changes does not match the one you used in your GitHub account. You can try to link your local email to your GitHub, or update your local email address and then amend your commits:

git config --global user.email "YOUR_GITHUB_EMAIL"
git commit --amend

Comment on lines 94 to 97
if is_main_process():
metrics_results = [metrics_results]
averaged_results = self.average_results(metrics_results)
else:
metrics_results = [None] # type: ignore
averaged_results = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

metrics_results need to process like this

broadcast_object_list(metrics_results)
broadcast_object_list([averaged_results])

return metrics_results[0], averaged_results
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return should be a single dict.

@serious-gist serious-gist force-pushed the feature/return_averaged_metric_over_all_dataset branch from d3b2e4f to 6faeb6f Compare July 27, 2023 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants