Skip to content
This repository has been archived by the owner on Sep 18, 2024. It is now read-only.

Feat evaluators #224

Merged
merged 9 commits into from
Nov 26, 2021
Merged

Feat evaluators #224

merged 9 commits into from
Nov 26, 2021

Conversation

maximilianwerk
Copy link
Member

First attempt on Evaluation with our dataset. Missing:

  • conversion from class dataset.
  • having summary with averaged NDCG etc.

@codecov
Copy link

codecov bot commented Nov 23, 2021

Codecov Report

Merging #224 (72d387a) into main (89511dc) will increase coverage by 0.93%.
The diff coverage is 98.21%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #224      +/-   ##
==========================================
+ Coverage   86.99%   87.92%   +0.93%     
==========================================
  Files          30       35       +5     
  Lines        1668     1781     +113     
==========================================
+ Hits         1451     1566     +115     
+ Misses        217      215       -2     
Flag Coverage Δ
finetuner 87.92% <98.21%> (+0.93%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
finetuner/tuner/evaluation.py 98.18% <98.18%> (ø)
finetuner/__init__.py 74.19% <100.00%> (+0.86%) ⬆️
finetuner/tailor/keras/__init__.py 83.63% <0.00%> (-0.68%) ⬇️
finetuner/helper.py 73.52% <0.00%> (ø)
finetuner/embedding.py 77.77% <0.00%> (ø)
finetuner/labeler/__init__.py 0.00% <0.00%> (ø)
finetuner/tuner/dataset/samplers.py 100.00% <0.00%> (ø)
finetuner/tuner/summary.py
finetuner/tuner/callback/progress_bar.py 98.07% <0.00%> (ø)
finetuner/tuner/dataset/base.py 92.68% <0.00%> (ø)
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 670adbe...72d387a. Read the comment docs.

finetuner/tuner/evaluation.py Outdated Show resolved Hide resolved
finetuner/tuner/evaluation.py Show resolved Hide resolved
finetuner/tuner/evaluation.py Outdated Show resolved Hide resolved
finetuner/tuner/evaluation.py Outdated Show resolved Hide resolved
finetuner/tuner/metrics.py Outdated Show resolved Hide resolved
Copy link
Member

@hanxiao hanxiao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

finetuner/tuner/metrics.py is redundant to me

@github-actions github-actions bot added size/l and removed size/m labels Nov 25, 2021
@gmastrapas gmastrapas self-assigned this Nov 25, 2021
Copy link
Member

@bwanglzu bwanglzu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one minor comment

@github-actions github-actions bot added size/m and removed size/l labels Nov 26, 2021
Copy link
Member

@bwanglzu bwanglzu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some comments

tests/unit/tuner/test_evaluation.py Outdated Show resolved Hide resolved
finetuner/tuner/evaluation.py Outdated Show resolved Hide resolved
finetuner/tuner/evaluation.py Outdated Show resolved Hide resolved
finetuner/tuner/evaluation.py Outdated Show resolved Hide resolved
finetuner/tuner/evaluation.py Outdated Show resolved Hide resolved
finetuner/tuner/evaluation.py Show resolved Hide resolved
finetuner/tuner/evaluation.py Show resolved Hide resolved
Copy link
Member

@bwanglzu bwanglzu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@gmastrapas
Copy link
Member

@hanxiao can you review the changes wrt the metrics.py file?

@gmastrapas gmastrapas merged commit dcadc56 into main Nov 26, 2021
@gmastrapas gmastrapas deleted the feat-evaluators branch November 26, 2021 15:53
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants