Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tuner): add miner v1 #180

Merged
merged 19 commits into from
Nov 2, 2021
Merged

feat(tuner): add miner v1 #180

merged 19 commits into from
Nov 2, 2021

Conversation

bwanglzu
Copy link
Member

@bwanglzu bwanglzu commented Oct 27, 2021

this PR implemented:

  1. first iteration of miner, including a base class and miner for pytorch.
  2. tests

rest to do:

  1. miner for paddle and tf. (same as this, but with more mining strategy supported, such as hard/semi-hard negative mining, need more discussion).
  2. old data style support (where user already give negatives)

@github-actions github-actions bot added the area/testing This issue/PR affects testing label Oct 27, 2021
finetuner/tuner/base.py Outdated Show resolved Hide resolved
finetuner/tuner/miner.py Outdated Show resolved Hide resolved
finetuner/tuner/miner.py Outdated Show resolved Hide resolved
finetuner/tuner/miner.py Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Oct 28, 2021

Codecov Report

Merging #180 (fdd7957) into main (7e9c04f) will increase coverage by 0.19%.
The diff coverage is 96.15%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #180      +/-   ##
==========================================
+ Coverage   86.45%   86.65%   +0.19%     
==========================================
  Files          25       26       +1     
  Lines        1211     1236      +25     
==========================================
+ Hits         1047     1071      +24     
- Misses        164      165       +1     
Flag Coverage Δ
finetuner 86.65% <96.15%> (+0.19%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
finetuner/tuner/base.py 87.93% <80.00%> (-0.96%) ⬇️
finetuner/helper.py 94.73% <100.00%> (+0.29%) ⬆️
finetuner/tuner/pytorch/miner.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7e9c04f...fdd7957. Read the comment docs.

@github-actions github-actions bot added size/m and removed size/s labels Oct 28, 2021
@bwanglzu bwanglzu changed the title feat(tuner): add miner feat(tuner): add miner v1 Oct 28, 2021
@bwanglzu bwanglzu marked this pull request as ready for review October 28, 2021 15:14
finetuner/tuner/pytorch/miner.py Outdated Show resolved Hide resolved
finetuner/tuner/pytorch/miner.py Outdated Show resolved Hide resolved
finetuner/tuner/pytorch/miner.py Outdated Show resolved Hide resolved
finetuner/tuner/pytorch/miner.py Outdated Show resolved Hide resolved
finetuner/tuner/pytorch/miner.py Outdated Show resolved Hide resolved
finetuner/tuner/base.py Outdated Show resolved Hide resolved
finetuner/tuner/pytorch/miner.py Outdated Show resolved Hide resolved
finetuner/tuner/pytorch/miner.py Outdated Show resolved Hide resolved
Copy link
Contributor

@tadejsv tadejsv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just one final comment

Comment on lines +20 to +23
return [
(left[0], right[0], 1) if left[1] == right[1] else (left[0], right[0], -1)
for left, right in combinations(enumerate(labels), 2)
]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry to be nit-picky here, but can you break down this return statement? it's hard to read in current form

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

formated by black, break down ci will fail

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I mean is to re-format this, so first constrict the list, then return it. This inline if-else should only be used if it can fit comfortably in one line

Copy link
Member Author

@bwanglzu bwanglzu Oct 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the break-up version has been commented here: #180 (comment), won't apply

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the comment is irrelevant to this, please apply

@bwanglzu bwanglzu merged commit 1e4a1ae into main Nov 2, 2021
@bwanglzu bwanglzu deleted the feat-miner branch November 2, 2021 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants