Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove host-side branch on F.accuracy with ignore_label #8364

Merged
merged 2 commits into from
Nov 1, 2019

Conversation

y1r
Copy link
Contributor

@y1r y1r commented Nov 1, 2019

Current F.accuracy with ignore_label implementation requires host-side branch if total == 0.
This causes unnecessary sync between forward and backward computation.

By this PR, we can avoid this synchronization so it can accelerate CPU-intensive model training (i.e. Mixed-precision training)
I've tried the Advanced Indexing feature, but it cannot help.

@niboshi
Copy link
Member

niboshi commented Nov 1, 2019

Thank you for the PR.

xp.asarray(count / total, dtype=y.dtype)

I believe it causes a warning if xp is numpy:

RuntimeWarning: divide by zero encountered in true_divide

How about using the previous code when xp is numpy?

@y1r
Copy link
Contributor Author

y1r commented Nov 1, 2019

Thank you for your quick review.
I've reflected your comment.

@niboshi
Copy link
Member

niboshi commented Nov 1, 2019

Thank you!
Jenkins, test this please

@niboshi niboshi added cat:bug Bug report or fix. st:test-and-merge State indicating that pull request is approved by a reviewer and can be merged after CI passes. to-be-backported Pull request that should be backported. labels Nov 1, 2019
@niboshi niboshi added this to the v7.0.0 milestone Nov 1, 2019
@niboshi niboshi changed the title Remove host-side branch on F.accuracy with ignore_label Remove host-side branch on F.accuracy with ignore_label Nov 1, 2019
@chainer-ci
Copy link
Member

Jenkins CI test (for commit 070a110, target branch master) failed with status FAILURE.

@niboshi
Copy link
Member

niboshi commented Nov 1, 2019

Test failure is unrelated.
LGTM

@niboshi niboshi merged commit 0532329 into chainer:master Nov 1, 2019
niboshi added a commit to niboshi/chainer that referenced this pull request Nov 1, 2019
Remove host-side branch on `F.accuracy` with `ignore_label`
@y1r y1r deleted the avoid_host_branch_accuracy branch November 3, 2019 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cat:bug Bug report or fix. st:test-and-merge State indicating that pull request is approved by a reviewer and can be merged after CI passes. to-be-backported Pull request that should be backported.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants