Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix recompute to check if data is cleared before backward without setup_recompute #883

Conversation

AkioHayakawa-sony
Copy link
Member

@AkioHayakawa-sony AkioHayakawa-sony commented May 25, 2021

Re-computation may perform without the necessary setup_recompute execution.

Example:

x -> F.affine -> h1 -> F.dropout -> h2 -> F.affine -> y

y.forward()
h2.data.clear()
y.backward() # grad is computed incorrectly

# Dropout::setup_recompute() must be executed before recomputation
# but not called during forward prop because h2 is cleared after forward prop.
# Therefore, h2.data is recomputed incorrectly.

This PR handles this case and raises runtime error.

@YukioOobuchi YukioOobuchi deleted the feature/20210520-fix-recompute-without-setup-recompute branch November 10, 2021 00:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note-bugfix Auto-release; Bugfix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants