Remove custom function in no_grad block error message #33896

zou3519 · 2020-02-27T20:44:40Z

Stack from ghstack:

Remove custom function in no_grad block error message #33896 Remove custom function in no_grad block error message

Fixes #32625. Previously, we'd receive an error message if we have a
custom function return a view of an input in a no_grad block:

class Alias(Function):
    @staticmethod
    def forward(ctx, x):
        return x[:]

    @staticmethod
    def backward(ctx, gx):
        return gx

inp = torch.rand(2, requires_grad=True)

with torch.no_grad():
    # Used to error out
    output = Alias.apply(inp)

After this change, the error no longer happens. The behavior changes to
become consistent to if we had implemented an operator that does the
same thing as the custom function:

the output requires_grad
we are able to detect (and error out) if the user tries to modify the
output in-place outside of the no_grad block.

Test Plan:

new test

Differential Revision: D20345601

@staticmethod

Fixes #32625. Previously, we'd receive an error message if we have a custom function return a view of an input in a no_grad block: ``` class Alias(Function): @staticmethod def forward(ctx, x): return x[:] @staticmethod def backward(ctx, gx): return gx inp = torch.rand(2, requires_grad=True) with torch.no_grad(): # Used to error out output = Alias.apply(inp) ``` After this change, the error no longer happens. The behavior changes to become consistent to if we had implemented an operator that does the same thing as the custom function: - the output requires_grad - we are able to detect (and error out) if the user tries to modify the output in-place outside of the no_grad block. Test Plan: - new test [ghstack-poisoned]

@staticmethod

Fixes #32625. Previously, we'd receive an error message if we have a custom function return a view of an input in a no_grad block: ``` class Alias(Function): @staticmethod def forward(ctx, x): return x[:] @staticmethod def backward(ctx, gx): return gx inp = torch.rand(2, requires_grad=True) with torch.no_grad(): # Used to error out output = Alias.apply(inp) ``` After this change, the error no longer happens. The behavior changes to become consistent to if we had implemented an operator that does the same thing as the custom function: - the output requires_grad - we are able to detect (and error out) if the user tries to modify the output in-place outside of the no_grad block. Test Plan: - new test ghstack-source-id: 233a597f6e42069208a934316b34146db1073984 Pull Request resolved: #33896

albanD

Looks good

dr-ci · 2020-02-27T21:28:10Z

💊 CircleCI build failures summary and remediations

As of commit 4cc7919 (more details on the Dr. CI page):

2/2 failures introduced in this PR

🕵️ 2 new failures recognized by patterns

The following build failures do not appear to be due to upstream breakages:

pytorch_linux_xenial_py3_clang5_asan_test (1/2)

Step: "Test" (full log | pattern match details)

Mar 09 19:10:15 caused by: Connection refused (os error 111)

Mar 09 19:10:15 +++ eval 'extract_trap_cmd ' 
Mar 09 19:10:15 ++++ extract_trap_cmd 
Mar 09 19:10:15 ++++ printf '%s\n' '' 
Mar 09 19:10:15 +++ printf '%s\n' cleanup 
Mar 09 19:10:15 ++ trap -- ' 
Mar 09 19:10:15 cleanup' EXIT 
Mar 09 19:10:15 ++ which sccache 
Mar 09 19:10:15 ++ sccache --stop-server 
Mar 09 19:10:15 Stopping sccache server... 
Mar 09 19:10:15 error: couldn't connect to server 
Mar 09 19:10:15 caused by: Connection refused (os error 111) 
Mar 09 19:10:15 ++ true 
Mar 09 19:10:15 ++ rm /var/lib/jenkins/sccache_error.log 
Mar 09 19:10:15 ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log 
Mar 09 19:10:15 ++ SCCACHE_IDLE_TIMEOUT=1200 
Mar 09 19:10:15 ++ RUST_LOG=sccache::server=error 
Mar 09 19:10:15 ++ sccache --start-server 
Mar 09 19:10:15 Starting sccache server... 
Mar 09 19:10:15 ++ sccache --zero-stats 
Mar 09 19:10:15 Compile requests                 0 
Mar 09 19:10:15 Compile requests executed        0

pytorch_windows_vs2019_py36_cuda10.1_test2 (2/2)

Step: "Test" (full log | pattern match details)

ERROR: test_ctc_loss_cuda (__main__.TestAutogradDeviceTypeCUDA)

test_where_broadcast_all_cuda (__main__.TestAutogradDeviceTypeCUDA) ... ok 
test_where_cuda (__main__.TestAutogradDeviceTypeCUDA) ... ok 
test_where_functional_cuda (__main__.TestAutogradDeviceTypeCUDA) ... ok 
test_where_scalar_broadcast_mask_cuda (__main__.TestAutogradDeviceTypeCUDA) ... ok 
test_where_scalar_broadcast_non_mask_cuda (__main__.TestAutogradDeviceTypeCUDA) ... ok 
test_where_scalar_cuda (__main__.TestAutogradDeviceTypeCUDA) ... ok 
test_zero__cuda (__main__.TestAutogradDeviceTypeCUDA) ... ok 
test_zero__scalar_cuda (__main__.TestAutogradDeviceTypeCUDA) ... ok 
 
====================================================================== 
ERROR: test_ctc_loss_cuda (__main__.TestAutogradDeviceTypeCUDA) 
---------------------------------------------------------------------- 
Traceback (most recent call last): 
  File "C:\Users\circleci\project\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 207, in instantiated_test 
    return test(self, device_arg) 
  File "C:\Users\circleci\project\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 365, in dep_fn 
    return fn(slf, device, *args, **kwargs) 
  File "test_autograd.py", line 4416, in test_ctc_loss 
    gradcheck(ctc_after_softmax, [x]) 
  File "C:\Users\circleci\project\build\win_tmp\build\torch\autograd\gradcheck.py", line 297, in gradcheck 
    'The tolerance for nondeterminism was {}.'.format(nondet_tol))

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

This comment has been revised 4 times.

@staticmethod

Fixes #32625. Previously, we'd receive an error message if we have a custom function return a view of an input in a no_grad block: ``` class Alias(Function): @staticmethod def forward(ctx, x): return x[:] @staticmethod def backward(ctx, gx): return gx inp = torch.rand(2, requires_grad=True) with torch.no_grad(): # Used to error out output = Alias.apply(inp) ``` After this change, the error no longer happens. The behavior changes to become consistent to if we had implemented an operator that does the same thing as the custom function: - the output requires_grad - we are able to detect (and error out) if the user tries to modify the output in-place outside of the no_grad block. Test Plan: - new test [ghstack-poisoned]

@staticmethod

Fixes #32625. Previously, we'd receive an error message if we have a custom function return a view of an input in a no_grad block: ``` class Alias(Function): @staticmethod def forward(ctx, x): return x[:] @staticmethod def backward(ctx, gx): return gx inp = torch.rand(2, requires_grad=True) with torch.no_grad(): # Used to error out output = Alias.apply(inp) ``` After this change, the error no longer happens. The behavior changes to become consistent to if we had implemented an operator that does the same thing as the custom function: - the output requires_grad - we are able to detect (and error out) if the user tries to modify the output in-place outside of the no_grad block. Test Plan: - new test ghstack-source-id: 53a30c4975cec4f1637e813e72ec97b6c81f7d22 Pull Request resolved: #33896

facebook-github-bot · 2020-03-10T16:10:51Z

@zou3519 merged this pull request in f5ee46f.

zou3519 requested review from albanD and apaszke as code owners February 27, 2020 20:44

zou3519 mentioned this pull request Feb 27, 2020

Add instructional error message for cudnn RNN double backward workaround #33884

Closed

albanD approved these changes Feb 27, 2020

View reviewed changes

facebook-github-bot closed this in f5ee46f Mar 10, 2020

facebook-github-bot added the merged label Mar 10, 2020

facebook-github-bot deleted the gh/zou3519/239/head branch March 14, 2020 14:16

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove custom function in no_grad block error message #33896

Remove custom function in no_grad block error message #33896

zou3519 commented Feb 27, 2020 •

edited

albanD left a comment

dr-ci bot commented Feb 27, 2020 •

edited

facebook-github-bot commented Mar 10, 2020

Remove custom function in no_grad block error message #33896

Remove custom function in no_grad block error message #33896

Conversation

zou3519 commented Feb 27, 2020 • edited

albanD left a comment

Choose a reason for hiding this comment

dr-ci bot commented Feb 27, 2020 • edited

💊 CircleCI build failures summary and remediations

🕵️ 2 new failures recognized by patterns

pytorch_linux_xenial_py3_clang5_asan_test (1/2)

pytorch_windows_vs2019_py36_cuda10.1_test2 (2/2)

facebook-github-bot commented Mar 10, 2020

zou3519 commented Feb 27, 2020 •

edited

dr-ci bot commented Feb 27, 2020 •

edited