Skip to content

Fully deprecate variadic inputs of checkpoint_sequential - #25985

Closed
sublee wants to merge 8 commits into
pytorch:masterfrom
sublee:deprecate-checkpoint-sequential-varargs
Closed

Fully deprecate variadic inputs of checkpoint_sequential#25985
sublee wants to merge 8 commits into
pytorch:masterfrom
sublee:deprecate-checkpoint-sequential-varargs

Conversation

@sublee

@sublee sublee commented Sep 11, 2019

Copy link
Copy Markdown
Contributor

To support variadic inputs of checkpoint_sequential was deprecated at #21006. This case should be warned with DeprecationWarning for PyTorch 1.2, but it should be simply failed with TypeError since PyTorch 1.3. This patch removes the DeprecationWarning for PyTorch 1.2.

@pytorchbot pytorchbot added module: activation checkpointing Related to activation checkpointing module: tests Issues related to tests (not the torch.testing module) labels Sep 11, 2019
@soumith

soumith commented Sep 24, 2019

Copy link
Copy Markdown
Collaborator

@pytorchbot rebase this please

@sublee

sublee commented Sep 24, 2019

Copy link
Copy Markdown
Contributor Author

@soumith Thank you for reviewing it.

@yf225

yf225 commented Sep 25, 2019

Copy link
Copy Markdown
Contributor

@sublee We can't land it yet because test is failing:

Sep 24 20:52:17 ======================================================================
Sep 24 20:52:17 ERROR: test_checkpoint_module_list_multiple_args (__main__.TestCheckpoint)
Sep 24 20:52:17 ----------------------------------------------------------------------
Sep 24 20:52:17 Traceback (most recent call last):
Sep 24 20:52:17   File "test_utils.py", line 188, in test_checkpoint_module_list_multiple_args
Sep 24 20:52:17     torch.randn(1, 60, requires_grad=True)
Sep 24 20:52:17   File "test_utils.py", line 69, in _check_checkpoint_sequential
Sep 24 20:52:17     out = checkpoint_sequential(model_to_compare, num_chunks, *detached_inputs)
Sep 24 20:52:17   File "/opt/conda/lib/python3.6/site-packages/torch/utils/checkpoint.py", line 217, in checkpoint_sequential
Sep 24 20:52:17     preserve_rng_state=preserve_rng_state)
Sep 24 20:52:17   File "/opt/conda/lib/python3.6/site-packages/torch/utils/checkpoint.py", line 155, in checkpoint
Sep 24 20:52:17     return CheckpointFunction.apply(function, preserve, *args)
Sep 24 20:52:17   File "/opt/conda/lib/python3.6/site-packages/torch/utils/checkpoint.py", line 62, in forward
Sep 24 20:52:17     if preserve_rng_state:
Sep 24 20:52:17 RuntimeError: bool value of Tensor with more than one value is ambiguous

@soumith soumith left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please fix test

@sublee

sublee commented Sep 26, 2019

Copy link
Copy Markdown
Contributor Author

@yf225, @soumith Sorry for my mistake. I ran only the test cases I added at #21006 but I had not to do. I just fixed those failures. Please review it.

@vincentqb vincentqb added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Sep 26, 2019
@sublee

sublee commented Oct 9, 2019

Copy link
Copy Markdown
Contributor Author

@soumith I guess you lost the last update. Would you please review this PR again?

@sublee
sublee force-pushed the deprecate-checkpoint-sequential-varargs branch from aa20b4a to a07700f Compare October 11, 2019 11:15
@sublee

sublee commented Oct 11, 2019

Copy link
Copy Markdown
Contributor Author

This patch was intended to deploy with v1.3.0. However, today PyTorch v1.3.0 has been released. I update the version to v1.4 in the deprecation documentation and rebase this branch onto the current master.

@albanD albanD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Details in the test to make it look slightly better.
Then I'll merge.

Thanks for the PR !

Comment thread test/test_utils.py Outdated
Comment thread test/test_utils.py Outdated
@albanD
albanD force-pushed the deprecate-checkpoint-sequential-varargs branch from 01bf464 to e61b020 Compare December 4, 2019 15:08
@albanD

albanD commented Dec 4, 2019

Copy link
Copy Markdown
Collaborator

Just rebased on top of master for CI.

And again as I rebased on top of a broken master...

@facebook-github-bot facebook-github-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@albanD has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@albanD
albanD force-pushed the deprecate-checkpoint-sequential-varargs branch from e61b020 to 5829119 Compare December 4, 2019 16:18

@facebook-github-bot facebook-github-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@albanD is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot

Copy link
Copy Markdown
Contributor

@albanD merged this pull request in fa251cf.

wuhuikx pushed a commit to wuhuikx/pytorch that referenced this pull request Jan 30, 2020
Summary:
To support variadic inputs of `checkpoint_sequential` was deprecated at pytorch#21006. This case should be warned with `DeprecationWarning` for PyTorch 1.2, but it should be simply failed with `TypeError` since PyTorch 1.3. This patch removes the `DeprecationWarning` for PyTorch 1.2.
Pull Request resolved: pytorch#25985

Differential Revision: D18809875

Pulled By: albanD

fbshipit-source-id: e84dd8629c04979c4b2dc63e8ada94292e8cedd0
@zou3519 zou3519 added the module: bc-breaking Related to a BC-breaking change label Mar 31, 2020
@zou3519

zou3519 commented Apr 8, 2020

Copy link
Copy Markdown
Contributor

Hmm, If I am understanding this correctly it is not BC breaking because it enables behavior that used to throw a TypeError

@zou3519 zou3519 removed the module: bc-breaking Related to a BC-breaking change label Apr 8, 2020
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 24, 2026
Summary:
To support variadic inputs of `checkpoint_sequential` was deprecated at pytorch#21006. This case should be warned with `DeprecationWarning` for PyTorch 1.2, but it should be simply failed with `TypeError` since PyTorch 1.3. This patch removes the `DeprecationWarning` for PyTorch 1.2.
Pull Request resolved: pytorch#25985

Differential Revision: D18809875

Pulled By: albanD

fbshipit-source-id: e84dd8629c04979c4b2dc63e8ada94292e8cedd0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Merged module: activation checkpointing Related to activation checkpointing module: tests Issues related to tests (not the torch.testing module) open source triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants