[optimizer] refactor RMSProp to use functional API #50410

wanchaol · 2021-01-12T01:25:38Z

Stack from ghstack:

[dist_optim] add warning to distributed optimizer #50630 [dist_optim] add warning to distributed optimizer
[dist_optim] add distributed functional AdamW optimizer #50620 [dist_optim] add distributed functional AdamW optimizer
[dist_optim] add distributed functional RMSprop optimizer #50619 [dist_optim] add distributed functional RMSprop optimizer
[dist_optim] add distributed functional Adadelta optimizer #50623 [dist_optim] add distributed functional Adadelta optimizer
[dist_optim] add distributed functional sgd optimizer #50618 [dist_optim] add distributed functional sgd optimizer
[dist_optim] add distributed functional Adam optimizer #50624 [dist_optim] add distributed functional Adam optimizer
[optimizer] refactor AdamW to use functional API #50411 [optimizer] refactor AdamW to use functional API
[optimizer] refactor RMSProp to use functional API #50410 [optimizer] refactor RMSProp to use functional API
[optimizer] refactor Adadelta to use functional API #50409 [optimizer] refactor Adadelta to use functional API
[optimizer] refactor SGD to use functional API #45597 [optimizer] refactor SGD to use functional API

Differential Revision: D25932779

[ghstack-poisoned]

facebook-github-bot · 2021-01-12T01:25:56Z

💊 CI failures summary and remediations

As of commit 7fe5ffd (more details on the Dr. CI page):

6/6 failures introduced in this PR

🕵️ 6 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_linux_bionic_py3_8_gcc9_coverage_test1 (1/6)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Jan 12 02:11:20 AssertionError: mypy failed: torch/optim/functional.py:128: error: Argument 1 to "add" of "_TensorBase" has incompatible type "Optional[Tensor]"; expected "Union[Tensor, Union[int, float, bool]]"  [arg-type]

Jan 12 02:10:56   test_type_hint_examples (__main__.TestTypeHints)
Jan 12 02:11:20 Runs mypy over all the test examples present in ... ok (23.583s)
Jan 12 02:11:20 
Jan 12 02:11:20 ======================================================================
Jan 12 02:11:20 FAIL [77.305s]: test_run_mypy (__main__.TestTypeHints)
Jan 12 02:11:20 Runs mypy over all files specified in mypy.ini
Jan 12 02:11:20 ----------------------------------------------------------------------
Jan 12 02:11:20 Traceback (most recent call last):
Jan 12 02:11:20   File "test_type_hints.py", line 214, in test_run_mypy
Jan 12 02:11:20     self.fail(f"mypy failed: {stdout} {stderr}")
Jan 12 02:11:20 AssertionError: mypy failed: torch/optim/functional.py:128: error: Argument 1 to "add" of "_TensorBase" has incompatible type "Optional[Tensor]"; expected "Union[Tensor, Union[int, float, bool]]"  [arg-type]
Jan 12 02:11:20 torch/optim/functional.py:130: error: Incompatible types in assignment (expression has type "Optional[Tensor]", variable has type "Tensor")  [assignment]
Jan 12 02:11:20 torch/distributed/optim/functional_adam.py:61: error: Need type annotation for 'state_sums' (hint: "state_sums: List[<type>] = ...")  [var-annotated]
Jan 12 02:11:20 Found 3 errors in 2 files (checked 1189 source files)
Jan 12 02:11:20  
Jan 12 02:11:20 
Jan 12 02:11:20 ----------------------------------------------------------------------
Jan 12 02:11:20 Ran 4 tests in 118.102s
Jan 12 02:11:20 
Jan 12 02:11:20 FAILED (failures=1)
Jan 12 02:11:20

pytorch_linux_bionic_py3_6_clang9_test (2/6)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Jan 12 02:03:31 AssertionError: mypy failed: torch/optim/functional.py:128: error: Argument 1 to "add" of "_TensorBase" has incompatible type "Optional[Tensor]"; expected "Union[Tensor, Union[int, float, bool]]"  [arg-type]

Jan 12 02:03:08   test_run_mypy (__main__.TestTypeHints) ... FAIL (61.249s)
Jan 12 02:03:11   test_run_mypy_strict (__main__.TestTypeHints) ... ok (3.016s)
Jan 12 02:03:31   test_type_hint_examples (__main__.TestTypeHints) ... ok (20.732s)
Jan 12 02:03:31 
Jan 12 02:03:31 ======================================================================
Jan 12 02:03:31 FAIL [61.249s]: test_run_mypy (__main__.TestTypeHints)
Jan 12 02:03:31 ----------------------------------------------------------------------
Jan 12 02:03:31 Traceback (most recent call last):
Jan 12 02:03:31   File "test_type_hints.py", line 214, in test_run_mypy
Jan 12 02:03:31     self.fail(f"mypy failed: {stdout} {stderr}")
Jan 12 02:03:31 AssertionError: mypy failed: torch/optim/functional.py:128: error: Argument 1 to "add" of "_TensorBase" has incompatible type "Optional[Tensor]"; expected "Union[Tensor, Union[int, float, bool]]"  [arg-type]
Jan 12 02:03:31 torch/optim/functional.py:130: error: Incompatible types in assignment (expression has type "Optional[Tensor]", variable has type "Tensor")  [assignment]
Jan 12 02:03:31 torch/distributed/optim/functional_adam.py:61: error: Need type annotation for 'state_sums' (hint: "state_sums: List[<type>] = ...")  [var-annotated]
Jan 12 02:03:31 Found 3 errors in 2 files (checked 1189 source files)
Jan 12 02:03:31  
Jan 12 02:03:31 
Jan 12 02:03:31 ----------------------------------------------------------------------
Jan 12 02:03:31 Ran 4 tests in 96.074s
Jan 12 02:03:31 
Jan 12 02:03:31 FAILED (failures=1)
Jan 12 02:03:31

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test1 (3/6)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Jan 12 02:25:50 AssertionError: mypy failed: torch/optim/functional.py:128: error: Argument 1 to "add" of "_TensorBase" has incompatible type "Optional[Tensor]"; expected "Union[Tensor, Union[int, float, bool]]"  [arg-type]

Jan 12 02:25:28   test_run_mypy (__main__.TestTypeHints) ... FAIL (62.272s)
Jan 12 02:25:31   test_run_mypy_strict (__main__.TestTypeHints) ... ok (2.898s)
Jan 12 02:25:50   test_type_hint_examples (__main__.TestTypeHints) ... ok (18.521s)
Jan 12 02:25:50 
Jan 12 02:25:50 ======================================================================
Jan 12 02:25:50 FAIL [62.272s]: test_run_mypy (__main__.TestTypeHints)
Jan 12 02:25:50 ----------------------------------------------------------------------
Jan 12 02:25:50 Traceback (most recent call last):
Jan 12 02:25:50   File "test_type_hints.py", line 214, in test_run_mypy
Jan 12 02:25:50     self.fail(f"mypy failed: {stdout} {stderr}")
Jan 12 02:25:50 AssertionError: mypy failed: torch/optim/functional.py:128: error: Argument 1 to "add" of "_TensorBase" has incompatible type "Optional[Tensor]"; expected "Union[Tensor, Union[int, float, bool]]"  [arg-type]
Jan 12 02:25:50 torch/optim/functional.py:130: error: Incompatible types in assignment (expression has type "Optional[Tensor]", variable has type "Tensor")  [assignment]
Jan 12 02:25:50 torch/distributed/optim/functional_adam.py:61: error: Need type annotation for 'state_sums' (hint: "state_sums: List[<type>] = ...")  [var-annotated]
Jan 12 02:25:50 Found 3 errors in 2 files (checked 1189 source files)
Jan 12 02:25:50  
Jan 12 02:25:50 
Jan 12 02:25:50 ----------------------------------------------------------------------
Jan 12 02:25:50 Ran 4 tests in 94.734s
Jan 12 02:25:50 
Jan 12 02:25:50 FAILED (failures=1)
Jan 12 02:25:50

pytorch_linux_xenial_py3_clang5_asan_test1 (4/6)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Jan 12 02:10:21 AssertionError: mypy failed: torch/optim/functional.py:128: error: Argument 1 to "add" of "_TensorBase" has incompatible type "Optional[Tensor]"; expected "Union[Tensor, Union[int, float, bool]]"  [arg-type]

Jan 12 02:09:49   test_run_mypy (__main__.TestTypeHints) ... FAIL (81.033s)
Jan 12 02:09:53   test_run_mypy_strict (__main__.TestTypeHints) ... ok (4.207s)
Jan 12 02:10:21   test_type_hint_examples (__main__.TestTypeHints) ... ok (27.864s)
Jan 12 02:10:21 
Jan 12 02:10:21 ======================================================================
Jan 12 02:10:21 FAIL [81.033s]: test_run_mypy (__main__.TestTypeHints)
Jan 12 02:10:21 ----------------------------------------------------------------------
Jan 12 02:10:21 Traceback (most recent call last):
Jan 12 02:10:21   File "test_type_hints.py", line 214, in test_run_mypy
Jan 12 02:10:21     self.fail(f"mypy failed: {stdout} {stderr}")
Jan 12 02:10:21 AssertionError: mypy failed: torch/optim/functional.py:128: error: Argument 1 to "add" of "_TensorBase" has incompatible type "Optional[Tensor]"; expected "Union[Tensor, Union[int, float, bool]]"  [arg-type]
Jan 12 02:10:21 torch/optim/functional.py:130: error: Incompatible types in assignment (expression has type "Optional[Tensor]", variable has type "Tensor")  [assignment]
Jan 12 02:10:21 torch/distributed/optim/functional_adam.py:61: error: Need type annotation for 'state_sums' (hint: "state_sums: List[<type>] = ...")  [var-annotated]
Jan 12 02:10:21 Found 3 errors in 2 files (checked 1189 source files)
Jan 12 02:10:21  
Jan 12 02:10:21 
Jan 12 02:10:21 ----------------------------------------------------------------------
Jan 12 02:10:21 Ran 4 tests in 127.874s
Jan 12 02:10:21 
Jan 12 02:10:21 FAILED (failures=1)
Jan 12 02:10:21

pytorch_windows_vs2019_py36_cuda10.1_test2 (5/6)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

AssertionError: False is not true : Tensors failed to compare as equal!With rtol=1e-07 and atol=1e-07, found 6 element(s) (out of 6) whose difference(s) exceeded the margin of error (including 0 nan comparisons). The greatest difference was 0.14733898893893732 (0.020583703180005117 vs. 0.16792269211894245), which occurred at index (2, 1).

======================================================================
FAIL [0.271s]: test_multi_tensor_optimizers (__main__.TestOptim)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\circleci\project\build\win_tmp\build\torch\testing\_internal\common_utils.py", line 423, in wrapper
    fn(*args, **kwargs)
  File "test_optim.py", line 380, in test_multi_tensor_optimizers
    self.assertEqual(p1, p2)
  File "C:\Users\circleci\project\build\win_tmp\build\torch\testing\_internal\common_utils.py", line 1180, in assertEqual
    super().assertTrue(result, msg=self._get_assert_msg(msg, debug_msg=debug_msg))
AssertionError: False is not true : Tensors failed to compare as equal!With rtol=1e-07 and atol=1e-07, found 6 element(s) (out of 6) whose difference(s) exceeded the margin of error (including 0 nan comparisons). The greatest difference was 0.14733898893893732 (0.020583703180005117 vs. 0.16792269211894245), which occurred at index (2, 1).

----------------------------------------------------------------------
Ran 104 tests in 56.775s

FAILED (failures=1)

Generating XML reports...
Generated XML report: test-reports\dist-gloo\TEST-TestLRScheduler-20210112032513.xml
Generated XML report: test-reports\dist-gloo\TEST-TestOptim-20210112032513.xml
Generated XML report: test-reports\dist-gloo\TEST-TestSWAUtils-20210112032513.xml

pytorch_linux_xenial_py3_6_gcc5_4_test (6/6)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Jan 12 02:10:01 AssertionError: mypy failed: torch/optim/functional.py:128: error: Argument 1 to "add" of "_TensorBase" has incompatible type "Optional[Tensor]"; expected "Union[Tensor, Union[int, float, bool]]"  [arg-type]

Jan 12 02:09:37   test_run_mypy (__main__.TestTypeHints) ... FAIL (61.656s)
Jan 12 02:09:40   test_run_mypy_strict (__main__.TestTypeHints) ... ok (3.116s)
Jan 12 02:10:01   test_type_hint_examples (__main__.TestTypeHints) ... ok (20.589s)
Jan 12 02:10:01 
Jan 12 02:10:01 ======================================================================
Jan 12 02:10:01 FAIL [61.656s]: test_run_mypy (__main__.TestTypeHints)
Jan 12 02:10:01 ----------------------------------------------------------------------
Jan 12 02:10:01 Traceback (most recent call last):
Jan 12 02:10:01   File "test_type_hints.py", line 214, in test_run_mypy
Jan 12 02:10:01     self.fail(f"mypy failed: {stdout} {stderr}")
Jan 12 02:10:01 AssertionError: mypy failed: torch/optim/functional.py:128: error: Argument 1 to "add" of "_TensorBase" has incompatible type "Optional[Tensor]"; expected "Union[Tensor, Union[int, float, bool]]"  [arg-type]
Jan 12 02:10:01 torch/optim/functional.py:130: error: Incompatible types in assignment (expression has type "Optional[Tensor]", variable has type "Tensor")  [assignment]
Jan 12 02:10:01 torch/distributed/optim/functional_adam.py:61: error: Need type annotation for 'state_sums' (hint: "state_sums: List[<type>] = ...")  [var-annotated]
Jan 12 02:10:01 Found 3 errors in 2 files (checked 1190 source files)
Jan 12 02:10:01  
Jan 12 02:10:01 
Jan 12 02:10:01 ----------------------------------------------------------------------
Jan 12 02:10:01 Ran 4 tests in 95.707s
Jan 12 02:10:01 
Jan 12 02:10:01 FAILED (failures=1)
Jan 12 02:10:01

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

This comment has been revised 9 times.

[ghstack-poisoned]

Differential Revision: [D25932779](https://our.internmc.facebook.com/intern/diff/D25932779) [ghstack-poisoned]

codecov · 2021-01-21T09:10:43Z

Codecov Report

Merging #50410 (044617a) into gh/wanchaol/150/base (5f3563e) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@                  Coverage Diff                  @@
##           gh/wanchaol/150/base   #50410   +/-   ##
=====================================================
  Coverage                 81.01%   81.02%           
=====================================================
  Files                      1916     1916           
  Lines                    209313   209328   +15     
=====================================================
+ Hits                     169585   169600   +15     
  Misses                    39728    39728

facebook-github-bot · 2021-01-21T19:01:19Z

@wanchaol merged this pull request in ce1781d.

ghstack-source-id: 434a7bfc4d0becfdc2b3e482edc1603f438872e6 Pull Request resolved: pytorch/pytorch#50410

[optimizer] refactor RMSProp to use functional API

7fe5ffd

[ghstack-poisoned]

This was referenced Jan 12, 2021

[dist_optim] add distributed functional Adam optimizer #45596

Merged

[optimizer] refactor SGD to use functional API #45597

Closed

facebook-github-bot added the cla signed label Jan 12, 2021

wanchaol mentioned this pull request Jan 12, 2021

[optimizer] refactor Adadelta to use functional API #50409

Closed

wanchaol mentioned this pull request Jan 12, 2021

[optimizer] refactor AdamW to use functional API #50411

Closed

wanchaol added 3 commits January 15, 2021 01:26

Update on "[optimizer] refactor RMSProp to use functional API"

b8d8489

[ghstack-poisoned]

Update on "[optimizer] refactor RMSProp to use functional API"

a9de5e7

[ghstack-poisoned]

Update on "[optimizer] refactor RMSProp to use functional API"

fc7b43a

[ghstack-poisoned]

wanchaol requested review from pritamdamania87, albanD, vincentqb, rohan-varma, izdeby and mrshenli January 15, 2021 20:40

Update on "[optimizer] refactor RMSProp to use functional API"

6f0d8e2

[ghstack-poisoned]

This was referenced Jan 15, 2021

[dist_optim] add distributed functional sgd optimizer #50618

Closed

[dist_optim] add distributed functional RMSprop optimizer #50619

Closed

[dist_optim] add distributed functional AdamW optimizer #50620

Closed

Update on "[optimizer] refactor RMSProp to use functional API"

945daef

[ghstack-poisoned]

This was referenced Jan 15, 2021

[dist_optim] add distributed functional Adadelta optimizer #50623

Closed

[dist_optim] add distributed functional Adam optimizer #50624

Closed

[dist_optim] add warning to distributed optimizer #50630

Closed

izdeby approved these changes Jan 19, 2021

View reviewed changes

wanchaol added 3 commits January 20, 2021 11:58

Update on "[optimizer] refactor RMSProp to use functional API"

4623acb

Differential Revision: [D25932779](https://our.internmc.facebook.com/intern/diff/D25932779) [ghstack-poisoned]

Update on "[optimizer] refactor RMSProp to use functional API"

a430fe8

Differential Revision: [D25932779](https://our.internmc.facebook.com/intern/diff/D25932779) [ghstack-poisoned]

Update on "[optimizer] refactor RMSProp to use functional API"

044617a

Differential Revision: [D25932779](https://our.internmc.facebook.com/intern/diff/D25932779) [ghstack-poisoned]

facebook-github-bot closed this in ce1781d Jan 21, 2021

facebook-github-bot added the Merged label Jan 21, 2021

facebook-github-bot deleted the gh/wanchaol/150/head branch January 25, 2021 15:19

jasperzhong pushed a commit to jasperzhong/swift that referenced this pull request Nov 25, 2021

[optimizer] refactor RMSProp to use functional API

9e53736

ghstack-source-id: 434a7bfc4d0becfdc2b3e482edc1603f438872e6 Pull Request resolved: pytorch/pytorch#50410

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[optimizer] refactor RMSProp to use functional API #50410

[optimizer] refactor RMSProp to use functional API #50410

wanchaol commented Jan 12, 2021 •

edited

facebook-github-bot commented Jan 12, 2021 •

edited

codecov bot commented Jan 21, 2021

facebook-github-bot commented Jan 21, 2021

[optimizer] refactor RMSProp to use functional API #50410

[optimizer] refactor RMSProp to use functional API #50410

Conversation

wanchaol commented Jan 12, 2021 • edited

facebook-github-bot commented Jan 12, 2021 • edited

💊 CI failures summary and remediations

🕵️ 6 new failures recognized by patterns

pytorch_linux_bionic_py3_8_gcc9_coverage_test1 (1/6)

pytorch_linux_bionic_py3_6_clang9_test (2/6)

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test1 (3/6)

pytorch_linux_xenial_py3_clang5_asan_test1 (4/6)

pytorch_windows_vs2019_py36_cuda10.1_test2 (5/6)

pytorch_linux_xenial_py3_6_gcc5_4_test (6/6)

codecov bot commented Jan 21, 2021

Codecov Report

facebook-github-bot commented Jan 21, 2021

wanchaol commented Jan 12, 2021 •

edited

facebook-github-bot commented Jan 12, 2021 •

edited