Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix bilinear backward pass #331

Merged
merged 2 commits into from
Feb 15, 2023
Merged

Fix bilinear backward pass #331

merged 2 commits into from
Feb 15, 2023

Conversation

DenisVieriu97
Copy link
Collaborator

@DenisVieriu97 DenisVieriu97 commented Feb 15, 2023 •
edited
Loading

Fixes backward pass for bilinear.

Summary of changes:

  • bilinear op is able to produce contiguous, non-view tensors with a storage offset, such as: shape=[1, 1, 1, 1], storage_offset=12. This seems a weird case, but it is valid, and for these type of tensors we wouldn't be able to gather/scatter since we look at the view flag (which is not set here). This change looks into storage_offset only rather than the is_view flag which is not being set
  • reduction sum must return a zeroed out output if passing an input with 0 elements (e.g a shape of (0, 5)).

@kulinseth
Copy link
Owner

This is great. @DenisVieriu97 can you check view() has been removed from all other places we are using it.

@kulinseth kulinseth merged commit 051bc9c into master Feb 15, 2023
DenisVieriu97 added a commit that referenced this pull request Feb 15, 2023
@DenisVieriu97
Fix bilinear backward pass (#331)
092e092 
* Fix bilinear backward pass

* Remove comment
@DenisVieriu97 DenisVieriu97 added the Upstreamed Change has been upstreamed to PyTorch master label Feb 15, 2023
kulinseth pushed a commit that referenced this pull request Feb 22, 2023
@DenisVieriu97 @kulinseth
Fix bilinear backward pass (#331)
a76aef7 
* Fix bilinear backward pass

* Remove comment
kulinseth added a commit that referenced this pull request Feb 28, 2023
@kulinseth
Update the test mps.
a3ca431 
Remove torch._six from test_mps (#326)

Fix test_zero_grad() (#330)

Fix bilinear backward pass (#331)

* Fix bilinear backward pass

* Remove comment

Update macOS 12 blocklist (#323)

* Update macOS 12 blocklist
- move sum, masked.var, mul to low precision list
- unblock them from running

* - mark __rdiv__ failures as accumulate error exceeds atol/rtol

Fix nn.functional.embedding grad (#335)

- casting the input tensor to float32 and cast back the output tensor
- unblock the test

Fix prelu backward (#334)

Reduction cast f16 to f32 only on macOS 12 (#332)

- unblock rdiv float16

Fix trace op (#340)

- give warnings of converting int64 for reduction ops
- use cast tensor for reduction sum on trace
- unblock trace from running

Update random result list (#339)

* - move nn.functional.feature_alpha_dropoutwith_train, normalnumber_mean, new_empty_strided to expected failures

* - update new_empty_strided

---------

Co-authored-by: Kulin Seth <kulin_seth@apple.com>

Enable int8 in TestConsistency (#347)

Dev/skotapati/copy broadcasting (#350)

* Handle broadcasting by expanding src tensor in Copy.mm

* Unblock linalg_matrix_power

* Improved formatting

Add the functionality to dump MPS ops.

1. DUMP_MPS_OPS to use LoggingTensor to dump out the ATen ops.
2. Skip running the EXPECTTEST list, as some tests are still
   seg-faulting

Fix lintrunner errors (#353)

* Fix lintrunner errors

* - move normal_in_place to random result list

Fixed the test_mps.

Test mps is updated.
DenisVieriu97 pushed a commit that referenced this pull request Feb 28, 2023
@kulinseth @DenisVieriu97
Update the test mps.
b3cdb2a 
Remove torch._six from test_mps (#326)

Fix test_zero_grad() (#330)

Fix bilinear backward pass (#331)

* Fix bilinear backward pass

* Remove comment

Update macOS 12 blocklist (#323)

* Update macOS 12 blocklist
- move sum, masked.var, mul to low precision list
- unblock them from running

* - mark __rdiv__ failures as accumulate error exceeds atol/rtol

Fix nn.functional.embedding grad (#335)

- casting the input tensor to float32 and cast back the output tensor
- unblock the test

Fix prelu backward (#334)

Reduction cast f16 to f32 only on macOS 12 (#332)

- unblock rdiv float16

Fix trace op (#340)

- give warnings of converting int64 for reduction ops
- use cast tensor for reduction sum on trace
- unblock trace from running

Update random result list (#339)

* - move nn.functional.feature_alpha_dropoutwith_train, normalnumber_mean, new_empty_strided to expected failures

* - update new_empty_strided

---------

Co-authored-by: Kulin Seth <kulin_seth@apple.com>

Enable int8 in TestConsistency (#347)

Dev/skotapati/copy broadcasting (#350)

* Handle broadcasting by expanding src tensor in Copy.mm

* Unblock linalg_matrix_power

* Improved formatting

Add the functionality to dump MPS ops.

1. DUMP_MPS_OPS to use LoggingTensor to dump out the ATen ops.
2. Skip running the EXPECTTEST list, as some tests are still
   seg-faulting

Fix lintrunner errors (#353)

* Fix lintrunner errors

* - move normal_in_place to random result list

Fixed the test_mps.
skotapati pushed a commit that referenced this pull request Apr 7, 2023
@kulinseth @skotapati
Update the test mps.
01707cc 
Remove torch._six from test_mps (#326)

Fix test_zero_grad() (#330)

Fix bilinear backward pass (#331)

* Fix bilinear backward pass

* Remove comment

Update macOS 12 blocklist (#323)

* Update macOS 12 blocklist
- move sum, masked.var, mul to low precision list
- unblock them from running

* - mark __rdiv__ failures as accumulate error exceeds atol/rtol

Fix nn.functional.embedding grad (#335)

- casting the input tensor to float32 and cast back the output tensor
- unblock the test

Fix prelu backward (#334)

Reduction cast f16 to f32 only on macOS 12 (#332)

- unblock rdiv float16

Fix trace op (#340)

- give warnings of converting int64 for reduction ops
- use cast tensor for reduction sum on trace
- unblock trace from running

Update random result list (#339)

* - move nn.functional.feature_alpha_dropoutwith_train, normalnumber_mean, new_empty_strided to expected failures

* - update new_empty_strided

---------

Co-authored-by: Kulin Seth <kulin_seth@apple.com>

Enable int8 in TestConsistency (#347)

Dev/skotapati/copy broadcasting (#350)

* Handle broadcasting by expanding src tensor in Copy.mm

* Unblock linalg_matrix_power

* Improved formatting

Add the functionality to dump MPS ops.

1. DUMP_MPS_OPS to use LoggingTensor to dump out the ATen ops.
2. Skip running the EXPECTTEST list, as some tests are still
   seg-faulting

Fix lintrunner errors (#353)

* Fix lintrunner errors

* - move normal_in_place to random result list

Fixed the test_mps.

Test mps is updated.
jhavukainen pushed a commit that referenced this pull request Mar 15, 2024
@kulinseth @jhavukainen
Update the test mps.
9cc57dd 
Remove torch._six from test_mps (#326)

Fix test_zero_grad() (#330)

Fix bilinear backward pass (#331)

* Fix bilinear backward pass

* Remove comment

Update macOS 12 blocklist (#323)

* Update macOS 12 blocklist
- move sum, masked.var, mul to low precision list
- unblock them from running

* - mark __rdiv__ failures as accumulate error exceeds atol/rtol

Fix nn.functional.embedding grad (#335)

- casting the input tensor to float32 and cast back the output tensor
- unblock the test

Fix prelu backward (#334)

Reduction cast f16 to f32 only on macOS 12 (#332)

- unblock rdiv float16

Fix trace op (#340)

- give warnings of converting int64 for reduction ops
- use cast tensor for reduction sum on trace
- unblock trace from running

Update random result list (#339)

* - move nn.functional.feature_alpha_dropoutwith_train, normalnumber_mean, new_empty_strided to expected failures

* - update new_empty_strided

---------

Co-authored-by: Kulin Seth <kulin_seth@apple.com>

Enable int8 in TestConsistency (#347)

Dev/skotapati/copy broadcasting (#350)

* Handle broadcasting by expanding src tensor in Copy.mm

* Unblock linalg_matrix_power

* Improved formatting

Add the functionality to dump MPS ops.

1. DUMP_MPS_OPS to use LoggingTensor to dump out the ATen ops.
2. Skip running the EXPECTTEST list, as some tests are still
   seg-faulting

Fix lintrunner errors (#353)

* Fix lintrunner errors

* - move normal_in_place to random result list

Fixed the test_mps.

Test mps is updated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kulinseth kulinseth kulinseth approved these changes

@skotapati skotapati Awaiting requested review from skotapati

@razarmehr razarmehr Awaiting requested review from razarmehr

@Ronian526 Ronian526 Awaiting requested review from Ronian526

@ssaladis ssaladis Awaiting requested review from ssaladis

@shuhand0 shuhand0 Awaiting requested review from shuhand0

Assignees
No one assigned
Labels
Upstreamed Change has been upstreamed to PyTorch master
Projects
None yet
Milestone
No milestone
2 participants
@DenisVieriu97 @kulinseth