Added computing matrix condition numbers (linalg.cond) #45832

IvanYashchuk · 2020-10-05T10:21:27Z

This PR adds torch.linalg.cond for NumPy compatibility.

Now for p=None and p=±2 condition number is computed with svd. Added errorchecks.

facebook-github-bot · 2020-10-30T17:26:49Z

Thank you for your pull request. We require contributors to sign our Contributor License Agreement, and yours needs attention.

You currently have a record in our system, but we do not have a signature on file.

In order for us to review and merge your code, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

dr-ci · 2020-11-03T13:03:10Z

💊 CI failures summary and remediations

As of commit 898f855 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 78 times.

aten/src/ATen/native/LinearAlgebra.cpp

IvanYashchuk · 2020-11-06T07:32:05Z

test/test_linalg.py

+                continue
+            run_test_case(a, ord)
+
+    # TODO: once "inverse_cuda" supports complex dtypes, they shall be added to above tests


#45034 got merged. This needs to be updated.

IvanYashchuk · 2020-11-06T07:42:35Z

aten/src/ATen/native/LinearAlgebra.cpp

+    squareCheckInputs(self);
+    std::array<int64_t, 2> dim_arr = {-2, -1};
+    optional<IntArrayRef> dim = IntArrayRef(dim_arr);
+    // Ignore errors if not invertible, result is INFINITY in this case


Probably should add a note here that if in the future it is decided that inverse routine shouldn't throw an error by default (for performance reasons). Then the behaviour here could be changed that the inverse matrix contains NaNs and the norm of such matrix is also NaN, and apply at::nan_to_num(result, INFINITY); to get the result.

We'll keep throwing errors by default but add _ex variants that return the errors instead of throwing them.

IvanYashchuk · 2020-11-06T07:43:49Z

@mruberry, this PR is ready for starting the review process.
It adds torch.linalg.cond a function that computes the condition number, that is given a matrix A compute norm(A)*norm(inverse(A)).
out= variant is implemented. Autograd and JIT tests pass as well.

torch/linalg/__init__.py

…so no need to use max and min

Added complex dtype tests cases; Renamed ord -> p

IvanYashchuk · 2020-11-30T22:02:27Z

@mruberry, I updated this pull request:

Modified the implementation to return zero for 0x0 matrices and added tests for that.
Added checks whether the provided norm type p is valid.
Updated documentation to include a reference to svd for explaining why non-square inputs are allowed for p={None, -2, 2}

torch/linalg/__init__.py

mruberry · 2020-12-02T08:27:39Z

Update looks pretty good, @IvanYashchuk, just a last couple doc tweaks.

IvanYashchuk · 2020-12-02T09:27:58Z

@mruberry, I updated the docs per your suggested tweaks.

mruberry · 2020-12-02T09:32:15Z

torch/linalg/__init__.py

+          For other norms, however, :attr:`input` must be a square matrix or a batch of square matrices,
+          and if this requirement is not satisfied a RuntimeError will be thrown.
+
+.. note:: If :attr:`input` is a non-invertible matrix then a tensor containing infinity will be returned.


Darn, I just realized this note will need to be updated, too, since non-invertbility doesn't apply to both cases, and the next sentence is redundant with the previous note.

Sorry I missed that on the last pass, @IvanYashchuk.

I updated this note mentioning that it applies only for some p.

.. note:: For ``p = {'fro', 'nuc', inf, -inf, 1, -1}`` if :attr:`input` is a non-invertible matrix then a tensor containing infinity will be returned. If :attr:`input` is a batch of matrices and one or more of them is not invertible then a RuntimeError will be thrown.

Specified that this note applies only for p = {'fro', 'nuc', inf, -inf, 1, -1}

mruberry · 2020-12-03T08:09:21Z

Lint failure looks real, unfortunately:

{
    path: 'torch/linalg/__init__.py',
    start_line: 354,
    end_line: 354,
    start_column: 141,
    end_column: 141,
    annotation_level: 'failure',
    message: '[B950] line too long (140 > 120 characters)'
  }

IvanYashchuk · 2020-12-03T10:54:36Z

@mruberry I'm sorry about that. Seems like I need to install pre-commit hooks to avoid these situations.
Now it's fixed.

facebook-github-bot

@mruberry has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2020-12-04T11:14:55Z

@mruberry merged this pull request in cb28508.

Summary: This PR makes documentation for `cond` available at https://pytorch.org/docs/master/linalg.html I forgot to include this change in #45832. Pull Request resolved: #48941 Reviewed By: ngimel Differential Revision: D25379244 Pulled By: mruberry fbshipit-source-id: c8c0a0b8a05c17025d6c3cea405b2add369e2019

Added linalg_cond

7ce8a20

IvanYashchuk added module: numpy Related to numpy support, and also numpy compatibility of our operators module: linear algebra Issues related to specialized linear algebra operations in PyTorch; includes matrix multiply matmul labels Oct 5, 2020

IvanYashchuk mentioned this pull request Oct 5, 2020

torch.linalg in PyTorch 1.10 tracker #42666

Closed

4 tasks

pytorchbot added the open source label Oct 5, 2020

Updated the implementation.

dcff9ef

Now for p=None and p=±2 condition number is computed with svd. Added errorchecks.

facebook-github-bot added the cla signed label Oct 31, 2020

IvanYashchuk added 6 commits November 3, 2020 04:14

Merge remote-tracking branch 'upstream/master' into linalg_cond

d179001

Updated non-invertible case implementation

c88ccc3

Updated docs

1e49003

Added entry to common_methods_invocations.py

db2de7c

Added overrides.py entry

843c07f

flake8

fd82325

IvanYashchuk marked this pull request as ready for review November 3, 2020 11:56

IvanYashchuk requested a review from mruberry November 3, 2020 11:57

Implement a trick for replacing FLT_MAX or DBL_MAX to INFINITY

ed1b7de

IvanYashchuk added 6 commits November 3, 2020 07:23

Added out= variant

7be03c7

Updated implementation and tests. Complex input is now supported.

f36e732

Fix typo

de025dc

Trying to fix initialization of 'dim'

43f52bc

Merge remote-tracking branch 'upstream/master' into linalg_cond

39cc93f

Trying to fix initialization of 'dim'

6bded45

ejguan added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Nov 4, 2020

IvanYashchuk commented Nov 6, 2020

View reviewed changes

aten/src/ATen/native/LinearAlgebra.cpp Show resolved Hide resolved

IvanYashchuk commented Nov 6, 2020

View reviewed changes

mruberry reviewed Nov 6, 2020

View reviewed changes

torch/linalg/__init__.py Outdated Show resolved Hide resolved

IvanYashchuk added 8 commits November 29, 2020 11:59

Use IntArrayRef for constructing result_shape

c082881

Allow batched input with zero dimensions; singular values are sorted …

c1e4bd9

…so no need to use max and min

Added empty batch size test inputs

f5296cb

Revert changes to common_methods_invocations.py

b452379

Return 0 for 0x0 matrices

b15fedd

Added tests case for 0x0 matrices

73046d1

Added complex dtype tests cases; Renamed ord -> p

Updated note for p = 2, -2 to include reference to svd

5ed8a77

Added checks for norm type

e407098

mruberry reviewed Dec 2, 2020

View reviewed changes

torch/linalg/__init__.py Outdated Show resolved Hide resolved

mruberry reviewed Dec 2, 2020

View reviewed changes

torch/linalg/__init__.py Outdated Show resolved Hide resolved

Updated documentation

9008c10

mruberry reviewed Dec 2, 2020

View reviewed changes

Changed the note on non-invertible input

f615332

Specified that this note applies only for p = {'fro', 'nuc', inf, -inf, 1, -1}

IvanYashchuk added 2 commits December 3, 2020 04:39

flake8

994cca6

Merge remote-tracking branch 'upstream/master' into linalg_cond

898f855

facebook-github-bot reviewed Dec 3, 2020

View reviewed changes

facebook-github-bot closed this in cb28508 Dec 4, 2020

facebook-github-bot added the Merged label Dec 4, 2020

IvanYashchuk mentioned this pull request Dec 7, 2020

Added entry for torch.linalg.cond to linalg.rst #48941

Closed

IvanYashchuk mentioned this pull request Apr 20, 2021

Improved docs for torch.linalg #56265

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added computing matrix condition numbers (linalg.cond) #45832

Added computing matrix condition numbers (linalg.cond) #45832

IvanYashchuk commented Oct 5, 2020 •

edited

facebook-github-bot commented Oct 30, 2020

dr-ci bot commented Nov 3, 2020 •

edited

IvanYashchuk Nov 6, 2020

IvanYashchuk Nov 6, 2020

mruberry Nov 10, 2020

IvanYashchuk commented Nov 6, 2020

IvanYashchuk commented Nov 30, 2020

mruberry commented Dec 2, 2020

IvanYashchuk commented Dec 2, 2020

mruberry Dec 2, 2020

IvanYashchuk Dec 2, 2020

mruberry commented Dec 3, 2020

IvanYashchuk commented Dec 3, 2020

facebook-github-bot left a comment

facebook-github-bot commented Dec 4, 2020

Added computing matrix condition numbers (linalg.cond) #45832

Added computing matrix condition numbers (linalg.cond) #45832

Conversation

IvanYashchuk commented Oct 5, 2020 • edited

facebook-github-bot commented Oct 30, 2020

dr-ci bot commented Nov 3, 2020 • edited

💊 CI failures summary and remediations

IvanYashchuk Nov 6, 2020

Choose a reason for hiding this comment

IvanYashchuk Nov 6, 2020

Choose a reason for hiding this comment

mruberry Nov 10, 2020

Choose a reason for hiding this comment

IvanYashchuk commented Nov 6, 2020

IvanYashchuk commented Nov 30, 2020

mruberry commented Dec 2, 2020

IvanYashchuk commented Dec 2, 2020

mruberry Dec 2, 2020

Choose a reason for hiding this comment

IvanYashchuk Dec 2, 2020

Choose a reason for hiding this comment

mruberry commented Dec 3, 2020

IvanYashchuk commented Dec 3, 2020

facebook-github-bot left a comment

Choose a reason for hiding this comment

facebook-github-bot commented Dec 4, 2020

IvanYashchuk commented Oct 5, 2020 •

edited

dr-ci bot commented Nov 3, 2020 •

edited