Feature/338 matrix inverse

[mtar]

Description

An implementation of the matrix inverse using Gauss-Jordan elimination when the matrix is distributed.
torch.linalg.inv is called on the local tensors otherwise.

Issue/s resolved: #338

Changes proposed:

• add linalg.inv

Type of change

• New feature

Due Diligence

• All split configurations tested
• Multiple dtypes tested in relevant functions
• Documentation updated (if needed)
• Updated changelog.md under the title "Pending Additions"

Does this change modify the behaviour of other functions? If so, which?

no

[codecov]

Codecov Report

Merging #875 (a17d126) into master (dbb8300) will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #875      +/-   ##
==========================================
+ Coverage   95.47%   95.49%   +0.02%     
==========================================
  Files          64       64              
  Lines        9741     9793      +52     
==========================================
+ Hits         9300     9352      +52     
  Misses        441      441              

Flag	Coverage Δ
gpu	94.58% <100.00%> (+0.02%)	⬆️
unit	91.07% <100.00%> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
heat/core/linalg/basics.py	95.52% <100.00%> (+0.26%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dbb8300...a17d126. Read the comment docs.

[coquelin77]

so ive done a bit of testing on this branch.

ht.random.seed(42)
split = 1
a = ht.random.random((20, 20), dtype=ht.float64, split=split)
ainv = ht.linalg.inv(a)
i = ht.eye(a.shape, split=split, dtype=a.dtype)
print(ht.max((a @ ainv) - i))
print(ht.allclose(a @ ainv, i))

The above code give this:

(base) daniel@daniel-LIFEBOOK-U749:~/.git/heat$ mpirun -np 1 python testing.py 
DNDarray([1.5736e-13], dtype=ht.float64, device=cpu:0, split=None)
True
(base) daniel@daniel-LIFEBOOK-U749:~/.git/heat$ mpirun -np 2 python testing.py 
DNDarray([3.5616e-13], dtype=ht.float64, device=cpu:0, split=None)
True

However, if i use split=0...

(base) daniel@daniel-LIFEBOOK-U749:~/.git/heat$ mpirun -np 1 python testing.py 
DNDarray([1.2335e-05], dtype=ht.float64, device=cpu:0, split=None)
False
(base) daniel@daniel-LIFEBOOK-U749:~/.git/heat$ mpirun -np 2 python testing.py 
DNDarray([1.2335e-05], dtype=ht.float64, device=cpu:0, split=None)
False

Is this some artifact of the algorithm?

[mtar]

@coquelin77 This is some weird behaviour of the indexing methods of DNDarray and the division operation . It's now the same precision on both splits.