Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Features/98 moments - skew and kurtosis #615

Merged
merged 19 commits into from
Jul 2, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,17 @@
- [#577](https://github.com/helmholtz-analytics/heat/pull/577) Add `ndim` property in dndarray
- [#578](https://github.com/helmholtz-analytics/heat/pull/578) Bugfix: Bad variable in reshape
- [#580](https://github.com/helmholtz-analytics/heat/pull/580) New feature: `fliplr()`
- [#581](https://github.com/helmholtz-analytics/heat/pull/581) New Feature: `DNDarray.tolist()`
- [#581](https://github.com/helmholtz-analytics/heat/pull/581) New feature: `DNDarray.tolist()`
- [#583](https://github.com/helmholtz-analytics/heat/pull/583) New feature: `rot90()`
- [#593](https://github.com/helmholtz-analytics/heat/pull/593) New feature `arctan2()`
- [#593](https://github.com/helmholtz-analytics/heat/pull/593) New feature: `arctan2()`
- [#594](https://github.com/helmholtz-analytics/heat/pull/594) New feature: Advanced indexing
- [#594](https://github.com/helmholtz-analytics/heat/pull/594) Bugfix: getitem and setitem memory consumption heavily reduced
- [#594](https://github.com/helmholtz-analytics/heat/pull/594) Bugfix: `__getitem__` and `__setitem__` memory consumption heavily reduced
- [#596](https://github.com/helmholtz-analytics/heat/pull/596) New feature: `outer()`
- [#598](https://github.com/helmholtz-analytics/heat/pull/598) Type casting changed to PyTorch style casting (i.e. intuitive casting) instead of safe casting
- [#600](https://github.com/helmholtz-analytics/heat/pull/600) New feature: `shape()`
- [#614](https://github.com/helmholtz-analytics/heat/pull/614) New feature: printing of DNDarrays and ``__repr__`` and ``__str__`` functions
- [#615](https://github.com/helmholtz-analytics/heat/pull/615) New feature: `skew()`
- [#615](https://github.com/helmholtz-analytics/heat/pull/615) New feature: `kurtosis()`
- [#618](https://github.com/helmholtz-analytics/heat/pull/618) Printing of unbalnced DNDarrays added

# v0.4.0
Expand Down
8 changes: 7 additions & 1 deletion heat/cluster/spectral.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,13 @@ def _spectral_embedding(self, X):
"""
L = self._laplacian.construct(X)
# 3. Eigenvalue and -vector calculation via Lanczos Algorithm
v0 = ht.ones((L.shape[0],), dtype=L.dtype, split=0, device=L.device) / math.sqrt(L.shape[0])
v0 = ht.full(
(L.shape[0],),
fill_value=1.0 / math.sqrt(L.shape[0]),
dtype=L.dtype,
split=0,
device=L.device,
)
V, T = ht.lanczos(L, self.n_lanczos, v0)

# 4. Calculate and Sort Eigenvalues and Eigenvectors of tridiagonal matrix T
Expand Down
56 changes: 51 additions & 5 deletions heat/core/dndarray.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
import math
import torch
import warnings
from typing import List
from typing import List, Union

from . import arithmetics
from . import devices
Expand Down Expand Up @@ -147,10 +147,7 @@ def size(self):
size : int
number of total elements of the tensor
"""
try:
return np.prod(self.__gshape)
except TypeError:
return 1
return torch.prod(torch.tensor(self.gshape, device=self.device.torch_device)).item()

@property
def gnumel(self):
Expand Down Expand Up @@ -1703,6 +1700,35 @@ def item(self):
"""
return self.__array.item()

def kurtosis(self, axis=None, unbiased=True, Fischer=True):
coquelin77 marked this conversation as resolved.
Show resolved Hide resolved
"""
Compute the kurtosis (Fisher or Pearson) of a dataset.
TODO: add return type annotation (DNDarray) and x annotation (DNDarray)

Kurtosis is the fourth central moment divided by the square of the variance.
If Fisher’s definition is used, then 3.0 is subtracted from the result to give 0.0 for a normal distribution.

If unbiased is True (defualt) then the kurtosis is calculated using k statistics to
eliminate bias coming from biased moment estimators

Parameters
----------
x : ht.DNDarray
Input array
axis : NoneType or Int
Axis along which skewness is calculated, Default is to compute over the whole array `x`
unbiased : Bool
if True (default) the calculations are corrected for bias
Fischer : bool
Whether use Fischer's definition or not. If true 3. is subtracted from the result.

coquelin77 marked this conversation as resolved.
Show resolved Hide resolved
Warnings
--------
UserWarning: Dependent on the axis given and the split configuration a UserWarning may be thrown during this
function as data is transferred between processes
"""
return statistics.kurtosis(self, axis, unbiased, Fischer)

def __le__(self, other):
"""
Element-wise rich comparison of relation "less than or equal" with values from second operand (scalar or tensor)
Expand Down Expand Up @@ -3227,6 +3253,26 @@ def sinh(self, out=None):
"""
return trigonometrics.sinh(self, out)

def skew(self, axis=None, unbiased=True):
coquelin77 marked this conversation as resolved.
Show resolved Hide resolved
"""
Compute the sample skewness of a data set.

Parameters
----------
x : ht.DNDarray
Input array
axis : NoneType or Int
Axis along which skewness is calculated, Default is to compute over the whole array `x`
unbiased : Bool
if True (default) the calculations are corrected for bias

Warnings
--------
UserWarning: Dependent on the axis given and the split configuration a UserWarning may be thrown during this
function as data is transferred between processes
"""
return statistics.skew(self, axis, unbiased)

def sqrt(self, out=None):
"""
Return the non-negative square-root of the tensor element-wise.
Expand Down
4 changes: 2 additions & 2 deletions heat/core/linalg/solver.py
Original file line number Diff line number Diff line change
Expand Up @@ -155,13 +155,13 @@ def lanczos(A, m, v0=None, V_out=None, T_out=None):
b = torch.dot(vi_loc, vi_loc)
A.comm.Allreduce(ht.communication.MPI.IN_PLACE, a, ht.communication.MPI.SUM)
A.comm.Allreduce(ht.communication.MPI.IN_PLACE, b, ht.communication.MPI.SUM)
vr._DNDarray__array = vr._DNDarray__array - a / b * vi_loc
vr._DNDarray__array -= a / b * vi_loc

vi = vr / ht.norm(vr)

w = ht.matmul(A, vi)
alpha = ht.dot(w, vi)
w = w - alpha * vi - beta * V[:, i - 1]
w -= alpha * vi - beta * V[:, i - 1]

T[i - 1, i] = beta
T[i, i - 1] = beta
Expand Down
Loading