Tensor spaces, second try #861

kohr-h · 2017-01-25T13:25:24Z

Copied from #753

TODOs:

Implement basic structures for tensors and tensor spaces
Replicate functionality of NtuplesBase, FnBase, NumpyNtuples and NumpyFn in the corresponding tensor classes
Remove Ntuples and related stuff
Add former Ntuples and the likes as special case to tensor classes
Change names with "tensor" to something more lightweight
Find place for ~~MatVecOperator~~MatrixOperator, see Reload modules in Spyder causes bugs #584. Solved in Issue 910 matvec op #911
Improve pretty-printing of elements
Fix broken downstream code (lp_discr, ...)
Change code that was restricted to 1D data containers
Port unit tests
Write some doctests
Write more doctests
Remove TensorSet and others (see Scrap FunctionSet, DiscretizedSet and similar #860) duck-type space properties
Check documentation for outdated information, e.g. regarding 1D data layout (Update: can't find anything explicit)
~~Use notion "rank" instead of "number of dimensions" to better distinguish between space.ndim and space.size~~ Not doing this
Implement indexing of TensorSpace. ~~Done, but kind of wrong, see here and here~~ Done properly now
~~Write extra documentation on tensors~~ Should be part of Representing vector- and tensor-valued functions #908
Port show methods (1D from Ntuples, 2D from DiscreteLp)
Add unit tests for unusual data types and the likes
Simplify doctests of tensor spaces
Fix FunctionSpaceElement in 1D (again)
Implement tensor-valued functions, see Representing vector- and tensor-valued functions #908
Fix interpolation from array of points, related to Representing vector- and tensor-valued functions #908
Cleanup (old stuff, commit history, ...)
Fix large-scale tests
Make test with power function more general
Add more tests for product space multi-indexing
Implement indexing of FunctionSpace as
- indexing the components of the output (i.e. raise an error for scalar-valued functions) in __getitem__
- by axis of the domain using a byaxis attribute
Implement indexing of DiscreteLp as in FunctionSpace
Implement __array_ufunc__ interface on Tensor and NumpyTensor (the latter only differs in weight propagation)
Implement __array_ufunc__ interface on DiscreteLpElement
Write more extensive tests for __array_ufunc__ intefaces (bad input etc.)

kohr-h · 2017-01-26T17:33:46Z

Implement indexing of TensorSpace

Regarding this topic: I have implemented a simple indexing capability for spaces, but it is nowhere near as powerful and flexible as the lazy approach of simply indexing the array and appropriately wrap whatever comes out.
So the earlier suggestion

arr = self.data[indices]
return self.space[indices].element(arr)

is much much less powerful than lazy indexing, if we don't put great amounts of work into it.

So I'd say the space indexing can be used for simple things like

>>> space = odl.rn((2, 3, 4))
>>> space[[0, 2]]
rn((2, 4))
>>> space[0]
rn(2)

but not for indexing of elements.

kohr-h · 2017-01-26T22:55:23Z

Regarding this topic: I have implemented a simple indexing capability for spaces, but it is nowhere near as powerful and flexible as the lazy approach of simply indexing the array and appropriately wrap whatever comes out.
So the earlier suggestion

arr = self.data[indices]
return self.space[indices].element(arr)

is much much less powerful than lazy indexing, if we don't put great amounts of work into it.

So I'd say the space indexing can be used for simple things like

space = odl.rn((2, 3, 4))
space[[0, 2]]
rn((2, 4))
space[0]
rn(2)

but not for indexing of elements.

I realized that indexing like this is not really helpful since it's not compatible with power spaces. I'll think about it a bit more.

adler-j · 2017-02-08T22:41:53Z

Is this in a state where it needs a review? If so I guess I need to get at it.

kohr-h · 2017-02-08T22:44:45Z

Is this in a state where it needs a review? If so I guess I need to get at it.

Yes it is. I deliberately didn't mention it so nobody burns time at it that they don't have :-)

adler-j · 2017-02-09T07:05:40Z

Could you do the rebase from master first? It seems rather big.

kohr-h · 2017-02-09T10:38:16Z

Could you do the rebase from master first? It seems rather big.

Yes, I've also spotted some glitches that I want to fix along the way.

kohr-h · 2017-02-10T18:01:25Z

Rebased

adler-j

Huge review done, but mostly small things. I have not reviewed the tests yet, will do that later on.

On the larger scale, I have a few comments:

The support for DiscreteLp with vector/tensor valued functions needs to be added and properly supported. This is the largest out-standing problem with this.
That all examples default to shape=(2,3) makes the doc much harder to read and adds little. I understand that during development this is important, but in release we should trim this in general, just like we dont use dtype='float32' everywhere.
You have added __hash__ in a few places where it shouldn't, i.e. in mutable objects. Also I think you may have missed adding __ne__ in some places, check that.
The matrix_representation/MatrixOperator thing needs to be fixed, but could use a new issue.

adler-j · 2017-01-25T13:31:50Z

odl/test/largescale/space/tensor_space_slow_test.py


-    x0 = np.arange(fn.size)
-    x = fn.element(x0)
+    x0 = np.arange(tspace.size)


tspace.shape?

Haven't looked much into this file, probably needs more love

adler-j · 2017-02-13T10:45:11Z

doc/source/dev/extend.rst

-Adding Fn spaces
----------------
-The abstract spaces `FnBase` and `NtuplesBase` are the workhorses of the ODL space machinery. They are used in both the discrete :math:`R^n` case, as well as data representation for discretized function spaces such as :math:`L^2([0, 1])` in the `DiscretizedSpace` class. These are in general created through the `rn` and `uniform_discr` functions who take an ``impl`` parameter, allowing users to select the backend to use.
+Adding Tensor spaces


Skipping the doc for now.

adler-j · 2017-02-13T10:45:48Z

doc/source/generate_doc.py

@@ -63,26 +63,26 @@


 def import_submodules(package, name=None, recursive=True):
-    """ Import all submodules of a module, recursively, including subpackages
-    """
+    """Recursively import all submodules of ``package``."""


Improve. I guess it returns the names etc?

adler-j · 2017-02-13T10:46:17Z

doc/source/guide/glossary.rst

@@ -11,7 +11,7 @@ Glossary

    array-like
        Any data structure which can be converted into a `numpy.ndarray` by the
-        `numpy.array` constructor. Includes all `NtuplesBaseVector` based classes.
+        `numpy.array` constructor. Includes all `Tensor` based classes.


The NtuplesBaseVector -> Tensor may be our best rename in a long time!

Yeah scrap those shitty old names!

adler-j · 2017-02-13T10:46:29Z

doc/source/guide/glossary.rst

@@ -22,7 +22,7 @@ Glossary

    discretization
        Structure to handle the mapping between abstract objects (e.g. functions) and concrete, finite realizations.
-        It encompasses an abstract `Set`, a finite data container (`NtuplesBaseVector` in general) and the mappings between them,         :term:`sampling` and :term:`interpolation`.
+        It encompasses an abstract `Set`, a finite data container (`GeneralizedTensor` in general) and the mappings between them,         :term:`sampling` and :term:`interpolation`.


Didn't we scrap GeneralizedTensor?

Yes, needs to be replaced

adler-j · 2017-02-13T14:45:07Z

odl/space/pspace.py

+            return ProductSpace(*self.spaces[indices], field=self.field)
+        elif isinstance(indices, tuple):
+            if len(indices) > 1:
+                raise ValueError('too many indices: {}'.format(indices))


Another option is to pass these through further down, i.e.

return ProductSpace(self.spaces[indices[0]][indices[1:]], field=self.field)

adler-j · 2017-02-13T14:46:04Z

odl/space/pspace.py

@@ -848,7 +854,7 @@ def show(self, title=None, indices=None, **kwargs):
        --------
        odl.discr.lp_discr.DiscreteLpElement.show :
            Display of a discretized function
-        odl.space.base_ntuples.NtuplesBaseVector.show :
+        odl.space.base_tensors.GeneralizedTensor.show :


find replace GeneralizedTensor

adler-j · 2017-02-13T14:46:19Z

odl/space/space_utils.py

-    impl : string, optional
-        The backend to use. See `odl.space.entry_points.NTUPLES_IMPLS` and
-        `odl.space.entry_points.FN_IMPLS` for available options.
+    order : {'C', 'F'}, optional


Clearly missing

adler-j · 2017-02-13T14:46:32Z

odl/space/space_utils.py

-        `odl.space.entry_points.FN_IMPLS` for available options.
+    order : {'C', 'F'}, optional
+        Axis ordering of the data storage.
+    impl : str


adler-j · 2017-02-13T14:48:54Z

odl/space/weighting.py

        if self.const <= 0:
            raise ValueError('expected positive constant, got {}'
                             ''.format(const))
        if not np.isfinite(self.const):
            raise ValueError('`const` {} is invalid'.format(const))

+    def __hash__(self):


Prefer if __hash__ is always right after __eq__ since they are so strongly connected.

kohr-h · 2017-02-13T22:06:22Z

Thanks a lot for the detailed review @adler-j, will try to get this done asap.

adler-j · 2017-02-14T15:54:17Z

odl/space/npy_tensors.py

+    """Weighting of a `NumpyTensorSpace` with constant 1."""
+
+    # Implement singleton pattern for efficiency in the default case
+    __instanc


You use a 3 dimensional array later.

Anyway, if we allow 2+, why not allow 1d as well, in which case this is an inner product.

Wrong place again, you need to hack it apparently.

The comment is w.r.t MatrixOperator where you assume the matrix is 2 or higher dimensional. I can easily see this working with 1 or even 0 dimensional arrays, so that condition is arbitrary.

The line "You use a 3 dimensional array later." refers to a comment that MatrixOperator input should be 2d, but you later use 3d. Also you later say that axis needs to be 1 or more, but the default is 0.

adler-j · 2017-03-28T07:59:35Z

What is the review status on this?

kohr-h · 2017-04-01T09:22:52Z

What is the review status on this?

I figured that sampling would need tensor-valued functions, so I implemented them. Now I'm in the middle of writing/fixing the tests for those.

adler-j · 2017-07-14T21:50:06Z

Can you re-open this for a new review, if you want one, else we really want to go ahead here.

kohr-h · 2017-08-10T10:35:35Z

After thinking about it some more, I think the following way of indexing spaces is most reasonable:

NumpyTensorSpace:
- __getitem__: Numpy indexing, should wrap an array indexed in the same way
- byaxis: index axes of the tensors
FunctionSpace:
- __getitem__: Numpy indexing for the out_shape part, i.e., for tensor-valued functions. Raise an error for scalar-valued functions.
- byaxis: index the domain per axis
DiscreteLp: same as FunctionSpace

Opinions?

kohr-h · 2017-08-11T08:40:14Z

The page for this PR has become unworkable once again, opening a new one.

kohr-h mentioned this pull request Jan 25, 2017

WIP: CPU tensors #753

Closed

19 tasks

kohr-h force-pushed the issue-342__tensor branch from f530983 to 8a6d51f Compare January 26, 2017 13:50

kohr-h mentioned this pull request Jan 26, 2017

Scrap FunctionSet, DiscretizedSet and similar #860

Closed

kohr-h force-pushed the issue-342__tensor branch from 59b0936 to ee061ea Compare January 26, 2017 15:55

kohr-h force-pushed the issue-342__tensor branch from 6a1dd6e to 5292f10 Compare February 10, 2017 18:00

kohr-h added the status: review needed label Feb 10, 2017

adler-j requested changes Feb 13, 2017

View reviewed changes

adler-j mentioned this pull request Feb 14, 2017

proximal operators with matix valued "step size" #904

Closed

adler-j reviewed Feb 14, 2017

View reviewed changes

This was referenced Feb 15, 2017

What do we mean by slicing in DiscreteLp spaces? #907

Open

Representing vector- and tensor-valued functions #908

Open

Weighting with list of 1d vectors #909

Open

Issue 910 matvec op #911

Merged

adler-j mentioned this pull request Feb 24, 2017

Sampling Operator (sample coefficients of discretised function) #936

Closed

kohr-h force-pushed the issue-342__tensor branch 2 times, most recently from 4fcd3a7 to 3183484 Compare February 27, 2017 22:28

adler-j mentioned this pull request Feb 28, 2017

multi indexing of functions #943

Closed

kohr-h force-pushed the issue-342__tensor branch 2 times, most recently from ebddf98 to ac45d52 Compare March 20, 2017 09:39

kohr-h mentioned this pull request Mar 20, 2017

initial version of sampling and embedding operators #940

Merged

MAINT: fix failing tests after rebasing

e28fdb1

kohr-h mentioned this pull request Jul 10, 2017

GPU example #1074

Open

Holger Kohr added 4 commits July 25, 2017 16:07

MAINT: port upstream optimizations to tensors

da3222c

ENH: add __array_ufunc__ interface to base_tensors

b338b3e

WIP: add __array_ufunc__ to DiscreteLpElement

47e5cae

ENH: add Numpy ufuncs and some tests

0c5cf2a

kohr-h mentioned this pull request Aug 7, 2017

Numpy 1.13 support #1085

Closed

Holger Kohr added 7 commits August 8, 2017 22:18

TST: add test for tensor __array_ufunc__ with Numpy array as out

9b4807a

MAINT: minor doc fix in base_tensors __array_ufunc__

8fd1b21

MAINT: remove unnecessary stuff from ufuncs.py

1ac314d

MAINT: allow data container in base __array_ufunc__

8cc41c3

WIP: implement DiscreteLp array_ufunc

c286f51

MAINT: input check fixes to __array_ufunc__

3cbbcf8

WIP: implement DiscreteLp array_ufunc

05d199d

Holger Kohr added 7 commits August 10, 2017 18:55

BUG: fix partition byaxis indexing with lists

94d760b

MAINT: fix failing doctest in IntervalProd

a9ef25c

ENH: implement byaxis[in/out] indexing for FunctionSpace

fdfaa41

MAINT: improve handling of weighting in base tensors

1dae25c

BUG: fix handling of out in tensor __array_ufunc__

8b13d8a

BUG: fix tensor space byaxis indexing for lists

f441496

WIP: implement DiscreteLp __array_ufunc__

2bd7f2b

kohr-h mentioned this pull request Aug 10, 2017

Indexing of multi-dimensional spaces #965

Open

Holger Kohr added 2 commits August 10, 2017 23:47

BUG: fix insert of grid and interval product for 0 args

ba8ab82

ENH: implement __array_ufunc__ for DiscreteLp

be428fe

kohr-h closed this Aug 11, 2017

kohr-h mentioned this pull request Aug 11, 2017

CPU tensors #1088

Merged

29 tasks

kohr-h removed the status: review needed label Oct 6, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tensor spaces, second try #861

Tensor spaces, second try #861

kohr-h commented Jan 25, 2017 •

edited

Loading

kohr-h commented Jan 26, 2017

kohr-h commented Jan 26, 2017

adler-j commented Feb 8, 2017

kohr-h commented Feb 8, 2017

adler-j commented Feb 9, 2017

kohr-h commented Feb 9, 2017

kohr-h commented Feb 10, 2017

adler-j left a comment

adler-j Jan 25, 2017

kohr-h Feb 13, 2017

adler-j Feb 13, 2017

adler-j Feb 13, 2017

kohr-h Feb 13, 2017

adler-j Feb 13, 2017

kohr-h Feb 27, 2017

adler-j Feb 13, 2017

kohr-h Feb 13, 2017

adler-j Feb 13, 2017

adler-j Feb 13, 2017

adler-j Feb 13, 2017

kohr-h Feb 13, 2017

adler-j Feb 13, 2017

adler-j Feb 13, 2017

kohr-h commented Feb 13, 2017

adler-j Feb 14, 2017

kohr-h Feb 15, 2017

adler-j Feb 15, 2017

adler-j commented Mar 28, 2017

kohr-h commented Apr 1, 2017

adler-j commented Jul 14, 2017

kohr-h commented Aug 10, 2017

kohr-h commented Aug 11, 2017

Tensor spaces, second try #861

Tensor spaces, second try #861

Conversation

kohr-h commented Jan 25, 2017 • edited Loading

kohr-h commented Jan 26, 2017

kohr-h commented Jan 26, 2017

adler-j commented Feb 8, 2017

kohr-h commented Feb 8, 2017

adler-j commented Feb 9, 2017

kohr-h commented Feb 9, 2017

kohr-h commented Feb 10, 2017

adler-j left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kohr-h commented Feb 13, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adler-j commented Mar 28, 2017

kohr-h commented Apr 1, 2017

adler-j commented Jul 14, 2017

kohr-h commented Aug 10, 2017

kohr-h commented Aug 11, 2017

kohr-h commented Jan 25, 2017 •

edited

Loading