transforms as standalone library

[clbarnes]

I was experimenting with some image slicing stuff and didn't find any nice generic standalone library providing an interface for general-purpose coordinate transforms. This kind of thing is obviously useful outside of neuron analysis and I like utilities you've got in navis.transforms. What would you think about splitting it out as a standalone library which navis depends on? The functions in navis could then be thin wrappers (if any wrapping was needed at all).

The hope would be that new coordinate transform libraries could use the base class provided by that library and be instantly compatible with downstream libraries, or thin wrappers could be added to the upstream library (like imageio/meshio). Downstream libraries would just have to be generic over the base class.

I hacked a bit on a simplified prototype with some API changes, some of which are much of a muchness:

minor

• use __call__() instead of xform(), so that they could be used interchangeably with regular free functions
• __add__() instead of .append() for producing a sequence (it would be quite nice to overload an operator for this, although I'm not sure __add__ is the right one)

could be added to navis

• add a class which simply wraps another callable

possibly more contentious

• every instance can hold a reference to which spaces it goes to and from - any hashable can be used as a space reference (probably strings in most cases). This allows you to build a transform sequences and a graph/ registry from an unordered bag of transforms, as well as doing some sanity checks.
• take coordinates in the form supported by scipy.ndimage.map_coordinates so that entire volumes can be transformed at once. In the 2D case, this is closer to what affine transforms use. It's also slightly closer to the output of numpy.meshgrid. It doesn't really matter in the 2D case as it's just a .T away, but having an arbitrary shape after the first dimension is handy for the image/volume case.

[schlegelp]

I'd be game for that. Would you want to outsource only the transforms or also the bridging system?

Re replacing .xform(): I actually went a bit off script there in that in nat this function is called xform_points and some transforms support transforming other types of data (e.g. images). This could off course be bundled into a __call__ function and simply raise NotImplementError if the data type is not supported. Might be easier from the outside to check if a transform has a xform_points and/or an xform_image method.

[schlegelp]

I went ahead and created a skeleton repo at https://github.com/schlegelp/xform

So far, I've only moved a bunch of code over from navis. I also reserved xform on PyPI but I'm not married to the name.

[schlegelp]

I now ripped out all navis-related stuff and made it so it can actually be imported. Not sure how much of a vision you have for this and if you want to have a crack at it now or later? In order to switch navis over, it would need to be fairly stable already.

[clbarnes]

Here was my prototype: https://github.com/clbarnes/transformnd (no tests or anything so it may not work). As you can see it's quite cut down, really targeted at being light and generic. navis' transformation needs wouldn't need to be entirely outsourced to a library which does 100% of what navis requires (it could always subclass or compose over the dependency) - I just had need of some core transformation logic elsewhere and was surprised there wasn't already an interface/aggregate library like imageio/meshio.

Would you want to outsource only the transforms or also the bridging system?

I think some sort of bridging system would be useful in the library but auto-loading transforms from particular directories and so on are probably best left at the application level (e.g. navis).

in nat this function is called xform_points and some transforms support transforming other types of data (e.g. images) ... Might be easier from the outside to check if a transform has a xform_points and/or an xform_image method.

Ah, got it. With scipy.ndimage.map_coordinates, anything which can transform coordinates can be used for image transformation, so a generic image transformation library could be one thing which is generic over BaseTransform subclasses. This issue was actually motivated by this experiment in arbitrary 2D slices of 3D images (a significant hole in napari), which ends up being able to do some 3Dscript-like stuff without too much effort - I started writing some transformation classes in imslice and it felt suspiciously familiar ;)

I went pretty hard on the "arbitrary shapes" thing in transformnd but I'm not absolutely set on it - the majority of transforms will need reshaping, and the arbitrary shapes is only useful to certain consumers, so it might best to provide a convenience function for those consumers to use and have the transformers only operate on DxN or NxD arrays.

[schlegelp]

🤦 I had actually been looking at your repos to check whether you had already made a start but must have missed this one.

What I would suggest is this:

You keep prototyping/trying out stuff in transformnd. While I get xform to a point that it would - in theory - work as drop-in replacement for navis. Then we get together and think about how to best combine the current base functionality with your improvements. Does that sound reasonable?

I very much like the idea of being able to transform image data and your other proposed changes. I would leave the code in xform close to what it is now and leave it to you to implement your ideas.

[schlegelp]

Looking more at your prototype, I see now that it is indeed very light (no H5 or CMTK transforms, minimal bridging capabilities). Do you prefer to keep it that way? I would have probably outsourced most of what's currently in navis.

Might be worth having a quick chat over this?

[clbarnes]

facepalm I had actually been looking at your repos to check whether you had already made a start but must have missed this one.

Only pushed it today!

no H5 or CMTK transforms

I would intend these to be included; they were just a bit more complex so weren't an immediate priority while I'm feeling it out. Mirror transforms too (for hyperplanes and possibly lower dimensions, e.g. reflecting around a point for 3D).

minimal bridging capabilities

I haven't worked with bridging transforms before - what else would be a core bridging feature for this kind of generic library, besides chaining together transforms between known spaces?