Tool to check for orientation

[JulesScholler]

Not really an issue but more of a question.

Is there a practical way to check the orientation of the data in a user friendly manner? I usually use BG-space to apply a transformation and then display the data along the Atlas but it requires some coding. I found out that the most difficult part of the registration for users is actually finding the orientation because if you mess up then the registration result is bad and people will start tweaking the parameters without realizing that the data was not aligned correctly.

Thanks and keep up the amazing work :).

[adamltyson]

Hey @JulesScholler, does this need a tool, or just better documentation? TBH I thought the docs were better, but it's certainly not trivial to work out the orientation from what we have.

You shouldn't have to reorient your data before using brainreg.

[JulesScholler]

You can add more documentation but I think most people are not going to understand right away how to define the orientation with the 3 letters scheme. Just displaying the transformed input data and the atlas before registration as a validation step is much easier (I think) to grasp if the input orientation is correct or not.

Don't get me wrong, the doc is great, but I still sometime fuck up the determination of the orientation and if I had visualized the transformed data beforehand I could have spotted sooner the problem.

MIRACL pipeline has another approach where they show the user max projections and ask what plane it is (sagital, horizontal, etc) and then figures out by itself the input orientation. I guess this could be an option in brainreg, or even better an automatic way of determining the input orientation using a simple classifier and a few features computed on the max-projections (e.g. sphericity and area of the segmented shape).

[adamltyson]

I like these ideas, but I'm hesitant to add more code to solve a documentation problem (I've got limited time to support/maintain this in my spare time), but more "interactive" docs (e.g. if your data is X, then the first letter is Y etc etc) may help this problem. Displaying data is also a pain as many users use the CLI tool for very large datasets on very slow network drives.

[JulesScholler]

In my view I would add a parsed argument --orientation-help that can be

• 'off' (default) and the behavior is like now
• 'gui' and you display for validation or use MIRACL approach
• 'automatic' and you use an automatic approach at your own risk

Doing that you add more flexibility without changing the behavior for current users and the default behavior is not adding any latency for network drives.

[adamltyson]

I'd love that, but TBH I just can't commit to anything other than bug fixes at this point.

[JulesScholler]

I will dedicate some time for this issue, i.e. having an automatic way of determining the input orientation and having an interactive way of fixing the output. Requirements are:

• Should work with single hemispheres
• Should work with not aligned data (to some extent)
• Should be quick to not add unwanted overhead

The workflow I see would be to add an orientation option 'interactive' or 'auto' which would try to automatically guess the input orientation. Then it would open a Napari window with reference and oriented input data and buttons to correct by rotating horizontally and vertically. When the data is oriented with the reference brain the user closes the Napari viewer and the rest of the pipeline is run.

How about that @adamltyson ? I will start working on automatically finding the input orientation.

[adamltyson]

Sounds great! My only thought is that if it starts up napari, maybe it's better placed in the brainreg-napari plugin, and the brainreg command line tool can continue to be a fully automated, headless version?

Users could use the napari plugin to begin with, then use the CLI once they're confident of the orientation, and have optimised registration parameters.

Thanks for looking into this, the orientation is confusing a few people!

[JulesScholler]

Ok so the 'automatic' option will be available in the CLI and we can add it later on in the Napari plugin. I agree the interactive approach should be added only in the Napari plugin since it requires this dependency.

[adamltyson]

Sounds great. Thanks @JulesScholler!

[JulesScholler]

So I tried different 'physics' things but they all failed when testing on different angles (I added a random rotation of 0 up to 15° around one of the three axis). Basically looking at the inertia tensor can tell you how to find the three axis but not the direction so you end up with 'a/p' 's/i' and 'l/r' without being sure which one it is.

I will focus on finding some symmetry measures to try and find the direction.

Last approach would we to train a random forest or equivalent classification tool using a few features on the maximum intensity projection but this will require extra work and might be a pain to package because the trained classifier should ship with brainreg.

[JulesScholler]

After playing quite a while with it I think that it is a bad idea to try and find automatically the orientation. Main argument is that the method will either be atlas specific or need machine learning and we don't want that in brainreg. I made a tool to interactively check that the orientation is good:

Top is the reference atlas, averaged for all directions and bottom is the aligned input data with wrong input

Top is the reference atlas, averaged for all directions and bottom is the aligned input data with correct input

Advantages is that it works out of the box with all the atlases, drawback is you need to load the atlas twice (once for interactive alignment and one when the pipeline is run). Let me know if you like the solution and I'll send a PR.

[adamltyson]

I think you just raised a PR, but this looks ideal, thanks! Loading the atlas twice isn't a bad idea, we could always load a lower-res version if available too.

machine learning and we don't want that in brainreg

Glad you agree. I'm trying to keep brainreg the "simple" registration tool, not necessarily the "best".

[JulesScholler]

We are testing it internally but so far so good! It takes 10 seconds to figure out how to orient correctly the brain and it is quite intuitive (swap letter when images are swapped or change letter when the symmetry is wrong).

Hopes this get merged soon and we can deploy it easily to our users :).

[adamltyson]

We are testing it internally but so far so good!

Awesome. I think we've got one more brainreg PR to merge on our side, then we can get the orientation PR merged, then this one. I'm answering a lot of orientation questions, so this would make my life a lot easier!

[adamltyson]

This is merged now. Thanks @JulesScholler. Hope you don't mind I've turned your comment above into some brief documentation.