normalizeForMatching not sufficiently general

[fcollman]

I am having trouble running the spark SIFT client and getting matches because it assumes i want to do normalizeForMatching, and normalizeForMatching assumes simply removing the last transform in the transform list will bring the tile back to something where putting x,y at 0,0 will bring the image into view. This is true if all you have a lens correction and a single transformation to bring it into the world, but in the case of longer lists of transforms this is not true. In particular I happen to have 4 transformations after using trakem2 to

not sure what the fix is... a few options i can think of to brainstorm.

1. make normalizeForMatching an option so those who don't want it can skip it. (seems like a bad solution given the need for evenly sized tiles is certain matchers). Incidentally i don't see how this solution guarantees consistently sized tiles if the lens corrections are sufficiently different.

2. give some options to hardcode the size of the tile to cutout, understanding that the user needs to make it large enough to cover the transformed tile given its current transformed location into the world.

3. completely rewrite the point match service to calculate and express matches in the original dataspace, meaning pre-transformed data space of the original mipmaplevel 0 image, not this sort of partially transformed standard that currently exists.
   (i realize that this is a major change...I don't see how you can really maintain meaningful matches across transformation sets any other way, and if matches are tied to a particular transformation set, than that relationship should be made explicit and tracked rather than ad hoc).

4. If the goal of normalization is to bring the tile back to this "original" state, but with only lens correction included, than the tilespec schema could be extended to more explicitly call out that this transformation (or set of transformations) as special/pre-alignment/raw-correction or something, and other transformations are not... and then the normalizeForMatching code should use this distinction to remove all the non special/pre-alignment/raw-correction type transformations. Could be as easy as adding a flag to the transform class of, is-raw or not.

5. do not assume that you can shift the origin to 0,0 after removing the transforms that you did.. instead dynamically calculate the bounding box after you remove the transformations that you have. This is I think the easiest fix i can think of.

I think I like 4 the best, but 5 would also be ok...
thoughts?

[fcollman]

In the interim I have hacked a solution which works for me because i have no lenscorrections, i suppose this corresponds to solution 3 above, as the image will be rendered in its original untransformed state.

'diff --git a/render-ws/src/main/java/org/janelia/render/service/TileDataService.java b/render-'ws/src/main/java/org/janelia/render/service/TileDataService.java
'index 51bef06..37b80ec 100644
'--- a/render-ws/src/main/java/org/janelia/render/service/TileDataService.java
'+++ b/render-ws/src/main/java/org/janelia/render/service/TileDataService.java
'@@ -187,13 +187,13 @@ public class TileDataService {
' tileRenderHeight = (int) (tileSpec.getHeight() * normalizationFactor);
' }
'
'- tileSpec.removeLastTransformSpec();
'+ //tileSpec.removeLastTransformSpec();
'
' // If the tile still has more than 3 transforms, remove all but the last 3.
' // This assumes that the last 3 transforms are for lens correction.
' // Hopefully at some point we'll label transforms so that it is possible to
' // explicitly include only lens correction transforms.
'- while (tileSpec.getTransforms().size() > 3) {
'+ while (tileSpec.getTransforms().size() > 0) {
' tileSpec.removeLastTransformSpec();
' }
'

[trautmane]

Folks here (Stephan, Khaled, Eric P.) concluded that matches should be derived from lens corrected data - not raw data. Since we don't currently have a generic way to identify the lens corrected form of a tile, I hacked the normalization option to work with our data as it was. I think it makes sense to add support for labeling transforms and update the normalization process to use those labels - basically the approach you mentioned in option 4.

I won't likely get to this for a while. I'm assuming your work-around is sufficient for the short term. If that is a poor assumption, let me know.

[fcollman]

OK, sounds good. I think i have a temporary solution... somewhat annoying to keep it synced up as you move forward in other areas, but I'll make it work. Most important wanted to document the issue somewhere in case others ran into it.

[fcollman]

Is this same issue a factor in toLayoutFileFormat
https://github.com/saalfeldlab/render/blob/master/render-app/src/main/java/org/janelia/alignment/spec/TileSpec.java

in that this is assuming that the last transform is a affine, and that all the other ones are "raw" transforms?

The EM aligner seems to not be working for me, and I can see that it's initial transformations are using this toLayoutfileFormat to retrieve the transformations.. but I have a rough alignment that is the composition of several transformations, so that last transform alone doesn't make much sense..

I'm happy to modify EM aligner or change the java but i'd like advice as to which is better to do.

[fcollman]

I think this is now addressed by the recent commits, perhaps it should remain open until we have more thoroughly tested it.

labels is now an added field to transformSpecs the can contain one or more labels

and

?excludeTransformsAfterLast=lens 
?excludeFirstTransformAndAllAfter=rough
?excludeAllTransforms=true

now implement various ways to filter out transformations.

I think this addresses the underlying issues at the render-ws api level. More work is needed to utilize this functionality in the clients (point matching, transformchanges, etc).

[fcollman]

I believe this issue is now closed due to the new filters we have.