Image Jacobians

[BenedictChannn]

Hi all,

If my residual is a pixel of a CV image (indexed sf.DataBuffer), the Jacobians and Hessians are set to zero. I need to account for the gradient of the CV image and also the Jacobian of the reprojection of a 3D point from a reference frame to the current frame. Defining a custom Jacobian would not be ideal since I am dealing with a lot of symbolic variables. Is there a way to deal with this? Especially since the CV image is now represented as a 1D array instead the standard Matrix.

Ideally, the autogenerated Jacobians should include the image intensity gradient and the Jacobian of the warping function as well.

Also, is there a way to handle conditional statements for symbolic computations? Are there other similar functions/ capabilities like those in logic.py

I also noticed that when using sf.logical_or operations with more than 2 arguments (unsafe = True) ,output is max(True, False) when it should by right be True?

I would appreciate any help on the above!! Thanks.

[aaron-skydio]

Addressed here, will continue discussion there: #348 (comment)

In particular, this is not a bug - a DataBuffer is a piecewise constant function of its index, and if you want a notion of an image gradient you need to represent that