[ENH]: Specify a custom focal length / FOV for the 3d camera

[scottshambaugh]

Problem

I'd like to be able to recreate the perspective-warping effects of changing focal length / FOV of a physical camera in matplotlib, for more fine-tuned camera control and for replicating real-world cameras in software.

Proposed solution

From this slide deck, we see how to generate a projection matrix:

This page has a nice derivation of the matrix.

The persp_transformation function in /lib/mpl_toolkits/mplot3d/proj3d.py implements this for a fixed focal distance of 1 (equivalent to a FOV of 90 deg):

def persp_transformation(zfront, zback):
    a = (zfront+zback)/(zfront-zback)
    b = -2*(zfront*zback)/(zfront-zback)
    return np.array([[1, 0, 0, 0],
                     [0, 1, 0, 0],
                     [0, 0, a, b],
                     [0, 0, -1, 0]])

It should not be too hard to update this function to take in a focal length. I think the biggest question is the user interface. The focal_length can be taken as fundamental and the current proj_type arguments to the Axes3D constructor can be mapped to focal_length = 1 for proj_type == 'persp'. To specify a custom focal length however, what should the user input?

Edit: I think the easiest thing would be a default focal_length = None argument, which can be set by the user but otherwise gets set to 1 if proj_type == 'persp'.

[tacaswell]

I think the best path forward here is a to open a PR to modify Axes3D.set_proj_type to take a callable in addition to the two strings we currently take for the projection.

You can then write a class like

class FocalProject:
    def __init__(self, focal_length):
        self.fl = focal_length
    def get_projection(self, zfront, zback):
         return ...(self.fl)

fp = FocalProject(5)
...
ax3d.set_proj_type(fp.get_projection)

and now you have an object that lets you control how the projection matrix is created that will be consulted by the rest of the machinery at draw time. Modifying set_proj is something we can only do upstream, but developing and testing these controllable projection class can be done in a third party package. If they get big uptake / have a stable (and nice to use) API we can then look at pulling them into core.

[timhoffm]

and for replicating real-world cameras in software.

This sounds like you are planning to do advanced 3d plots in Matplotlib. You should be aware that Matplotlib does only pseudo-3d rendering by projecting the objects in the right way and drawing them in the right order. For example, this will fail for mutually overlapping objects where parts of one objects are in front of and other parts are behind another object. So before diving deep here, make sure that Matplotlib is good enough for your application.

[scottshambaugh]

@tacaswell that works, though I'm not sure how much sense it makes to fully generalize the API when afaik there's only one canonical perspective matrix used in computer graphics (plus its simplified forms). More complex nonlinear transformations (ie a fisheye lens) can't be implemented via a simple matrix multiplication, so they wouldn't be able to slot in here anyways.

A small note, the matrix in the original post is actually a simplification of the most general form, with the assumption of a symmetric viewing volume. Which I believe is pretty much the only use case, and so it's not worth fully generalizing right now.

Any yeah, I only want to use this for wireframes / points / convex objects, so matplotlib works well with me. Can minimize my dependencies, embed these in notebooks, and don't have to train myself/others on different frameworks.

[jklymak]

It seems to me that if there is a more general matrix, and it reduces to the same matrix that we are now doing for focal_length=1, is there any harm to using the more general matrix, and coming up with a simple API to pass the more general parameters?

[scottshambaugh]

I'll draft something up next week and see what you guys think.

[tacaswell]

though I'm not sure how much sense it makes to fully generalize the API when afaik there's only one canonical perspective matrix used in computer graphics (plus its simplified forms).

Fair! I do not know enough about 3D graphics to be sure about that one way or the other. Taking a callable is is likely powerful enough, makes sure we do not lock our selves into bad API choices, and is a common pattern (it is used throughout scipy and for our color maps) so I suggested it as the most conservative path forward ;) If there is only one reasonable paramatarization of this matrix, then I agree with @jklymak .

---

One API suggestion would be to extend set_proj to take a dictionary (or named tuple or dataclass) of the parameters to generate the matrix, replace our two hard-coded functions with 1 function that given those parametetrs returns a function with def get_matrix(zfront, zback) signature, and then the special strings pick out a set of parameters rather than a function?