Sensor Models

Camera

Cameras project a point in space resolved in the camera frame into the image space as a pixel with a set of parameters:

$\mathbf{t}^s_{sx}$ is the world point resolved in the camera frame.

$\mathbf{p}$ is pixel location of that point as observed by the camera.

$\boldsymbol{\beta}$ is a vector of intrinsics parameters.

$\mathbf{P}$ is the projection function.

$$\mathbf{p} = \mathbf{P}\left(\mathbf{t}^s_{sx}, \boldsymbol{\beta}\right)$$ $$\mathbf{t}^s_{sx} = \left[\begin{matrix}t_x\\t_y\\t_z\end{matrix}\right]$$

5-parameter OpenCv Model

Link to API

$$\boldsymbol{\beta} = \left[\begin{matrix}f&,&c_x&,&c_y&,&k_1&,&k_2&,&p_1&,&p_2&,&k_3\end{matrix}\right]$$

$f$ = focal length

$c_x, c_y$ = image center

$k_1, k_2, k_3$ = Radial distortion parameters

$p_1, p_2$ = Tangential distortion parameters

$$\mathbf{P}\left(\mathbf{t}^s_{sx_i}, \boldsymbol{\beta}\right) = \left[\begin{matrix}f & 0 \\ 0 & f \end{matrix}\right]\mathbf{p}_d + \left[\begin{array}{c}c_x\\c_y\end{array}\right]$$ $$\mathbf{p}_d = s\mathbf{p}_m + \left(2\mathbf{p}_m\mathbf{p}_m^T + r^2\mathbf{I}\right)\left[\begin{array}{c}p_2\\p_1\end{array}\right]$$ $$s = 1 + k_1r^2 + k_2r^4 + k_3r^6$$ $$r^2 = \mathbf{p}_m^T\mathbf{p}_m$$ $$\mathbf{p}_m = \left[\begin{array}{c}t_x / t_z\\ t_y / t_z\end{array}\right]$$

Kannala-Brandt Model

Link to API

$$\boldsymbol{\beta} = \left[\begin{matrix}f&,&c_x&,&c_y&,&k_1&,&k_2&,&k_3&,&k_4\end{matrix}\right]$$

$f$ = focal length

$c_x, c_y$ = image center

$k_1, k_2, k_3, k_4$ = Radial distortion parameters

$$\mathbf{P}\left(\mathbf{t}^s_{sx_i}, \boldsymbol{\beta}\right) = \left[\begin{matrix}f&0\\0&f\end{matrix}\right]\mathbf{p}_d + \left[\begin{matrix}c_x\\c_y\end{matrix}\right]$$ $$\mathbf{p}_d = \frac{\theta_d}{r}\mathbf{p}_m$$ $$\theta_d = \theta + k_1\theta^3 + k_2\theta^5 + k_3\theta^7 + k_4\theta^9$$ $$\theta = \arctan\left(r\right)$$ $$r^2 = {\mathbf{p}_m}^T\mathbf{p}_m$$ $$\mathbf{p}_m = \left[\begin{matrix}t_x / t_z\\t_y/t_z\end{matrix}\right]$$

Gyroscope

Accelerometer