Hardware limit for camera frame rates

Behavior camera

• Camera: SP-5000M-CXP4
• Frame grabber: Euresys Coaxlink Quad G3
• Interface: CXP6 x4

Frame rate calculation

The following calculation is based on the manual "Manual_SP-5000-CXP4" available from https://www.jai.com/products/sp-5000m-cxp4/, section 7.1.2 "Calculation of frame rate" (pages 32-33).

Frame readout time:

$$t_{\rm readout} = \frac{t_{\rm row} (N_{\rm row} + K + 2) + 1 + 3 t_{\rm row} + N_{\rm cycleF}}{f_{\rm sys}}.$$

Maximum frame rate possible:

$$f_{\rm max} = \frac{1}{t_{\rm readout}},$$

where

$$K = \left\lceil \frac{203}{t_{\rm row}} \right\rceil + 3,$$

$$f_{\rm sys} = 86,400,000.$$

The following values depend on ROI width, bit format, and link type. When ROI width is 1984, bit format is mono8, and the link type is CXP6 x4,

$$t_{\rm row} = H_{\rm blanking} = 130,$$

$$N_{\rm cycleF} = 1022.$$

Therefore, when $N_{\rm row} = 1500$, $N_{\rm col} = 1984$, under the conditions above,

$$K=\left\lceil\frac{203}{130}\right\rceil + 3 = 5,$$

$$ f_{\rm max} = \frac{1}{t_{\rm readout}} = \frac{86,400,000}{130(1500 + 5 + 2) + 1 + 3\cdot 130 + 1022} = 437.86 {\rm Hz} $$

Muscle camera

• Camera: pco.panda 4.2

Frame rate calculation

The following is based on the pco.panda 4.2 manual from https://www.excelitas.com/product/pcopanda-42-scmos-camera, section 6 "Rolling shutter" (pages 11-14).

A previous implementation (#79) triggers the start of a new frame after the previous frame has been fully acquired and read out (figure on page 13 of the manual, "exposure time < sensor frame readout time"). We now wish to implement a continuous rolling shutter where each line starts exposure for a new frame as soon as its readout period of the previous frame has completed, even if the lines under it are still being exposed or read out (figure on page 12 of the manual, "exposure time > sensor frame readout time"), as depicted in the sketch below.

where

• The thin, green lines are the exposure periods.
• The thick, red lines are the sensor readout periods.
• $t_{\rm so}$ is the shutter-open time (i.e., exposure time communicated to the muscle camera).
  • Note: Since we're emulating a "global shutter" by turning on illumination only during the common time, this exposure time is not exposed to the higher-level API. As far as the caller is concerned, the exposure time is the duration in which the light is switched on.
• $t_{\rm lo}$ is the duration in which the light is switched on (i.e., "exposure time" exposed to the caller, as noted above).
• $t_{\rm r}$ is the shutter rolling time (i.e., amount of time between the start of exposure of the first line and that of the last line while acquiring the same frame).
• $\Delta t$ is the per-line readout time (i.e., for the camera to register the data and prepare for the next frame).
• $t_{\rm c}$ is the common time (i.e., time during which all lines are exposed).
  • Note: In this continuous rolling mode, the only way to change the frame rate is to extend or reduce the common time. Therefore, to decouple frame rate control from light-on time control, we can add a buffer time $t_{\rm b}$ during which all lines are exposed, but the light is off.
• $t_{\rm b}$ is the buffer time as described above.
• $T$ is the recording interval (i.e., 1 / frame rate).

From the manual,

• The line time (amount of time between start of two adjacent lines during shutter rolling) is $12.136\ {\rm \mu s}$.
  • Therefore, at 1120-by-1120 px resolution, $t_{\rm r} = 1120\cdot 12.136 = 13592.32\ {\rm \mu s}$.
• $\Delta t = 4.5\ {\rm \mu s}$.

Therefore, the frame rate is:

$$ f = \frac{10^6}{t_{\rm r} + t_{\rm lo} + t_{\rm b} + \Delta t} = \frac{10^6}{13596.82 + t_{\rm lo} + t_{\rm b}}\ {\rm Hz}. $$

$$ t_{\rm b} = \frac{10^6}{f} - t_{\rm r} - t_{\rm lo} - \Delta t. $$

The maximum frame rate is reached when $t_{\rm b} = 0$:

$$ f_{\rm max} = \frac{10^6}{13596.82 + t_{\rm lo}}\ {\rm Hz}.$$

At 1 ms effective exposure time, $f_{\rm max} = 68.5\ {\rm Hz}$.

Conclusion

A good trade-off seems to be:

• Behavior images recorded at H=1500, W=1984 pixels at 396 Hz.
• Muscle images recorded at H=1120, W=1120 pixels at 66 Hz.
• The synchronization ration between behavior and muscle cameras is 6.
• This requires $t_{\rm b}$ to be $10^6/66 - 13592.32 - 1000 - 4.5 = 554.695\ {\mu s}$.