Data in shared memory

This page offers a detailed description of data that are shared between threads.

programState

This is a ProgramState data structure (struct) that contains:

• toQuit (atomic boolean): Whether the user has requested to quit the program, whether by clicking the "X" button on the GUI window or by interrupting the process with the SIGINT signal (e.g. by pressing Control-C in the terminal).
• isRecording (atomic boolean): Whether a recording is in session (i.e. images are being saved to disk rather than simply streamed to the GUI program).

This struct is defined in TODO.

behaviorRecordingState

This is a BehaviorRecordingState struct that contains:

• behaviorCamera (BehaviorCamera interface as previously defined): This is the single entry point to access the behavior camera.
• behaviorImageQueue (queue): A queue of frames that have been acquired by the behavior image acquisition thread, pending to be processed and saved by the behavior image saving threads.
• behaviorImageQueueMutex (mutex, a previously discussed synchronization primitive): A "lock" that ensures only one thread can access the behavior image queue at a time.
• behaviorImageQueueCondVar (condition variable, a previously discussed synchronization primitive): An object that the behavior image acquisition thread can use to "wake up" one behavior image saving thread to do the work.
• latestBehaviorFrameHolder (a custom LatestFrame class): A data holder for the most recently acquired behavior image. Other threads, namely the tracking control thread and the GUI thread, can take the latest image from this object for their respective tasks (i.e. computing where the motion stages should move to and streaming the current image in the GUI). The LatestFrame class, implemented in TODO, offers a getLatestFrameData method and a setLatestFrameData method. Internally, LatestFrame maintains its own mutex to ensure that no thread can read from it when the acquisition thread is writing to it.

This struct is implemented in TODO.

Note

Pseudo-BGR JPEG saving trick: In fact, the members of behaviorImageQueue are instances of the GroupOfThreeFrames struct. As the name suggests, the behavior images are packaged in groups of three. This is because we save every three adjacent frames as the blue, green, and red channels of a single JPEG image. The JPEG format is designed to efficiently compress three images that are structurally very similar (the three color channels of an image usually contain the same objects). We can take advantage of this design to save behavior data more compactly, as the fly doesn't move much between three adjacent frames. This trick strikes a balance between time efficiency (i.e. the need to compress image in real time) and space efficiency (fewer bytes per frame).

muscleRecordingState

The MuscleRecordingState struct is very similar to behaviorRecordingState. It contains:

• muscleCamera (MuscleCamera)
• muscleImageQueue (queue)
• muscleImageQueueMutex (mutex)
• muscleImageQueueCondVar (condition variable)
• latestBehaviorFrameHolder (LatestFrame) Refer to behaviorRecordingState for detailed explanations.

This struct is defined in TODO.

trackingControlState

This is a TrackingControlState struct that contains:

• motionControlHandlerReady (atomic boolean): Whether the motion stage controller is ready.
• latestMotionStagePosition (a custom MotionStagePosition class): The last sensed position of the motion stages. The frequency at which the current position can be read out is limited, so we update the stage position in a separate thread, and the updates are not synchronized with image acquisition.
• latestMotionStagePositionMutex (mutex, a previously discussed synchronization primitive): A "lock" that ensures no thread can read out the latest motion stage position while the motion control thread is writing to it.
• trackingOn (atomic boolean): Whether fly tracking is on. It can be handy to turn tracking off, for example, when you are manually handling the behavior stage. This can be toggled on the GUI.
• shouldOverrideTracking (atomic boolean): Whether we should override tracking. This boolean is set to true when the user has asked the motion stages to move to a specific location by clicking on the arena overview widget on the GUI, and the stages are still en route to that position. During this period, the motion stages moves independent of what the behavior camera sees.
• overridingPosX (atomic double): The X coordinate of the position that the user has asked the stages to move to (by clicking on the arena overview widget on the GUI).
• overridingPosY (atomic double): Same as above but for Y.

This struct is defined in TODO.

arduinoCommunication

The ArduinoCommunication class (not a struct!) that offers the following public methods:

• setBehaviorRecordingFPS
• setSyncRatio
• setBehaviorExposureTime
• setMuscleExposureTime
• startRecording
• stopRecording
• stopCommunication

The use of these methods is explained in Hardware interface, Arduino.

Under the hood, this class contains private members that allows the implementation of the methods above. These include, for example:

• A serial port connection that allows sending and receiving messages via USB.
• A queue of pending messages to be sent to the Arduino.
• A mutex and a condition variable that ensures only one thread can push a new message to the queue at a time.
• A background thread (the Arduino communication thread) that runs in a loop and sends any new message to the Arduino.

This class is implemented in recorder/src/peripherals/arduinoCommunication.cpp and arduinoCommunication.hpp.

saveDirectory

The SaveDirectory class (not struct!) offers the following public methods:

• setDirectory: Set a given directory as the save directory
• getDirectory: Read out the save directory
• initialize: Create the save directory and its subdirectories if they do not already exist

The class has the following private members:

• A std::filesystem::path object that keeps track of what the currently set directory is
• A mutex that ensures that the currently set directory cannot be read from and written to at the same time. This is important because multiple threads will need to read from (e.g. image saver threads) and write to (e.g. GUI) the path.

This class is implemented in TODO.

programmedRecordingStop

This is a ProgrammedStop struct that contains the following:

• numBehaviorFramesExpected (int): After this many frames have been acquired by the behavior camera, recording should stop.
• numMuscleFramesExpected (int): Same as above but for the muscle camera.
• hasEndedFlagForGUI (atomic boolean): When the behavior camera notices that it should stop recording now, it sets this flag to true. When the GUI thread sees that this flag is false, it shows that the recording has terminated in a pop-up window and toggles the flag back to false.

The numbers of behavior and muscle frames expected are set by the GUI thread based on the experiment protocol that the user has entered, particular the "stop" instruction. See The experiment protocol string for details.

The ProgrammedStop struct is implemented in TODO.

recorderConfig

This is a RecorderConfig class that allows the caller to access parameters set in [the "recorder_config.yaml" file] (TODO). A RecorderConfig object is initialized from a path to the "recorder_config.yaml" file. Once initialized, the user can use the getParameter method to read out parameters in the YAML file and cast them to the appropriate data types automatically.

behaviorCamCalibrationParams

This is an object of the CalibrationParams, which is a wrapper around the camera calibration file, generated following TODO and TODO. The caller can use the stagePosAndPixelPosToPhysicalPos and stagePosAndPhysicalPosToPixelPos methods to apply the mappings $f_b$ and $g_b$ mappings described in Calibration algorithm.