Background

Why an Open Sourced MoCap Camera?

Well, frankly I need one in order to draw developers to contribute to the Gimli project. There would be no point in researching and writing the MoCap Processing Software if there was no free way of doing some MoCap. There will be many compromises made in the pursuit of a working Open MoCap Camera - and certainly if you want 4MP @ 500fps with perfect Shutter sync, you should be thinking about buying Vicon, Mac, or commissioning a custom SD device. This project is for 'The little guy'.

A pi? How can this possibly work?

My Contentions

1. The RPi 3 has a quad core, 64-Bit, 1.2GHz CPU, with an advanced Vectorized instruction set (NEON). The 'stock' Raspian OS provides all of the conveniences of a Linux system, and Programming interfaces for lots of helpful stuff, eg. Berkeley Sockets & select. If I were making a commercial MoCap camera, I would be writing my own RTOS to run on an ALTERA FPGA. To complex, a pi gives us so much for free out of the box.

2. A 'Typical' scene imaged by a MoCap camera will contain 20~60 bright spots (the markers reflecting light back to the camera) being 2x2 to 20x20 pixels in size. More than 98% of the image will be dark (or specifically, below a threshold value)

   • 2.1. I Argue that we can skip through the majority of the image very quickly by doing a vectorized 'less-than-or-equal' comparison to the threshold. This can be upto 16 Bytes wide with 64-Bit NEON SIMD.
   • 2.2. The Image could be further split into 2, 3, or 4 regions for parallel evaluation - however in testing 1 core performs surprisingly well with simulated data.
   • 2.3. Thus, the pi will be capable of Centroid detection at a usable realtime framerate

3. There are several COTS (Commercial Off The Shelf) Camera modules available for the pi

   • 3.1 The Rpi Cam2 is 1280x720 @ 60Hz, and is available in an IR mode and with an IR strobe ring. It's image is 'published' in a YUV format through the default API, so the first 'block' of uint8 bytes will be the B&W image unraveled. Even though we get (unwanted) chroma information, we can skip it entirely.
   • 3.2 This arducam has a Global Shutter, and apparently an external sync (TTL?) Input.
   • 3.3 Even better the 'Authentic' pi V2 HQ Camera now either emits 'source' or accepts 'sink' a timing signal.

4. Using non-exotic hardware, and a widely available platform, I hope this system can be put in the most peoples hands for the lowest possible price.

   • 4.1 Also attempting to minimize the complexity to setup a pi to become part of the system reduces barriers to entry for new users / developers.

Challenges

1. Execute connected components to find RoIs in the image DONE
2. Develop some process to circle fit or compute a centroid DONE
3. Timely threaded execution on a non-RTS Threading not yet required
4. Hook into Camera system and centroid detect at 60fps Wip
5. Syncronization of multiple cameras Wip

Tasks

Camera

• 'Main Loop' for camera receiving and interpreting commands, taking pictures, returning data
• 'connected components' algo
• 'centroid computation' algo
• evaluate timing of these algos
• possible threaded implementation of these algos & main loop
• c implementation of best result
• syncronization strategy research c.f. Hermann-SW
• RTOS R&D if needed
• Strobe Triggering

Algo

• Connected components Monolithic
• Connected components Parallel
• Centroids CircleFit
• Centroids AvgCenter
• Centroids HuMoments
• Monolithic Vs Parallel Implementations
• SIMD Implementations

Syncronization

• NTP Protocol
• PtP Protocol Wip
• Multi-Cast trigger ??
• PtP Trigger ??
• GPIO Trigger ??

Custom Distro

• Build minimal Raspian distribution ????
• Camera I/O
• Network I/O
• File system I/O
• Sync system
• Telnet

C&C Program

• Emit simple 'Verb:Noun' commands (UDP, 8-Bit)
• Communication broker to send & receive data from a camera
• simulated camera(s) to emulate a full system to build C&C Program
• Camera Setup tool
• Data Recording tool