Add IMU (+ Magnetometor) Support

[Luxonis-Brandon]

Start with the why:

Although not intended for V-SLAM, DepthAI could provide decent V-SLAM in parallel with other functions (through depth + feature-tracking (VTRACK)) stream outputs.

To do so, having an integrated IMU would enable this additional/parallel use-case.

Move to the how:

We also have initially prototyped support for the BMI160 and BMM150 and have the IO for it brought out through the 100-pin connector of the BW1099 100-pin interface.

We've also prototyped the BMX055/BMI055, which are other options. And discussing offline, it seems the BNO085 may be an excellent choice, as it provides an absolute orientation result in one package.

We could alternatively add these directly to the BW1099 module, but it probably makes sense to leave the choice as to whether they are included (and where) to the designer of the baseboard... as IMU location can be of critical important depending on the constraints on the design and colocation of image sensors/IMU/etc. to pertinent center locations on the design. So having the IMU/magnetometer on the baseboard affords these degrees of freedoms.

WRT the USB transfer of IMU data, XLink won't be a good option if we want low latency, as large transfers (from video streams) would delay the transmission.

We could add an interrupt endpoint to the USB device (could use HID class for example). XLink currently uses a single endpoint, bulk. And we can have on DepthAI a total of 3 different endpoints for output to host.

The what:

Add the IMU & Magnetometer to the following baseboard designs, in this priority:

• BW1092 BW1092: DepthAI ESP32 Reference Design | Embedded DepthAI Reference Design #10
• BW1098OBC
• BW1097
• BW1098FFC
• BW1098EMB

Implement an interrupt endpoint to the USB device for super-low-latency 500 or 1,000 Hz update to the host from the IMU/Mag (comb).

[Luxonis-Brandon]

Some notes from discussions offline to consider when implementing:

1. Does the BMI160 give you out an absolute orientation like the BNO-055 and BNO-080 do?
2. Selecting an IMU that gives absolute orientation out allows not having to roll our own IMU filter inside DepthAO.
3. From an expert in the field, stuff on the IMU will always be better than some general algorithm after the fact so its nice to have an IMU that gives you that out of the box.

[Luxonis-Brandon]

To update from some additional research:

1. Likely the BNO085 is attractive as it does provide the full absolute orientation output (i.e. the sensor fusion is done on-chip instead of off-chip), which helps on 2 and 3.

[Luxonis-Brian]

Notes:

The BNO080/085 use Bosch sensor BMF055. The Bosch equivalent to the BNO080/085 is the BNO055, the main differences in the SoMs being Hillcrest Labs' sensor fusion software: "Though both can be used in inertial sensing applications, choose the BNO085 for applications that demand high dynamic accuracy, low latency, and context/event classification. For embedded applications, the BNO080/BNO085 also includes RTOS support."

From ceva-dsp website: "For new designs, Hillcrest Labs recommends customers consider using the BNO085. The BNO085 is backwards compatible to the BNO080 in fit, form, and function; there are no changes required in software or operation of the part for current designs. The BNO085 simply adds a new capability which allows operation with future application specific external software libraries provided by Hillcrest Labs. The first such library provides additional advanced capabilities for use with VR controllers.

The BNO080/085 provides application specific configurations, features, and performance in a small turn-key component that benefits OEMs through faster time-to-market, reductions in development time and BOM cost, and delivering the highest precision and quality."

See RESOURCES AND DOWNLOADS section of this same webpage for datasheets, calibration guides, 080/085 migration guide, development kit info, etc.

[Luxonis-Brandon]

This is initially implemented in #10

[Luxonis-Brandon]

We just got the 1098EMB back with IMU support:

[Luxonis-Brandon]

We added the IMU to the new Pi CM4 POE version here: #54

[ladrians]

Hi there, I am using the OAK D PRO W and would be a great feature to use the magnetometer.
Not sure if this is the thread to ask for? It would be great to publish at least a raw IMU message which can be complemented with a filter to get an absolute orientation, as not all use-cases need it. For example for us is mandatory an absolute orientation for outdoor navigation; for indoor just getting the changes in orientation are good enough as we fuse it with other sensors with a kalman filter.
thanks in advance for any comment you can give us.