Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IMU (+ Magnetometor) Support #8

Closed
5 tasks done
Luxonis-Brandon opened this issue Jun 22, 2020 · 10 comments
Closed
5 tasks done

Add IMU (+ Magnetometor) Support #8

Luxonis-Brandon opened this issue Jun 22, 2020 · 10 comments
Assignees
Labels
enhancement New feature or request Gen2 Feature for or issue with Gen2

Comments

@Luxonis-Brandon
Copy link
Contributor

Luxonis-Brandon commented Jun 22, 2020

Start with the why:

Although not intended for V-SLAM, DepthAI could provide decent V-SLAM in parallel with other functions (through depth + feature-tracking (VTRACK)) stream outputs.

To do so, having an integrated IMU would enable this additional/parallel use-case.

Move to the how:

We also have initially prototyped support for the BMI160 and BMM150 and have the IO for it brought out through the 100-pin connector of the BW1099 100-pin interface.

We've also prototyped the BMX055/BMI055, which are other options. And discussing offline, it seems the BNO085 may be an excellent choice, as it provides an absolute orientation result in one package.

We could alternatively add these directly to the BW1099 module, but it probably makes sense to leave the choice as to whether they are included (and where) to the designer of the baseboard... as IMU location can be of critical important depending on the constraints on the design and colocation of image sensors/IMU/etc. to pertinent center locations on the design. So having the IMU/magnetometer on the baseboard affords these degrees of freedoms.

WRT the USB transfer of IMU data, XLink won't be a good option if we want low latency, as large transfers (from video streams) would delay the transmission.

We could add an interrupt endpoint to the USB device (could use HID class for example). XLink currently uses a single endpoint, bulk. And we can have on DepthAI a total of 3 different endpoints for output to host.

The what:

Add the IMU & Magnetometer to the following baseboard designs, in this priority:

Implement an interrupt endpoint to the USB device for super-low-latency 500 or 1,000 Hz update to the host from the IMU/Mag (comb).

@Luxonis-Brandon Luxonis-Brandon added the enhancement New feature or request label Jun 22, 2020
@Luxonis-Brandon Luxonis-Brandon changed the title Add BMI160 IMU & BMM150 Magnetometer Support Add IMU (+ Magnetometor) Support Jun 22, 2020
@Luxonis-Brandon
Copy link
Contributor Author

Some notes from discussions offline to consider when implementing:

  1. Does the BMI160 give you out an absolute orientation like the BNO-055 and BNO-080 do?
  2. Selecting an IMU that gives absolute orientation out allows not having to roll our own IMU filter inside DepthAO.
  3. From an expert in the field, stuff on the IMU will always be better than some general algorithm after the fact so its nice to have an IMU that gives you that out of the box.

@Luxonis-Brandon
Copy link
Contributor Author

To update from some additional research:

  1. Likely the BNO085 is attractive as it does provide the full absolute orientation output (i.e. the sensor fusion is done on-chip instead of off-chip), which helps on 2 and 3.

@Luxonis-Brian
Copy link
Contributor

Luxonis-Brian commented Jun 26, 2020

Notes:

The BNO080/085 use Bosch sensor BMF055. The Bosch equivalent to the BNO080/085 is the BNO055, the main differences in the SoMs being Hillcrest Labs' sensor fusion software: "Though both can be used in inertial sensing applications, choose the BNO085 for applications that demand high dynamic accuracy, low latency, and context/event classification. For embedded applications, the BNO080/BNO085 also includes RTOS support."

From ceva-dsp website: "For new designs, Hillcrest Labs recommends customers consider using the BNO085. The BNO085 is backwards compatible to the BNO080 in fit, form, and function; there are no changes required in software or operation of the part for current designs. The BNO085 simply adds a new capability which allows operation with future application specific external software libraries provided by Hillcrest Labs. The first such library provides additional advanced capabilities for use with VR controllers.

The BNO080/085 provides application specific configurations, features, and performance in a small turn-key component that benefits OEMs through faster time-to-market, reductions in development time and BOM cost, and delivering the highest precision and quality."

See RESOURCES AND DOWNLOADS section of this same webpage for datasheets, calibration guides, 080/085 migration guide, development kit info, etc.

@Luxonis-Brandon
Copy link
Contributor Author

This is initially implemented in #10

@Luxonis-Brandon Luxonis-Brandon added the Gen2 Feature for or issue with Gen2 label Oct 20, 2020
@Luxonis-Brandon Luxonis-Brandon moved this from Roadmap to In progress in Gen2 Phase I Delivery Nov 11, 2020
@Luxonis-Brandon
Copy link
Contributor Author

We just got the 1098EMB back with IMU support:

image
image

@Luxonis-Brandon
Copy link
Contributor Author

We added the IMU to the new Pi CM4 POE version here: #54

@ladrians
Copy link

Hi there, I am using the OAK D PRO W and would be a great feature to use the magnetometer.
Not sure if this is the thread to ask for? It would be great to publish at least a raw IMU message which can be complemented with a filter to get an absolute orientation, as not all use-cases need it. For example for us is mandatory an absolute orientation for outdoor navigation; for indoor just getting the changes in orientation are good enough as we fuse it with other sensors with a kalman filter.
thanks in advance for any comment you can give us.

@Erol444
Copy link
Member

Erol444 commented Mar 13, 2023

Hi @ladrians ,
We already support IMU (see depthai IMU node), and OAK-D Pro W should have IMU integrated - it might not be 9axis (with magnetometer) though - as first batch we didn't have the 9axis IMU in stock and we populated the 6axis one instead. If that's the case, and magnetometer is essential to your project, please reach out via support@luxonis.com and we will resolve the issue.
Thanks, Erik

@ladrians
Copy link

thanks for the comments @Erol444 , now we use an external IMU for that.
I do not know if my camera is in the batch that does not include the 9axis IMU and was shipped with the 6axis version, will contact support for that, thanks

@Erol444
Copy link
Member

Erol444 commented Mar 13, 2023

@ladrians best to check with the example - Rotation vector example is only supported by BNO085 (9axis), while Accelero/Gyro example is supported by both IMU sensors.
Thanks, Erik

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Gen2 Feature for or issue with Gen2
Development

No branches or pull requests

4 participants