New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faulty depth measurements near object edges #7

Open
AlexReimann opened this Issue Jun 13, 2016 · 33 comments

Comments

Projects
None yet
@AlexReimann
Copy link

AlexReimann commented Jun 13, 2016

When using the driver there is a huge noise in the point cloud at edges of objects.

Pointing at a flat surface:
orbbec noise

A video of it can be found here:
https://www.youtube.com/watch?v=p2L59TLhsg4

I guess some interpolation calculation is going wrong.

This was not the case with an older driver I got from the Orbbec website (the link is down now).

@AlexReimann AlexReimann changed the title Faulty depth measurements near the border Faulty depth measurements near object edges Jun 13, 2016

@procopiostein

This comment has been minimized.

Copy link

procopiostein commented Jun 13, 2016

I can confirm this issue affects both ASTRA S and ASTRA cameras.

@ob-tim-liu

This comment has been minimized.

Copy link
Collaborator

ob-tim-liu commented Jun 13, 2016

This was not the case with an older driver I got from the Orbbec website (the link is down now).

Could you provide the link of older driver? we will check it .

@ruvindad

This comment has been minimized.

Copy link

ruvindad commented Jun 28, 2016

I too can confirm that there is a significant difference between the older driver that was hosted on dropbox and this one. I am attaching the old driver for your reference:
OpenNI-2-Linux-x64-Orbbec3D.tar.gz

@tfoote

This comment has been minimized.

Copy link
Contributor

tfoote commented Jul 6, 2016

These artifacts are a challenge for navigating with the TurtleBot. It generally works but with noteably lower performance due to the extra noise needing to be rejected. And the avoidance of phantom obstacles.

Here's a screenshot from my debugging.

astra_ray_noise

It would be really great to add the filters to the released version of the driver.

@bit-pirate

This comment has been minimized.

Copy link

bit-pirate commented Jul 6, 2016

A bugfix for this is required before we can consider integrating the Orbbec Astra with our products. Any idea when we could expect a fix for this?

@JoshBlake

This comment has been minimized.

Copy link

JoshBlake commented Jul 6, 2016

@bit-pirate The binary releases of our openni2 drivers include this filter. You can find them as part of Astra SDK's openni2 folder or at https://3dclub.orbbec3d.com/t/full-openni-2-environment-download/34. Note that ARM builds exclude the filter due to the limited processing power of most ARM CPUs. If you need additional support on this please contact david@orbbec3d.com.

@bit-pirate

This comment has been minimized.

Copy link

bit-pirate commented Jul 25, 2016

@tfoote is the OpenNI2 bundle linked above by @JoshBlake included in our ROS package for the Astra? "Our ROS package" being your fork, if I understand the current situation correctly.

@tfoote

This comment has been minimized.

Copy link
Contributor

tfoote commented Jul 26, 2016

All my changes are merged upstream. https://github.com/tfoote/ros_astra_camera/network I did the release from my branch for speed only. All development and the next release should come from the orbbec main branch.

@AlexReimann

This comment has been minimized.

Copy link

AlexReimann commented Jul 29, 2016

Looks like it the filter is currently in the filterlibrary branch and I guess will be eventually merged Looks much better with that.

Any chance to get any info on how this filter works?

@ob-tim-liu

This comment has been minimized.

Copy link
Collaborator

ob-tim-liu commented Jul 29, 2016

@AlexReimann the branch of filerlibrary is a temporary solution , that provider function same to master with extra filter driver . but it will not merged to the master. you can use codes by reference to README.

@AlexReimann

This comment has been minimized.

Copy link

AlexReimann commented Aug 1, 2016

So what's the planned solution?

@mintar

This comment has been minimized.

Copy link
Contributor

mintar commented Sep 21, 2016

Yup, waiting for a solution as well... any news?

@AlexReimann

This comment has been minimized.

Copy link

AlexReimann commented Oct 17, 2016

Bump ._.'

@bit-pirate

This comment has been minimized.

Copy link

bit-pirate commented Nov 4, 2016

@JoshBlake @ob-tim-liu it would be very important for us and our products, like the TurtleBot, to get a fix for this edge noise into the released driver. Could you let us know about the progress?

@ob-tim-liu

This comment has been minimized.

Copy link
Collaborator

ob-tim-liu commented Nov 4, 2016

now, we provider the pull request to tfoot about orbbec driver with soft filter library.
before merge this request , you can use the 'filterlibrary' branch , the only different is
astra_openni2 in the local dir.


刘华
座机:0755-86329228
邮箱:liuhua@orbbec.com
手机:13603061506
深圳奥比中光科技有限公司 www.orbbec.com.cn
地址:深圳市南山区粤兴三道8号中国地质大学产学研基地中地大楼A808

From: Marcus Liebhardt
Date: 2016-11-04 14:42
To: orbbec/ros_astra_camera
CC: ob-tim-liu; Mention
Subject: Re: [orbbec/ros_astra_camera] Faulty depth measurements near object edges (#7)
@JoshBlake @ob-tim-liu it would be very important for us and our products, like the TurtleBot, to get a fix for this edge noise into the released driver. Could you let us know about the progress?

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@mintar

This comment has been minimized.

Copy link
Contributor

mintar commented Nov 4, 2016

In case people are wondering, @ob-tim-liu is talking about tfoote/OpenNI2#1 . Once that is merged, the master branch in this repo should do the same as the filterlibrary branch. The only difference is that the filterlibrary branch contains a copy of tfoote/OpenNI2, whereas master uses tfoote/OpenNI2 as an external dependency (which is better, of course).

@bit-pirate

This comment has been minimized.

Copy link

bit-pirate commented Nov 4, 2016

Understood. Thanks for the update, guys.

@tfoote could you notify us here when that PR has been merged and the package re-released? Then we will test it again (and hopefully close the issue).

@AlexReimann

This comment has been minimized.

Copy link

AlexReimann commented Jan 24, 2017

@tfoote bounced this back

So.. bump :p

@bit-pirate

This comment has been minimized.

Copy link

bit-pirate commented Feb 13, 2017

@ob-tim-liu as @tfoote mentioned in his comment in the open PR (tfoote/OpenNI2#1), Orbbec should start release their ROS S/W drivers from their own, official repo.

Could you please do the required changes and release the important changes in the filterlibrary branch?

@ob-tim-liu

This comment has been minimized.

Copy link
Collaborator

ob-tim-liu commented Feb 13, 2017

@bit-pirate i have change the link of OpenNI2 from tfoote's github to orbbec, you can use the latest release on master branch

@bit-pirate

This comment has been minimized.

Copy link

bit-pirate commented Feb 13, 2017

Thanks @ob-tim-liu ! @AlexReimann please verify.

@tfoote tfoote referenced this issue Feb 14, 2017

Closed

add softfilter #1

@AlexReimann

This comment has been minimized.

Copy link

AlexReimann commented Feb 23, 2017

Latest deb still doesn't have the filter (not released yet?) and the master branch here doesn't show any depth image since the last commit with our Astras.

If I roll-back the master branch one commit, the filter is in and works.

So just need to release the deb and fix that issue?

@bit-pirate

This comment has been minimized.

Copy link

bit-pirate commented Feb 23, 2017

The release deb is still at version 0.1.5. So the commits from Sep 22 to Dec 16 are not released yet.

I'd say, 1. fix depth image issue and 2. release the softfilter changes and depth image bugfix with version 0.1.6.

@mintar

This comment has been minimized.

Copy link
Contributor

mintar commented Feb 23, 2017

Also, the deb is still released from tfoote/ros_astra_camera, not from this repo. Given the low responsiveness of the Orbbec people, I think I would even prefer it if somebody other than them would maintain the deb releases. We would just need somebody to volunteer for the job.

@mintar

This comment has been minimized.

Copy link
Contributor

mintar commented Feb 23, 2017

Also, the filter library is not really a fix, but a horrible workaround. It's so computationally expensive that it's impossible to do on an ARM board, and since it's baked into the OpenNI driver instead of a standalone node you can't run it on an external PC either.

This is what I have observed so far:

  • The random pixels in the depth image only occur near object edges, and only if depth registration is enabled in the driver.
  • They seem truly random, not interpolated between foreground and background.

Depth registration has to reproject each depth value from the depth into the rgb frame. My theory is that the random pixels occur in all pixels that are visible from the rgb frame, but not the depth frame (hence near object edges). Probably they just allocated some uninitialized memory for the registered depth image, then overwrote each pixel for which they can compute a depth value, leaving the rest uninitialized (i.e. "random").

The proper (and easy!) fix would be to just initialize the registered depth image with "NaN". If Orbbec could upgrade their firmware to do just that, we wouldn't need the filter library any more, and the Astra would finally become usable.

@wjwwood

This comment has been minimized.

Copy link

wjwwood commented Feb 24, 2017

I don't know how I can help, but I wanted to add another data point (and to make sure what I'm seeing isn't a new issue) I get something like this when the astra is too close to an object or it is looking at a surface that is out of range:

screenshot from 2017-02-24 15-37-52

Here's a gif of me moving it and show it in a glass door and if I put my hand close: https://gfycat.com/MelodicBlondKudu

Notice the left-right smears in the depth image.

@ob-tim-liu

This comment has been minimized.

Copy link
Collaborator

ob-tim-liu commented Feb 27, 2017

@tfoote

This comment has been minimized.

Copy link
Contributor

tfoote commented Feb 27, 2017

@ob-tim-liu Most users use the prebuilt debian packages in which the filter library is not included. A new release which includes the filter here: https://github.com/tfoote/OpenNI2/pull/1/files would give most users a much better experience with the astra when used with ROS.

@AlexReimann

This comment has been minimized.

Copy link

AlexReimann commented Mar 13, 2017

@mintar we get the problem also when there is no depth registration

Update: The problem was one of our engineers was using the wrong repository / branch

Besides that we run into a new problem case:

sensor_noise1

sensor_noise2

In this case we get noise depth pixels up to a certain pixel height (you can clearly see that in the lower right corner) in the areas where the sensor can't get a proper depth value. This looks to me like some memory initialization or corruption problem.

I agree on @mintar that fixing this by fixing the firmware or driver properly is the way to go instead of hiding the problem by using some filter.

@ob-tim-liu Please could you let someone make sure you are properly initializing stuff (I'm humbly suggesting memcheck)?

I will do some more test over the day and see if there is a way around this.

Update: The problem was one of our engineers was using the wrong repository / branch

@alberto-tellaeche

This comment has been minimized.

Copy link

alberto-tellaeche commented Mar 28, 2017

Hello all;
After having problems with the astra driver from indigo repositories and having issues, I have compiled this driver from source testing the flag -DFILTER to both values ON and OFF on filter branch of this repository. Seems to work better. In any case, any plans to move these changes to master?
Is an alternative fix planned for the package in ROS or an alternative solution in Astra FW?

Thanks,
Alberto

@aswinsarang

This comment has been minimized.

Copy link

aswinsarang commented Apr 3, 2017

astra 1
Is there a bug fix?

@ob-jackson

This comment has been minimized.

Copy link

ob-jackson commented Apr 3, 2017

@aswinsarang Please use the filter library from the above comments.

@AlexReimann

This comment has been minimized.

Copy link

AlexReimann commented Apr 7, 2017

The problem stated 4 posts above was because someone used a branch / repository without the filter library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment