Question about Eye-to-hand Scenario Usage

[zyaqcl]

Hi

I have a depth camera mounted on the base and an object mounted on the end effector of robot arm. After scanning, hand_T_cam was calculated and used as the pose and sent along the scan result for fusion. I thought I could get the result including a clear model of the object and a messy background. However, on the contrary, I got a clear background and a messy foreground. Why is that?

Thanks

[benemer]

Note that the points that are passed to the integrate function need to be in a common frame. Is that the case on your side?

[zyaqcl]

@benemer Thanks for your reply. I have all the point clouds mapped to the same Base frame and performed point cloud alignment and got transformation for each point cloud. What should I choose as the input of intergrate? Should that be aligned point clouds with identical poses?

[benemer]

These should be the transformed points and the poses in the same coordinate frame.

So, if you have transformations from each local frame to the base frame, pass those poses together with the points transformed to the base frame.

[zyaqcl]

@benemer Thank you fro your patience and apologies for not being specific.
I have a set of point clouds under base frame marked as ^BaseP, which are transfromed from their own local frame with ^BaseT_Local.
After performing point cloud alignment, I got ^AlignT for each point cloud. With ^BaseP. transform(^AlignT) I can get ^AlignedP.
I have tried 4 setups for the input.
If input is (^BaseP, ^BaseT_Local) I can get correct but not aligned points.
Input (^BaseP, ^AlignT@^BaseT_Local) I get similar result as upper one.
If input is (^AlignedP. ^AlignT@^BaseT_Local) or (^AlignedP. ^BaseT_Local) , the results do not seem correcrt.

[benemer]

What exactly is your base frame representing? How are you aligning the points?

Can you maybe share the point clouds with us for testing?

First looking at this, (^AlignedP. ^BaseT_Local) seems to be correct.

[zyaqcl]

@benemer
Thank you. Your solution seems correct.
The base frame is the frame of robot arm end effector which the object is mounted on.
I did the alignment with open3d.registration.registration_icp. The ^AlignT indicates the result.
Previously I used the ^AlignT incorrectly so the results are wrong.
It seems that there is little diference between using (^AlignedP, ^BaseT_Local) and (^AlignedP, ^AlignT @ ^BaseT_Local)