RayTracing: Does sometimes only work partially or even not at all

[andreasdr]

Hi @DanielChappuis

I have created a isolated test, written in TDME2, that shows random behaviour of RayCasting in RP3D. This test uses no multithreading or something, its just a basic single threaded test.

See here:

• https://github.com/andreasdr/tdme2/blob/master/src/tdme/tests/RayTracingTest.cpp

You can build and start with:

• make clean && make -j hardware_threads mains
• ./bin/tdme/tests/RayTracingTest (from main tdme2 folder)

You can find my raytracing logic here:

• https://github.com/andreasdr/tdme2/blob/master/src/tdme/engine/physics/World.cpp#L355

Expected would be that interaction table will be raytraced from bottom to top, but
sometimes it only works on the upper OBB part or sometimes even not at all
We have similar issues with other shapes, but lets start with this one.

You can change "player" X, Y rotation with mouse dragging and move with A,W,S,D keys.

If you have any questions or something please tell me. As this happens randomly I suggest no error in TDME2 but I can check deeper if required.

Many thanx and best regards
Andreas

[DanielChappuis]

Hello,

Thanks for the test.
I was able to run the RayTracingTest but I am trying to understand what I should look at.

I see a red capsule (the player) and several tables but where is ray casting in this scene ? Where are the rays ? What should I see if when it's working and what should I see when it's not working ?

[andreasdr]

Hi,

the ray tracing starts where the cam looks from and has it endpoint somewhere between where cam looks from to where the cam looks at.

In the middle you see a text message with a info "interact with ..." rigid body id if the ray hit something.
Maybe the message is a bit small and only seen with fullscreen window. I can make it bigger if required.

If you got this just move around and ray cast some interaction tables.
You see that sometimes the ground is raycasted and not the interaction table typically at the bottom of the interaction table. There is a crosshair also but its very small.

I can also do a video if required.

Maybe I should show the line for debugging purposes. Hmm.

Many thanx and

Best regards
Andreas

[DanielChappuis]

Maybe displaying the ray line is not useful because, if I understand correctly, it is always perfectly aligned with the camera and therefore we will probably not see it. However, I think that displaying the raycast hit point could be very helpful.

[andreasdr]

I can say that the hit point always was correct if the ray cast was computed properly.
If you need Ill do it anyways. Could you reproduce the issue and see what I meant?
To me its like the dynamic AABB does not work sometimes or something as I can shoot through some of the objects in our map but I can not walk through. So really ray casting seem to have some issue. No idea what exactly.

[DJLinux]

inside of:

bool AABB::testRayIntersect(const Ray& ray) const

I'm not sure but in some situations this can may be a problem:

   const decimal epsilon = 0.00001;
   adx += epsilon;   ady += epsilon;   adz += epsilon;

I would prefer something like this:

  const decimal epsilon = 0.00001;
  if (IsNearZero(adx))
    adx += epsilon;
  if (IsNearZero(ady))
    ady += epsilon;
  if (IsNearZero(adz))
    adz += epsilon;

By the way here are my more hand optimized version "speed for free" :-)
In my version #if DO_OPTIMATION are not defined all the original code are build
otherwise all my hand optimized code are used.
one example:

// Return true if the ray intersects the AABB
/// This method use the line vs AABB raycasting technique described in
/// Real-time Collision Detection by Christer Ericson.
bool AABB::testRayIntersect(const Ray& ray) const {
#ifndef DO_OPTIMATION
  const Vector3 point2 = ray.point1 + ray.maxFraction * (ray.point2 - ray.point1);
  const Vector3 e = mMaxCoordinates - mMinCoordinates;
  const Vector3 d = point2 - ray.point1;
  const Vector3 m = ray.point1 + point2 - mMinCoordinates - mMaxCoordinates;

  // Test if the AABB face normals are separating axis
  decimal adx = std::abs(d.x);
  if (std::abs(m.x) > e.x + adx) return false;
  decimal ady = std::abs(d.y);
  if (std::abs(m.y) > e.y + ady) return false;
  decimal adz = std::abs(d.z);
  if (std::abs(m.z) > e.z + adz) return false;

  // Add in an epsilon term to counteract arithmetic errors when segment is
  // (near) parallel to a coordinate axis (see text for detail)
  const decimal epsilon = 0.00001;
  adx += epsilon;
  ady += epsilon;
  adz += epsilon;

  // Test if the cross products between face normals and ray direction are
  // separating axis
  if (std::abs(m.y * d.z - m.z * d.y) > e.y * adz + e.z * ady) return false;
  if (std::abs(m.z * d.x - m.x * d.z) > e.x * adz + e.z * adx) return false;
  if (std::abs(m.x * d.y - m.y * d.x) > e.x * ady + e.y * adx) return false;

  // No separating axis has been found
  return true;

#else
  // define std::abs() different
  #define stdabs(a) a > decimal(0.0) ? a : -a
  const decimal epsilon = 0.00001;
  // test only X at first
        decimal p2 = ray.point1.x + ray.maxFraction * (ray.point2.x - ray.point1.x);
  const decimal ex = mMaxCoordinates.x - mMinCoordinates.x;
  const decimal dx = p2 - ray.point1.x;
  const decimal mx = ray.point1.x + p2 - mMinCoordinates.x - mMaxCoordinates.x;
  decimal adx = stdabs(dx);
  if (stdabs(mx) > ex + adx) return false;

  // ok we need Y also
                p2 = ray.point1.y + ray.maxFraction * (ray.point2.y - ray.point1.y);
  const decimal ey = mMaxCoordinates.y - mMinCoordinates.y;
  const decimal dy = p2 - ray.point1.y;
  const decimal my = ray.point1.y + p2 - mMinCoordinates.y - mMaxCoordinates.y;
  decimal ady = stdabs(dy);
  if (stdabs(my) > ey + ady) return false;

  // test XY cross product at first
  adx += epsilon;
  ady += epsilon;
  decimal absterm = mx*dy - my*dx;
  absterm = stdabs(absterm);
  if (absterm > ex*ady + ey*adx) return false;

  // ok we need Z also
                p2 = ray.point1.z + ray.maxFraction * (ray.point2.z - ray.point1.z);
  const decimal ez = mMaxCoordinates.z - mMinCoordinates.z;
  const decimal dz = p2 - ray.point1.z;
  const decimal mz = ray.point1.z + p2 - mMinCoordinates.z - mMaxCoordinates.z;
  decimal adz = stdabs(dz);
  if (stdabs(mz) > ez + adz) return false;
  adz += epsilon;

  // test YZ cross product as next
  absterm = my*dz - mz*dy;
  absterm = stdabs(absterm);
  if (absterm > ey*adz + ez*ady) return false;

  // test ZX cross product at least
  absterm = mz*dx - mx*dz;
  absterm = stdabs(absterm);
  if (absterm > ex*adz + ez*adx) return false;

  // No separating axis has been found
  return true;
  #undef stdabs
#endif
}

[DanielChappuis]

I would like to import the interaction table 3D model inside my testbed application to test if I can reproduce the issue. Could you send me this mesh in .obj file for instance ? I have found the .fbx.tmm file but I can't really load this.

Moreover, I am not sure exactly where in your code you set the CollisionShape for this object. Is it a concave or convex mesh ? Is it a single collision shape or a body with multiple collision shapes ?

[andreasdr]

Hi, I was on vacation. Sorry for the delay.

Its just a AABB/OBB with the following properties.

• "centerx":0.0635
• "centery":0.75
• "centerz":0.00100002
• "halfextensionx": 0.4395
• "halfextensiony":0.75
• "halfextensionz":0.37

Does this help?

[andreasdr]

Relevant code lives here:

• https://github.com/andreasdr/tdme2/blob/master/src/tdme/engine/primitives/OrientedBoundingBox.cpp#L146
• https://github.com/andreasdr/tdme2/blob/master/src/tdme/engine/physics/Body.cpp#L224

Maybe I have an error, but I am really unsure as collision detection is working like 100%

[DanielChappuis]

No problem. I was also on vacations.

I will try to deep into your raycast test but it's a bit hard without displaying the box and the ray on the screen.

[andreasdr]

Ok. I can display the box for you. The ray is aligned with the camera. So there should only be a point, right under the UI crosshair. Hmm.

Give me 1-2 days for the box maximum.

[andreasdr]

I will also try to display the line.

[DanielChappuis]

It would be great. Are you also able to place the player and camera in such a way that you expect a raycast hit but there is none at the beginning of the physics test ?

I mean, it would be great that when you launch the raycast test demo, we have a failling raycast situation without having to move the player around. This way, I can launch the demo and deep into the debugger directly because I know that we expect a raycast hit a the beginning of the simulation but we don't have one.

I hope it's clear enough :)

[andreasdr]

Ok. I'll try.

[andreasdr]

Hi, totally did that. Not sure if it covers all issues still. But its a start. Can you have a look at the issue please?

RayTracingTest now looks like:

• https://imgur.com/a/ZL042fI

Many thanx

[andreasdr]

Please ignore the box shading. This has to do with transparent faces rendering. Its a box.
Also this looks like a edge case, but I hope fixing this covers the other issues too.

[DanielChappuis]

Thanks a lot for this test. Just to make sure I understand this test. When a start the test, I cannot move the camera and I see a ray entering the box and it says that the ray hit the ground. This is a raycast test that is not working properly because instead of reporting a ground hit it should report a table hit. Am I correct ?

[andreasdr]

Yes!!! Please tell me if it works for you.

[DanielChappuis]

I have found something strange in the CustomCallbackClass::notifyRaycastHit() method. If you take a look at the documentation for raycasting in the user manual here, you can see what value should be returned in this method.

Here is how it works. In this method, you need to return a hitFraction value (between 0 and 1) that specify whether you want to discard or not other raycast hits along this same ray. Basically, the value you return is the next maxHitFraction value that allows you to clip the current ray for the next ray cast tests for the other objects of the scene. All the objects of the scene are tested for raycasting in any order. During the first raycast test, the minHitFraction=0 and maxHitFraction=1, which means that the library will report a raycast hit with an object, if the hitFraction value is between 0 and 1. Now, if in this call of the notifyRaycastHit() for the first hit object, you return a value of 0.5, this will be the maxHitFraction value for the next raycast tests along this ray. Therefore, when we test raycasting with the next object, we have minHitFraction=0 and maxHitFraction=0.5 which means that you now only want to test for raycast hits in the first half of the ray for the next objects of the scene.

Now consider the following example. We have a ray that crosses three objects A, B and C in this order from starting point of the ray to the end. Now, if you want to receive a notifyRaycastHit() callback for all the hit objects (A, B and C). You need to return the value 1.0 in this method. This way, the next maxFraction of the ray will always be 1.0 and the ray will never be clipped during the raycasting and therefore all the objects hit by the ray will be reported. Note that the physics library might first test raycasting with object B and then C and then A for instance (the order is never known).

Now, with the same example, if in the notifyRaycastHit() method, you return a value of 0.0, it means that the next maxFraction value will be zero for the second hit object to be tested for raycasting. Which means that the ray is completely clipped away and therefore of length zero. In this example, only the first hit object found by the library will be reported with a call to notifyRaycastHit() method. Note that it can be object either object A, B or C but we never know which one. We want to return 0.0 only if we only want a callback for any hit object (without any preference) but not the others. What we are doing here is clipping away completely the ray after the first hit so that we discard the others hit objects because we are not interested in them.

The third scenario, is probably the most useful. Here we are interested only by the hit that is closest to the starting point of the ray (object A in our example). We are not interested in the raycast hits with the object B and C because they are hidden by object A. We want this if we use raycasting to simulate a gun shot for instance. How do we do that? The idea is to clip the ray with the current hitFraction after each hit object. Therefore, we return the current hitFraction value in the notifyRaycastHit() method. The idea here is that when we have found a hit object, we clip the ray at the current object. Therefore, we will only receive the hits for objects that are closer in the next calls to notifyRaycastHit(). However, we can still receive hits with many object and we need to sort them at the end based on the hitFraction value to get the closest one. In this scenario, if the library first finds a hit with object B with a hitFraction=0.7, we return this value in the notifyRaycastHit() method and therefore we have maxHitFraction=0.7 for the next hit tests. Now if the library finds another hit with object C with hitFraction=0.9 because this object is further, we discard this hit because it is not closer than maxHitFraction=0.7. Next, we find a hit with object A with hitFraction=0.2, we also report this hit because it is smaller than maxHitFraction=0.7. In the notifyRaycastHit() callback for object A, we return 0.2 (current hitFraction) to clip again the ray. Now we will consider only the ray part with minHitFraction=0 and maxHitFraction=0.2 but there are no others objects to be tested in our example. Therefore, in this example, we have reported a hit with object B with hitFraction=0.7 and object A with hitFraction=0.2 but you still need to use the hitFraction value to get the closest one (closest to the start of the ray).

If you take a look at your code in the CustomCallbackClass::notifyRaycastHit() method (around line 365), there is a case where you return 0. As discussed previously. This will report only one hit object along your ray but it can be any object. In your example, there are two objects along your ray: the table and the ground. If I run your example, a hit with the ground is reported and then you return 0 which means that you are not interested in other hit objects along the same ray but it's probably wrong. I guess in your example, that you are interested in the closest hit from the starting point of the ray which is the table. Therefore, you should probably return the current hitFraction value here instead of 0. If you do this, the library will report hits that are closer and closer to the starting point of your ray but you still need to sort them at the end to see that the closest one is the table and not the ground. Also, I am not sure why your return 1.0 in the "else" condition of your "if" statement.

I hope it is clear enough :)
Let me know if it is not the case.

[andreasdr]

Hi Daniel, this seem to fix my problems. Thank you very much!

• andreasdr/tdme2@13444b9

[DanielChappuis]

Great news ! Can I close this issue ?

[andreasdr]

Sure. I should have read the documentation more seriously. I am sorry for that.

[DanielChappuis]

No problem. I also think that the documentation could be more clear :)