Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mat4 inverse precision #700

Closed
recp opened this issue Nov 17, 2017 · 1 comment
Closed

mat4 inverse precision #700

recp opened this issue Nov 17, 2017 · 1 comment

Comments

@recp
Copy link

recp commented Nov 17, 2017

Hello,

Recently I have written a test to compare mat4 results in cglm repo (http://github.com/recp/cglm),
but I've precision problem, related test fails. What I'm trying to do is:

A = random matrix
B = inv(A)
C = inv(B)

assert(A == C)

I'm comparing float values via FLT_EPSILON (I guess it is 0.000001) but it doesn't seem to right value to compare?

I have tested the same with original glm but the result is same. Only this ( 0.0001) precision looks work :/

Test suite:

  ...
  glm::mat4 A = glm::make_mat4(cglm_matrix[0]);
  glm::mat4 B = glm::inverse(A);
  glm::mat4 C = glm::inverse(B);

  for (i = 0; i < m; i++) {
    for (j = 0; j < n; j++) {
      for (k = 0; k < m; k++) {
        /* fails  */
        assert(fabsf(A[i][j] - C[i][j]) <= FLT_EPSILON);

        /*
--------> this is always passing (on macos):
          assert(fabsf(A[i][j] - C[i][j]) <= 0.0001);
         */
      }
    }
  }
  ...

Any idea why the precision is too low (not FLT_EPSILON)? Or should I always use 0.0001 as epsilon to compare results?

@recp
Copy link
Author

recp commented Nov 17, 2017

The random matrix was not an affine transform, now I tested with affine transform it looks good.

I'm closing this issue.

@recp recp closed this as completed Nov 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant