You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Recently I have written a test to compare mat4 results in cglm repo (http://github.com/recp/cglm),
but I've precision problem, related test fails. What I'm trying to do is:
A = random matrix
B = inv(A)
C = inv(B)
assert(A == C)
I'm comparing float values via FLT_EPSILON (I guess it is 0.000001) but it doesn't seem to right value to compare?
I have tested the same with original glm but the result is same. Only this ( 0.0001) precision looks work :/
Test suite:
...
glm::mat4 A = glm::make_mat4(cglm_matrix[0]);
glm::mat4 B = glm::inverse(A);
glm::mat4 C = glm::inverse(B);
for (i = 0; i < m; i++) {
for (j = 0; j < n; j++) {
for (k = 0; k < m; k++) {
/* fails */assert(fabsf(A[i][j] - C[i][j]) <= FLT_EPSILON);
/*--------> this is always passing (on macos): assert(fabsf(A[i][j] - C[i][j]) <= 0.0001);*/
}
}
}
...
Any idea why the precision is too low (not FLT_EPSILON)? Or should I always use 0.0001 as epsilon to compare results?
The text was updated successfully, but these errors were encountered:
Hello,
Recently I have written a test to compare mat4 results in cglm repo (http://github.com/recp/cglm),
but I've precision problem, related test fails. What I'm trying to do is:
A = random matrix
B = inv(A)
C = inv(B)
assert(A == C)
I'm comparing float values via
FLT_EPSILON
(I guess it is 0.000001) but it doesn't seem to right value to compare?I have tested the same with original glm but the result is same. Only this (
0.0001
) precision looks work :/Test suite:
Any idea why the precision is too low (not FLT_EPSILON)? Or should I always use
0.0001
as epsilon to compare results?The text was updated successfully, but these errors were encountered: