Unconditional failing when Floats equality is asserted breaks Elm Verify Examples

[tad-lispy]

test/src/Expect.elm

Lines 747 to 748 in f42d322

	if usesFloats then
	fail floatError

This creates some problems with elm-verify-examples. See stoeffel/elm-verify-examples#83. Perhaps there could be a command line switch or something like Expect.strictlyEqual to allow to do this kind of comparison.

For reference here is a discussion that introduced this feature: elm-community/elm-test#230

[mgold]

Oof. No obvious solution here so I'll invite brainstorming.

[drathier]

Floating point functions change depending on os, browser and cpu generation, so I don't think float equality is a good idea here either. Sooner or later someone will test a non-trivial float function, and it'll break. Maybe Stoeffel can add a second syntax for Expect.within checks, or leave out the ability to check floats?

[harrysarson]

There is another related issue where records that contain floating point values are really hard to test as you have to compare all keys one by one.

---

Floating point functions change depending on os, browser and cpu generation

As an aside - I believe the IEEE floating point spec has in fact locked down the behaviour of floating point functions.

[drathier]

Yes, but IEEE754 is so relaxed that it allows for all these implementation-specific differences. I ran into this issue with elm-test in 2018, where I had unit tests for exact equality of floats, and the tests passed on mac but not on windows.

[tad-lispy]

Maybe Stoeffel can add a second syntax for Expect.within checks, or leave out the ability to check floats?

Hmm... maybe something like:

0.1 --> 0.1 (+/- 0.0000001)

There was a related discussion here: stoeffel/elm-verify-examples#39. Personally I agree with @stoeffel - these are just examples, so the additional syntax should be as straight forward as possible.

Perhaps it could be possible to detect floats (like this package is doing) and generate different test code using within instead of equal. Is there a tolerance that would always be safe? Meaning that it would allow for rounding errors but other than that would require equality? I don't think so, because the errors can accumulate, right?

[drathier]

Unfortunately, the errors build up. You'd have to specify both a relative (multiplicative) tolerance, and an absolute (additive) tolerance.

[mgold]

Sooner or later someone will test a non-trivial float function, and it'll break.

Sure, but not as an example in the docs, right? If we can come up with a reasonable tolerance I think that would be ideal. Maybe allowing it to pass if within an absolute or a relative tolerance?

[drathier]

We can give it a try and see what happens ¯\_(ツ)_/¯ I think exact equality would be best in that case.

[tad-lispy]

Sure, but not as an example in the docs, right?

Why not? I mean the examples should be simple, but the internals of a tested function can be complex. I'm not saying it's impossible to come up with a reasonable tolerance, but the first assumption doesn't seem sound.