Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Edge cases mode ignored for lists #119

Closed
jdmarble opened this issue Aug 30, 2020 · 8 comments
Closed

Edge cases mode ignored for lists #119

jdmarble opened this issue Aug 30, 2020 · 8 comments
Labels

Comments

@jdmarble
Copy link

jdmarble commented Aug 30, 2020

Testing Problem

I'm getting many thousands of lines of warnings logged per test if I generate a list of objects with lots of edge cases. This happens despite setting edgeCases = EdgeCasesMode.NONE on the @Property and defaultEdgeCases = NONE in jqwik.properties.

See this revised gist for a minimal example. Output:

Aug 30, 2020 2:22:58 PM net.jqwik.engine.facades.EdgeCasesFacadeImpl logTooManyEdgeCases
WARNING: Combinatorial explosion of edge case generation. Stopped creating more after 10000 generated cases.
// exactly 10k-2 lines skipped 
Aug 30, 2020 2:23:15 PM net.jqwik.engine.facades.EdgeCasesFacadeImpl logTooManyEdgeCases
WARNING: Combinatorial explosion of edge case generation. Stopped creating more after 10000 generated cases.
timestamp = 2020-08-30T14:23:15.227451100, ThingTest:my property = 
                              |--------------------jqwik--------------------
tries = 1000                  | # of calls to property
checks = 1000                 | # of not rejected calls
generation = RANDOMIZED       | parameters are randomly generated
after-failure = PREVIOUS_SEED | use the previous seed
edge-cases#mode = NONE        | edge cases are not explicitly generated
seed = 4337682503445002671    | random seed to reproduce generated values

Suggested Solution

I think the intent was to make edgeCases = EdgeCasesMode.NONE work on lists, but it's not working as intended. Perhaps an oversite or bug is preventing the setting from propagating through list generation.

Discussion

I'm not sure if my build is slowing down because of the edge case generation or the added logging.

@jlink
Copy link
Owner

jlink commented Aug 31, 2020

That’s a lot of log lines :-(

I’ll look at it when I’m back from holidays in two weeks.

@jlink jlink added the bug label Aug 31, 2020
@jlink
Copy link
Owner

jlink commented Sep 14, 2020

@jdmarble Do you have a property handy that reproduces the explosion of log lines? Need not necessarily be 10k lines, just a few would suffice ;-)

@jlink
Copy link
Owner

jlink commented Sep 14, 2020

Don't worry. I just noticed the gist.

@jlink
Copy link
Owner

jlink commented Sep 14, 2020

BTW, If I remove edgeCases = EdgeCasesMode.NONE I only get 4 lines of warning. AND the tests runs faster probably due to saving all the synchronized log output.

@jlink
Copy link
Owner

jlink commented Sep 14, 2020

To (partially) explain this behaviour: edgeCases = EdgeCasesMode.NONE only applies to the combined edge case generation of all parameters. Individual generators will always (as of version 1.3.5) inject edge cases. This is not really consistent and I'm thinking about getting rid of this inconsistency but it requires deeper refactoring.

@jlink
Copy link
Owner

jlink commented Sep 15, 2020

I was wrong again :-(

edgeCases = EdgeCasesMode.NONE suppresses generation of non edge cases "on my machine" since I use edgeCases = EdgeCasesMode.FIRST as default. So ignore my previous comment.

@jlink jlink closed this as completed in ee0de5f Sep 15, 2020
@jlink
Copy link
Owner

jlink commented Sep 15, 2020

I changed two things:

  • The warning about combinatorial explosion will only be logged max once per property
  • I reduced the max number of edge cases to be generated to 1000, which cuts the runtime of your property considerably

The changes are already available in 1.3.6-SNAPSHOT

I'll probably do a significant change to how edge cases will be generated in the more recent future. Maybe already for the 1.3.6 release but maybe not.

@jdmarble
Copy link
Author

jdmarble commented Sep 24, 2020

Thank you so much! With 1.3.6, I've finally been able to upgrade from 1.2.7.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants