Skip to content

Increase numberOfTestsPerMethod to 100 for testDeterministicAndNonDeterministicProvidersReturnValues #745

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 29, 2023

Conversation

snuyanzin
Copy link
Collaborator

It looks like 10 is not enough for some cases e.g. https://github.com/datafaker-net/datafaker/actions/runs/4554807351/jobs/8033243612

Also there is no need to continue invocations in case set's size is >= 2

@what-the-diff
Copy link

what-the-diff bot commented Mar 29, 2023

PR Summary

  • Increase in test iterations
    The number of tests performed for each method is increased from 10 to 100.
  • Introduction of new variable and loop condition
    A new variable, currentSize, is introduced and used in the loop condition to ensure at least 100 iterations are executed, but stopping when there are more than one unique values generated by a provider. This helps in identifying non-deterministic behavior.

@codecov-commenter
Copy link

Codecov Report

Merging #745 (470381d) into main (f4699de) will increase coverage by 0.01%.
The diff coverage is n/a.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@             Coverage Diff              @@
##               main     #745      +/-   ##
============================================
+ Coverage     92.84%   92.85%   +0.01%     
+ Complexity     2592     2591       -1     
============================================
  Files           281      281              
  Lines          5126     5126              
  Branches        530      530              
============================================
+ Hits           4759     4760       +1     
+ Misses          240      238       -2     
- Partials        127      128       +1     

see 3 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Collaborator

@kingthorin kingthorin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logically this should be fine, but technically we could end-up with situations where low choice sets (sex, boolean, etc) do generate the same value 100 times. (Improbable but not impossible.) In such cases the failing GitHub action/workflow could just be re-run from the WebUI.

@snuyanzin
Copy link
Collaborator Author

yes, should be very rare case.
Locally I ran this test 1000 times in a row and no failures

@snuyanzin snuyanzin merged commit 4a51133 into datafaker-net:main Mar 29, 2023
snuyanzin added a commit to snuyanzin/datafaker that referenced this pull request Apr 17, 2023
snuyanzin added a commit to snuyanzin/datafaker that referenced this pull request Apr 17, 2023
snuyanzin added a commit that referenced this pull request Apr 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants