Skip to content
This repository has been archived by the owner on Aug 30, 2022. It is now read-only.

Add distribution threshold test #35

Merged
merged 1 commit into from
Aug 18, 2021
Merged

Conversation

zhouzhuojie
Copy link
Contributor

@zhouzhuojie zhouzhuojie commented Aug 17, 2021

Description

To make sure that the distribution result from eval flags are as close as expected with the hashing function that flagr is using (i.e. CRC32). In these tests, we are targeting 1e6 samples and a 0.5% error bar (usually it's less than 0.1%, e.g. 0.0436% from the secure random key generator, but for reducing the flakiness of unit tests, we are raising the threshold to 0.5%).

Motivation and Context

Address some of the concerns raised from openflagr/flagr#464

How Has This Been Tested?

unit tests

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@codecov-commenter
Copy link

Codecov Report

Merging #35 (0f17204) into main (f161c56) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##             main      #35   +/-   ##
=======================================
  Coverage   81.55%   81.55%           
=======================================
  Files          27       27           
  Lines        2174     2174           
=======================================
  Hits         1773     1773           
  Misses        314      314           
  Partials       87       87           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f161c56...0f17204. Read the comment docs.

@zhouzhuojie zhouzhuojie merged commit f6a8dd2 into main Aug 18, 2021
@zhouzhuojie zhouzhuojie deleted the add-distribution-threshold-test branch August 18, 2021 01:56
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants