Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUGFIX] Add new high precision rule for mean and stdev in OnboardingDataAssistant #5276

Conversation

NathanFarmer
Copy link
Contributor

@NathanFarmer NathanFarmer commented Jun 9, 2022

Changes proposed in this pull request:

  • GREAT-984
  • Separate out mean and standard deviation from numeric rule because precision will always be greater than or equal to starting precision for these metrics
  • Update profile notebook renderer with new rule

Before (expectation fails approximately 2/3 of the time):

image
image

After (expectation fails at approximately the false_positive_rate of 0.05):

image
image

Definition of Done

  • My code follows the Great Expectations style guide
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added unit tests where applicable and made sure that new and existing tests are passing.
  • I have run any local integration tests and made sure that nothing is broken.

@netlify
Copy link

netlify bot commented Jun 9, 2022

Deploy Preview for niobium-lead-7998 ready!

Name Link
🔨 Latest commit 593ea17
🔍 Latest deploy log https://app.netlify.com/sites/niobium-lead-7998/deploys/62a37255df9a0d0009fd7274
😎 Deploy Preview https://deploy-preview-5276--niobium-lead-7998.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@ghost
Copy link

ghost commented Jun 9, 2022

👇 Click on the image for a new way to code review
  • Make big changes easier — review code in small groups of related files

  • Know where to start — see the whole change at a glance

  • Take a code tour — explore the change with an interactive tour

  • Make comments and review — all fully sync’ed with github

    Try it now!

Review these changes using an interactive CodeSee Map

Legend

CodeSee Map Legend

@NathanFarmer NathanFarmer marked this pull request as ready for review June 9, 2022 13:48
@NathanFarmer NathanFarmer requested a review from cdkini June 9, 2022 17:35
Copy link
Contributor

@alexsherstinsky alexsherstinsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NathanFarmer We also need to update the V3 Profile Jupyter Notebook renderer in a way that changes the rule names in the stock example for how to configure the variables. For example, you can copy the existing rule settings twice (with the appropriate name changes) and leaving your new tolerances equal to your new low/high precision defaults. The corresponding tests as well as the notebook(s) created by @Shinnnyshinshin will also need to be updated (this is a simple copy/paste from the above). Happy to discuss. Thanks!

Copy link
Contributor

@alexsherstinsky alexsherstinsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NathanFarmer All good regarding rule naming and implementations -- just need to take care of the renderer (you can find which renderer by searching for the OnboardingDataAssistantResut in the repository (including tests). Thanks!
P.S.: Please do not forget about any applicable tests/integration and docs notebooks. Thanks!

Nathan Farmer and others added 4 commits June 9, 2022 21:34
…ion-rounding-error' of github.com:great-expectations/great_expectations into bugfix/OnboardingDataAssistant-mean-and-standard-deviation-rounding-error
Nathan Farmer and others added 2 commits June 10, 2022 08:39
@NathanFarmer NathanFarmer enabled auto-merge (squash) June 10, 2022 14:07
Copy link
Contributor

@alexsherstinsky alexsherstinsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NathanFarmer A couple of "consistency" requests -- almost there! Thanks!

Copy link
Contributor

@alexsherstinsky alexsherstinsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NathanFarmer saw tiny inconsistency regarding round_decimals in two places. Almost! Thanks!

github-actions bot and others added 3 commits June 10, 2022 16:32
…ion-rounding-error' of github.com:great-expectations/great_expectations into bugfix/OnboardingDataAssistant-mean-and-standard-deviation-rounding-error
Copy link
Contributor

@alexsherstinsky alexsherstinsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!!! Yay!

@NathanFarmer NathanFarmer merged commit f23253e into develop Jun 10, 2022
@NathanFarmer NathanFarmer deleted the bugfix/OnboardingDataAssistant-mean-and-standard-deviation-rounding-error branch June 10, 2022 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants