Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SummaryConstraint: Table shape constraints #398

Merged
merged 20 commits into from
Jan 25, 2022

Conversation

pecop2
Copy link
Contributor

@pecop2 pecop2 commented Dec 27, 2021

Description

columnExistsConstraint
numberOfRowsConstraint
columnsMatchSetConstraint

Evaluated using apply_table_shape_constraints
Attribute table_shape_constraints in DatasetConstraints
Attribute total_row_number in DatasetProfile
Tests for DatasetConstraints serialization/deserialization as changes in proto DatasetConstraintMsg were required
Tests for table shape constraints

General Checklist

  • Tests added for this feature/bug
    if it was a bug, test must cover it.
  • Conform by the style guides, by using formatter
  • Documentation updated
  • (optional) Please add a label to your PR

Copy link

@Taxuspt Taxuspt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.
I left a few comments, mostly about styles.
I don't know if this was discussed with Jamie, but should the code be PEP8 compliant?

src/whylogs/core/statistics/constraints.py Show resolved Hide resolved
src/whylogs/core/statistics/constraints.py Outdated Show resolved Hide resolved
src/whylogs/core/statistics/constraints.py Outdated Show resolved Hide resolved
src/whylogs/core/statistics/constraints.py Outdated Show resolved Hide resolved
src/whylogs/core/statistics/constraints.py Show resolved Hide resolved
@pecop2 pecop2 force-pushed the dev/loka/table_shape_constraints branch from d909dc9 to 76cd87f Compare January 11, 2022 14:04
Copy link
Contributor

@jamie256 jamie256 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When ready, point to mainline and I can take another look.

src/whylogs/core/statistics/constraints.py Outdated Show resolved Hide resolved
@pecop2 pecop2 changed the base branch from dev/loka/constraints to mainline January 18, 2022 16:53
Copy link
Contributor

@jamie256 jamie256 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please resolve conflicts (hopefully that also triggers CI to run and report test status, looks like the last test run never reported status).

@coveralls
Copy link

coveralls commented Jan 20, 2022

Pull Request Test Coverage Report for Build 1746469670

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 90 of 92 (97.83%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall first build on dev/loka/table_shape_constraints at 82.668%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/whylogs/core/statistics/constraints.py 78 80 97.5%
Totals Coverage Status
Change from base Build 1743181373: 82.7%
Covered Lines: 4388
Relevant Lines: 5196

💛 - Coveralls

@pecop2 pecop2 requested a review from Taxuspt January 21, 2022 00:41
Copy link
Contributor

@jamie256 jamie256 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General implementation looks good. I think there are some edge cases not handled I left a comment on.

src/whylogs/core/datasetprofile.py Outdated Show resolved Hide resolved
@jamie256 jamie256 merged commit 7434e09 into mainline Jan 25, 2022
@jamie256 jamie256 deleted the dev/loka/table_shape_constraints branch January 25, 2022 22:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants