Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Implement TupleFilesystemStoreBackend::get_all #9687

Merged
merged 4 commits into from Apr 2, 2024

Conversation

tyler-hoffman
Copy link
Contributor

  • Description of PR changes above includes a link to an existing GitHub issue
  • PR title is prefixed with one of: [BUGFIX], [FEATURE], [DOCS], [MAINTENANCE], [CONTRIB]
  • Code is linted - run invoke lint (uses ruff format + ruff check)
  • Appropriate tests and docs have been updated

For more information about contributing, see Contribute.

After you submit your PR, keep the page open and monitor the statuses of the various checks made by our continuous integration process at the bottom of the page. Please fix any issues that come up and reach out on Slack if you need help. Thanks for contributing!

Copy link

netlify bot commented Apr 1, 2024

Deploy Preview for niobium-lead-7998 canceled.

Name Link
🔨 Latest commit 0e717e0
🔍 Latest deploy log https://app.netlify.com/sites/niobium-lead-7998/deploys/660c6147b6e7cc0008b5210e

Copy link

codecov bot commented Apr 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.55%. Comparing base (0b7b2aa) to head (0e717e0).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #9687      +/-   ##
===========================================
- Coverage    82.55%   82.55%   -0.01%     
===========================================
  Files          511      511              
  Lines        46437    46439       +2     
===========================================
+ Hits         38335    38336       +1     
- Misses        8102     8103       +1     
Flag Coverage Δ
3.10 64.62% <0.00%> (-0.01%) ⬇️
3.10 athena or clickhouse or openpyxl or pyarrow or project or sqlite or aws_creds ?
3.10 aws_deps ?
3.10 big ?
3.10 databricks ?
3.10 filesystem ?
3.10 mssql ?
3.10 mysql ?
3.10 postgresql ?
3.10 snowflake ?
3.10 spark ?
3.10 trino ?
3.11 64.62% <0.00%> (-0.01%) ⬇️
3.11 athena or clickhouse or openpyxl or pyarrow or project or sqlite or aws_creds 53.96% <0.00%> (-0.01%) ⬇️
3.11 aws_deps 48.98% <0.00%> (-0.01%) ⬇️
3.11 big 63.97% <0.00%> (-0.01%) ⬇️
3.11 databricks 48.21% <0.00%> (-0.01%) ⬇️
3.11 filesystem 63.78% <100.00%> (+<0.01%) ⬆️
3.11 mssql 47.43% <0.00%> (-0.01%) ⬇️
3.11 mysql 47.48% <0.00%> (-0.01%) ⬇️
3.11 postgresql 54.26% <0.00%> (-0.01%) ⬇️
3.11 snowflake 48.74% <0.00%> (-0.01%) ⬇️
3.11 spark 60.65% <0.00%> (-0.01%) ⬇️
3.11 trino 53.89% <0.00%> (-0.01%) ⬇️
3.8 64.62% <0.00%> (-0.01%) ⬇️
3.8 athena or clickhouse or openpyxl or pyarrow or project or sqlite or aws_creds 53.97% <0.00%> (-0.01%) ⬇️
3.8 aws_deps 48.99% <0.00%> (-0.01%) ⬇️
3.8 big 63.98% <0.00%> (-0.01%) ⬇️
3.8 databricks 48.22% <0.00%> (-0.01%) ⬇️
3.8 filesystem 63.78% <100.00%> (+<0.01%) ⬆️
3.8 mssql 47.41% <0.00%> (-0.01%) ⬇️
3.8 mysql 47.46% <0.00%> (-0.01%) ⬇️
3.8 postgresql 54.25% <0.00%> (-0.01%) ⬇️
3.8 snowflake 48.76% <0.00%> (+<0.01%) ⬆️
3.8 spark 60.61% <0.00%> (-0.01%) ⬇️
3.8 trino 53.88% <0.00%> (-0.01%) ⬇️
3.9 64.62% <0.00%> (-0.01%) ⬇️
3.9 athena or clickhouse or openpyxl or pyarrow or project or sqlite or aws_creds ?
3.9 aws_deps ?
3.9 big ?
3.9 databricks ?
3.9 filesystem ?
3.9 mssql ?
3.9 mysql ?
3.9 postgresql ?
3.9 snowflake ?
3.9 spark ?
3.9 trino ?
cloud 0.00% <0.00%> (ø)
docs-basic 54.49% <0.00%> (-0.01%) ⬇️
docs-creds-needed 55.06% <0.00%> (-0.01%) ⬇️
docs-spark 54.59% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -316,7 +316,8 @@ def _get(self, key):

@override
def _get_all(self) -> list[Any]:
raise NotImplementedError
keys = [key for key in self.list_keys() if key != StoreBackend.STORE_BACKEND_ID_KEY]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels weird to have to exclude this myself. Is there a better option?

Copy link
Member

@Kilo59 Kilo59 Apr 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is getting reported correctly (eventually) but codecov begins commenting/reporting immediately.
https://app.codecov.io/gh/great-expectations/great_expectations/pull/9687/blob/great_expectations/data_context/store/tuple_store_backend.py

image

https://docs.codecov.com/docs/pull-request-comments#after_n_builds

By default Codecov will post and/or update the pull request comment after it processes each report uploaded for a particular pull request commit. If your CI process uploads many reports to codecov, this can be confusing for team members viewing the pull request as reports are processing.

To remedy this, you can delay the posting of a pull request comment until a certain number of reports are received and processed by Codecov, using the after_n_builds setting.

@tyler-hoffman tyler-hoffman added this pull request to the merge queue Apr 2, 2024
Merged via the queue into develop with commit e0c61f8 Apr 2, 2024
69 of 70 checks passed
@tyler-hoffman tyler-hoffman deleted the f/v1-231/TupleFilesystemStoreBackend-get_all branch April 2, 2024 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants