[FEATURE] Add Azure CI/CD action to aid with style guide enforcement (docstrings) #4617
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…dation Dependencies 2022 03 23 66 (#4531)
* removing upper bound on mistune * remove deprecated depedencies * adds untracked dependency * adds untracked dependency * adds untracked dependency * moving dependencies * removes dependencies added to lite from core | adds missing dependencies Co-authored-by: Chetan Kini <chetan@superconductive.com>
…ectationConfigurationBuilder Self-Contained with its own validation_parameter_builder settings (#4547)
* Add validate_configuration to 2 core Expectations that are passing all their tests * Comment out examples for expect_column_values_to_match_regex.py... its test_definitions JSON has many more tests * Update all validate_configuration methods to have type hints and return None * Update all doc snippet references that were effected
* - Adds technical tags to all documents in the Connect to data: Cloud section of the docs. (Note, the term in the <WhereToRunCode /> imported component was tagged in a different PR.) - Some additional editing was done to bring documents in line with the documentation and how-to guide standards. * - Fixed extra </Tabs> and </TabItem> closures from prior commit to resolve conflicts with develop.
- Minor updates to correct formatting and spelling issues.
…in the ToC (#4551) - Minor edit to title of "How to write integration documentation" to conform to ToC standards (not title cased unless containing a Technical Term).
- Updates the image file link for the overview image on the intro page
* - DOC-184: Specify in the tutorial that Spark and SqlAlchemy are also supported Execution Engines. * - DOC-183: In the Execution Engine technical term page, list the class names for Execution Engines and specify that spark is supported as a scalable alternative to Pandas. * - DOC-182: In the connect to data: overview section for "configuring your datasource's execution engine" list the class names for execution engines.
* - add support for technical term tags. * - added technical term tags. - Updated how to validate data without a checkpoint to mention the replacement workflow and that additional documentation is forthcoming. - NOTE: how_to_implement_custom_notifications needs to be rewritten, and was not edited as part of this PR. * - added technical term tags. - Updated how to validate data without a checkpoint to mention the replacement workflow and that additional documentation is forthcoming. - NOTE: how_to_implement_custom_notifications needs to be rewritten, and was not edited as part of this PR.
* - Edits to bring docs up to documentation and how-to guide standards. * - add technical term tags to documents. - minor formatting edits (technical terms missing capitalization, etc).
…e modules/classes for better separation of concerns (#4553)
* chore: use black directives to temporarily disable linting * chore: more black directives to temporarily disable linting * chore: finish remaining * refactor: start cleaning up conftest * refactor: more refactoring of conftest * refactor: even more refactoring of conftest
* Update library_metadata check to provide details when it doesn't pass * In linting check, if snake_case doesn't match filename, show computed snake_case * Change class name for expect_column_values_to_be_valid_ipv4.py and remove package attr * Update convert_to_json_serializable to handle bytes * Update build_gallery.py script to convert diagnostics to JSON in separate try/except * Update build_gallery.py script to write expectation_library_v2.json file with indenting * Update _check_input_validation to tell if custom assert statements are used in validate_configuration
…rn (#4485) * feat: init commit * refactor: shift all logic over to base class * feat: start impl of anonymize on Anonymizer * feat: get ProfilerRunAnonymizer working * refactor: remove constructor from ProfilerRunAnonymizer * refactor: start on CheckpointRunAnonymizer * fix: clean up broken checkpoint tests * fix: ensure *args and **kwargs are propogated through * refactor: start work on datasource anonymizers * refactor: remove all anonymizers except Anonymizer from usage stats attrs * fix: update isinstance checks * refactor: move helper into checkpoint_run_anonymizer * refactor: move helper into datasource_anonymizer * refactor: make anonymize string private and place in strategy * refactor: make anonymize batch info private and place in strategy * refactor: move build_init_payload to Anonymizer * refactor: make remainder of anonymize methods private * refactor: add store info to strategy * refactor: add dataconnector info to strategy * refactor: consolidate profiler info and profiler run anonymization * refactor: remove *args from signatures * refactor: updates around checkpoint anonymization * chore: misc cleanup of Anonymizer * feat: final touch up before review * chore: remove 'else' statements * fix: ensure appropriate checkpoint method gets called * chore: misc updates from review * refactor: move init_payload back to usage stats * chore: misc type hinting * refactor: start using individual classes again * chore: continue updating individual anonymizer classes * feat: further updates to child classes * feat: update anonymize_init_payload * fix: get checkpoint payloads working * refactor: ensure all methods have obj * fix: misc fixes * fix: make misc updates to conditional checks for obj * refactor: rename ExpectationAnonymizer to ExpectationSuiteAnonymizer * refactor: rename Checkpoint and Profiler anonymizers * feat: leverage aggregate anonymizer downstream * feature: conditionally create aggregate_anonymizer in constructor * feat: add cache retrieve or instantiate util * chore: add batch_request can_handle * feat: ensure that salt has a default value in anonymizers * refactor: require aggregate anonymizer in constructor * refactor: instantiate all strategies in aggregate * fix: fix broken tests * refactor: rename internal getter Co-authored-by: Don Heppner <donald.heppner@gmail.com>
…o create a batch of data from an in memory data frame (#4573) * -Corrected the line references and added <snippet> tags to source code for Spark version of guide. * -Corrected the line references and added <snippet> tags to source code for Pandas version of guide. * -lint reformat w/black * -correcting line numbers after lint formatting.
… 1 (#4546) Usage stats instrumentation of package dependencies
… v8.1.0 (#4577) * chore: bump version * chore: test change * chore: update all instances of black * chore: new test changes * chore: revert test changes
* Add missing links. * Fix some typos * Simplify flow and grammar in a few places Co-authored-by: Austin Ziech Robinson <44794138+austiezr@users.noreply.github.com>
- updated links in admonition box to point to current technical documentation rather than old core concepts documents.
Little bit of cleanup in our execution engine and validator
…ation (#4590) * fix : misconfigured ExpectationConfigurationBuilder * pushing fix * clean up before submitting for review * bugfix : remove sorting * remove extra line
* Add dependencies to data_context.__init__ event
…_expectations into feature/gh-action-for-code-quality
…om/great-expectations/great_expectations into feature/gh-action-for-code-quality
donaldheppner
requested changes
Apr 8, 2022
scripts/check_docstring_coverage.py
Outdated
| assert ( | ||
| total_failed <= DOCSTRING_ERROR_THRESHOLD | ||
| ), f"""A public function without a docstring was introduced; please resolve the matter before merging. | ||
| We expect there to be {total_failed} or fewer violations of the style guide (actual: {total_failed})""" |
There was a problem hiding this comment.
total_failed should not appear twice (only in the actual: ...) area. Also, right now the threshold is below the actual, if merged this check would always fail
There was a problem hiding this comment.
good call! should be up to date now
…pectations into feature/gh-action-for-code-quality
…_expectations into feature/gh-action-for-code-quality
cdkini
changed the title
cdkini
changed the title
donaldheppner
approved these changes
Apr 19, 2022
…_expectations into feature/gh-action-for-code-quality
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes proposed in this pull request:
mypyto ensure that no further type-hint violations are introduced to the codebase.Definition of Done
Please delete options that are not relevant.
Thank you for submitting!
Definition of Done
Please delete options that are not relevant.
Thank you for submitting!