[FEATURE] Script to automate proper triggering of Docs Azure pipeline #3003
Conversation
|
✔️ Deploy Preview for knoxpod ready! 🔨 Explore the source changes: 66bd06c 🔍 Inspect the deploy log: https://app.netlify.com/sites/knoxpod/deploys/60f5dfc7a37a6f0007a531f8 😎 Browse the preview: https://deploy-preview-3003--knoxpod.netlify.app |
HOWDY! This is your friendly 🤖 CHANGELOG bot 🤖Please don't forget to add a clear and succinct description of your change under the Develop header in ✨ Thank you! ✨ |
There was a problem hiding this comment.
Made changes per our last discussion @alexsherstinsky please take a quick look whenever you have a minute.
I've left a comment or two but we should hopefully be good to have this merged in shortly!
scripts/check_for_docs_deps_changes
Outdated
|
|
||
| pip install -e . > /dev/null 2>&1 # Necessary to get GE helper methods working for `trace_docs_deps.py` | ||
|
|
There was a problem hiding this comment.
@alexsherstinsky we actually need to install GE if we want to use the file_relative_path() helper method in our dependencies script.
I don't think this is worth it as the remainder of script works as intended without any external dependencies.
scripts/trace_docs_deps.py
Outdated
| Please note that this solution has some inherit disclaimers one should be familiar with. As there is no actual running | ||
| of source code (but rather parsing of the source file's AST), it is hard to exactly pinpoint the location of a given | ||
| import. Utilizing the `__file__` attr would provide the exact path of an import but without that, we need to assume that | ||
| an ast.Import node corresponds to an accurate path in our codebase (i.e. `great_expectations.core.batch` results in | ||
| great_expectations/core/batch.py). | ||
|
|
||
| As long as imports correspond to the relative path of modules, this method will continue to work as intended. Deviations | ||
| here would result in certain imports not being picked up by the script, reducing the accuracy and frequency of the | ||
| Azure Docs Integration pipeline's triggers. As GE has maintained this consistency throughout its development, this | ||
| script and the methods used herein are deemed to be appropriate. |
There was a problem hiding this comment.
@alexsherstinsky added some color around possible pitfalls. Let me know if that's adequate or if I can better clarify anything.
scripts/trace_docs_deps.py
Outdated
| for line in open(doc): | ||
| if re.search(pattern, line): | ||
| file: str = _parse_file_from_docusaurus_link(line) | ||
| path: str = file_relative_path(doc, file) |
There was a problem hiding this comment.
I think this should be os.path.join(os.path.dirname(doc), file) (what file_relative_path is actually doing) so we can remove GE as a dependency from this script.
Please annotate your PR title to describe what the PR does, then give a brief bulleted description of your PR below. PR titles should begin with [BUGFIX], [FEATURE], [DOCS], or [MAINTENANCE]. If a new feature introduces breaking changes for the Great Expectations API or configuration files, please also add [BREAKING]. You can read about the tags in our contributor checklist.
Changes proposed in this pull request:
docs/has ongreat_expectations/.After submitting your PR, CI checks will run and @tiny-tim-bot will check for your CLA signature.
For a PR with nontrivial changes, we review with both design-centric and code-centric lenses.
In a design review, we aim to ensure that the PR is consistent with our relationship to the open source community, with our software architecture and abstractions, and with our users' needs and expectations. That review often starts well before a PR, for example in github issues or slack, so please link to relevant conversations in notes below to help reviewers understand and approve your PR more quickly (e.g.
closes #123).Previous Design Review notes:
Definition of Done
Please delete options that are not relevant.
Thank you for submitting!