Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MAINTENANCE] Small refactor for tests that allows DB setup to be done from all tests #4012

Merged

Conversation

Shinnnyshinshin
Copy link
Contributor

@Shinnnyshinshin Shinnnyshinshin commented Jan 18, 2022

Changes proposed in this pull request:

  • [DOCS] Migration Guide now includes example for Databases #4005 requires a util function that was previously only used by tests in the folder docusaurus/connecting_to_your_data/database, which loads data into a test DB. The "simplest" way to make sure the Migration Guide tests also had access to this function this involved making an additional copy of the util.py file, which we wanted to avoid.

  • This is made slightly more complicated by the fact that test directories for Docusaurus integration tests are generated fresh for each test, and we don't have access to great_expectations/tests/ directory within each test.

This PR does two things.

  1. Moves the necessary function load_data_into_test_database() into the tests/test_utils.py file where it can be accessed from all tests. In the process we also add a boolean flag that determines whether the full dataset or only the first 10 lines will be loaded into the test database (first 10 lines by default).
  2. Refactors the test_script_runner.py file + associated tests to use this new function.

Previous Design Review notes:

  • GREAT-504

Definition of Done

Please delete options that are not relevant.

  • My code follows the Great Expectations style guide
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added unit tests where applicable and made sure that new and existing tests are passing.
  • I have run any local integration tests and made sure that nothing is broken.

Thank you for submitting!

@netlify
Copy link

netlify bot commented Jan 18, 2022

✔️ Deploy Preview for niobium-lead-7998 ready!

🔨 Explore the source changes: a53b495

🔍 Inspect the deploy log: https://app.netlify.com/sites/niobium-lead-7998/deploys/61e7148f8756d90007b61cee

😎 Browse the preview: https://deploy-preview-4012--niobium-lead-7998.netlify.app

@github-actions
Copy link
Contributor

HOWDY! This is your friendly 🤖 CHANGELOG bot 🤖

Please don't forget to add a clear and succinct description of your change under the Develop header in docs_rtd/changelog.rst, if applicable. This will help us with the release process. See the Contribution checklist in the Great Expectations documentation for the type of labels to use!

Thank you!

@Shinnnyshinshin Shinnnyshinshin changed the title [MAINTENANCE] Small refactor that allows test DB setup to be done from all tests [MAINTENANCE] Small refactor for tests that allows DB setup to be done from all tests Jan 18, 2022
@Shinnnyshinshin Shinnnyshinshin self-assigned this Jan 18, 2022
@@ -104,25 +104,21 @@ class BackendDependencies(enum.Enum):
"user_flow_script": "tests/integration/docusaurus/connecting_to_your_data/how_to_choose_which_dataconnector_to_use.py",
"data_context_dir": "tests/integration/fixtures/no_datasources/great_expectations",
"data_dir": "tests/test_sets/dataconnector_docs",
"util_script": "tests/integration/docusaurus/connecting_to_your_data/database/util.py",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

util_script is not needed for data_connector tests

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙇🏻

Copy link
Contributor

@NathanFarmer NathanFarmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! So simple, yet now it is so much clearer what the purpose of that function is.

@Shinnnyshinshin Shinnnyshinshin enabled auto-merge (squash) January 18, 2022 19:54
@Shinnnyshinshin Shinnnyshinshin merged commit 93b95b0 into develop Jan 18, 2022
@Shinnnyshinshin Shinnnyshinshin deleted the MAINTENANCE/GREAT-504/move-util-to-somewhere-better branch January 18, 2022 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants