Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MAINTENANCE] Write integration/E2E tests for both GCSDataConnectors #3301

Conversation

cdkini
Copy link
Member

@cdkini cdkini commented Aug 23, 2021

Please annotate your PR title to describe what the PR does, then give a brief bulleted description of your PR below. PR titles should begin with [BUGFIX], [FEATURE], [DOCS], or [MAINTENANCE]. If a new feature introduces breaking changes for the Great Expectations API or configuration files, please also add [BREAKING]. You can read about the tags in our contributor checklist.

Changes proposed in this pull request:

Definition of Done

Please delete options that are not relevant.

  • My code follows the Great Expectations style guide
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added unit tests where applicable and made sure that new and existing tests are passing.
  • I have run any local integration tests and made sure that nothing is broken.

…s_to_pandas_execution_engine_to_facilitate_azure_connectors
…s_to_pandas_execution_engine_to_facilitate_azure_connectors
…tions into FEATURE/GREAT-71/GREAT-81/implement_configured_asset_gcs_data_connector
…ion_tests_for_inferred_asset_gcs_data_connector
@cdkini cdkini marked this pull request as ready for review August 24, 2021 13:17
…inferred_asset_gcs_data_connector' of github.com:great-expectations/great_expectations into feature/GREAT-71/GREAT-179/write_integration_tests_for_inferred_asset_gcs_data_connector
…tions into feature/GREAT-71/GREAT-179/write_integration_tests_for_inferred_asset_gcs_data_connector
…tions into feature/GREAT-71/GREAT-179/write_integration_tests_for_inferred_asset_gcs_data_connector
…ecution_engine_to_facilitate_azure_connectors' into feature/GREAT-71/GREAT-179/write_integration_tests_for_inferred_asset_gcs_data_connector
@cdkini cdkini force-pushed the feature/GREAT-71/GREAT-179/write_integration_tests_for_inferred_asset_gcs_data_connector branch from f0690f6 to aa98dc3 Compare August 24, 2021 16:51
@cdkini cdkini force-pushed the feature/GREAT-71/GREAT-179/write_integration_tests_for_inferred_asset_gcs_data_connector branch from 64d1265 to 991c79c Compare August 24, 2021 21:22
Comment on lines +50 to +57
# "user_flow_script": "tests/integration/docusaurus/connecting_to_your_data/cloud/azure/pandas/inferred_and_runtime_yaml_example.py",
# "data_context_dir": "tests/integration/fixtures/no_datasources/great_expectations",
# },
# {
# "user_flow_script": "tests/integration/docusaurus/connecting_to_your_data/cloud/azure/pandas/inferred_and_runtime_python_example.py",
# "data_context_dir": "tests/integration/fixtures/no_datasources/great_expectations",
# },
# GCS
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Segmenting by cloud provider and writing out the fixtures for the remaining Azure tests. If we like this, we could apply these changes to the S3 tests (once we've fixed them up!)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

…ion_tests_for_inferred_asset_gcs_data_connector
Copy link
Contributor

@Shinnnyshinshin Shinnnyshinshin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks GREAT to me

Comment on lines +59 to +73
"user_flow_script": "tests/integration/docusaurus/connecting_to_your_data/cloud/gcs/pandas/configured_yaml_example.py",
"data_context_dir": "tests/integration/fixtures/no_datasources/great_expectations",
},
{
"user_flow_script": "tests/integration/docusaurus/connecting_to_your_data/cloud/gcs/pandas/configured_python_example.py",
"data_context_dir": "tests/integration/fixtures/no_datasources/great_expectations",
},
{
"user_flow_script": "tests/integration/docusaurus/connecting_to_your_data/cloud/gcs/pandas/inferred_and_runtime_yaml_example.py",
"data_context_dir": "tests/integration/fixtures/no_datasources/great_expectations",
},
{
"user_flow_script": "tests/integration/docusaurus/connecting_to_your_data/cloud/gcs/pandas/inferred_and_runtime_python_example.py",
"data_context_dir": "tests/integration/fixtures/no_datasources/great_expectations",
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that these will be running on Azure makes me very happy 😁

…ion_tests_for_inferred_asset_gcs_data_connector
@cdkini cdkini enabled auto-merge (squash) August 26, 2021 11:25
@cdkini cdkini merged commit 4d19a58 into develop Aug 26, 2021
@cdkini cdkini deleted the feature/GREAT-71/GREAT-179/write_integration_tests_for_inferred_asset_gcs_data_connector branch August 26, 2021 12:01
Shinnnyshinshin pushed a commit that referenced this pull request Aug 28, 2021
…data_context' into MAINTENANCE/GDOC-219/tests_datasource_sparkdf_pandas

* MAINTENANCE/GDOC-2221/runtime_data_connector_sql_tests_data_context:
  after review
  [MAINTENANCE] Constructor for `BatchRequest` requires "triplet", and `RuntimeBatchRequest` also requires `runtime_parameters` and `batch_identifiers` (#3317)
  rename of methods to be more specific
  Release Prep release-prep-2021-08-26 (#3320)
  Docs] hide stubbed core skills (#3316)
  [MAINTENANCE] Write integration/E2E tests for both `GCSDataConnectors` (#3301)
  [DOCS] Standardize capitalization of various technologies in `docs` (#3312)
  [DOCS] Fix misc errors in "How to create renderers for Custom Expectations" (#3315)
Shinnnyshinshin pushed a commit that referenced this pull request Aug 28, 2021
…data_context' into MAINTENANCE/GDOC-222/tests_datacontext_sparkdf_pandas

* MAINTENANCE/GDOC-2221/runtime_data_connector_sql_tests_data_context:
  after review
  [MAINTENANCE] Constructor for `BatchRequest` requires "triplet", and `RuntimeBatchRequest` also requires `runtime_parameters` and `batch_identifiers` (#3317)
  rename of methods to be more specific
  Release Prep release-prep-2021-08-26 (#3320)
  Docs] hide stubbed core skills (#3316)
  [MAINTENANCE] Write integration/E2E tests for both `GCSDataConnectors` (#3301)
  [DOCS] Standardize capitalization of various technologies in `docs` (#3312)
  [DOCS] Fix misc errors in "How to create renderers for Custom Expectations" (#3315)
Shinnnyshinshin pushed a commit to jdimatteo/great_expectations that referenced this pull request Aug 31, 2021
…matteo/great_expectations into feature/add_performance_test

* 'feature/add_performance_test' of https://github.com/jdimatteo/great_expectations:
  [MAINTENANCE] Tests for RuntimeDataConnector at Datasource-level (Spark and Pandas) (great-expectations#3318)
  [MAINTENANCE] Tests for RuntimeDataConnector at DataContext-level (great-expectations#3304)
  [BUGFIX] SQL dialect doesn't register for BigQuery for V2 (great-expectations#3324)
  [WIP] [FEATURE] add backend args to run_diagnostics (great-expectations#3257)
  Release Prep release-prep-2021-08-26 (great-expectations#3320)
  Docs] hide stubbed core skills (great-expectations#3316)
  [MAINTENANCE] Write integration/E2E tests for both `GCSDataConnectors` (great-expectations#3301)
  [DOCS] Standardize capitalization of various technologies in `docs` (great-expectations#3312)
  [DOCS] Fix misc errors in "How to create renderers for Custom Expectations" (great-expectations#3315)
  docs: Remove misc TODOs to tidy up docs (great-expectations#3313)
  [DOCS] GDOC-217 remove stub links (great-expectations#3314)
  [FEATURE] Enable `GCS DataConnector` integration with `PandasExecutionEngine` (great-expectations#3264)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants