Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUGFIX] Fix docs_integration pipeline #5734

Merged
merged 22 commits into from
Aug 12, 2022
Merged

Conversation

kenwade4
Copy link
Contributor

@kenwade4 kenwade4 commented Aug 10, 2022

Changes proposed in this pull request:

  • Install aws-glue-libs from github repo in docusaurus_tests stage
  • Use SparkContext.getOrCreate() in aws_glue_deployment_patterns.py
  • Use correct suite name in bigquery integration test
  • Use the name of the temp table instead of 'ge_temp' in bigquery integation test
  • Use correct validator var in bigquery integration test
  • Add requirements-dev-azure.txt to requirements-dev.txt
  • Ignore type checking on 4 variables
  • Add some logger.debug messages

@github-actions github-actions bot added the devrel This item is being addressed by the Developer Relations Team label Aug 10, 2022
@netlify
Copy link

netlify bot commented Aug 10, 2022

Deploy Preview for niobium-lead-7998 ready!

Name Link
🔨 Latest commit 24ecacb
🔍 Latest deploy log https://app.netlify.com/sites/niobium-lead-7998/deploys/62f5bed3429fc600086291c5
😎 Deploy Preview https://deploy-preview-5734--niobium-lead-7998.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@ghost
Copy link

ghost commented Aug 10, 2022

👇 Click on the image for a new way to code review
  • Make big changes easier — review code in small groups of related files

  • Know where to start — see the whole change at a glance

  • Take a code tour — explore the change with an interactive tour

  • Make comments and review — all fully sync’ed with github

    Try it now!

Review these changes using an interactive CodeSee Map

Legend

CodeSee Map Legend

Causing ValueError: Cannot run multiple SparkContexts at once

See https://stackoverflow.com/a/47731444
great_expectations.exceptions.exceptions.DataContextError: expectation_suite test_suite_2 not found
Also, "the truth value of a DataFrame is ambiguous" so don't just assert
the dataframe that comes back from validator.head(..)
@kenwade4 kenwade4 changed the title Bug/docs integration tests [BUGFIX] Fix docs_integration pipeline Aug 11, 2022
@kenwade4 kenwade4 marked this pull request as ready for review August 11, 2022 23:45
@@ -4,6 +4,7 @@
--requirement requirements-dev-sqlalchemy.txt

--requirement requirements-dev-arrow.txt
--requirement requirements-dev-azure.txt
Copy link
Member

@Kilo59 Kilo59 Aug 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's already included.

Copy link
Contributor

@Shinnnyshinshin Shinnnyshinshin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't thank you enough for this @kenwade4. LGTM pending @Kilo59 's question

@kenwade4 kenwade4 enabled auto-merge (squash) August 12, 2022 02:43
@kenwade4 kenwade4 merged commit 63b5e26 into develop Aug 12, 2022
@kenwade4 kenwade4 deleted the bug/docs_integration_tests branch August 12, 2022 04:02
Shinnnyshinshin pushed a commit that referenced this pull request Aug 14, 2022
…herstinsky/row_condition_json_serialization_datetime_rule_quantiles_column_partion_and_parameter_builder_datetime_handling-2022_08_13-211' into feature/GREAT-906/updated-data-assistant-notebooks-onboarding

* pre_pr-prototype/maintenance/GREAT-761/GREAT-ABCD/alexsherstinsky/row_condition_json_serialization_datetime_rule_quantiles_column_partion_and_parameter_builder_datetime_handling-2022_08_13-211: (24 commits)
  fix regex parameter builder and its test
  datetime handling for kde estimation
  datetime handling for bootstrap estimation
  datetime handling
  [FEATURE] Enable passing "exact_estimation" boolean at `DataAssistant.run()` level (default value is True) (#5744)
  clean up
  clean up
  [MAINTENANCE] opt-out type-checking  (#5713)
  Update DataAssistantRunner API to accept estimation type argument.  Also, sort all domain arguments according to their class signatures.
  Add plist to build docker test image daily. (#5754)
  [BUGFIX] Patch flaky E2E Cloud test with randomized suite names (#5752)
  [MAINTENANCE] invoke docker (#5703)
  [MAINTENANCE] Clean up type hints around class constructors (#5738)
  [MAINTENANCE]profile_numeric_columns_diff_expectation (#5741)
  [MAINTENANCE] Store Refactor - cloud store return types & http-errors (#5730)
  [MAINTENANCE] Rename GX Cloud "suite_validation_result" resource to "validation_result" (#5749)
  [BUGFIX] Fix docs_integration pipeline (#5734)
  cleanup
  dir update for data_assistant_result
  makefun version pinning
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devrel This item is being addressed by the Developer Relations Team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants