Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GreatExpectationsBigQueryOperator is failing on Airflow #14

Closed
marktanios7 opened this issue Mar 8, 2021 · 1 comment · Fixed by #15
Closed

GreatExpectationsBigQueryOperator is failing on Airflow #14

marktanios7 opened this issue Mar 8, 2021 · 1 comment · Fixed by #15
Assignees

Comments

@marktanios7
Copy link

marktanios7 commented Mar 8, 2021

Describe the bug
Hi,
I have deployed Great Expectations with Google Cloud Composer. I've created an Airflow DAG to insert data into table from another table and then validate if the data is correct. Then I generated suite file using the scaffold example. I added all of GE files to GCP bucket. But I'm receiving this error: DataContextError: No validation operator 'action_list_operator' was found in your project. Please verify this in your great_expectations.yml. I'm sure that action_list_operator exists in my great_expectations.yml file.

Error message:
DataContextError: No validation operator 'action_list_operator' was found in your project. Please verify this in your great_expectations.yml. I'm sure that action_list_operator exists in my great_expectations.yml file.

How to reproduce:

  1. Installed pybigquery, great_expectations, airflow-provider-great-expectations on the composer environment.
  2. Followed this example with the same folder structure.

Environment Versions:
Airflow Version: 1.10.9+composer
Great Expectations Version: 0.13.11

I've also attached my DAG and great_expectations.yml files.

Workaround:
One workaround to fix this is that I've created my custom operator using this code and added action_list_operator part manually inside create_data_context_config function. But I'm receiving another error: IndexError: tuple index out of range

Workaround Error:
image

attached_files.zip

Thanks in advance!

@spbail
Copy link
Contributor

spbail commented Mar 10, 2021

Thanks for filing this issue! We're looking into this, there have recently been changes to the Checkpoint/Validation Operator functionality in core Great Expectations, so the operator needs to be adapted for that.

@spbail spbail linked a pull request Mar 25, 2021 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants