vdk-impala: Introduce checks for scd1 template #1472
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
What: Adding functionality to handle the scd1 template behavior if there are checks provided by the user Why: It is linked to #1361 Tests: provided positive and negative regression tests
for more information, see https://pre-commit.ci
antoniivanov
reviewed
Jan 4, 2023
...s/vdk-impala/src/vdk/plugin/impala/templates/load/dimension/scd1/02-handle-quality-checks.py
Show resolved
Hide resolved
...s/vdk-impala/src/vdk/plugin/impala/templates/load/dimension/scd1/02-handle-quality-checks.py
Outdated
Show resolved
Hide resolved
...s/vdk-impala/src/vdk/plugin/impala/templates/load/dimension/scd1/02-handle-quality-checks.py
Outdated
Show resolved
Hide resolved
projects/vdk-plugins/vdk-impala/tests/functional/template_regression_test.py
Show resolved
Hide resolved
...s/vdk-impala/src/vdk/plugin/impala/templates/load/dimension/scd1/02-handle-quality-checks.py
Show resolved
Hide resolved
murphp15
reviewed
Jan 4, 2023
...s/vdk-impala/src/vdk/plugin/impala/templates/load/dimension/scd1/02-handle-quality-checks.py
Show resolved
Hide resolved
murphp15
reviewed
Jan 4, 2023
...s/vdk-impala/src/vdk/plugin/impala/templates/load/dimension/scd1/02-handle-quality-checks.py
Outdated
Show resolved
Hide resolved
…es' of https://github.com/vmware/versatile-data-kit into person/sbuldeev/add-quality-check-to-processing-templates
for more information, see https://pre-commit.ci
|
I don't know if you've noticed but there are some failing tests: https://gitlab.com/vmware-analytics/versatile-data-kit/-/jobs/3552673519 Have you be able to run them locally ? In IDE ? |
antoniivanov
changed the title
These tests are failing, because at some point the execute_tempalate behavior is trying to dump all the arguments and since there is a function passed as an argument this is not possible. |
…es' of https://github.com/vmware/versatile-data-kit into person/sbuldeev/add-quality-check-to-processing-templates
for more information, see https://pre-commit.ci
…es' of https://github.com/vmware/versatile-data-kit into person/sbuldeev/add-quality-check-to-processing-templates
for more information, see https://pre-commit.ci
…es' of https://github.com/vmware/versatile-data-kit into person/sbuldeev/add-quality-check-to-processing-templates
for more information, see https://pre-commit.ci
…es' of https://github.com/vmware/versatile-data-kit into person/sbuldeev/add-quality-check-to-processing-templates
…es' of https://github.com/vmware/versatile-data-kit into person/sbuldeev/add-quality-check-to-processing-templates
for more information, see https://pre-commit.ci
…es' of https://github.com/vmware/versatile-data-kit into person/sbuldeev/add-quality-check-to-processing-templates
projects/vdk-plugins/vdk-impala/tests/functional/template_regression_test.py
Outdated
Show resolved
Hide resolved
projects/vdk-plugins/vdk-impala/tests/functional/template_regression_test.py
Outdated
Show resolved
Hide resolved
...s/vdk-impala/src/vdk/plugin/impala/templates/load/dimension/scd1/02-handle-quality-checks.py
Show resolved
Hide resolved
for more information, see https://pre-commit.ci
antoniivanov
approved these changes
Jan 23, 2023
|
@sbuldeev, before merging do ping me so I can walk you through the proper process for merging a change. A) It's fairly important that there's one commit (in this case particularly) - so you need to use Squash option |
sbuldeev
deleted the
person/sbuldeev/add-quality-check-to-processing-templates
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What:
Adding functionality to handle the scd1 template behavior if there are checks provided by the user
Why:
It is linked to #1361
Tests: provided positive and negative regression tests