Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apply GDPR DoNotProcess for Zuora Account on the ophan lake side #400

Merged

Conversation

mariogalic
Copy link
Contributor

@mariogalic mariogalic commented Sep 2, 2019

https://trello.com/c/5QPjfBvi/331-understand-the-reasons-for-the-differences-between-rawzu-cleanzuora

Corresponding lake PR: https://github.com/guardian/ophan-data-lake/pull/4423

Fix for why

thousands are in clean and not raw:*

SELECT approx_distinct(name) FROM clean.zuora_subscription WHERE name NOT IN (
    SELECT DISTINCT subscription_name FROM raw.zu_subs
) 

Having mutable fields in filter does not work with Zuora stateful export API so we have to add corresponding fields to export and filter them on the ophan-data-lake side.

@coveralls
Copy link

Coverage Status

Coverage remained the same at 58.984% when pulling 1ad22d7 on apply-gdpr-DoNotProcess-for-zuora-account-on-ophan-side into 1aad211 on master.

@mariogalic mariogalic merged commit 05ad92c into master Sep 2, 2019
@mariogalic mariogalic deleted the apply-gdpr-DoNotProcess-for-zuora-account-on-ophan-side branch September 2, 2019 21:20
@mariogalic
Copy link
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants