Repository navigation
bug(Source facebook-marketing): upgrade cdk to filter invalid fields - #41958
Aldo Gonzalez (aldogonzalez8) merged 3 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
Octavia Squidington III (octavia-squidington-iii)
added
the
area/connectors
Octavia Squidington III (octavia-squidington-iii)
added
the
area/documentation
Aldo Gonzalez (aldogonzalez8)
changed the title
| [tool.poetry.dependencies] | ||
| python = "^3.9,<3.12" | ||
| airbyte-cdk = "^2.3.0" | ||
| airbyte-cdk = "^3.5.0" |
There was a problem hiding this comment.
this is a pretty big bump. Let's make sure to test with live-tests
There was a problem hiding this comment.
Live tests are ongoing...
airbyte-ci connectors --name=source-facebook-marketing build --architecture=linux/arm64
poetry run pytest src/live_tests/regression_tests --connector-image=airbyte/source-facebook-marketing --pr-url=https://github.com/airbytehq/airbyte/pull/41958 --connection-id=81d16659-48bf-44f3-ad98-17f4561e225eWill let you know results
There was a problem hiding this comment.
Changing connection id as the other was not available for live tests (EU)
poetry run pytest src/live_tests/regression_tests --connector-image=airbyte/source-facebook-marketing --pr-url=https://github.com/airbytehq/airbyte/pull/41958 --connection-id=8d7631c6-edd4-4fe6-86ac-7d7efa3261f3
There was a problem hiding this comment.
Note: there is a problem currently with requests count and url in live tests that are not reflected in facebook-marketing
I can see the logging we added in the target logs on my local:
{"level": "WARN", "message": "Stream ads_insights_action_type: the following fields are deprecated and cannot be synced. {'conversion_lead_rate', 'cost_per_conversion_lead'}. Refresh the connection's source schema to resolve this warning.", "stack_trace": null}
{"level": "WARN", "message": "Stream ads_insights: the following fields are deprecated and cannot be synced. {'conversion_lead_rate', 'cost_per_conversion_lead'}. Refresh the connection's source schema to resolve this warning.", "stack_trace": null}
There was a problem hiding this comment.
I did more testing on a different connection. It was not reported to have problems but wanted to check another case to grow confidence in this solution cc Alexandre Girard (@girarda)
Record count is consistent:
Also no errors found in airbyte messages of target:
Note: I couldn't find our warning for "the following fields are deprecated..." but this would be expected if there were no problems with the configured catalog; I just wanted to check that a healthy running connection would continue healthy.
This branch was successfully deployed
3 participants
What
We have seen a recent problem with configured catalog having deprecated fields in configuration. We now have a fix for this in CDK.
How
Upgrade CDK version to get fix on invalid fields.
User Impact
After unpinned old facebook-marketing version to now use the latest users should not see anymore this error.
Can this PR be safely reverted and rolled back?