🐛 Source Amazon Ads: Fix unexpected column for SponsoredProductCampaigns and SponsoredBrandsKeywords #30679

lazebnyi · 2023-09-21T22:09:47Z

What

Fix due to comments after checklist review - https://docs.google.com/spreadsheets/d/12JrATeBFEvjp-H-Uxo3x5VgGDwadxFsq35HCMKlPXBg/edit#gid=1700962388

How

Add field to schema
Add pattern, format to start date and airbyte_secret to client_id
Delete inapp doc to use main one

🚨 User Impact 🚨

No

Pre-merge Actions

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Unit & integration tests added

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.

vercel · 2023-09-21T22:09:54Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	⬜️ Ignored (Inspect)	Visit Preview		Sep 22, 2023 3:24pm

github-actions · 2023-09-21T22:10:18Z

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

PR name follows PR naming conventions
Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
Secrets in the connector's spec are annotated with airbyte_secret
All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

Check for hidden checklists in your PR description
Toggle the github label checklist-action-run on/off to re-run the checklist CI.

airbyte-oss-build-runner · 2023-09-21T22:15:34Z

source-amazon-ads test report (commit `58df40edc1`) - ✅

⏲️ Total pipeline duration: 03mn10s

Step	Result
Connector package install	✅
Build source-amazon-ads docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅
Code format checks	✅
Validate airbyte-integrations/connectors/source-amazon-ads/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-amazon-ads test

airbyte-oss-build-runner · 2023-09-21T23:13:44Z

source-amazon-ads test report (commit `16070afae6`) - ✅

⏲️ Total pipeline duration: 02mn49s

Step	Result
Connector package install	✅
Build source-amazon-ads docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅
Code format checks	✅
Validate airbyte-integrations/connectors/source-amazon-ads/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-amazon-ads test

airbyte-integrations/connectors/source-amazon-ads/acceptance-test-config.yml

airbyte-integrations/connectors/source-amazon-ads/Dockerfile

davydov-d · 2023-09-22T09:04:39Z

airbyte-integrations/connectors/source-amazon-ads/unit_tests/test_source.py


    assert source.check(None, config_gen(start_date="2022-02-20")) == AirbyteConnectionStatus(status=Status.SUCCEEDED)
-    assert len(responses.calls) == 6
+    assert len(responses.calls) == 4


why did the number of calls change? I couldn't find any related changes in the code

we update case in line 67-68 and now we handle wrong pattern, so call qty reduced

davydov-d · 2023-09-22T09:05:09Z

airbyte-integrations/connectors/source-amazon-ads/unit_tests/test_source.py

-    assert command_check(source, config_gen(start_date="")) == AirbyteConnectionStatus(status=Status.SUCCEEDED)
-    assert len(responses.calls) == 4
+    with pytest.raises(Exception):
+        command_check(source, config_gen(start_date=""))


isn't start date optional?

Yes, it is optional. But we're adding it with the wrong template in that case, that's why the error rise.

airbyte-oss-build-runner · 2023-09-22T11:10:25Z

source-amazon-ads test report (commit `11a97ab423`) - ✅

⏲️ Total pipeline duration: 02mn07s

Step	Result
Connector package install	✅
Build source-amazon-ads docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅
Code format checks	✅
Validate airbyte-integrations/connectors/source-amazon-ads/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-amazon-ads test

lazebnyi · 2023-09-22T15:24:50Z

airbyte-integrations/connectors/source-amazon-ads/unit_tests/utils.py

@@ -41,6 +42,7 @@ def read_full_refresh(stream_instance: Stream):
 def command_check(source: Source, config):
    logger = mock.MagicMock()
    connector_config, _ = split_config(config)
+    connector_config = MigrateStartDate.modify_and_save("unit_tests/config.json", source, connector_config)


to test modify logic with check tests

airbyte-oss-build-runner · 2023-09-22T15:29:44Z

source-amazon-ads test report (commit `535d1b74be`) - ✅

⏲️ Total pipeline duration: 03mn19s

Step	Result
Connector package install	✅
Build source-amazon-ads docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅
Code format checks	✅
Validate airbyte-integrations/connectors/source-amazon-ads/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-amazon-ads test

…gns and SponsoredBrandsKeywords (#30679) Co-authored-by: lazebnyi <lazebnyi@users.noreply.github.com>

lazebnyi added 2 commits September 22, 2023 00:03

Schema fixed

4314a65

Bump version

58df40e

octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/amazon-ads labels Sep 21, 2023

PR number updated

65a8235

vercel bot deployed to Preview September 21, 2023 22:13 View deployment

This comment was marked as outdated.

Sign in to view

lazebnyi added 2 commits September 22, 2023 00:41

Add format and patern to spec

d2368e7

Set client id as airbyte_secret

2aed614

This comment was marked as outdated.

Sign in to view

Fix unit tests

7bdc6a9

This comment was marked as outdated.

Sign in to view

Deleted inapp doc

22b0043

vercel bot deployed to Preview September 21, 2023 23:05 View deployment

Added format, pattern and airbyte_secret to spec.json

16070af

This comment was marked as outdated.

Sign in to view

lazebnyi requested review from brianjlai and davydov-d and removed request for brianjlai September 21, 2023 23:18

davydov-d requested changes Sep 22, 2023

View reviewed changes

Bumped version

dcc7771

lazebnyi requested a review from davydov-d September 22, 2023 10:59

vercel bot deployed to Preview September 22, 2023 11:00 View deployment

Automated Commit - Formatting Changes

ad2de99

vercel bot deployed to Preview September 22, 2023 11:07 View deployment

This comment was marked as outdated.

Sign in to view

lazebnyi added 2 commits September 22, 2023 17:20

Add migration for config

db7fa52

Merge master to branch

b1f0798

octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels Sep 22, 2023

Fix typo

535d1b7

vercel bot deployed to Preview September 22, 2023 15:23 View deployment

lazebnyi commented Sep 22, 2023

View reviewed changes

davydov-d approved these changes Sep 22, 2023

View reviewed changes

lazebnyi linked an issue Sep 22, 2023 that may be closed by this pull request

[EPIC] Source Amazon Ads to Silver #28212

Closed

lazebnyi merged commit 50faaa6 into master Sep 22, 2023
29 of 31 checks passed

lazebnyi deleted the lazebnyi/source-amazon-ads-fix-schema branch September 22, 2023 17:42

jbfbell pushed a commit that referenced this pull request Sep 22, 2023

🐛 Source Amazon Ads: Fix unexpected column for SponsoredProductCampai…

97901f0

…gns and SponsoredBrandsKeywords (#30679) Co-authored-by: lazebnyi <lazebnyi@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 Source Amazon Ads: Fix unexpected column for SponsoredProductCampaigns and SponsoredBrandsKeywords #30679

🐛 Source Amazon Ads: Fix unexpected column for SponsoredProductCampaigns and SponsoredBrandsKeywords #30679

lazebnyi commented Sep 21, 2023 •

edited

vercel bot commented Sep 21, 2023 •

edited

github-actions bot commented Sep 21, 2023 •

edited by lazebnyi

airbyte-oss-build-runner commented Sep 21, 2023

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

airbyte-oss-build-runner commented Sep 21, 2023

davydov-d Sep 22, 2023

lazebnyi Sep 22, 2023

davydov-d Sep 22, 2023

lazebnyi Sep 22, 2023

airbyte-oss-build-runner commented Sep 22, 2023

This comment was marked as outdated.

lazebnyi Sep 22, 2023

airbyte-oss-build-runner commented Sep 22, 2023

🐛 Source Amazon Ads: Fix unexpected column for SponsoredProductCampaigns and SponsoredBrandsKeywords #30679

🐛 Source Amazon Ads: Fix unexpected column for SponsoredProductCampaigns and SponsoredBrandsKeywords #30679

Conversation

lazebnyi commented Sep 21, 2023 • edited

What

How

🚨 User Impact 🚨

Pre-merge Actions

Community member or Airbyter

Airbyter

vercel bot commented Sep 21, 2023 • edited

github-actions bot commented Sep 21, 2023 • edited by lazebnyi

Before Merging a Connector Pull Request

airbyte-oss-build-runner commented Sep 21, 2023

source-amazon-ads test report (commit 58df40edc1) - ✅

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

airbyte-oss-build-runner commented Sep 21, 2023

source-amazon-ads test report (commit 16070afae6) - ✅

davydov-d Sep 22, 2023

Choose a reason for hiding this comment

lazebnyi Sep 22, 2023

Choose a reason for hiding this comment

davydov-d Sep 22, 2023

Choose a reason for hiding this comment

lazebnyi Sep 22, 2023

Choose a reason for hiding this comment

airbyte-oss-build-runner commented Sep 22, 2023

source-amazon-ads test report (commit 11a97ab423) - ✅

This comment was marked as outdated.

lazebnyi Sep 22, 2023

Choose a reason for hiding this comment

airbyte-oss-build-runner commented Sep 22, 2023

source-amazon-ads test report (commit 535d1b74be) - ✅

lazebnyi commented Sep 21, 2023 •

edited

vercel bot commented Sep 21, 2023 •

edited

github-actions bot commented Sep 21, 2023 •

edited by lazebnyi

source-amazon-ads test report (commit `58df40edc1`) - ✅

source-amazon-ads test report (commit `16070afae6`) - ✅

source-amazon-ads test report (commit `11a97ab423`) - ✅

source-amazon-ads test report (commit `535d1b74be`) - ✅