🎉 Source Amazon Ads: increase report_wait_timeout 30 -> 60 minutes #16453

grubberr · 2022-09-08T17:20:44Z

Signed-off-by: Sergey Chvalyuk grubberr@gmail.com

What

This PR is attempt to find more optimal default for report_wait_timeout param.
I don't want to setup default to high to allow connector to re-try if needed.

https://github.com/airbytehq/alpha-beta-issues/issues/193
https://github.com/airbytehq/alpha-beta-issues/issues/209

How

Change: 30 -> 60 minutes

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub and connector version bumped by running the /publish command described here

Tests

Unit

Put your unit tests output here.

Integration

Put your integration tests output here.

Acceptance

Put your acceptance tests output here.

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr · 2022-09-08T17:38:21Z

/test connector=connectors/source-amazon-ads

🕑 connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3017115986
❌ connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3017115986
🐛 https://gradle.com/s/x7gtuyxtfl25o

Build Failed

Test summary info:

Could not find result summary

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr · 2022-09-08T19:08:00Z

/test connector=connectors/source-amazon-ads

🕑 connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3017631381
❌ connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3017631381
🐛 https://gradle.com/s/ylgp722fznbte

Build Failed

Test summary info:

Could not find result summary

grubberr · 2022-09-08T20:01:08Z

/test connector=connectors/source-amazon-ads

🕑 connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3017925144
❌ connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3017925144
🐛 https://gradle.com/s/yj6lg4vhsnudy

Build Failed

Test summary info:

Could not find result summary

grubberr · 2022-09-08T21:48:59Z

/test connector=connectors/source-amazon-ads

🕑 connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3018495346
❌ connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3018495346
🐛 https://gradle.com/s/ehay26ugkbeim

Build Failed

Test summary info:

Could not find result summary

grubberr · 2022-09-09T05:32:13Z

/test connector=connectors/source-amazon-ads

🕑 connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3020186407
❌ connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3020186407
🐛 https://gradle.com/s/fl3wsvbrbonum

Build Failed

Test summary info:

Could not find result summary

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr · 2022-09-09T11:59:45Z

/test connector=connectors/source-amazon-ads

🕑 connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3022241104
❌ connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3022241104
🐛 https://gradle.com/s/4yj66zerssbrg

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestSpec::test_match_expected[inputs0] - AssertionError:...
================== 1 failed, 31 passed in 5150.20s (1:25:50) ===================

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr · 2022-09-09T13:51:04Z

/test connector=connectors/source-amazon-ads

🕑 connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3022902376
❌ connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3022902376
🐛 https://gradle.com/s/7y3d7yn4v2oou

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_incremental.py::TestIncremental::test_two_sequential_reads[inputs0]
================== 1 failed, 31 passed in 3108.67s (0:51:48) ===================

grubberr · 2022-09-09T17:14:14Z

/test connector=connectors/source-amazon-ads

🕑 connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3024148392
✅ connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3024148392
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
source_amazon_ads/utils.py                                            9      0   100%
source_amazon_ads/streams/sponsored_products.py                      32      0   100%
source_amazon_ads/streams/sponsored_display.py                       22      0   100%
source_amazon_ads/streams/sponsored_brands.py                        17      0   100%
source_amazon_ads/streams/report_streams/products_report.py          18      0   100%
source_amazon_ads/streams/report_streams/display_report.py           16      0   100%
source_amazon_ads/streams/report_streams/brands_video_report.py      10      0   100%
source_amazon_ads/streams/report_streams/brands_report.py            10      0   100%
source_amazon_ads/streams/report_streams/__init__.py                  5      0   100%
source_amazon_ads/streams/profiles.py                                21      0   100%
source_amazon_ads/streams/__init__.py                                 6      0   100%
source_amazon_ads/schemas/sponsored_products.py                      37      0   100%
source_amazon_ads/schemas/sponsored_display.py                       31      0   100%
source_amazon_ads/schemas/sponsored_brands.py                        22      0   100%
source_amazon_ads/schemas/profile.py                                 16      0   100%
source_amazon_ads/schemas/__init__.py                                 6      0   100%
source_amazon_ads/constants.py                                        1      0   100%
source_amazon_ads/__init__.py                                         2      0   100%
source_amazon_ads/streams/common.py                                  76      1    99%
source_amazon_ads/schemas/common.py                                  51      1    98%
source_amazon_ads/source.py                                          44      1    98%
source_amazon_ads/streams/report_streams/report_streams.py          237     18    92%
-------------------------------------------------------------------------------------
TOTAL                                                               689     21    97%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          10      4    60%   15-18
	 source_acceptance_test/config.py                        83      6    93%   78-80, 84-86
	 source_acceptance_test/conftest.py                     164    164     0%   6-282
	 source_acceptance_test/plugin.py                        48     48     0%   6-104
	 source_acceptance_test/tests/test_core.py              329    111    66%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 374-376, 379, 439-448, 477-478, 484, 487, 520-530, 543-568, 573-577
	 source_acceptance_test/tests/test_full_refresh.py       52      2    96%   34, 65
	 source_acceptance_test/tests/test_incremental.py       121     25    79%   21-23, 29-31, 36-43, 48-61, 208-216
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     17    78%   15-16, 24-30, 47-54, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       110     48    56%   23-26, 32, 36, 39-64, 67-69, 72-74, 77-79, 82-84, 87-89, 92-110, 144-146
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1322    463    65%

Build Passed

Test summary info:

All Passed

grubberr · 2022-09-10T08:28:35Z

/publish connector=connectors/source-amazon-ads

🕑 Publishing the following connectors:
connectors/source-amazon-ads
https://github.com/airbytehq/airbyte/actions/runs/3027310338

Connector	Did it publish?	Were definitions generated?
connectors/source-amazon-ads	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

…irbytehq#16453) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

increse report_wait_timeout 30 -> 60 minutes

2729fe4

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

github-actions bot added the area/connectors Connector related issues label Sep 8, 2022

octavia-squidington-iii added the connectors/source/amazon-ads label Sep 8, 2022

30, 60 -> 45

7a8e3dc

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

github-actions bot added the area/documentation Improvements or additions to documentation label Sep 8, 2022

grubberr changed the title ~~🎉 Source Amazon Ads: increase report_wait_timeout 30 -> 60 minutes~~ 🎉 Source Amazon Ads: increase report_wait_timeout 30 -> 45 minutes Sep 8, 2022

grubberr self-assigned this Sep 8, 2022

grubberr added 3 commits September 8, 2022 22:05

Merge branch 'master' into grubberr/alpha-beta-209-source-amazon-ads

b1ccbc3

45 -> 60

5350bf0

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

amazon-ads.md updated

e445ea1

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr changed the title ~~🎉 Source Amazon Ads: increase report_wait_timeout 30 -> 45 minutes~~ 🎉 Source Amazon Ads: increase report_wait_timeout 30 -> 60 minutes Sep 8, 2022

grubberr added 2 commits September 9, 2022 14:57

Merge branch 'master' into grubberr/alpha-beta-209-source-amazon-ads

19db280

fix unit_tests

eeafc60

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

30 -> 60

8eb91bb

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr requested a review from arsenlosenko September 9, 2022 22:49

lazebnyi approved these changes Sep 10, 2022

View reviewed changes

arsenlosenko approved these changes Sep 10, 2022

View reviewed changes

auto-bump connector version [ci skip]

2fb3037

grubberr merged commit 0ab777f into master Sep 10, 2022

grubberr deleted the grubberr/alpha-beta-209-source-amazon-ads branch September 10, 2022 10:48

octavia-squidington-iii mentioned this pull request Sep 12, 2022

Bump Airbyte version from 0.40.4 to 0.40.5 #16610

Merged

robbinhan pushed a commit to robbinhan/airbyte that referenced this pull request Sep 29, 2022

🎉 Source Amazon Ads: increase report_wait_timeout 30 -> 60 minutes (a…

06600f6

…irbytehq#16453) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

jhammarstedt pushed a commit to jhammarstedt/airbyte that referenced this pull request Oct 31, 2022

🎉 Source Amazon Ads: increase report_wait_timeout 30 -> 60 minutes (a…

adafa36

…irbytehq#16453) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 Source Amazon Ads: increase report_wait_timeout 30 -> 60 minutes #16453

🎉 Source Amazon Ads: increase report_wait_timeout 30 -> 60 minutes #16453

grubberr commented Sep 8, 2022 •

edited

Loading

grubberr commented Sep 8, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 8, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 8, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 8, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 9, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 9, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 9, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 9, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 10, 2022 •

edited by github-actions bot

Loading

🎉 Source Amazon Ads: increase report_wait_timeout 30 -> 60 minutes #16453

🎉 Source Amazon Ads: increase report_wait_timeout 30 -> 60 minutes #16453

Conversation

grubberr commented Sep 8, 2022 • edited Loading

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Airbyter

Tests

grubberr commented Sep 8, 2022 • edited by github-actions bot Loading

Build Failed

grubberr commented Sep 8, 2022 • edited by github-actions bot Loading

Build Failed

grubberr commented Sep 8, 2022 • edited by github-actions bot Loading

Build Failed

grubberr commented Sep 8, 2022 • edited by github-actions bot Loading

Build Failed

grubberr commented Sep 9, 2022 • edited by github-actions bot Loading

Build Failed

grubberr commented Sep 9, 2022 • edited by github-actions bot Loading

Build Failed

grubberr commented Sep 9, 2022 • edited by github-actions bot Loading

Build Failed

grubberr commented Sep 9, 2022 • edited by github-actions bot Loading

Build Passed

grubberr commented Sep 10, 2022 • edited by github-actions bot Loading

grubberr commented Sep 8, 2022 •

edited

Loading

grubberr commented Sep 8, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 8, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 8, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 8, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 9, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 9, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 9, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 9, 2022 •

edited by github-actions bot

Loading

grubberr commented Sep 10, 2022 •

edited by github-actions bot

Loading