Source Amazon Ads: remove "report_wait_timeout", "report_generation_max_retries" from config #18985

grubberr · 2022-11-04T20:28:44Z

Signed-off-by: Sergey Chvalyuk grubberr@gmail.com

What

Remove "report_wait_timeout", "report_generation_max_retries" - there is not reason to bother customer with this internal settings
get REPORT_WAIT_TIMEOUT=180, REPORT_GENERATION_MAX_RETRIES=5 from environment if variable are not defined get default values.
Change "get_error_display_message" message

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub and connector version bumped by running the /publish command described here

Tests

Unit

Put your unit tests output here.

Integration

Put your integration tests output here.

Acceptance

Put your acceptance tests output here.

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

grubberr · 2022-11-08T07:07:29Z

/test connector=connectors/source-amazon-ads

🕑 connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3417151744
❌ connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3417151744
🐛 https://gradle.com/s/cpriyzfs62p3k

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_incremental.py::TestIncremental::test_read_sequential_slices[inputs0]
============ 1 failed, 31 passed, 29 warnings in 4031.47s (1:07:11) ============

grubberr · 2022-11-08T08:45:59Z

/test connector=connectors/source-amazon-ads

🕑 connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3417763517
❌ connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3417763517
🐛 https://gradle.com/s/qwq3hmfit5j7u

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_incremental.py::TestIncremental::test_two_sequential_reads[inputs0]
============ 1 failed, 31 passed, 29 warnings in 5521.17s (1:32:01) ============

grubberr · 2022-11-08T10:53:42Z

/test connector=connectors/source-amazon-ads

🕑 connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3418720206
✅ connectors/source-amazon-ads https://github.com/airbytehq/airbyte/actions/runs/3418720206
Python tests coverage:

Name                                                              Stmts   Miss  Cover
-------------------------------------------------------------------------------------
source_amazon_ads/utils.py                                           22      0   100%
source_amazon_ads/streams/sponsored_products.py                      41      0   100%
source_amazon_ads/streams/sponsored_display.py                       31      0   100%
source_amazon_ads/streams/sponsored_brands.py                        26      0   100%
source_amazon_ads/streams/report_streams/products_report.py          18      0   100%
source_amazon_ads/streams/report_streams/display_report.py           16      0   100%
source_amazon_ads/streams/report_streams/brands_video_report.py      10      0   100%
source_amazon_ads/streams/report_streams/brands_report.py            10      0   100%
source_amazon_ads/streams/report_streams/__init__.py                  5      0   100%
source_amazon_ads/streams/profiles.py                                21      0   100%
source_amazon_ads/streams/attribution_report.py                      81      0   100%
source_amazon_ads/streams/__init__.py                                 7      0   100%
source_amazon_ads/schemas/sponsored_products.py                      37      0   100%
source_amazon_ads/schemas/sponsored_display.py                       31      0   100%
source_amazon_ads/schemas/sponsored_brands.py                        22      0   100%
source_amazon_ads/schemas/profile.py                                 16      0   100%
source_amazon_ads/schemas/attribution_report.py                      21      0   100%
source_amazon_ads/schemas/__init__.py                                 7      0   100%
source_amazon_ads/constants.py                                        1      0   100%
source_amazon_ads/__init__.py                                         2      0   100%
source_amazon_ads/streams/common.py                                  79      1    99%
source_amazon_ads/schemas/common.py                                  51      1    98%
source_amazon_ads/source.py                                          44      1    98%
source_amazon_ads/streams/report_streams/report_streams.py          240     19    92%
-------------------------------------------------------------------------------------
TOTAL                                                               839     22    97%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       133      3    98%   87, 93, 230
	 source_acceptance_test/conftest.py                     196     92    53%   35, 41-43, 48, 54, 60, 66, 72-74, 93, 98-100, 106-108, 114-115, 120-121, 126, 132, 141-150, 156-161, 176, 200, 231, 237, 243-248, 256-261, 269-282, 287-293, 300-311, 318-334
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              345    110    68%   53, 64-72, 77-84, 88-89, 93-94, 178, 216-233, 242-250, 254-259, 265, 298-303, 341-348, 391-393, 396, 461-469, 481-484, 489, 545-546, 552, 555, 591-601, 614-639
	 source_acceptance_test/tests/test_incremental.py       145     20    86%   21-23, 29-31, 36-43, 48-61, 224
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/config_migration.py        23     23     0%   5-37
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-68, 71-73, 76-78, 81-83, 86-88, 91-93, 96-114, 148-150
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1512    375    75%

Build Passed

Test summary info:

All Passed

grubberr · 2022-11-08T16:58:25Z

/publish connector=connectors/source-amazon-ads

🕑 Publishing the following connectors:
connectors/source-amazon-ads
https://github.com/airbytehq/airbyte/actions/runs/3421513822

Connector	Did it publish?	Were definitions generated?

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

grubberr · 2022-11-08T20:57:50Z

/publish connector=connectors/source-amazon-ads run-tests=false

🕑 Publishing the following connectors:
connectors/source-amazon-ads
https://github.com/airbytehq/airbyte/actions/runs/3423133027

Connector	Did it publish?	Were definitions generated?
connectors/source-amazon-ads	❌	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

grubberr · 2022-11-08T21:24:38Z

/publish connector=connectors/source-amazon-ads run-tests=false

🕑 Publishing the following connectors:
connectors/source-amazon-ads
https://github.com/airbytehq/airbyte/actions/runs/3423314898

Connector	Did it publish?	Were definitions generated?
connectors/source-amazon-ads	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

…ax_retries" from config (#18985) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

get_typed_env added

a6313ec

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

github-actions bot added the area/connectors Connector related issues label Nov 4, 2022

octavia-squidington-iv added the connectors/source/amazon-ads label Nov 4, 2022

grubberr self-assigned this Nov 4, 2022

grubberr linked an issue Nov 4, 2022 that may be closed by this pull request

Source Amazon Ads: Deprecate report_wait_timeout and report_generation_max_retries fields #18948

Closed

grubberr added 6 commits November 8, 2022 07:59

Merge branch 'master' into grubberr/18948-source-amazon-ads

9224de5

REPORT_WAIT_TIMEOUT 60 -> 180

5d27ce8

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

test_get_typed_env added

ec55a43

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

get_error_display_message updated

a52a4ed

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

bump 0.1.25

72d87c1

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

amazon-ads.md updated

9d4b513

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

github-actions bot added the area/documentation Improvements or additions to documentation label Nov 8, 2022

grubberr requested review from darynaishchenko and girarda November 8, 2022 11:53

girarda approved these changes Nov 8, 2022

View reviewed changes

Merge branch 'master' into grubberr/18948-source-amazon-ads

794ca99

auto-bump connector version

a4f7954

octavia-squidington-iii temporarily deployed to more-secrets November 8, 2022 21:43 Inactive

grubberr merged commit 7063563 into master Nov 8, 2022

grubberr deleted the grubberr/18948-source-amazon-ads branch November 8, 2022 22:11

grubberr mentioned this pull request Nov 9, 2022

Source Amazon Ads: remove "report_wait_timeout", "report_generation_max_retries" - Documentation fixed #19202

Merged

github-actions bot mentioned this pull request Nov 15, 2022

Bump helm chart version reference to 0.40.43 #19453

Closed

octavia-squidington-iii mentioned this pull request Nov 18, 2022

Bump Airbyte version from 0.40.18 to 0.40.19 #19579

Merged

akashkulk pushed a commit that referenced this pull request Dec 2, 2022

Source Amazon Ads: remove "report_wait_timeout", "report_generation_m…

3d2bf06

…ax_retries" from config (#18985) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Source Amazon Ads: remove "report_wait_timeout", "report_generation_max_retries" from config #18985

Source Amazon Ads: remove "report_wait_timeout", "report_generation_max_retries" from config #18985

grubberr commented Nov 4, 2022 •

edited

grubberr commented Nov 8, 2022 •

edited by github-actions bot

grubberr commented Nov 8, 2022 •

edited by github-actions bot

grubberr commented Nov 8, 2022 •

edited by github-actions bot

grubberr commented Nov 8, 2022 •

edited by github-actions bot

grubberr commented Nov 8, 2022 •

edited by github-actions bot

grubberr commented Nov 8, 2022 •

edited by github-actions bot

Source Amazon Ads: remove "report_wait_timeout", "report_generation_max_retries" from config #18985

Source Amazon Ads: remove "report_wait_timeout", "report_generation_max_retries" from config #18985

Conversation

grubberr commented Nov 4, 2022 • edited

What

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Airbyter

Tests

grubberr commented Nov 8, 2022 • edited by github-actions bot

Build Failed

grubberr commented Nov 8, 2022 • edited by github-actions bot

Build Failed

grubberr commented Nov 8, 2022 • edited by github-actions bot

Build Passed

grubberr commented Nov 8, 2022 • edited by github-actions bot

grubberr commented Nov 8, 2022 • edited by github-actions bot

grubberr commented Nov 8, 2022 • edited by github-actions bot

grubberr commented Nov 4, 2022 •

edited

grubberr commented Nov 8, 2022 •

edited by github-actions bot

grubberr commented Nov 8, 2022 •

edited by github-actions bot

grubberr commented Nov 8, 2022 •

edited by github-actions bot

grubberr commented Nov 8, 2022 •

edited by github-actions bot

grubberr commented Nov 8, 2022 •

edited by github-actions bot

grubberr commented Nov 8, 2022 •

edited by github-actions bot