Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source Amazon Ads: fix bug with handling: "Report date is too far in the past." (partial revert of #20662) #21082

Merged
merged 14 commits into from
Jan 6, 2023
Merged
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
- name: Amazon Ads
sourceDefinitionId: c6b0a29e-1da9-4512-9002-7bfd0cba2246
dockerRepository: airbyte/source-amazon-ads
dockerImageTag: 0.1.26
dockerImageTag: 0.1.27
documentationUrl: https://docs.airbyte.com/integrations/sources/amazon-ads
icon: amazonads.svg
sourceType: api
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -719,7 +719,7 @@
supportsNormalization: false
supportsDBT: false
supported_destination_sync_modes: []
- dockerImage: "airbyte/source-amazon-ads:0.1.26"
- dockerImage: "airbyte/source-amazon-ads:0.1.27"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/sources/amazon-ads"
connectionSpecification:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,5 @@ ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]


LABEL io.airbyte.version=0.1.26
LABEL io.airbyte.version=0.1.27
LABEL io.airbyte.name=airbyte/source-amazon-ads
Original file line number Diff line number Diff line change
Expand Up @@ -36,26 +36,17 @@
SponsoredProductsReportStream,
SponsoredProductTargetings,
)
from .streams.report_streams.report_streams import ReportStream

# Oauth 2.0 authentication URL for amazon
TOKEN_URL = "https://api.amazon.com/auth/o2/token"
CONFIG_DATE_FORMAT = "YYYY-MM-DD"


class SourceAmazonAds(AbstractSource):
def _validate_and_transform(self, config: Mapping[str, Any], check=False):
def _validate_and_transform(self, config: Mapping[str, Any]):
start_date = config.get("start_date")
if start_date:
config["start_date"] = pendulum.from_format(start_date, CONFIG_DATE_FORMAT).date()
if check:
# This validation is used only for improving the user UX experience.
# The connector can work without this low boundary checking.
# We use REPORTING_PERIOD plus ONE additional day because
# amazon account can have multiple profiles in different time zones.
min_date = pendulum.today().date().subtract(days=ReportStream.REPORTING_PERIOD + 1)
if config["start_date"] < min_date:
raise Exception(f"Start Date: minimum allowed value is {min_date}")
else:
config["start_date"] = None
if not config.get("region"):
Expand All @@ -70,7 +61,7 @@ def check_connection(self, logger: logging.Logger, config: Mapping[str, Any]) ->
:return Tuple[bool, any]: (True, None) if the input config can be used to connect to the API successfully, (False, error) otherwise.
"""
try:
config = self._validate_and_transform(config, check=True)
config = self._validate_and_transform(config)
except Exception as e:
return False, str(e)
# Check connection by sending list of profiles request. Its most simple
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,8 @@
# Copyright (c) 2022 Airbyte, Inc., all rights reserved.
#

import pendulum
import responses
from airbyte_cdk.models import AirbyteConnectionStatus, AirbyteMessage, ConnectorSpecification, Status, Type
from freezegun import freeze_time
from jsonschema import Draft4Validator
from source_amazon_ads import SourceAmazonAds

Expand Down Expand Up @@ -43,7 +41,6 @@ def test_spec():


@responses.activate
@freeze_time("2022-12-20 10:00:00")
def test_check(config_gen):
setup_responses()
source = SourceAmazonAds()
Expand All @@ -54,13 +51,7 @@ def test_check(config_gen):
assert command_check(source, config_gen(start_date="")) == AirbyteConnectionStatus(status=Status.SUCCEEDED)
assert len(responses.calls) == 4

assert source.check(None, config_gen(start_date="1900-01-01")) == AirbyteConnectionStatus(
status=Status.FAILED, message="'Start Date: minimum allowed value is 2022-10-20'"
)
assert len(responses.calls) == 4

start_date = pendulum.today().format("YYYY-MM-DD")
assert source.check(None, config_gen(start_date=start_date)) == AirbyteConnectionStatus(status=Status.SUCCEEDED)
assert source.check(None, config_gen(start_date="2022-02-20")) == AirbyteConnectionStatus(status=Status.SUCCEEDED)
assert len(responses.calls) == 6

assert command_check(source, config_gen(start_date="2022-20-02")) == AirbyteConnectionStatus(
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/amazon-ads.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ Information about expected report generation waiting time you may find [here](ht

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------|
| 0.1.27 | 2023-01-05 | [21082](https://github.com/airbytehq/airbyte/pull/21082) | Fix bug with handling: "Report date is too far in the past." - partial revert of #20662 |
| 0.1.26 | 2022-12-19 | [20662](https://github.com/airbytehq/airbyte/pull/20662) | Fix bug with handling: "Report date is too far in the past." |
| 0.1.25 | 2022-11-08 | [18985](https://github.com/airbytehq/airbyte/pull/18985) | Remove "report_wait_timeout", "report_generation_max_retries" from config |
| 0.1.24 | 2022-10-19 | [17475](https://github.com/airbytehq/airbyte/pull/17475) | Add filters for state on brand, product and display campaigns |
Expand Down