🚨 🚨 Source Amazon Ads: fix schemas (airbytehq#27868)

* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* airbytehq#1955 source amazon ads: fix stream schemas

* source amazon ads: upd changelog

* source amazon ads: upd CAT config

* upd backwards incompatible version

* update metadata with breakingChanges entry

* fix CATs

* fix md

airbyte-integrations/connectors/source-amazon-ads/Dockerfile

@@ -13,5 +13,5 @@ ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
 ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
 
 
-LABEL io.airbyte.version=2.3.1
+LABEL io.airbyte.version=3.0.0
 LABEL io.airbyte.name=airbyte/source-amazon-ads

airbyte-integrations/connectors/source-amazon-ads/acceptance-test-config.yml

@@ -50,7 +50,7 @@ acceptance_tests:
     tests:
       - config_path: secrets/config.json
         backward_compatibility_tests_config:
-          disable_for_version: 2.1.0
+          disable_for_version: 2.3.1
   full_refresh:
     tests:
       - config_path: secrets/config.json

airbyte-integrations/connectors/source-amazon-ads/metadata.yaml

@@ -8,7 +8,7 @@ data:
   connectorSubtype: api
   connectorType: source
   definitionId: c6b0a29e-1da9-4512-9002-7bfd0cba2246
-  dockerImageTag: 2.3.1
+  dockerImageTag: 3.0.0
   dockerRepository: airbyte/source-amazon-ads
   githubIssueLabel: source-amazon-ads
   icon: amazonads.svg
@@ -23,4 +23,9 @@ data:
   documentationUrl: https://docs.airbyte.com/integrations/sources/amazon-ads
   tags:
     - language:python
+  releases:
+    breakingChanges:
+      3.0.0:
+        message: "Attribution report stream schemas fix."
+        upgradeDeadline: "2023-07-24"
 metadataSpecVersion: "1.0"

airbyte-integrations/connectors/source-amazon-ads/source_amazon_ads/schemas/attribution_report.py

@@ -2,12 +2,10 @@
 # Copyright (c) 2023 Airbyte, Inc., all rights reserved.
 #
 
-from typing import List
-
 from .common import CatalogModel
 
 
-class Report(CatalogModel):
+class AttributionReportModel(CatalogModel):
     date: str
     brandName: str
     marketplace: str
@@ -22,9 +20,3 @@ class Report(CatalogModel):
     productSubcategory: str
     productGroup: str
     publisher: str
-
-
-class AttributionReportModel(CatalogModel):
-    reports: List[Report]
-    size: int
-    cursorId: str

airbyte-integrations/connectors/source-amazon-ads/source_amazon_ads/streams/attribution_report.py

@@ -60,7 +60,7 @@ class AttributionReport(AmazonAdsStream):
     page_size = 300
 
     report_type = ""
-    metrics = ""
+    custom_metrics = []
     group_by = ""
 
     _next_page_token_field = "cursorId"
@@ -74,13 +74,23 @@ def __init__(self, config: Mapping[str, Any], *args, **kwargs):
         self._start_date = config.get("start_date")
         super().__init__(config, *args, **kwargs)
 
+    @property
+    def metrics(self):
+        return METRICS_MAP[self.report_type] + self.custom_metrics
+
     @property
     def http_method(self) -> str:
         return "POST"
 
     def path(self, **kvargs) -> str:
         return "/attribution/report"
 
+    def get_json_schema(self):
+        schema = super().get_json_schema()
+        metrics_type_map = {metric: {"type": ["null", "string"]} for metric in self.metrics}
+        schema["properties"].update(metrics_type_map)
+        return schema
+
     def stream_slices(
         self, sync_mode: SyncMode, cursor_field: List[str] = None, stream_state: Mapping[str, Any] = None
     ) -> Iterable[Optional[Mapping[str, Any]]]:
@@ -135,7 +145,7 @@ def request_body_json(
         body = {
             "reportType": self.report_type,
             "count": self.page_size,
-            "metrics": self.metrics,
+            "metrics": ",".join(self.metrics),
             "startDate": stream_slice["startDate"],
             "endDate": stream_slice["endDate"],
             self._next_page_token_field: "",
@@ -152,35 +162,21 @@ def request_body_json(
 
 class AttributionReportProducts(AttributionReport):
     report_type = "PRODUCTS"
-
-    metrics = ",".join(METRICS_MAP[report_type])
-
     group_by = ""
 
 
 class AttributionReportPerformanceCreative(AttributionReport):
     report_type = "PERFORMANCE"
-
-    metrics = ",".join(METRICS_MAP[report_type])
-
     group_by = "CREATIVE"
 
 
 class AttributionReportPerformanceAdgroup(AttributionReport):
     report_type = "PERFORMANCE"
-
-    metrics_list = METRICS_MAP[report_type]
-    metrics_list.append(BRAND_REFERRAL_BONUS)
-    metrics = ",".join(metrics_list)
-
+    custom_metrics = [BRAND_REFERRAL_BONUS]
     group_by = "ADGROUP"
 
 
 class AttributionReportPerformanceCampaign(AttributionReport):
     report_type = "PERFORMANCE"
-
-    metrics_list = METRICS_MAP[report_type]
-    metrics_list.append(BRAND_REFERRAL_BONUS)
-    metrics = ",".join(metrics_list)
-
+    custom_metrics = [BRAND_REFERRAL_BONUS]
     group_by = "CAMPAIGN"

airbyte-integrations/connectors/source-amazon-ads/unit_tests/test_attribution_report.py

@@ -70,6 +70,7 @@ def test_attribution_report_schema(config, profiles_response, attribution_report
     profile_stream = get_stream_by_name(streams, "profiles")
     attribution_report_stream = get_stream_by_name(streams, stream_name)
     schema = attribution_report_stream.get_json_schema()
+    schema["additionalProperties"] = False
 
     profile_records = list(read_full_refresh(profile_stream))
     attribution_records = list(read_full_refresh(attribution_report_stream))

docs/integrations/sources/amazon-ads-migrations.md

@@ -0,0 +1,6 @@
+# Amazon Ads Migration Guide
+
+## Upgrading to 3.0.0
+
+A major update of attribution report stream schemas.
+For a smooth migration, a data reset and a schema refresh are needed.

docs/integrations/sources/amazon-ads.md

@@ -97,8 +97,10 @@ Information about expected report generation waiting time you may find [here](ht
 
 ## CHANGELOG
 
+
 | Version | Date       | Pull Request                                             | Subject                                                                                                         |
 |:--------|:-----------|:---------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------|
+| 3.0.0   | 2023-07-24 | [27868](https://github.com/airbytehq/airbyte/pull/27868) | Fix attribution report stream schemas                                                                           |
 | 2.3.1   | 2023-07-11 | [28155](https://github.com/airbytehq/airbyte/pull/28155) | Bugfix: validation error when record values are missing                                                         |
 | 2.3.0   | 2023-07-06 | [28002](https://github.com/airbytehq/airbyte/pull/28002) | Add sponsored_product_ad_group_suggested_keywords, sponsored_product_ad_group_bid_recommendations streams       |
 | 2.2.0   | 2023-07-05 | [27607](https://github.com/airbytehq/airbyte/pull/27607) | Add stream for sponsored brands v3 purchased product reports                                                    |