Skip to content

Commit

Permalink
Source Recharge: Use TypeTransformer to convert integer values in Ord…
Browse files Browse the repository at this point in the history
…ers stream (#16592)

* Use TypeTransformer to convert integer values in Orders stream

* Update changelog

* Leave just the declaration of transformer
  • Loading branch information
arsenlosenko committed Sep 12, 2022
1 parent cc965b3 commit 0f26426
Show file tree
Hide file tree
Showing 3 changed files with 4 additions and 14 deletions.
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-recharge/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ RUN pip install .
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.8
LABEL io.airbyte.version=0.1.9
LABEL io.airbyte.name=airbyte/source-recharge
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import pendulum
import requests
from airbyte_cdk.sources.streams.http import HttpStream
from airbyte_cdk.sources.utils.transform import TransformConfig, TypeTransformer


class RechargeStream(HttpStream, ABC):
Expand Down Expand Up @@ -163,19 +164,7 @@ class Orders(IncrementalRechargeStream):
Orders Stream: https://developer.rechargepayments.com/v1-shopify?python#list-orders
"""

def get_stream_data(self, response_data: Any) -> List[dict]:
# We expect total_weight to be an integer, but the API is returning numbers like 42.0
# Cast these down to int if possible, and error if not.
data = super().get_stream_data(response_data)
for record in data:
if "total_weight" in record:
total_weight = record["total_weight"]
int_total_weight = int(total_weight)
if total_weight == int_total_weight:
record["total_weight"] = int_total_weight
else:
raise ValueError(f"Expected total_weight to be an integer, got {total_weight}")
return data
transformer: TypeTransformer = TypeTransformer(TransformConfig.DefaultSchemaNormalization)


class Products(RechargeStream):
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/recharge.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ The Recharge connector should gracefully handle Recharge API limitations under n

| Version | Date | Pull Request | Subject |
| :--- | :--- | :--- | :--- |
| 0.1.9 | 2022-09-12 | [16592](https://github.com/airbytehq/airbyte/pull/16592) | Use TypeTransformer to reliably convert to int values in Orders stream
| 0.1.8 | 2022-08-27 | [16045](https://github.com/airbytehq/airbyte/pull/16045) | Force total_weight to be an integer |
| 0.1.7 | 2022-07-24 | [14978](https://github.com/airbytehq/airbyte/pull/14978) | Set `additionalProperties` to True, to guarantee backward cababilities |
| 0.1.6 | 2022-07-21 | [14902](https://github.com/airbytehq/airbyte/pull/14902) | Increased test coverage, fixed broken `charges`, `orders` schemas, added state checkpoint |
Expand Down

0 comments on commit 0f26426

Please sign in to comment.