🐛 CDK: fix number of records mismatch (#5767)

* CDK: fix number of records mismatch Increment `record_counter` after we check whether we reached the limit or not. * Update docs. Bump version to 0.1.17
airbytehq · Sep 2, 2021 · 58c9b84 · 58c9b84
1 parent 95171b8
commit 58c9b84
Show file tree

Hide file tree

Showing 3 changed files with 7 additions and 4 deletions.
diff --git a/airbyte-cdk/python/CHANGELOG.md b/airbyte-cdk/python/CHANGELOG.md
@@ -1,7 +1,10 @@
 # Changelog
 
+## 0.1.17
+Fix mismatching between number of records actually read and number of records in logs by 1: https://github.com/airbytehq/airbyte/pull/5767
+
 ## 0.1.16
-Update generated AirbyteProtocol models to contain [Oauth changes](https://github.com/airbytehq/airbyte/pull/5776). 
+Update generated AirbyteProtocol models to contain [Oauth changes](https://github.com/airbytehq/airbyte/pull/5776).
 
 ## 0.1.15
 Add \_limit and \_page_size as internal config parameters for SAT

diff --git a/airbyte-cdk/python/airbyte_cdk/sources/abstract_source.py b/airbyte-cdk/python/airbyte_cdk/sources/abstract_source.py
@@ -146,10 +146,10 @@ def _read_stream(
         logger.info(f"Syncing stream: {stream_name} ")
         for record in record_iterator:
             if record.type == MessageType.RECORD:
-                record_counter += 1
-                if internal_config.limit and record_counter > internal_config.limit:
+                if internal_config.limit and record_counter >= internal_config.limit:
                     logger.info(f"Reached limit defined by internal config ({internal_config.limit}), stop reading")
                     break
+                record_counter += 1
             yield record
 
         logger.info(f"Read {record_counter} records from {stream_name} stream")

diff --git a/airbyte-cdk/python/setup.py b/airbyte-cdk/python/setup.py
@@ -35,7 +35,7 @@
 
 setup(
     name="airbyte-cdk",
-    version="0.1.16",
+    version="0.1.17",
     description="A framework for writing Airbyte Connectors.",
     long_description=README,
     long_description_content_type="text/markdown",