fix: Make incremental replication opt-in for streams that support it (#…

…175)

Co-authored-by: Adam Roderick <aroder@gmail.com>
Co-authored-by: Adam Roderick <23650+aroder@users.noreply.github.com>

README.md

@@ -40,6 +40,39 @@ Singer Tap for Jotform. Built with the [Meltano Singer SDK](https://sdk.meltano.
 
 A full list of supported settings and capabilities is available by running: `tap-jotform --about`
 
+## Streams
+
+| Stream name | API endpoint      | API docs                                       | Notes |
+| :---------- | :---------------- | :--------------------------------------------- | :---- |
+| forms       | /user/forms       | https://api.jotform.com/docs/#user-forms       | Replication for this stream is opt-in. See instructions [below](#configuring-incremental-replication). |
+| questions   | /form/{form_id}/questions | https://api.jotform.com/docs/#form-id-questions | |
+| submissions | /user/submissions | https://api.jotform.com/docs/#user-submissions | Replication for this stream is opt-in. See instructions [below](#configuring-incremental-replication).  |
+| reports     | /user/reports     | https://api.jotform.com/docs/#user-reports | |
+| user_history | /user/history    | https://api.jotform.com/docs/#user-history | |
+
+
+### Configuring incremental replication
+
+By default, the `forms` and `submissions` stream are synced with `FULL_TABLE` replication. Incremental replication can be enabled by setting the replication metadata in the stream's entry in the catalog file:
+
+* `replication_method`: set to`INCREMENTAL`
+* `replication_key` set to `created_at` or `updated_at`. The former will omit updated submissions, while the latter will omit new submissions.
+
+For example, to enable incremental replication for the `submissions` stream:
+
+```json
+{
+  "streams": [
+    {
+      "tap_stream_id": "submissions",
+      "stream": "submissions",
+      "replication_method": "INCREMENTAL",
+      "replication_key": "updated_at",
+    }
+  ]
+}
+```
+
 ### Source Authentication and Authorization
 
 To generate an API key, follow the instructions in https://api.jotform.com/docs/#gettingstarted.

tap_jotform/client.py

@@ -133,8 +133,6 @@ def requests_session(self) -> requests_cache.CachedSession | requests.Session:
 class JotformPaginatedStream(JotformStream):
     """A Jotform stream with pagination."""
 
-    replication_key = "updated_at"
-
     def get_new_paginator(self) -> JotformPaginator:
         """Return a new instance of a paginator.
 
@@ -160,7 +158,7 @@ def get_url_params(
         params: dict[str, t.Any] = {"limit": self.page_size}
 
         starting_value = self.get_starting_timestamp(context)
-        if starting_value:
+        if starting_value and self.replication_key:
             self.logger.info(
                 "Bookmark found %(bookmark)s",
                 extra={"bookmark": starting_value},