Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

State bookmarks should use ids instead of repo names #149

Open
laurentS opened this issue Jun 22, 2022 · 1 comment
Open

State bookmarks should use ids instead of repo names #149

laurentS opened this issue Jun 22, 2022 · 1 comment
Labels
bug Something isn't working

Comments

@laurentS
Copy link
Contributor

Following the same logic as what is described in #110 and the linked PR, the state bookmarks should use stable IDs for their context, as opposed to the current org / repo:
If a repo is renamed between 2 runs of the tap, the later run would ignore the bookmark, as the repo name would not match anymore. While not a disaster, it would essentially force a full resync of the history of the stream, which is a waste of time/energy/api quotas.

Example of a state bookmark right now:

    {
      "context": { "org": "scipy", "repo": "scipy" },  // this should use a stable id
      "replication_key": "updated_at",
      "replication_key_value": "2022-05-06T14:29:18Z",
      "replication_key_signpost": "2022-06-22T09:07:14.047100+00:00",
      "starting_replication_value": "2022-05-06T14:29:18Z",
      "progress_markers": {
        "Note": "Progress is not resumable if interrupted.",
        "replication_key": "updated_at",
        "replication_key_value": "2022-06-22T07:55:32Z"
      }
    },
@laurentS laurentS added the bug Something isn't working label Jun 22, 2022
@laurentS
Copy link
Contributor Author

Relates to #122

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant