Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start sync: don't wait for schedule #1036

Closed
bastienbeurier opened this issue Sep 20, 2023 · 1 comment · Fixed by #1055
Closed

Start sync: don't wait for schedule #1036

bastienbeurier opened this issue Sep 20, 2023 · 1 comment · Fixed by #1055
Assignees
Labels
enhancement New feature or request

Comments

@bastienbeurier
Copy link
Member

We've recently introduced the feature to pause and resume syncs.

This feature proves handy when setting metadata for a sync:

  1. Initiate OAuth (with auto_start: false)
  2. Input metadata
  3. Begin the sync

However, a challenge arises in this flow. Even if these three steps are executed quickly, the sync, once started, must wait until the next scheduled run to begin syncing the initial data. This means we inadvertently create the maximum possible delay before the data starts syncing, based on the sync frequency specified by the developer.

To address this, we should adjust the schedule to start counting from the moment the sync is initiated.

@bastienbeurier bastienbeurier added the enhancement New feature or request label Sep 20, 2023
khaliqgant added a commit that referenced this issue Sep 28, 2023
• link activity log id again by session_id and add index
• Clean up error state display on connection details
• add sync_config_id earlier in the run service for proper linking
@khaliqgant
Copy link
Member

@bastienbeurier #1055 updates the logic so that on a sync start (called enum UNPAUSE in code):

  1. The schedule is unpaused via the temporal sdk
  2. The sync is immediately triggered to run
  3. The schedule offset is updated so that the schedule runs according to the moment it was unpaused which satisfies your comment "we should adjust the schedule to start counting from the moment the sync is initiated."

This replaces previous behavior where:

  1. Schedule is unpaused via the temporal SDK

This left us with no immediate schedule trigger and the previous schedule coming into effect with the original offset intact so it would pick up where it originally would have been schedule to run. Explained by an example:

I create a sync at 1335 with a runs property of "every 30 minuutes". That means the next sync run will be at 1405. I pause the sync at 1337. I unpause the sync at 1340. The sync won't run until 1405 still.

With the updated implementation the sync will run right when I unpause it and then will run with an updated schedule next at 1410.

khaliqgant added a commit that referenced this issue Sep 29, 2023
* [gh-#1036] unpause sync starts right away and adjusts the offset

• link activity log id again by session_id and add index
• Clean up error state display on connection details
• add sync_config_id earlier in the run service for proper linking

* [gh-#1036] beefen up interval and offset logic and tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants