feat: toggle event ordering for workspace/destination #4278

Sidddddarth · 2024-01-03T01:23:23Z

Description

Provides config to toggle event ordering at a workspace/destination level.("Router.orderingDisabledWorkspaceIDs", "Router.orderingDisabledDestinationIDs")

On setting required workspace/destinationID, the barrier for that BarrierKey is disabled. Turns back to processing events in order once the config has been reset and after passing of eventOrderDisabledStateDuration and eventOrderHalfEnabledStateDuration duration.

Introduces BarrierKey which is just a composite:

type BarrierKey struct {
	DestinationID, UserID, WorkspaceID string
}

so the barrier methods now take this instead of the string(<destID>:<userID>) earlier

Linear Ticket

linear

Security

The code changed/added as part of this pull request won't create any security issues with how the software is being used.

coderabbitai · 2024-01-03T01:23:28Z

Important

Auto Review Skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository.

To trigger a single review, invoke the @coderabbitai review command.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>.
- Generate unit-tests for this file.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit tests for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai generate interesting stats about this repository from git and render them as a table.
- @coderabbitai show all the console.log statements in this repository.
- @coderabbitai read src/utils.ts and generate unit tests.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
The JSON schema for the configuration file is available here.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

CodeRabbit Discord Community

Join our Discord Community to get help, request features, and share feedback.

codecov · 2024-01-03T01:40:10Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (99db4e2) 75.64% compared to head (8baecb7) 75.69%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4278      +/-   ##
==========================================
+ Coverage   75.64%   75.69%   +0.05%     
==========================================
  Files         381      381              
  Lines       46077    46120      +43     
==========================================
+ Hits        34854    34912      +58     
- Misses       8939     8942       +3     
+ Partials     2284     2266      -18

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2024-01-30T01:56:50Z

This PR is considered to be stale. It has been open 20 days with no further activity thus it is going to be closed in 7 days. To avoid such a case please consider removing the stale label manually or add a comment to the PR.

mihir20

LGTM

mihir20 · 2024-02-08T10:11:03Z

router/router_test.go

+			require.ErrorIs(t, err, types.ErrJobOrderBlocked)
+		})
+
+		t.Run("job not blocked after event ordering is disabled", func(t *testing.T) {


should we add a similar test for workspace ID as well?

mihir20 · 2024-02-08T10:15:57Z

router/worker.go

@@ -871,7 +883,7 @@ func (w *worker) prepareResponsesForJobs(destinationJob *types.DestinationJobT,
 	return respStatusCodes, respBodys
 }

-func (w *worker) canSendJobToDestination(failedJobOrderKeys map[string]struct{}, destinationJob *types.DestinationJobT) bool {
+func (w *worker) canSendJobToDestination(failedJobOrderKeys map[eventorder.BarrierKey]struct{}, destinationJob *types.DestinationJobT) bool {
 	if !w.rt.guaranteeUserEventOrder {


should we add this check here as well?

(rt.guaranteeUserEventOrder && rt.barrier.Disabled(orderKey)) || (rt.eventOrderingDisabledForWorkspace(job.WorkspaceId) || rt.eventOrderingDisabledForDestination(parameters.DestinationID))

Sidddddarth requested a review from atzoum January 3, 2024 01:23

Sidddddarth marked this pull request as draft January 3, 2024 01:23

Sidddddarth force-pushed the feat.eventOrderingToggleForWorkspaceOrDestination branch from 20eb0fc to 9988150 Compare January 3, 2024 01:24

Sidddddarth force-pushed the feat.eventOrderingToggleForWorkspaceOrDestination branch 2 times, most recently from db54ee0 to d5fffa5 Compare January 8, 2024 04:46

Sidddddarth marked this pull request as ready for review January 8, 2024 06:19

Sidddddarth force-pushed the feat.eventOrderingToggleForWorkspaceOrDestination branch 2 times, most recently from 01a1935 to 11c37b2 Compare January 9, 2024 15:28

github-actions bot added Stale and removed Stale labels Jan 30, 2024

Sidddddarth force-pushed the feat.eventOrderingToggleForWorkspaceOrDestination branch from 11c37b2 to 9e0031c Compare February 7, 2024 13:35

mihir20 approved these changes Feb 8, 2024

View reviewed changes

Sidddddarth force-pushed the feat.eventOrderingToggleForWorkspaceOrDestination branch from 0f0abde to 1170a58 Compare February 9, 2024 09:10

Sidddddarth requested review from cisse21, lvrach, BonapartePC, fracasula and achettyiitr February 9, 2024 14:16

Sidddddarth added 4 commits February 13, 2024 09:25

feat: toggle event ordering for workspace/destination

b037221

fixup! feat: toggle event ordering for workspace/destination

9636510

standarding eventOrderKey: using the same everywhere

bdf9ce1

review

8baecb7

Sidddddarth force-pushed the feat.eventOrderingToggleForWorkspaceOrDestination branch from 1170a58 to 8baecb7 Compare February 13, 2024 03:55

cisse21 approved these changes Feb 14, 2024

View reviewed changes

cisse21 merged commit 3d43b69 into master Feb 14, 2024
42 checks passed

cisse21 deleted the feat.eventOrderingToggleForWorkspaceOrDestination branch February 14, 2024 07:02

devops-github-rudderstack mentioned this pull request Mar 4, 2024

chore: prerelease 1.21.0-rc.1 #4439

Closed

This was referenced Mar 4, 2024

chore: release 1.21.0 #4440

Merged

chore: prerelease 1.21.0-rc.1 #4441

Merged

chore: prerelease 1.21.0-rc.2 #4445

Closed

chore: prerelease 1.21.0-rc.2 #4447

Merged

chore: prerelease 1.21.0-rc.3 #4451

Closed

chore: prerelease 1.21.0-rc.3 #4453

Merged

This was referenced Mar 14, 2024

chore: prerelease 1.22.0-rc.1 #4480

Closed

chore: prerelease 1.22.0-rc.1 #4489

Merged

chore: prerelease 1.22.0-rc.2 #4490

Closed

chore: prerelease 1.22.0-rc.2 #4498

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: toggle event ordering for workspace/destination #4278

feat: toggle event ordering for workspace/destination #4278

Sidddddarth commented Jan 3, 2024 •

edited by achettyiitr

coderabbitai bot commented Jan 3, 2024 •

edited

Auto Review Skipped

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (`.coderabbit.yaml`)

CodeRabbit Discord Community

codecov bot commented Jan 3, 2024 •

edited

github-actions bot commented Jan 30, 2024

mihir20 left a comment

mihir20 Feb 8, 2024

mihir20 Feb 8, 2024

feat: toggle event ordering for workspace/destination #4278

feat: toggle event ordering for workspace/destination #4278

Conversation

Sidddddarth commented Jan 3, 2024 • edited by achettyiitr

Description

Linear Ticket

Security

coderabbitai bot commented Jan 3, 2024 • edited

Auto Review Skipped

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (.coderabbit.yaml)

CodeRabbit Discord Community

codecov bot commented Jan 3, 2024 • edited

Codecov Report

github-actions bot commented Jan 30, 2024

mihir20 left a comment

Choose a reason for hiding this comment

mihir20 Feb 8, 2024

Choose a reason for hiding this comment

mihir20 Feb 8, 2024

Choose a reason for hiding this comment

Sidddddarth commented Jan 3, 2024 •

edited by achettyiitr

coderabbitai bot commented Jan 3, 2024 •

edited

CodeRabbit Configration File (`.coderabbit.yaml`)

codecov bot commented Jan 3, 2024 •

edited