chore: support generic rules to have routers drain events

[Sidddddarth]

Description

The topic of draining events from a sync gave birth to this.
This is a way to pass generic rules by which (batch)router and processor(only jobRunID now) can drain events. How this is set to config is another story.

Routers, BatchRouters now have a drainer which provides the method:
Drain(job *jobsdb.JobT) (bool, string) to check if their jobs can be drained. Support for draining via configurable destinationIDs("Router.toAbortDestinationIDs") and jobRunID("RSources.toAbortJobRunIDs") present now. Adding more drain configurations should be straightforward -> just need to add logic in type drainer and (d *drainer) Drain(...).

Processor has a simpler approach, a new config field is added to accommodate configurations to drain by, jobRunId for now. Also injecting *config.Config into processor now. Plan is to proliferate it's use instead of config.Default in processor package(story for another PR).

Linear Ticket

drain events for a cancelled sync

Security

• The code changed/added as part of this pull request won't create any security issues with how the software is being used.

[codecov]

Codecov Report

Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (408046d) 71.75% compared to head (edf6dc1) 71.78%.
Report is 6 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3856      +/-   ##
==========================================
+ Coverage   71.75%   71.78%   +0.03%     
==========================================
  Files         374      372       -2     
  Lines       54919    54987      +68     
==========================================
+ Hits        39407    39474      +67     
- Misses      13188    13190       +2     
+ Partials     2324     2323       -1     

Files	Coverage Δ
processor/manager.go	92.68% <100.00%> (+0.68%)	⬆️
router/batchrouter/factory.go	100.00% <100.00%> (ø)
router/batchrouter/handle.go	72.05% <100.00%> (ø)
router/batchrouter/handle_async.go	8.62% <100.00%> (ø)
router/batchrouter/handle_lifecycle.go	62.00% <100.00%> (+0.77%)	⬆️
router/batchrouter/worker.go	88.72% <100.00%> (-0.06%)	⬇️
router/handle.go	70.75% <100.00%> (+0.86%)	⬆️
router/handle_lifecycle.go	95.45% <100.00%> (+0.10%)	⬆️
router/worker.go	81.72% <100.00%> (-0.40%)	⬇️
processor/processor.go	89.06% <95.31%> (+0.11%)	⬆️
... and 1 more

... and 13 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[atzoum]

Router has more parameters than batchrouter. Who is using rudderAccountId?

RecordID                interface{} `json:"record_id"`
WorkspaceID             string      `json:"workspaceId"`
RudderAccountID         string      `json:"rudderAccountId"`

[Sidddddarth]

yeah
the new struct contains a combo of both

[Sidddddarth]

as for RudderAccountId, it is present in multiple structs in the repo.
I will tackle them in another PR. Or perhaps defer to integrations, since they seem to have introduced it.

[cisse21]

RudderAccountID is used for OAuth destination IIRC

[atzoum]

Nobody is reading it in server's code though, @sanpj2292 ?

[lvrach]

the package name is not very indicative of its contents, and in general, util/misc names should be avoided.

We can have a type package for JobParameters.
and introduce a new drainer package for NewDrainer