Skip to content

Enumeration in worker executor service #295

@vigoo

Description

@vigoo

Create a new service trait and implementation in worker-executor-service-base with two public operations (in pseudo code):

  • get-running-workers(template_id, filter) -> vec<metadata>
  • scan-workers(template_id, filter, precise, cursor) -> option<(cursor, vec<metadata>)>

This service should have a default “live implementation”, which:

  • uses the ActiveWorkers to enumerate all workers in memory
  • uses a combination of Redis and the WorkerService to perform the SCAN to lazily enumerate all workers

Both operations should use a common implementation of applying the filter to the chunk of worker metadata.

We should have a precise bool flag (or some mode enum with two modes for now) with the following meaning:

  • When it is true, scan-workers must check every worker metadata’s cached part (WorkerStatusRecord in the code) to see if it’s oplog_index is the same as the oplog’s size (requires a XLEN redis operation - to be done through OplogService). If not, an up-to-date WorkerStatusRecord needs to be generated calculate_last_known_status. This needs to be done before applying the filters (at least that part of the filter which is related to status)

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions