Schedular requirements

Requirements for Scheduler

Current status: awaiting estimation / agreement on specs Current estimate from Petr: xxx hrs

Front end

• There is a screen to manage scans which is accesible to adimistrators.

• That screen you can schedule scans with a resolution of minute

• Wildcard values are allowed for:

  • Hour
  • Day
  • Weekday
  • Month

• It should be possible to schedule scans on the xth monday,tu,we,th,etc of the month

• Scans can be scanned as a one off or as a reoccuring scan

• It should be possible to see the scanning agenda for the coming month

• It should be possible to limit a scan to a certain scanner machine

• Should support priorities

  • Lazy: somewhere after this time if not otherwise busy
  • Normal
  • High: should be picked up first even if other jobs are queued

• It should be possible to schedule a scan to run once now.

Back end

Database changes:

add "status" and "message" field to runs table current "timestamp" field is when the scan started execution 2 additional fields: "queued_time", "end_time" maybe: add pid and priority

New table queue which shows which scans are queued for execution or currently running.

Queue manager API should implement following calls:

1. enqueue scanner job with parameters similar to bin/do-scan (workplace and scan) and priority ("lazy", "normal" and "high")
2. list current queue entries (question: do we need any internal representation of the queue other than runs table? if there is none, we do not need "list" api method, but we need to keep pid and priority there as well)
3. cancel the scan job

The way API works is totally irrelevant. It could be a local executable, a unix socket or a web server, either will work as long as these functions are provided.

The queue manager backend should implement following functions:

1. Running scans. Check for available resources (memory, cpu, sockets, scans already running, whatever limits we impose, to be defined later). If there is enough resources, take (fifo) high priority scan from the queue and run it, repeat until no high priority scans left. If there is enough resources and there is no high priority scans in queue, take (fifo) normal priority scan from the queue and run it, repeat until no normal scans left. If there is enough resources and we are within "preferred scanning window" (time configured as lowest network load), take (fifo) lazy priority scan from the queue and run it, repeat until no lazy scans left. (NB: we may have a vast distributed system with different scanning windows across the locations. thus, preferred scanning window should probably be a scan property (and do we need to add it to asset info as well?))
2. Cancelling scans
3. Checking scans health status in background
4. Should support having more than one executor at the same time (host name is identifier)

Other

When a scan is being run, it changes status from "queued" to "running". "running" scan may become either "completed", "stopped" (the same thing as cancelled, but for scan that already has beein started), "failed" (scan script returned failure status), "stalled" (did not show any activity within defined timeout and killed), "crashed" (the scan process disappeared without gracefully failing).

All timestamps in runs table should be updated accordingly.

Note: EVERY (scanners/*/scan) script should be reviewed for error handling. "just dying" on any stage is not an option. If there is an error anywhere: scan, fetching results, XML import, database import -- non-zero return status should be raised and a meaningful message should be returned.

TBD: UI requirements.