SAVE components & plan of work

SAVE-core (CLI tool)

• Investigate existing test frameworks and configuration options (LIT, kotlin compiler, gcc, ...)
• calculate text diff from kotlin/native - kotlin-multiplatform-diff
• ktoml
• Describe types of test scenarios
• Create CLI tool
• Execute part of tests batch (by id, not only by toml config)

SAVE Cloud

SAVE common

• Describe domain model for projects and their test statistics
• Test Result: status <- [PASSED, FAILED, IGNORED, INTERNAL_ERROR (CLI exit 1), TEST_ERROR (CLI exit 42)]

SAVE Backend

• API endpoint to upload executable and script for its run configuration
• CRUD for project data, return this data to frontend
• Monitoring (Actuator + Prometheus)

SAVE Orchestrator

• Describe DB schema for event states and execution history
  • Events: Active Agent - ID of test case - Time started - Time finished (nullable) - Status
• Responds to Agent with batches of test IDs from Database, has some atomics to avoid race condition between multiple Agents.
• Build base docker images for different test suites; include SAVED in them and set CMD to SAVED
• Distributed (among containers) test execution - orchestrator should distribute work among workers, monitor and consolidate results
• Write execution to DB, use this data for consolidation and backups
• Write Dockerfile for orchestrator image; mount docker socket to communicate with daemon from inside the container: -v /var/run/docker.sock:/var/run/docker.sock
• Scale tests batch size according to execution duration, compares with heartbeat interval

SAVE Frontend

• initialize it - some template, needed dependencies, kotlin/js support
• copy frontend resources to backend jar during build (see diktat-demo as an example)
• create a form to upload executables (or, in general, add your project, via github link or other ways)
• project view - name, owner or organization, history of tests execution
• dashboard view - table with all analyzed projects

SAVE agent (SAVED - SAVE Daemon?)

• Lightweight process with HTTP API (kotlin/native with ktor-client based on libcurl)
• Runs SAVE CLI inside container (kotlin platform.posix.system)
• Heathbeats to orchestrator (status, percent of tests done from batch)
• When batch finished - streams data to backend, after 200 starts queryng orchestrator for next batch or Finish command (orchestrator waits for Finish from backend first, responds Wait to client and sets Waiting status in DB for this agent). As soon as backend sends Finish to Orchestrator, it starts responding to agent with either Finish or set of test IDs.
• Also should send SAVE execution logs (to Orc?) to store them on the box
• New states: CLI_FAILED, BACKEND_FAILURE, BACKEND_UNREACHABLE instad of ERROR

Other

Repo

• Split into two repos: save and save-cloud
• code style (diktat, detekt)
• releases (upload CLI to GH releases, do we need maven central for other artifacts?)
• automatic deployments via github actions (once we have deployment system)

Database

• MySQL in Cloud
• Tables:
  • Projects
  • Tests execution history
  • Execution states (written by orchestrator)

Deployment

• Docker swarm seems easy enough: shipped with docker out-of-the-box, better than writing own tool. Alternatively: k8s, custom bash script.
• Prometheus with Docker swarm SD
• Grafana