New snapshot handling

[tizava]

• Flaw in the current architecture prevents restoration of snapshot
• Proposal for new snapshot handling

[github-actions]

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run //:license-check

Status: ⚠️ Needs Review

Click to expand output

[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.3.0) and connecting to it...
INFO: Invocation ID: 4ef1c1b6-7ee3-45ff-a4da-92e5cc2b59ad
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
DEBUG: Rule 'rust_qnx8_toolchain+' indicated that a canonical reproducible form can be obtained by modifying arguments integrity = "sha256-oEubHgeZDdT0svMmBKJx7c3/2TdSI/vfwRUyDn+TPGA="
DEBUG: Repository rust_qnx8_toolchain+ instantiated at:
  <builtin>: in <toplevel>
Repository rule http_archive defined at:
  /home/runner/.bazel/external/bazel_tools/tools/build_defs/repo/http.bzl:394:31: in <toplevel>
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
WARNING: For repository 'score_process', the root module requires module version score_process@1.3.2, but got score_process@1.4.0 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
Computing main repo mapping: 
Loading: 
Loading: 3 packages loaded
Loading: 3 packages loaded
    currently loading: 
Analyzing: target //:license-check (4 packages loaded, 0 targets configured)
Analyzing: target //:license-check (4 packages loaded, 0 targets configured)

Analyzing: target //:license-check (87 packages loaded, 10 targets configured)

Analyzing: target //:license-check (153 packages loaded, 2123 targets configured)

Analyzing: target //:license-check (159 packages loaded, 6997 targets configured)

Analyzing: target //:license-check (159 packages loaded, 6997 targets configured)

INFO: Analyzed target //:license-check (162 packages loaded, 9013 targets configured).
[13 / 14] [Prepa] Generating Dash formatted dependency file ...
INFO: From Generating Dash formatted dependency file ...:
INFO: Successfully converted 62 packages from Cargo.lock to bazel-out/k8-fastbuild/bin/formatted.txt
INFO: Found 1 target...
Target //:license.check.license_check up-to-date:
  bazel-bin/license.check.license_check
  bazel-bin/license.check.license_check.jar
INFO: Elapsed time: 26.807s, Critical Path: 0.39s
INFO: 14 processes: 5 disk cache hit, 9 internal.
INFO: Build completed successfully, 14 total actions
INFO: Running command line: bazel-bin/license.check.license_check ./formatted.txt <args omitted>
usage: org.eclipse.dash.licenses.cli.Main [-batch <int>] [-cd <url>]
       [-confidence <int>] [-ef <url>] [-excludeSources <sources>] [-help] [-lic
       <url>] [-project <shortname>] [-repo <url>] [-review] [-summary <file>]
       [-timeout <seconds>] [-token <token>]

[github-actions]

The created documentation from the pull request is available at: docu-html

[arkjedrz]

I created some notes and thoughts on the topic based on Tibor's work. Some comments might rephrase what Tibor proposed.

Assumed user expectations

• Are snapshots for restoring previous versions of data?
  • Maybe keep multiple versions as diffs? More compute-heavy, less space needed.
• Are snapshots to ensure robust KVS load on unexpected unavailability - e.g., corrupted file?

KVS initialization issue

For sure - it is a must that KVS can be initialized for non-zero snapshot ID.
Current ideas:

• Make optional builder parameter for snapshot ID.
  • 0 as a default.
• Make automatic load and return snapshot ID that was used.
  • Load until successful.
  • Not preferred - implicit and not really obvious operation happens underneath.
    • But more comments on that later.

Replace flush with snapshot_create

Current approach is that each flush introduces a new snapshot implicitly.
Those are indexed using snapshot IDs containing an increasing number.

Maybe it'd be okay to have two methods:

• store - implicitly to current snapshot ID, direct replacement to flush
  • Some predefined constant name can be used for current snapshot ID?
• store_as - with snapshot ID provided as parameter

This way it is possible to have a clear way to store e.g., pre-update state.
This could work in conjunction to snapshot_remove to remove such snapshot once update is done and succesful.

Correctness of current approach to snapshot IDs

Should a number used for ID be replaced with a name?
How to check availables IDs before init?

Add snapshot_remove

This is currently missing.
How to approach IDs in case of removal?

• Should they keep their IDs?
  • This would require changes to the implementation to accomodate that.
• Should existing snapshots be rotated into correct position?
  • It would also be surprising and difficult to track - user would have to recreate rotating logic to create a mapping.
  • But to be fair it is consistent with current approach - user is expected to understand and map implicit behavior of rotating snapshots.
• Having named snapshots removes this issues.