Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add filewatcher capabilities to kanto-auto-deployer #35

Merged
merged 11 commits into from
Mar 30, 2023

Conversation

vasilvas99
Copy link
Contributor

@vasilvas99 vasilvas99 commented Mar 28, 2023

Problem

We want kanto-auto-deployer to be able to run in two modes:

  • one-shot (deploy all manifests in a provided directory and exit)
  • daemon (deploy all manifests in a provided directory, keep monitoring it for changes and redeploy any manifests added later/old manifests changed)

Other issues:

  • Cargo.toml was not using exact semvers for dependencies, leading to compilation breaking sometimes with rustc 1.59.0
  • A more elaborate handling of CLI arguments is required as now we provide more user choice.

Solution

  • All dependencies were set to a specific semver and the cargo.lock was regenerated. Now compilations are consistent.
  • clap was used as a cli parsing library as it's very flexible and parses the CLI arguments as a strongly typed rust data structure + provides the ability to set default values.

A new cli flag --daemon/-d was added that sets the operation mode of KAD.

If the flag is not passed, KAD works in "one-shot" mode (minimal changes here)

If the flag is passed:
The one-shot directory deployment is ran and then an async_watcher is set-up through the new module fs_watch. This module polls the directory every 10 seconds (const POLL_SECONDS: f64 = 10.0;) and if FS events have occurred during that interval it de-bounces them (multiple modifications -> single trigger at the end) it triggers a callback function.

That callback redeploy_on_change goes through all newly added json files/modified existing ones and deploys the containers specified in those manifests. If the container already exists (e.g. it's already deployed) it is first stopped, then removed and created again. This ensures that the latest state described in the manifest will be used when running as a daemon.


The "filewatcher" feature flag

To reduce binary bloat, all filewatcher/deamon functionality has been "namespaced" under the "filewatcher" compilation flag. This is a default feature, so running cargo build --release will yield a version of KAD that has this mode available for use.

To compile without daemon mode run cargo build --release --no-default-features. This will yield a binary that only has the "one-shot" mode.


Screenshots:
Help:
image

One-shot mode:
image

Daemon mode:

image

As it can be seen on the screenshots only the created/modified files are taken into account. If the manifest is correct (e.g. vum.json) it will be redeployed. If not (e.g. wrong.json) it would log an error and continue with the monitoring.

Compiled with --no-default-features:
image

Footprint (on QEMUx86 + kvm) in daemon mode (service):

Size with filewatcher: 4.2 M (most of the increase is due to the clap CLI parser)
RAM usage: 1.4 M
CPU: 0.0 all cores (from htop)

@vasilvas99 vasilvas99 marked this pull request as ready for review March 30, 2023 07:41
Copy link
Contributor

@stlachev stlachev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@stlachev stlachev merged commit cead2d0 into eclipse-leda:main Mar 30, 2023
@stlachev stlachev deleted the KAD-fswatcher branch March 30, 2023 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants