Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support environment variables as alternate config mechanism #2162

Merged
merged 25 commits into from Apr 1, 2022

Conversation

mavam
Copy link
Member

@mavam mavam commented Mar 24, 2022

This PR adds new functionality to use environment variables as alternate input to a YAML config file.

馃摑 Checklist

  • Add utility function to enumerate environment
  • Translate VAST_ and CAF_ env vars into settings
  • Add changelog entry
  • Add tests
  • Update docs

馃幆 Review Instructions

Commit-by-commit.

@mavam mavam added the feature New functionality label Mar 24, 2022
@mavam mavam force-pushed the topic/environment-variables branch from 8aad83a to d74d900 Compare March 25, 2022 05:31
@mavam mavam marked this pull request as ready for review March 25, 2022 05:40
@mavam mavam added feature New functionality and removed feature New functionality labels Mar 25, 2022
Copy link
Member

@dominiklohmann dominiklohmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat hack, but I think this needs a bit of work and most importantly tests.

libvast/src/system/configuration.cpp Outdated Show resolved Hide resolved
libvast/src/system/configuration.cpp Outdated Show resolved Hide resolved
libvast/src/system/configuration.cpp Outdated Show resolved Hide resolved
libvast/src/system/configuration.cpp Outdated Show resolved Hide resolved
libvast/src/system/configuration.cpp Outdated Show resolved Hide resolved
libvast/src/system/configuration.cpp Outdated Show resolved Hide resolved
@mavam
Copy link
Member Author

mavam commented Mar 25, 2022

@tenzir/vast how would you test this feature?

Copy link
Member

@dominiklohmann dominiklohmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left some final remarks inline. Other than that let's ship it!

libvast/src/system/configuration.cpp Outdated Show resolved Hide resolved
libvast/src/system/configuration.cpp Outdated Show resolved Hide resolved
libvast/src/system/configuration.cpp Show resolved Hide resolved
@mavam mavam added feature New functionality and removed feature New functionality labels Mar 26, 2022
libvast/src/plugin.cpp Outdated Show resolved Hide resolved
Copy link
Member

@lava lava left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not to derail the PR, but some additional minor issues.

libvast/src/detail/env.cpp Outdated Show resolved Hide resolved
libvast/src/system/configuration.cpp Show resolved Hide resolved
libvast/src/system/configuration.cpp Show resolved Hide resolved
libvast/src/system/configuration.cpp Outdated Show resolved Hide resolved
This is another stab at breaking up the giant monolith function and
making it more side-effect free. This one might appear as zero-sum
refactoring because we're adding a singleton, but the bigger evil is the
giant parsing function that we need to break down first.
@mavam mavam force-pushed the topic/environment-variables branch 2 times, most recently from 048fe80 to 4da964a Compare March 30, 2022 04:40
To the user, it doesn't matter whether locking takes place under the
hood. They just want to get their environment worked. This is why the
locked_ prefix no longer exists.
It turns out that other functions operate the same way, e.g.,
get_plugin_dirs takes also an actor_system_config. So we'll keep this as
transport medium for now.
@mavam mavam force-pushed the topic/environment-variables branch from 4da964a to 3cd4bf5 Compare March 30, 2022 05:05
@mavam mavam requested a review from lava March 31, 2022 11:54
Copy link
Member

@lava lava left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I investigated the last commit for a while, and it looks like the whole "convert settings based on the type in the configuration" is only done for options that come directly from caf. For the vast. options it looks like the default YAML/vast::data parser does (accidentally?) the correct thing already.

The logic on this part is pretty complicated so there may be some config parsing bugs left here, but even if so they would apply to the existing code and not be introduced by this PR so merging this should be fine.

@mavam mavam merged commit 4121692 into master Apr 1, 2022
@mavam mavam deleted the topic/environment-variables branch April 1, 2022 09:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New functionality
Projects
None yet
3 participants