feat: hot-reload pipeline files alongside rules#68
Merged
Conversation
Store pipeline file paths in RuntimeEngine so that load_rules() re-reads pipeline YAML from disk before rebuilding the engine. Extend the notify file watcher to also watch individual pipeline files. If any pipeline file fails to parse during reload, the entire reload aborts and the old engine stays active. - RuntimeEngine: add pipeline_paths field, set_pipeline_paths(), pipeline_paths(), and reload_pipelines() helper - LogProcessor::reload_rules(): carry pipeline_paths through to new engine - spawn_file_watcher(): accept and watch additional pipeline file paths - DaemonConfig: add pipeline_paths field, wire through from CLI - 2 new tests: reload_re_reads_pipelines_from_disk, reload_with_broken_pipeline_keeps_old_engine
Mention that pipeline files passed via -p are also watched for changes in daemon mode, and that rules + pipelines are re-read together on any reload trigger.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RuntimeEngineso thatload_rules()re-reads pipeline YAML from disk before rebuilding the engine, instead of carrying over stale in-memory pipelines.notifyfile watcher to also watch individual pipeline files (in addition to the rules directory). Any YAML change triggers the same debounced reload signal.Changes
rsigma-runtime/src/engine.rspipeline_pathsfield,set_pipeline_paths(),pipeline_paths(),reload_pipelines()helperrsigma-runtime/src/processor.rsreload_rules()carriespipeline_pathsthrough to the new engine; 2 new testsrsigma-cli/src/daemon/reload.rsspawn_file_watcher()accepts and watches additional pipeline file pathsrsigma-cli/src/daemon/server.rsDaemonConfig.pipeline_paths, engine setup, watcher setup, updated log messagesrsigma-cli/src/main.rspipeline_pathsintoDaemonConfigTest plan
reload_re_reads_pipelines_from_disk: verifies field mapping changes take effect after reloadreload_with_broken_pipeline_keeps_old_engine: verifies failed reload leaves old engine activereload_rules_preserves_enginestill passescargo test --workspace, zero failures)cargo clippy --workspace --all-features -- -D warningsclean