Event specific filters #101
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add a verbose flag -v which implies printing additional info. This is passed down to lua during load_rules and sets the per-module verbose value for the compiler and parser modules. Later commits will use this to print additional info when loading rules.
Instead of combining all rules into one huge filter expression and giving it to the inspector, keep each filter expression separate and annotate it with the events for which the rule applies. This uses the capabilties in draios/sysdig#627 to have multiple sets of event-specific filters. Change traverse_ast to allow a set of node types instead of a single node type. Within the compiler, a new pass over the ast get_evttypes looks for evt.type clauses, converts the evt.type as a string to any event type ids for which it may apply, and passes that back with the compiled rule. As rule conditions may refer to evt.types in negative contexts (i.e. evt.type != XXX, or not evt.type = XXX), this pass prefers rules that list event type checks at the beginning of conditions, and allows other rules with a warning. When traversing the ast looking for evt.type checks, once any "!=" or "not ..." is seen, no other evt.type checks are "allowed". If one is found, the rule is considered ambiguous wrt event types. In this case, a warning is printed and the rule is associated with a catchall set that runs for all event types. Also, instead of rejecting rules with no event type check, print a warning and associate it with the catchall set. In the rule loader, create a new global events that maps each event as a string to the list of event ids for which it may apply. Instead of calling install_filter once after all rules have been loaded, call a new function add_filter for each rule. In turn, it passes the rule and list of event ids to the inspector using add_evttype_filter(). Also, with -v (verbose) also print the exact set of events found for each event type. This is used by a upcoming change to the set of unit tests.
Add shell scripts to make it easier to collect performance results from traces, live tests, and phoronix tests. With run_performance_tests.sh you specify the following: - a subject program to run, using --root - a name to give to this set of results, using --variant - a test to run, using --test - a file to write the results to, using --results. For tests that start with "trace", the script runs falco/sysdig on the trace file and measures the time taken to read the file. For other tests, he script handles starting falco/sysdig, starting a cpu measurement script (a wrapper around top, just to provide identical values to what you would see using top) to measure the cpu usage of falco/sysdig, and running a live test. The measurement interval for cpu usage depends on the test being run--10 seconds for most tests, 2 seconds for shorter tests. The output is written as json to the file specified in --results. Also add R scripts to easily display the results from the shell script. plot-live.r shows a linechart of the cpu usage for the provided variants over time. plot-traces.r shows grouped barcharts showing user/system/total time taken for the provided variants and traces. One bug--you have to make the results file actual json by adding leading/trailing []s.
mstemm
force-pushed
the
event-specific-filters
branch
from
6f519bc
to
cf0d36e
Compare
- Move evt.type checks to the front of rules. This is necessary to avoid warnings now that event types are automatically extracted during rule parsing and used to bind each rule with a specific set of events. - Explicitly specify open for O_CREAT. With the change to event-specific filters, it's necessary to associate a search for O_CREAT with evt.type=open.
Add tests that verify that the event type identification functionality
is working. Notable changes:
- Modify falco_test.py to additionally check for warnings when loading
any set of rules and verify that the event types for each rule match
expected values. This is controlled by the new multiplex fields
"rules_warning" and "rules_events".
- Instead of starting with an empty falco_tests.yaml from scratch from
the downloaded trace files, use a checked-in version which defines
two tests:
- Loading the checked-in falco_rules.yaml and verify that no rules
have warnings.
- A sample falco_rules_warnings.yaml that has ~30 different
mutations of rule filtering expressions. The test verifies for each
rule whether or not the rule should result in a warning and what the
extracted event types are.
The generated tests from the trace files are appended to this file.
- Add an empty .scap file to use with the above tests.
mstemm
force-pushed
the
event-specific-filters
branch
from
1da269f
to
7b68fc2
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes to switch from a single global filter expression that works for all event types to a collection of individual filter expressions, each associated with a list of event types.
This allows falco to only consider those rules associated with a given event's type when trying to match rules against an event, which is much faster.
This depends on draios/sysdig#627.