Allow multiple filters #627

mstemm · 2016-07-14T19:24:58Z

Used by falco to do a very fast initial categorization of rules by event type.

@gianlucaborello @luca3m @ldegio wanna take a look?

Putting whitespace changes in a separate commit.

Instead of combining all rules into one huge filter expression and giving it to the inspector, keep each filter expression separate and annotate it with the events for which the rule applies. This uses the capabilties in draios/sysdig#627 to have multiple sets of event-specific filters. Within the compiler, a new pass over the ast get_evttypes looks for evt.type clauses, converts the evt.type as a string to any event type ids for which it may apply, and passes that back with the compiled rule. As rule conditions may refer to evt.types in negative contexts (i.e. evt.type != XXX, or not evt.type = XXX), this pass prefers rules that list event type checks at the beginning of conditions, and allows other rules with a warning. When traversing the ast looking for evt.type checks, once any "!=" or "not ..." is seen, no other evt.type checks are "allowed". If one is found, the rule is considered ambiguous wrt event types. In this case, a warning is printed and the rule is associated with a catchall set that runs for all event types. Also, instead of rejecting rules with no event type check, print a warning and associate it with the catchall set. In the rule loader, create a new global events that maps each event as a string to the list of event ids for which it may apply. Also there's a callback add_filter from lua which is called for each rule. In turn, it passes the rule and list of event ids to the inspector using add_evttype_filter(). Also, with -v (verbose) also print the exact set of events found for each event type. This is used by a upcoming change to the set of unit tests. TODO: - There are some remnants of the old parsing code left in the rule - loader that should be removed. - The string splitting within the compiler is ugly and should be cleaned up.

Instead of combining all rules into one huge filter expression and giving it to the inspector, keep each filter expression separate and annotate it with the events for which the rule applies. This uses the capabilties in draios/sysdig#627 to have multiple sets of event-specific filters. Within the compiler, a new pass over the ast get_evttypes looks for evt.type clauses, converts the evt.type as a string to any event type ids for which it may apply, and passes that back with the compiled rule. As rule conditions may refer to evt.types in negative contexts (i.e. evt.type != XXX, or not evt.type = XXX), this pass prefers rules that list event type checks at the beginning of conditions, and allows other rules with a warning. When traversing the ast looking for evt.type checks, once any "!=" or "not ..." is seen, no other evt.type checks are "allowed". If one is found, the rule is considered ambiguous wrt event types. In this case, a warning is printed and the rule is associated with a catchall set that runs for all event types. Also, instead of rejecting rules with no event type check, print a warning and associate it with the catchall set. In the rule loader, create a new global events that maps each event as a string to the list of event ids for which it may apply. Also there's a callback add_filter from lua which is called for each rule. In turn, it passes the rule and list of event ids to the inspector using add_evttype_filter(). Also, with -v (verbose) also print the exact set of events found for each event type. This is used by a upcoming change to the set of unit tests.

Instead of combining all rules into one huge filter expression and giving it to the inspector, keep each filter expression separate and annotate it with the events for which the rule applies. This uses the capabilties in draios/sysdig#627 to have multiple sets of event-specific filters. Change traverse_ast to allow a set of node types instead of a single node type. Within the compiler, a new pass over the ast get_evttypes looks for evt.type clauses, converts the evt.type as a string to any event type ids for which it may apply, and passes that back with the compiled rule. As rule conditions may refer to evt.types in negative contexts (i.e. evt.type != XXX, or not evt.type = XXX), this pass prefers rules that list event type checks at the beginning of conditions, and allows other rules with a warning. When traversing the ast looking for evt.type checks, once any "!=" or "not ..." is seen, no other evt.type checks are "allowed". If one is found, the rule is considered ambiguous wrt event types. In this case, a warning is printed and the rule is associated with a catchall set that runs for all event types. Also, instead of rejecting rules with no event type check, print a warning and associate it with the catchall set. In the rule loader, create a new global events that maps each event as a string to the list of event ids for which it may apply. Instead of calling install_filter once after all rules have been loaded, call a new function add_filter for each rule. In turn, it passes the rule and list of event ids to the inspector using add_evttype_filter(). Also, with -v (verbose) also print the exact set of events found for each event type. This is used by a upcoming change to the set of unit tests.

Add support for an additional set of individual filters that can run alongside the main filter set in sinsp::set_filter(). Each additional filter comes with a list of event types for which the filter should run. sinsp::add_evttype_filter adds an event type filter to the inspector. It provides a list of event types and a sinsp_filter pointer. The list of event types can be empty, implying the filter is a catchall filter and should run for all event types. This does not replace the main function set_filter()--the idea is that you can specify a global filter as well as per-event type filters. Both will run for all events. An array m_filter_by_evttype maps from event type to a list of sinsp_filter pointers. Each list at index i holds the filters that should run for events with type i. A filter can run for multiple event types, so the sinsp_filter pointers can appear in multiple lists. There's a separate list m_catchall_evttype_filters that contains filters that should run for all event types. The individual filter pointers are also held in a list m_evttype_filters so they can be cleaned up on close(). A new method run_filters_on_evt() handles the details of running the main filter, catchall filters, and the individual filters. It finds the filters related to the event's type, and runs those filters. Falco uses the lua callbacks in lua_parser{_api} to populate filters as the lua code parses the rule's condition, so also modify lua_parser to add a optional flag to get_parser that steals the m_filter object and creates a new one. Falco calls get_parser for each rule's condition, which allows it to create a sinsp_filter object for each rule.

Instead of combining all rules into one huge filter expression and giving it to the inspector, keep each filter expression separate and annotate it with the events for which the rule applies. This uses the capabilties in draios/sysdig#627 to have multiple sets of event-specific filters. Change traverse_ast to allow a set of node types instead of a single node type. Within the compiler, a new pass over the ast get_evttypes looks for evt.type clauses, converts the evt.type as a string to any event type ids for which it may apply, and passes that back with the compiled rule. As rule conditions may refer to evt.types in negative contexts (i.e. evt.type != XXX, or not evt.type = XXX), this pass prefers rules that list event type checks at the beginning of conditions, and allows other rules with a warning. When traversing the ast looking for evt.type checks, once any "!=" or "not ..." is seen, no other evt.type checks are "allowed". If one is found, the rule is considered ambiguous wrt event types. In this case, a warning is printed and the rule is associated with a catchall set that runs for all event types. Also, instead of rejecting rules with no event type check, print a warning and associate it with the catchall set. In the rule loader, create a new global events that maps each event as a string to the list of event ids for which it may apply. Instead of calling install_filter once after all rules have been loaded, call a new function add_filter for each rule. In turn, it passes the rule and list of event ids to the inspector using add_evttype_filter(). Also, with -v (verbose) also print the exact set of events found for each event type. This is used by a upcoming change to the set of unit tests.

* Whitespace diffs. Putting whitespace changes in a separate commit. * Add optional support for multiple filters. Add support for an additional set of individual filters that can run alongside the main filter set in sinsp::set_filter(). Each additional filter comes with a list of event types for which the filter should run. sinsp::add_evttype_filter adds an event type filter to the inspector. It provides a list of event types and a sinsp_filter pointer. The list of event types can be empty, implying the filter is a catchall filter and should run for all event types. This does not replace the main function set_filter()--the idea is that you can specify a global filter as well as per-event type filters. Both will run for all events. An array m_filter_by_evttype maps from event type to a list of sinsp_filter pointers. Each list at index i holds the filters that should run for events with type i. A filter can run for multiple event types, so the sinsp_filter pointers can appear in multiple lists. There's a separate list m_catchall_evttype_filters that contains filters that should run for all event types. The individual filter pointers are also held in a list m_evttype_filters so they can be cleaned up on close(). A new method run_filters_on_evt() handles the details of running the main filter, catchall filters, and the individual filters. It finds the filters related to the event's type, and runs those filters. Falco uses the lua callbacks in lua_parser{_api} to populate filters as the lua code parses the rule's condition, so also modify lua_parser to add a optional flag to get_parser that steals the m_filter object and creates a new one. Falco calls get_parser for each rule's condition, which allows it to create a sinsp_filter object for each rule.

Whitespace diffs.

c15f0ff

Putting whitespace changes in a separate commit.

mstemm mentioned this pull request Jul 14, 2016

Event specific filters falcosecurity/falco#101

Merged

mstemm force-pushed the allow-multiple-filters branch from c1e1ec7 to b3ebbe1 Compare July 14, 2016 20:04

mstemm merged commit a7a8b59 into dev Jul 18, 2016

mstemm deleted the allow-multiple-filters branch July 18, 2016 16:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow multiple filters #627

Allow multiple filters #627

mstemm commented Jul 14, 2016

Allow multiple filters #627

Allow multiple filters #627

Conversation

mstemm commented Jul 14, 2016