Skip to content

Commit

Permalink
Create embeddable falco engine.
Browse files Browse the repository at this point in the history
Create standalone classes falco_engine/falco_outputs that can be
embedded in other programs. falco_engine is responsible for matching
events against rules, and falco_output is responsible for formatting an
alert string given an event and writing the alert string to all
configured outputs.

falco_engine's main interfaces are:

 - load_rules/load_rules_file: Given a path to a rules file or a string
   containing a set of rules, load the rules. Also loads needed lua code.
 - process_event(): check the event against the set of rules and return
   the results of a match, if any.
 - describe_rule(): print details on a specific rule or all rules.
 - print_stats(): print stats on the rules that matched.
 - enable_rule(): enable/disable any individual rule. New falco command
   line option -D allows you to disable one or more rules on the
   command line.

falco_output's main interfaces are:
 - init(): load needed lua code.
 - add_output(): add an output channel for alert notifications.
 - handle_event(): given an event that matches one or more rules, format
   an alert message and send it to any output channels.

Each of falco_engine/falco_output maintains a separate lua state and
loads separate sets of lua files. The code to create and initialize the
lua state is in a base class falco_common.

falco_engine no longer logs anything. In the case of errors, it throws
exceptions. falco_logger is now only used as a logging mechanism for
falco itself and as an output method for alert messages. (This should
really probably be split, but it's ok for now).

falco_engine contains an sinsp_evttype_filter object containing the set
of eventtype filters. Instead of calling
m_inspector->add_evttype_filter() to add a filter created by the
compiler, call falco_engine::add_evttype_filter() instead. This means
that the inspector runs with a NULL filter and all events are returned
from do_inspect. This depends on
draios/sysdig#633 which has a wrapper around a
set of eventtype filters.

Some additional changes along with creating these classes:

- Some cleanups of unnecessary header files, cmake include_directory()s,
  etc to only include necessary includes and only include them in header
  files when required.

- Try to avoid 'using namespace std' in header files, or assuming
  someone else has done that. Generally add 'using namespace std' to all
  source files.

- Instead of using sinsp_exception for all errors, define a
  falco_engine_exception class for exceptions coming from the falco
  engine and use it instead. For falco program code, switch to general
  exceptions under std::exception and catch + display an error for all
  exceptions, not just sinsp_exceptions.

- Remove fields.{cpp,h}. This was dead code.

- Start tracking counts of rules by priority string (i.e. what's in the
  falco rules file) as compared to priority level (i.e. roughtly
  corresponding to a syslog level). This keeps the rule processing and
  rule output halves separate. This led to some test changes. The regex
  used in the test is now case insensitive to be a bit more flexible.

- Now that draios/sysdig#632 is merged, we can
  delete the rules object (and its lua_parser) safely.

Finally, fix most memory leaks found by valgrind:

 - falco_configuration wasn't deleting the allocated m_config yaml
   config.
 - several ifstreams were being created simply to test which falco
   config file to use.
 - In the lua output methods, an event formatter was being created using
   falco.formatter() but there was no corresponding free_formatter().

This depends on changes in draios/sysdig#640.
  • Loading branch information
mstemm committed Aug 10, 2016
1 parent b57eb86 commit 3bf94ee
Show file tree
Hide file tree
Showing 22 changed files with 696 additions and 436 deletions.
2 changes: 1 addition & 1 deletion test/falco_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ def check_detections(self, res):
if events_detected == 0:
self.fail("Detected {} events when should have detected > 0".format(events_detected))

level_line = '{}: (\d+)'.format(self.detect_level)
level_line = '(?i){}: (\d+)'.format(self.detect_level)
match = re.search(level_line, res.stdout)

if match is None:
Expand Down
10 changes: 5 additions & 5 deletions test/run_regression_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,12 @@ EOF
function prepare_multiplex_file() {
cp $SCRIPTDIR/falco_tests.yaml.in $MULT_FILE

prepare_multiplex_fileset traces-positive True Warning False
prepare_multiplex_fileset traces-negative False Warning True
prepare_multiplex_fileset traces-info True Informational False
prepare_multiplex_fileset traces-positive True WARNING False
prepare_multiplex_fileset traces-negative False WARNING True
prepare_multiplex_fileset traces-info True INFO False

prepare_multiplex_fileset traces-positive True Warning True
prepare_multiplex_fileset traces-info True Informational True
prepare_multiplex_fileset traces-positive True WARNING True
prepare_multiplex_fileset traces-info True INFO True

echo "Contents of $MULT_FILE:"
cat $MULT_FILE
Expand Down
3 changes: 1 addition & 2 deletions userspace/falco/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ include_directories("${CURL_INCLUDE_DIR}")
include_directories("${YAMLCPP_INCLUDE_DIR}")
include_directories("${DRAIOS_DEPENDENCIES_DIR}/yaml-${DRAIOS_YAML_VERSION}/target/include")

add_executable(falco configuration.cpp formats.cpp fields.cpp rules.cpp logger.cpp falco.cpp)
add_executable(falco configuration.cpp formats.cpp rules.cpp logger.cpp falco_common.cpp falco_engine.cpp falco_outputs.cpp falco.cpp)

target_link_libraries(falco sinsp)
target_link_libraries(falco
Expand All @@ -18,7 +18,6 @@ target_link_libraries(falco
"${YAMLCPP_LIB}")


set(FALCO_LUA_MAIN "rule_loader.lua")
configure_file(config_falco.h.in config_falco.h)

install(TARGETS falco DESTINATION bin)
Expand Down
3 changes: 0 additions & 3 deletions userspace/falco/config_falco.h.in
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,4 @@
#define FALCO_INSTALL_CONF_FILE "/etc/falco.yaml"
#define FALCO_SOURCE_LUA_DIR "${PROJECT_SOURCE_DIR}/userspace/falco/lua/"


#define FALCO_LUA_MAIN "${FALCO_LUA_MAIN}"

#define PROBE_NAME "${PROBE_NAME}"
38 changes: 24 additions & 14 deletions userspace/falco/configuration.cpp
Original file line number Diff line number Diff line change
@@ -1,22 +1,32 @@
#include "configuration.h"
#include "config_falco.h"
#include "sinsp.h"
#include "logger.h"

using namespace std;

falco_configuration::falco_configuration()
: m_config(NULL)
{
}

falco_configuration::~falco_configuration()
{
if (m_config)
{
delete m_config;
}
}

// If we don't have a configuration file, we just use stdout output and all other defaults
void falco_configuration::init(std::list<std::string> &cmdline_options)
void falco_configuration::init(list<string> &cmdline_options)
{
init_cmdline_options(cmdline_options);

output_config stdout_output;
falco_outputs::output_config stdout_output;
stdout_output.name = "stdout";
m_outputs.push_back(stdout_output);
}

void falco_configuration::init(string conf_filename, std::list<std::string> &cmdline_options)
void falco_configuration::init(string conf_filename, list<string> &cmdline_options)
{
string m_config_file = conf_filename;
m_config = new yaml_configuration(m_config_file);
Expand All @@ -26,35 +36,35 @@ void falco_configuration::init(string conf_filename, std::list<std::string> &cmd
m_rules_filename = m_config->get_scalar<string>("rules_file", "/etc/falco_rules.yaml");
m_json_output = m_config->get_scalar<bool>("json_output", false);

output_config file_output;
falco_outputs::output_config file_output;
file_output.name = "file";
if (m_config->get_scalar<bool>("file_output", "enabled", false))
{
string filename;
filename = m_config->get_scalar<string>("file_output", "filename", "");
if (filename == string(""))
{
throw sinsp_exception("Error reading config file (" + m_config_file + "): file output enabled but no filename in configuration block");
throw invalid_argument("Error reading config file (" + m_config_file + "): file output enabled but no filename in configuration block");
}
file_output.options["filename"] = filename;
m_outputs.push_back(file_output);
}

output_config stdout_output;
falco_outputs::output_config stdout_output;
stdout_output.name = "stdout";
if (m_config->get_scalar<bool>("stdout_output", "enabled", false))
{
m_outputs.push_back(stdout_output);
}

output_config syslog_output;
falco_outputs::output_config syslog_output;
syslog_output.name = "syslog";
if (m_config->get_scalar<bool>("syslog_output", "enabled", false))
{
m_outputs.push_back(syslog_output);
}

output_config program_output;
falco_outputs::output_config program_output;
program_output.name = "program";
if (m_config->get_scalar<bool>("program_output", "enabled", false))
{
Expand All @@ -70,7 +80,7 @@ void falco_configuration::init(string conf_filename, std::list<std::string> &cmd

if (m_outputs.size() == 0)
{
throw sinsp_exception("Error reading config file (" + m_config_file + "): No outputs configured. Please configure at least one output file output enabled but no filename in configuration block");
throw invalid_argument("Error reading config file (" + m_config_file + "): No outputs configured. Please configure at least one output file output enabled but no filename in configuration block");
}

falco_logger::log_stderr = m_config->get_scalar<bool>("log_stderr", false);
Expand All @@ -90,21 +100,21 @@ static bool split(const string &str, char delim, pair<string,string> &parts)
return true;
}

void falco_configuration::init_cmdline_options(std::list<std::string> &cmdline_options)
void falco_configuration::init_cmdline_options(list<string> &cmdline_options)
{
for(const string &option : cmdline_options)
{
set_cmdline_option(option);
}
}

void falco_configuration::set_cmdline_option(const std::string &opt)
void falco_configuration::set_cmdline_option(const string &opt)
{
pair<string,string> keyval;
pair<string,string> subkey;

if (! split(opt, '=', keyval)) {
throw sinsp_exception("Error parsing config option \"" + opt + "\". Must be of the form key=val or key.subkey=val");
throw invalid_argument("Error parsing config option \"" + opt + "\". Must be of the form key=val or key.subkey=val");
}

if (split(keyval.first, '.', subkey)) {
Expand Down
16 changes: 9 additions & 7 deletions userspace/falco/configuration.h
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
#pragma once

#include <yaml-cpp/yaml.h>
#include <string>
#include <vector>
#include <list>
#include <iostream>

struct output_config
{
std::string name;
std::map<std::string, std::string> options;
};
#include "falco_outputs.h"

class yaml_configuration
{
Expand All @@ -17,7 +16,7 @@ class yaml_configuration
{
m_path = path;
YAML::Node config;
std::vector<output_config> outputs;
std::vector<falco_outputs::output_config> outputs;
try
{
m_root = YAML::LoadFile(path);
Expand Down Expand Up @@ -118,12 +117,15 @@ class yaml_configuration
class falco_configuration
{
public:
falco_configuration();
virtual ~falco_configuration();

void init(std::string conf_filename, std::list<std::string> &cmdline_options);
void init(std::list<std::string> &cmdline_options);

std::string m_rules_filename;
bool m_json_output;
std::vector<output_config> m_outputs;
std::vector<falco_outputs::output_config> m_outputs;
private:
void init_cmdline_options(std::list<std::string> &cmdline_options);

Expand Down
Loading

0 comments on commit 3bf94ee

Please sign in to comment.