Repository navigation
using osqueryd
osqueryd is the host monitoring daemon which is included with osquery.
osqueryd allows you to schedule queries for execution across your infrastructure. The daemon aggregates query results over time and generates logs which indicate state change in your infrastructure. These logs can be used to gain insight into changes you may be interested in.
By default, queries are read from a local JSON file. If you're interested in using your existing configuration management system with osqueryd, read the registering config plugins guide.
Here is an example config:
{
"scheduledQueries": [
{
"name": "macosx_kextstat",
"query": "SELECT * FROM kextstat;",
"interval": 10
},
{
"name": "foobar",
"query": "SELECT foo, bar, pid FROM foobar_table;",
"interval": 600
}
]
}This config tells osqueryd to schedule two queries: "macosx_kextstat" and "foobar".
- the "name" must be unique
- the "interval" specifies query frequency, in seconds
The first query will document changes to an OSX host's kernel extensions, with a query interval of 10 seconds.
The results of your query are cached on disk via RocksDB. On first query run, all of the results are stored in RocksDB. On subsequent runs, only result-set changes are logged to RocksDB.
Running osqueryd --help will tell you the default locations of logs. Currently, this is /var/log/osquery/, but this is subject to change.
There are two sets of logs:
- Application logs (info, warning, error and fatal)
- query results logs
You can see this illustrated here:
$ ls -l /var/log/osquery/
total 24
lrwxr-xr-x 1 root wheel 77 Sep 30 17:37 osqueryd.INFO -> osqueryd.hostname.root.log.INFO.20140930-173730.97830
-rw-r--r-- 1 root wheel 1226 Sep 30 17:37 osqueryd.hostname.root.log.INFO.20140930-173730.97830
-rw-r--r-- 1 root wheel 388 Sep 30 17:37 osqueryd.results.log
Application logs are generated by the glog logging framework.
As the above directory listing reveals, "osqueryd.INFO" is a symlink to the most recent execution's INFO log. The same is true for the warning, error and fatal logs.
For more information on the format of GLOG logs, please refer to the GLOG documentation.
The results of your scheduled queries are logged to the "results log".
Each log line is a JSON string that indicates what data has been added/removed by which query.
There are two format options, "single" and "batched"
This is the default. Each log line represents a state change.
This format works best for log aggregation systems like Logstash or Splunk.
Example output of SELECT name, path, pid FROM processes; (whitespace added for readability)
{
"action": "added",
"columns": {
"name": "osqueryd",
"path": "/usr/local/bin/osqueryd",
"pid": "97830"
},
"name": "processes",
"hostname": "hostname.local",
"calendarTime": "Tue Sep 30 17:37:30 2014",
"unixTime": "1412123850"
}{
"action": "removed",
"columns": {
"name": "osqueryd",
"path": "/usr/local/bin/osqueryd",
"pid": "97650"
},
"name": "processes",
"hostname": "hostname.local",
"calendarTime": "Tue Sep 30 17:37:30 2014",
"unixTime": "1412123850"
}This tells us that a binary called "osqueryd" was stopped and a new binary with the same name was started (note the different pids). The data is generated by keeping a cache of previous query results and only logging when the cache changes. If no new processes are started or stopped, the query won't log any results.
If a query identifies multiple state changes, the batched format will include all results in a single log line. If you're programmatically parsing lines and loading them into a backend datastore, this is probably the best solution.
To enable batch log lines, launch osqueryd with the --log_result_events=false argument.
Example output of SELECT name, path, pid FROM processes; (whitespace added for readability)
{
"diffResults": {
"added": [
{
"name": "osqueryd",
"path": "/usr/local/bin/osqueryd",
"pid": "97830"
}
],
"removed": [
{
"name": "osqueryd",
"path": "/usr/local/bin/osqueryd",
"pid": "97650"
}
]
},
"name": "processes",
"hostname": "hostname.local",
"calendarTime": "Tue Sep 30 17:37:30 2014",
"unixTime": "1412123850"
}As previously mentioned, osqueryd's application logs are written to the filesystem. If you want to aggregate and collect these logs from the filesystem, you can use an agent like fluentd, rsyslog, logstash, or Splunk universal forwarder.
For the results logs, if you prefer to directly transport your logs to something like Scribe or Flume, you can utilize the plugin architecture and use a logger plugin.