Trail of Bits osquery Extensions
Extensions are a type of osquery add-on that can be loaded at runtime to provide new virtual tables. The extensions interface allows organizations to implement proprietary detection methods, or address their individual needs. Here, we use it to demonstrate other pioneering use cases of osquery.
In extensions, we can add capabilities that go beyond what would be possible in osquery core. Trail of Bits has developed extensions to provide tables that can manage service configurations as well as view them, or that can cross-check information on the host with external third-party services.
To learn more about osquery extensions development and why developing outside of 'core' is encouraged for demonstrating new use cases or novel functionality, view our talk (slides, video) from QueryCon 2018.
|efigy||Integrates osquery with the Duo Labs EFIgy API to determine if the EFI firmware on your Mac fleet is up-to-date.||macOS|
|santa||Integrates osquery with the Santa application whitelisting solution. Check DENY events and manage the whitelist/blacklist rules.||macOS|
|fwctl||Provides osquery with the ability to view and manage the OS-native firewall rules and
||macOS, Linux, Windows|
|ntfs_forensics||Provides osquery with NTFS-specific forensic information for incident responders.||Windows|
|windows_sync_objects||Provides osquery with the ability of listing and locking Windows synchronization objects (mutants, events, semaphores).||Windows|
|darwin_unified_log||Provides an event driven table that contains entries from the unified system log on MacOS.||macOS|
|iptables||Provides a superset of the information supplied by the default
|(more to come)||...||...|
- network_monitor: Provides an event-based table that lists DNS requests performed by the endpoint. Uses libpcap and Pcap++ to capture and parse network requests.
Note: the releases page has download links for our extensions. The instructions below are only necessary for those interested in building from source.
Boost library (all platforms)
The full Boost library is required to build the Trail of Bits osquery extensions. Unfortunately, the version of Boost that osquery core builds against does not currently include all of the necessary Boost components.
We've submitted a PR to osquery core, to fix the issue: https://github.com/facebook/osquery/pull/4339
For the time being, you can include the entire Boost library in the osquery core build by applying the commits from that branch to your local copy of the osquery repository.
For Linux or macOS, you only need to apply the patch as follows, and nothing else is necessary to build the full Boost dependency for these operating systems:
# assuming you have checked out the osquery core code to './osquery' cd osquery curl https://patch-diff.githubusercontent.com/raw/facebook/osquery/pull/4339.patch | git am
Additional steps are required for Windows (see below).
You will additionally need to have:
- Xcode (installed from the App Store)
- openssl and curl (install from Homebrew:
brew install openssl curl)
- a user account with sudo (in order to run the script that installs the other osquery build dependencies)
For Windows only, you must also rebuild the Boost package from source (due to a bug in the binaries that were uploaded to the S3 repository). When doing this, you must work in a folder close to the root of the drive, like
C:\Projects\osquery. This is because the Boost script will generate many nested folders, and often hits the path size limit and fails, a problem not very apparent to the Chocolatey package manager.
After cloning the osquery repository and cherry-picking the full set of commits from the PR mentioned above, the remaining steps are:
- Run the following script once:
- Uninstall the boost-msvc14 package that osquery's scripts just installed:
choco uninstall boost-msvc14
- Build the Boost package from source (at a Powershell prompt):
- Enter the folder where the package was created:
choco install -s . .\boost-msvc220.127.116.11-r2.nupkgto install the Boost package you just built.
Remember, if you'd just rather have a binary package, you'll find one in the releases page.
At a high-level, the steps are:
- Clone the osquery and osquery-extensions repositories
- Symlink the osquery-extensions folder into
- Run the osquery scripts to install dependencies and build osquery, which also builds the extensions
Additionally, the osquery-extensions repository has git submodules that need to be pulled.
Here are example steps for each platform:
macOS or Linux
Note: Due to the current way
bison is built, you may need to add some symlinks to your system. This is only necessary if you want to build the
network_monitor extension, or are building all extensions.
/home/linuxbrew/.linuxbrew/Cellar -> /usr/local/osquery/Cellar /home/linuxbrew/.linuxbrew/opt -> /usr/local/osquery/opt
Then you can run the following commands.
cd /src git clone https://github.com/facebook/osquery.git git clone https://github.com/trailofbits/osquery-extensions.git cd /src/osquery-extensions git submodule init git submodule update --recursive cd /src/osquery ln -s /src/osquery-extensions /src/osquery/external/extension_trailofbits make sysprep make deps # If using macOS, replace `nproc` with `sysctl -n hw.ncpu` make -j `nproc`
cd \Projects git clone https://github.com/facebook/osquery.git git clone https://github.com/trailofbits/osquery-extensions.git cd \Projects\osquery-extensions git submodule init git submodule update --recursive # Symbolically link the extensions repo into the osquery core repo: mklink /D "\Projects\osquery\external\extension_trailofbits" "\Projects\osquery-extensions" # From a shell with Administrator privileges: cd \Projects\osquery .\tools\make-win64-dev-env.bat .\tools\make-win64-binaries.bat # To additionally build the extensions, now: cd build\windows10 cmake --build . --config Release --target trailofbits_osquery_extensions
If you see the following warning, it can be ignored:
-- Cannot find Doxygen executable in path
Specifying the extensions to be built
By default, all of the extensions in our repository are built into one executable. It's also possible to select which extensions to build, using the
TRAILOFBITS_EXTENSIONS_TO_BUILD environment variable and specifying a comma separated list of extension names. For example, if you wish to build both the
fwctl extensions on Windows, you can set it to:
$env:TRAILOFBITS_EXTENSIONS_TO_BUILD = "windows_sync_objects,fwctl"
Finding the executable binary
This is where the extension should be available once it has been built:
Running the automated tests
macOS or Linux: once osquery has been built with tests enabled (i.e., without the
SKIP_TESTS variable), enter the build/<platform_name> folder and run the following command:
Windows: tests are not yet supported on Windows.
To quickly test an extension, you can either start it from the osqueryi shell, or launch it manually and wait for it to connect to the running osquery instance.
By default, osquery does not want to load extensions not owned by root. You can either change the ownership of
trailofbits_osquery_extensions.ext to root, or run osquery with the
osqueryi --extension /path/to/trailofbits_osquery_extensions.ext
$ sudo osqueryi --extension osquery/build/darwin/external/trailofbits_osquery_extensions.ext Using a virtual database. Need help, type '.help' osquery> SELECT * FROM efigy; +--------------------+-----------------+--------------------+-------------------+------------+---------------------+ | latest_efi_version | efi_version | efi_version_status | latest_os_version | os_version | build_number_status | +--------------------+-----------------+--------------------+-------------------+------------+---------------------+ | MBP142.0167.B00 | MBP142.0167.B00 | success | 10.12.6 | 10.12.6 | success | +--------------------+-----------------+--------------------+-------------------+------------+---------------------+ osquery>
See the osquery documentation on extensions for further information.
Do you have an idea for an osquery extension? Please file an issue for it. We welcome contributions of bug fixes, bug reports, feature requests, and new extensions. For more information on how you can contribute, see our Contributing Guidelines.
When troubleshooting, ensure you are running osqueryd/osqueryi with the
- As mentioned, if you encounter the following error, you need change the owner of
trailofbits_osquery_extensions.extto be the root account, or else run osquery with the
watcher.cpp:535] [Ref #1382] Extension binary has unsafe permissions:1
The code in this repository is licensed under the Apache 2.0 license.