Skip to content

/ osquery

Permalink
master

osquery/CHANGELOG.md

Go to file
@directionless
directionless Changelog 4.5.1 (#6692)
Latest commit 335c013 Oct 5, 2020 History 
Changelog for 4.5.1
7 contributors

Users who have contributed to this file

@directionless @theopolis @Smjert @mike-myers-tob @zwass @sharvilshah @reedloden
598 lines (474 sloc) 48.2 KB
Raw Blame

osquery Changelog

4.5.1

Git Commits

Under the Hood improvements

  • Improve carver tests by faking postCarve (#6659)
  • Emit an error during carving, if the carve SQL function is disabled (#6658)
  • Update carves specs to allow full scan (#6657)
  • Update carves table to use JSON (#6656)
  • Improve performance and accuracy of Windows registry querying (#6647)
  • Refactor ephemeral database plugin into core and simplify tests (#6648)

Table Changes

  • Support for Office MRU (most recently used) entries (#6587)
  • Implement configurable timeout through WHERE clause on curl_certificate (#6641)
  • Add atom_packages table spec to window (#6649)
  • Add signature information to authenticode table on windows (#6677)
  • Add additional AWS regions (#6666)

Bug Fixes

  • Fix container overflow in curl_certificate (#6664)
  • Fix handling of invalid array bound error with EvtNext function (#6660)
  • Fix wmi_bios_info table searching (#5246)
  • Fix image column within drivers table on Windows (#6652)
  • Fix windows dirPathsAreEqual to use the documented way (#6690)
  • Fix incorrect stat() return checking within process_events (#6694)
  • Always flush stdout when called with --help (#6693)

Documentation

  • Document max scheduled query interval (#6683)
  • Update documentation around build steps (#6681)
  • Documentation copy editing (#6676, #6665, #6662)
  • Add 4.5.0 CHANGELOG (#6646)
  • Add 4.5.1 CHANGELOG (#6692)

Build

  • Improve flaky python test handling (#6654)
  • Restore test_osqueryi (#6631)
  • Limit osqueryd CPU usage to 20% in systemd unit file (#6644)
  • Improve flaky test_osqueryi (#6688)
  • Add cppcheck support to macOS (#6685)

Hardening

  • Add exception catching for table execution (#6689)

4.5.0

Git Commits

We would like to thank all of the contributors working on bootstrapping the ARM64/AARCH64 support and Windows 32bit support. Additionally, we want to thank those working on Unicode support and all the bug fixes, documentation improvements, and new features. Thank you! 👏

New Features

  • ARM64/AARCH64 beta support for Linux (#6612)
  • Windows 32bit support (#6543)
  • Fix buildup of RocksDB SST files (#6606)

Under the Hood improvements

  • Remove selectAllFrom from Linux process_events callback (#6638)
  • Remove database read only concept (#6637)
  • Move database initialization retry logic into DB API (#6633)
  • Move osquery/include files into respective CMake targets (#6557)
  • Memoize EventFactory::getType (#6555)
  • Update schedule counter behavior (#6223)
  • Define UNICODE and _UNICODE preprocessors for windows (#6338)
  • Add WMI utility function to convert datetime to FILETIME (#5901)
  • Move osquery shutdown logic outside of Initializer (#6530)

Table Changes

  • Support for Windows Background Activity Moderator (#6585)
  • Add apparmor_events table to Linux (#4982)
  • Add sigurl column to get YARA signatures from an HTTPS server (#6607)
  • Add sigrules column to pass YARA signatures within queries (#6568)
  • Add non-evented table for querying windows_event_log (#6563)
  • Improve chassis_types and security_breach columns within chassis_info (#6608)
  • Fix bool type usage in powershell_events (#6584)
  • Add FileVersionRaw column to file table for Windows (#5771)
  • Enable YARA table on Windows (#6564)
  • Add dns_cache table for Windows (#6505)
  • Add support for processing KILL syscall (#6435)
  • Add startup_items table for Linux (#6502)
  • Add shimcache table (#6463)
  • Refactor shell_history to use generators (it will use less memory) (#6541)

Bug Fixes

  • Set thread names correctly on macOS and Linux (#6627)
  • Apply --scheduler_timeout correctly (#6618)
  • Add check for character_frequencies size (#6625)
  • Fix race in removing external TablePlugins (#6623)
  • Force shell to disable watchdog and logger (#6621)
  • Return early within the shell if relative flags are used (#6605)
  • Apply watcher delay each time the worker is started (#6604)
  • Set global output function for Thrift (#6592)
  • Fix incorrect readFile params in createPidFile (#6578)
  • Fix call to LocalFree on deinit ptr inside getUidFromSid (#6579)
  • Fix readFile to observe requested read size (#6569)
  • Replace fstream within syslog_events with a custom non-blocking getline (#6539)
  • Only fire events if a publisher exists (#6553)
  • Fix Leak in psidToString (#6548)
  • Fix memory leaks in rpm_package_files (#6544)
  • Change "Symlink loop" message from warning to verbose (#6545)

Documentation

  • Update process auditing docs schema link (#6645)
  • Improve descriptions for the processes table (#6596)
  • Replace slackin with Slack shared invite (#6617)
  • Update copyright notices to osquery foundation (#6589, #6590)

Build

  • Fix Windows build by removing non existing C11 conformance (#6629)
  • Remove ExecStartPre from systemd service unit (#6586)
  • Fix pip upgrade warning within CI (#6576)
  • Detect MAJOR_IN_SYSMACROS/MKDEV for librpm in CMake (#6554)
  • Add curl_certificate tests (#5281)
  • Update YARA library to 4.0.2 (#6559)
  • Improve testing assumptions and flush fsevents when stopping (#6552)
  • Fix the test utility to allow Windows profiling (#6550)
  • Support ASAN for boost coroutine2 using ucontext (#6531)
  • Update instructions for CPack package building (#6529)
  • Use specific RPM variables to set the package name (#6527)
  • Update compiler version used to v142 within Azure (#6528)

Hardening

  • Restore PIE support being dropped on Linux (#6611)

4.4.0

Git Commits

New Features / Under the Hood improvements

  • Implement container access from tables on Linux (#6209, #6485)
  • Update language to use 'allow list' and 'deny list' (#6489, #6487, #6488, #6493)
  • macos: Automatic configuration of the OpenBSM audit rules (#6447)
  • macos: Add polling to OpenBSM publisher (#6436)
  • Add messages to distributed query results (#6352)
  • Implement event batching support for Windows tables (#6280)

Table Changes

  • Add container access to the os_version table (#6413)
  • Add container access to DEB, RPM, NPM packages tables (#6414)
  • Add fields auid, fs{u,g}id, s{u,g}id to auditd based tables (#6362)
  • Improve apt_sources resiliency (#6482)
  • Make file and hash container columns hidden (#6486)
  • Add 'maintainer', 'section', 'priority' columns to deb_packages (#6442)
  • Add 'vendor', 'package_group' columns to rpm_packages (#6443)
  • Add 'arch' column to os_version (#6444)
  • Add 'board_xxx' columns to system_info table (#6398)
  • Windows: omit non-interactive sessions from logged_in_users (#6375)
  • Fixes to package_bom table (#6457, #6461)
  • Add chassis_info table for windows (#5282)
  • Add Azure tables (#6507)

Bug Fixes

  • Update hash cache inode number in query cache (#6440)
  • Only explode registry key if it can be tokenized (#6474)
  • Change ErrorBase::takeUnderlyingError to non const (#6483)
  • Use RapidJSON to fix event format results and the Kafka Logger (#6449)
  • Correct the 'cwd' and 'root' columns of processes table on Windows (#6459)
  • Correct some SQLite types (#6392)
  • Partial fix for md_devices issue (#6417)
  • Fix the handling of empty args strings, on Windows (#6460)
  • Refactor shutdown logging, and remove explicit syslog call (#6376)
  • Change the Windows registry LIKE path constraint to filter recursively (#6448)
  • Use sync resolve within http client (#6490)
  • Fix typed_row table caching (#6508)
  • Do not use system proxy for AWS local authority (#6512)
  • Only populate table cache with star-like selects (#6513)

Documentation

  • Update osquery security policy (#6425)
  • Updating changelog for 4.3.0 release (#6387)
  • Improve the new table tutorial (#6479)
  • Add Auto Table Construction to docs (#6476)
  • Add documentation for enabling socket_events on macOS (#6407)
  • Update winbaseobj table description (#6429)
  • Fixing the description of failed_login_count from account_policy_data (#6415)
  • Remove references to brew in macOS install (#6494)
  • Add note to bump the Homebrew cask (#6519)
  • Updating docs on cpack usage to include Chocolatey (#6022)
  • Changelog for 4.4.0 (#6492, #6523))

Build

  • Fix Userassist.test_sanity test sometimes failing (#6396)
  • Drop the facebook and source_migration layers (#6473)
  • Move ssdeep-cpp to source_migration (#6464)
  • Move smartmontools to source_migration (#6465)
  • Build augeas from source on macOS (#6399)
  • Build lldpd from source on macOS (#6406)
  • Build linenoise-ng from source on macOS and Windows (#6412)
  • Build sleuthkit from source on macOS (#6416)
  • Build popt from source on macOS (#6409)
  • Fix libelfin build on ossfuzz and LLVM/Clang 10 (#6472)
  • Use the patched libelfin version (#6480)
  • codegen: Port Jinja2 to Templite (#6470)
  • Pass the minimum macOS SDK version to openssl only if explicitly set (#6471)
  • Add git-lfs as dep for macOS build in documentation (#6384)
  • Update openssl from 1.1.1f to 1.1.1g (#6432)
  • Build openssl with the macOS SDK version taken from CMake (#6469)
  • Do not install openssl docs (#6441)
  • Update build configuration of ReadTheDocs (#6434, #6456)
  • Link librdkafka on Windows (#6454)
  • Build sleuthkit on Windows (#6445)
  • Add nupkg cpack build option and update Windows deployment script (#6262)
  • Fix rpm and deb package name format (#6468)
  • Fix atom_packages, processes, rpm_packages tests (#6518)
  • Fixes and cleanup for Windows compiler flags (#6521)
  • Correct macOS framework linking (#6522)

Security Issues

  • Disable openssl compression support (#6433)

Hardening

  • Use LOAD_LIBRARY_SEARCH_SYSTEM32 for LoadLibrary (#6458)

4.3.0

Git Commits

New Features / Under the Hood improvements

  • Change verbosity of scheduled query execution messages from INFO to verbose only (#6271)
  • Updated the unwanted-chrome-extensions queries to include all users, not the osquery process owner only (#6265)
  • Check for errors in the return status of the extension tables and report them (#6108)
  • First steps to properly support UTF8 strings on Windows (#6190)
  • Display the undelying API error string when udev monitoring fails (#6186)
  • Add the path column to the ATC generate specs (#6278)
  • Add Kafka support to Microsoft Windows (#6095)
  • Log a warning message if osquery fails to get the service description on Microsoft Windows (#6281)
  • Make AWS kinesis status logging configurable (#6135)
  • Add an integration test for the disk_info table (#6323)
  • Use -1 for missing ppid in the process_events table (#6339)
  • Remove error when converting empty numeric rows (#6371)
  • Change verbosity from ERROR to INFO of access failures to system processes on Microsoft Windows (#6370)
  • Make possible to get verbose messages from the dispatcher service management on Microsoft Windows too (#6369)

Build

  • Fix codegen template for extension group (#6244)
  • Update SQLite from 3.30.1-1 to 3.31.1 (#6252)
  • Update the osquery-toolchain to version 1.1.0 which uses LLVM/Clang 9.0.1 (#6315)
  • Update openssl to version 1.1.1f (#6302, #6359)
  • Simplify formula-based third party libraries build (#6303)
  • Removed the Buck build system (#6361)

Bug Fixes

  • Fix CFNumber conversion when the type was a Float64/32 instead of a Double (#6273)
  • Fix duplicate results being returned by the chrome_extensions table (#6277)
  • Fix flaky ProcessOpenFilesTest.test_sanity (#6185)
  • Fix the --database_dump flag for RocksDB not outputting anything (#6272)
  • Fix the pci_devices table pci ids extraction in non-existing paths (#6297)
  • Fix parsing an invalid decorators config (#6317)
  • Fix flaky TLSConfigTests.test_runner_and_scheduler (#6308)
  • Fix chromeExtensions.test_sanity (#6324)
  • Fix broken Unicode filename searches on Microsoft Windows (#6291)
  • Fix a use-after-free when sqlite attempts to access the entire rows data at the end of a query (#6328)
  • Keep proc instance for test_base and test_osqueryd (#6335)
  • Fix osquery not exiting when given check or dump requests (#6334)
  • Fix process table cmdline parsing (#6340)
  • Fix a crash when parsing files with libmagic (#6363)
  • Fix a sporadic readFile API failure when using non-blocking I/O (#6368)
  • Fix the MSI package not always installing in the system drive by default (#6379)
  • Ensure the extensions uuid is never 0 (#6377)
  • Fix a race condition making the watcher act as a worker on Microsoft Windows (#6372)
  • Fix extensions tables detaching which was sometimes failing (#6373)
  • Fix an issue with extensions re-registration (#6374)
  • Fix a crash due to a race condition in accessing the iokit port on Darwin (Apple OS X) (#6380)

Hardening

  • Limit SQL functions regex_match and regex_split regex size (#6267)
  • Prevent a stack overflow when parsing deeply nested configs (#6325)

Table Changes

  • Added table chrome_extension_content_scripts to All Platforms (#6140)
  • Added table docker_container_fs_changes to POSIX-compatible Plaforms (#6178)
  • Added table windows_security_center to Microsoft Windows (#6256)
  • Added many new tables to Linux to query lxd (#6249)
  • Added table screenlock to Darwin (Apple OS X) (#6243)
  • Added table userassist to Microsoft Windows (#5539)
  • Added column status (TEXT) to table deb_packages (#6341)
  • Added many new columns to the curl_certificate table (#6176)
  • Added table socket_events to Darwin (Apple OS X) (#6028)
  • Added table hvci_status, previously inadvertly left out from the build, to Microsoft Windows (#6378)

4.2.0

Git Commits

New Features / Under the Hood improvements

  • TLS Testing infrastructure has been overhauled (#6170)
  • Boost regex has been replaced with std (#6236)
  • community_id_v1 added as a SQL function (#6211)

Build

  • Fix format checking on Windows (#6188)
  • Fix format folder exclusions for build checks (#6201)
  • Fix the linking for extensions in build (#6219)
  • Fix build to include windows optional features table (#6207)

Security Issues

  • [CVE-2020-1887] osquery does not properly verify the SNI hostname (#6197)

Bug Fixes

  • Carver no longer returns empty carves for hidden files (#6183)
  • Address a race in the Dispatcher logic (#6145)
  • Fix validation in 'last' table (#6147)
  • Fix flaky logger testing (#6171)
  • Fix JSON format assumptions in file_paths parsing (#6159)
  • Fix windows WMI BSTR to be wstrings (#6175)
  • Fix windows string <-> wstring conversion functions (#6187)
  • Enable more intelligent path expansion on Windows (#6153)
  • Fix heap buffer overflow in callDoubleFunc and powerFunc (#6225)

Table Changes

  • Added table firefox_addons to All Platforms (#6200)
  • Added table ssh_configs to All Platforms (#6161)
  • Added table user_ssh_keys to All Platforms (#6161)
  • Added table mdls to Darwin (Apple OS X) (#4825)
  • Added table hvci_status to Microsoft Windows (#5426)
  • Added table ntfs_journal_events to Microsoft Windows (#5371)
  • Added table docker_image_layers to POSIX-compatible Plaforms (#6154)
  • Added table process_open_pipes to POSIX-compatible Plaforms (#6142)
  • Added table apparmor_profiles to Ubuntu, CentOS (#6138)
  • Added table selinux_settings to Ubuntu, CentOS (#6118)
  • Added column lock_status (INTEGER_TYPE) to table bitlocker_info (#6155)
  • Added column percentage_encrypted (INTEGER_TYPE) to table bitlocker_info (#6155)
  • Added column version (INTEGER_TYPE) to table bitlocker_info (#6155)
  • Added column optional_permissions (TEXT_TYPE) to table chrome_extensions (#6115)
  • Removed table firefox_addons from POSIX-compatible Plaforms (#6200)
  • Removed table ssh_configs from POSIX-compatible Plaforms (#6161)
  • Removed table user_ssh_keys from POSIX-compatible Plaforms (#6161)

4.1.2

Git Commits

New Features / Under the Hood improvements

  • Add more tests throughout the codebase (#5908), (#6071), (#6126)
  • The chrome_extensions table now supports Chromium and Brave (#6126)

Build

  • Require Python 3.5 and greater (#6081), (#6120)
  • Prepare Python tests for CI (lots of effort!) (#6068)
  • Restore osqueryd integration test (#6116)

Bug Fixes

  • Continue to use com.facebook.osquery.plist for Launch Daemon configuration (#6093)
  • Update systemd service to use KillMode=control-group (#6096)
  • RPM and DEB packages both have post-install scripts to reload systemd (#6097)
  • Update Windows package build script to include cert bundle (#6114)
  • Update table specs to fix constraints passing (#6103), (#6104), (#6105), (#6106), (#6122)

Table Changes

  • Added tables azure_instance_tags and azure_instance_metadata to Linux and Microsoft Windows (#5434)
  • Added column install_time (INTEGER_TYPE) to table rpm_packages (#6113)
  • Added column bsd_flags (TEST_TYPE) to table file on Darwin (#5981)

4.1.1

Git Commits

New Features / Under the Hood improvements

  • Improve nvram table to use input variable names (#6053)
  • Improve apt_sources source detection (#6047)
  • Change atom_packages to use user constraints (#6052)
  • Re-enable required-column warning messages (#6038)

Build

  • Migrate several libraries to the CMake source layer (#5902), (#6023)
  • Update SQLite from 3.29.0-3 to 3.30.1-1 (#6020)
  • Recommend building with MacOS 10.11 SDK (#6000)

Bug Fixes

  • Fix Linux audit incorrect read and handle leak (#5959)
  • Change "logNumericsAsNumbers" to "numerics" logger top-level key (#6002)
  • Restore INDEX behavior for extensions (#6006)
  • Fix potential JSON parsing issues in ATC plugin (#6029)
  • Avoid scanning special files with YARA (#5971)
  • Fix use-after-move in YARA subscriber (#6054)
  • Handle relative redirects in internal HTTP clients (#6049)
  • Apply options config parsing before others (#6050)

Table Changes

  • Added table windows_optional_features to Microsoft Windows #5991)

4.1.0

Git Commits

New Features / Under the Hood improvements

Build

Hardening

Bug Fixes

  • Set Windows MSI ErrorControl to normal instead of critical (#5818)
  • Wrap flagfile with quotes for Windows install flag (#5824)
  • Improve submodule usages in CMake (#5850), (#5880), (#5892), (#5897), (#5907)
  • Improve locking support in internal APIs (#5841), (#5906), (#5943), (#5944)
  • Fixes for macOS application layer firewall tables (#5378)
  • Fixes within BPF event tables (#5874)
  • Refactor and improve PCI device tables on Linux (#5446)
  • Implement PID indexing on Windows processes table (#5919)
  • Improve WHERE IN() performance (#5924), (#5938)
  • Improve the internal HTTP client (#5891), (#5946), (#5947)
  • Fix Windows version codename lookup (#5887)

Table Changes

  • Added table alf_services to Darwin (Apple OS X) (#5378)
  • Added table connectivity to Microsoft Windows (#5500)
  • Added table default_environment to Microsoft Windows (#5441)
  • Added table windows_security_products to Microsoft Windows (#5479)
  • Added column platform_mask (INTEGER_TYPE) to table osquery_info (#5898)

4.0.2

This release fixes crashes identified in 4.0.1. There are no changes in functionality.

Git Commits

Bug Fixes

  • Fix configuration of AWS libraries to address crash in Linux (#5799)
  • Remove RocksDB optimization causing crash (#5797)

4.0.1

This release has two major focuses. It is the first release since osquery transitioned to a Linux Foundation project.

It features a heavily reworked build system. This aims to provide flexibility and stability.

Git Commits

New Features / Under the Hood improvements

  • Linux Audit process_events Implement support for fork/vfork/clone/execveat (#5701)
  • New SQLite function regex_match to match across columns (#5444)
  • LRU cache for syscall tracing (#5521)
  • Basic tracing via eBPF on Linux (#5403, #5386, #5384)
  • Experimental kill and setuid syscall tracing in Linux via eBPF (#5519)
  • New eventing (ev2) framework (#5401)
  • Improved table performance profiles (#5187)
  • macOS query pack: detect SearchAwesome malware (#5713)
  • macOS query pack: detect when a process is tapping keyboard event (#5345)

Build

Hardening

  • Link binaries with Full RELRO on Linux (#5748)
  • Remove FTS features from SQLite (#5703, #5702)
  • Fix SQLite API usage errors (#5551)
  • Fix issues reported by ASAN (#5665)
  • Handle bad FDs in md_tables (#5553)
  • Fix lock resource leak in events/syslog (#5552)
  • Fix memory leak in macOS keychain_items and extended_attributes tables (#5550, #5538)
  • Fix memory leak in genLoggedInUsers (Windows). Update WTSFreeMemoryEx to WTSFreeMemory (#5642)
  • Fix potential null dereferences in smbios_tables (#5332)
  • Fix osquery exiting with wrong status (3824c2e6)
  • Add additional install and uninstall flag incompatibility check (85eb77a0)
  • Fix warning with constants initialisation in magic (2a624f2f)
  • Fix sign compare warning in file_compression (b93069b3)
  • Refactored logical_drives table on Windows (#5400)
  • Refactored core/windows/wmi to use smart pointers (#5492)
  • Fixed various potential crashes in the virtual table implementaion (6ade85a5)
  • Increase the amount of MaxRecvRetries for Thrift sockets (#5390)

Bug Fixes

  • Fix the reading of the serial of a certificate (little-endian big int) (#5742)
  • Fix bugs and update pathname variables in MSI package build script (#5733)
  • Fix registry table exception closing an uninitialized key handle (#5718)
  • Config views are now recreated on startup (#5732)
  • Change MSI Service Error handling on Windows (#5467)
  • Allow mounting SQLite DBs using WAL journaling with ATC (#5525, #5633)
  • Fix mount table interacting with direct autofs (#5635)
  • Fix HTTP Host Header to include port (#5576)
  • Various fixes to the Windows certificates table and expansion to include Personal certificates (#5697), (#5696), (#5640), (#5631)
  • Add optimization back to macOS users and groups (#5684)
  • Do not return a row for macOS battery if no data is present (#5650)
  • Fix several integer conversions in process_ops (#5614)
  • Include weekends on the kernel_panics table (#5298)
  • Fix key_strength bug for Windows certificates table (#5304)
  • The interface column of routes table could be empty on Windows (bcf0ab8e)
  • The name column of programs table could be empty on Windows (7bceba4b)
  • Fix disable_watcher flag (08dc11b7)
  • Populate path column correctly in firefox_addons table (#5462)
  • Fix numeric monitoring plugin not being registered (#5484)
  • Fix wrong error code returned when querying the Windows registry (#5621)
  • Fix logical_drives boot partition detection (#5477)
  • Replace sync calls by async within the HTTP client implementation (#5606)
  • Fix RocksDB crash related to OptimizeForSmallDb (a31d7582)
  • Fix bug in table column data validator (e3037331)
  • Fix random port problem (a32ed7c4)
  • Refactor battery table and return information even if advanced information is missing (6a64e353)

Table Changes

  • Added table ibridge_info on macOS (Notebooks only) (#5707)
  • Added table running_apps on macOS (#5216)
  • Added table atom_packages on macOS and Linux (6d159d40)
  • Remove EC2 tables on Windows (#5657)
  • Add column win_timestamp to time table on Windows (3bbe6c51)
  • Add column is_hidden to users and groups table on macOS (#5368)
  • Add column profile to chrome_extensions table (#5213)
  • Add column epoch to rpm_packages table on Linux (#5248)
  • Add column sid to logged_in_users table on Windows (#5454)
  • Add column registry_hive to logged_in_users table on Windows (#5454)
  • Add column sid to certificates table on Windows (#5631)
  • Add column store_location to certificates table on Windows (#5631)
  • Add column store to certificates table on Windows (#5631)
  • Add column username to certificates table on Windows (#5631)
  • Add column store_id to certificates table on Windows (#5631)
  • Add column product_version to file table on Windows (#5431)
  • Add column source to sudoers table on POSIX systems (#5350)
You can’t perform that action at this time.