Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logscale: add logscale() destination #4472

Merged
merged 2 commits into from
May 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
23 changes: 23 additions & 0 deletions news/feature-4472.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
Sending messages to CrowdStrike Falcon LogScale (Humio)

The `logscale()` destination feeds LogScale via the [Ingest API](https://library.humio.com/falcon-logscale/api-ingest.html#api-ingest-structured-data).

Minimal config:
```
destination d_logscale {
logscale(
token("my-token")
);
};
```
Additional options include:
* `url()`
* `rawstring()`
* `timestamp()`
* `timezone()`
* `attributes()`
* `extra-headers()`
* `content-type()`


__TODO: Add Andreas Friedmann and Ryan Faircloth to Contributors!!!!!__
1 change: 1 addition & 0 deletions scl/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ set(SCL_DIRS
loadbalancer
loggly
logmatic
logscale
mariadb
mbox
netskope
Expand Down
1 change: 1 addition & 0 deletions scl/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ SCL_SUBDIRS = \
loadbalancer \
loggly \
logmatic \
logscale \
mariadb \
mbox \
netskope \
Expand Down
76 changes: 76 additions & 0 deletions scl/logscale/logscale.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
#############################################################################
# Copyright (c) 2023 Attila Szakacs
#
# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License version 2 as published
# by the Free Software Foundation, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
#
# As an additional exemption you are allowed to compile & link against the
# OpenSSL libraries as published by the OpenSSL project. See the file
# COPYING for details.
#
#############################################################################

# Batching best practices: https://library.humio.com/falcon-logscale/api-ingest.html#api-ingest-best-practices

@requires json-plugin


# https://library.humio.com/falcon-logscale/api-ingest.html#api-ingest-structured-data
block destination logscale(
url("https://cloud.humio.com")
token()

rawstring("${MESSAGE}")
timestamp("${S_ISODATE}")
timezone("")
attributes("--scope rfc5424 --exclude MESSAGE --exclude DATE --leave-initial-dot")

batch_lines(1000)
batch_bytes(1024kB)
batch_timeout(1)
workers(8)
timeout(10)

content_type("application/json")
extra_headers("")
use_system_cert_store(yes)
...)
{

@requires http "The logscale() driver depends on the syslog-ng http module, please install the syslog-ng-mod-http (Debian & derivatives) or the syslog-ng-http (RHEL & co) package"

http(
url("`url`/api/v1/ingest/humio-structured")
headers(
"Authorization: Bearer `token`"
"Content-Type: `content_type`"
`extra_headers`
)
delimiter(",")
body-prefix('[{"events":[')
body('$(format-json --scope none --omit-empty-values
rawstring=`rawstring`
timestamp=`timestamp`
timezone=`timezone`
attributes=$(if ("`attributes`" ne "") $(format-json --scope none `attributes`) ""))'
)
body-suffix(']}]')
batch-lines(`batch_lines`)
batch-bytes(`batch_bytes`)
batch-timeout(`batch_timeout`)
timeout(`timeout`)
workers(`workers`)
use_system_cert_store(`use_system_cert_store`)
`__VARARGS__`
);
};
1 change: 1 addition & 0 deletions tests/copyright/policy
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,7 @@ modules/python/python-flags.(c|h)$
modules/python/tests/test_python_options.c$
scl/fortigate/.*\.conf$
scl/cee/.*\.conf$
scl/logscale/logscale\.conf$
scl/mariadb/.*\.conf$
scl/python/python-modules\.conf$
scl/splunk/splunk\.conf$
Expand Down