syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, queueing, SQL & NoSQL.
C Java Python Makefile CMake Shell Other
Latest commit f55f258 Jul 27, 2016 @lbudai lbudai committed on GitHub Merge pull request #1124 from kvch/f/plugin-creator
plugin skeleton creator
Failed to load latest commit information.
Mk cmake: add CMakeLists.txt file to Mk directory May 21, 2016
cmake/Modules cmake/Modules: prefix public domain and BSD license with our LGPL lic… May 21, 2016
contrib contrib/selinux/syslog_ng.sh: Fixed a bug in order to ensure the --he… Jul 21, 2016
debian debian: added add-contextual-data package Jul 25, 2016
dev-utils/plugin_skeleton_creator plugin-skeleton-creator: add author name && year to copyright Jul 15, 2016
doc patterndb-5.xsd: remove test_tags element May 26, 2016
lib Merge pull request #1130 from kvch/nvhandle_32bit_revisited Jul 25, 2016
libtest testutils: add assert_msg_field_unset() function Jul 15, 2016
m4 check_java_support.m4: fix relative path canonicalization Jun 28, 2016
modules add-context-data: added initial version Jul 25, 2016
scl ElasticSearchDestination-v2: Determine default port values by client … Jun 21, 2016
scripts scripts/get-libjvm-path.sh: similar to how it is done in check java supp May 31, 2016
syslog-ng-ctl cmake: add CMakeLists.txt file to syslog-ng-ctl directory May 21, 2016
syslog-ng cmake: add CMakeLists.txt file to syslog-ng directory May 21, 2016
tests Merge pull request #1124 from kvch/f/plugin-creator Jul 27, 2016
tgz2build syslog-ng: set OpenSSL as a hard dependency Jan 22, 2015
.gitignore autogen.sh: warn the user if autoconf-archive is not installed Nov 17, 2015
.gitmodules afmongodb: replaced submodule limongo-client with mongo-c-driver Jan 27, 2016
.travis.yml .travis.yml: force enable modules, install func-test, split long lines May 31, 2016
AUTHORS lib/compat/strcasestr.c: added copyright of origin, extended COPYING … Jan 22, 2016
CMakeLists.txt cmake: set SYSLOG_NG_ENABLE_DEBUG based on release type May 23, 2016
CONTRIBUTING.md COPYING, CONTRIBUTING.md: updated directory structure and clarified l… Jan 22, 2016
COPYING modules/native: relicensed under the LGPL Jan 22, 2016
GPL.txt relicense syslog-ng to a combination of GPL/LGPL Jul 14, 2010
INSTALL make dynamic linking the default Jun 9, 2008
LGPL.txt Major copyright & license fixups Nov 12, 2012
Makefile.am Makefile.am: propagate $(AM_CFLAGS) via $(TEST_CFLAGS) May 20, 2016
NEWS.md Preparations for syslog-ng 3.7.2 Oct 26, 2015
README.md README: add quickstart section May 20, 2016
VERSION VERSION: bumped to 3.8.0alpha0 Nov 4, 2015
autogen.sh build scripts: added copyright declaration, renamed system-expand.sh Apr 1, 2016
configure.ac Merge pull request #981 from bkil-syslogng/f/libmongo-client-compatib… Jun 1, 2016
dist.conf.in added dist.conf and added automatic substitution of some dynamic vari… Apr 14, 2007
requirements.txt requirements: update pylint and specify concrete astroid version Nov 30, 2015
sub-configure.sh build scripts: added copyright declaration, renamed system-expand.sh Apr 1, 2016
syslog-ng-config.h.in cmake: set SYSLOG_NG_ENABLE_DEBUG based on release type May 23, 2016
syslog-ng-native-connector.pc.cmake cmake: add syslog-ng-native-connector.pc.cmake file May 21, 2016
syslog-ng-native-connector.pc.in Makefile: add pkg-config file for syslog-ng-native-connector.a Dec 8, 2015
syslog-ng.pc.cmake cmake: add syslog-ng.pc.cmake May 21, 2016
syslog-ng.pc.in configure: use PACKAGE_VERSION variable instead of VERSION Mar 4, 2016
syslog-ng.spec.in configure: use PACKAGE_VERSION variable instead of VERSION Mar 4, 2016
syslog-ng.supp Makefile: add valgrind suppressions Dec 10, 2015

README.md

Gitter Build Status Build Status

syslog-ng

syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, message queues, databases (SQL and NoSQL alike) and more.

Quickstart

The easiest configuration that accepts system logs on /dev/log (from applications or forwarded by systemd) and writes everything to a single file:

@version: 3.8
@include "scl.conf"

log {
    source { system(); };
    destination { file("/var/log/syslog"); };
};

This one also processes logs from the network (TCP/514 by default):

@version: 3.8
@include "scl.conf"

log {
    source {
        system();
        network();
    };
    destination { file("/var/log/syslog"); };
};

Structured/application logging, local submission via JSON, output in key=value format.

@version: 3.8
@include "scl.conf"

log {
    source { system(); };
    destination { file("/var/log/app.log" template("$(format-welf --subkeys .cim.)\n")); };
};

Here's how to submit a structured message using "logger":

$ logger '@cim: {"name1":"value1", "name2":"value2"}'

and the result will be:

name1=value1 name2=value2

Features

  • receive and send RFC3164 and RFC5424 style syslog messages
  • work with any kind of unstructured data
  • receive and send JSON formatted messages
  • classify and structure logs with builtin parsers (csv-parser(), db-parser(), kv-parser(), ...)
  • normalize, crunch and process logs as they flow through the system
  • hand on messages for further processing using message queues (like AMQP), files or databases (like PostgreSQL or MongoDB), and
  • forward log messages to big data tools like Elasticsearch, Apache Kafka, or Apache Hadoop.

Performance:

  • syslog-ng provides performance levels comparable to a large cluster while running on a single node.
  • In the simplest use-case it scales up 600-800k messages per second.
  • But classification, parsing and filtering still produces several tens of thousands messages per second.

Community:

  • syslog-ng is developed by a community of volunteers, the best way to connect us is via our github project page project, our gitter channel or our mailing list
  • syslog-ng is integrated into almost all Linux distributions & BSDs, it is also incorporated into a number of products, see our [powered by syslog-ng] page for more details.

Sponsors:

  • Balabit is the original creator and the largest current sponsor of the syslog-ng project, they provide support, professional services and addons you might be interested in.

Feedback

We are really interested in who uses our software, so if you do and you like what you see, please tell us about it. A simple "thanks" is also enough, but learning about your usecase, experience, things to improve would be most appreciated.

Just send an email to feedback (at) syslog-ng.org.

Should not take more than a minute, right? Now go ahead. Please.

FeedbackPowersOpenSource.

Installation from Source

Releases and tarballs ready to compile are are made available at GitHub.

To compile from source, the usual drill applies (assuming you have the required dependencies):

$ ./configure && make && make install

Some of the functionality is compiled only in case the required development libraries are present. The configure script displays a summary of enabled features at the end of its run. For details, see the syslog-ng compiling instructions

Installation from Binaries

Binaries are available in various Linux distributions and contributors maintain packages of the latest and greatest syslog-ng version for various OSes.

Debian/Ubuntu

Simply invoke the following command as root:

# apt-get install syslog-ng

Latest versions of syslog-ng are available for a wide range of Debian and Ubuntu releases and architectures from an unofficial repository.

Fedora

syslog-ng is available as a Fedora package that you can install using yum:

# yum install syslog-ng

You can download packages for the latest versions from here.

Others

Binaries for other platforms are listed at the official third party page.

Documentation

The documentation of the latest released version of syslog-ng Open Source Edition is available here. For earlier versions, see the Balabit Documentation Page. For ancient versions, see the Balabit Documentation Archive.

Contributing

If you want to modify the source of syslog-ng, for example, to correct a bug or develop a new module, the syslog-ng gitbook helps you to take the first steps with the code base.