New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debugging syslog table configuration #1964

Closed
zwass opened this Issue Mar 25, 2016 · 7 comments

Comments

Projects
None yet
5 participants
@zwass
Contributor

zwass commented Mar 25, 2016

This issue attempts to help debug configuration errors with the Linux syslog table.

osquery reads syslog logs from rsyslogd via a named pipe. This pipe, and rsyslogd must be configured properly in order to query syslog.

Debugging Suggestions

  1. Ensure that you are using the --enable_syslog flag (as well as --disable_events=false when using with osqueryi).
  2. Turn on verbose logging (verbose configuration flag).
    • Is osquery able to create/open the pipe file? If so, skip to step 4.
  3. If osquery cannot create/open the pipe:
    • Does /var/osquery/ (or the directory you used for syslog_pipe_path) exist? Is it read/writeable by the osquery process?
    • Does /var/osquery/syslog_pipe (or the path you used for syslog_pipe_path) exist? Is it readable by the osquery process?
  4. If osquery does not report an error opening the pipe, but no logs are in the syslog table:
    • Check the permissions of the pipe file:

      • If rsyslogd runs as user syslog (default Ubuntu setup):
      $ ls -l /var/osquery/syslog_pipe
      pr--rw---- 1 root syslog 0 Apr  4 18:50 /var/osquery/syslog_pipe
      
      • If rsyslogd runs as user root:
      $ ls -l /var/osquery/syslog_pipe
      pr--rw---- 1 root root 0 Apr  4 18:50 /var/osquery/syslog_pipe
      
    • Make sure that rsyslogd is able to write to the pipe.

      1. Kill all osquery processes.
      2. In a separate shell: cat /var/osquery/syslog_pipe (or the path you used for syslog_pipe_path).
      3. Try sending a log using the logger utility in your shell: logger "test log foo". Did the output show up in the shell running cat?
    • If the above steps fail:

      1. Check that rsyslogd is properly configured to write to the pipe. See the configuration docs.
      2. Restart rsyslogd: sudo service rsyslog restart.
      3. Does rsyslogd report any errors in /var/log/syslog?

Other considerations

  • Is SELinux turned on? Are SELinux policies preventing rsyslogd or osquery from accessing the pipe file?

@zwass zwass closed this Mar 25, 2016

@theopolis theopolis added the reference label Mar 25, 2016

@zwass zwass changed the title from Syslog table configuration errors to Debugging syslog table configuration Apr 4, 2016

@my-janala

This comment has been minimized.

my-janala commented Jul 19, 2016

I am struggling to fix syslog table, still nothing is being populated on syslog table. The log shows

I0719 15:32:07.685186 5266 syslog.cpp:87] Successfully opened pipe for syslog ingestion

@jacknagz

This comment has been minimized.

Contributor

jacknagz commented Aug 3, 2016

Does rsyslogd report any errors in /etc/log/sysog?

shouldn't that be /var/log/syslog

@zwass

This comment has been minimized.

Contributor

zwass commented Aug 3, 2016

@jacknagz Yes, will correct above.

@zwass

This comment has been minimized.

Contributor

zwass commented Aug 3, 2016

@my-janala Come on over to the osquery Slack if you would like more help debugging. You can catch my attention by mentioning me with @zwass in there.

@DwijadasDey

This comment has been minimized.

DwijadasDey commented Apr 16, 2017

@zwass The steps for debugging syslog table is nice one. I am also facing the same issue but with syslog-ng. In my case syslogs do logged in the file /var/log/osquery/osqueryd.results.log if i use system() in syslog-ng source but not pipe as a source.
.......................
pipe("/var/osquery/syslog_pipe");
unix-stream("/dev/log");
.......................

Obviously syslogs does not appear in the table syslog. The name pipe has been created and syslog-ng is running as root:adm. Although i have checked with root:root as well.

ls -l /var/osquery/syslog_pipe
pr--rw---- 1 root adm 0 Apr 14 15:41 /var/osquery/syslog_pipe

There is no error in /var/log/syslog as well. I am using Ubuntu 16.10 and not firewall is running.

@zwass

This comment has been minimized.

Contributor

zwass commented Apr 17, 2017

@DwijadasDey I don't understand the issue you are experiencing. Please join the osquery Slack if you would like more help debugging. You can catch my attention by mentioning me with @zwass in there.

@zwass

This comment has been minimized.

Contributor

zwass commented Apr 18, 2017

If anyone is interested in in using the syslog table with syslog-ng, please take a look at this discussion for more information.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment