New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not enable audit #959

Closed
opoplawski opened this Issue Aug 14, 2015 · 51 comments

Comments

@opoplawski

opoplawski commented Aug 14, 2015

See https://bugzilla.redhat.com/show_bug.cgi?id=1227379#c25 from Steve Grubb:

I think the root of the problem here is journald. There is a function, server_open_audit() which enables audit even if people don't want it on. Journald should not enable audit, that is the audit daemon's job. What journald should do is:

  1. Stop enabling audit.
  2. Make it configurable as to whether or not to include audit data in the logs. Because its potentially mixing Top Secret data with unclassified data, there needs to be a way for people to shut that off when needed. If enabled, just attach to the multicast group and listen. AVC's will still come out. If not, open a bug on that. But enabling audit will slow the system down. If you want the highest performance of your system, audit must be disabled since boot and never turned on.
  3. Make it configurable as to whether or not to pass audit data to syslog. Again, journald is the only thing that knows the provenance of the data stream. Therefore its incumbent on journald to prevent unintended reclassification of data.

The audit subsystem leaves sending event to syslog as a configuration option. It defaults to off because that is what the majority of the people wanted. Some people want it on, though, so that its aggregated and searchable by splunk. But this is the minority.

@poettering

This comment has been minimized.

Show comment
Hide comment
@poettering

poettering Aug 14, 2015

Member

Well, I very much disagree with this. Audit can be potentially useful, and we should centralize it by default in the journal. I really don't buy the "top secret" non-sense, UNIX doesn't work that way. Either you have access or you don't, but there aren't multiple levels of "root" defined. It's a completely valid option though to turn audit off (run "auditctl -e0" on boot, after journald; or "audit=0" on the kernel cmdline), and they can still: all we did is change things from opt-in to opt-out.

On Fedora auditing has always been on, as has been selinux. I am pretty sure that we should make use of it if it's available. If you don't want it, then turn off the whole subsystem with "audit=0" on the kernel command line. But if you don't explicitly disable it that way, then we should capture that information and add it to everything else that is logged.

rsyslog reads the data from the journal, it's not the journal that pushes the data to syslog anymore. Hence, if rsyslog doesn't want the audit data, then it should filter it out (by ignoring all log entries with _TRANSPORT=audit for example), but the journal really has no control over the filtering rsyslog applies on the data it collects.

Sorry, I don't think there's anything to fix here.

Member

poettering commented Aug 14, 2015

Well, I very much disagree with this. Audit can be potentially useful, and we should centralize it by default in the journal. I really don't buy the "top secret" non-sense, UNIX doesn't work that way. Either you have access or you don't, but there aren't multiple levels of "root" defined. It's a completely valid option though to turn audit off (run "auditctl -e0" on boot, after journald; or "audit=0" on the kernel cmdline), and they can still: all we did is change things from opt-in to opt-out.

On Fedora auditing has always been on, as has been selinux. I am pretty sure that we should make use of it if it's available. If you don't want it, then turn off the whole subsystem with "audit=0" on the kernel command line. But if you don't explicitly disable it that way, then we should capture that information and add it to everything else that is logged.

rsyslog reads the data from the journal, it's not the journal that pushes the data to syslog anymore. Hence, if rsyslog doesn't want the audit data, then it should filter it out (by ignoring all log entries with _TRANSPORT=audit for example), but the journal really has no control over the filtering rsyslog applies on the data it collects.

Sorry, I don't think there's anything to fix here.

@klic

This comment has been minimized.

Show comment
Hide comment
@klic

klic Aug 15, 2015

Sorry, but you're wrong here. Every modern Unix has the possibility to use multiple levels of access. Think rbac or selinux MCs/mls. Ever heard of Bell/La Padua ? If you mix it here, you break that model, and either distributions for enterprises will have to fork systemd, or lose that market share. This is another era away from distinctions like uid==0 or not

Klaus

klic commented Aug 15, 2015

Sorry, but you're wrong here. Every modern Unix has the possibility to use multiple levels of access. Think rbac or selinux MCs/mls. Ever heard of Bell/La Padua ? If you mix it here, you break that model, and either distributions for enterprises will have to fork systemd, or lose that market share. This is another era away from distinctions like uid==0 or not

Klaus

@davidelang

This comment has been minimized.

Show comment
Hide comment
@davidelang

davidelang Aug 22, 2015

having ti as an option is a good thing for those who want to combine all the data (which does include some enterprise users)

having it mandatory with no option and not passing the information on the source to syslog is a bad thing.

When logs are passed to syslog, I know that the logs can be passed (almost) as if journald wasn't involved. Is there also an option to pass structured data (i.e. JSON in the body of the log) so that the filtering can be done on this metadata inside the syslog infrastructure?

davidelang commented Aug 22, 2015

having ti as an option is a good thing for those who want to combine all the data (which does include some enterprise users)

having it mandatory with no option and not passing the information on the source to syslog is a bad thing.

When logs are passed to syslog, I know that the logs can be passed (almost) as if journald wasn't involved. Is there also an option to pass structured data (i.e. JSON in the body of the log) so that the filtering can be done on this metadata inside the syslog infrastructure?

@poettering

This comment has been minimized.

Show comment
Hide comment
@poettering

poettering Aug 22, 2015

Member

@davidelang as I wrote already, rsyslog pulls the data from the journal these days, journald's syslog forwarding is not used anymore. One of the reasons syslog was changed was to get the structured data.

Member

poettering commented Aug 22, 2015

@davidelang as I wrote already, rsyslog pulls the data from the journal these days, journald's syslog forwarding is not used anymore. One of the reasons syslog was changed was to get the structured data.

@mbiebl

This comment has been minimized.

Show comment
Hide comment
@mbiebl

mbiebl Aug 22, 2015

Contributor

@poettering on Debian/Ubuntu we still use the push model instead of letting rsyslog pull the data.
One reason is that not all existing syslog implementations support pulling data out of journald directly.

Contributor

mbiebl commented Aug 22, 2015

@poettering on Debian/Ubuntu we still use the push model instead of letting rsyslog pull the data.
One reason is that not all existing syslog implementations support pulling data out of journald directly.

@mbiebl

This comment has been minimized.

Show comment
Hide comment
@mbiebl

mbiebl Aug 22, 2015

Contributor

@poettering also, afaik, the imjournal module in rsyslog is not really maintained by rsyslog upstream directly, but it's mostly maintained by external contributor. At least from my understanding rsyslog upstream still recommends using the socket forwarding mode.

Contributor

mbiebl commented Aug 22, 2015

@poettering also, afaik, the imjournal module in rsyslog is not really maintained by rsyslog upstream directly, but it's mostly maintained by external contributor. At least from my understanding rsyslog upstream still recommends using the socket forwarding mode.

@johannbg

This comment has been minimized.

Show comment
Hide comment
@johannbg

johannbg Aug 24, 2015

Contributor

@mbiebl which other syslog solution beside rsyslog ( the imjournal module is not officially supported upstream and I question of rsyslog will ever officially support journal since last time I checked Rainer Gerhards had some axe to grind with Lennart or binary logging in general ) does Debian have that does not support pulling data out of the journal directly ( syslog-ng does that and defaults to do that since what 3.6 or there about ).

Arguably distributions ( or rather their server versions since it makes no sense shipping both on the desktop alternatives ) should be defaulting to shipping syslog-ng ( due to that supported upstream integration ) in addition to the journal itself to provide that forwarding to centralized log server capability ( which is what administrators have mostly criticised and found lacking with the journal ).

Contributor

johannbg commented Aug 24, 2015

@mbiebl which other syslog solution beside rsyslog ( the imjournal module is not officially supported upstream and I question of rsyslog will ever officially support journal since last time I checked Rainer Gerhards had some axe to grind with Lennart or binary logging in general ) does Debian have that does not support pulling data out of the journal directly ( syslog-ng does that and defaults to do that since what 3.6 or there about ).

Arguably distributions ( or rather their server versions since it makes no sense shipping both on the desktop alternatives ) should be defaulting to shipping syslog-ng ( due to that supported upstream integration ) in addition to the journal itself to provide that forwarding to centralized log server capability ( which is what administrators have mostly criticised and found lacking with the journal ).

@doverride

This comment has been minimized.

Show comment
Hide comment
@doverride

doverride Aug 24, 2015

We should have an option to disable this in journald only. You can tell us what (you think) UNIX is and what (you think) it is not but you should give us the oportunity to opt-out at least, and preferable to opt-in.

You shouldnt try to push your agenda on others when you are in a position such as yourself.

doverride commented Aug 24, 2015

We should have an option to disable this in journald only. You can tell us what (you think) UNIX is and what (you think) it is not but you should give us the oportunity to opt-out at least, and preferable to opt-in.

You shouldnt try to push your agenda on others when you are in a position such as yourself.

@bigon

This comment has been minimized.

Show comment
Hide comment
@bigon

bigon Aug 24, 2015

@johannbg Querying the apt cache, there are several packages that provide system-log-daemon( Provides: system-log-daemon in packages metadata)

$ LANG=C apt-cache search system-log-daemon
busybox-syslogd - Provides syslogd and klogd using busybox
dsyslog - advanced modular syslog daemon
inetutils-syslogd - system logging daemon
rsyslog - reliable system and kernel logging daemon
socklog-run - system and kernel logging services
syslog-ng-core - Enhanced system logging daemon (core)

bigon commented Aug 24, 2015

@johannbg Querying the apt cache, there are several packages that provide system-log-daemon( Provides: system-log-daemon in packages metadata)

$ LANG=C apt-cache search system-log-daemon
busybox-syslogd - Provides syslogd and klogd using busybox
dsyslog - advanced modular syslog daemon
inetutils-syslogd - system logging daemon
rsyslog - reliable system and kernel logging daemon
socklog-run - system and kernel logging services
syslog-ng-core - Enhanced system logging daemon (core)

@johannbg

This comment has been minimized.

Show comment
Hide comment
@johannbg

johannbg Aug 24, 2015

Contributor

@doverride the unix philosophy ( written sometime in the 1970s and relates to the environment and the computation evolution at that time ) is just a set of developers guidelines ( aimed at developers not end users or administrators )which developers should just keep in the back of their head when developing but developers are under no obligation to follow those guidelines no more than they like but through time people seem to have obfuscated those development guidelines and taken the term "UNIX" and put what ever they think it means, meaning to it and expect others to follow it.

Systemd is not "UNIX" by those individuals measurements ( or in general and it's completely irrelevant to anyone what meaning people seem to be putting into it since nobody is under any obligations to follow it ) but most certainly if you look the unix philosophy and apply the development guidelines and principals the unix philosophy outlines to systemd and it's code and development you can clearly see those development guidelines have been followed to certain extent.

That said the systemd-journald is not optional and never will be so there is no availability for "opt-in" or "opt-out" capability hence traditional textual file based syslog applications need to be made systemd-journald compatible ( by directly reading from the journal and or from using /run/systemd/journal/syslog ) with the most common ones ( rsyslog and syslog-ng ) already being that ( for the the rest of what @bigon lists here I cannot say and there are ways you can configure rsyslog and syslog-ng to provide you with text file logging in conjunction with the journal ).

Distribution that ship syslogging components that are not systemd-journal compatible ( which effectively just means those components are systemd incompatible ) will need to conflict those components with systemd.

If distributions are supporting more than one init system and at the same time are providing components that are systemd ( journal or not ) compatible they will need to maintain two or more versions of their components, one that supports and setups the defaults for that component to be used in conjunction with systemd ( compile options,type units and or configuration defaults ) and one or more that support and setups the alternative init system they support ( their compile options,init script or configuration files and or configuration defaults ).

Contributor

johannbg commented Aug 24, 2015

@doverride the unix philosophy ( written sometime in the 1970s and relates to the environment and the computation evolution at that time ) is just a set of developers guidelines ( aimed at developers not end users or administrators )which developers should just keep in the back of their head when developing but developers are under no obligation to follow those guidelines no more than they like but through time people seem to have obfuscated those development guidelines and taken the term "UNIX" and put what ever they think it means, meaning to it and expect others to follow it.

Systemd is not "UNIX" by those individuals measurements ( or in general and it's completely irrelevant to anyone what meaning people seem to be putting into it since nobody is under any obligations to follow it ) but most certainly if you look the unix philosophy and apply the development guidelines and principals the unix philosophy outlines to systemd and it's code and development you can clearly see those development guidelines have been followed to certain extent.

That said the systemd-journald is not optional and never will be so there is no availability for "opt-in" or "opt-out" capability hence traditional textual file based syslog applications need to be made systemd-journald compatible ( by directly reading from the journal and or from using /run/systemd/journal/syslog ) with the most common ones ( rsyslog and syslog-ng ) already being that ( for the the rest of what @bigon lists here I cannot say and there are ways you can configure rsyslog and syslog-ng to provide you with text file logging in conjunction with the journal ).

Distribution that ship syslogging components that are not systemd-journal compatible ( which effectively just means those components are systemd incompatible ) will need to conflict those components with systemd.

If distributions are supporting more than one init system and at the same time are providing components that are systemd ( journal or not ) compatible they will need to maintain two or more versions of their components, one that supports and setups the defaults for that component to be used in conjunction with systemd ( compile options,type units and or configuration defaults ) and one or more that support and setups the alternative init system they support ( their compile options,init script or configuration files and or configuration defaults ).

@davidelang

This comment has been minimized.

Show comment
Hide comment
@davidelang

davidelang Aug 24, 2015

so you are saying that Linux systems running systemd are not Unix systems?

boy will that be a money quote for the anti-systemd people

davidelang commented Aug 24, 2015

so you are saying that Linux systems running systemd are not Unix systems?

boy will that be a money quote for the anti-systemd people

@doverride

This comment has been minimized.

Show comment
Hide comment
@doverride

doverride Aug 24, 2015

@johannbg I am not suggesting opt-in/-out of systemd-journald as a whole. Instead i am suggesting opt-in/-out of the ability to enclose audit related messages with the journal.

People with serious audit requirements cannot do with journald/journalctl as it is way too limited. This audience needs tools to be able to effectively analyze these audit messages as well. E.g. tools like ausearch, aureport. These tools cannot parse the journal. So that means that if you have a serious audit requirement that you must use auditd. Now we don't want duplicate audit logs (in journal as well as /var/log/audit) so we want to be able to tell journald:

"hey, were using another audit facility. so to avoid duplicate audit logs we kindly instruct you to stop logging audit messages altogether"

doverride commented Aug 24, 2015

@johannbg I am not suggesting opt-in/-out of systemd-journald as a whole. Instead i am suggesting opt-in/-out of the ability to enclose audit related messages with the journal.

People with serious audit requirements cannot do with journald/journalctl as it is way too limited. This audience needs tools to be able to effectively analyze these audit messages as well. E.g. tools like ausearch, aureport. These tools cannot parse the journal. So that means that if you have a serious audit requirement that you must use auditd. Now we don't want duplicate audit logs (in journal as well as /var/log/audit) so we want to be able to tell journald:

"hey, were using another audit facility. so to avoid duplicate audit logs we kindly instruct you to stop logging audit messages altogether"

@davidelang

This comment has been minimized.

Show comment
Hide comment
@davidelang

davidelang Aug 24, 2015

systemd does not control all of Linux, and it should not try. Nobody should control all of Linux.

It's true that there is no love lost between rsyslog and systemd, but that can be said about many projects and systemd, and it should not matter.

There are many different log daemons out there, and systemd should support them as opposed to requiring that they change to work with the journal.

Requiring that they either parse files periodically, or call an api periodically to fetch logs,is far from ideal. Especially when this means that the log message must get serialized and stored by jounrald and then searched for, retrieved, deserialized from storage, and then serialized to be passed through the api.

at the point that journald is processing the log message, it has all the data in RAM and it can serialize it out to a syslog daemon as JSON at very little cost. This will work with every syslog daemon that exists, and any one that is written in the future to conform to the POSIX standards (not all of them will want JSON or the full metadata, so it should be an option)

davidelang commented Aug 24, 2015

systemd does not control all of Linux, and it should not try. Nobody should control all of Linux.

It's true that there is no love lost between rsyslog and systemd, but that can be said about many projects and systemd, and it should not matter.

There are many different log daemons out there, and systemd should support them as opposed to requiring that they change to work with the journal.

Requiring that they either parse files periodically, or call an api periodically to fetch logs,is far from ideal. Especially when this means that the log message must get serialized and stored by jounrald and then searched for, retrieved, deserialized from storage, and then serialized to be passed through the api.

at the point that journald is processing the log message, it has all the data in RAM and it can serialize it out to a syslog daemon as JSON at very little cost. This will work with every syslog daemon that exists, and any one that is written in the future to conform to the POSIX standards (not all of them will want JSON or the full metadata, so it should be an option)

@doverride

This comment has been minimized.

Show comment
Hide comment
@doverride

doverride Aug 24, 2015

It is obvious to me that systemd maintainer has no clue about audit or selinux. journald with audit functionality is useless if you cannot analyze/parse the logs properly. When dealing with high volumes of audit data there must be a way to process all that data. journald/journalctl lacks that ability. The tools that can be used to process these usually large quantities of audit data cannot be used with journal.

Thus journalds' audit functionality is mostly useless.

doverride commented Aug 24, 2015

It is obvious to me that systemd maintainer has no clue about audit or selinux. journald with audit functionality is useless if you cannot analyze/parse the logs properly. When dealing with high volumes of audit data there must be a way to process all that data. journald/journalctl lacks that ability. The tools that can be used to process these usually large quantities of audit data cannot be used with journal.

Thus journalds' audit functionality is mostly useless.

@poettering

This comment has been minimized.

Show comment
Hide comment
@poettering

poettering Aug 24, 2015

Member

@davidelang, @mbiebl ah, wasn't aware that only Fedora ships that as default. I assumed this was an upstream thing. Either way: if you want structured journal data in rsyslog, that's still the way to go. journald only delivers the actual text messages via the the syslog AF_UNIX socket, in a way strictly compatible with the way glibc does it. As glibc's BSD syslog support does not allow structured data to be passed, we don't allow that either...

Member

poettering commented Aug 24, 2015

@davidelang, @mbiebl ah, wasn't aware that only Fedora ships that as default. I assumed this was an upstream thing. Either way: if you want structured journal data in rsyslog, that's still the way to go. journald only delivers the actual text messages via the the syslog AF_UNIX socket, in a way strictly compatible with the way glibc does it. As glibc's BSD syslog support does not allow structured data to be passed, we don't allow that either...

@poettering

This comment has been minimized.

Show comment
Hide comment
@poettering

poettering Aug 24, 2015

Member

@doverride you do have the option to turn off the audit firehose, as mentioned above. journald just makes it opt-out instead of opt-in. That's all.

Member

poettering commented Aug 24, 2015

@doverride you do have the option to turn off the audit firehose, as mentioned above. journald just makes it opt-out instead of opt-in. That's all.

@poettering

This comment has been minimized.

Show comment
Hide comment
@poettering

poettering Aug 24, 2015

Member

@doverride and thank you very much, I hacked up the audit code in systemd and journald. I am pretty sure I have a vague understanding of it...

Member

poettering commented Aug 24, 2015

@doverride and thank you very much, I hacked up the audit code in systemd and journald. I am pretty sure I have a vague understanding of it...

@doverride

This comment has been minimized.

Show comment
Hide comment
@doverride

doverride Aug 24, 2015

@poettering Sorry i did not mean to upset anyone. Also from the above i was (and i am still) not able to determine that journald can be instructed to not log audit messages. and how that is done (is that documented somewhere?)

If you knew that audit data must be analyzed to be useful then why did you add this functionality to journald? I mean what good does it do if you can' t (really) do anything (or much) with this data?

doverride commented Aug 24, 2015

@poettering Sorry i did not mean to upset anyone. Also from the above i was (and i am still) not able to determine that journald can be instructed to not log audit messages. and how that is done (is that documented somewhere?)

If you knew that audit data must be analyzed to be useful then why did you add this functionality to journald? I mean what good does it do if you can' t (really) do anything (or much) with this data?

@poettering

This comment has been minimized.

Show comment
Hide comment
@poettering

poettering Aug 24, 2015

Member

@doverride see above, use "auditctl -c0" or "audit=0" on the kernel cmdline...

Not sure what you mean by "analyzing". But do note, that the journald is a lot more useful in many ways than ausearch for working with audit msgs, since we split up all fields and index by them individually, so that you can search for audit fields like for any other structured logging fields.

Member

poettering commented Aug 24, 2015

@doverride see above, use "auditctl -c0" or "audit=0" on the kernel cmdline...

Not sure what you mean by "analyzing". But do note, that the journald is a lot more useful in many ways than ausearch for working with audit msgs, since we split up all fields and index by them individually, so that you can search for audit fields like for any other structured logging fields.

@johannbg

This comment has been minimized.

Show comment
Hide comment
@johannbg

johannbg Aug 24, 2015

Contributor

@doverride I'm not sure what you are meaning by that.

At least in the case of selinux it's upstream maintainer have been working closely with the systemd community in fact so closely that he planned on eliminating the setroubleshoot database totally and just use journald as its information database

Administrators can for example easily access their denials via "journalctl -r -o verbose _COMM=setroubleshootd" and follow what the MESSAGE field says ( you can access the audit entries visa something like "journalctl -r -o verbose SYSLOG_IDENTIFIER=audit" and disable it altogether with what got mentioned in comment 2 " )

So a) systemd supports the means that those two are enabled and disabled and b) it provides the administrator with effective means to search for relevant entries for those solutions so what more do you want or expect from systemd and the journal?

If and how other application chose to query and filter based on journal entries in relation to selinux or audit is entirely up to those developing those applications. Systemd and it's community has absolutely no saying in that.

@poettering
I think you can safely assume that distribution that are not using the imjournal in rsyslog are doing something like this ( if not they should be doing something like this )...

/etc/rsyslog.d/journald.conf
module(load="imuxsock" syssock.use="off")
input(type="imuxsock" Socket="/run/systemd/journal/syslog" CreatePath="off" Unlink="off")
...

ln -s /lib/systemd/system/rsyslog.service /etc/systemd/system/syslog.service

with or without the storage option set to volatile in /etc/systemd/journald.conf ( depending on whether they want to keep the journal files on reboots or just the traditional text file(s) )

Contributor

johannbg commented Aug 24, 2015

@doverride I'm not sure what you are meaning by that.

At least in the case of selinux it's upstream maintainer have been working closely with the systemd community in fact so closely that he planned on eliminating the setroubleshoot database totally and just use journald as its information database

Administrators can for example easily access their denials via "journalctl -r -o verbose _COMM=setroubleshootd" and follow what the MESSAGE field says ( you can access the audit entries visa something like "journalctl -r -o verbose SYSLOG_IDENTIFIER=audit" and disable it altogether with what got mentioned in comment 2 " )

So a) systemd supports the means that those two are enabled and disabled and b) it provides the administrator with effective means to search for relevant entries for those solutions so what more do you want or expect from systemd and the journal?

If and how other application chose to query and filter based on journal entries in relation to selinux or audit is entirely up to those developing those applications. Systemd and it's community has absolutely no saying in that.

@poettering
I think you can safely assume that distribution that are not using the imjournal in rsyslog are doing something like this ( if not they should be doing something like this )...

/etc/rsyslog.d/journald.conf
module(load="imuxsock" syssock.use="off")
input(type="imuxsock" Socket="/run/systemd/journal/syslog" CreatePath="off" Unlink="off")
...

ln -s /lib/systemd/system/rsyslog.service /etc/systemd/system/syslog.service

with or without the storage option set to volatile in /etc/systemd/journald.conf ( depending on whether they want to keep the journal files on reboots or just the traditional text file(s) )

@doverride

This comment has been minimized.

Show comment
Hide comment
@doverride

doverride Aug 24, 2015

@poettering Excuse my ignorance but "auditctl -c0" or "audit=0" are not journald specific.

Also, maybe instead you meant "auditctl -e0"

-e [0..2]
Set enabled flag. When 0 is passed, this can be used to temporarily disable auditing.

These will disable audit altogether. (So that will also apply to auditd then i suspect). What if i want to use auditd for audit logging but not journald?

Also journalctl has nice features, granted. For audit log processing it seems insufficient. How do i "interpret" fields like : arch=c000003e syscall=44 (what arch is that, what syscall is that?) ausearch has the -i option that will interpret that and other hex strings youll find troughout the various audit messages. Also what about audit reports (aureport)

doverride commented Aug 24, 2015

@poettering Excuse my ignorance but "auditctl -c0" or "audit=0" are not journald specific.

Also, maybe instead you meant "auditctl -e0"

-e [0..2]
Set enabled flag. When 0 is passed, this can be used to temporarily disable auditing.

These will disable audit altogether. (So that will also apply to auditd then i suspect). What if i want to use auditd for audit logging but not journald?

Also journalctl has nice features, granted. For audit log processing it seems insufficient. How do i "interpret" fields like : arch=c000003e syscall=44 (what arch is that, what syscall is that?) ausearch has the -i option that will interpret that and other hex strings youll find troughout the various audit messages. Also what about audit reports (aureport)

@johannbg

This comment has been minimized.

Show comment
Hide comment
@johannbg

johannbg Aug 24, 2015

Contributor

@doverride
Consult upstream documentation for audit and selinux to see if they support logging into separated log file in addition to the journal, whether their native command support searching natively in the journal and or just use that log file if it's logged separated and what fields in the journal match corresponding field in the traditional log text file ( if it's still supported or only supported ).

Contributor

johannbg commented Aug 24, 2015

@doverride
Consult upstream documentation for audit and selinux to see if they support logging into separated log file in addition to the journal, whether their native command support searching natively in the journal and or just use that log file if it's logged separated and what fields in the journal match corresponding field in the traditional log text file ( if it's still supported or only supported ).

@doverride

This comment has been minimized.

Show comment
Hide comment
@doverride

doverride Aug 24, 2015

@johannbg I do not want "in addition to the journal". I want to be able to use auditd for audit logging instead of journald.

So i need an option in journald.conf: DO_NOT_LOG_AUDIT_MSGS=yes

I do not want to end up with both auditd as well as journald logging audit messages if i choose to use auditd.

Anyhow it seems i do not have the skills to make my point. I simply cannot get one to see the issue i am trying to address. Thus i will take my loss and let this go. Hopefully someone else with better communication skills will be able to explain these issues.

doverride commented Aug 24, 2015

@johannbg I do not want "in addition to the journal". I want to be able to use auditd for audit logging instead of journald.

So i need an option in journald.conf: DO_NOT_LOG_AUDIT_MSGS=yes

I do not want to end up with both auditd as well as journald logging audit messages if i choose to use auditd.

Anyhow it seems i do not have the skills to make my point. I simply cannot get one to see the issue i am trying to address. Thus i will take my loss and let this go. Hopefully someone else with better communication skills will be able to explain these issues.

@davidelang

This comment has been minimized.

Show comment
Hide comment
@davidelang

davidelang Aug 24, 2015

As glibc's BSD syslog support does not allow structured data to be passed, we don't allow that either...

glibc's BSD syslog support absolutely allows structured data to be passed, just put JSON in the message section and you have your structured data. The major syslog daemons (rsyslog, syslog-ng, nxlog, and logstash that I know of personally) are able to directly access the various fields, and JSON parsers are available for all languages, and many databases are able to understand JSON directly as well. As a result of all of this, taking all the metadata that you have and serializing it in JSON will work very well and improve reliability and performance.

I agree that the glibc syslog interface does not give you a good way to send RFC5424 structured data, but since almost nobody uses it, it doesn't actually hurt much that this is the case.

davidelang commented Aug 24, 2015

As glibc's BSD syslog support does not allow structured data to be passed, we don't allow that either...

glibc's BSD syslog support absolutely allows structured data to be passed, just put JSON in the message section and you have your structured data. The major syslog daemons (rsyslog, syslog-ng, nxlog, and logstash that I know of personally) are able to directly access the various fields, and JSON parsers are available for all languages, and many databases are able to understand JSON directly as well. As a result of all of this, taking all the metadata that you have and serializing it in JSON will work very well and improve reliability and performance.

I agree that the glibc syslog interface does not give you a good way to send RFC5424 structured data, but since almost nobody uses it, it doesn't actually hurt much that this is the case.

@bigon

This comment has been minimized.

Show comment
Hide comment
@bigon

bigon Aug 24, 2015

@johannbg well my point listing the different syslog implementations we have in debian was to show that we should be able to support other implementations than syslog-ng or rsyslog

bigon commented Aug 24, 2015

@johannbg well my point listing the different syslog implementations we have in debian was to show that we should be able to support other implementations than syslog-ng or rsyslog

@johannbg

This comment has been minimized.

Show comment
Hide comment
@johannbg

johannbg Aug 24, 2015

Contributor

@bigon that's really up to those other solutions and worst case scenario with or without centralized log solution you can do something as ugly like this...

[Unit]
Description=Send Journal

[Service]
TimeoutStartSec=0
ExecStart=/bin/sh -c '/usr/bin/journalctl -o json -f | /usr/bin/ncat syslog 515'
Restart=always
RestartSec=5s

[Install]
WantedBy=multi-user.target

Contributor

johannbg commented Aug 24, 2015

@bigon that's really up to those other solutions and worst case scenario with or without centralized log solution you can do something as ugly like this...

[Unit]
Description=Send Journal

[Service]
TimeoutStartSec=0
ExecStart=/bin/sh -c '/usr/bin/journalctl -o json -f | /usr/bin/ncat syslog 515'
Restart=always
RestartSec=5s

[Install]
WantedBy=multi-user.target

@poettering

This comment has been minimized.

Show comment
Hide comment
@poettering

poettering Aug 24, 2015

Member

@davidelang well, i can also pretend my printer would understand JSON by formatting JSON as text and then print that text document on it...

Can you reference any spec/RFC that documents how to use json via BSD syslog? Something that defines the vocabulary of fields to use then? I am not aware of any, and as such this is not really useful at all..

Member

poettering commented Aug 24, 2015

@davidelang well, i can also pretend my printer would understand JSON by formatting JSON as text and then print that text document on it...

Can you reference any spec/RFC that documents how to use json via BSD syslog? Something that defines the vocabulary of fields to use then? I am not aware of any, and as such this is not really useful at all..

@davidelang

This comment has been minimized.

Show comment
Hide comment
@davidelang

davidelang Aug 24, 2015

There isn't a spec for the body of a syslog message because there isn't any piece of data that is going to be present for all possible log messages. That's why it allows arbitrary text with newline as the delimiter.

single-line JSON meets this spec, it's text with no newlines in it.

If journald already has JSON output, then that is exactly what it should put in the message portion of the syslog message.

davidelang commented Aug 24, 2015

There isn't a spec for the body of a syslog message because there isn't any piece of data that is going to be present for all possible log messages. That's why it allows arbitrary text with newline as the delimiter.

single-line JSON meets this spec, it's text with no newlines in it.

If journald already has JSON output, then that is exactly what it should put in the message portion of the syslog message.

@poettering

This comment has been minimized.

Show comment
Hide comment
@poettering

poettering Aug 24, 2015

Member

@davidelang well, the syslog rfcs I know don't mention JSON anywhere. Also looking at the various bits of data we get fed into the journal via glibc syslog() I have never seen any such JSON msg.

I think what you think is a "standard" is none, not even a "de-facto" one...

But anyway, this discussion is pointless. We won't add new features to compat solutions beyond what the original implementation we try to be compatible to provided. BSD syslog is such a compat feature for us, hence there's really no point in trying to convince me to add this.

But again, nothing stops you from doing this, and maybe rsyslog or syslog-ng already do. If they don't please ask them, they might add this for you...

Member

poettering commented Aug 24, 2015

@davidelang well, the syslog rfcs I know don't mention JSON anywhere. Also looking at the various bits of data we get fed into the journal via glibc syslog() I have never seen any such JSON msg.

I think what you think is a "standard" is none, not even a "de-facto" one...

But anyway, this discussion is pointless. We won't add new features to compat solutions beyond what the original implementation we try to be compatible to provided. BSD syslog is such a compat feature for us, hence there's really no point in trying to convince me to add this.

But again, nothing stops you from doing this, and maybe rsyslog or syslog-ng already do. If they don't please ask them, they might add this for you...

@doverride

This comment has been minimized.

Show comment
Hide comment
@doverride

doverride Aug 24, 2015

@poettering Thanks, It is not particularely pretty but if it works and if it does not cause DoS then that addresses my concerns.

Could the following be more prominently documented in maybe man journald?:

to disable audit support in only the journald just drop the CAP_AUDIT_READ and CAP_AUDIT_CONTROL caps from journald. Add /etc/systemd/system/systemd-journald.service.d/no-audit.conf and add to it:

[Service]
CapabilityBoundingSet=CAP_SYS_ADMIN CAP_DAC_OVERRIDE CAP_SYS_PTRACE CAP_SYSLOG CAP_CHOWN CAP_DAC_READ_SEARCH CAP_FOWNER CAP_SETUID CAP_SETGID CAP_MAC_OVERRIDE

doverride commented Aug 24, 2015

@poettering Thanks, It is not particularely pretty but if it works and if it does not cause DoS then that addresses my concerns.

Could the following be more prominently documented in maybe man journald?:

to disable audit support in only the journald just drop the CAP_AUDIT_READ and CAP_AUDIT_CONTROL caps from journald. Add /etc/systemd/system/systemd-journald.service.d/no-audit.conf and add to it:

[Service]
CapabilityBoundingSet=CAP_SYS_ADMIN CAP_DAC_OVERRIDE CAP_SYS_PTRACE CAP_SYSLOG CAP_CHOWN CAP_DAC_READ_SEARCH CAP_FOWNER CAP_SETUID CAP_SETGID CAP_MAC_OVERRIDE

@klic

This comment has been minimized.

Show comment
Hide comment
@klic

klic Aug 24, 2015

Am 24.08.2015 um 17:51 schrieb doverride:

@johannbg https://github.com/johannbg I do not want "in addition to
the journal". I want to be able to use auditd for audit logging instead
of journald.

So i need an option in journald.conf: DO_NOT_LOG_AUDIT_MSGS=yes

I do not want to end up with both auditd as well as journald logging
audit messages if i choose to use auditd.

Anyhow it seems i do not have the skills to make my point. I simply
cannot get one to see the issue i am trying to address. Thus i will take
my loss and let this go. Hopefully someone else with better
communication skills will be able to explain these issues.


Reply to this email directly or view it on GitHub
#959 (comment).

In the very end, two daemons logging audit data can definitely be a
performance problem. Auditd does a very good job of it (probably
journald as well), but two aren't needed.

And actually, I don't need the au* tools (except for auditctl :-)) on
the system, as every audit message gets forwarded to a SIEM system
(which is a performance topic on its own, granted)

Klaus


Klaus Lichtenwalder, Dipl. Inform., http://www.lichtenwalder.name/
PGP Key fingerprint: 0CF4 4A47 355A DD36 65B3 56F3 2696 79B2 7F3D 1020

klic commented Aug 24, 2015

Am 24.08.2015 um 17:51 schrieb doverride:

@johannbg https://github.com/johannbg I do not want "in addition to
the journal". I want to be able to use auditd for audit logging instead
of journald.

So i need an option in journald.conf: DO_NOT_LOG_AUDIT_MSGS=yes

I do not want to end up with both auditd as well as journald logging
audit messages if i choose to use auditd.

Anyhow it seems i do not have the skills to make my point. I simply
cannot get one to see the issue i am trying to address. Thus i will take
my loss and let this go. Hopefully someone else with better
communication skills will be able to explain these issues.


Reply to this email directly or view it on GitHub
#959 (comment).

In the very end, two daemons logging audit data can definitely be a
performance problem. Auditd does a very good job of it (probably
journald as well), but two aren't needed.

And actually, I don't need the au* tools (except for auditctl :-)) on
the system, as every audit message gets forwarded to a SIEM system
(which is a performance topic on its own, granted)

Klaus


Klaus Lichtenwalder, Dipl. Inform., http://www.lichtenwalder.name/
PGP Key fingerprint: 0CF4 4A47 355A DD36 65B3 56F3 2696 79B2 7F3D 1020

@doverride

This comment has been minimized.

Show comment
Hide comment
@doverride

doverride Aug 24, 2015

@klic probably proprietory then (the SIEM solution i mean)?

I use to use prelude when it was still available from the Fedora repositories. Still looking for interesting FOSS alternatives.

doverride commented Aug 24, 2015

@klic probably proprietory then (the SIEM solution i mean)?

I use to use prelude when it was still available from the Fedora repositories. Still looking for interesting FOSS alternatives.

@klic

This comment has been minimized.

Show comment
Hide comment
@klic

klic Aug 24, 2015

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Am 24.08.2015 um 21:25 schrieb doverride:

@klic https://github.com/klic probably proprietory then (the
SIEM solution i mean)?

Well, depends on how you see it. It's the commercial version of splunk.
That's because it's a regulated enterprise, and has to go this way.

I use to use prelude when it was still available from the Fedora
repositories. Still looking for interesting FOSS alternatives.

I guess even elastic search is a viable alternative? Haven't looked
into it, yet, though, but want to do it for more non controlled log
data searches

Klaus



Klaus Lichtenwalder, Dipl. Inform., http://www.lichtenwalder.name/
PGP Key fingerprint: 0CF4 4A47 355A DD36 65B3 56F3 2696 79B2 7F3D 1020
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCgAGBQJV23B5AAoJECaWebJ/PRAgE+cP/Aw4wPsmJdp22VwjhfqnZbWX
Rtm0MvFRxJwQ11in0cBVTSiaD832jVxBFHAPwxoOjssWksaf7dPpjZA/H+T1FqEU
/xrJjDCIt4WjjN4GjyfMcE2XIPahNN5cNHOEh/BBLhqF1Gn4CMsAQCEK1bYwQq2s
QJ1fxB/z/6zLed1SGQn0Hc6gS9lYE3OkarrtA1eEcD/4c6uRc7a06g/3JmAaFrLm
UmvGL1AvAuQrvYivdx40CZXG3BHGzCIIMelu1XeXfsFnfKfyFezSjzFoC6xmWjYi
eOgC8hCyyd584hBbxyV6ObRTZ9dESD6BBVb0TIfClRBb9MVH1+yJg1yQ5P7wKUS/
WNkuyrrNq1lD1mzXveyiruopsrPRlksXgI8EeZc0+EiNWHTTK49O6Xi6ljKaJUz3
YEKQuh5p42fvfPwkk8OYwBqfv51JYaV54YJl00fml2UNnalAk0JvXQFYZGHVYG4p
dDkr8h9Wg1jOkiJNhr/1z3ji2zvFvCFgI+QnoJrHQdd45Q1q/DAglqf7aopzm54V
3N65zjWEq79qVjWrAkTIhU5zl+M6a8Dxf75TsuHJbbQZpi2rGYXznha9o6bEV/mT
+hN0lHNGVPOv7m125IVPtZ6/qU2PmXN13DdUW+eAi2Mm5gAJ6fqut8ANU2cnyGEl
pJKs4WHl8grJs4oqcLfN
=dMc+
-----END PGP SIGNATURE-----

klic commented Aug 24, 2015

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Am 24.08.2015 um 21:25 schrieb doverride:

@klic https://github.com/klic probably proprietory then (the
SIEM solution i mean)?

Well, depends on how you see it. It's the commercial version of splunk.
That's because it's a regulated enterprise, and has to go this way.

I use to use prelude when it was still available from the Fedora
repositories. Still looking for interesting FOSS alternatives.

I guess even elastic search is a viable alternative? Haven't looked
into it, yet, though, but want to do it for more non controlled log
data searches

Klaus



Klaus Lichtenwalder, Dipl. Inform., http://www.lichtenwalder.name/
PGP Key fingerprint: 0CF4 4A47 355A DD36 65B3 56F3 2696 79B2 7F3D 1020
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCgAGBQJV23B5AAoJECaWebJ/PRAgE+cP/Aw4wPsmJdp22VwjhfqnZbWX
Rtm0MvFRxJwQ11in0cBVTSiaD832jVxBFHAPwxoOjssWksaf7dPpjZA/H+T1FqEU
/xrJjDCIt4WjjN4GjyfMcE2XIPahNN5cNHOEh/BBLhqF1Gn4CMsAQCEK1bYwQq2s
QJ1fxB/z/6zLed1SGQn0Hc6gS9lYE3OkarrtA1eEcD/4c6uRc7a06g/3JmAaFrLm
UmvGL1AvAuQrvYivdx40CZXG3BHGzCIIMelu1XeXfsFnfKfyFezSjzFoC6xmWjYi
eOgC8hCyyd584hBbxyV6ObRTZ9dESD6BBVb0TIfClRBb9MVH1+yJg1yQ5P7wKUS/
WNkuyrrNq1lD1mzXveyiruopsrPRlksXgI8EeZc0+EiNWHTTK49O6Xi6ljKaJUz3
YEKQuh5p42fvfPwkk8OYwBqfv51JYaV54YJl00fml2UNnalAk0JvXQFYZGHVYG4p
dDkr8h9Wg1jOkiJNhr/1z3ji2zvFvCFgI+QnoJrHQdd45Q1q/DAglqf7aopzm54V
3N65zjWEq79qVjWrAkTIhU5zl+M6a8Dxf75TsuHJbbQZpi2rGYXznha9o6bEV/mT
+hN0lHNGVPOv7m125IVPtZ6/qU2PmXN13DdUW+eAi2Mm5gAJ6fqut8ANU2cnyGEl
pJKs4WHl8grJs4oqcLfN
=dMc+
-----END PGP SIGNATURE-----

@davidelang

This comment has been minimized.

Show comment
Hide comment
@davidelang

davidelang Aug 24, 2015

ElasticSearch is roughly the free software equivalent of the Splunk datastore. Kibana is a GUI that lets you do a lot of what the Splunk GUI does. The pair are a great way to explore your data.

The company that sponsors ElasticSearch and Kibana has coined the term the ELK stack (ElasticSearch Logstash Kibana) watch out for the performance of Logstash and keep in mind that nxlog and rsyslog can both parse data into JSON and insert it into ElasticSearch at high speed. I don't know if syslog-ng has this or not, I don't watch their capabilities much

davidelang commented Aug 24, 2015

ElasticSearch is roughly the free software equivalent of the Splunk datastore. Kibana is a GUI that lets you do a lot of what the Splunk GUI does. The pair are a great way to explore your data.

The company that sponsors ElasticSearch and Kibana has coined the term the ELK stack (ElasticSearch Logstash Kibana) watch out for the performance of Logstash and keep in mind that nxlog and rsyslog can both parse data into JSON and insert it into ElasticSearch at high speed. I don't know if syslog-ng has this or not, I don't watch their capabilities much

@rgerhards

This comment has been minimized.

Show comment
Hide comment
@rgerhards

rgerhards Aug 25, 2015

Contributor

@johannbg you haven't checked the status for several years, as it looks ;) I've come across this topic by dealing with a rsyslog bug tracker, but please let me clarify a bit. We recommend the journal for low-end systems (e.g. typical notebook user) and say it's up to the end user's philosophy in enterprise environments. There are ample of integration tools. We are doing this since quite a while, see for example this LinuxTag presentation from 2013: http://de.slideshare.net/rainergerhards1/rsyslog-vs-systemd-journal-presentation (to go up to the point, look at slide 12+).

It is true that we do not recommend to use imjournal, except when needed (see http://www.rsyslog.com/doc/v8-stable/configuration/modules/imjournal.html ). This is the result of our experience (and user reports) of problems showing up every now and then, not being rooted in imjournal or rsyslog. We also got feedback that imjournal is much more performance-intense than the classical interface. As rsyslog is often used for high-performance environments, that is an concern.. So if you do not need metadata, there is no real point in opening up for some extra problems. If there is strong opinion we should change that policy, and good reason to do so, I am open to it.

Finally, we do not actively maintain imjournal because there is no need to: the main contributor is Red Hat and we do obviously get sufficiently frequent updates because very few problems are reported against this module in general. With a long todo list and far fewer contributors than systemd, the rsyslog project needs to focus on what's most in demand, and imjournal doesn't look like to be such a component. I hope this clarifies.

Contributor

rgerhards commented Aug 25, 2015

@johannbg you haven't checked the status for several years, as it looks ;) I've come across this topic by dealing with a rsyslog bug tracker, but please let me clarify a bit. We recommend the journal for low-end systems (e.g. typical notebook user) and say it's up to the end user's philosophy in enterprise environments. There are ample of integration tools. We are doing this since quite a while, see for example this LinuxTag presentation from 2013: http://de.slideshare.net/rainergerhards1/rsyslog-vs-systemd-journal-presentation (to go up to the point, look at slide 12+).

It is true that we do not recommend to use imjournal, except when needed (see http://www.rsyslog.com/doc/v8-stable/configuration/modules/imjournal.html ). This is the result of our experience (and user reports) of problems showing up every now and then, not being rooted in imjournal or rsyslog. We also got feedback that imjournal is much more performance-intense than the classical interface. As rsyslog is often used for high-performance environments, that is an concern.. So if you do not need metadata, there is no real point in opening up for some extra problems. If there is strong opinion we should change that policy, and good reason to do so, I am open to it.

Finally, we do not actively maintain imjournal because there is no need to: the main contributor is Red Hat and we do obviously get sufficiently frequent updates because very few problems are reported against this module in general. With a long todo list and far fewer contributors than systemd, the rsyslog project needs to focus on what's most in demand, and imjournal doesn't look like to be such a component. I hope this clarifies.

@johannbg

This comment has been minimized.

Show comment
Hide comment
@johannbg

johannbg Aug 25, 2015

Contributor

@rgerhards I have not been paying attention to centralized syslog servers in general very much since distribution more or less ( as of last year ) has stop shipping rsylog as part of their default install and or ship it disabled since the introduction of systemd's journal.

Rsyslog has moved away from being a "generic" syslog solution to becoming one of the many "centralized" syslog solution available on the market to host locally or are used for customer based cloud solutions.

With the exception of server administrators that are still in the "petting their server" mindset and or have general hard time adapting and learning new things or for what ever ( infrastructure ) reason they still have to rely on textual log files residing on host itself, everyone has move to create "centralized-syslog.service" which consist of more or less of what I have outlined here before ( to work around lack of an proper configurable implementation of doing this in the configuration for the journal which has been the nr.1 criticize the journal has received in the past years ) in my previous comments ( using mostly the json format ) and configure the storage of the journal to be volatile. There are few solution,setups out there that want to deal with the json file directly locally but then you pretty much just use the same thing I outlined before but with ">" + PATH and have a timer unit for the classic file rotation so still no need for a full blown centralized syslog solution being installed on the host.

Contributor

johannbg commented Aug 25, 2015

@rgerhards I have not been paying attention to centralized syslog servers in general very much since distribution more or less ( as of last year ) has stop shipping rsylog as part of their default install and or ship it disabled since the introduction of systemd's journal.

Rsyslog has moved away from being a "generic" syslog solution to becoming one of the many "centralized" syslog solution available on the market to host locally or are used for customer based cloud solutions.

With the exception of server administrators that are still in the "petting their server" mindset and or have general hard time adapting and learning new things or for what ever ( infrastructure ) reason they still have to rely on textual log files residing on host itself, everyone has move to create "centralized-syslog.service" which consist of more or less of what I have outlined here before ( to work around lack of an proper configurable implementation of doing this in the configuration for the journal which has been the nr.1 criticize the journal has received in the past years ) in my previous comments ( using mostly the json format ) and configure the storage of the journal to be volatile. There are few solution,setups out there that want to deal with the json file directly locally but then you pretty much just use the same thing I outlined before but with ">" + PATH and have a timer unit for the classic file rotation so still no need for a full blown centralized syslog solution being installed on the host.

@rgerhards

This comment has been minimized.

Show comment
Hide comment
@rgerhards

rgerhards Aug 25, 2015

Contributor

@johannbg I should have mentioned that I specifically tried to address this part of your posting: "( the imjournal module is not officially supported upstream and I question of rsyslog will ever officially support journal since last time I checked Rainer Gerhards had some axe to grind with Lennart or binary logging in general )". To me, this sounds like I try to "sabotage" journal integration. Quite the opposite is true. Of course, I didn't like the way the journal was introduced, but that's history. No need to bother ever and ever again about it.

On this thread, I am not trying to to request anything from the journal community or make any other point in it. Hwo the journal deals with audit is between audit and journal. Not interested in it (except that I don't intend to add work-arounds in rsyslog for those that don't like what journal does).

Besides, I think we are not that far away in our view. There are different needs, and journal and (r)syslog are different solutions for different needs. I still think rsyslog is a (very) "generic" syslog solution, but that depends on your definitions of "generic" and "syslog solution", so no point to argue.

Bottom line: journal & rsyslog do coexist, both projects have different use case bases and priorities, and, yes, for the casual end user there is absolutely no need to worry about syslog. The picture slightly changes for the tech-savy end user who wants to monitor his router --- this is why we have omjournal, which permits to write those router syslog messages to the journal, so that the user can deal with journal tools, only. And since two or three years I recommend to distros to ship this configuration as a standard optional component for users with this need -- instead of a full blown rsyslog config. Unfortunately, this advise is not followed.

Contributor

rgerhards commented Aug 25, 2015

@johannbg I should have mentioned that I specifically tried to address this part of your posting: "( the imjournal module is not officially supported upstream and I question of rsyslog will ever officially support journal since last time I checked Rainer Gerhards had some axe to grind with Lennart or binary logging in general )". To me, this sounds like I try to "sabotage" journal integration. Quite the opposite is true. Of course, I didn't like the way the journal was introduced, but that's history. No need to bother ever and ever again about it.

On this thread, I am not trying to to request anything from the journal community or make any other point in it. Hwo the journal deals with audit is between audit and journal. Not interested in it (except that I don't intend to add work-arounds in rsyslog for those that don't like what journal does).

Besides, I think we are not that far away in our view. There are different needs, and journal and (r)syslog are different solutions for different needs. I still think rsyslog is a (very) "generic" syslog solution, but that depends on your definitions of "generic" and "syslog solution", so no point to argue.

Bottom line: journal & rsyslog do coexist, both projects have different use case bases and priorities, and, yes, for the casual end user there is absolutely no need to worry about syslog. The picture slightly changes for the tech-savy end user who wants to monitor his router --- this is why we have omjournal, which permits to write those router syslog messages to the journal, so that the user can deal with journal tools, only. And since two or three years I recommend to distros to ship this configuration as a standard optional component for users with this need -- instead of a full blown rsyslog config. Unfortunately, this advise is not followed.

@davidelang

This comment has been minimized.

Show comment
Hide comment
@davidelang

davidelang Aug 25, 2015

@johannbg There are significant security reasons for getting the logs off the box ASAP to a central place (this is required for PCI for example)

It's also much better to do the analysis and alerting on logs in a central place that doesn't impact the performance of the servers that your users are using.

It's not just fuddy-duddy sysadmins with a "petting their server" mindset. Centralized logging is even more important in a cloud/virtualized environment where systems come and go because you won't be able to look at the logs on a system that's been stopped/deleted.

davidelang commented Aug 25, 2015

@johannbg There are significant security reasons for getting the logs off the box ASAP to a central place (this is required for PCI for example)

It's also much better to do the analysis and alerting on logs in a central place that doesn't impact the performance of the servers that your users are using.

It's not just fuddy-duddy sysadmins with a "petting their server" mindset. Centralized logging is even more important in a cloud/virtualized environment where systems come and go because you won't be able to look at the logs on a system that's been stopped/deleted.

@johannbg

This comment has been minimized.

Show comment
Hide comment
@johannbg

johannbg Aug 25, 2015

Contributor

@davidelang I work in pci compliance environments so I'm perfectly aware of the importance of centralised syslog servers and I even pointed out how everybody is pushing journal entries to those servers without the use of textual log files and rsyslog and the likes so I'm not sure what point you are trying to make here.

Contributor

johannbg commented Aug 25, 2015

@davidelang I work in pci compliance environments so I'm perfectly aware of the importance of centralised syslog servers and I even pointed out how everybody is pushing journal entries to those servers without the use of textual log files and rsyslog and the likes so I'm not sure what point you are trying to make here.

@peterbowey

This comment has been minimized.

Show comment
Hide comment
@peterbowey

peterbowey Jan 23, 2016

@poettering

I am so very relieved that "audit=0" on the kernel cmdline works (to shut-up this [new] forced "audit" thing), as since upgrading to Fedora 23 then 24 Fedora 24 (rawhide), I have been seeing far too much "/var/log/messages" noise related to "audit" logging: (6Mb logs per day is a bit too much).

I find it strange that 'audit' persists even when it is 'removed' by the traditional and normal means?

With "audit=0" on the kernel cmdline, no longer do I have to look through many pages of this typical audit logging:

Jan 23 22:29:04 ns1 kernel: audit: type=2404 audit(1453550344.061:660): pid=1635 uid=0 auid=4294967295 ses=4294967295 msg='op=destroy kind=server fp=SHA256:e8:74:20:bd:fc:7a:6d:db:fc:a0:3d:f2:42:60:70:cd:ea:16:4
9:d3:57:9e:74:dd:40:07:1d:ed:43:ec:1e:f2 direction=? spid=1635 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
Jan 23 22:29:04 ns1 kernel: audit: type=2404 audit(1453550344.061:661): pid=1635 uid=0 auid=4294967295 ses=4294967295 msg='op=destroy kind=server fp=SHA256:30:cb:e9:2b:fb:74:21:02:4e:03:66:6f:64:c1:5a:4e:1d:53:b
8:79:2b:85:30:ee:2b:75:75:d1:02:13:44:dd direction=? spid=1635 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
Jan 23 22:29:04 ns1 kernel: audit: type=2404 audit(1453550344.061:662): pid=1635 uid=0 auid=4294967295 ses=4294967295 msg='op=destroy kind=server fp=SHA256:c5:72:55:bf:b9:ef:8c:26:2c:a8:5b:da:a1:66:0d:e9:8f:3f:5
1:08:99:06:04:7c:a6:41:56:e7:24:3e:51:c7 direction=? spid=1635 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
Jan 23 22:29:04 ns1 kernel: audit: type=2404 audit(1453550344.061:663): pid=1635 uid=0 auid=4294967295 ses=4294967295 msg='op=destroy kind=server fp=SHA256:cb:4a:76:c6:78:71:fa:4e:e7:1e:38:7f:b6:71:82:7f:86:21:6
6:25:61:51:9b:d1:f5:24:66:b0:4f:a1:84:f9 direction=? spid=1635 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
Jan 23 22:29:04 ns1 kernel: audit: type=1106 audit(1453550344.062:664): pid=2605 uid=0 auid=0 ses=1 msg='op=PAM:session_close grantors=pam_selinux,pam_loginuid,pam_selinux,pam_namespace,pam_keyinit,pam_keyinit,p
am_limits,pam_systemd,pam_unix,pam_lastlog acct="root" exe="/usr/sbin/sshd" hostname=165.228.91.94 addr=165.228.91.94 terminal=ssh res=success'
Jan 23 22:29:04 ns1 kernel: audit: type=1104 audit(1453550344.062:665): pid=2605 uid=0 auid=0 ses=1 msg='op=PAM:setcred grantors=pam_env,pam_unix acct="root" exe="/usr/sbin/sshd" hostname=165.228.91.94 addr=165.
228.91.94 terminal=ssh res=success'

peterbowey commented Jan 23, 2016

@poettering

I am so very relieved that "audit=0" on the kernel cmdline works (to shut-up this [new] forced "audit" thing), as since upgrading to Fedora 23 then 24 Fedora 24 (rawhide), I have been seeing far too much "/var/log/messages" noise related to "audit" logging: (6Mb logs per day is a bit too much).

I find it strange that 'audit' persists even when it is 'removed' by the traditional and normal means?

With "audit=0" on the kernel cmdline, no longer do I have to look through many pages of this typical audit logging:

Jan 23 22:29:04 ns1 kernel: audit: type=2404 audit(1453550344.061:660): pid=1635 uid=0 auid=4294967295 ses=4294967295 msg='op=destroy kind=server fp=SHA256:e8:74:20:bd:fc:7a:6d:db:fc:a0:3d:f2:42:60:70:cd:ea:16:4
9:d3:57:9e:74:dd:40:07:1d:ed:43:ec:1e:f2 direction=? spid=1635 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
Jan 23 22:29:04 ns1 kernel: audit: type=2404 audit(1453550344.061:661): pid=1635 uid=0 auid=4294967295 ses=4294967295 msg='op=destroy kind=server fp=SHA256:30:cb:e9:2b:fb:74:21:02:4e:03:66:6f:64:c1:5a:4e:1d:53:b
8:79:2b:85:30:ee:2b:75:75:d1:02:13:44:dd direction=? spid=1635 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
Jan 23 22:29:04 ns1 kernel: audit: type=2404 audit(1453550344.061:662): pid=1635 uid=0 auid=4294967295 ses=4294967295 msg='op=destroy kind=server fp=SHA256:c5:72:55:bf:b9:ef:8c:26:2c:a8:5b:da:a1:66:0d:e9:8f:3f:5
1:08:99:06:04:7c:a6:41:56:e7:24:3e:51:c7 direction=? spid=1635 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
Jan 23 22:29:04 ns1 kernel: audit: type=2404 audit(1453550344.061:663): pid=1635 uid=0 auid=4294967295 ses=4294967295 msg='op=destroy kind=server fp=SHA256:cb:4a:76:c6:78:71:fa:4e:e7:1e:38:7f:b6:71:82:7f:86:21:6
6:25:61:51:9b:d1:f5:24:66:b0:4f:a1:84:f9 direction=? spid=1635 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
Jan 23 22:29:04 ns1 kernel: audit: type=1106 audit(1453550344.062:664): pid=2605 uid=0 auid=0 ses=1 msg='op=PAM:session_close grantors=pam_selinux,pam_loginuid,pam_selinux,pam_namespace,pam_keyinit,pam_keyinit,p
am_limits,pam_systemd,pam_unix,pam_lastlog acct="root" exe="/usr/sbin/sshd" hostname=165.228.91.94 addr=165.228.91.94 terminal=ssh res=success'
Jan 23 22:29:04 ns1 kernel: audit: type=1104 audit(1453550344.062:665): pid=2605 uid=0 auid=0 ses=1 msg='op=PAM:setcred grantors=pam_env,pam_unix acct="root" exe="/usr/sbin/sshd" hostname=165.228.91.94 addr=165.
228.91.94 terminal=ssh res=success'
@keszybz

This comment has been minimized.

Show comment
Hide comment
@keszybz

keszybz Jan 25, 2016

Member
systemctl mask systemd-journald-audit.socket

should do the trick and is easiest solution to disable audit logs currently.

Member

keszybz commented Jan 25, 2016

systemctl mask systemd-journald-audit.socket

should do the trick and is easiest solution to disable audit logs currently.

fbuihuu added a commit to openSUSE/systemd that referenced this issue Jun 30, 2016

journald: disable audit support completely from the journal
This patch not only prevents journald to enable audit system
unconditionally very early at boot but also prevents it to receive
audit messages for the audit netlink and to push them into the
journal.

The first reason is that when journald enables kernel audit, it does
not disable syscall audit (it doesn't load the audit rules), which
introduced a global performance hit. This can be minimized if audit
service is started but that's not the case for all systems.

The second reason is that for systems where audit was disabled by
default they will suddenly have audit enabled (unless audit=0 was
already passed to the kernel command line). This means tons of audit
messages will be sent to dmesg, syslog, journal files, etc...

Note also that audit messages are duplicated in the journal since they
are received both from kmsg and from the audit netlink. A related bug
report can be found here:
https://bugzilla.redhat.com/show_bug.cgi?id=1160046.

This basically reverts the following upstream commits:

 - 875c2e2
 - 4d9ced9

Upstream issue:
systemd/systemd#959

So disable all of this for now until a better option is found or
someone comes up with a real use case.

[fbui: bsc#984034]

fbuihuu added a commit to openSUSE/systemd that referenced this issue Aug 16, 2016

journald: disable audit support completely from the journal
This patch not only prevents journald to enable audit system
unconditionally very early at boot but also prevents it to receive
audit messages for the audit netlink and to push them into the
journal.

The first reason is that when journald enables kernel audit, it does
not disable syscall audit (it doesn't load the audit rules), which
introduced a global performance hit. This can be minimized if audit
service is started but that's not the case for all systems.

The second reason is that for systems where audit was disabled by
default they will suddenly have audit enabled (unless audit=0 was
already passed to the kernel command line). This means tons of audit
messages will be sent to dmesg, syslog, journal files, etc...

Note also that audit messages are duplicated in the journal since they
are received both from kmsg and from the audit netlink. A related bug
report can be found here:
https://bugzilla.redhat.com/show_bug.cgi?id=1160046.

This basically reverts the following upstream commits:

 - 875c2e2
 - 4d9ced9

Upstream issue:
systemd/systemd#959

So disable all of this for now until a better option is found or
someone comes up with a real use case.

[fbui: bsc#984034]

fbuihuu added a commit to openSUSE/systemd that referenced this issue Aug 22, 2016

journald: disable audit support completely from the journal
This patch not only prevents journald to enable audit system
unconditionally very early at boot but also prevents it to receive
audit messages for the audit netlink and to push them into the
journal.

The first reason is that when journald enables kernel audit, it does
not disable syscall audit (it doesn't load the audit rules), which
introduced a global performance hit. This can be minimized if audit
service is started but that's not the case for all systems.

The second reason is that for systems where audit was disabled by
default they will suddenly have audit enabled (unless audit=0 was
already passed to the kernel command line). This means tons of audit
messages will be sent to dmesg, syslog, journal files, etc...

Note also that audit messages are duplicated in the journal since they
are received both from kmsg and from the audit netlink. A related bug
report can be found here:
https://bugzilla.redhat.com/show_bug.cgi?id=1160046.

This basically reverts the following upstream commits:

 - 875c2e2
 - 4d9ced9

Upstream issue:
systemd/systemd#959

So disable all of this for now until a better option is found or
someone comes up with a real use case.

[fbui: bsc#984034]

dezgeg added a commit to dezgeg/nixpkgs that referenced this issue Aug 22, 2016

audit module: Disable audit via kernel option if module is not enabled
Disable it via the big hammer (audit=0 in kernel params) to prevent
journald from starting the audit system anyway (as having audit enabled
but doing nothing still takes a performance hit).

Some reading:
    - https://fedorahosted.org/fesco/ticket/1311
    - systemd/systemd#959
    - openSUSE/systemd@64f83d3

Unfortunately, this means enabling/disabling audit now needs a reboot.
But typically that option won't be changed too often.

dezgeg added a commit to dezgeg/nixpkgs that referenced this issue Aug 28, 2016

dezgeg added a commit to NixOS/nixpkgs that referenced this issue Aug 31, 2016

fbuihuu added a commit to fbuihuu/systemd-opensuse-next that referenced this issue Nov 9, 2016

journald: disable audit support completely from the journal
This patch not only prevents journald to enable audit system
unconditionally very early at boot but also prevents it to receive
audit messages for the audit netlink and to push them into the
journal.

The first reason is that when journald enables kernel audit, it does
not disable syscall audit (it doesn't load the audit rules), which
introduced a global performance hit. This can be minimized if audit
service is started but that's not the case for all systems.

The second reason is that for systems where audit was disabled by
default they will suddenly have audit enabled (unless audit=0 was
already passed to the kernel command line). This means tons of audit
messages will be sent to dmesg, syslog, journal files, etc...

Note also that audit messages are duplicated in the journal since they
are received both from kmsg and from the audit netlink. A related bug
report can be found here:
https://bugzilla.redhat.com/show_bug.cgi?id=1160046.

This basically reverts the following upstream commits:

 - 875c2e2
 - 4d9ced9

Upstream issue:
systemd/systemd#959

So disable all of this for now until a better option is found or
someone comes up with a real use case.

[fbui: bsc#984034]

fbuihuu added a commit to fbuihuu/systemd-opensuse-next that referenced this issue Mar 23, 2017

journald: disable audit support completely from the journal
This patch not only prevents journald to enable audit system
unconditionally very early at boot but also prevents it to receive
audit messages for the audit netlink and to push them into the
journal.

The first reason is that when journald enables kernel audit, it does
not disable syscall audit (it doesn't load the audit rules), which
introduced a global performance hit. This can be minimized if audit
service is started but that's not the case for all systems.

The second reason is that for systems where audit was disabled by
default they will suddenly have audit enabled (unless audit=0 was
already passed to the kernel command line). This means tons of audit
messages will be sent to dmesg, syslog, journal files, etc...

Note also that audit messages are duplicated in the journal since they
are received both from kmsg and from the audit netlink. A related bug
report can be found here:
https://bugzilla.redhat.com/show_bug.cgi?id=1160046.

This basically reverts the following upstream commits:

 - 875c2e2
 - 4d9ced9

Upstream issue:
systemd/systemd#959

So disable all of this for now until a better option is found or
someone comes up with a real use case.

[fbui: bsc#984034]

fbuihuu added a commit to fbuihuu/systemd-opensuse-next that referenced this issue May 24, 2017

journald: disable audit support completely from the journal
This patch not only prevents journald to enable audit system
unconditionally very early at boot but also prevents it to receive
audit messages for the audit netlink and to push them into the
journal.

The first reason is that when journald enables kernel audit, it does
not disable syscall audit (it doesn't load the audit rules), which
introduced a global performance hit. This can be minimized if audit
service is started but that's not the case for all systems.

The second reason is that for systems where audit was disabled by
default they will suddenly have audit enabled (unless audit=0 was
already passed to the kernel command line). This means tons of audit
messages will be sent to dmesg, syslog, journal files, etc...

Note also that audit messages are duplicated in the journal since they
are received both from kmsg and from the audit netlink. A related bug
report can be found here:
https://bugzilla.redhat.com/show_bug.cgi?id=1160046.

This basically reverts the following upstream commits:

 - 875c2e2
 - 4d9ced9

Upstream issue:
systemd/systemd#959

So disable all of this for now until a better option is found or
someone comes up with a real use case.

[fbui: bsc#984034]

damentz added a commit to damentz/liquorix-package that referenced this issue May 31, 2017

Make systemd auditing opt-in
Per github issue below, the devs behind systemd are not interested in
making audit messages more sane or automatically logged.  Therefore, we
will make auditing force opt-in in Liquorix.

systemd/systemd#959

fbuihuu added a commit to fbuihuu/systemd-opensuse-next that referenced this issue Jun 1, 2017

journald: disable audit support completely from the journal
This patch not only prevents journald to enable audit system
unconditionally very early at boot but also prevents it to receive
audit messages for the audit netlink and to push them into the
journal.

The first reason is that when journald enables kernel audit, it does
not disable syscall audit (it doesn't load the audit rules), which
introduced a global performance hit. This can be minimized if audit
service is started but that's not the case for all systems.

The second reason is that for systems where audit was disabled by
default they will suddenly have audit enabled (unless audit=0 was
already passed to the kernel command line). This means tons of audit
messages will be sent to dmesg, syslog, journal files, etc...

Note also that audit messages are duplicated in the journal since they
are received both from kmsg and from the audit netlink. A related bug
report can be found here:
https://bugzilla.redhat.com/show_bug.cgi?id=1160046.

This basically reverts the following upstream commits:

 - 875c2e2
 - 4d9ced9

Upstream issue:
systemd/systemd#959

So disable all of this for now until a better option is found or
someone comes up with a real use case.

[fbui: bsc#984034]

fbuihuu added a commit to openSUSE/systemd that referenced this issue Jul 21, 2017

journald: disable audit support completely from the journal
This patch not only prevents journald to enable audit system
unconditionally very early at boot but also prevents it to receive
audit messages for the audit netlink and to push them into the
journal.

The first reason is that when journald enables kernel audit, it does
not disable syscall audit (it doesn't load the audit rules), which
introduced a global performance hit. This can be minimized if audit
service is started but that's not the case for all systems.

The second reason is that for systems where audit was disabled by
default they will suddenly have audit enabled (unless audit=0 was
already passed to the kernel command line). This means tons of audit
messages will be sent to dmesg, syslog, journal files, etc...

Note also that audit messages are duplicated in the journal since they
are received both from kmsg and from the audit netlink. A related bug
report can be found here:
https://bugzilla.redhat.com/show_bug.cgi?id=1160046.

This basically reverts the following upstream commits:

 - 875c2e2
 - 4d9ced9

Upstream issue:
systemd/systemd#959

So disable all of this for now until a better option is found or
someone comes up with a real use case.

[fbui: bsc#984034]

fbuihuu added a commit to openSUSE/systemd that referenced this issue Mar 1, 2018

journald: disable audit support completely from the journal
This patch not only prevents journald to enable audit system
unconditionally very early at boot but also prevents it to receive
audit messages for the audit netlink and to push them into the
journal.

The first reason is that when journald enables kernel audit, it does
not disable syscall audit (it doesn't load the audit rules), which
introduced a global performance hit. This can be minimized if audit
service is started but that's not the case for all systems.

The second reason is that for systems where audit was disabled by
default they will suddenly have audit enabled (unless audit=0 was
already passed to the kernel command line). This means tons of audit
messages will be sent to dmesg, syslog, journal files, etc...

Note also that audit messages are duplicated in the journal since they
are received both from kmsg and from the audit netlink. A related bug
report can be found here:
https://bugzilla.redhat.com/show_bug.cgi?id=1160046.

This basically reverts the following upstream commits:

 - 875c2e2
 - 4d9ced9

Upstream issue:
systemd/systemd#959

So disable all of this for now until a better option is found or
someone comes up with a real use case.

[fbui: bsc#984034]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment