Fixes #22493 - structured logging support #5237
Conversation
|
Issues: #22493 |
lib/foreman/logging.rb
Outdated
| when 'json' | ||
| ::Logging::Layouts::Parseable.json(:items => @config[:items]) | ||
| when 'pattern' | ||
| ::Logging::Layouts::pattern(:pattern => pattern, :color_scheme => colorize ? 'bright' : nil) |
lzap
force-pushed
the
structured-logging-22493
branch
from
9863c30
to
5f4dae5
Compare
| - mdc | ||
| - ndc | ||
| :pattern: "%d [%.1l|%.3c|%.5X{request}] %m\n" | ||
| :sys_pattern: "[%.1l|%.3c|%.5X{request}] %m\n" |
There was a problem hiding this comment.
Option sys_pattern is only used for syslog/journal, note there is no date prefix as it's not needed in this case.
There was a problem hiding this comment.
Is this really used though? I have layout: multiline_pattern and type: journald - but my logs look like:
3月 13 10:18:35 centos7-devel.lobatolan.home foreman[17690]: 2018-03-13T10:18:35 [I|app|a08c8] Rendered /home/vagrant/foreman-plugin/redhat-access/app/views/dashboard/_insights_issues_widget.html.erb (1732.3ms)
3月 13 10:18:35 centos7-devel.lobatolan.home foreman[17690]: 2018-03-13T10:18:35 [I|app|a08c8] Completed 200 OK in 3329ms (Views: 1727.8ms | ActiveRecord: 1558.9ms)
So the date is printed twice.
| @@ -170,8 +169,7 @@ class Application < Rails::Application | |||
| # Catching Invalid JSON Parse Errors with Rack Middleware | |||
| config.middleware.use Middleware::CatchJsonParseErrors | |||
|
|
|||
| # Record request ID in logging MDC storage | |||
| config.middleware.insert_before Rails::Rack::Logger, Middleware::TaggedLogging | |||
There was a problem hiding this comment.
This patch drops TaggedLogging middleware and incorporate everything into SessionSafeLogging one.
| # variables set by app | ||
| ::Logging.mdc.delete('user_login') | ||
| ::Logging.mdc.delete('org_id') | ||
| ::Logging.mdc.delete('loc_id') | ||
| end |
There was a problem hiding this comment.
Previously there were three mdc variables: request, session and session_safe. Since original "session" was not safe, this changes it to: request, session and remote_ip (additional field).
|
|
||
| @app.call(env) | ||
| ensure | ||
| ::Logging.mdc.delete('session_safe') | ||
| # variables set by middleware | ||
| ::Logging.mdc.delete('remote_ip') |
There was a problem hiding this comment.
is there anything we would like to keep here? Perhaps it would be safer just to .clean this.
There was a problem hiding this comment.
Yes good idea, diagnostic context should start from scratch on each request, adding clear then.
| ::Logging.mdc.delete('session_safe') | ||
| # variables set by middleware | ||
| ::Logging.mdc.delete('remote_ip') | ||
| ::Logging.mdc.delete('request') |
There was a problem hiding this comment.
Since we do more here than just session_safe setup, we should probably rename the class to LoggingMdcSetup or something like that.
There was a problem hiding this comment.
Done, named it LoggingContext then.
lib/foreman/logging.rb
Outdated
| when 'file' | ||
| build_file_appender(name, options) | ||
| else | ||
| fail 'unsupported logger type, please choose syslog or file' | ||
| end | ||
| end | ||
|
|
||
| def options_with_syslog_facility(options, defaults) | ||
| opts = options.reverse_merge(defaults) | ||
| opts[:facility] = self.class.const_get("::Syslog::Constants::#{opts[:facility]}") |
There was a problem hiding this comment.
As a developer, in which case I would want to override the :facility? Should generic foreman code expose this via API?
There was a problem hiding this comment.
User wants to change facility when processing logs in syslog. Not following.
There was a problem hiding this comment.
I take it as "also log foreman exception code along with other fields", good idea. Added:
extra_fields[:foreman_code] = exception.code if exception.respond_to?(:code)
There was a problem hiding this comment.
I haven't meant that (I mean foreman source code), but good idea anyway :)
I might just not understand the meaning of the :facility and when in the source code would I want to use the non-default one.
config/logging.yaml
Outdated
| # Formatter, one of 'pattern', 'multiline_pattern' or 'json'. The multiline pattern prepends | ||
| # '|' characters on all additional lines for better readability. | ||
| # | ||
| # :items: |
There was a problem hiding this comment.
s/items/json_items, otherwise, it's not clear what this value is being used for.
|
Thanks for initial review, rebasing. diff --git a/config/logging.yaml b/config/logging.yaml
index 036e1972d..d7851dec5 100644
--- a/config/logging.yaml
+++ b/config/logging.yaml
@@ -37,7 +37,7 @@
# Formatter, one of 'pattern', 'multiline_pattern' or 'json'. The multiline pattern prepends
# '|' characters on all additional lines for better readability.
#
-# :items:
+# :json_items:
# Items to include in JSON. Only applies when 'json' layout is set. Possible items are
# logger timestamp level message file line method hostname pid millis thread_id thread
# mdc ndc. Global fields like session or organization are available under 'mdc' item,
@@ -77,7 +77,7 @@
:level: info
:type: journald
:layout: multiline_pattern
- :items:
+ :json_items:
- logger
- timestamp
- level
diff --git a/lib/foreman/logging.rb b/lib/foreman/logging.rb
index e6ee40801..c6304afab 100644
--- a/lib/foreman/logging.rb
+++ b/lib/foreman/logging.rb
@@ -86,6 +86,7 @@ module Foreman
exception_message: exception.message,
exception_backtrace: backtrace
}
+ extra_fields[:foreman_code] = exception.code if exception.respond_to?(:code)
with_fields(extra_fields) do
self.logger(logger_name).public_send(level) do
([context_message, "#{exception.class}: #{exception.message}"] + backtrace).join("\n")
@@ -190,7 +191,7 @@ module Foreman
colorize = nil unless enable_colors
case @config[:layout]
when 'json'
- ::Logging::Layouts::Parseable.json(:items => @config[:items])
+ ::Logging::Layouts::Parseable.json(:items => @config[:json_items])
when 'pattern'
::Logging::Layouts.pattern(:pattern => pattern, :color_scheme => colorize ? 'bright' : nil)
when 'multiline_pattern |
lzap
force-pushed
the
structured-logging-22493
branch
from
5f4dae5
to
ad6285b
Compare
|
You mentioned that logging-journald would become a hard dependency. Does this mean that journald existing will also be a hard dependency? I assume console logging, even of structured data, will continue to exist with this change. |
Gemfile
Outdated
| @@ -31,6 +31,7 @@ gem 'gettext_i18n_rails', '~> 1.0' | |||
| gem 'rails-i18n', '~> 5.0.0' | |||
| gem 'turbolinks', '>= 2.5.4', '< 3' | |||
| gem 'logging', '>= 1.8.0', '< 3.0.0' | |||
| gem 'logging-journald', '~> 1.0' | |||
There was a problem hiding this comment.
We could pre-emptively open a packaging PR for this.
There was a problem hiding this comment.
I am extracting this into bundler.d/journald.rb so developers can easily do bundle install --without journald in order to avoid installing systemd-devel.
| @@ -73,6 +73,7 @@ def current=(o) | |||
| Rails.logger.info "Current user: #{o.login} (#{o.admin? ? 'administrator' : 'regular user'})" | |||
| end | |||
| Rails.logger.debug "Setting current user thread-local variable to " + (o.is_a?(User) ? o.login : 'nil') | |||
| ::Logging.mdc['user_login'] = o.try(:login) | |||
There was a problem hiding this comment.
How does this know which logger to send this information to? Does it default to app logger unless otherwise specified? Can you specify which logger? e.g. if I wanted to do this with katello logger?
There was a problem hiding this comment.
There are two diagnostic contexts: message (mdc) and nested (ndc). The former one can be seen as "global" - fields in here gets logged along every single message. If you want only to attach fields to particular message, then use ndc. There is a helper with_fields described in the text of this PR.
Now what you are asking for is something in the middle, I don't have code for that. Can you give examples of fields which should go only into specific logger?
| # Only messages with severity >= this level from all loggers will be logged | ||
| # values: debug, info, warn, error, fatal, unknown | ||
| # | ||
| # :type: | ||
| # Type of logging, for file variant you can specify other options below like filename | ||
| # values: file, syslog | ||
| # values: file, syslog or journald |
There was a problem hiding this comment.
Pardon my ignorance here, do you mind a brief reminder of what the difference between syslog and journald are?
There was a problem hiding this comment.
Sure, syslog has an "API" that only allows sending strings while system journal supports message and arbitrary amount of key=value pairs. Supporting both give options to users:
- send logs to stdout/file in plain text
- send logs to syslog in plain text
- send logs to syslog in JSON
- send logs to journal in plain text with additional fields
- send logs to journal in JSON with additional fields
If you want to integrate with logstask (ELK) or similar 3rd party, you will likely use syslog/JSON configuration. If you want to keep standalone logging, the most flexible configuration (I am going to propose as sane default) will be journal/plain text. Advantages:
- messages are human-readable and it is approachable by users
- log entries are enriched with extra fields
- by default journald forwards to rsyslog on EL7 so extra data is lost
- user can decide to disable volatile journal and start capturing logs with extra fields
- extra fields are still visible/searchable in volatile journal (50MB memory buffer by default)
- thank to syslog facility setting, installer will configure rsyslog to send those into production.log (no big change for users - everything stays "as-is")
- journald can be easily configured to pull logs from remote hosts in the same format (smart proxies - the next step in my effort)
- with very little configuration changes (disable volatile, configure fetching from proxies) user can achive good central structured logging with extra features like log sealing and filtering
Hope it makes more sense now. The key is ability to integrate with both worlds.
config/logging.yaml
Outdated
| # :facility: | ||
| # System log/journal facility to use. One of LOG_AUTH, LOG_AUTHPRIV, LOG_CONSOLE, | ||
| # LOG_CRON, LOG_DAEMON, LOG_FTP, LOG_KERN, LOG_LPR, LOG_MAIL, LOG_NEWS, LOG_NTP, | ||
| # LOG_SECURITY, LOG_SYSLOG, LOG_USER, LOG_UUCP, LOG_LOCAL0 to LOG_LOCAL7. |
There was a problem hiding this comment.
Is there a good resource to look up what these facilities mean? If yes, should we consider putting a link here in the comments?
There was a problem hiding this comment.
Sure, adding this to the text. It's in man 3 syslog.
|
The first thing I meant to say is, awesome! Glad you are taking this on and got an implementation up. The third question I had was around clarity of configuration. I, wrongly assumed, that I'd be able to say the kinda of logger I want (console, file, syslong, journald) and the type of structure I'd want (traditional or json). How I go about doing that is unclear to me looking at the configuration options. |
|
First of all, the only requirement for the native gem to be built is libsystemd and systemd/sd-journal.h - these are both shipped in |
|
Answer to the third one (hopefully I did not miss anything): the complexity stems from logging gem, it is very flexible (which I like). Maybe we can find easier way of configuring this, how about commented lines in Here are some examples: # Log to production.log via file appender and multi-line friendly pattern
#:logging:
# :level: info
# :production:
# :type: file
# :layout: multiline_pattern
# Log to syslog in JSON format for integrating with 3rd parties (ELK)
#:logging:
# :level: info
# :production:
# :type: syslog
# :layout: json
# Send logs to system journal with structured data with simple text pattern
#:logging:
# :level: info
# :production:
# :type: journald
# :layout: pattern |
|
Hmm I wonder why jenkins complains while I pushed it already: https://rubygems.org/gems/logging-journald |
lzap
force-pushed
the
structured-logging-22493
branch
from
ad6285b
to
28763fb
Compare
config/logging.yaml
Outdated
| :default: | ||
| :colorize: false | ||
| :console_inline: false | ||
| :log_trace: false | ||
| :level: info | ||
| :type: file | ||
| :pattern: "%d %.8X{request} [%c] [%.1l] %m\n" | ||
| :type: journald |
There was a problem hiding this comment.
I changed type to journald by mistake. By default it should remain as file for development environments, it's our installer which will set it to journald by default.
lzap
force-pushed
the
structured-logging-22493
branch
from
28763fb
to
48c0c8a
Compare
|
Rebase after chat with @iNecas, he found that default setting for dev mode was not |
|
For the record, here is additional discussion about this PR: https://community.theforeman.org/t/rfc-unified-and-structured-logging/8334/5 |
bundler.d/journald.rb
Outdated
| @@ -0,0 +1 @@ | |||
| gem 'logging-journald', '~> 1.0' | |||
There was a problem hiding this comment.
you need to specify the group here.
group :logging do
gem 'logging-journald', '~> 1.0'
end
lzap
force-pushed
the
structured-logging-22493
branch
from
48c0c8a
to
2239de0
Compare
lzap
force-pushed
the
structured-logging-22493
branch
from
2239de0
to
ee39819
Compare
theforeman-bot
added
Needs re-review
Needs testing
and removed
Waiting on contributor
labels
|
Rebased, ready for test @dLobatog ping me for questions on IRC. |
|
I know @timogoebel has an ELK stack, would appreciate your feedback on this. |
|
[test upgrade] |
|
I am pushing a small change on top - support for logging arbitrary blobs (multiline texts). For more info see the discussion here: https://community.theforeman.org/t/rfc-file-audit-for-template-rendering/7670/4 In short, we want to log rendered template contents on every render for logging/audit purposes. There is now new helper method called |
diff --git a/config/application.rb b/config/application.rb
index f14abf41f..ce86b66f4 100644
--- a/config/application.rb
+++ b/config/application.rb
@@ -211,7 +211,9 @@ module Foreman
:templates => {:enabled => true},
:notifications => {:enabled => true},
:background => {:enabled => true},
- :dynflow => {:enabled => true}
+ :dynflow => {:enabled => true},
+ :telemetry => {:enabled => true},
+ :blob => {:enabled => true},
))
config.logger = Foreman::Logging.logger('app')
diff --git a/config/settings.yaml.example b/config/settings.yaml.example
index 377a40429..73c8d0aeb 100644
--- a/config/settings.yaml.example
+++ b/config/settings.yaml.example
@@ -93,6 +93,8 @@
# :enabled: true
# :telemetry:
# :enabled: true
+# :blob:
+# :enabled: true
# Foreman telemetry has three destinations: prometheus, statsd and rails log.
:telemetry:
diff --git a/lib/foreman/logging.rb b/lib/foreman/logging.rb
index f033c6077..a34bc91f6 100644
--- a/lib/foreman/logging.rb
+++ b/lib/foreman/logging.rb
@@ -94,6 +94,16 @@ module Foreman
end
end
+ def blob(message, contents, extra_fields = {})
+ logger_name = extra_fields[:logger] || 'blob'
+ with_fields(extra_fields) do
+ self.logger(logger_name).info do
+ message + "\n" + contents
+ end
+ end
+ contents
+ end
+
private
def load_config(environment, overrides = {})
diff --git a/lib/foreman/renderer.rb b/lib/foreman/renderer.rb
index 400c94318..4d952ae23 100644
--- a/lib/foreman/renderer.rb
+++ b/lib/foreman/renderer.rb
@@ -243,7 +243,15 @@ module Foreman
raise ::Foreman::Exception.new(N_("Template '%s' is either missing or has an invalid organization or location"), @template_name) if template.nil?
content = template.respond_to?(:template) ? template.template : template
allowed_variables = allowed_variables_mapping(ALLOWED_VARIABLES)
- render_safe content, ALLOWED_HELPERS, allowed_variables, scope_variables
+ result = render_safe content, ALLOWED_HELPERS, allowed_variables, scope_variables
+ digest = Digest::SHA256.hexdigest(result)
+ Foreman::Logging.blob("Unattended render of '#{@template_name}' = '#{digest}'", result,
+ template_digest: digest,
+ template_name: @template_name,
+ template_context: self.class.name,
+ template_host_name: @host.try(:name),
+ template_host_id: @host.try(:id))
+ result
end
def unattended_render_to_temp_file(content, prefix = id.to_s, options = {}) |
lzap
force-pushed
the
structured-logging-22493
branch
from
ee39819
to
b5d98f7
Compare
|
@lzap: I have asked my co-workers - who know more about Logging then I do - to take a look. |
|
travis needs also updating to get the systemd development files installed. |
lzap
force-pushed
the
structured-logging-22493
branch
from
b5d98f7
to
4cab476
Compare
config/application.rb
Outdated
| :dynflow => {:enabled => true} | ||
| :dynflow => {:enabled => true}, | ||
| :telemetry => {:enabled => true}, | ||
| :blob => {:enabled => true}, |
There was a problem hiding this comment.
Avoid comma after the last parameter of a method call.
config/application.rb
Outdated
| :dynflow => {:enabled => true} | ||
| :dynflow => {:enabled => true}, | ||
| :telemetry => {:enabled => true}, | ||
| :blob => {:enabled => true}, |
There was a problem hiding this comment.
Avoid comma after the last parameter of a method call.
|
Added thanks @mmoll but CI is failing with: Cliver::Dependency::NotFound: Could not find an executable ["/var/lib/workspace/workspace/test_develop_pr_core/database/postgresql/ruby/2.4/slave/fast/node_modules/.bin/phantomjs"] on your path. No idea what this is. |
|
[test foreman] |
lzap
force-pushed
the
structured-logging-22493
branch
from
4cab476
to
5e9d322
Compare
config/logging.yaml
Outdated
| @@ -49,15 +59,36 @@ | |||
| # [%X{session}] session ID from cookies, else the request-ID | |||
| # [%X{session_safe}] random ID assigned per session, not available early on during | |||
There was a problem hiding this comment.
You seem to have removed the session_safe ID?
There was a problem hiding this comment.
Good catch, it made no sense to have two. Will remove.
config/application.rb
Outdated
| @@ -213,7 +211,9 @@ class Application < Rails::Application | |||
| :templates => {:enabled => true}, | |||
| :notifications => {:enabled => true}, | |||
| :background => {:enabled => true}, | |||
| :dynflow => {:enabled => true} | |||
| :dynflow => {:enabled => true}, | |||
| :telemetry => {:enabled => true}, | |||
There was a problem hiding this comment.
Telemetry isn't enabled by default as far as I remember, shouldn't the log here be set to false?
|
|
||
| if env["rack.session"].id.present? | ||
| # use random token to prevent hijack. | ||
| session['logging_token'] ||= SecureRandom.uuid |
There was a problem hiding this comment.
Why not keeping logging_token as session_safe like before, so the log format in the yaml is backwards compatible?
There was a problem hiding this comment.
There is a reason - the stack is unable to remove keys from MDC context, it simply logs everything. We would log the session id twice. Since this is a big change in logging which will eventually lead to changes of config if users want to enable journald/syslog I thought it's good time for a change and cleanup.
| - mdc | ||
| - ndc | ||
| :pattern: "%d [%.1l|%.3c|%.5X{request}] %m\n" | ||
| :sys_pattern: "[%.1l|%.3c|%.5X{request}] %m\n" |
config/logging.yaml
Outdated
| :default: | ||
| :colorize: false | ||
| :console_inline: false | ||
| :log_trace: false | ||
| :level: info | ||
| :type: file | ||
| :pattern: "%d %.8X{request} [%c] [%.1l] %m\n" | ||
| #:layout: multiline_pattern |
| - mdc | ||
| - ndc | ||
| :pattern: "%d [%.1l|%.3c|%.5X{request}] %m\n" | ||
| :sys_pattern: "[%.1l|%.3c|%.5X{request}] %m\n" |
There was a problem hiding this comment.
Is this really used though? I have layout: multiline_pattern and type: journald - but my logs look like:
3月 13 10:18:35 centos7-devel.lobatolan.home foreman[17690]: 2018-03-13T10:18:35 [I|app|a08c8] Rendered /home/vagrant/foreman-plugin/redhat-access/app/views/dashboard/_insights_issues_widget.html.erb (1732.3ms)
3月 13 10:18:35 centos7-devel.lobatolan.home foreman[17690]: 2018-03-13T10:18:35 [I|app|a08c8] Completed 200 OK in 3329ms (Views: 1727.8ms | ActiveRecord: 1558.9ms)
So the date is printed twice.
| @@ -116,6 +117,7 @@ def current=(organization) | |||
| end | |||
|
|
|||
| Rails.logger.debug "Setting current organization thread-local variable to #{organization || 'none'}" | |||
| ::Logging.mdc['org_id'] = organization.try(:id) | |||
There was a problem hiding this comment.
Adding if SETTING[:organizations_enabled] would be nice, as otherwise we see 2 fields on the JSON output like "loc_id":null.
There was a problem hiding this comment.
I did not put this intentionally because that's an extra database call (or two), but I found a better way.
config/logging.yaml
Outdated
| :default: | ||
| :colorize: false | ||
| :console_inline: false | ||
| :log_trace: false | ||
| :level: info | ||
| :type: file | ||
| :pattern: "%d %.8X{request} [%c] [%.1l] %m\n" | ||
| #:layout: multiline_pattern | ||
| :layout: json |
There was a problem hiding this comment.
I'd rather default to multiline & journald, isn't it?
|
I don't see comment box on github for "Is this really used though?..." so my answer is: Yes, a bug. Fixed. |
|
Done, thanks! diff --git a/app/models/concerns/foreman/thread_session.rb b/app/models/concerns/foreman/thread_session.rb
index edf9a327c..37b17a058 100644
--- a/app/models/concerns/foreman/thread_session.rb
+++ b/app/models/concerns/foreman/thread_session.rb
@@ -117,7 +117,8 @@ module Foreman
end
Rails.logger.debug "Setting current organization thread-local variable to #{organization || 'none'}"
- ::Logging.mdc['org_id'] = organization.try(:id)
+ org_id = organization.try(:id)
+ ::Logging.mdc['org_id'] = org_id if org_id
Thread.current[:organization] = organization
end
@@ -153,7 +154,8 @@ module Foreman
end
Rails.logger.debug "Setting current location thread-local variable to #{location || 'none'}"
- ::Logging.mdc['loc_id'] = location.try(:id)
+ loc_id = location.try(:id)
+ ::Logging.mdc['loc_id'] = loc_id if loc_id
Thread.current[:location] = location
end
diff --git a/config/application.rb b/config/application.rb
index 6d3c10d98..b4e31aef7 100644
--- a/config/application.rb
+++ b/config/application.rb
@@ -212,7 +212,7 @@ module Foreman
:notifications => {:enabled => true},
:background => {:enabled => true},
:dynflow => {:enabled => true},
- :telemetry => {:enabled => true},
+ :telemetry => {:enabled => false},
:blob => {:enabled => true}
))
diff --git a/config/logging.yaml b/config/logging.yaml
index 76b72fc84..fb699c71c 100644
--- a/config/logging.yaml
+++ b/config/logging.yaml
@@ -56,9 +56,8 @@
# [%L] line number where the logging request was issued
# [%M] method name where the logging request was issued
# [%X{request}] request-ID set in HTTP headers, or random UUID
-# [%X{session}] session ID from cookies, else the request-ID
-# [%X{session_safe}] random ID assigned per session, not available early on during
-# request logging, but prevents disclosure of the session ID
+# [%X{session}] random ID assigned per session, not available early on during
+# request logging, but prevents disclosure of the session ID
# [%X{string}] variable set via MDC (::Logging.mdc['string'])
# [%x{string}] variable set via NDC (::Logging.ndc['string'])
#
@@ -77,8 +76,7 @@
:log_trace: false
:level: info
:type: file
- #:layout: multiline_pattern
- :layout: json
+ :layout: multiline_pattern
:json_items:
- logger
- timestamp
diff --git a/lib/foreman/logging.rb b/lib/foreman/logging.rb
index a34bc91f6..1fdc7a326 100644
--- a/lib/foreman/logging.rb
+++ b/lib/foreman/logging.rb
@@ -193,6 +193,7 @@ module Foreman
def build_layout(enable_colors = true)
pattern, colorize = @config[:pattern], @config[:colorize]
+ pattern = @config[:sys_pattern] if @config[:type] =~ /^(journald?|syslog)$/i
colorize = nil unless enable_colors
case @config[:layout]
when 'json |
lzap
force-pushed
the
structured-logging-22493
branch
from
5e9d322
to
3a952f6
Compare
|
Thanks for merging, setting the release to 1.18 in redmine. |
Foreman uses logging gem for all logging, this has new plugin which enables structured logging into system journal (https://github.com/lzap/logging-journald). This feature will add configuration options to use the logging plugin:
:layout: Formatter, one of 'pattern', 'multiline_pattern' or 'json'. The multiline pattern prepends
'|' characters on all additional lines for better readability.
:items: Items to include in JSON. Only applies when 'json' layout is set. Possible items are
logger timestamp level message file line method hostname pid millis thread_id thread
mdc ndc. Global fields like session or organization are available under 'mdc' item,
local fields like exception class or backtrace are under 'ndc' item.
:sys_pattern: Logger line pattern for system log/journal. See :pattern for more info.
:facility: System log/journal facility to use. One of LOG_AUTH, LOG_AUTHPRIV, LOG_CONSOLE,
LOG_CRON, LOG_DAEMON, LOG_FTP, LOG_KERN, LOG_LPR, LOG_MAIL, LOG_NEWS, LOG_NTP,
LOG_SECURITY, LOG_SYSLOG, LOG_USER, LOG_UUCP, LOG_LOCAL0 to LOG_LOCAL7.
Foreman already uses MDC (mapped diagnostic context) for session/request id. This patch extends this and adds more fields to the MDC, complete list:
There is also NDC (nested diagnostic context) array in logging gem which can be used to send additional fields per individual log records. It is currently used when logging an exception - it adds exception class, message and backtrace into separate fields. To use NDC in code or plugin, use helper:
As part of this, logging-journald is becoming a soft dependency of foreman-core via bundler group. Nothing is changing for development logging configuration, but for production we will consider making changes in the installer to send logs into journald and splitting them via rsyslog back into production.log for easier transition for existing users and tools. To be able to do that, all logs will be sent to facility local6 (configurable option). It was not yet decided if we do this as the default or opt-in in the installer.
When JSON output is set, log messages looks something like:
When exception was logged via
Foreman::Logging.exceptionthen there are additional fields inndchash:Note in this case backtrace is sent both in additional fields and into the message itself for best UX. Most users will stick with default configuration which is - journald ignores additional flags and sends the data to rsyslog.
To see the flags in journald use
journalctl -o verboseto see all flags:Foreman logger name is available as FOREMAN_LOGGER, level is PRIORITY and all other fields are available as well. Note that field names are in uppercase, this is converted by system journal daemon automatically.
Further reading: