Add Support for the current Syslog Protocol (RFC 5424)#1223
Draft
VadimZhestikov wants to merge 8 commits intonginx:masterfrom
Draft
Add Support for the current Syslog Protocol (RFC 5424)#1223VadimZhestikov wants to merge 8 commits intonginx:masterfrom
VadimZhestikov wants to merge 8 commits intonginx:masterfrom
Conversation
Add "rfc=rfc3164|rfc5424" to the syslog directive. The default remains rfc3164 for backward compatibility. Tag validation is moved from ngx_syslog_parse_args() to ngx_syslog_process_conf() so that the character set and length constraints can be conditioned on the chosen protocol version. RFC 5424 (APP-NAME) allows up to 48 printable US-ASCII characters (0x21-0x7E), while RFC 3164 (TAG) restricts to alphanumeric characters and underscore with a 32-character limit.
Test that the "rfc=" parameter of the syslog directive accepts "rfc3164" and "rfc5424", and rejects unknown values. Test the protocol-version-dependent tag validation: RFC 5424 (APP-NAME) allows hyphenated/dotted tags up to 48 printable US-ASCII characters; RFC 3164 (TAG) restricts to alphanumeric characters and underscore with a 32-character limit.
When the "rfc=rfc5424" syslog parameter is used, format syslog messages according to RFC 5424 instead of RFC 3164. The RFC 5424 HEADER contains VERSION (always "1"), an ISO 8601 TIMESTAMP with millisecond precision and UTC offset (e.g. "2003-10-11T22:14:15.003+05:30"), HOSTNAME, APP-NAME, PROCID (nginx process PID), and nil values for MSGID and STRUCTURED-DATA. Add cached_syslog_rfc5424_time to ngx_times.c populated alongside the existing cached_syslog_time in both ngx_time_update() and ngx_time_sigsafe_update(). The signal-safe path has no millisecond counter available and emits ".000" for the sub-second field. Expand NGX_SYSLOG_MAX_STR to accommodate the larger RFC 5424 header.
Test each field of the RFC 5424 HEADER for an access_log syslog
destination: PRI, VERSION ("1"), ISO 8601 timestamp with milliseconds
and UTC offset, HOSTNAME, APP-NAME (printable US-ASCII), PROCID
(decimal integer PID), nil MSGID, nil STRUCTURED-DATA, and non-empty MSG.
Additionally test:
- nohostname: HOSTNAME is the nil value "-"
- Custom hyphenated tag (valid in APP-NAME per RFC 5424)
- facility=user encoded as facility 1 in the PRI field
- Global error_log using rfc5424 produces messages with VERSION "1"
- Default (rfc3164) format is unchanged: BSD timestamp, no VERSION field
Remove the per-second cached_syslog_rfc5424_time slot array that was
added in the previous commit. The cached approach would populate the
millisecond field with the value at second rollover (usually near 0),
defeating the sub-second precision that RFC 5424 was requested for.
Instead, in ngx_syslog_add_header(), build the RFC 5424 TIMESTAMP
field inline:
- The date, time, and UTC offset are read from
ngx_cached_http_log_iso8601 ("YYYY-MM-DDTHH:MM:SS±HH:MM"),
splitting the 25-byte string at position 19.
- The millisecond sub-field is taken from ngx_timeofday()->msec,
which is updated on every event-loop tick by ngx_time_update().
This produces timestamps accurate to the resolution of the event
loop (typically 1 ms) at negligible per-message cost.
Update syslog_rfc5424.t: add a test that sends several messages with 20 ms gaps and verifies that the millisecond field in the RFC 5424 timestamp is not permanently stuck at "000", confirming that the implementation reads the live ngx_timeofday()->msec value rather than a once-per-second cached value. Add syslog_rfc5424_stream.t: exercise the RFC 5424 syslog format via the stream access_log directive using the same per-field checks as the HTTP test (PRI, VERSION, ISO 8601 timestamp, HOSTNAME, APP-NAME, PROCID, nil MSGID, nil STRUCTURED-DATA, MSG). Also test that the "nohostname" parameter produces the RFC 5424 nil HOSTNAME "-".
Add "msgid=<id>" to the syslog directive. When rfc=rfc5424 is in use, the configured value is placed in the MSGID field of the RFC 5424 HEADER (§6.2.7) instead of the nil value "-". MSGID is at most 32 printable US-ASCII characters (0x21-0x7E; space and controls are excluded). Specifying "msgid=" without "rfc=rfc5424" is a configuration error. The default MSGID is "-". Expand NGX_SYSLOG_MAX_STR by 31 bytes to accommodate a maximum-length MSGID field.
Config tests (syslog_rfc5424_config.t): - msgid= accepted with rfc=rfc5424 - msgid= rejected when rfc=rfc5424 is not set - msgid= with a non-ASCII byte (>0x7E) rejected - msgid= exceeding 32 characters rejected - 32-character msgid= accepted Format tests (syslog_rfc5424.t): - Explicit msgid=MYAPP appears in the MSGID field (position 6 of the RFC 5424 HEADER) of the emitted syslog message. - Without msgid=, the MSGID field defaults to the nil value "-".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add Support for the current Syslog Protocol (RFC 5424)
DRAFT