Schema Comparison between Structured Logging Libraries

The goal of this document is to summarize the differences between various structured logging libraries, in order to encourage a discussion so that a consensus can be reached for a common format.

Please use the GitHub issues for discussion, and submit GitHub Pull requests to add/fix the information in the tables.

Message and Payload

Behold the following logging command that you might find in an application. There is a "message" string, and a "payload" object:

logger.info("Hello World", {"animal": "cat", "numLegs": 4})
             ^^^^^^^^^^^   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
             Message       Payload object

Some libraries merge the payload object with the root object, so the event will look like this:

{
    "timestamp": "2018-06-18T23:16:45.000Z",
    "message": "Hello World",
    "animal": "cat",
    "numLegs": 4
}

While other libraries nest the payload object beneath some key, with a result like this:

{
    "timestamp": "2018-06-18T23:16:45.000Z",
    "message": "Hello World",
    "data": {
        "animal": "cat",
        "numLegs": 4
    }
}

Library	Message JSON Key	Payload Object JSON Key
bunyan (JavaScript)	msg	Merged with root
katip (Haskell)	msg	data
logrus (Go)	msg	Merged with root
logstash	message	Merged with root
ougai (Ruby)	msg	Merged with root
pygogo (Python)	message	Merged with root
roarr (JavaScript)	message	context
semantic logger (Ruby)	message	payload
serilog (C#)	@m / @mt	Merged with root
slog (Rust)	msg	Merged with root
structlog (Python)	msg	HELP NEEDED

Timestamp JSON Key and format

All libraries automatically add a timestamp to all events.

Library	Timestamp JSON Key	Timestamp format
bunyan (JavaScript)	time	ISO 8601
katip (Haskell)	at	ISO 8601
logrus (Go)	time	ISO 8601
logstash	timestamp / @timestamp	ISO 8601 / varies
ougai (Ruby)	time	ISO 8601
pygogo (Python)	time	ISO 8601
roarr (JavaScript)	time	Epoch milliseconds
semantic logger (Ruby)	timestamp	ISO 8601
serilog (C#)	Timestamp / @t	ISO 8601
slog (Rust)	ts	ISO 8601
structlog (Python)	timestamp	ISO 8601

Log Level

All libraries require that that all events are tagged with a level, chosen from a pre-defined list of available levels.

Log Level JSON Key

Library	Log Level JSON Key
bunyan (JavaScript)	level
katip (Haskell)	sev
logrus (Go)	level
logstash	HELP NEEDED
ougai (Ruby)	level
pygogo (Python)	level
roarr (JavaScript)	context.logLevel
semantic logger (Ruby)	level
serilog (C#)	Level / @l
slog (Rust)	level
structlog (Python)	level

Available Log Levels

Library	VERBOSE	TRACE	DEBUG	INFO	NOTICE	WARNING	ERROR	CRITICAL	ALERT	FATAL	PANIC	EMERGENCY
bunyan (JavaScript)	N/A	10	20	30	N/A	40	50	N/A	N/A	60	N/A	N/A
katip (Haskell)	N/A	N/A	Debug	Info	Notice	Warning	Error	Critical	Alert	N/A	N/A	Emergency
logrus (Go)	N/A	N/A	debug	info	N/A	warning	error	N/A	N/A	fatal	panic	N/A
logstash	HELP NEEDED	HELP NEEDED	HELP NEEDED	HELP NEEDED	HELP NEEDED	HELP NEEDED	HELP NEEDED	HELP NEEDED	HELP NEEDED	HELP NEEDED	HELP NEEDED	HELP NEEDED
ougai (Ruby)	N/A	10	20	30	N/A	40	50	N/A	N/A	60	N/A	N/A
pygogo (Python)	N/A	N/A	DEBUG	INFO	N/A	WARNING	ERROR	CRITICAL	N/A	N/A	N/A	N/A
roarr (JavaScript)	N/A	10	20	30	N/A	40	50	N/A	N/A	60	N/A	N/A
semantic logger (Ruby)	N/A	trace	debug	info	N/A	warn	error	N/A	N/A	fatal	N/A	N/A
serilog (C#)	Verbose	N/A	Debug	Information	N/A	Warning	Error	N/A	N/A	Fatal	N/A	N/A
slog (Rust)	N/A	TRACE	DEBUG	INFO	N/A	WARN	ERROR	CRITICAL	N/A	N/A	N/A	N/A
structlog (Python)	N/A	N/A	debug	info	N/A	warning	error	critical	N/A	N/A	N/A	N/A

System Context Data

Some libraries automatically add additional fields to all events (such as process ID and thread ID) and/or allow you to configure a few pre-determined global fields suchs the current environment and application.

Library	Environment	Application	Namespace	Package	Machine Hostname	Process ID	Thread ID
bunyan (JavaScript)	N/A	N/A	N/A	N/A	hostname	pid	N/A
katip (Haskell)	env	app (Array of strings)	ns (Array of strings)	N/A	host	pid	thread
logrus (Go)	N/A	N/A	N/A	N/A	N/A	N/A	N/A
logstash	HELP NEEDED	HELP NEEDED	HELP NEEDED	HELP NEEDED	HELP NEEDED	HELP NEEDED	HELP NEEDED
ougai (Ruby)	N/A	app	N/A	N/A	hostname	pid	N/A
pygogo (Python)	N/A	N/A	N/A	N/A	N/A	N/A	N/A
roarr (JavaScript)	N/A	context.application	context.namespace	context.package	context.hostname	N/A	N/A
semantic logger (Ruby)	environment	application	N/A	N/A	host	pid	thread
serilog (C#)	N/A	N/A	N/A	N/A	N/A	N/A	N/A
slog (Rust)	N/A	N/A	N/A	N/A	N/A	N/A	N/A
structlog (Python)	N/A	N/A	N/A	N/A	N/A	N/A	N/A

HTTP Request Context Data

Logging of HTTP requests is very common. Let's figure out a common format for HTTP method, host, url, headers, remote IP, remote IP geo-lookup, response status code, response size, etc...

!!! TODO !!!