You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some of the software we run inside WDLs supports JSON logging. Also, we want to be able to configure miniWDL itself to emit its logs as JSON. Combining the two, if a log line on stderr is inferred to be a JSON object, and miniWDL's logger is configured to emit JSON, then the log line should be deserialized and nested into the miniWDL logger's JSON, instead of ending up being double-serialized.
The text was updated successfully, but these errors were encountered:
The miniwdl internal logs flow through the standard Python logging framework including the structured logging "recipe". We should be able to hook up the python-json-logging handler including "custom object serialization."
The task standard error streams through a file ($run_directory/stderr.txt) which miniwdl tails when in verbose logging mode. Thinking ahead to production use cases where miniwdl's GIL may present a scalability bottleneck, we might want to look at publishing that filename for some other process to tail and parse/forward.
@kislyuk#380 adds --log-json and a hook permitting a plugin to override how each line of the streaming stderr log is handled in verbose mode. I'd still like to explore if there's a reasonable solution that avoids miniwdl having to examine and parse the stderr stream contents, since that might cause a scalability bottleneck down the road.
Some of the software we run inside WDLs supports JSON logging. Also, we want to be able to configure miniWDL itself to emit its logs as JSON. Combining the two, if a log line on stderr is inferred to be a JSON object, and miniWDL's logger is configured to emit JSON, then the log line should be deserialized and nested into the miniWDL logger's JSON, instead of ending up being double-serialized.
The text was updated successfully, but these errors were encountered: