Structural logging: JSON format

[Zarathustra2]

We have deployed meilisearch on kubernetes on GKE but the logs-explorer picks every log up as an error.

Are there any plans on adding an environment variable ENABLE_JSON_LOGGING which would log every log message as a json?

If not I would happily try to implement it.

[gmourier]

Hi @Zarathustra2,

It may be because the logging is displayed on stderr; Information should probably be on stdout while error, warning etc should be on stderr. @Kerollmops WDYT?

However, the idea of having a JSON structure for the log message is interesting.

There could be a unique identifier by web request to allow you to merge the query log with the response log later in your analytics dashboard.

[Kerollmops]

The fact that all kinds of logs are output to stderr looks like the way to go. The only content that should be displayed on the stdout should be the content requested by the program user. But maybe we could reconsider this.

About using structured logging: we could remove our dependency on the log library to something more structured like slog or tracing.rs. Unfortunately, a lot of libraries already use log internally, how will this new structured logging library interact with the already common log one?

We have deployed meilisearch on kubernetes on GKE but the logs-explorer picks every log up as an error.

I recommend you to look at the logging format that we use and find a way to match the term to determine the severity of it i.e. ERROR, WARN, DEBUG.

[gajus]

badly need this to support Meilisearch in our infrastructure.

We are ingesting all the logs in Google Cloud Logging backend. The unstructured logs are quite useless.

[gmourier]

Hey @gajus 👋

What do you mean by structured logs? Can you be more explicit?

Thank you!

[gajus]

JSON logs

[gmourier]

Is it useless because you need something missing from the current log lines, or is it because you need them to be in JSON format?

[gajus]

I need them to be in JSON format to be able to trigger alerts based on the contents of the logs. Otherwise, we have to parse text logs ourselves looking for keywords, which is far from ideal. That's what we eventually did, but it is not great experience.

[gmourier]

Thank you @gajus

[irevoire]

Hello everyone!
We just implemented this feature as an experimental feature and would love to get your feedback about it!
See the associated discussion and please posts your feedback in it: https://github.com/orgs/meilisearch/discussions/723
It should be available in the next RC (v1.7.0-rc.1)