Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request logging encoding stopped working in 2.0.x #1654

Open
quantoid opened this issue Oct 2, 2017 · 8 comments
Open

Request logging encoding stopped working in 2.0.x #1654

quantoid opened this issue Oct 2, 2017 · 8 comments

Comments

@quantoid
Copy link

@quantoid quantoid commented Oct 2, 2017

I'm using the --log-req-encoder option to encode request logging as JSON. This has been working fine for a while, but recently stopped having any effect (the logging is now plain text).

Options used:

fmt=$'{"timestamp": "${strftime:%FT%TZ}", "level": "DEBUG", "name": "uwsgi", "message": "${msg}"}\n'
uwsgi --need-app --module=myapp.main:app \
  --http=0.0.0.0:8080 --uid=leaf --master --workers=8 --enable-threads \
  --env="TZ=UTC" --log-req-encoder="json ${fmt}" --logformat="%(status) [%(msecs)ms] %(method) %(uri)"

What I see on stdout:

[log-req-encoder] registered json {"timestamp": "${strftime:%FT%TZ}", "level": "DEBUG", "name": "uwsgi", "message": "${msg}"}
:::
spawned uWSGI http 1 (pid: 76846)
{"timestamp": "2017-10-02T23:11:54.135Z", "message": "Descriptor requested", "request": "1470df66-a7c7-11e7-b483-186590cba8eb", "name": "myapp.main", "level": "INFO"}
200 [11ms] GET /descriptor.json

The JSON encoded line is the output from my Flask app, which uses a python log formatter to encode as JSON.

The next line logging the GET request/response used to be JSON encoded but is not any more.

@quantoid quantoid changed the title Request logging encoding stopped working in 2.0.15 Request logging encoding stopped working in 2.0.x Oct 3, 2017
@quantoid

This comment has been minimized.

Copy link
Author

@quantoid quantoid commented Oct 3, 2017

I'd also be interested to know how to use --log-encoder to encode stdout as JSON without also encoding the output from the Flask app (which is already JSON encoded).

@jwineinger

This comment has been minimized.

Copy link

@jwineinger jwineinger commented Nov 10, 2017

@quantoid did you have any luck with the second part (not encoding your flask app output)? I also found that I had to go back to 2.0.14 to get any of the json encoding to work

@f-f

This comment has been minimized.

Copy link

@f-f f-f commented Feb 20, 2018

+1 on this, I'm running into the same issues: I'm defining json logger throughout all of our stack, and I cannot make it work on uwsgi.

I would say there is two different issues here:

  1. It is not possible to define log-req-encoder without having to define also a log-encoder
  2. It is not possible to "passthrough" the app logs
@quantoid

This comment has been minimized.

Copy link
Author

@quantoid quantoid commented Feb 21, 2018

Finally solved this today: you have to include —logger-req=stdio and then the json encoder starts working.

@quantoid

This comment has been minimized.

Copy link
Author

@quantoid quantoid commented Feb 21, 2018

But still haven’t found a way to stop app logging from being encoded along with other stdout.

@quantoid

This comment has been minimized.

Copy link
Author

@quantoid quantoid commented Feb 28, 2018

What I think we need here is an option to redirect stdout and stderr from uwsgi itself (but not from the app) to a logger, with the appropriate logging level. Then you can keep control of where all your logging goes and how it gets formatted in the usual python way.

The output from the app (and hence from its loggers) should be left alone in this case, because it's probably already output from loggers. I guess I could configure the root logger to just output the message to stdout and rely on uwsgi formatting as logging, but then I lose the name and level attributes.

@jwineinger

This comment has been minimized.

Copy link

@jwineinger jwineinger commented Feb 28, 2018

@iborko

This comment has been minimized.

Copy link

@iborko iborko commented Feb 10, 2020

I've recently managed to configure uWSGI to output all logs in JSON format and pass-through the application logs (as they are already in JSON format). I also ended up writing a uWSGI logging plugin with custom logchunks. You can read more about it on my blog: https://www.velebit.ai/blog/tech-blog-uwsgi-json-logging/

There is also a link on my repo that contains the source code of my uWSGI plugin and all required instructions to build and use it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.