Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent logging initialization #1969

Closed
abitrolly opened this issue Nov 16, 2020 · 3 comments
Closed

Inconsistent logging initialization #1969

abitrolly opened this issue Nov 16, 2020 · 3 comments

Comments

@abitrolly
Copy link

werkzeug sometimes initializes logging before app import, resulting in double logging entries.

This is app1.py that uses logging_tree to print logging configuration.

from logging_tree import printout as pt
pt()

import logging
logging.warn('------------[app restart]---')

pt()

from flask import Flask

app = Flask(__name__)

@app.route("/")
def heartbeat():
    return "tick"

import werkzeug
print(werkzeug.__version__)

The application is run with flask run in development mode.

FLASK_ENV=development FLASK_APP=bin/app1.py flask run -p 2999

Then, using combination of touch app1.py and curl localhost:2999 it should be easy to catch condition when after `logging`` tree is not empty, and log lines are duplicated. Here is the output. What happens there.

  1. werkzeug reloaded app1.py. app1.py found no logging configuration, implicitly installed own root handler through logging.warn. When request from curl arrived, it was logged with this root handler as INFO:werkzeug:127.0.0.1.. line. No other lines with 200 status are printed so far.
  2. touch event made werkzeug reload app1.py again. This time the logging configuration appears non-empty, app1.py again adds root handler, and when request arrives, it is printed twice.
INFO:werkzeug: * Detected change in '/root/cheat.sh/bin/app1.py', reloading
 * Restarting with stat
<--""
   Level WARNING
WARNING:root:------------[app restart]---
<--""
   Level WARNING
   Handler Stream <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>
     Formatter fmt='%(levelname)s:%(name)s:%(message)s' datefmt=None
1.0.1
WARNING:werkzeug: * Debugger is active!
INFO:werkzeug: * Debugger PIN: 770-386-719
INFO:werkzeug:127.0.0.1 - - [16/Nov/2020 16:33:30] "GET / HTTP/1.1" 200 -
INFO:werkzeug: * Detected change in '/root/cheat.sh/bin/app1.py', reloading
 * Restarting with stat
 * Debugger is active!
<--""
   Level WARNING
   |
   o<--"werkzeug"
       Level INFO
       Handler Stream <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>
WARNING:root:------------[app restart]---
<--""
   Level WARNING
   Handler Stream <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>
     Formatter fmt='%(levelname)s:%(name)s:%(message)s' datefmt=None
   |
   o<--"werkzeug"
       Level INFO
       Handler Stream <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>
1.0.1
 * Debugger PIN: 770-386-719
INFO:werkzeug: * Debugger PIN: 770-386-719
127.0.0.1 - - [16/Nov/2020 16:36:06] "GET / HTTP/1.1" 200 -
INFO:werkzeug:127.0.0.1 - - [16/Nov/2020 16:36:06] "GET / HTTP/1.1" 200 -

I expected werkzeug to initialize the logging module after app is imported to prevent double entries in stderr logs.

Environment:

Python 3.9.0
Flask 1.1.2
Werkzeug 1.0.1
@abitrolly abitrolly changed the title Inconsistent logging configuration Inconsistent logging initialization Nov 16, 2020
abitrolly added a commit to chubin/cheat.sh that referenced this issue Nov 17, 2020
Sometimes there is nothing, sometimes lines are duplicated.

Duplicated lines problem appears, because Python root logger
handler and Flask's `werkzeug` logger handler both write the
same message. `cheat.sh` is hit, because it sets this root
logging handler.

Normally `werkzeug` doesn't setup its own handler if it sees
that there are some handlers that process its messages. This
in case of `cheat.sh` resulted in no stderr logging at all.
No Exceptions either. Because `werkzeug` catches the
exceptions and logs them.

Hovewer, sometimes `werkzeug` starts too early, befora app
is imported (pallets/werkzeug#1969)
and sets its logger anyway, before the app itself, resulting
in duplicating lines. In that case app root handler needs to
skip lines from `werkzeug`.

Kudos to @brandon-rhodes for `logging_tree` awesomeness
@abitrolly
Copy link
Author

If werkzeug threading doesn't allow to wait until app is imported, one solution is to explicitly remove the handler after the user app is initialized and before it is served. Explicit means that removal will be accompanied with the notification message.

 * User logging detected, removing `werkzeug` handler

@davidism
Copy link
Member

I can't reproduce this with the latest code. Seems likely to have been addressed by #1500.

@abitrolly
Copy link
Author

@davidism did you manage to reproduce it with the versions specified? #1500 can not fix it, because its commits 7cf9e37 were merged in 0.15.2 and I used 1.0.1

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 28, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants