Show decoded urls in log

[dima74]

Hello! Werkzeug logs urls as encoded strings. For example if I have route with Cyrillic characters: @app.route('/привет') in Flask (https://gist.github.com/dima74/5e025da90ee90a78b65bc3e3c69623ea), then Werkzeug will log GET /%D0%BF%D1%80%D0%B8%D0%B2%D0%B5%D1%82 HTTP/1.1. It would be very nice if there is option for Werkzeug to log decoded urls. (Thanks for Werkzeug!)

[untitaker]

I suppose this really is a problem for people with non-latin languages. However, we need to make sure we're not unescaping any other characters, such as terminal control sequences, otherwise people can probably send weird requests, the server admin looks at the log and then their terminal does funny things. This is where it gets much harder to implement.

…

On Fri, May 05, 2017 at 10:46:10AM -0700, Dmitry Murzin wrote: Hello! Werkzeug logs urls as encoded strings. For example if I have route with cyrillic characters: @app.route('/привет') in Flask (https://gist.github.com/dima74/5e025da90ee90a78b65bc3e3c69623ea), than Werkzeug will log "GET /%D0%BF%D1%80%D0%B8%D0%B2%D0%B5%D1%82 HTTP/1.1". It would be very nice if there is option to get that Werkzeug logs urls decoded. (Thanks for Werkzeug!) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #1115

[dima74]

Yes, I agree with you, thank you

[dima74]

What if only letters will be decode, that is characters c for which c.isalpha() == True. Something like this:

import urllib.parse
import re


def decode_letters_in_url(url_encoded):
    result = ''
    url_encoded_parts = re.split('((?:%..)+)', url_encoded)
    for part in url_encoded_parts:
        if part.startswith('%'):
            part_decoded = urllib.parse.unquote(part)
            for letter in part_decoded:
                result += letter if letter.isalpha() else urllib.parse.quote(letter)
        else:
            result += part
    return result

>>> decode_letters_in_url('/query?args=1,2')
'/query?args=1,2'
>>> decode_letters_in_url('/russian/%D0%BF%D1%80%D0%B8%D0%B2%D0%B5%D1%82')
'/russian/привет'
>>> decode_letters_in_url('/args?%D0%B0%D1%80%D0%B3%D1%83%D0%BC%D0%B5%D0%BD%D1%82%D1%8B=1,2,3')
'/args?аргументы=1,2,3'
>>> decode_letters_in_url('/query?spaces=_%20_%20_')
'/query?spaces=_%20_%20_'
>>> decode_letters_in_url('/%D0%BF%D1%80%D0%B8%D0%B2%D0%B5%D1%82?space=%20russian=%D1%80%D1%83%D1%81%D1%81%D0%BA%D0%B8%D0%B9')
'/привет?space=%20russian=русский'

[untitaker]

Do you need this for production or for development? In development we can make this much simpler since we don't need to worry about the developer owning himself.

[dima74]

For development. As far as I know Werkzeug is for development, not for production, doesn't it? In production I use Gunicorn, and I want that Gunicorn logs decoded url too, but it is another story.

[untitaker]

The Werkzeug server is used for development, but there's more stuff in Werkzeug than the server.

…

On Sat, May 06, 2017 at 09:08:36AM -0700, Dmitry Murzin wrote: For development. As far as I know Werkzeug is for development, not for production, doesn't it? In production I use Gunicorn, and I want that Gunicorn logs decoded url too, but it is another story. -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: #1115 (comment)

[dima74]

OK. Now I use only Werkzeug server for development

[untitaker]

Cool, so I think code like yours would be OK for the Werkzeug dev server, but I would like somebody else's feedback on this. Even though this is just the dev server, I am worried about performance of this.

…

On Sat, May 06, 2017 at 09:23:25AM -0700, Dmitry Murzin wrote: OK. Now I use only Werkzeug server for development -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: #1115 (comment)

[davidism]

uri_to_iri has been fixed to leave unsafe and invalid characters quoted, so this should probably be possible without a separate function.