Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Full Debug / Dump of a Request #331

Closed
VeNoMouS opened this issue Sep 9, 2019 · 7 comments
Closed

Full Debug / Dump of a Request #331

VeNoMouS opened this issue Sep 9, 2019 · 7 comments

Comments

@VeNoMouS
Copy link

VeNoMouS commented Sep 9, 2019 •
edited

Is there any functionality that replicates dump from requests_toolbelt.utils , that dumps out full headers and content etc from the response so you can debug your requests... as I cannot find any reference to debugging in the documentation...

IE in requests

import requests
from requests_toolbelt.utils import dump

def debugRequest(req):
    try:
        print(dump.dump_all(req).decode('utf-8'))
    except:  # noqa
        pass

ret = requests.get('http://example.com')
debugRequest(ret)

Would result in..

< GET / HTTP/1.1
< Host: example.com
< User-Agent: python-requests/2.22.0
< Accept-Encoding: gzip, deflate
< Accept: */*
< Connection: keep-alive
<

> HTTP/1.1 200 OK
> Content-Encoding: gzip
> Accept-Ranges: bytes
> Cache-Control: max-age=604800
> Content-Type: text/html; charset=UTF-8
> Date: Mon, 09 Sep 2019 21:02:17 GMT
> Etag: "1541025663"
> Expires: Mon, 16 Sep 2019 21:02:17 GMT
> Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
> Server: ECS (oxr/830D)
> Vary: Accept-Encoding
> X-Cache: HIT
> Content-Length: 606
>
<!doctype html>
<html>
<head>
    <title>Example Domain</title>

    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <style type="text/css">
    body {
        background-color: #f0f0f2;
        margin: 0;
        padding: 0;
        font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;

    }
    div {
        width: 600px;
        margin: 5em auto;
        padding: 50px;
        background-color: #fff;
        border-radius: 1em;
    }
    a:link, a:visited {
        color: #38488f;
        text-decoration: none;
    }
    @media (max-width: 700px) {
        body {
            background-color: #fff;
        }
        div {
            width: auto;
            margin: 0 auto;
            border-radius: 0;
            padding: 1em;
        }
    }
    </style>
</head>

<body>
<div>
    <h1>Example Domain</h1>
    <p>This domain is established to be used for illustrative examples in documents. You may use this
    domain in examples without prior coordination or asking for permission.</p>
    <p><a href="http://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>

Cheers
@sethmlarson
Copy link
Contributor

sethmlarson commented Sep 10, 2019 •
edited

This wouldn't be too hard to implement. IMO we should look to requests-toolbelt for inspiration on how some functionality like multipart form encoding to work like that by default. Toolbelt is basically Requests+

Not sure about this exact feature but some of them might be good for us.

@StephenBrown2
Copy link
Contributor

Agreed. I too wanted that functionality and so copy-pasted toolbelt's version and tweaked it for httpx, making it much less configurable in the process:

# Inspired by https://github.com/requests/toolbelt/blob/0.9.1/requests_toolbelt/utils/dump.py
def _log_response(response: httpx.models.BaseResponse) -> str:
    req_prefix = "< "
    res_prefix = "> "
    request = response.request
    output = []

    output.append(f"{req_prefix}{request.method} {request.url} {response.protocol}")

    for name, value in request.headers.items():
        output.append(f"{req_prefix}{name}: {value}")

    output.append(req_prefix)

    if isinstance(request.content, (str, bytes)):
        output.append(f"{req_prefix}{request.content}")
    else:
        output.append("<< Request body is not a string-like type >>")

    output.append("")

    output.append(
        f"{res_prefix}{response.protocol} {response.status_code} {response.reason_phrase}"
    )

    for name, value in response.headers.items():
        output.append(f"{res_prefix}{name}: {value}")

    output.append(res_prefix)

    output.append(f"{res_prefix}{response.text}")

    return "\n".join(output)


def dump_all_requests(response: httpx.models.BaseResponse) -> str:
    """Dump all requests and responses including redirects.
    """
    data = []

    history = list(response.history[:])
    history.append(response)

    for response in history:
        data.append(_log_response(response))

    return "\n".join(data)

This could also be done with a LoggingDispatcher, and I would be my preferred method I think, but I haven't actually implemented that just yet. Take this as inspiration, but not a definitive answer.

@sethmlarson
Copy link
Contributor

You could probably use the newly added middleware support :D

@florimondmanca
Copy link
Member

You could probably use the newly added middleware support :D

Do we allow wrapping other middleware from the outside already? Don’t think we do, and that might be a nice next step as well as documenting things. :)

@sethmlarson
Copy link
Contributor

Ah right, sorry! I'm just too excited about custom middleware. :)

@tomchristie
Copy link
Member

Yeah third party middleware will be the right tack for this.
It'd actually be a great proof-of-functionality for us to ensure that the middleware API is sufficiently complete.

@tomchristie
Copy link
Member

Closing this as out-of-scope to the core package.
We're tracking the middleware API in #345, which is also somewhat relevant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@VeNoMouS @tomchristie @StephenBrown2 @florimondmanca @sethmlarson