Skip to content
This repository has been archived by the owner on Feb 22, 2023. It is now read-only.

Invalid payload when trying to scrape the /shots?data=json page #25

Closed
pdehaan opened this issue Mar 21, 2017 · 0 comments
Closed

Invalid payload when trying to scrape the /shots?data=json page #25

pdehaan opened this issue Mar 21, 2017 · 0 comments

Comments

@pdehaan
Copy link
Collaborator

pdehaan commented Mar 21, 2017

Ref: #23

In my @global_setup(), we're uploading 2 images (so we can test the load on list_shots retrieval). Then, in the async list_shots(), we do:

@scenario(100)
async def list_shots(session):
    res = await utils.list_shots(session)

    print("\n\n\n!!!!!!!!\n\n")
    payload = await res.json()
    print(payload)
    print("\n\n ¯\_(ツ)_/¯ ")

So it looks like the await res.json() is returning None for the payload. Not sure why that is, but the one suspicious but below about the response says **** Binary content ****.
When @rpappalax and I were testing this API using cURL, we were getting some JSON back -- so not sure if this is me not passing the right headers through molotov, or if molotov is adding some unwanted headers which is giving me back gzipped binary back instead of the JSON payload.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

GET https://pageshot.stage.mozaws.net/shots?data=json
Content-Type: application/json
Accept: application/json, */*
Accept-Encoding: gzip, deflate
Host: pageshot.stage.mozaws.net
User-Agent: Python/3.6 aiohttp/1.3.3
Cookie: abtests=eyJoaWdobGlnaHRCdXR0b25Pbkluc3RhbGwiOnsidmFsdWUiOiJjb250cm9sIiwiZ2FGaWVsZCI6ImNkMyIsInZlcnNpb24iOjF9LCJzdHlsZU15U2hvdHNCdXR0b24iOnsidmFsdWUiOiJjb250cm9sIiwiZ2FGaWVsZCI6ImNkNCIsInZlcnNpb24iOjF9LCJhdXRvT3BlblNoYXJlUGFuZWwiOnsidmFsdWUiOiJjb250cm9sIiwiZ2FGaWVsZCI6ImNkNSIsInZlcnNpb24iOjEsInNob3RGaWVsZCI6ImNkNiJ9fQ==; abtests.sig=RaLSzsm7ATg732SFBD0t8KvwrBU; user=b0ad183a0e7b11e7b1e828cfe91ef31b; user.sig=RF9Uamjog2jkTIL_CW6A4Ep_SCI
Content-Length: 0
SUCCESSES: 0 | FAILURES: 0


!!!!!!!!


None


 ¯\_(ツ)_/¯

=============================================

HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Security-Policy: default-src 'self'; img-src 'self' www.google-analytics.com pageshotcontent.stage.mozaws.net data:; script-src 'self' www.google-analytics.com 'nonce-cff58350-fea5-4dd6-bd20-0279e3e03744'; style-src 'self' 'unsafe-inline' https://code.cdn.mozilla.net; connect-src 'self' www.google-analytics.com sentry.prod.mozaws.net; font-src https://code.cdn.mozilla.net;
Content-Type: application/json; charset=utf-8
Date: Tue, 21 Mar 2017 21:16:48 GMT
Etag: W/"7a4-G6eZewYkHll3ub/IdCAOvg"
Strict-Transport-Security: max-age=86400000
Vary: Accept-Encoding
X-Frame-Options: DENY
X-Powered-By: Express
Content-Length: 792
Connection: keep-alive

**** Binary content ****

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

/cc @tarekziade

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant