Receiving malformed response (bad response or corrupted JSON) #631
Comments
|
Hey @PaulNewton thanks for opening the issue! The error messageA malformed response should not be returned in normal usages, I will work on making that error message more informative however you should not be receiving that message. (Which is probably why that error message sucks in the first place) Deeper debuggingI would love it if you could download a free copy of Charles Proxy When you run it the first time it should start recording with a proxy running on port 8888. Then you can run This will record all the requests from themekit. You can inspect the body of each requests to see how they look. If you notice anything you can let me know. Otherwise if you rightclick on the parent you will be able to export the report and send it to me tim.anema@shopify.com
Some last moment questions
Thanks for you help! |
|
I ran ahead and assumed normal troubleshooting problems on my end including bad connection, so changed network, restarted my machine, deleted the current folders&files and started again.
there was an auth gateway so possible proxy or filter. For now I changed networks and themekit get is working without errors so possible correlation.
Running commands in cmd.exe
Haven't tried powershell yet on prior network
Not my first install of themekit
Until this error Themekit has worked like a dream, thank you so much.
I installed themekit straight from the github download A note on finding if anythings missing doing a file compare of the admin exported theme.zip and themekit download. The export zip will insert rendered versions of css.liquid and javascript.liquid increase the file count in the zip. I've gotten so used to themekit I forgot that happened. |
I have seen this cause issues in the past so I can probably give you my 80~90% confidence that it was this that was interfering with themekit. I would love to make themekit more resilient to this issue, so if you run into the problem again I would love it if you could capture some of the network traffic and hopefully I can prevent these sort of issues in the future. |
|
I'm encountering these errors, for a development store. @tanema I've emailed you a Charles Proxy export file.
|
|
I think my particular case these "received a malformed response" themekit errors might be because I'm trying to download a trial theme. When I used themekit to download the default debut theme, it's working fine. |
|
@monting I haven't had a chance to look at your proxy output yet but thankyou for that extra info. I will test with a trial theme as well. That would indicate that we are experiencing 2 problems here though |
|
Running into this issue as well. Will try diff'ing theme downloaded via zip archive against theme download command, as well as inspecting Charles Proxy logs. |
|
@cameroncowden and @monting if you are on OSX you can use mitmproxy. Install it with Mitmproxy is more minimal, easy to use, and easier for me to parse. |
|
Hey Tim, |
|
hrm, that might indicate that it has something to do with ssl/https since ssl is disabled while proxying so that all traffic can be inspected. It would also explain why the logs that everyone sent me were completely unintelligible because they were still encoded for some reason. It might have something to do with this other issue: #544 A fix for this is on master and you can try it out with |
|
Roger that. I'm currently seeing this error at a coffee shop and the wifi has no password. I'm not sure if I've seen this on my home wifi (wpa2 I think). Not sure if that's helpful. I also have the theme I'm downloading in a git repo, and even when the errors is reported there doesn't appear to be anything incorrect or jumbled in the files that are updated or added. (I wouldn't know if it "skipped" one of the new files though) |
|
I have released 1.0.1 with the fix for #544 in it. Please give it a try and see if it solves your issues. |
tanema
changed the title
|
I am getting the error on my site page: This page is temporarily unavailable because a device from your location is sending large amounts of web requests. Visitors from other locations can still view the page. Theme kit gives me the error: This page is temporarily unavailable because a device from your location is sending large amounts of web requests. Visitors from other locations can still view the page. The errors (which happen simultaneously so I assume its the same issue) are intermittent. Everything works fine; then it doesn't. If I CTRL-C and wait a few minutes, it starts working again. But then soon I get the same error. I am using the Chrome extension LivePage to refresh my page since PrePros doesn't work on my system for some reason. I've set my config.yml file to include: bucket_size: 40, refill_rate: 2, concurrency: 2, timeout: 30s. I'm posting here because the error is the same but I don't know if the cause is the same. I tried running the Charles Proxy command as listed above but I get a response: Proxy URL detected from Configuration[http://127.0.0.1:8888] SSL Certificate Validation will be disabled! Get https://tales-of-murder-press.myshopify.com/meta.json: proxyconnect tcp: dial tcp 127.0.0.1:8888: connect: connection refused. I don't believe I'm behind a proxy (I didn't set up anything). I'm running Manjaro Linux (up to date). Chrome. |
|
@flaturtha you definitely have a proxy configured if that message is happening. Check your config file or your THEMEKIT_PROXY environment variable. Also are you working in an office with other theme developers all sharing the same IP? If so it is just likely that your whole office is being throttled because of a lot of traffic. |
|
I’m working solo in a home office so no other users. I’ll check my config and environment variables. I am using the Chrome extension LivePage so maybe that’s the root of the proxy issue. The problem goes away when it’s turned off but of course I lose browser refresh, etc. Thanks. I’m learning but still very much a noob so I’m not always sure where an issue comes from. |
|
I'm having this issue all of a sudden. I've been working for a week with my theme. All of a sudden tonight while using
I'm not going to lie I'm kind of freaking out. I even rolled back to a few version of my theme in git and no dice. Any ideas? I adjusted the timeout to 10min. Still the same errors. I just tried it on different ISPs. I don't have any proxy installed. Additionally the files it complains about if I upload them with
Here you can take a look. My theme is totally borked. It's using the wrong style, colors, and has a whole bunch of modules I normally don't have. When I go into the shopify admin it claims some file I've never edited is broken. (I checked git history)
Thanks for any help! |
|
@ckizer I would recommend restarting your terminal and/or your computer and try deploying again. The deploy command will upload a copy of each file to make sure the theme is fully sync so that makes sense if a single file in the theme is corrupted. If restarting does not work I would recommend trying different wifi, like your home wifi. It may also be that Shopify was having issues at the time you were deploying, and it may be clear now. |
|
Same error this morning. 200|227 [====================================================================>---------] 88 % |
|
I've restarted. The issues started around 12:30am at Shopify. They aren't fixed still. It's def on Shopify's servers as the same thing is happening on a new computer with a brand new checked out copy. I consider myself a decent developer, and have no idea how to proceed. I even tried looking for the assets.json file to delete. Any of those protocol error files above if I manually deploy them it works fine |
|
It seems like your issue is not the same issue as this. Yours is actually a bit lower in the stack before json is unmarshalled so I have opened a new issue to track this #659 Please follow there |
|
I have also noticed that this might happen when shopify returns html error pages if something is wrong. I will try to add some checks to see if I can at least handle that instance. |
|
So, is it so then that Trial Themes shoul work fine? Im having exactly this same problem for a theme trial. If I try in the same shop theme kit for an free one which is Live now it works, but for the Theme Trial it does not. |
|
Theme trials do not let you access the themes code neither through the
admin or the api AFAIK , though this seems like this should be a 403 access
forbidden instead of this error.
…On Wed, Apr 8, 2020, 4:33 AM oscarsan ***@***.***> wrote:
So, is it so then that Trial Themes shoul work fine? Im having exactly
this same problem for a theme trial. If I try in the same shop theme kit
for an free one which is Live now it works, but for the Theme Trial it does
not.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#631 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ALIKKMMQRU2UU54LOG6U42DRLQZDXANCNFSM4HIG5EHQ>
.
|
|
I have solved this issue, just use a proxy server and it is works for me. This is probably happen because my IP is unstable that changed automatically in several short time |
|
I tried using a proxy server as well but to no avail. Currently the only way I've been able to get this manageable is by re-publishing until the only errors I have are images that have no bearing on the theme functionality. Currently my internet is pretty slow due to where I'm living at the moment but I cannot change that right now (3mb down, 1mb up). I've also set the timeout to 10 minutes but again same errors. Also this has been with Shopify Plus as well basic Shopify plans. Running on MacOS 10.15.4 using VS Code (zsh shell) using ThemeKit 1.0.3). Anyone found insights into this? Thanks! |
What happens if explicitly download everything BUT images? Or making a config that ignores all image files |
|
I haven't had issues with download but with publishing. Using the ignore_files setup you had I only had issues with MP4/WebM files (I then re-ran it with those file types being ignored). On second run, I had a 71kb js file cause this issues on publishing:
So it definitely seemed to reduce the amount of files having issues (in this case often it's 5-7). But even when I ran it through a proxy...it seems the PUT commands get queued and at some point they must time out or hit some limitation from the server? |
Probably unrelated and merely a coincidence but that js file indicates it's the same theme that first gave me these types of errors. Which at the time seemed like a combo of the wifi portal used and maybe the large images assets.
Unfortunately I think repeatability is the problem here, I've not seen it again since themekit was updated with #544 thanks to tanema, however I also haven't touched that theme again but I made a laundry list to go through if it happened again since it kills productivity when it happens.
That at least narrows it more to the theme itself or some themekit bug and not single network oddness
|
|
Okay, just to let you know I am still working on this. I just opened a PR that will give us better error outputs for this. The error will include the HTTP Status Code so that we can figure out what sort of response we are getting. I am also flushing the response body to a temporary file, so that we can examine the response. I am pretty sure that the response we are getting is an html error page (which we cannot unmarshal) so if we can see the html page, we can get what the error message is. |
|
Since we will not be merging anything around Black Friday/Cyber Monday, I have created a prerelease with the additions for this issue included in it. You can install the prerelease by running |
|
Not quite sure if my issue is related to this one, but...
|
|
@alexissel for the images, do you have any ignore rules in your config for the development environment? |
|
Nope Edit: |
|
|
|
I am having the same issue as above but when calling a specific
All instances fail with the same problem as above: [development] Errors encountered:
[development] error downloading assets/Gotham-Bold.woff2: could not read response body
This may mean that the request was not able to finish successfully
Http Response Status: 200
Request ID: 0bcf25df-9182-4a7d-8ff1-95fdee1bfa29
Error: context canceled
[development] error downloading assets/Gotham-BookItalic.woff2: could not read response body
This may mean that the request was not able to finish successfully
Http Response Status: 200
Request ID: e8d3f185-3010-4914-aaaf-986743d39e76
Error: context canceled
finished command with errors[development] Errors encountered:
[development] error downloading assets/bundle.ajax-search.js: could not read response body
This may mean that the request was not able to finish successfully
Http Response Status: 200
Request ID: 7ff048f1-48c5-4e34-b5d1-7cfc28ce8aa1
Error: context canceled
finished command with errors[development] Errors encountered:
[development] error downloading assets/Gotham-MediumItalic.woff2: could not read response body
This may mean that the request was not able to finish successfully
Http Response Status: 200
Request ID: 89ab149a-1323-4be2-b0e6-a9342db2a05d
Error: context canceled
[development] error downloading assets/Gotham-LightItalic.woff2: could not read response body
This may mean that the request was not able to finish successfully
Http Response Status: 200
Request ID: 4ebbfca3-ba9d-4f0a-b5a6-56bc22499387
Error: context canceled
[development] error downloading assets/Gotham-BookItalic.woff2: could not read response body
This may mean that the request was not able to finish successfully
Http Response Status: 200
Request ID: 1991085b-07d0-46f1-8f45-876a2fa4964b
Error: context canceled
finished command with errors |
|
Also getting these kinds of errors on After a retry it usually works, sometimes I need two retries. Has been like this for months now. |
Is your feature request related to a problem? Please describe.
Using version 1.0.0 windows 10 64 bit
When downloading theme files, using themekit get -t=THEMEID, you may get the following message output:
error downloading asset: received a malformed responseWhich seems to be from ErrMalformedResponse
Without the verbose (-v) flag after the download process finishes this may be displayed multiple times.
And the progress count indicates all files have downloaded but some files are actually missing when checking the folders&files on the OS.
With the verbose flag I will see the above error in between messages of "Successfully wrote X to disk" so creates an ambiguity because what does that mean when following an error?
Without the filename or a checksum system you have to download the theme zip and do a comparison if you want to soothe the file paranoia, and then copy any missing files
Describe the solution you'd like
Included the relevant filename so interpretation of the message is easier.
Or clarifying if a file needs to be redownloaded either in the console or in the documentation.
Or provide a log of affected files to then pipe back into themekit for individual download.
Or prompt user to retry missing files
Describe alternatives you've considered
Alternative is to manually download theme zip from the store admin via email kinda rendering themekits download capability pointless when these errors occur.
Additional context
I'm not able to reliably reproduce the same output with the -v flag or narrow down if it's specific file types, or as a problem on shopifys end or my end.
The theme was created less than an hour before running command
Increasing the timeout to 1minute doesn't make it go away.
The text was updated successfully, but these errors were encountered: