Download from GitHub Releases stalled

[probonopd]

GitHub Releases does support HTTP Range Requests as we can verify:

curl --header "Range: bytes=200-500" "https://github-cloud.s3.amazonaws.com/releases/9435153/856d1954-a481-11e6-8b97-7a5fd7f3f61e.AppImage?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAISTNZFOVBIJMK3TQ%2F20161107%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20161107T000238Z&X-Amz-Expires=300&X-Amz-Signature=83b9c0951194d8f9ce3dbc0a6d08782ed6d1d283d1c9fc9795bcc19cec3e6fb8&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dappimagetool-x86_64.AppImage&response-content-type=application%2Foctet-stream"

Nevertheless, appimageupdate stalls when trying to download from GitHub Releases. Is this a bug in zsync-curl?

[mike-whittaker-work]

Switch on the more verbose libcurl output, by setting
export CURLOPT_VERBOSE=1

[mike-whittaker-work]

See #16 - but looks as if S3 syncs cause glitches in the reported content-size, which seems to be able to cause zsync-curl to stall on its final block.

It needs to be able to recover from the situation in a deterministic way, rather than forever repeating the request.

[probonopd]

Just for clarification, do you see "S3 syncs cause glitches in the reported content-size" as S3 issue (that could be reported to AWS, and possibly additionally worked around in zsync-curl) or an issue of how zsync-curl uses the (correct) data coming from AWS?

[mike-whittaker-work]

I do not know - but see apparently invalid changes in Content-Length, which seems to be correlated with z-c misbehaviour.

Hence I would say z-c could do with a deterministic path out of all potential loops ! Ideally it would spot a "stuck" condition and restart the whole transfer - this behaviour enabled by cmd-line option of course.

[probonopd]

@mike-whittaker-work Do you also get "Empty reply from server" when using CURLOPT_VERBOSE=1 as in #17?

[mike-whittaker-work]

I have not seen that message - but as implied before, development in this area has now stopped so will not be able to try any other experiments currently. That's one reason I dumped those logs into the Issues comments, so at least there would be something to be going on with.

[probonopd]

So I am going to assume this is the same issue as analyzed in #17. Let's reopen if it proves otherwise.

[probonopd]

Just fixed a bug in appimagetool that would result in the SHA-1 of the zsync file to be wrong when the AppImage was signed (since the zsync file had been generated prior to signing the AppImage). AppImage/AppImageKit@88853e9

To verify that this is not the case with your AppImage, run

wget https://github.com/probonopd/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage
wget https://github.com/probonopd/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage.zsync

strings *zsync |  grep SHA-1 | cut -d " " -f 2
sha1sum *.AppImage

The signatures must match. If they don't, update your appimagetool.

With this in place, I am now able to download from GitHub Releases with zsync-curl successfully. Hence closing. Feel free to comment if you think that this needs to be reopened.