-R isn't working for failed uploads

[fasterthanlime]

cf. original discussion at github-release#39

[fasterthanlime]

Here's what happens: failed uploads aren't listed in the /releases endpoint (which is what ReleaseOfTag uses), only in the /releases/:release_id/assets endpoint.

The replace logic needs another API call, no big deal :)

[mfonville]

I will check this patch out :-)

[mfonville]

It seems to work right!

Maybe this patch somebody had made for github-release is also interesting, to at least discover when receiving the 502: kidanekal@088eb5f

[fasterthanlime]

Ah, interesting — so GitHub is falsely returning 502 now and then, even when the upload went fine?

I'd rather have GitHub fix their (long-standing) upload issues than add cruft to work around it, though. (That's a self-serving point of view btw — I mostly run github-release on CI servers with very fast network connections, so I don't really mind having it upload twice! It was the manual intervention part that was annoying).

I'm really disappointed in GitHub on that one though. Network failures happen, but that's clearly a bug on their part 😢

[mfonville]

@fasterthanlime it seems that it happens sometimes, but I did run with that fork for a day, but it was not reliable. Sometimes it would not detect that an upload actually did fail... But at least processing the response code from GitHub would be useful, because currently I just wrap the upload-command inside a timeout, and if it takes too long I assume it dies and it will retry.

Also my server's connection is fast, so uploading twice ain't the problem, but reliably spotting a failed upload attempt would speed up the upload process (not having to wait for the timeout).

GitHub should indeed fix this issue, I am in contact with them about this since August last year, with multiple of their staff. But they are not able to reproduce it. I provided them with info from my failed upload attempts, but they are apparently not able to investigate their server's logs.
I doubt it will ever get fixed :-(

[fasterthanlime]

because currently I just wrap the upload-command inside a timeout, and if it takes too long I assume it dies and it will retry.

Ohhh since it's a naive golang executable it probably just hangs forever if the connection drops, too! I've had that problem with https://github.com/itchio/butler too, we're now using https://github.com/getlantern/idletiming with reasonable values to detect timeouts.

[mfonville]

ok, apparently the -R flag does not work completely for 100% of the cases... :(
Last night this happened:
The original upload attempt:
error: could not upload, status code (502 Bad Gateway), msg: Error uploading asset, errors:
and after that each subsequent retry:
error: could not upload, status code (422 status code 422), msg: Validation Failed, errors: [field: name, code: already_exists]

When I browsed to GitHub manually I could remove that failed upload.
So apparently assets can still block the upload.

If this would happen again in the future, what (e.g. API) command should I run to help with the debugging of this situation?

[mfonville]

Speaking of the devil. I just received another blocked upload. I will attach what the API returns in 'assets'; assets.txt
The file we try upload open_gapps-x86-5.1-stock-20160510.zip is not mentioned in the assets.
I will also attach some HTML from GitHub UI where it is actually removable.
brokenupload.txt

I wrote about GitHub too, when I was contacting them, why these half-uploaded files did not show up in the assets, and whether they please could make this possible.
But I understand that in your API asset requests
they are visible ???

[mfonville]

I think I got the root cause of the problem found: API pagination!
Just as described on github-release#45 I have so many assets, that they don't fit on a single API page!

Would you be able to implement pagination? https://developer.github.com/v3/#pagination
I have zero go-knowledge so I wouldn't be able to do so myself :-(

This is how it is done in Python, through an abstraction: https://github.com/PyGithub/PyGithub/blob/master/github/PaginatedList.py

(an temporary workaround would be to set per_page=100 and assume there are never more than 100 assets, but my projects at the moment up to 94, but that will become more when android N will be released :-/)

[fasterthanlime]

@mfonville ah, I did just set per_page to 100 in order to have a temporary fix :)