Skip to content
This repository has been archived by the owner on Oct 2, 2023. It is now read-only.

Commit

Permalink
pacman-helper: bring the big hammer against those timeout issues
Browse files Browse the repository at this point in the history
For some reason (quota? but why would it be only occasionally?), BinTray
recently started timing out on me frequently. Like, *frequently*. Like,
*every single time* I needed to upload new packages, wasting my time
trying to clean up inconsistent states (because the database cannot be
uploaded transactionally, for technical reasons: BinTray does not allow
different versions to use the same file path).

The problem seems to occur only with the package database, where 16
relatively small files are uploaded in quick succession. So maybe it
*does* have something to do with a usage quota. But why, then, does it
succeed occasionally (and used to succeed all the time)? Be that as it
may, we really need to get a working workaround for this issue because
maintenance is hard enough and I do not exactly need the stress involved
in a failed upload that leaves Git for Windows' Pacman repositories in a
non-working state.

So let's get out the big toolbox and throw *everything* we have at the
problem. All of cURL's timeout options, set to 5 minutes (instead of the
20 seconds it seems to take to timeout). And since those options seem
not to help even one bit, let's also look at the exit code (which is
thankfully quite informative for cURL) and if we detect the same issue
(exit code 7: Failed to connect to host), we sleep 5 seconds and try
again. And again. And again. Until it succeeds, or until the sun burns
out, whichever comes first.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
  • Loading branch information
dscho committed Aug 9, 2017
1 parent 9dc1758 commit 056ed64
Showing 1 changed file with 12 additions and 2 deletions.
14 changes: 12 additions & 2 deletions pacman-helper.sh
Expand Up @@ -111,8 +111,18 @@ publish () { # <package> <version>
echo "publish: curl --netrc -fX POST $content_url/$1/$2/publish"
return
}
curl --netrc --retry 5 -fX POST "$content_url/$1/$2/publish" ||
die "Could not publish $2 in $1"
curl --netrc --connect-timeout 300 --max-time 300 \
--expect100-timeout 300 --speed-time 300 --retry 5 \
-fX POST "$content_url/$1/$2/publish" ||
while test $? = 7
do
echo "Timed out connecting to host, retrying in 5" >&2
sleep 5
curl --netrc --connect-timeout 300 --max-time 300 \
--expect100-timeout 300 --speed-time 300 --retry 5 \
-fX POST "$content_url/$1/$2/publish"
done ||
die "Could not publish $2 in $1 (exit code $?)"
}


Expand Down

0 comments on commit 056ed64

Please sign in to comment.