Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support retrying downloads if they transiently fail #1594

Closed
wants to merge 8 commits into from
Closed

Support retrying downloads if they transiently fail #1594

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
73226ae
lib/repo-pull: Support retrying requests on transient network errors
pwithnall May 22, 2018
5b78601
lib/fetcher-soup: Map more SoupStatus codes to known GIOErrors
pwithnall May 25, 2018
63b1ba4
ostree/trivial-httpd: Add --random-408s command line option
pwithnall May 28, 2018
30b6c08
tests: Test pull behaviour when network timeouts occur
pwithnall May 28, 2018
3c51ee4
fixup! lib/repo-pull: Support retrying requests on transient network …
pwithnall May 30, 2018
d8ad578
lib/repo-pull: Add some missing assertions for progress statistics
pwithnall May 30, 2018
ba8a265
lib/fetcher: Factor out HTTP status code handling from soup and curl
pwithnall May 30, 2018
e283a05
fixup! lib/fetcher: Factor out HTTP status code handling from soup an…
pwithnall May 30, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions src/ostree/ostree-trivial-httpd.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,14 @@ static int opt_random_500s_percentage;
/* We have a strong upper bound for any unlikely
* cases involving repeated random 500s. */
static int opt_random_500s_max = 100;
static int opt_random_408s_percentage;
static int opt_random_408s_max = 100;
static gint opt_port = 0;
static gchar **opt_expected_cookies;
static gchar **opt_expected_headers;

static guint emitted_random_500s_count = 0;
static guint emitted_random_408s_count = 0;

typedef struct {
int root_dfd;
Expand All @@ -70,6 +73,8 @@ static GOptionEntry options[] = {
{ "force-range-requests", 0, 0, G_OPTION_ARG_NONE, &opt_force_ranges, "Force range requests by only serving half of files", NULL },
{ "random-500s", 0, 0, G_OPTION_ARG_INT, &opt_random_500s_percentage, "Generate random HTTP 500 errors approximately for PERCENTAGE requests", "PERCENTAGE" },
{ "random-500s-max", 0, 0, G_OPTION_ARG_INT, &opt_random_500s_max, "Limit HTTP 500 errors to MAX (default 100)", "MAX" },
{ "random-408s", 0, 0, G_OPTION_ARG_INT, &opt_random_408s_percentage, "Generate random HTTP 408 errors approximately for PERCENTAGE requests", "PERCENTAGE" },
{ "random-408s-max", 0, 0, G_OPTION_ARG_INT, &opt_random_408s_max, "Limit HTTP 408 errors to MAX (default 100)", "MAX" },
{ "log-file", 0, 0, G_OPTION_ARG_FILENAME, &opt_log, "Put logs here (use - for stdout)", "PATH" },
{ "expected-cookies", 0, 0, G_OPTION_ARG_STRING_ARRAY, &opt_expected_cookies, "Expect given cookies in the http request", "KEY=VALUE" },
{ "expected-header", 0, 0, G_OPTION_ARG_STRING_ARRAY, &opt_expected_headers, "Expect given headers in the http request", "KEY=VALUE" },
Expand Down Expand Up @@ -291,6 +296,14 @@ do_get (OtTrivialHttpd *self,
soup_message_set_status (msg, SOUP_STATUS_INTERNAL_SERVER_ERROR);
goto out;
}
else if (opt_random_408s_percentage > 0 &&
emitted_random_408s_count < opt_random_408s_max &&
g_random_int_range (0, 100) < opt_random_408s_percentage)
{
emitted_random_408s_count++;
soup_message_set_status (msg, SOUP_STATUS_REQUEST_TIMEOUT);
goto out;
}

while (path[0] == '/')
path++;
Expand Down Expand Up @@ -511,6 +524,13 @@ run (int argc, char **argv, GCancellable *cancellable, GError **error)
goto out;
}

if (!(opt_random_408s_percentage >= 0 && opt_random_408s_percentage <= 99))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: the way I like to write these range type checks usually is:

if (!(0 <= opt_random_408s_percentage && opt_random_408s_percentage <= 99))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh that's OK, I see you're just following the existing formatting of the 500s one.

{
g_set_error (error, G_IO_ERROR, G_IO_ERROR_FAILED,
"Invalid --random-408s=%u", opt_random_408s_percentage);
goto out;
}

if (opt_log)
{
GOutputStream *stream = NULL;
Expand Down