Skip to content

[flutter_tools] tool exit after repeated network failure #64878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Aug 31, 2020

Conversation

jonahwilliams
Copy link
Contributor

@jonahwilliams jonahwilliams commented Aug 29, 2020

Description

Exit the tool after a repeated network error to download. previously we were returning null and continuing on, leading to a ProcessException when we unzipped a missing file.

Fixes #64876
Fixes #64857

@flutter-dashboard flutter-dashboard bot added the tool Affects the "flutter" command-line tool. See also t: labels. label Aug 29, 2020
@jonahwilliams
Copy link
Contributor Author

Possibly #52437 (comment) too

@jonahwilliams jonahwilliams requested a review from zanderso August 31, 2020 16:04
@@ -70,7 +70,7 @@ class Net {

if (maxAttempts != null && attempts >= maxAttempts) {
_logger.printStatus('Download failed -- retry $attempts');
return null;
throw Exception('Failed to download $url');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_attempt could return null on success, and the exception object on an error. Then here you can rethrow the most recent error. Then at the callsite, you can catch IOException, or something more narrow, since it doesn't look like the text 'Failed to download...' is used there. The caller also presumably already knows what the url is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done?

@jonahwilliams jonahwilliams requested a review from zanderso August 31, 2020 16:41
@@ -1469,6 +1469,14 @@ class ArtifactUpdater {
try {
_ensureExists(tempFile.parent);
await _net.fetchUrl(url, destFile: tempFile, maxAttempts: 2);
} on Exception {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ehhhh. Okay, sorry. Maybe that wasn't the best suggestion. If there's some way to avoid catching Exception here that would be good, though, since it's only the IOExceptions that probably indicate a network issue on the end-user's side.

}
assert(response != null);

if (response.statusCode != HttpStatus.ok) {
_logger.printTrace('Download error: ${response.statusCode} ${response.reasonPhrase}');
final Exception exception = Exception(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where did the throwTooleExit go?

@@ -120,7 +121,7 @@ class Net {
exitCode: kNetworkProblemExitCode,);
}
_logger.printError(error.toString());
rethrow;
return error;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe these should all rethrow, fetchUrl can do a try {} catch on IOException {} with a continue in the catch if there are still attempts left? doesRemoteFileExist would also need a try {} catch {}, too, then I guess.

@jonahwilliams
Copy link
Contributor Author

I think the net logic is quite convoluted actually, which is part of the issue with trying to update it. It's not really clear from the contract what sorts of exceptions it should be handling, and which ones to leave to the callers.

If I were going to spend some time on it, what I would do is more or less replace Net with a wrapper that accepted an http client and made the request. The only job would be to turn a non-200 response into an exception so that the caller could handle that in a similar way to other failures.

This would let the artifact updater contain the logic for writing the bytes, which would allow it to handle any IOexceptions from writing.

@jonahwilliams
Copy link
Contributor Author

(also removing the retry logic, to allow the callers to include that if they wanted to)

Copy link
Member

@zanderso zanderso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were there any other users of Net?

Should the number of retries be wired up to method arguments or command line flags of e.g. precache?

@jonahwilliams
Copy link
Contributor Author

There is some usage in create.dart (for snippet creation), update_packages for downloading coverage (which doesn't need to be that resilient), and then the cache check that artifacts exist.

As a follow up these could be refactored/removed. Likely we'll want to keep the cache check for existence (header only request) and find out how to remove the others.

@jonahwilliams
Copy link
Contributor Author

I could hoist the retry count into a constant for now, and we could make it configurable if there is a feature request to do so

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
tool Affects the "flutter" command-line tool. See also t: labels.
Projects
None yet
3 participants