Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[iOS] Dispose of log readers and port forwarders if launch fails #127140

Merged
merged 4 commits into from May 19, 2023

Conversation

vashworth
Copy link
Contributor

@vashworth vashworth commented May 18, 2023

When tests run in our CI using flutter drive, if there is a failure it will loop and try again.

while (attempt < _kLaunchAttempts) {
result = await device.startApp(
_applicationPackage,
mainPath: mainPath,
route: route,
debuggingOptions: debuggingOptions,
platformArgs: platformArgs,
userIdentifier: userIdentifier,
prebuiltApplication: prebuiltApplication,
);

However, it's using the same device instance for each iteration. So what was happening was when it would fail to launch, it would tell its listeners that it was cancelled.

if (_debuggerOutput.hasListener) {
// Tell listeners the process died.
await _debuggerOutput.close();
}

Then when the next iteration started, the vmServiceDiscovery would immediately return with null because the deviceLogReader would be cached from the previous iteration and would already be cancelled. Therefore, bypassing and cancelling the timer.

localUri = await vmServiceDiscovery?.uri;
}
timer.cancel();
if (localUri == null) {
await iosDeployDebugger?.stopAndDumpBacktrace();
return LaunchResult.failed();
}

return _logReaders.putIfAbsent(app, () => IOSDeviceLogReader.create(

In addition, it seems like sometimes the stop would fail and therefore the the drain would never get the signal that it was done and therefore would hang forever. There was no indication that the stop had failed though because the logs were going to the stream that had no listeners since deviceLogReader was already cancelled.

Future<void> stopAndDumpBacktrace() async {
if (!debuggerAttached) {
return;
}
try {
// Stop the app, which will prompt the backtrace to be printed for all threads in the stdoutSubscription handler.
_iosDeployProcess?.stdin.writeln(_signalStop);
} on SocketException catch (error) {
// Best effort, try to detach, but maybe the app already exited or already detached.
_logger.printTrace('Could not stop app from debugger: $error');
}
// Wait for logging to finish on process exit.
return logLines.drain();
}

Fixes #127141

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
  • I signed the CLA.
  • I listed at least one issue that this PR fixes in the description above.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or this PR is test-exempt.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@flutter-dashboard flutter-dashboard bot added platform-ios iOS applications specifically tool Affects the "flutter" command-line tool. See also t: labels. labels May 18, 2023
@vashworth vashworth marked this pull request as ready for review May 18, 2023 22:09
return LaunchResult.failed();
}
return LaunchResult.succeeded(vmServiceUri: localUri);
} on ProcessException catch (e) {
await iosDeployDebugger?.stopAndDumpBacktrace();
_logger.printError(e.message);
await dispose();
return LaunchResult.failed();
} finally {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason not to just call dispose() from the finally block?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want it to dispose when it launches successfully

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahh yeah, good point :)

final FakeProcessManager processManager = FakeProcessManager.list(<FakeCommand>[
FakeCommand(
command: const <String>['ios-deploy'],
stdout: '(lldb) run\r\nsuccess\r\nsuccess\r\nprocess signal SIGSTOP\r\n\r\nerror: Failed to send signal 17: failed to send signal 17',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, would we actually expect to see '\r' here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh good question, I'm not sure. I just grabbed that from the test above. I'll try and figure out

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not worth investigating, just wondering if there was a legit reason.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just happened to come across the answer to this:

// The lldb console stream from ios-deploy is separated lines by an extra \r\n.
// To avoid all lines being double spaced, if the last line from the
// debugger was not an empty line, skip this empty line.
// This will still cause "legit" logged newlines to be doubled...

@@ -147,6 +147,33 @@ void main () {
expect(logger.traceText, contains('* thread #1'));
});

testWithoutContext('debugger attached and stop failed', () async {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this test verifying any of your changes here? It's passing for me if I revert your other changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, let me rework it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay updated it

Copy link
Member

@christopherfujino christopherfujino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for this fix!

@vashworth vashworth added the autosubmit Merge PR when tree becomes green via auto submit App label May 19, 2023
@auto-submit auto-submit bot merged commit b9e8b0a into flutter:master May 19, 2023
124 checks passed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 21, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 21, 2023
auto-submit bot pushed a commit to flutter/packages that referenced this pull request May 22, 2023
flutter/flutter@077d644...ab57304

2023-05-20 engine-flutter-autoroll@skia.org Roll Flutter Engine from 482c99af9c69 to aac09195688d (1 revision) (flutter/flutter#127241)
2023-05-20 andrewrkolos@gmail.com Reland "[tool] Move Java functions to their own file" (flutter/flutter#126577)
2023-05-20 engine-flutter-autoroll@skia.org Roll Flutter Engine from f0c02aee69db to 482c99af9c69 (1 revision) (flutter/flutter#127240)
2023-05-19 82336674+gilnobrega@users.noreply.github.com Do not animate `TabBarView` if controller is invalid (flutter/flutter#123442)
2023-05-19 54558023+keyonghan@users.noreply.github.com Run Mac intel only targets on both intel and arm (flutter/flutter#127230)
2023-05-19 737941+loic-sharma@users.noreply.github.com [Windows] Ensure window is shown (flutter/flutter#127046)
2023-05-19 engine-flutter-autoroll@skia.org Roll Flutter Engine from 3267fa29491a to f0c02aee69db (4 revisions) (flutter/flutter#127233)
2023-05-19 engine-flutter-autoroll@skia.org Roll Flutter Engine from 2b14f8a1f21c to 3267fa29491a (4 revisions) (flutter/flutter#127224)
2023-05-19 pq@users.noreply.github.com fixes to anticipate next Dart linter release (flutter/flutter#127211)
2023-05-19 katelovett@google.com Remove deprecated OverscrollIndicatorNotification.disallowGlow (flutter/flutter#127050)
2023-05-19 engine-flutter-autoroll@skia.org Roll Flutter Engine from f471b37a2146 to 2b14f8a1f21c (1 revision) (flutter/flutter#127221)
2023-05-19 christopherfujino@gmail.com [flutter_tools] only try to take a screenshot from flutter drive if the --screenshot flag is passed (flutter/flutter#127150)
2023-05-19 zanderso@users.noreply.github.com Roll goldctl to f808dcff91b221ae313e540c09d79696cd08b8de (flutter/flutter#127218)
2023-05-19 engine-flutter-autoroll@skia.org Roll Packages from b31a128 to 1e214d7 (3 revisions) (flutter/flutter#127217)
2023-05-19 engine-flutter-autoroll@skia.org Roll Flutter Engine from a0ea4d2d9ea5 to f471b37a2146 (1 revision) (flutter/flutter#127212)
2023-05-19 joshualitt@google.com Revert "Migrate benchmarks to package:web" (flutter/flutter#127207)
2023-05-19 ychris@google.com [tool] delete xcresult bundle file before each xcode retry. (flutter/flutter#127144)
2023-05-19 vashworth@google.com [iOS] Dispose of log readers and port forwarders if launch fails (flutter/flutter#127140)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages
Please CC dit@google.com,rmistry@google.com,stuartmorgan@google.com on the revert to ensure that a human
is aware of the problem.

To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
CaseyHillers pushed a commit to CaseyHillers/flutter that referenced this pull request May 24, 2023
…tter#127140)

When tests run in our CI using `flutter drive`, if there is a failure it will loop and try again.

https://github.com/flutter/flutter/blob/434b81f1a58b8528299a63ae7c3ded2aa519e3c9/packages/flutter_tools/lib/src/drive/drive_service.dart#L177-L186

However, it's using the same `device` instance for each iteration. So what was happening was when it would fail to launch, it would tell its listeners that it was cancelled.
https://github.com/flutter/flutter/blob/434b81f1a58b8528299a63ae7c3ded2aa519e3c9/packages/flutter_tools/lib/src/ios/ios_deploy.dart#L486-L489 

Then when the next iteration started, the `vmServiceDiscovery` would immediately return with null because the `deviceLogReader` would be cached from the previous iteration and would already be cancelled. Therefore, bypassing and cancelling the timer.

https://github.com/flutter/flutter/blob/434b81f1a58b8528299a63ae7c3ded2aa519e3c9/packages/flutter_tools/lib/src/ios/devices.dart#L585-L591

https://github.com/flutter/flutter/blob/434b81f1a58b8528299a63ae7c3ded2aa519e3c9/packages/flutter_tools/lib/src/ios/devices.dart#L627

In addition, it seems like sometimes the stop would fail and therefore the the drain would never get the signal that it was done and therefore would hang forever. There was no indication that the stop had failed though because the logs were going to the stream that had no listeners since `deviceLogReader` was already cancelled.
https://github.com/flutter/flutter/blob/434b81f1a58b8528299a63ae7c3ded2aa519e3c9/packages/flutter_tools/lib/src/ios/ios_deploy.dart#L563-L576

Fixes flutter#127141
nploi pushed a commit to nploi/packages that referenced this pull request Jul 16, 2023
…r#4051)

flutter/flutter@077d644...ab57304

2023-05-20 engine-flutter-autoroll@skia.org Roll Flutter Engine from 482c99af9c69 to aac09195688d (1 revision) (flutter/flutter#127241)
2023-05-20 andrewrkolos@gmail.com Reland "[tool] Move Java functions to their own file" (flutter/flutter#126577)
2023-05-20 engine-flutter-autoroll@skia.org Roll Flutter Engine from f0c02aee69db to 482c99af9c69 (1 revision) (flutter/flutter#127240)
2023-05-19 82336674+gilnobrega@users.noreply.github.com Do not animate `TabBarView` if controller is invalid (flutter/flutter#123442)
2023-05-19 54558023+keyonghan@users.noreply.github.com Run Mac intel only targets on both intel and arm (flutter/flutter#127230)
2023-05-19 737941+loic-sharma@users.noreply.github.com [Windows] Ensure window is shown (flutter/flutter#127046)
2023-05-19 engine-flutter-autoroll@skia.org Roll Flutter Engine from 3267fa29491a to f0c02aee69db (4 revisions) (flutter/flutter#127233)
2023-05-19 engine-flutter-autoroll@skia.org Roll Flutter Engine from 2b14f8a1f21c to 3267fa29491a (4 revisions) (flutter/flutter#127224)
2023-05-19 pq@users.noreply.github.com fixes to anticipate next Dart linter release (flutter/flutter#127211)
2023-05-19 katelovett@google.com Remove deprecated OverscrollIndicatorNotification.disallowGlow (flutter/flutter#127050)
2023-05-19 engine-flutter-autoroll@skia.org Roll Flutter Engine from f471b37a2146 to 2b14f8a1f21c (1 revision) (flutter/flutter#127221)
2023-05-19 christopherfujino@gmail.com [flutter_tools] only try to take a screenshot from flutter drive if the --screenshot flag is passed (flutter/flutter#127150)
2023-05-19 zanderso@users.noreply.github.com Roll goldctl to f808dcff91b221ae313e540c09d79696cd08b8de (flutter/flutter#127218)
2023-05-19 engine-flutter-autoroll@skia.org Roll Packages from b31a128 to 1e214d7 (3 revisions) (flutter/flutter#127217)
2023-05-19 engine-flutter-autoroll@skia.org Roll Flutter Engine from a0ea4d2d9ea5 to f471b37a2146 (1 revision) (flutter/flutter#127212)
2023-05-19 joshualitt@google.com Revert "Migrate benchmarks to package:web" (flutter/flutter#127207)
2023-05-19 ychris@google.com [tool] delete xcresult bundle file before each xcode retry. (flutter/flutter#127144)
2023-05-19 vashworth@google.com [iOS] Dispose of log readers and port forwarders if launch fails (flutter/flutter#127140)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages
Please CC dit@google.com,rmistry@google.com,stuartmorgan@google.com on the revert to ensure that a human
is aware of the problem.

To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 16, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 17, 2023
@vashworth vashworth deleted the timeout-fix-1 branch September 27, 2023 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
autosubmit Merge PR when tree becomes green via auto submit App platform-ios iOS applications specifically tool Affects the "flutter" command-line tool. See also t: labels.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Stuck waiting for Dart VM url with no stack trace
2 participants