Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix for flakey analyze test #111895

Merged
merged 7 commits into from Sep 19, 2022
Merged

fix for flakey analyze test #111895

merged 7 commits into from Sep 19, 2022

Conversation

eliasyishak
Copy link
Contributor

@eliasyishak eliasyishak commented Sep 19, 2022 •
edited

This PR fixes #111272 by improving how we parse the stdout from flutter analyze to find only the relevant lines

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
  • I signed the CLA.
  • I listed at least one issue that this PR fixes in the description above.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or this PR is test-exempt.
  • All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@flutter-dashboard flutter-dashboard bot added the tool Affects the "flutter" command-line tool. See also t: labels. label Sep 19, 2022
@eliasyishak eliasyishak marked this pull request as ready for review September 19, 2022 18:29
christopherfujino
christopherfujino reviewed Sep 19, 2022
View reviewed changes
packages/flutter_tools/test/commands.shard/permeable/create_test.dart Outdated
@@ -40,6 +40,7 @@ const String _kNoPlatformsMessage = "You've created a plugin project that doesn'
const String frameworkRevision = '12345678';
const String frameworkChannel = 'omega';
const String _kDisabledPlatformRequestedMessage = 'currently not supported on your local environment.';
const String _kLintingErrorPattern = r"^\s*(?<type>error|info)(\s*[•|-]\s*)(?<desc>[\(\)`@\w\s'-]+)(\s*[•|-]\s*)(?<path>[\\\w'-\/:]+)(\s*[•|-]\s*)(?<lint>[\w'-]+)$";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pattern is pretty inscrutable :) Is this approach superior to only collecting errors AFTER 'Analyzing flutter_project'?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, the pattern is hard to decipher. But I went with this approach because I wasn't sure if the analyze tool could potentially output more lines after the Analyzing... line.

I can make the necessary updates to only check lines after that particular line if you think that is better?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To help with deciphering it as well, I added named groups within the regex matches so it could potentially make it easier to debug... but adding the named groups also makes the pattern string pretty long

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, the pattern is hard to decipher. But I went with this approach because I wasn't sure if the analyze tool could potentially output more lines after the Analyzing... line.

"Analyzing flutter_project..." occurs after we're done with the pub get flow: https://github.com/flutter/flutter/blob/master/packages/flutter_tools/lib/src/commands/analyze_once.dart#L115. It's not perfect, but at least if we regress this I'm pretty sure it will be in our own tool code.

@eliasyishak
removing difficult regex pattern
3f39614 
instead looking for the first instance of `Analyzing flutter_project...` to determine when to start capturing error outputs from `flutter analyze`
christopherfujino
christopherfujino approved these changes Sep 19, 2022
View reviewed changes
Copy link
Member

@christopherfujino christopherfujino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eliasyishak eliasyishak changed the title regex pattern approach for flakey analyze test fix for flakey analyze test Sep 19, 2022
@eliasyishak eliasyishak added the autosubmit Merge PR when tree becomes green via auto submit App label Sep 19, 2022
@auto-submit auto-submit bot merged commit 9e87a5b into flutter:master Sep 19, 2022
This was referenced Sep 20, 2022
Merged
Merged
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Sep 20, 2022
@engine-flutter-autoroll
9e87a5b08 fix for flakey analyze test (flutter/flutter#111895)
7e97a95
engine-flutter-autoroll added a commit to engine-flutter-autoroll/plugins that referenced this pull request Sep 20, 2022
@engine-flutter-autoroll
9e87a5b fix for flakey analyze test (flutter/flutter#111895)
431665e
@eliasyishak eliasyishak deleted the analyze_flakey_test_fix branch September 20, 2022 15:16
@eliasyishak eliasyishak mentioned this pull request Sep 20, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@christopherfujino christopherfujino christopherfujino approved these changes

Assignees
No one assigned
Labels
autosubmit Merge PR when tree becomes green via auto submit App tool Affects the "flutter" command-line tool. See also t: labels.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Shard tool_tests_commands is severe flaky at test create_test
2 participants
@eliasyishak @christopherfujino