Skip to content

Gold Pre-submit flow for contributors without permissions#47551

Merged
fluttergithubbot merged 20 commits intoflutter:masterfrom
Piinks:noAuthGold
Jan 9, 2020
Merged

Gold Pre-submit flow for contributors without permissions#47551
fluttergithubbot merged 20 commits intoflutter:masterfrom
Piinks:noAuthGold

Conversation

@Piinks
Copy link
Copy Markdown
Contributor

@Piinks Piinks commented Dec 20, 2019

Description

There is currently a workaround in place to un-block first time/un-privileged contributors that cannot execute pre-submit tryjobs, but it does not allow these contributors to land new or updated golden file changes.
This is a more complete solution that removes that work around and provides a workflow for these contributors.

For permission-prohibited contributors, the presubmit comparator will instead use the goldctl imgtest check feature to lookup image hashes to confirm results. If a check fails, it will retrieve and use the raw expectations to confirm if it is a new test, allowing it to pass through, or if it is an expected change via Gold's ignore feature.

This very similar to the pre-tryjob implementation, without requesting images and manually comparing, and without the need to access Gold's details API.

Better error messages have been included in this change to provide more informative and actionable messaging in all of these use cases.

Related Issues

Fixes #46687
Fixes #47898

Tests

As an authorized contributor in this case, I reversed the logic on an earlier commit (and since reverted) to ensure that the presubmit checks completed correctly for the unauthorized use case.

  • Restores all ignore tests from before tryjobs so that first-time contributors can make use of the feature
  • The presubmit comparator makes the correct choice for authorized and unauthorized test cases
  • The unauthorized comparator uses ignores to allow for expected changes and new tests

Checklist

Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I signed the CLA.
  • I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I updated/added relevant documentation (doc comments with ///).
  • All existing and new tests are passing.
  • The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
  • I am willing to follow-up on review comments in a timely manner.

Breaking Change

Did any tests fail when you ran them? Please read Handling breaking changes.

  • No, no existing tests failed, so this is not a breaking change.

@Piinks Piinks added a: tests "flutter test", flutter_test, or one of our tests c: contributor-productivity Team-specific productivity, code health, technical debt. framework flutter/packages/flutter repository. See also f: labels. work in progress; do not review team-infra Owned by Infrastructure team labels Dec 20, 2019
@Piinks Piinks mentioned this pull request Dec 30, 2019
13 tasks
}

/// Creates an animation controller with no upper or lower bound for its value.
/// Creates an animation controller with no upper or lower bound for its
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trivial framework change to trigger the appropriate test shards.

Comment thread packages/flutter_goldens_client/lib/skia_client.dart Outdated
@Piinks Piinks added a: error message Error messages from the Flutter framework c: regression It was better in the past than it is now labels Dec 30, 2019
@Piinks Piinks requested a review from goderbauer December 30, 2019 20:20
@Piinks Piinks changed the title [WIP] Gold Pre-submit flow for contributors without permissions Gold Pre-submit flow for contributors without permissions Dec 30, 2019
Comment thread packages/flutter_goldens/lib/flutter_goldens.dart Outdated
Comment thread packages/flutter_goldens/lib/flutter_goldens.dart Outdated
@Piinks Piinks added work in progress; do not review and removed c: regression It was better in the past than it is now labels Jan 6, 2020
@Piinks
Copy link
Copy Markdown
Contributor Author

Piinks commented Jan 8, 2020

This is ready for review. @dnfield, this is the full solution for users without the proper permissions on Cirrus, and replaces the workaround I put in place before the holiday.

return FlutterPreSubmitFileComparator(baseDirectory.uri, goldens);

final bool hasWritePermission = platform.environment['CIRRUS_USER_PERMISSION'] == 'admin'
|| platform.environment['CIRRUS_USER_PERMISSION'] == 'write';
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this how we check elsewhere?

I think it would be better if we check the actual env variable that has the service account key and see if it starts with ENCRYPTED[ or something like that. That's what we really care about, right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do! :)

Copy link
Copy Markdown
Contributor

@dnfield dnfield left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

a: error message Error messages from the Flutter framework a: tests "flutter test", flutter_test, or one of our tests c: contributor-productivity Team-specific productivity, code health, technical debt. framework flutter/packages/flutter repository. See also f: labels. team-infra Owned by Infrastructure team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The output when there's a Skia Gold error is inscrutable Gold tryjobs cannot be executed by first-time contributors

5 participants