Make PRs from external user repositories pass or skip CI jobs

[teor2345]

Motivation

Currently, Zebra's CI fails on PRs from external contributors. This can be a really confusing and negative experience for a first-time contributor.

Example: (recent examples first)
#7516
#4527

Jobs that need to be skipped

We don't want external PRs running on our Google Cloud, so we need to skip all those jobs.
We want to skip release and deployment jobs as well, because they use secret keys.

See the full list here:
#4529 (comment)

Jobs that need to be fixed

All other jobs should be fixed if they don't work for external PRs.

Designs

We can skip some tests that would otherwise fail, because Mergify will catch them when it does a full test run. But we need to disable in-place merges to make sure Mergify does a separate run.

This is more secure, because we review code, approve it, then run Mergify on it.

Related Work

Our previous attempts to fix this issue didn't work:

• Use OpenID Connect instead of the long-lived API keys for GH-actions deployments #3419

[teor2345]

We're getting closer to the release candidate series, so this is a medium priority now.

[teor2345]

Attracting external developers is not one of the goals of the release candidate, so this is a low priority.

[mpguerra]

Is it just as simple as skipping the CI Docker workflow or are there others that would need to be skipped?

How will this change affect the complexity of our CI rules and/or setup?

[teor2345]

Is it just as simple as skipping the CI Docker workflow or are there others that would need to be skipped?

We'll also need to skip other workflows that write to our GitHub or Docker:

• release drafter (PR auto-labels)
• zcash-lightwalletd (google cloud Docker image push)

And avoid sending our GitHub access token to the protoc installer, by doing one of these things:

• just skip those entire workflows (simplest option, and a tiny amount of work)
• skip the protoc and --all-features parts of those workflows for external users (slightly more work)
• commit the generated files to our git repository, and have a workflow that checks they are up to date (this would take slightly more work, but simplify a lot of other workflows, and decrease our CI failure

If we decide we really want one of these workflows to run, we can do the more complicated thing for it later.

How will this change affect the complexity of our CI rules and/or setup?

If we choose the simplest option, it is:

• change the workflow triggers for about 5 workflows by adding 2 lines of conditions
• change the patch workflows for those workflows to run instead

If we change the patch workflows, I don't think we even need to change Mergify at all. Which keeps things a lot simpler.

[gustavovalverde]

I'll be taking this into account for the redesign as we should really solve this for contributors.

[mpguerra]

Let's try to do this in 2023 Sprint 13 if we have time

[teor2345]

This is a higher priority now we know QEDIT is going to start building on Zebra.

[teor2345]

Jobs that need to be skipped

We don't want external PRs running on our Google Cloud, so we need to skip all those jobs.
We want to skip release and deployment jobs as well, because they use secret keys.

Jobs that need to be fixed

All other jobs should be fixed if they don't work for external PRs.

[gustavovalverde]

While researching this (as I previously made changes to allow this behavior, by removing most secrets) I recently realized there's an open discussion as GitHub variables are impeding this from happening. And reverting from variables to fix values is a no-go: https://github.com/orgs/community/discussions/44322

In the meanwhile, the "best" approach is to request reviewers to trigger the actions after validating the proposed code, which would use the reviewer permissions.

I made this change as a workaround, and I'll be closing this ticket as not-planned for now.

[teor2345]

What does this look like? Is there a test PR somewhere?

[gustavovalverde]

I've seen this in other open-source repos were I've contribute to. It's basically a button around here, asking for permissions to run the workflows

[teor2345]

This fix does not work completely. Most CI runs successfully, but anything that uses Google Cloud doesn't work due to authentication issues. Let's test the next fix either before or after making it?

[teor2345]

Here's a quick solution to this issue:

• skip the jobs that are failing if a PR isn't from a branch in ZcashFoundation/zebra, and add patch jobs (or change their run conditions)
• let mergify check those jobs once the PR is approved