Push rule private beta!

[patrick-knight]

We want to hear from you on push rules!

Who put an EXE file in my iOS mobile App repo? Why can I push a 20MiB mp3 when this repo is all .MD docs files? All these mysteries will soon be answerable and controllable.

It's been a year since we showed y'all the future of protecting branches with the public beta of repository rules.

But we had one lingering thought this whole time. How do we let you to prohibit certain things from entering a repository?

The repository rule release only let you protect refs, meaning the commit is in the repository already and can be reachable. Often, that's fine because rules are great for preventing unintended changes or enforcing governance decisions. But what about risky files in a repo due to size, type, or location? Thats where push rules come in.

If you’ve had a chance to try out the repo rules that are now in public beta, we’re curious to know your thoughts and how we can make this feature work better for you!

🐛 Known issues

REST and GraphQL API will be available soon.

When push rules are in active or evaluate mode the usage of both the /git/trees and /git/blobs API endpoints are restricted.

[archon810]

@patrick-knight Can these be used to enforce LFS on large files? Something like "if file extension is *.foo AND file size > 20MB AND file is not on LFS => deny"? From the screenshot, it looks like it may work, although it's ambiguous - if the file bar.foo is using LFS and the rule set is for *.foo and size > 20MB, will it be denied no matter its size or will it be allowed no matter its size?

Can a rule be overridden by an admin on a case by case basis to push an exception to the rule?

[archon810]

Just tested this on a test repo, and unfortunately it doesn't work the way I would like it to.

The ruleset:

The test when not using LFS correctly blocks this 25MB file due to extension and size.

But the test when using LFS still blocks it due to extension:

testrepo> git push
Uploading LFS objects: 100% (1/1), 24 MB | 3.0 MB/s, done.
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 2 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 546 bytes | 42.00 KiB/s, done.
Total 4 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
remote: error: GH013: Repository rule violations found for refs/heads/master.
remote: Review all repository rules at http://github.com/BLA/testrepo/rules?ref=refs%2Fheads%2Fmaster
remote:
remote: - GITHUB PUSH PROTECTION
remote:   —————————————————————————————————————————
remote:     Resolve the following violations before pushing again
remote:
remote:     - Files cannot include restricted file extensions.
remote:       Found 1 violation:
remote:
remote:       testlargeapks/foo.apk
remote:
To github.com:BLA/testrepo.git
 ! [remote rejected] master -> master (push declined due to repository rule violations)
error: failed to push some refs to 'github.com:BLA/testrepo.git'

[patrick-knight]

We evaluate each rule independently, so pushes cannot contain any *.apk or any file over 5MB. and because the apk hit the rule it will get blocked regardless of LFS.

Can a rule be overridden by an admin on a case by case basis to push an exception to the rule?

Not today, but it sure is something we are thinking a lot about as these rules do introduce new points of friction.

[archon810]

We evaluate each rule independently, so pushes cannot contain any *.apk or any file over 5MB. and because the apk hit the rule it will get blocked regardless of LFS.

In this case, sadly this system is not powerful enough right now. Please consider more complex rules that would AND instead of OR.

[patrick-knight]

Thanks for the feedback!

[archon810]

UI-wise, this is confusing IMO:

I took it as you can provide a comma-separated list of extensions, like zip, rar, but the end result looked wrong, which made me re-evaluate the instructions and only then provide one extension per entry.

I think this UI can be clarified, and also it should block commas.

[patrick-knight]

Thanks, that's a good note.

[TWiStErRob]

Comma is a valid file name (and therefore extension) character, please don't block it outright.

[Brend-Smits]

Awesome addition!
It seems like the REST API has not been updated yet to reflect this new addition. When can we expect an update for this API so we can access and update the rule programmatically?
See documentation: https://docs.github.com/en/rest/repos/rules?apiVersion=2022-11-28#update-a-repository-ruleset

[patrick-knight]

Thanks.

That API is not yet ready, but should be available quite soon.

[ViacheslavKudinov]

But we got it in advance 😎
integrations/terraform-provider-github#2235

[bewuethr]

I would like to prevent certain files, such as .DS_Store, from being committed. If use a file path restriction for **/.DS_Store, it does indeed prevent adding these files, but it also prevents removing them, because the rule is

Prevent commits that include changes in specified file paths from being pushed to the commit graph.

Ideally, I would be able to say "do not allow adding matching files, but deleting is okay".

[patrick-knight]

Thanks for the suggestion. How common would the removal be for folks that don't have bypass? Would you see that as a daily occurrence, weekly?

[bewuethr]

Once per repository, really, just to clean up the instances where we already such a file committed.

[Haarolean]

Why is this availble only for private repos? As an open source maintainer, I'd like to be able to deny pushes with 10-mb binary files in PRs without having to manually inspect every part of one.

[archon810]

Where do you see that it's only available for private repos?

Rulesets are available in public repositories with GitHub Free and GitHub Free for organizations, and in public and private repositories with GitHub Pro, GitHub Team, and GitHub Enterprise Cloud. For more information, see "GitHub’s plans."

Push rulesets are available for the GitHub Team plan in internal and private repositories, and forks of repositories that have push rulesets enabled.

[Haarolean]

@archon810
https://github.blog/changelog/2024-04-18-push-rules-public-beta/

Push rules are now available for private and internal repositories for GitHub Teams, and across organizations for GitHub Enterprise Cloud.

Public repos don't even have a menu item for push rulesets:

[archon810]

Oh, right, push rulesets specifically, not rulesets in general. Disregard me!

Hopefully, they'll bring them to public repos after the beta period once it hits GA.

[patrick-knight]

Great question.

The limitation for public repo is driven by the behavior with forks. As push rules are designed to protect a repo and it's fork there are some concerns we have about how this would be received in the open source community. So we opted away from it for now. The public beta is a good way to listen to the feedback if we need to change that stance.

[TWiStErRob]

I wanted to give feedback on a branch ruleset, but the "Give feedback" button opens this page...

[patrick-knight]

😬 Whoops. Thats should only be on the new push rule beta page. We'll get that fixed up soon.

If you want to provide feedback on the existing branch and tag rules, feel free to jump into this discussion post.

[siMobin]

Why is this ruleset not for the organization's private repo?

[patrick-knight]

Hey @siMobin,

Push rules are available for private and internal repos for GitHub Team and across organizations for GitHub Enterprise.

[chrisxaustin]

Excellent new feature, thank you for doing this.

My first rule set was to restrict workflow changes, only allowing maintainer/admin roles in each repo to reduce the risk of malicious 3rd party actions being added accidentally. This is only a workaround though - I'd strongly prefer to have an org-level policy to only use the workflow that's in main. I don't like that someone can modify the workflow in a PR and have it applied before it has been reviewed/approved.

[TWiStErRob]

@chrisxaustin Are you aware of this GitHub feature? It might not do exactly what you're after, but it's actually a more powerful way of protection in my reading. If your workflows call just a single command or script from your clone on the runner, your repository is vulnerable to external code execution, because an actor can raise a PR with the script/build script changed and that will execute even without touching the workflow files.

[bewuethr]

We set up a rule to prevent pushing .DS_Store files, but then disabled it (set it to "evaluate) because it prevented also removing those files (see here).

We bumped into another issue with the same rule: even though set to evaluate, it resulted in workflows being unable to push commits to a third-party repository. We use fjogeleit/yaml-update-action to update a version in a YAML file. This step started failing with

HttpError: Operation is blocked due to enforced push rulesets.

Deleting the push rule, even though it was only in evaluation mode, made the issue go away. Blocking in eval-only mode seems to be a bug.

We use a PAT to check out the repo and push to it, for what it's worth, and the bot user for which the PAT was generated has admin permissions on the repo; this lets the pushes override a "must create PR first" branch protection rule.

[patrick-knight]

Quick glance in that repos looks like there is use of the trees and blobs endpoints which causes a known behavior we are working on resolving.

[mkushakov]

We have encountered the same issue. Disabling rule seems unblocked us, so we are waiting for fix before enforcing it again. @patrick-knight please let us know when we can test it again.

[patrick-knight]

Will do!

[nicholasgriffintn]

We were also getting this with Renovate, I think that's probably the same issue, in case anyone comes along.

[technologik]

@patrick-knight +1, we found evaluate mode restricting pushes of blobs during our automation that uploads blobs during PR creation.

[rcmarques3]

Hi @patrick-knight,

Using rulesets (and branch protection policies before that) we always faced a few issues while enforcing commit signatures.
Unsigned commit objects can get pushed to the repository, and it's the ref update that rejected certain operations that would make them part of the branches covered by the rules.
This ref validation sometimes causes problems for us, in situations where you're merging say a previously existing support branch into main. Those commits usually were valid at the time they were pushed to that branch, but when entering main's history their signature is once again checked.
This can fail due to:

1. the signing GPG key was set to expire, the user has generated a new one but incorrectly removed the old one from his profile
2. some of those committers are no longer valid user accounts
3. those past commits were part of a migration / GitHub onboarding and signatures were bypassed by an admin at the time

Any of these situations will cause a (signed) merge commit on main to get blocked, because some of the baggage it's bringing along does not presently yield a valid signature.

This new pre-receive enforcement seems like a great opportunity to address this use case.
Any chance you could support enforcing Code Signatures in the push rulesets, so we could use it there instead of branch rulesets and thus avoiding altogether what I described?

And BTW looking at the existing branch rulesets, enforcing commit metadata (ex: message patterns) does seem to follow the same kind of reasoning. Not that I have an immediate need for it, but it does make sense to me that it could be enforceable in the push phase as well.

[patrick-knight]

Great question @rcmarques3.

There's a lot of threads to pull here, but at the moment we don't have plans to move those into push rules. There are scenarios for large monorepos where keeping some of these behaviors as branch level rules continues to make sense.

[rcmarques3]

Of course having this as an option in the push rules doesn't mean it should be removed from branch rules :) because different use cases do need different policies.
Just wanted to bring this to your attention since IIRC the branch rulesets did suffer some small adjustments to their behavior in beta regarding signatures, but never really fixed the problem for us.
Push rulesets seem like the perfect lifecycle spot to enforce this validation - and I did suggest something along those lines at the time, but no such hook existed back then.
I understand if there's no bandwidth to address everything, but do hope this manages to get implemented at some point. Thanks!

[zoezhangmattr]

hey there, this is awesome feature, i tried to enable one, and bypass github app, but it doesnt seem work for the bypass, it still get the
Error: Unhandled error: HttpError: Operation is blocked due to enforced push rulesets

[patrick-knight]

We do have a know issue with the usage of both the /git/trees and /git/blobs API endpoints and rules. Do you know if those are in use in your scenario?

[zoezhangmattr]

heya thanks for reply, yes, the github app is using these endpoints. is there any plan to support by passing the github app, we use it in quite a few scenarios. hopefully the new feature can support it as well?

[DrNeil]

hi, Push Rules are a great idea, thank you for getting them this far.
I work with repos where it would be more valuable to have a list of allowed file types, than a restricted list.
We can keep adding to the restricted list, however eventually it is going to be every file type other than the handful we want to support in the repos.
Please consider the inverse of restricted files, as a rule; allowed file types

[patrick-knight]

Thanks @DrNeil great suggestion. We're taking a look at data across the beta on how we best help with scenarios like this as well as "bundling" common file and path types for future itterations.

[bodgit]

We use a tool called release-please on pushes to master which creates releases/tags based on semantic commits. Part of this involves updating the current version in .release-please-manifest.json and we don't want regular users to edit this file so I thought this would be a good candidate for a push protection rule.

We run the workflow as a dedicated GitHub App so I created the push ruleset with this App in the bypass list. The workflow cannot push a change to this file if the rulset is either active or evaluating, only disabled and nothing ever appears in the rule insights. I've confirmed with the gh cli that the token is for the correct App account so this feels like a bug currently.

Happy to provide more information if needed.

[patrick-knight]

Hey @bodgit is there use of the trees or blobs endpoints in that tool? Based on the behavior of blocking on evaluate it seems like it might be the same behavior we are working on.

[bodgit]

I can't easily tell, but the error message I get is something like:

Error: release-please failed: Error adding to tree: <commit sha>

So based on the presence of "tree", perhaps that is the case?

[TBeijen]

Stumbled upon this (push ruleset) looking for a way to protect pipeline definitions.

At a first glance seems to work as expected. And solves a real problem in a simple way that is otherwise hard to solve. 👌
(Some thoughts: Intuitively I was aiming at solving this in the platform that accesses our resources, in my current case Azure Devops, but thinking it through: We also trust GitHub to protect our main branch, and by that the integrity of our systems. This is similar)

About this warning:

Push rulesets are applied to a repository and its downstream repositories. To review the downstream repositories that will inherit this ruleset, visit the forks page.

I get it (I think) but I can also imagine it complicating contributing by others.

One more thought: When having enterprise plan, to what extent will it be possible to impose certain rules in bulk to all repositories?

[patrick-knight]

I can also imagine it complicating contributing by others.

That drove our initial work to support just private and internal repos to limit the scope of impact there.

You can 100% apply push and branch/tag rules to all repos in an org when on GitHub enterprise today, with the exception of Enterprise Server where push rules are not currently availible.

[oed-sassmannc]

Thanks for this feature! As far as we could test, it perfectly fits our needs to protect our Workflows.
Are there any notes about GA release? Can't wait to use this in production.

[patrick-knight]

We are targeting a GA sometime between June and August assuming everything goes to plan.