Automatically respond to crates.io API token leaks on GitHub (via GitHub secret scanning)

[greysteil]

Is your feature request related to a problem? Please describe.
Token leaks happen. There's been previous discussion about avoiding them in cargo but I couldn't find any discussion about automatically responding to them.

I'd like crates.io to automatically revoke any credentials that are leaked into a GitHub public repo. I run the GitHub secret scanning team, so I can provide you with a feed of potential matches we find in public repos, if you're willing to take action on them automatically.

We have a similar setup with folks like AWS - if you leak your AWS credentials on GitHub you'll quickly get an email from AWS saying the credential has been quarantined. Under the hood, they're receiving the feed of matches from GitHub.

Describe the solution you'd like
crates.io would have an endpoint for receiving details of leaked secrets from GitHub, following the specification here. crates.io would automatically revoke leaked secrets, and would email their owners to notify them that they had been revoked. The email would include details of where the secret had been exposed (as provided by GitHub).

Describe alternatives you've considered
crates.io could notify users of leaked secrets, but not automatically revoke them. However, this would leave leaked credentials exposed.

[Turbo87]

@greysteil would #2653 fit what you're requesting here?

[greysteil]

If API tokens were able to self-revoke then we (GitHub) could hit the self-revoke endpoint with our findings, yep. It might feel a bit weird, though - since the tokens we find don't "belong" to us, it's not totally natural that we would presume we can automatically revoke them. They do "belong" to crates (together with the user), so it feels more natural that you would initiate the revoke (and GitHub is just informing you of the leak).

With other partners, we sometimes have large numbers of matches, so it's helpful for us to be able to provide results in bulk. The crates.io token format looks relatively distinctive, though - I've just shipped the pattern to our scanning service so I can see what quantity of matches we're seeing.

[Turbo87]

@greysteil we've discussed this topic in our team meeting today. we generally agree that participating in the automated token scanning would be beneficial. we are however currently in the transition phase from Mozilla to the Rust Foundation (see https://foundation.rust-lang.org), and there are a few open question that the foundation would have to answer before we can move forward here.

in other words: we will have to put this issue on hold for a few weeks until the situation is a bit more structured, but we do plan to return to this :)

[greysteil]

Thanks for the update @Turbo87. Just let me know when you've had a chance do discuss further.

[greysteil]

Hey @Turbo87, revisiting this now the Rust Foundation has had a chance to get established.

The offer here is obviously an evergreen one, and the GitHub team and I will be here to help if you ever want to receive details of leaked secrets from GitHub. 😄

[Turbo87]

@greysteil we now have an experimental endpoint deployed to https://crates.io/api/github/secret-scanning/verify. do you have a way to test this on your end? can we assist with that in some way?

[greysteil]

That's great! OK, on the GitHub side we just need to add your endpoint to our detection service and deploy that change. I'll get it done today. Once that's done you'll be able to test in production by committing a token (for an account which doesn't have any associated crates).

I'll let you know when everything is deployed.

[Turbo87]

not sure if you already know this, but our current token format is cio[a-zA-Z0-9]{32} :)

[greysteil]

Yep! And all deployed - you should be able to test now.

A couple of follow ups:

• There are some (very straightforward) legals associated with the programme that basically say GitHub is allowed to use any information you provide us (like the regex above, or details of whether a detection is a true or false positive) - no real obligations on you/crates.io, it's just an IP grant to make sure GitHub doesn't ever get in trouble for using info you tell us. Can you email secret-scanning@github.com so we can send those over to you?
• Having scanned for the pattern above for the last few months, we're seeing about 20% of findings are true positives. Any chance of updating the pattern to make it more identifiable (e.g., with an _ following the prefix and perhaps a checksum)? If so we'd be able to add this pattern to push protection (which we'd like to roll out to public repos in 2023)