Auto-triage rules for Dependabot

[erinhav]

What do you think about Dependabot's new auto-triage feature?

At GitHub, we’ve been thinking deeply about how to responsibly address long-running issues around alert fatigue and false positives. Rather than over-indexing on one or two criteria like reachability or dependency scope, we believe that a responsibly-designed solution should be able to detect and reason on a rich set of complex, contextual alert metadata.

That’s why, moving forward, we’re releasing a series of ships powered by an underlying, all-new, flexible and powerful alert rules engine!

If you’ve had a chance to try out the beta feature, we'd love to hear your feedback on 1) what's working for you, 2) what needs to change, and 3) what you'd like us to tackle next!

What are auto-triage rules?

Auto-triage rules help you responsibly reduce the amount of alerts needing immediate assessment, without compromising on any potentially concerning alerts. GitHub curates suggested rulesets for everyone, which have caught millions of false positive alerts since the May 2 release. For those that need more control, custom rules are available to help you scale your security strategy based on your risk tolerance and unique contextual needs.

📖 Helpful information:

• Feature FAQ and announcement
• Dependabot alerts product documentation
• Dependabot alerts REST reference
• Dependabot alerts GraphQL reference
• Dependabot alerts webhook reference

[testworksau]

We are on GHEC, and like the ability to be able to enable Dependabot at an org level.

Will you consider adding an option to allow organisation administrators to enable auto-dismissal across all repositories, rather than having to do this on a repo-by-repo basis?

[erinhav]

Absolutely! That's currently roadmapped as the next major project to focus on, after this quarter's ship of custom configurability at the repository level.

[jeremie0]

Will enabling this feature at the repo level be available via the REST API? (at least until we can enable org-wide)

[erinhav]

@jeremie0 would you still make use of enablement via API if we had better UI-based enablement (across org-levels)?

[jeremie0]

If we could enable org-wide via the UI that would replace the need for API.

[erinhav]

Thank you!

[dduzgun-security]

Would love to see support for yarn as well to reduce alerts on devDependencies 🙏
As of now, it seems to work only with npm

[erinhav]

Noted! We're investigating this now. ❤️

[dduzgun-security]

@erinhav I was curious to know if there were any updates on the support for yarn?

[SalimBensiali]

Hi @erinhav any further update on this? Currently I am able to work around this limitation by running: npm ls <pkg-name> --prod in a Github Action that fetches Dependabot alerts for the yarn ecosystem via Graphql, but the problem with this approach is there is no way to update the dependencyScope for the alert metadata via any API if it turns out the scope is indeed DEVELOPMENT instead of RUNTIME

[SalimBensiali]

HI @erinhav 👋
Would you have more info on "custom alert rules" you mentioned in your post?

• I would love to see yarn getting the same support as npm, currently all alerts have the production scope, which renders this auto-dismissal feature unusable.
• I would love to programatically be able to add extra metadata to dependabot alerts. Currently, if Dependabot has successfully managed to raise a PR, I am able to leverage GH Actions to add metadata as tags on the associated PR, but I would love to do this at the alert level so I can also cover the alerts for which Dependabot was unable to raise a PR.

[lasselasse]

This is a great feature which would help us to reduce alert noise.

However, after I enabled the auto-dismiss feature for a private repo, I was surprised to discover after a few days that dependabot had auto-dismissed all open alerts (besides NPM also PIP dependencies which are not even supported according to the docs). I looked at a few alerts to see if the CWEs match the published low impact list but also failed to see a match there. Something clearly seems to be wrong here!

Regarding alert custom rules: It would be great to have the option to auto-dismiss alerts of certain severity levels (e.g. everything low or moderate), configurable per repository and ideally also being able to set an organization default.

[erinhav]

Fix going out now. Thanks for letting us know! ❤️

[lasselasse]

+1 All 3 are in scope. For organization defaults, would those be always on, or would you want some available to all repositories (but off by default, so not enforced initially)?

Great to hear! Any idea about an ETA? (I'm about to write a custom action to dismiss low-severity alerts and I'd gladly save myself that effort...)

Regarding settings: Ideally there would be an organizational default (off by default as you propose) that can then be overridden per repository.

[lasselasse]

Oh dear! That's not right. Have you opened a support ticket? You can also email me details at erinhav@github.com, so we can take a look.

Thanks, I'll get in touch by email for details as this concerns a private org repo.

[lasselasse]

The bugs of all open alerts having been auto-dismissed seems to be fixed.
As of today, there is only a single auto-dismissed alert in the repo in question, which is correctly classified (CWE-1333).
I haven't yet checked if there are other alerts that should have been auto-dismissed...

[Ameralameri]

Good

[Techiral]

Dependabot’s new auto-dismissal feature is a great addition to the tool. It helps to reduce the number of alerts that need immediate assessment, without compromising on any potentially concerning alerts. This feature is made possible by GitHub-curated vulnerability patterns, which take into account contextual information about how you’re using the dependency in order to substantially reduce the number of alerts that make it to your inbox. The initial release focuses on a subset of low impact development-scoped (devDependency) alerts for npm. If you’ve had a chance to try out the beta feature, GitHub would love to hear your feedback on what’s working for you, what needs to change, and which cases of vulnerabilities you’d like them to tackle next! Additionally, custom alert rules are also being introduced, allowing users to write their own logic in order to let Dependabot know which alerts should get a Dependabot pull request, and perhaps even which high-risk alerts should be auto-merged after passing built-in guardrails. This is an exciting development and I’m looking forward to seeing how it evolves! Is there anything else you would like to know? 😊

[henriquevcosta]

This is a highly desirable feature for us since dependabot has been generating a lot of noise with false positives and with it some friction with developers - however rolling this out on a repo by repo basis is problematic for us (few thousand repos) - what's the timeline to have this available as an org-level UI setting OR exposed at REST API level?

[erinhav]

We're actively developing org-level rules, which will ship within the quarter. Private beta will open next month, if you'd like to be a part of that!

[lasselasse]

✋ I'd like to join the private beta.

[PaulBernhardt]

I haven't seen how to turn this on, but from the blog post it sounds like we could maybe say certain sub dependencies, or dev vs prod, can be auto dismissed. I think that is not going to be enough. What I would like to be able to say is something like

If ALL of the root level dependencies this sub dependency was brought in by are on this list, AND the severity is not critical, dismiss it

So say my list of acceptable root dependencies is jest and eslint, and a few vulnerabilities come in:

Package A
Brought in by: jest, eslint
Severity: Medium
Action: Dismiss
A medium vulnerability that only impacts jest and eslint cannot possible influence production, close it

Package B
Brought in by: jest, webpack
Severity: Medium
Action: Keep Open
Webpack, though a dev dependency, could be including the vulnerable code in the built project. Not safe to auto dismiss

Package C
Brought in by: jest
Severity: Critical
Action: Keep Open
Maybe this is something like 'library compromised, it will email all your code to some bad guy', can't safely dismiss that.

Also, I would like to be able to set this up at the org level.

[erinhav]

That's great feedback. We're 🤏 this close to completing a major rearchitecture of our dependency graph, which will enable us to surface transitive dependency info in places like alert details and our rules engine.

While this dependency graph work is in flight, unfortunately I see the next step of exposing this data as still being months out (our supply chain team is small).

On the bright side, org-level custom rules and APIs are planned to ship this quarter. Any additional requests regarding those ships?

[lriho]

Could this functionality be enhanced for general dependency handling regardless of security implications? E.g. we keep getting dependabot alerts about latest version of @types/node (20.x.x) even though we're using Node v18 (LTS). So we would like to suppress alerts and have visibility on those suppressions. The current workflow of searching for "@dependabot ignore" in:comments leaves a lot to be desired. Also additional rules to dismiss alerts until minor or major versions are available would be handy.

[ansereb]

I would loke to have an OR operator in rule's criteria field. Something like:
cwe:400 or severity:low
Currently i have to create 2 rules: one with cwe:400 and other with severity:low
Also by some reason cwe:400,1333 is invalid construction and i have to write cwe:400 cwe:1333 .... It becomes very nasty strings with multiple CWE

[vtjballeng]

Dependabot keeps flagging older versions that are in my NuGet migration backups. We are on the current version but get these false positives from our backups. It's a nuisance warning.

[carogalvin]

OK, so to make sure I understand: you have a folder in your repository that represents a backup, and not the code that is being actively maintained used, so you would like to be able to tell Dependabot to ignore that folder for alerts and PRs, is that correct?

[vtjballeng]

@carogalvin , yes that is correct.

It would be more convenient if dependabot made this distinction on it's own. If it cannot then a manual directory exclusion is the next best option.

[carogalvin]

What would you expect Dependabot to use to tell the difference between a directory that it should and should not update, in this case?

[vtjballeng]

@carogalvin , it is certainly possible to recognize a VS repo and Migration Backup as an automatic exclusion as a user convenience. I think that methodology would help the most users seamlessly and prevent dependabot abandonment for the extra hassle from false positives it will inherently cause as-is.

Otherwise a manual user programable directory exclusion is the next best option.

[carogalvin]

Thanks for the feedback! If you have custom auto-triage rules available for your repository, you can automatically dismiss alerts for those manifests. We also have improvements to configuration for Dependabot updates on our roadmap for later this year, where we will be addressing things like telling Dependabot to ignore particular directories for updates.

[bordenit]

It doesn't seem to catch much in my Ruby or PHP based repositories. Would be good if it did more for those. But, for NuGet and .NET it works great.

[carogalvin]

@bordenit could you clarify what you mean by "it doesn't seem to catch much" - are you saying the auto-triage rules aren't dismissing when you expect it to, or that you are not receiving alerts that you are expecting?

[filips123]

I don't know if this is the right place to report this, but I have a problem that GitHub considers all dependencies as "runtime dependencies", even though they are only included as transitive dependencies of devDependencies and not used anywhere else. Because of this, I often receive Dependabot alerts for vulnerabilities in some transitive development dependencies that my project doesn't even use, even though Auto-triage rules are enabled. This happens in projects that use Yarn v1 and with normal package.json and yarn.lock files.

[carogalvin]

@filips123 thanks for your feedback! This is a limitation in our support for Yarn today, unfortunately

[filips123]

Thanks for the notice. Are there any plans to improve support for Yarn, and if so, are there any details approximately when it will be available? Also, is there any detailed explanation about which package managers (for JS and also other languages) are supported with which Dependentbot features?

[carogalvin]

We're investigating the level of effort required here but don't have a target date yet. You can see a list of supported package managers across Dependabot alerts here: https://docs.github.com/en/code-security/dependabot/dependabot-alerts/viewing-and-updating-dependabot-alerts