Proposal: extracting vulnerable functions from CodeFix data (step 1 of the reachability RFC #1313)

[kwado-tech]

Hi all,

I'd like to help with the first step of the vulnerability reachability RFC (aboutcode-org/vulnerablecode#1313): finding which functions and methods were vulnerable, using the fix commits VulnerableCode already collects in CodeFix. I commented on the issue a couple of weeks ago and wanted to bring it here for wider visibility.

What I have in mind:

• An improver pipeline that diffs the pre-fix and fixed code with tree-sitter and maps changed lines to qualified function and method names, starting with PyPI, npm with Maven as the next ecosystem (I've narrowed the first phase since my comment on #1313).
• A confidence score per symbol, with low-confidence results routed to curation.
• Validation against the Go vulnerability database, which already records vulnerable symbols, plus hand-labelled samples per ecosystem.
• The resulting data published openly, with the licence agreed with you.

This data could also feed DejaCode's planned reachability integration (aboutcode-org/dejacode#103).

A few questions:

1. Is anyone already working on this, within AboutCode, CodeSupply or with AppThreat's atom work?
2. Would a small data-model addition to store affected symbols per advisory and package be welcome? Any preferred shape?
3. Given that VulnerableCode's data is CC-BY-SA-4.0, what licence would you expect for derived symbol data?

For transparency: I run Toevan, a small company in Estonia, and I'm planning to apply to the CodeSupply open call to fund this work. I'd rather build it to fit VulnerableCode than alongside it.

cc @pombredanne @ziadhany , and @DennisClark in case this is useful for DejaCode.

Thanks!
Christian