Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
revise security program for higher signal #789
We should get the 300th report this week, time to make statistics and suggestions!
Since the bounty started in June 2015, we got 12.4% of "Resolved" reports (reports leading to a commit or more in our codebase), 15.5% of "N/A" (unrelated to our infrastructure or code, third-party products... we don't use this category anymore), 34.6% of "Informative" (not a vulnerability issue, risk to low to be a really risk…) and… 37.5% of "Duplicates" (some of the informative ones are even in fact duplicates). The reports categorized as "low-quality" ("Informative" + "N/A", even if we should count the duplicates of already public reports) increased since your (very good) post about the creation of Gratipay's bounty program.
First point: I find this amount of duplicates incredibly high. It's definitely related to our high delay to properly resolve issues and to the lack of information given in response of the reports we refuse and make publicly available. We need to address this by trying to fix it as soon the report is triaged. Handling reports takes time: don't close any hazardously formulated but valid reports, try to do the same steps, validate the find or not, reply personally to the researcher… enough reasons to avoid getting duplicates. We'll win time, make quicker fixes, lead to less duplicates, etc.
We get a lot of copied and pasted reports related to "Best practices" that have been reported (and not always awarded…) elsewhere. This kind of "Informative" reports belong more to our Github queue than HackerOne.
We don't attract the right researchers: it's rare to get a report originating from somebody with a positive Signal value. This mechanism is already a good indicator of the serious of the researcher, in addition the followings points:
Some researchers are often trying to make "easy" money by sending a lot of (often low-quality) reports, "just in case, because it was awarded elsewhere", even if it's not applicable. It's not what we want to receive, but that's up to us give them the opportunity to improve their reports and help them expand their skill set by giving detailed answers and not sending generic messages.
My suggestions are the following:
Thoughts on theses (draft) ideas?
And from #782 (comment) ...