Add intel module to sync CVE data from NIST #794
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This is very interesting. Aside, the lack of a cleanup job is concerning, and we'll need to carefully consider the possibility of the graph growing too large too quickly. |
|
Well, for this there's an upper bound on growth. Nodes are only added if
there's new CVEs. Also, there's no cleanup because CVEs are never deleted
from the source.
…On Thu, Apr 7, 2022, 1:49 AM Ramon Petgrave ***@***.***> wrote:
This is very interesting.
Aside, the lack of a cleanup job is concerning, and we'll need to
carefully consider the possibility of the graph growing too large too
quickly.
—
Reply to this email directly, view it on GitHub
<#794 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AALXSMUVCLPQ3BEMHX4XP3TVDW6ARANCNFSM5SHNLEFQ>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
|
Not stale |
achantavy
reviewed
Jun 2, 2022
|
@achantavy done! |
achantavy
approved these changes
Jun 4, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change adds a cve intel module, which syncs CVE data from the NIST json feeds, using the v4 json format feeds.
This module is a bit special, since it doesn't have a cleanup job, and it introspects the graph to determine what it needs to sync.
CVE data is generally historical and isn't deleted. The data is updated, but is updated through a feed. Historical data is grouped by year, and updates are available in special feeds that are updated daily, and contain updates for the past 8 days. The yearly data is also updated daily, but assuming the yearly data has been synced, it's only necessary to pull in the recent and modified feeds after that point.
The module syncs each year, then syncs the recent and historical data. For each type (year, recent, modified), it records the sync via SyncMetadata. Prior to syncing the yearly data, the module checks to see if the yearly data has been synced by introspecting the graph for the SyncMetadata for the yearly data. It only syncs years that have not yet been recorded. This allows the module to normally only sync the modified and recent feeds.