New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

process: Add process functions #2

Merged
merged 11 commits into from Jan 14, 2018

Conversation

Projects
None yet
2 participants
@ashcrow
Member

ashcrow commented Jan 10, 2018

This includes downloading packages and submitting them to hashing
services.

PTAL @jasinner as this interfaces with the Java Hash Service.

process: Add process functions
This includes downloading packages and submitting them to hashing
services.

Signed-off-by: Steve Milner <smilner@redhat.com>

@ashcrow ashcrow requested a review from jasinner Jan 10, 2018

@jasinner

This comment has been minimized.

Show comment
Hide comment
@jasinner

jasinner Jan 11, 2018

Member

So far so good. I was expecting to see some git diffing going on with the victims-cve-db repository in order to determine the files to submit. Or does it act of a webhook from victims-cve-db?
Will there also be code for downloading from Maven?

Member

jasinner commented Jan 11, 2018

So far so good. I was expecting to see some git diffing going on with the victims-cve-db repository in order to determine the files to submit. Or does it act of a webhook from victims-cve-db?
Will there also be code for downloading from Maven?

@ashcrow

This comment has been minimized.

Show comment
Hide comment
@ashcrow

ashcrow Jan 11, 2018

Member

So far so good.

😸

I was expecting to see some git diffing going on with the victims-cve-db repository in order to determine the files to submit.

I have the start of looking at the changes in the code already but just haven't finished it.

Or does it act of a webhook from victims-cve-db?

When we merge a PR to the victims-cve-db GitHub will send a notification with the change to this bot. This bot will look to see if:

  1. The change was committed by someone other than itself
  2. The change adds a file to the cve db portion of the repo

If the above are both true then the bot:

  1. Clones the repository locally
  2. Looks at each added file in the merge related to the cve db
  • Downloads the artifact
  • Passes the artifact to the proper hashing service
  • Gets the results from the hashing service
  • Updates the cve db file itself with hashes
  • Commits the updated file
  1. Pushes the commits to the cve db

Clients will end up being updated (per previous discussions) to pull from the cve-db via git for updates rather than a remote service

Will there also be code for downloading from Maven?

With PURL we will eventually be able to translate from a maven URI to a download location. For now we'll want to use a URI such as http://central.maven.org/maven2/org/springframework/spring-core/5.0.2.RELEASE/spring-core-5.0.2.RELEASE.jar. In either case we will want to add a field to the cve-db. Maybe package-uri? We can then use it for http now but later also support PURL.

Validation would be done by the person merging the PR. It's up to them to review the pull to ensure the package url is the right one 😄.

Thoughts?

Member

ashcrow commented Jan 11, 2018

So far so good.

😸

I was expecting to see some git diffing going on with the victims-cve-db repository in order to determine the files to submit.

I have the start of looking at the changes in the code already but just haven't finished it.

Or does it act of a webhook from victims-cve-db?

When we merge a PR to the victims-cve-db GitHub will send a notification with the change to this bot. This bot will look to see if:

  1. The change was committed by someone other than itself
  2. The change adds a file to the cve db portion of the repo

If the above are both true then the bot:

  1. Clones the repository locally
  2. Looks at each added file in the merge related to the cve db
  • Downloads the artifact
  • Passes the artifact to the proper hashing service
  • Gets the results from the hashing service
  • Updates the cve db file itself with hashes
  • Commits the updated file
  1. Pushes the commits to the cve db

Clients will end up being updated (per previous discussions) to pull from the cve-db via git for updates rather than a remote service

Will there also be code for downloading from Maven?

With PURL we will eventually be able to translate from a maven URI to a download location. For now we'll want to use a URI such as http://central.maven.org/maven2/org/springframework/spring-core/5.0.2.RELEASE/spring-core-5.0.2.RELEASE.jar. In either case we will want to add a field to the cve-db. Maybe package-uri? We can then use it for http now but later also support PURL.

Validation would be done by the person merging the PR. It's up to them to review the pull to ensure the package url is the right one 😄.

Thoughts?

@ashcrow

This comment has been minimized.

Show comment
Hide comment
@ashcrow

ashcrow Jan 11, 2018

Member

And to keep backwards compat for a short period of time we can host the result of current API output as a static file at the same location for a few months.

Member

ashcrow commented Jan 11, 2018

And to keep backwards compat for a short period of time we can host the result of current API output as a static file at the same location for a few months.

ashcrow added some commits Jan 12, 2018

gh: Add CommitChange
Signed-off-by: Steve Milner <smilner@redhat.com>
web: Update Hook with more logic
Signed-off-by: Steve Milner <smilner@redhat.com>
gh: Add Push function
Signed-off-by: Steve Milner <stevem@gnulinux.net>
Add yaml lib and file editing operation
Signed-off-by: Steve Milner <stevem@gnulinux.net>
web: Update Hook with recent lib changes
Signed-off-by: Steve Milner <stevem@gnulinux.net>
gh: Add PingEvent type
Signed-off-by: Steve Milner <stevem@gnulinux.net>
web: Hook handles events with functions
Signed-off-by: Steve Milner <stevem@gnulinux.net>
@ashcrow

This comment has been minimized.

Show comment
Hide comment
@ashcrow

ashcrow Jan 13, 2018

Member

@jasinner recent pushes should show the logic flow better. The types will probably move into common in the near future but for now I'm keeping them within local packages.

Member

ashcrow commented Jan 13, 2018

@jasinner recent pushes should show the logic flow better. The types will probably move into common in the near future but for now I'm keeping them within local packages.

ashcrow added some commits Jan 13, 2018

utils: Add more flags and env support
Signed-off-by: Steve Milner <stevem@gnulinux.net>
web: Use new config items and only push if a commit was made
Signed-off-by: Steve Milner <stevem@gnulinux.net>
README.md: Add configuration info
Signed-off-by: Steve Milner <stevem@gnulinux.net>
@jasinner

This comment has been minimized.

Show comment
Hide comment
@jasinner

jasinner Jan 14, 2018

Member

Looks great

Member

jasinner commented Jan 14, 2018

Looks great

@jasinner jasinner merged commit bf079e2 into victims:master Jan 14, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment