New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds configurable batch insert for References and Vulnerabilities #1014

Merged
merged 5 commits into from Dec 10, 2017

Conversation

Projects
None yet
2 participants
@alefq
Copy link
Contributor

alefq commented Nov 30, 2017

Fixes Issue

Makes use of batch capabilities of PreparedStatement. Executes updates every n rows (configurable), slowing down time for vulnerabilities updates by a near 10x factor on external databases (Postgresql for instance).
Tested on H2 (default), and Postgresql.

Description of Change

Applies batch inserts for reference and vulnerability tables, solves
slow one-by-one insert process, for Vulnerabilities with several
references/vulnerabilities associated.
Feature is configurable through properties: database.batchinsert.enabled
and database.batchinsert.maxsize.

Have test cases been added to cover the new functionality?

no

Tests available already cover the process. All tests where executed before pushing code to forked repository

Ale Feltes added some commits Nov 17, 2017

Ale Feltes
Added Batch insert for References and Vulnerabilities
Applies batch inserts for reference and vulnerability tables,
solves the slow one-by-one insert process, for Vulnerabilities
with several references/vulnerabilities associated.
Ale Feltes
Adds configurable batch insert for References and Vulnerabilities
Applies batch inserts for reference and vulnerability tables, solves
slow one-by-one insert process, for Vulnerabilities with several
references/vulnerabilities associated.
Feature is configurable through properties: database.batchinsert.enabled
and database.batchinsert.maxsize.
Ale Feltes
Restore CVE start year
cve.startyear value is 2002.
@jeremylong

This comment has been minimized.

Copy link
Owner

jeremylong commented Nov 30, 2017

Can one of the admins verify this patch?

Update CveDB.java
Remove unused variable.
@jeremylong

This comment has been minimized.

Copy link
Owner

jeremylong commented Dec 4, 2017

Thanks! It will likely be next weekend before I can take a look at this update.

@jeremylong jeremylong merged commit b46d13f into jeremylong:master Dec 10, 2017

3 checks passed

Codacy/PR Quality Review Good work! A positive pull request.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
lgtm analysis: Java No alert changes
Details
@jeremylong

This comment has been minimized.

Copy link
Owner

jeremylong commented Dec 10, 2017

Thanks for the PR - I've done some testing on mysql and the earlier issues identified with batching look like they have been resolved via other changes to the tool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment