Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added new security scanner: nuclei #605

Merged
merged 21 commits into from
Aug 27, 2021
Merged

Added new security scanner: nuclei #605

merged 21 commits into from
Aug 27, 2021

Conversation

rseedorff
Copy link
Member

Description

This PR if applied integrates a new security scanner with the secureCodeBox: nuclei

Checklist

  • Test your changes as thoroughly as possible before you commit them. Preferably, automate your test by unit/integration tests.
  • Make sure npm test runs for the whole project.
  • Make codeclimate checks happy

rseedorff and others added 3 commits August 21, 2021 00:34
@rseedorff
initial integration of the nuclei scanner
87d5804 
Signed-off-by: Robert Seedorff <Robert.Seedorff@iteratec.com>
@rseedorff @secureCodeBoxBot
Updating Helm Docs
46c2555 
Signed-off-by: GitHub Actions <securecodebox@iteratec.com>
@rseedorff
Added nuclei to ci pipeline
8476318 
Signed-off-by: Robert Seedorff <Robert.Seedorff@iteratec.com>
@rseedorff rseedorff added the scanner Implement or update a security scanner label Aug 20, 2021
@rseedorff rseedorff added this to the v3.1.0 milestone Aug 20, 2021
@rseedorff rseedorff added this to In progress in secureCodeBox v3 via automation Aug 20, 2021
@rseedorff rseedorff marked this pull request as draft August 20, 2021 22:40
rseedorff and others added 8 commits August 21, 2021 01:06
@rseedorff
Fixed scanner and parser issues
5217246 
Signed-off-by: Robert Seedorff <Robert.Seedorff@iteratec.com>
@rseedorff @secureCodeBoxBot
Updating Helm Docs
4c28a24 
Signed-off-by: GitHub Actions <securecodebox@iteratec.com>
@rseedorff
cleanup stuff
173b572 
Signed-off-by: Robert Seedorff <Robert.Seedorff@iteratec.com>
@rseedorff
Removing dependency
9cdc615 
to implement the function by ourself

Signed-off-by: Robert Seedorff <Robert.Seedorff@iteratec.com>
@rseedorff
Fixed Docker image
046d79e 
Signed-off-by: Robert Seedorff <Robert.Seedorff@iteratec.com>
@rseedorff
Fixed DefectDojo severity Mapping
d5379af
@rseedorff
Added a persistentVolume to cache
0133857 
the nuclei-templates for all pods

Signed-off-by: Robert Seedorff <Robert.Seedorff@iteratec.com>
@rseedorff
Updating Helm Docs
48031e4 
Signed-off-by: GitHub Actions <securecodebox@iteratec.com>
Signed-off-by: Robert Seedorff <Robert.Seedorff@iteratec.com>
rseedorff and others added 3 commits August 21, 2021 14:30
@rseedorff
Fixed unit tests
5c080a5 
Signed-off-by: Robert Seedorff <Robert.Seedorff@iteratec.com>
@rseedorff
Added activeDeadlineSeconds as
2d6a3ca 
configuration option

Signed-off-by: Robert Seedorff <Robert.Seedorff@iteratec.com>
@rseedorff @secureCodeBoxBot
Updating Helm Docs
ec6bc24 
Signed-off-by: GitHub Actions <securecodebox@iteratec.com>
@rseedorff rseedorff marked this pull request as ready for review August 22, 2021 20:18
SebieF
SebieF requested changes Aug 24, 2021
View reviewed changes
Copy link
Contributor

@SebieF SebieF left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nicely done :) Would suggest to add integration tests, however.

scanners/nuclei/parser/parser.js Outdated Show resolved Hide resolved
scanners/nuclei/parser/parser.test.js Outdated Show resolved Hide resolved
secureCodeBox v3 automation moved this from In progress to To Review Aug 24, 2021
@rseedorff rseedorff self-assigned this Aug 24, 2021
@rseedorff
Rework after Code Review
736d0bc 
Signed-off-by: Robert Seedorff <Robert.Seedorff@iteratec.com>
SebieF
SebieF previously approved these changes Aug 25, 2021
View reviewed changes
secureCodeBox v3 automation moved this from To Review to Reviewer approved Aug 25, 2021
@SebieF SebieF self-requested a review August 25, 2021 09:13
@SebieF
Copy link
Contributor

SebieF commented Aug 25, 2021 •
edited

When I try to install the scanner via helm, I get the following error:
helm upgrade --install nuclei scanners/nuclei/

Release "nuclei" does not exist. Installing it now. Error: ScanType.execution.securecodebox.io "nuclei" is invalid: spec.jobTemplate.spec.activeDeadlineSeconds: Invalid value: "null": spec.jobTemplate.spec.activeDeadlineSeconds in body must be of type integer: "null"

Seems like the value for activeDeadlineSeconds must be set manually before the scanner can be installed. I guess this behavior is not intended?

@J12934
Copy link
Member

J12934 commented Aug 25, 2021

Wouldn't the cache volume currently block multiple nuclei scanners from running at the same time?
To my understanding ReadOnlyMany can only be mounted multiple times when the volume mount is marked as readOnly which doesn't seem to be the case.

And should the cache volume be filled by something like a CronJob which periodically refreshes the cache and the scans having this volume mounted as readOnly?

Also should this be turned on by default, as many storage k8s environments don't support ReadOnlyMany: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes

@rseedorff
Copy link
Member Author

@J12934 Yes you are right, in practice this happens sometimes, the nuclei scan then will be executed in a more serialized way, but is is not always the case 🤷🏻‍♂️

I already tried to find another solution at the nuclei project side, but this will take some time to finish: projectdiscovery/nuclei#895

@rseedorff
Introduced a correct PCV with an dedicated
d934b3c 
update Job to ensure always up-to-date nuclei templates
without hitting the GH ratelimit

Signed-off-by: Robert Seedorff <Robert.Seedorff@iteratec.com>
secureCodeBox v3 automation moved this from Reviewer approved to To Review Aug 27, 2021
@rseedorff @secureCodeBoxBot
Updating Helm Docs
2a11ec7 
Signed-off-by: GitHub Actions <securecodebox@iteratec.com>
@rseedorff rseedorff requested a review from J12934 August 27, 2021 09:54
@J12934
Update apiVersion to current batch version
0807a8e 
Signed-off-by: Jannik Hollenbach <jannik.hollenbach@iteratec.com>
@J12934
Fix typo
b3ac061 
Signed-off-by: Jannik Hollenbach <jannik.hollenbach@iteratec.com>
@J12934
Convert ternary to switch statement
0ac6a8c 
Signed-off-by: Jannik Hollenbach <jannik.hollenbach@iteratec.com>
@J12934
Add job to populate the cache directly after install
4ebcd58 
The cache would otherwise be empty untill the first cronjob gets triggered.
Untill then every scan would fail.

Signed-off-by: Jannik Hollenbach <jannik.hollenbach@iteratec.com>
J12934
J12934 approved these changes Aug 27, 2021
View reviewed changes
Copy link
Member

@J12934 J12934 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pushed some small changes / fixes.
PR is ready to merge now IMO 👍

secureCodeBox v3 automation moved this from To Review to Reviewer approved Aug 27, 2021
@rseedorff rseedorff merged commit d6e5522 into main Aug 27, 2021
@rseedorff rseedorff deleted the feature/add-scanner-nuclei branch August 27, 2021 20:03
secureCodeBox v3 automation moved this from Reviewer approved to Done Aug 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@J12934 J12934 J12934 approved these changes

@SebieF SebieF Awaiting requested review from SebieF

Assignees

@rseedorff rseedorff

Labels
scanner Implement or update a security scanner
Projects
No open projects
secureCodeBox v3
  
Done
Milestone
v3.1.0
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@rseedorff @SebieF @J12934