Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve Makefiles for generating scanner documentation #950

Merged
merged 24 commits into from
Feb 3, 2022
Merged

Improve Makefiles for generating scanner documentation #950

merged 24 commits into from
Feb 3, 2022

Conversation

Weltraumschaf
Copy link
Member

No description provided.

@Weltraumschaf
#754 Add some vars w/ locations
6b6ce7c 
It is easier to maintain the location of directories at one
place.

Also use built in functions to find files w/ glob patterns
instead of convoluted shell script magic.

Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Remove .ONESHELL from scanner-docs target
5451704 
Since macOS and maybe BSD does not support .ONESHELL for
make targets we must removethis. To make the shell commands
work ;\ must be appended to each line, except block open
keywords such as 'then' or 'else'.

This path also utilizes the SCANNERS_CHART_LIST variable
instead of the find.

Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Break long lines for better readability
0e275ec 
Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Use variable to point to thehelm docs templates
c82f909 
Jumping around with relative dirs is very brittle. Better pattern
is to define a variable with absolute location and references this.

Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Extratc variable for docs dir
3e472d9 
Also use absolute path to be more stable for refactorings.

Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Extratc variable for parser dir
b30dd41 
Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Extratc variable for scanner dir
e434eba 
Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Change dir directly at necessary place
489ff5d 
Forunknown reason helm-docs need to be in the scanner dir. Maybe
it assumes some files relative to the Chart.yaml

Until I discovered this issue using the cd as workaround.

Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Extract thew hole generation code into a shell script
21148e1 
Because it is shell script anyway and editing shell script in
Makefiles is tedious I decided to extract the code into a separate
shell script. So we can invoke this from the make targets.

AFAICS there is lots of duplicatedcode for hooks and parser. Seems
to be nearly the same as for the scanners. So we may reuse thisscript.

This will elad into a cleaner Makefile and less code to maintain.

Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf Weltraumschaf added the enhancement New feature or request label Feb 1, 2022
@Weltraumschaf Weltraumschaf self-assigned this Feb 1, 2022
@Weltraumschaf Weltraumschaf added this to Backlog in secureCodeBox v3 via automation Feb 1, 2022
@Weltraumschaf Weltraumschaf marked this pull request as draft February 1, 2022 20:42
@Weltraumschaf Weltraumschaf moved this from Backlog to In progress in secureCodeBox v3 Feb 1, 2022
@Weltraumschaf Weltraumschaf linked an issue Feb 1, 2022 that may be closed by this pull request
make scanner-docs / hook-docs fails on macOS #754
Closed
@Weltraumschaf Weltraumschaf marked this pull request as ready for review February 1, 2022 20:46
@Weltraumschaf Weltraumschaf moved this from In progress to To Review in secureCodeBox v3 Feb 1, 2022
@malexmave
Copy link
Member

malexmave commented Feb 2, 2022 •
edited

Does not seem to work for me...

Test process:

  • Check out branch
  • Make change to scanners/gitleaks/.helm-docs.gotmpl
  • In root directory of repo, run make scanner-docs
  • Process terminates without showing an error code, but READMEs are not updated to reflect the change in the template.

Relevant log snippet:

Parser found at: /Users/mmaass/Documents/Code/SecureCodeBox/scanners/gitleaks/parser
INFO[2022-02-02T08:14:04+01:00] Found Chart directories [.]                  
INFO[2022-02-02T08:14:04+01:00] Generating README Documentation for chart /Users/mmaass/Documents/Code/SecureCodeBox/scanners/gitleaks 
WARN[2022-02-02T08:14:04+01:00] Could not open chart README file /Users/mmaass/Documents/Code/SecureCodeBox/scanners/gitleaks/README.md, skipping chart 
Scanner found at: /Users/mmaass/Documents/Code/SecureCodeBox/scanners/gitleaks/scanner
INFO[2022-02-02T08:14:04+01:00] Found Chart directories [.]                  
INFO[2022-02-02T08:14:04+01:00] Generating README Documentation for chart /Users/mmaass/Documents/Code/SecureCodeBox/scanners/gitleaks 
WARN[2022-02-02T08:14:04+01:00] Could not open chart README file /Users/mmaass/Documents/Code/SecureCodeBox/scanners/gitleaks/README.md, skipping chart 
INFO[2022-02-02T08:14:04+01:00] Found Chart directories [.]                  
INFO[2022-02-02T08:14:04+01:00] Generating README Documentation for chart /Users/mmaass/Documents/Code/SecureCodeBox/scanners/gitleaks 
WARN[2022-02-02T08:14:04+01:00] Could not open chart README file /Users/mmaass/Documents/Code/SecureCodeBox/scanners/gitleaks/README.md, skipping chart

Let me know if I can help debug this in any way.

@Weltraumschaf
Copy link
Member Author

ACK. I thought this was a general warning, but I cross checked on a Debian machine: No such warnings. I have no clue why it occurs because the printed filename is accessible and I have no idea why helm-docs can't open it.

@malexmave
Copy link
Member

Maybe @J12934 has an idea what is going on there?

@Weltraumschaf
Copy link
Member Author

@J12934 Also something I don't understand: Why is it necessary forheln-docs to be executed inside the scanner directory, when I reference all given template files?

@Weltraumschaf Weltraumschaf force-pushed the bugfix/issue_754 branch 2 times, most recently from 110854c to 4531c4d Compare February 2, 2022 11:04
@Weltraumschaf
Copy link
Member Author

Loglevel trace ofhelm-docs is not helpfull.

@Weltraumschaf
Copy link
Member Author

@malexmave Fixed! Would you please test on your machine?

@malexmave
Copy link
Member

malexmave commented Feb 2, 2022 •
edited

Works on my machine now 🥳 . However, only for scanner-docs, not for hook-docs.

@Weltraumschaf
#754 Use the doc generation script in the Makefile
d05fc03 
Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Use helm docs dir from variable
2841dfd 
Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Add missing help messages for make targets
18734d5 
Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Early return if docs dir is missing
bbb0ab5 
Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Use less generic variable name for the main scanner dir
800c1f2 
Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Suround var output with quotes so we can see accidentally empty …
99dc17c 
…values

Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Fix unbound variable error
c527704 
Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Use dedicated log function to prepend an eyecatcher
9cae59c 
Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Remove unneccessary log output
5e67257 
We log this anyway in the called shell script anddoes not need
the info twice.

Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Improve the log output
7861ba9 
Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Add chart search root as parameter
6ed965e 
So we do not need to cd around.

Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Remove docs_dir from helm-doc output-file option
8438965 
helm-docs expets only a filename which is relative to the found
Chart.yaml. So we simply add the 'docs' part.

Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf
#754 Use a better name for the scanner image dir variable
c257b87 
Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf Weltraumschaf force-pushed the bugfix/issue_754 branch 2 times, most recently from f569c41 to f255186 Compare February 2, 2022 13:29
@Weltraumschaf
Use latest helm-docs binary
f7e61f7 
Signed-off-by: Sven Strittmatter <sven.strittmatter@iteratec.com>
@Weltraumschaf @secureCodeBoxBot
Updating Helm Docs
dd6b0f4 
Signed-off-by: GitHub Actions <securecodebox@iteratec.com>
secureCodeBox v3 automation moved this from To Review to Reviewer approved Feb 2, 2022
malexmave
malexmave approved these changes Feb 2, 2022
View reviewed changes
Copy link
Member

@malexmave malexmave left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me, and results are now also consistent with those obtained by the CI job.

@Weltraumschaf
Copy link
Member Author

Can you merge? You should have the rights.

@malexmave malexmave changed the title Bugfix/issue 754 Improve Makefiles for generating scanner documentation Feb 3, 2022
@malexmave malexmave merged commit 644adba into main Feb 3, 2022
secureCodeBox v3 automation moved this from Reviewer approved to Done Feb 3, 2022
@malexmave malexmave deleted the bugfix/issue_754 branch February 3, 2022 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@malexmave malexmave malexmave approved these changes

Assignees

@Weltraumschaf Weltraumschaf

Labels
enhancement New feature or request
Projects
No open projects
secureCodeBox v3
  
Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

make scanner-docs / hook-docs fails on macOS
2 participants
@Weltraumschaf @malexmave