render enhancements

[JimBugwadia]

From: kyverno/policies#7 (comment)

Might be good to make this process more "go newbie" friendly such as:
Add step to chmod +x the render binary after go build process.
May need to change the output of the build process to be something other than render if executed in the root of the website dir since render already exists as a directory.
Link to instructions on how to install go(?)
When I run render on the current state of the policies repo, it's generating title and linkTitle with dashes and not with spaces (as the MD files currently have). weight is also different. Haven't checked the code.

[chipzoller]

May need a diffing ability to support policy YAML removals. Rendering is one way currently (YAML source => MD files), and when an MD is rendered, the source policy file is removed, the resulting MD has to be manually deleted. May be best to implement this as a flag like --clear which, when pointed at a dir with any files present, requires setting said flag in order to run. The target dir would then get cleared before any files are rendered, ensuring the policies and the MD are in sync after each run. If pointed to a dir with any files and --clear is not set, render exits printing a message to this effect.

[JimBugwadia]

@chipzoller - do we need anything at this point, or should we close this?

[chipzoller]

We could really use some fixes here including solving of the double slash and ability to render into a relative directory with ability to overwrite existing files. Currently the render program is a multi-step process when trying to refresh all the rendered policies.

[JimBugwadia]

Chip, can you please provide more details on each requirement? These do not seem difficult to do, but want to make sure I understand the asks correctly.

[chipzoller]

Yes, certainly.

1. When calling render, somewhere in the Go it produces double slashes within the Markdown files. This is also documented here. While this does not cause any noticeable problems, it isn't correct. For example, this is what is rendered under the Policy Definition header which provides the link back to the corresponding policy YAML file in kyverno/policies: <a href="https://github.com/kyverno/policies/raw/main//best-practices/disallow_cri_sock_mount/disallow_cri_sock_mount.yaml" target="-blank">/best-practices/disallow_cri_sock_mount/disallow_cri_sock_mount.yaml</a> . You can observe the double slash between main and best-practices in the a href tag.
2. When cloning kyverno/website locally, which means it will be cloned with the Markdown files for all the policies, and then running render, all the policies will be rendered to the working directory for render which is the /render directory. The render binary does not accept a location for these files in relation to the working directory, for example ../content/en/policies (which is where they all end up).
3. Further to number two above, the render program should be able to, if an optional flag is passed, overwrite files in the destination directory if present. The result of numbers two and three would be only a single-step process to render new policies whereas, presently, it's more than that as they need to be rendered, the destination directory cleared (optional), then moved, then committed.
4. When there's a failure encountered by render in the output, it can be tricky to spot because you have to scroll back through all the output (which gets lengthier the more policies we add) and ensure there were no errors thrown. Sometimes this has resulted in Markdown files being absent when such an error is encountered. It'd be very nice if render could collect any errors encountered and show them as the final output on a run (and exiting with a 1 for automation purposes) so it's clear what steps need to be taken.
5. YAML files appear to be parsed regardless of their names and if their spec is an array then render will print an error like failed to decode file \openshift\disallow-deprecated-apis\resources.yaml: json: cannot unmarshal array into Go struct field ClusterPolicy.spec of type v1.Spec
6. The rendering needs to be more selective on which YAML files it considers. We are now seeing issues with it rendering a YAML file of a Kasten K10 Policy resource which is not the same as a Kyverno Policy. Example here. For now, these require manual removal after the rendering process which is not ideal.

Hopefully this all makes sense.

[chipzoller]

Include #547

[chipzoller]

Double slashes still appear in output.

[JimBugwadia]

Please create an issue with the full command details.