refactor(misconf): Refactor defsec components

[simar7]

As part of refactoring and breaking out defsec, we'd like to break out defsec into smaller components. A high level overview of this would like the following

This will allow for the following benefits:

1. Smaller repos: Today defsec has grown quite large and sometimes is not easy to manage.
2. Decoupling content from engine: trivy-rules will be a repo of its own to manage policies (content)
3. Frequent content releases: We can release trivy-rules more often than defsec releases to ensure newer bundles with newer policies are available sooner.
4. Smaller dependency footprint: Today defsec brings in a lot of dependencies as a result of being an amalgamation of the three above but upon refactoring, only the required dependencies will be needed by the respective clients.

[simar7]

A PoC for trivy-rules is here https://github.com/simar7/trivy-misconf-rules

[simar7]

A PoC for trivy-cloud is here https://github.com/nikpivkin/trivy-cloud

[nikpivkin]

POC for trivy-iac https://github.com/nikpivkin/trivy-iac

[nikpivkin]

@simar7 testing it with Trivy.
Some of the tests need to be moved to trivy-misconf-rules:

• rego_test.go
• bundle_test.go
• loader.go

[simar7]

@simar7 testing it with Trivy. Some of the tests need to be moved to trivy-misconf-rules:

• rego_test.go
• bundle_test.go
• loader.go

Added aquasecurity/trivy-checks@405b831

[nikpivkin]

@simar7 setup_test.go uses parsers and scanners, and this test should be in trivy-iac. I think it's worth deleting it and adding the rego_test.go test.

[simar7]

@simar7 setup_test.go uses parsers and scanners, and this test should be in trivy-iac. I think it's worth deleting it and adding the rego_test.go test.

I merged the PR https://github.com/simar7/trivy-policies and renamed the repo to trivy-policies

[simar7]

Updated refactor overview

[simar7]

Updated overview

Component lifecycle timelines

Key to note is that when we download policies for services that Trivy does not know about just yet, we handle that gracefully.

[knqyf263]

Thanks for the diagram. Looks good! I have two questions.

1. Release cycle of trivy-iac
   Trivy is planning to change the release cycle. Then, trivy-iac needs to sync with that. It can be a bit complex. Is it ok?
   docs: add release cycle #5201

2. Unsupported policies
   In the second diagram, Trivy fetches a new bundle, including some policies not yet supported. Doesn't it fail to scan? It is ok if Trivy can just ignore those policies.
   As you replied here, I understand it will handle them gracefully. Please correct me if I'm wrong.

[simar7]

Thanks for the diagram. Looks good! I have two questions.

1. Release cycle of trivy-iac
   Trivy is planning to change the release cycle. Then, trivy-iac needs to sync with that. It can be a bit complex. Is it ok?
   docs: add release cycle #5201

Yes trivy-iac and trivy will have the same release cycle. It will be similar to how defsec and trivy work today for release cycles. trivy-iac will have a release prior to a trivy release so that all the fixes for iac scanning go into the next release of Trivy.

As for trivy-policies and trivy-aws they are not subject to the same release cycle at the moment so expect more frequent releases. Rest assured, they will be updated prior to the next Trivy release.

2. Unsupported policies
   In the second diagram, Trivy fetches a new bundle, including some policies not yet supported. Doesn't it fail to scan? It is ok if Trivy can just ignore those policies.
   As you replied here, I understand it will handle them gracefully. Please correct me if I'm wrong.

Yes that's right. Trivy today already has the logic to ignore policies that are either syntactically broken or are for services that that particular version of Trivy does not support today. By graceful handling, it means that those policies are simply ignored and not compiled (and loaded) in memory when loading OPA upon Trivy startup.

If the user decides to upgrade the version of Trivy, it will add support for newer services. This will allow the new version of Trivy to start using these policies for newly supported services as the bundle will still be present on disk from the old version, if still under the cache expiration limit. If not, Trivy will download a fresh policy bundle as expected.

[knqyf263]

Thanks for explaining! It sounds great.