Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce a script to compare ComplianceAsCode versions #10768

Merged
merged 6 commits into from
Jul 3, 2023

Conversation

jan-cerny
Copy link
Collaborator

Description:

This PR is blocked by #10761 and will be rebased after it will be merged.

This commit introduces a new script compare_versions.py which can show differences between two ComplianceAsCode versions. Lists added or removed rules, profiles, changes in profile composition and changes in remediations and platforms. For comparison, you can use git tags or ComplianceAsCode JSON manifest files directly. This new feature leverages the ComplianceAsCode JSON manifests, which have been introduced by #10761

Rationale:

Helps track changes between content and product.

Review Hints:

To compare 2 ComplianceAsCode JSON manifests, provide the manifest files.

python3 utils/compare_versions.py compare_manifests ~/manifests/old.json ~/manifests/new.json

To compare 2 upstream versions, you need to specify the version git tags and a product ID.

$ python3 utils/compare_versions.py compare_tags v0.1.67 v0.1.68 rhel9

@jan-cerny jan-cerny added enhancement General enhancements to the project. Highlight This PR/Issue should make it to the featured changelog. good first issue Good candidates to get started contributing to CaC. labels Jun 29, 2023
@jan-cerny jan-cerny added this to the 0.1.69 milestone Jun 29, 2023
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Used by openshift-ci bot. label Jun 29, 2023
@openshift-ci
Copy link

openshift-ci bot commented Jun 29, 2023

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@github-actions
Copy link

Start a new ephemeral environment with changes proposed in this pull request:

Fedora Environment
Open in Gitpod

Oracle Linux 8 Environment
Open in Gitpod

@jan-cerny jan-cerny removed the good first issue Good candidates to get started contributing to CaC. label Jun 29, 2023
@matejak
Copy link
Member

matejak commented Jun 30, 2023

I suggest decoupling the difference representation and the text interpretation of thereof. That way, not only will it be possible to test the code, but tweaks to the output won't interfere with the code doing other things.
I am thinking of a generic approach that would e.g. given the manifest and key rules would create a dict-like diff structure with e.g. rules:added, rules:removed, and rules:stable keys. Perhaps applying this somehow recursively would largely do the trick of producing a diff structure?

@jan-cerny jan-cerny marked this pull request as ready for review June 30, 2023 13:58
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Used by openshift-ci bot. label Jun 30, 2023
@jan-cerny
Copy link
Collaborator Author

I have rebased this PR on the top of the latest upstream master branch.
I have slightly updated the logic to little split the logic from printing, but it's far away from what you wanted. The question is is it worth doing now or do we want to wait for the manifest adoption.

This commit introduces a new script compare_versions.py which can show
differences between two ComplianceAsCode versions.  Lists added or
removed rules, profiles, changes in profile composition and changes in
remediations and platforms.  For comparison, you can use git tags or
ComplianceAsCode JSON manifest files directly.  This new feature
leverages the ComplianceAsCode JSON manifests, which have been
introduced by ComplianceAsCode#10761
By refactoring the code into smaller methods we try to reply to
CodeClimate warnings.
@jan-cerny
Copy link
Collaborator Author

I have resolved remaining CodeClimate problem and I have rebased this on the top of the latest upstream master branch.

@codeclimate
Copy link

codeclimate bot commented Jul 3, 2023

Code Climate has analyzed commit 59d7280 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 53.5% (0.0% change).

View more on Code Climate.

Copy link
Member

@matejak matejak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the improvements.
The script works, and is able to demonstrate the potential of manifests. If the concept gains serious traction, it can be improved accordingly, but expending excess resources in this direction wouldn't be smart.

@matejak matejak merged commit 9d16f42 into ComplianceAsCode:master Jul 3, 2023
33 of 34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement General enhancements to the project. Highlight This PR/Issue should make it to the featured changelog.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants