Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Release 32.0.0rc1 prep #3150

Merged
merged 90 commits into from
Jan 4, 2023
Merged

Release 32.0.0rc1 prep #3150

merged 90 commits into from
Jan 4, 2023

Conversation

AyanSinhaMahapatra
Copy link
Member

Tasks

  • Reviewed contribution guidelines
  • PR is descriptively titled 📑 and links the original issue above 🔗
  • Tests pass -- look for a green checkbox ✔️ a few minutes after opening your PR
    Run tests locally to check for errors.
  • Commits are in uniquely-named feature branch and has no merge conflicts 📁

@AyanSinhaMahapatra
Remove deprecated images from CI and release-script
5ada2dd 
The following images are deprecated in GitHub actions and Azure DevOps:

* `ubuntu-18.04` : actions/runner-images#6002
* `macos-10.15` : actions/runner-images#5583

Due to this there was failing tests due to planned brownouts.

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Bump versions to 32.0.0b1
b3aaa47 
Also update spdx license list version as it was updated previously.

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Add license detection rules
27b15fc 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Fix test failures
969f5cf 
- Update test fails due to spdx license list update
- Update test fails due to inconsistent rule data

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Copy link
Member Author

@pombredanne I've fixed the test failures, but let's keep this open as discussed for the two changes i.e. top level unique license detections and license references as default, I'll open PRs for those into this branch/merge those onto develop and then merge this just before pushing the 32.0.0b1 tag?

@AyanSinhaMahapatra
Fix license/rules dump format
e5d3a19 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Merge branch 'develop' into release-32-rc1-prep
fd60bb8 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Reformat licenses with a load/dump
7cbdb17 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Reformat rules with a load/dump
279732f 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Copy link
Member Author

@DennisClark I would need your review on the following commit: 088e106 i.e. license additions in scancode-toolkit from spdx license list 3.19

@DennisClark
Copy link
Member

@AyanSinhaMahapatra you are right, thanks. the spdx x11vnc license is the very same as our openssl-exception-gpl-2.0 if you follow the SPDX matching guidelines. So we need to update the SPDX identifier of our license with x11vnc-openssl-exception instead of the current licenseref value and not create a new license.

@AyanSinhaMahapatra
Add license updates from spdx license list 3.19
84186d0 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Vendor python-forntmatter completely
162cffb 
We used to vendor parts of python frontmatter and also use it as
a dependency which was problematic. This vendors python-frontmatter
fully and only uses the code needed.

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra AyanSinhaMahapatra force-pushed the release-32-rc1-prep branch 2 times, most recently from bca071f to d776a10 Compare December 15, 2022 15:33
@AyanSinhaMahapatra
Update synclic script and misc test expectations
462d0e6 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Add new license category "CLA" #3147 #3038
1ea6303 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Deprecate licenses and rules
826b261 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Add new and updated licenses and rules
6921c9d 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Fix test failures
d0ebced 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Merge branch 'license-detection-follow-up' into release-32-rc1-prep
755f28b 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Update debian and spdx output plugins
15318be 
* fix bug in debian output plugin
* update plugins to not fetch matched_text from reference data

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Add license dump CLI option back
e9886d4 
* the license dump CLI option was removed by mistake in a merge
  conflict resolution, this adds it back.

Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@AyanSinhaMahapatra
Add new docstrings and update old docstrings
2b67d0b 
Signed-off-by: Ayan Sinha Mahapatra <ayansmahapatra@gmail.com>
@pombredanne
Add support for FIPS
ae4c63f 
Use sha1 correctly for UUID seed

Reported-by: RayGozer @RayGozer
Reference: #3165
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Regen test with new rule identifier
021ad24 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Update tests expectations
d2b4fc7 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Apply cosmetic refactoring
c0590ca 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Improve Rule check with is_synthetic check
ac4f7dc 
Also move test utilities to test module

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Use correct imports in tests
080e163 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Avoit using copyright as a varibale name
83a0db7 
Prefer copyrght to avoid name collision with the built-in
copyright

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Rename and streamline get_rule_object_from_match
8ee4406 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne pombredanne mentioned this pull request Jan 2, 2023
Open
@pombredanne
Make license reference its own plugin
ccc79a4 
It kicks in if license or package are requested as a post-scan plugin

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Remove deprecated is_license_text plugin
d7c682d 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Use correct planned release date
e6fca98 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Improve license and rule references handling
d292aac 
Streamline code now that we do not deal with references in the main
detection.

Improve models accordingly

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Do not include SPDX expression in low level matches
ccef6f8 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Update test expectations for RPMs
f27e61c 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Update test expectations
0298a5c 
Align with latest fixesa nd the move of license details to
license references

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Do not keep is_deprecated License flag
edaef02 
No deprecated license can participate in detection.

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Enable LicenseReference plugin correctly
b3acde0 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Update test expectations
857962c 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Merge upstream branch update-ci-images
299790f 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Generate correct Rule URL
88b5144 
Additional rules and synthetic rules have no URL for now.
The logic to create such URL is now under Rule.rule_url

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Use plain JSON test file
2b64449 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Copy link
Member

@AyanSinhaMahapatra this is ready for your review. The key changes are:

  • the license references are now always created with a new separate post scan plugin. This plugin is not commanded by its own command line option. Instead it is enabled if either the license or package option is requested. Because of this I streamlined or removed quite a bit of license reference-related code.
  • I updated which fields are included in the license and rule references to include many more fields.
  • the package license detection do not contain any license/rule references anymore, this is now only top level.
  • I removed the deprecated "is_license_text" plugin
  • I replaced using md5 with sha1 for Remove MD5 use to allow FIPS compliance. #3165 and eventual future FIPS support
  • I applied cosmetic refactoring and imports cleanup across the codebase

@pombredanne
Correct license mappings for Alpine
2548f43 
We were missing out certain openssl exception

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Generate correct URLs for license and rules
676a471 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne pombredanne mentioned this pull request Jan 4, 2023
Closed
@pombredanne
Use correct URLs for license and rules
993d4af 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne
Copy link
Member

Back to green! I am merging this now!

@pombredanne pombredanne merged commit 3021c74 into develop Jan 4, 2023
@pombredanne pombredanne deleted the release-32-rc1-prep branch January 4, 2023 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pombredanne pombredanne pombredanne approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@AyanSinhaMahapatra @DennisClark @pombredanne