Skip to content

Support advanced package assembly in multi-flavour repository #3604

@AyanSinhaMahapatra

Description

@AyanSinhaMahapatra

When there are package manifests from multiple package ecosystems and we perform a package assembly for the package manifest that we find first, a top level package is created out of it, and while assigning resources, we ignore any package manifest of different type from package assembly as they do not match the data-file patterns of this package handler.

Suggested fix: While doing Package assembly even if a package data is in the seen_resource_paths here: https://github.com/nexB/scancode-toolkit/blob/develop/src/packagedcode/plugin_package.py#L351 but it has a package data in it which has different type from the top-level package that made this resource seen, do not ignore this resource. To track this we have to make the seen_resource_paths a dictionary with {"path": "package_type"} instead of a plain list.

See example at https://github.com/nexB/scancode-toolkit/blob/support-cargo-workspaces/tests/packagedcode/data/cargo/cargo-with-workspace.expected.json#L1527 where the package from this resource is not assembled because of a package.json at the root.

See debug logs for the issue:

DEBUG    packagedcode.plugin_package:plugin_package.py:56 get_package_and_deps: location: /home/ayansinha/nexB/write_access/scancode-extra/tests/packagedcode/data/cargo/cargo-with-workspace/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56   get_package_and_deps: package_data.purl: None
DEBUG    packagedcode.plugin_package:plugin_package.py:56   get_package_and_deps: handler: <class 'packagedcode.cargo.CargoTomlHandler'>
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Package: pkg:cargo/tauri@2.0.0-alpha.17
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Package: pkg:cargo/tauri-runtime@1.0.0-alpha.4
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-runtime/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-runtime/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Package: pkg:cargo/tauri-build@2.0.0-alpha.11
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-build/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-build/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Package: pkg:cargo/restart@0.1.0
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tests/restart/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tests/restart/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56 get_package_and_deps: location: /home/ayansinha/nexB/write_access/scancode-extra/tests/packagedcode/data/cargo/cargo-with-workspace/package.json
DEBUG    packagedcode.plugin_package:plugin_package.py:56   get_package_and_deps: package_data.purl: pkg:npm/tauri-workspace@0.0.0
DEBUG    packagedcode.plugin_package:plugin_package.py:56   get_package_and_deps: handler: <class 'packagedcode.npm.NpmPackageJsonHandler'>
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Package: pkg:npm/tauri-workspace@0.0.0
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/LICENSE.spdx
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/LICENSE_APACHE-2.0
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/LICENSE_MIT
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/package.json
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/README.md
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-build
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-build/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-runtime
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tauri-runtime/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tests
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tests/restart
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/core/tests/restart/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/examples
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/examples/api
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/examples/api/package.json
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/examples/api/src-tauri
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/examples/api/src-tauri/Cargo.lock
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/examples/api/src-tauri/Cargo.toml
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/package.json
DEBUG    packagedcode.plugin_package:plugin_package.py:56     get_package_and_deps: Resource: cargo-with-workspace/package.json
DEBUG    packagedcode.plugin_package:plugin_package.py:56 get_package_and_deps: location: /home/ayansinha/nexB/write_access/scancode-extra/tests/packagedcode/data/cargo/cargo-with-workspace

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions