Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot assign multiple resources to the same Package using update_or_create_package #681

Closed
JonoYang opened this issue Apr 18, 2023 · 1 comment
Assignees

Comments

@JonoYang
Copy link
Contributor

In my pipeline, I am trying to associate multiple resources to the same package using update_or_create_package (https://github.com/nexB/purldb/blob/49-on-demand-mining/matchcode-toolkit/src/matchcode_toolkit/pipelines/matching.py#L227). However, this only results in one Resource being associated to a package, despite update_or_create_package being called on all of the Resources that should be associated.

The issue occurs in update_or_create_package at https://github.com/nexB/scancode.io/blob/main/scanpipe/pipes/__init__.py#L123 where if the package already exists, we just update the package data and don't associate a codebase resource to that package, even if a codebase resource has been passed into the function.

@JonoYang JonoYang self-assigned this Apr 18, 2023
JonoYang added a commit that referenced this issue Apr 18, 2023
    * Update test to see if we can assign multiple Resources to the same Package with update_or_create_package

Signed-off-by: Jono Yang <jyang@nexb.com>
tdruez added a commit that referenced this issue Apr 25, 2023
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Apr 25, 2023
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Apr 25, 2023
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Apr 25, 2023
* Fix logic in update_or_create_package #681

    * Update test to see if we can assign multiple Resources to the same Package with update_or_create_package

Signed-off-by: Jono Yang <jyang@nexb.com>

* Simplify the update_or_create_package logic #681

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Add ability to assign multiple codebase resources #681

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Force no package_uid as empty string for proper lookups #681

Signed-off-by: Thomas Druez <tdruez@nexb.com>

---------

Signed-off-by: Jono Yang <jyang@nexb.com>
Signed-off-by: Thomas Druez <tdruez@nexb.com>
Co-authored-by: Thomas Druez <tdruez@nexb.com>
@tdruez
Copy link
Contributor

tdruez commented Apr 25, 2023

@JonoYang Thanks for starting this.
I've added multiple changes to the PR adding the ability to provide a list of codebase resources to the function along fixing some bugs related to the package_uid.
This should be much more efficient than calling the function multiple times (less DB lookup too).
Let me know if those work for you :)

@tdruez tdruez closed this as completed Apr 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants