Fix crash when uploading a package through scoped API key

[gabm]

fixes #642

Problem

• the uploader_id gets set to the user_id of the uploader when uploading package files
  • in case a scoped API key is used, that user_id points to an "anonymous user"
• when listing packages, that anonymous user does not have a name (null) and provokes a crash

Solution

• when uploading through an API key, always retrieve the owner_id and use that as uploader_id

[janjagusch]

@gabm, can you please rebase with the latest commit on main?

[gabm]

done

[janjagusch]

@gabm, can you fix the lint, please?

[gabm]

actually the linter was very helpful.. I fixed the problems.. pls have a look

[gabm]

I am actually not sure whether we also want to use the owner in other places... the theory goes like this

• the user might be a "virtual user" corresponding to "downgraded" privileges
• the owner is the "physical user" that issued the API key and therefore the "parent" of the "virtual user" above

In some places we want to use one and in other places the other. For example when uploading a package we want to set the owner as the owner of the pkgs, but in other places we need the reduced privileges of the virtual user...

I am not familiar enough to review all the other uses or get_user and assert_user, but some might be replaced the owner method in some places. But then again its "dangerous" to touch, as it also escalates the privileges again..

[AndreasAlbertQC]

@gabm thanks for the PR. I appreciate the clarity of your fix. On the code itself, I mostly have nitpicky suggestions, I hope you don't mind.
One actually important thing though: Would you mind also writing a test that captures the bug (==fails with the old implementation, but works with your fix?)

[AndreasAlbertQC]

I am actually not sure whether we also want to use the owner in other places... the theory goes like this

• the user might be a "virtual user" corresponding to "downgraded" privileges
• the owner is the "physical user" that issued the API key and therefore the "parent" of the "virtual user" above

In some places we want to use one and in other places the other. For example when uploading a package we want to set the owner as the owner of the pkgs, but in other places we need the reduced privileges of the virtual user...

I am not familiar enough to review all the other uses or get_user and assert_user, but some might be replaced the owner method in some places. But then again its "dangerous" to touch, as it also escalates the privileges again..

My intuition here would be that the current implementation before your fix, where an anonymous user is used, is the most robust in the sense that it never leaks permissions. If anything, it is problematic because you sometimes get too few permissions. Therefore, unless we are fixing a specific problem (as you did in this PR), I would not change the existing system for fear of accidentally leaking permissions for no gain.

[gabm]

thanks for those hints @AndreasAlbertQC . I added the tests as well.. pls have a look.

P.S.: I had a hard time working through the pytest fixtures, they are all over the place :/

[AndreasAlbertQC]

Thanks for the additional work and tests @gabm ! Tests look good to me, I would just add a a try/finally around the yields in the fixtures to headaches if a test throws an exception. Apart from that, this is ready to go IMO

[beenje]

Hi! Thanks for working on this. I have the same issue :-)

I did a quick test and this branch indeed works for new uploaded packages.
What about existing packages? Should this PR include a migration script to fix the database?

Also just wondering if an alternative solution could not have been to show "anonymous/none" for the uploader.
There must be a reason for the user_id to be set to this user?

[AndreasAlbertQC]

As @ivergara kindly pointed out to me, pytest already takes care of the exception handling for yielding fixtures such that the try/except block I asked for is not needed. My bad!

[AndreasAlbertQC]

The CI is fixed on main (pydantic dependency pin). @gabm rebasing your branch on main should fix the CI here.

[gabm]

done

[gabm]

What about existing packages? Should this PR include a migration script to fix the database?

For us its not a huge problem, we fixed it manually so far. Currently there is no straight forward way to fix it in the DB because the uploader information is lost iirc.

Also just wondering if an alternative solution could not have been to show "anonymous/none" for the uploader. There must be a reason for the user_id to be set to this user?

The full details are outlined here #642. In short: the current user model mixes "identity" and "authorization" to some extent. When you create an API key with a scope, then a new user without username is created and added as a member/maintainer to the packages/channels etc... When uploading a package with that API Key, it correctly resolves to that new, anonymous user that has the restricted authorization - thus preventing privilege escalation. Unfortunately, as this anonymous user has no identity (no name etc), the uploader field remains unset and that causes the error. The uploader field should be set with the identity of the "owner of the API Key". This identity should imho not be "anonymous"...

[beenje]

Also just wondering if an alternative solution could not have been to show "anonymous/none" for the uploader. There must be a reason for the user_id to be set to this user?

The full details are outlined here #642. In short: the current user model mixes "identity" and "authorization" to some extent. When you create an API key with a scope, then a new user without username is created and added as a member/maintainer to the packages/channels etc... When uploading a package with that API Key, it correctly resolves to that new, anonymous user that has the restricted authorization - thus preventing privilege escalation. Unfortunately, as this anonymous user has no identity (no name etc), the uploader field remains unset and that causes the error. The uploader field should be set with the identity of the "owner of the API Key". This identity should imho not be "anonymous"...

Thanks for the additional information about restricted authorization. I agree using the owner_id makes sense.

What about existing packages? Should this PR include a migration script to fix the database?

For us its not a huge problem, we fixed it manually so far. Currently there is no straight forward way to fix it in the DB because the uploader information is lost iirc.

I think it's possible to have a migration script:

• we can get null users from users table
• in the api_keys table, we have the link user_id / owner_id
• In package_members table, we want to replace the user_id of null users with owner_id from the api_keys
• In package_versions table, we also have to replace the uploader_id with the same owner_id

Or I guess we don't want to change package_members, right? Only package_versions.

[AndreasAlbertQC]

Thanks @gabm!

@beenje Do you want to break out the migration followup into a separate issue or PR? In the interest of somewhat timely turnaround, I'd like to merge this PR now so that we at least have the fix in the main.

[beenje]

Thanks @gabm!

@beenje Do you want to break out the migration followup into a separate issue or PR? In the interest of somewhat timely turnaround, I'd like to merge this PR now so that we at least have the fix in the main.

Yes, go ahead! I will try to create another PR for the migration.