Refactor Authorization

[dstufft]

This refactors our use of the legacy IAuthorizationPolicy to migrate fully onto ISecurityPolicy.

Important things to note:

• The MultiSecurityPolicy remembers what policy provided an identity, and will only dispatch to that policy to call permits().
  • This works using a RequestLocalCache which will store a weak reference to the request and return the same results each time, this keeps the logic encapsulated inside MultiSecurityPolicy.
  • All of our security policies ultimately dispatch to Pyramid's ACL system, so permissions are the same across all the security policies except for the policy specific bits (Macaroons, 2FA, etc).
  • This keeps each security policy focused, you no longer have to figure out a complex interaction between multiple authorization policies that wrap each other.
• 2FA enforcement is no longer it's own AuthZ policy, but part of the Basic Auth and Session policies.
• Any object that we use for request.identity now knows how to generate it's own principals, this keeps ACLs and Principals both "near" the objects that implement them.
• Removed some group:with_admin_dashboard_access group and related permission that, as far as I can tell, were not used for anything and were being granted to every logged in user, which felt wrong and confusing to me.
• Removed the group:with_admin_dashboard_access and granted the admin_dashboard_access permission to our 3 admin groups.

Making this a draft because I want to do some manual testing tomorrow when I'm more awake.

Thanks to @tnytown for a great start with #13754!

Closes #13754
Fixed #13690

[ewdurbin]

Grossest diff ever, but this makes sense after four cups of coffee and a bit too much time switching between diffs and the files as they sit.

[ewdurbin]

I bet this made you so happy @dstufft

[dstufft]

[miketheman]

I'm not a Pyramid security policy expert, but from what I could follow, this makes sense and simplifies the policy stack.

Should these changes (and the ones that follow) be represented somewhere in the dev docs? https://github.com/pypi/warehouse/blob/main/docs/dev/security.rst
Overall security approach/strategy for internal authN/authZ might be helpful to understand how the thing is meant to work. I think you wrote a fair amount already in Slack messages, and have some more ideas up your sleeve.

[miketheman]

praise: ‏love that the assertion is now equivalent with the object, so any extra keys would be sniffed out

[miketheman]

thought: ‏I thought using assert could be tricky, if the runtime ever chose to run under -O https://docs.python.org/3.11/using/cmdline.html#cmdoption-O
I don't think we're doing that today, but on the off chance we ever do run the warehouse with the flag (or envvar PYTHONOPTIMIZE), this would always return None. Wouldn't running under optimize mode effectively bypass this check?
assert is used another couple of times here, as well as other existing cases in warehouse/macaroons/caveats/__init__.py and warehouse/http.py so this isn't a new thing, but If You See Something, Say Something.

[dstufft]

It would technically bypass this check, but the outcome of doing that is we would just fail later with a more inscrutable error message. For example in this method we would fail trying to access `request.identity.has_two_factor``.

In all the places I used asserts, it should be fine to remove them because the structure of the code means that any other condition should not be possible, and would likely error out due to an assumption later.

So why do I have the assertions at all?

1. They serve as verifiable documentation for these functions that they are only valid to be called in cases where the assertion is true.
2. They teach type checkers like MyPy what type is being used, so it can do better linting than assuming request.identity is Any.
3. If we somehow get into a state where a non user is passed in, we'll just try to fail early on with a more reasonable error message.

(3) is the only one really affected by PYTHONOPTIMIZE, and it's the weakest reason of the bunch. It would also likely only be something we do in prod so if we started violating these assumptions we'd likely notice it in development first.

The alternatives here would be to do nothing and not assert (which is what most Python code bases do, they just assume that you've passed in a type that looks like what they expect, e.g. duck typing) or explicitly do type checking with a real error. The latter has all of the same benefits really, but we end up having to test impossible conditions then, for something that shouldn't ever happen.