Privilege reduction for poetry

[daniel-j-h]

Issue Kind

Brand new capability

Description

The recent xz dilemma made me think about trust (among other things) and because poetry is central to the python ecosystem I wanted to bring this up here. I couldn't find a previous ticket on this; please let me know if there has been discussions around this in the past:

Can we reduce poetry's privileges and sandbox poetry so that it e.g. doesn't have unrestricted access to the full filesystem?

By now we have pretty good sandboxing mechanisms: the recent kernel APIs like landlock, seccomp, namespaces, the bubblewrap sandboxing tool underlying e.g. flatpak's security model.

When users run poetry would it be possible to e.g. make use of landlock to give it only access to filesystem directories and files it needs to have access to but otherwise don't expose the developer's full filesystem? For example, give it access only to: a cache directory and to the virtualenv directory.

I do understand that there are many more ways malicious python packages or social engineering could do damage. The reason I'm opening this issue is to have a discussion around poetry's role in all of this as a package manager interacting with arbitrary user-provided packages.

Impact

The motivation behind this ticket and privilege reduction for poetry is to think about what role poetry plays in the supply-chain and if there is a status quo we should re-think in the light of the xz dilemma.

Workarounds

One workaround is running poetry in a docker container shielding the host's filesystem from the command.

[neersighted]

Hi,

I'm not going to close this outright, as it might result in some interesting discussion, but this is definitely out of scope for Poetry.

Poetry is a tool with a very different privilege model from liblzma or even other system libraries, but more importantly, Poetry is a Python tool. Poetry benefits from being simple and running everywhere; the features you propose are simply not available on Windows, macOS, and the BSDs, which are equally important target platforms for us. Similarly, being Python means Poetry is easy to install anywhere and more easily auditable; Poetry has no interest in shipping native code at this time.

Also, any effort along the lines you suggest would likely mean we'd have to internalize all our dependencies (e.g. virtualenv, installer), or convince them to participate. I find this to be doubtful.

Finally, any effort would have to not significantly impede or disadvantage feature development and other improvements to Poetry. This is a project run by and contributed to by volunteers, and we must be respectful of their time and ability to make contributions.

In any case, I think this is the wrong layer of the stack -- if you want to sandbox Poetry with an OS-specific mechanism, you are welcome to maintain your own packaging to do so. As you have noted there are various mechanisms one can use, and there are others you did not mention (e.g. UWP/AppX packaging, FreeBSD Jails, etc.).

However, as this is a cross-platform tool designed to perform packaging tasks/workflows first and foremost, I don't think it's in scope for us.

One should also recognize that building any Python package entails arbitrary code execution, and in many cases even gathering dependency metadata requires arbitrary code execution; such is the nature of the ecosystem we work in (and again, if one is concerned, sandboxing mechanisms do exist/are recommended).

[dimbleby]

Can we reduce poetry's privileges and sandbox poetry so that it e.g. doesn't have unrestricted access to the full filesystem?

pretty much what neersighted said: yes, you can! - just like any other tool - through the mechanisms you mention and others

Indeed unless you are for some reason running poetry as root - don't do that - you already are arranging that it doesn't have unrestricted access to the full filesystem

by all means experiment with other techniques and share if you come up with anything interesting

[daniel-j-h]

Thanks for your thoughts! Valid point of views for sure. Leaving some comments below:

Poetry benefits from being simple and running everywhere; the features you propose are simply not available on Windows, macOS, and the BSDs, which are equally important target platforms for us. Similarly, being Python means Poetry is easy to install anywhere and more easily auditable; Poetry has no interest in shipping native code at this time.

I get that poetry targets multiple platforms but similar functionality such as landlock on Linux exists e.g. with unveil() on OpenBSD. In addition if we start out sandboxing one or two platforms that doesn't mean the others need to be sandboxed from the start or at all. It's not all-or-nothing situation.

Regarding native code, at least the Linux landlock functionality is available as syscalls you can get access to via ctypes, so no third party dependencies or otherwise native code needed. As an example there is a nice higher level abstraction show-casing how it can work from Python and how to build a simple API similar to unveil() here: https://github.com/Edward-Knight/landlock

In any case, I think this is the wrong layer of the stack -- if you want to sandbox Poetry with an OS-specific mechanism, you are welcome to maintain your own packaging to do so. As you have noted there are various mechanisms one can use, and there are others you did not mention (e.g. UWP/AppX packaging, FreeBSD Jails, etc.).

However, as this is a cross-platform tool designed to perform packaging tasks/workflows first and foremost, I don't think it's in scope for us.

One should also recognize that building any Python package entails arbitrary code execution, and in many cases even gathering dependency metadata requires arbitrary code execution; such is the nature of the ecosystem we work in (and again, if one is concerned, sandboxing mechanisms do exist/are recommended).

You are right in theory we can sandbox poetry already today: be it running in a bwrap shell or isolated namespaces or whatnot. In practice the vast majority of developers will use poetry without any such sandboxing techniques which means all dependencies in the dependency tree can e.g. read ~/.aws/credentials or similar and send it off via the network because both filesystem access as well as network access is possible. That is the reason I was wondering if poetry is interested in limiting the blast radius any such dependency supply-chain attack could cause.