Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkg_resources.resource_string allows absolute paths and paths with .. - contrary to docs #1635

Open
Mekk opened this issue Jan 14, 2019 · 7 comments
Assignees
Labels

Comments

@Mekk
Copy link

@Mekk Mekk commented Jan 14, 2019

The https://setuptools.readthedocs.io/en/latest/pkg_resources.html ("Basic Resource Access") page claims:

Note that resource names must be /-separated paths and cannot be absolute (i.e. no leading /)
or contain relative names like "..".

Let's see:

>>> pkg_resources.resource_string('multiprocessing', '/__init__.py')
'#\n# Package analogous …

>>> pkg_resources.resource_string('multiprocessing', '../../../../etc/passwd')
'root:x:0:0:root…

I'd say some validation is missing.

Tested on both python2.7 and python3.6, with pkg_resources as in Ubuntu 18.04

@Mekk
Copy link
Author

@Mekk Mekk commented Jan 14, 2019

Note that this may have security impact… I already saw the code, which assumed that using pkg_resources guarantees that given path belongs to the package (so can be shown/safely used/…).

@jaraco jaraco added the bug label Jan 17, 2019
@jaraco jaraco self-assigned this Jan 17, 2019
jaraco added a commit that referenced this issue Jan 17, 2019
@jaraco
Copy link
Member

@jaraco jaraco commented Jan 17, 2019

I've started working on this in https://github.com/pypa/setuptools/tree/bugfix/1635-disallow-parent-paths but with that change, I notice that there are many DeprecationWarnings in the setuptools codebase itself, including this call:

for subitem in metadata.resource_listdir('/'):

Due to the way the value is used, by splitting on / and passing the result to os.path.join, the leading / has no effect:

>>> '/'.split('/')
['', '']
>>> os.path.join('foo', '', '')
'foo/'
>>> os.path.join('foo', '', '', '')
'foo/'

So in the case of '/', I suggest just changing the docs to update the expectation.

jaraco added a commit that referenced this issue Jan 17, 2019
@jaraco
Copy link
Member

@jaraco jaraco commented Jan 18, 2019

@Mekk Would you review the PR and share your thoughts?

@Mekk
Copy link
Author

@Mekk Mekk commented Jan 18, 2019

  1. In my opinion leading / should be forbidden. It simply adds alternative confusing syntax. And there are various subtle implications.

    To give you a taste:

    • there exist real code which translates between url-s and pkg_resource apis, for example add_static_view in Pyramid. And in the context of url translation such extra slash may have non-trivial meaning…

    • such a slash may have meaning in case of non-filesystem-based resources, like various archives (some rar or another zip may be happy to extract some/file but not necessarily /some/file). Making restrictions consistent avoid such issues (this is my real case behind this bug, people used /slash/leading/paths, this worked for them on dev, then on true archive-based installation it failed)

    In general: pkg_resources should work the same whether package is on filesystem, in egg, or even provided by some custom loader (as long as that one fulfills specs).. Having some paths working on fs-based installation but not working elsewhere is painful.

  2. Regarding actual patch: what's going to happen on Windows, if \..\ happens (backslash-based illegal path)?

  3. Regarding docs: I'd add some note that currently invalid paths cause only warnings in case of filesystem-installed packages, but this may change into error in the future, and may fail with custom loaders even just now.

jaraco added a commit that referenced this issue Jan 21, 2019
jaraco added a commit that referenced this issue Jan 21, 2019
@jaraco
Copy link
Member

@jaraco jaraco commented Jan 21, 2019

The latest patch also disallows the leading '/'. I imagine there are hundreds of cases that will be affected by this change, so a longer deprecation period will be required.

what's going to happen on Windows?

Possibly a similar issue. There was no protection for this issue before.

I'd add some note that currently invalid paths cause only warnings

I'd like to rely on the changelog to document transient aspects and allow the documentation to reflect the more permanent intentions/expectations.

@jaraco
Copy link
Member

@jaraco jaraco commented Jan 27, 2019

what's going to happen on Windows?

On Windows, if a \ or C:\ is present at the beginning, it will give access to the whole system:

>>> ntpath.join('foo', 'C:\\bar.txt')                                                                                                         
'C:\\bar.txt'

So I've updated the PR to unconditionally disallow Windows-based absolute paths.

@airvzxf
Copy link

@airvzxf airvzxf commented Jun 18, 2021

Hi, I am disagreed to this change because it is not preventing the security since I am able to use setup.py and take any file from the computer, or I am able to modify manually the system file setuptools/config.py and remove the lines of code, I mean it is not a binary compiled and striped. With all of this, I try to say that the philosophy of Python is that it create a freedom kingdom for developers because all of us are adults.

Second, this change is blocking to me and others, since I searched on Internet about this problem. Why setuptools/setup.cfg is forcing me to have all the files in one directory, I understand that the structure having all files in the root is very common (same problem with Dockerfile), but at the same time is nasty, here is my tree directory. In this real example the code is in the main folder, the read me file is in ./README.md and the source code is in ./src/ which include the PyPi setup files.

.
├── LICENSE
├── README.md
├── requirements.txt
├── src
│   ├── build
│   │   ├── bdist.linux-x86_64
│   │   └── lib
│   │       └── sniparinject
│   │           ├── core
│   │           │   ├── game.py
│   │           │   ├── __init__.py
│   │           │   ├── settings.py
│   │           │   ├── text_style.py
│   │           │   └── utility.py
│   │           ├── __init__.py
│   │           └── network_sniffer.py
│   ├── dist
│   │   ├── sniparinject-0.0.0.dev5-py3-none-any.whl
│   │   └── sniparinject-0.0.0.dev5.tar.gz
│   ├── __init__.py
│   ├── pyproject.toml
│   ├── setup.cfg
│   ├── sniparinject
│   │   ├── core
│   │   │   ├── game.py
│   │   │   ├── __init__.py
│   │   │   ├── settings.py
│   │   │   ├── text_style.py
│   │   │   └── utility.py
│   │   ├── __init__.py
│   │   ├── network_sniffer.py
│   └── sniparinject.egg-info
│       ├── dependency_links.txt
│       ├── PKG-INFO
│       ├── requires.txt
│       ├── SOURCES.txt
│       └── top_level.txt
├── test
│   ├── __init__.py
│   ├── pytest.ini
│   ├── test_game.py
│   ├── test_network_sniffer.py
│   ├── test_settings.py
│   └── test_utility.py
└── TODO.md

The setup.cfg file contains:

[metadata]
; .... more set up here
long_description = file: ../README.md
long_description_content_type = text/markdown
url = https://github.com/airvzxf/sniparinject
; .... more set up here

As, I mentioned above, it is easy to fix with setup.py but I want to use setup.cfg because I like it. Means here the security is not covered since I can do it in one place and not in the other.

import setuptools

with open('../README.md', 'r', encoding="utf-8") as fh:
    long_description = fh.read()

setuptools.setup(
    long_description=long_description,
)

Other important reason is that it is using only to read files and put the content inside the PKG-INFO, which copy the content of the README.md file into the file mentioned. I understand the part of copy files from root, but copy a text file to the PKG-INFO?

In the file setuptools/config.py:

    @classmethod
    def _parse_file(cls, value):
        """Represents value as a string, allowing including text
        from nearest files using `file:` directive.

        Directive is sandboxed and won't reach anything outside
        directory with setup.py.

        Examples:
            file: README.rst, CHANGELOG.md, src/file.txt

        :param str value:
        :rtype: str
        """
        include_directive = 'file:'

        if not isinstance(value, str):
            return value

        if not value.startswith(include_directive):
            return value

        spec = value[len(include_directive):]
        filepaths = (os.path.abspath(path.strip()) for path in spec.split(','))
        return '\n'.join(
            cls._read_file(path)
            for path in filepaths
            if (cls._assert_local(path) or True)
            and os.path.isfile(path)
        )

    @staticmethod
    def _assert_local(filepath):
        if not filepath.startswith(os.getcwd()):
            raise DistutilsOptionError(
                '`file:` directive can not access %s' % filepath)

    @staticmethod
    def _read_file(filepath):
        with io.open(filepath, encoding='utf-8') as f:
            return f.read()

More information in the issue that I created: #2699

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants