Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Locking may fail while unparsing setup.py if version is read from environment #4274

Closed
peresc opened this issue May 28, 2020 · 1 comment · Fixed by #4302
Closed

Locking may fail while unparsing setup.py if version is read from environment #4274

peresc opened this issue May 28, 2020 · 1 comment · Fixed by #4302
Labels
Type: Bug 🐛 Type: Regression Type: Vendored Dependencies

Comments

@peresc
Copy link

@peresc peresc commented May 28, 2020

Issue description

When the source directory is added to the Pipfile (using pipenv install -e .) and the version number defined in setyp.py is read from the environment, then unparsing setup.py may throw an unexpected exception when running pipenv lock.

Running pipenv lock --clear, as suggested on the "diagnose documentation" page, throws the same exception.

Expected result

I would expect locking operation to continue (and complete), while not necessarily detecting the version number of the source package.

Actual result

The operation aborts due to an unexpected exception:

❯ pipenv lock --verbose
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Building requirements...
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1151, in ast_unparse
    unparsed[unparse(k)] = unparse(v)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1040, in ast_unparse
    items = unparse(analyzer.assignments[assignment])
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1140, in ast_unparse
    val = unparse(item.value, recurse=False)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1004, in ast_unparse
    unparsed = unparsed[unparse(item.slice.value)]
TypeError: 'Attribute' object is not subscriptable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/cli/command.py", line 370, in lock
    write=not state.quiet,
  File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 1117, in do_lock
    keep_outdated=keep_outdated
  File "/usr/local/lib/python3.7/site-packages/pipenv/utils.py", line 1318, in venv_resolve_deps
    deps, project, r=False, include_index=True
  File "/usr/local/lib/python3.7/site-packages/pipenv/utils.py", line 1434, in convert_deps_to_pip
    new_dep = Requirement.from_pipfile(dep_name, dep)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2743, in from_pipfile
    r = FileRequirement.from_pipfile(name, pipfile)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 1843, in from_pipfile
    arg_dict["setup_info"] = arg_dict["parsed_line"].setup_info
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 813, in setup_info
    self.setup_info = self.get_setup_info()
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 802, in get_setup_info
    setup_info = SetupInfo.from_ireq(self.ireq, subdir=self.subdirectory)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1882, in from_ireq
    ireq.source_dir, subdirectory=subdir, ireq=ireq, kwargs=kwargs, stack=stack
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1918, in create
    created.get_initial_info()
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1747, in get_initial_info
    self.update_from_dict(self.parse_setup_py())
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1504, in parse_setup_py
    parsed = ast_parse_setup_py(self.setup_py.as_posix())
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1202, in ast_parse_setup_py
    function_names = ast_analyzer.parse_functions()
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 962, in parse_functions
    retries = self.parse_function_names(function_map=self.function_map)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 958, in parse_function_names
    self.resolved_function_names[fn_name] = ast_unparse(v, analyzer=self)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1153, in ast_unparse
    unparsed[k] = unparse(v)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1040, in ast_unparse
    items = unparse(analyzer.assignments[assignment])
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1140, in ast_unparse
    val = unparse(item.value, recurse=False)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1004, in ast_unparse
    unparsed = unparsed[unparse(item.slice.value)]
TypeError: 'Attribute' object is not subscriptable

Steps to replicate

I think this is the simplest way I can reproduce it. With a setup.py file of:

from setuptools import setup, find_packages
import os

if 'PACKAGE_VERSION' in os.environ:
    my_version = os.environ['PACKAGE_VERSION']
else:
    my_version = '0.0.1'

setup(
    name='pipenv-unparse',
    version=my_version,
    install_requires=[
        'flask==1.1.1'
    ],
    python_requires='~=3.6',
    packages=find_packages('.')
)

And a Pipfile of:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pipenv-unparse = {editable = true, path = "."}

[requires]
python_version = "3.7"

Running the command pipfile lock fails with the described error.

Notes:

  • using the value directly in the keyword argument (version=os.environ['PACKAGE_VERSION']) works, but would obviously fail if run without the environment variable defined
  • using os.environ.get('PACKAGE_VERSION', '0.0.1') works as well
  • python is installed using homebrew; pipenv is installed using python3 -m pip install pipenv

$ pipenv --support

Pipenv version: '2020.5.28'

Pipenv location: '/usr/local/lib/python3.7/site-packages/pipenv'

Python location: '/usr/local/opt/python/bin/python3.7'

Python installations found:

  • 3.7.7: /usr/local/bin/python3
  • 3.7.7: /usr/local/bin/python3.7m
  • 3.7.7: /usr/local/bin/python3.7
  • 3.7.3: /usr/bin/python3
  • 2.7.16: /usr/bin/python2
  • 2.7.16: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.7',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '19.5.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 19.5.0: Thu Apr 30 18:25:59 PDT '
                     '2020; root:xnu-6153.121.1~7/RELEASE_X86_64',
 'python_full_version': '3.7.7',
 'python_version': '3.7',
 'sys_platform': 'darwin'}

System environment variables:

  • TERM_SESSION_ID
  • SSH_AUTH_SOCK
  • LC_TERMINAL_VERSION
  • COLORFGBG
  • ITERM_PROFILE
  • XPC_FLAGS
  • PWD
  • SHELL
  • LC_CTYPE
  • TERM_PROGRAM_VERSION
  • TERM_PROGRAM
  • PATH
  • DISPLAY
  • LC_TERMINAL
  • COLORTERM
  • TERM
  • HOME
  • TMPDIR
  • USER
  • XPC_SERVICE_NAME
  • LOGNAME
  • ITERM_SESSION_ID
  • __CF_USER_TEXT_ENCODING
  • SHLVL
  • OLDPWD
  • P9K_TTY
  • ZSH
  • LESS
  • PAGER
  • LSCOLORS
  • P9K_SSH
  • LC_ALL
  • LANG
  • _
  • PIP_DISABLE_PIP_VERSION_CHECK
  • PYTHONDONTWRITEBYTECODE
  • PIP_SHIMS_BASE_MODULE
  • PIP_PYTHON_PATH
  • PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

  • PATH: /Users/pes/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/Apple/usr/bin
  • SHELL: /bin/zsh
  • LANG: en_US.UTF-8
  • PWD: /Users/pes/work/Projects/CUE/work/pipenv-unparse

Contents of Pipfile ('/Users/pes/work/Projects/CUE/work/pipenv-unparse/Pipfile'):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pipenv-unparse = {editable = true, path = "."}

[requires]
python_version = "3.7"

Contents of Pipfile.lock ('/Users/pes/work/Projects/CUE/work/pipenv-unparse/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "aea1c7e6f5b86dd435a104a000cfad6d7c5664098cb0812ddd89ce3e20070f46"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "click": {
            "hashes": [
                "sha256:d2b5255c7c6349bc1bd1e59e08cd12acbbd63ce649f2588755783aa94dfb6b1a",
                "sha256:dacca89f4bfadd5de3d7489b7c8a566eee0d3676333fbb50030263894c38c0dc"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
            "version": "==7.1.2"
        },
        "flask": {
            "hashes": [
                "sha256:13f9f196f330c7c2c5d7a5cf91af894110ca0215ac051b5844701f2bfd934d52",
                "sha256:45eb5a6fd193d6cf7e0cf5d8a5b31f83d5faae0293695626f539a823e93b13f6"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
            "version": "==1.1.1"
        },
        "itsdangerous": {
            "hashes": [
                "sha256:321b033d07f2a4136d3ec762eac9f16a10ccd60f53c0c91af90217ace7ba1f19",
                "sha256:b12271b2047cb23eeb98c8b5622e2e5c5e9abd9784a153e9d8ef9cb4dd09d749"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==1.1.0"
        },
        "jinja2": {
            "hashes": [
                "sha256:89aab215427ef59c34ad58735269eb58b1a5808103067f7bb9d5836c651b3bb0",
                "sha256:f0a4641d3cf955324a89c04f3d94663aa4d638abe8f733ecd3582848e1c37035"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
            "version": "==2.11.2"
        },
        "markupsafe": {
            "hashes": [
                "sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473",
                "sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161",
                "sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235",
                "sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5",
                "sha256:13d3144e1e340870b25e7b10b98d779608c02016d5184cfb9927a9f10c689f42",
                "sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff",
                "sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b",
                "sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1",
                "sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e",
                "sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183",
                "sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66",
                "sha256:596510de112c685489095da617b5bcbbac7dd6384aeebeda4df6025d0256a81b",
                "sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1",
                "sha256:6788b695d50a51edb699cb55e35487e430fa21f1ed838122d722e0ff0ac5ba15",
                "sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1",
                "sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e",
                "sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b",
                "sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905",
                "sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735",
                "sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d",
                "sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e",
                "sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d",
                "sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c",
                "sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
                "sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
                "sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
                "sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
                "sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
                "sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
                "sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
                "sha256:cdb132fc825c38e1aeec2c8aa9338310d29d337bebbd7baa06889d09a60a1fa2",
                "sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7",
                "sha256:e8313f01ba26fbbe36c7be1966a7b7424942f670f38e666995b88d012765b9be"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==1.1.1"
        },
        "pipenv-unparse": {
            "editable": true,
            "path": "."
        },
        "werkzeug": {
            "hashes": [
                "sha256:2de2a5db0baeae7b2d2664949077c2ac63fbd16d98da0ff71837f7d1dea3fd43",
                "sha256:6c80b1e5ad3665290ea39320b91e1be1e0d5f60652b964a3070216de83d2e47c"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
            "version": "==1.0.1"
        }
    },
    "develop": {}
}
@techalchemy techalchemy added Type: Bug 🐛 Type: Regression Type: Vendored Dependencies labels May 28, 2020
@techalchemy
Copy link
Member

@techalchemy techalchemy commented May 28, 2020

sorry for the issue! I can see why this would cause problems. Thanks for reporting!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Bug 🐛 Type: Regression Type: Vendored Dependencies
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants