pipenv uninstall --all-dev uninstalls shared default dependencies

[fernandrone]

Issue description

pipenv uninstall --all-dev uninstalls shared default dependencies

Expected result

When running pipenv uninstall --all-dev I'd expect all dev dependencies and their dependencies to be removed, except those shared by default dependencies. Doing so breaks the application and forces me to run pipenv install.

Actual result

At least on one scenario that I observed, pipenv uninstall --all-dev will actually remove dev dependencies that are shared by default dependencies.

Steps to replicate

I'm using this sample application of mine to reproduce the behavior: https://github.com/fbcbarbosa/flask-chuck-norris

You'll notice that click is shared by black (a dev dependency) and Flask (a "default" dependency. When running pipenv uninstall --all-dev, click is also removed, breaking Flask.

$ pipenv install
...
$ pipenv graph
black==19.3b0
  - appdirs [required: Any, installed: 1.4.3]
  - attrs [required: >=18.1.0, installed: 19.1.0]
  - click [required: >=6.5, installed: 7.0]
  - toml [required: >=0.9.4, installed: 0.10.0]
Flask==1.0.2
  - click [required: >=5.1, installed: 7.0]
  - itsdangerous [required: >=0.24, installed: 1.1.0]
  - Jinja2 [required: >=2.10, installed: 2.10.1]
    - MarkupSafe [required: >=0.23, installed: 1.1.1]
  - Werkzeug [required: >=0.14, installed: 0.15.2]
pyjokes==0.5.0
pytest==4.4.1
  - atomicwrites [required: >=1.0, installed: 1.3.0]
  - attrs [required: >=17.4.0, installed: 19.1.0]
  - more-itertools [required: >=4.0.0, installed: 7.0.0]
  - pluggy [required: >=0.9, installed: 0.9.0]
  - py [required: >=1.5.0, installed: 1.8.0]
  - setuptools [required: Any, installed: 41.0.1]
  - six [required: >=1.10.0, installed: 1.12.0]
$ pipenv uninstall --all-dev
...
Removing black from Pipfile…
Uninstalling click…
Uninstalling Click-7.0:
  Successfully uninstalled Click-7.0

No package click to remove from Pipfile.
...
$ pipenv graph # graph says click is installed, but it isn't
Flask==1.0.2
  - click [required: >=5.1, installed: 7.0]
  - itsdangerous [required: >=0.24, installed: 1.1.0]
  - Jinja2 [required: >=2.10, installed: 2.10.1]
    - MarkupSafe [required: >=0.23, installed: 1.1.1]
  - Werkzeug [required: >=0.14, installed: 0.15.2]
pyjokes==0.5.0
$ pipenv shell
Launching subshell in virtual environment…
...
$ pip freeze
Flask==1.0.2
itsdangerous==1.1.0
Jinja2==2.10.1
MarkupSafe==1.1.1
pyjokes==0.5.0
Werkzeug==0.15.2

Running the application will throw an error, because click is not installed.

To tell the truth, the documentation is not clear on the expected behavior:

--all-dev             Un-install all package from [dev-packages].

Still, I find it unlikely that this is working as intended. Since in any reasonably sized project, dev dependencies and default dependencies will share some common dependencies, running pipenv uninstall --all-dev would always need to be followed by pipenv install.

My use case is that I want to run pipenv as part of my CI, and after running my tests I'd like to prune all dev dependencies before shipping my application - specifically, I'll be copying the pipenv environment inside a docker image.

To give another example, I'd expect it to work like npm prune --production.

---

$ pipenv --support

Pipenv version: '2018.11.26'

Pipenv location: '/usr/lib/python3.7/site-packages/pipenv'

Python location: '/usr/bin/python3'

Python installations found:

• 3.7.3: /home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/.venv/bin/python3
• 3.7.3: /usr/bin/python3.7
• 3.7.3: /usr/bin/python3.7m
• 2.7.15: /usr/bin/python2

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.3',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.0.9-200.fc29.x86_64',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Mon Apr 22 00:55:30 UTC 2019',
 'python_full_version': '3.7.3',
 'python_version': '3.7',
 'sys_platform': 'linux'}

System environment variables:

• PIPENV_VENV_IN_PROJECT
• NVM_DIR
• LS_COLORS
• TERMINATOR_UUID
• XDG_MENU_PREFIX
• MODULES_RUN_QUARANTINE
• LANG
• GDM_LANG
• HISTCONTROL
• LESS
• DISPLAY
• HOSTNAME
• OLDPWD
• SDKMAN_CANDIDATES_API
• KUBECONFIG
• EDITOR
• COLORTERM
• NVM_CD_FLAGS
• FPATH
• DESKTOP_AUTOSTART_ID
• USERNAME
• JAVA_HOME
• KDEDIRS
• XDG_VTNR
• GIO_LAUNCHED_DESKTOP_FILE_PID
• ZSH
• SSH_AUTH_SOCK
• XDG_SESSION_ID
• MODULES_CMD
• USER
• ENV
• PAGER
• LSCOLORS
• DESKTOP_SESSION
• GOPATH
• GRADLE_HOME
• PWD
• HOME
• KOPS_CLUSTER_NAME
• SSH_AGENT_PID
• XDG_SESSION_TYPE
• BASH_ENV
• XDG_DATA_DIRS
• TERMINATOR_DBUS_NAME
• XDG_SESSION_DESKTOP
• LOADEDMODULES
• SDKMAN_DIR
• TERMINATOR_DBUS_PATH
• MAIL
• ZSH_CUSTOM
• WINDOWPATH
• SHELL
• VTE_VERSION
• TERM
• QT_IM_MODULE
• XMODIFIERS
• NVM_BIN
• SDKMAN_CANDIDATES_DIR
• XDG_CURRENT_DESKTOP
• GIO_LAUNCHED_DESKTOP_FILE
• KOPS_STATE_STORE
• XDG_SEAT
• SHLVL
• MANPATH
• MODULEPATH
• GDMSESSION
• LOGNAME
• DBUS_SESSION_BUS_ADDRESS
• XDG_RUNTIME_DIR
• XAUTHORITY
• MODULEPATH_modshare
• GOBIN
• PATH
• SDKMAN_VERSION
• DATASTORE_TYPE
• MODULESHOME
• HISTSIZE
• SDKMAN_PLATFORM
• SESSION_MANAGER
• LESSOPEN
• BASH_FUNC_module%%
• BASH_FUNC__module_raw%%
• BASH_FUNC_switchml%%
• _
• APPLICATION_INSIGHTS_NO_DIAGNOSTIC_CHANNEL
• TERM_PROGRAM
• TERM_PROGRAM_VERSION
• PIP_DISABLE_PIP_VERSION_CHECK
• PYTHONDONTWRITEBYTECODE
• PIP_PYTHON_PATH
• PIPENV_ACTIVE
• LC_CTYPE
• VIRTUAL_ENV
• PS1
• PIP_SHIMS_BASE_MODULE
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

• PIPENV_VENV_IN_PROJECT: true
• PIPENV_ACTIVE: 1

Debug–specific environment variables:

• PATH: /home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/.venv/bin:/home/walker/.nvm/versions/node/v12.1.0/bin:/home/walker/.sdkman/candidates/java/current/bin:/home/walker/.sdkman/candidates/gradle/current/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/walker/bin:/var/lib/snapd/snap/bin:/home/walker/Workspace/bin:/home/walker/go/bin:/home/walker/Workspace/bin:/home/walker/go/bin:/home/walker/Workspace/bin:/home/walker/go/bin
• SHELL: /bin/zsh
• EDITOR: vim
• LANG: en_US.UTF-8
• PWD: /home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck
• VIRTUAL_ENV: /home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/.venv

---

Contents of Pipfile ('/home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/Pipfile'):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
flask = "==1.0.2"
pyjokes = "==0.5.0"

[requires]
python_version = "3.7.3"

Contents of Pipfile.lock ('/home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "6a10557b3e2690522705a2875ca320334b9903a13e8bf0af2fb0273707afdbf0"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7.3"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "click": {
            "hashes": [
                "sha256:2335065e6395b9e67ca716de5f7526736bfa6ceead690adf616d925bdc622b13",
                "sha256:5b94b49521f6456670fdb30cd82a4eca9412788a93fa6dd6df72c94d5a8ff2d7"
            ],
            "version": "==7.0"
        },
        "flask": {
            "hashes": [
                "sha256:2271c0070dbcb5275fad4a82e29f23ab92682dc45f9dfbc22c02ba9b9322ce48",
                "sha256:a080b744b7e345ccfcbc77954861cb05b3c63786e93f2b3875e0913d44b43f05"
            ],
            "index": "pypi",
            "version": "==1.0.2"
        },
        "itsdangerous": {
            "hashes": [
                "sha256:321b033d07f2a4136d3ec762eac9f16a10ccd60f53c0c91af90217ace7ba1f19",
                "sha256:b12271b2047cb23eeb98c8b5622e2e5c5e9abd9784a153e9d8ef9cb4dd09d749"
            ],
            "version": "==1.1.0"
        },
        "jinja2": {
            "hashes": [
                "sha256:065c4f02ebe7f7cf559e49ee5a95fb800a9e4528727aec6f24402a5374c65013",
                "sha256:14dd6caf1527abb21f08f86c784eac40853ba93edb79552aa1e4b8aef1b61c7b"
            ],
            "version": "==2.10.1"
        },
        "markupsafe": {
            "hashes": [
                "sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473",
                "sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161",
                "sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235",
                "sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5",
                "sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff",
                "sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b",
                "sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1",
                "sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e",
                "sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183",
                "sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66",
                "sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1",
                "sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1",
                "sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e",
                "sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b",
                "sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905",
                "sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735",
                "sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d",
                "sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e",
                "sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d",
                "sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c",
                "sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
                "sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
                "sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
                "sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
                "sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
                "sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
                "sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
                "sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7"
            ],
            "version": "==1.1.1"
        },
        "pyjokes": {
            "hashes": [
                "sha256:01b90474d5c889c21cae88f0d5fc8db1334b2891a16df75cbf9a0886bfdea653",
                "sha256:745299c99b7a4015251b4876ecd59dd62803f458a9f0540d036f0f6627dcc67e"
            ],
            "index": "pypi",
            "version": "==0.5.0"
        },
        "werkzeug": {
            "hashes": [
                "sha256:0a73e8bb2ff2feecfc5d56e6f458f5b99290ef34f565ffb2665801ff7de6af7a",
                "sha256:7fad9770a8778f9576693f0cc29c7dcc36964df916b83734f4431c0e612a7fbc"
            ],
            "version": "==0.15.2"
        }
    },
    "develop": {}
}

[fernandrone]

Also, if this is confirmed to be an issue, I'd happily open a PR and contribute!

This is the first sane Python package management workflow I've found, and the first one that almost enables me to purge dev dependencies from the environment (almost due to this bug... at least it's not that bad, because I can always run pipenv install again and it will only reinstall missing shared dependencies).

[jxltom]

Currently pipenv just simply removes all packages in dev but not smart to detect the shared dependencies of default and dev packages. Personally I feels what you are proposing is quite reasonable.

[techalchemy]

Yeah I'd definitely merge a PR which improves the behavior but I think we'd need an accompanying addition to /peeps describing what exactly we expect here, I know I haven't spent too much time considering this because we aren't looking to add too many knobs so e just made it as simple as possible -- just remove the things

it does seem a bit obvious that we shouldn't remove the things that are also in the other section though...

[cur33]

Bump—This is still an issue. Not too difficult to fix when it happens, but annoying and definitely unexpected.

[TTMaZa]

May I draw your attention to this behaviour:

pipenv install deprecation         # note: this will install pyparsing since its a dependency of deprecation
pipenv install --dev pyparsing 
pipenv uninstall --all-dev         # note: this will uninstall pyparsing even if it is in the [packages] section of the Pipfile!!!
pipenv sync                        # note: this will reinstall pyparsing

So to sumarize the problem from my view:

• What not works: pipenv uninstall --all-dev. This removes things that are still needed 👎
• What works: pipenv --rm && pipenv sync. This realy slow but rock solid 👍
• What I'm looking for is something such as 'pipenv clean --keep-non-dev-only' or pipenv uninstall --all-dev-only. This would by rock solid and fast 👍 👍