Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipenv no longer working under the root directory is not listed as a breaking change #4273

Closed
gps035 opened this issue May 28, 2020 · 28 comments · Fixed by #4275
Closed

Pipenv no longer working under the root directory is not listed as a breaking change #4273

gps035 opened this issue May 28, 2020 · 28 comments · Fixed by #4275
Labels
Type: Bug 🐛 Type: Regression

Comments

@gps035
Copy link

@gps035 gps035 commented May 28, 2020

Issue description

Our docker builds failed this morning due to a change in behaviour when running pipenv install in the root directory.
When running with the latest version (2020.5.28) an error is thrown after dependencies are built and installed ERROR: Pipenv is not intended to work under the root directory, please choose another path.
It looks like this is a change introduced in #3386, relating to an issue raised in #3434.
This issue is really just to point out that this behaviour change should be listed as breaking in the changelog, as it doesn't currently appear to be mentioned.
For the time being we have worked around this by pinning pipenv to the last version (2018.11.26).

Expected result

N/A

Actual result

N/A

Steps to replicate

N/A

@mohamedMok
Copy link

@mohamedMok mohamedMok commented May 28, 2020

hi @gps035,

I'm having the same issue with pipenv.
Could you please show how you've pinned the pipenv version ?

Thanks

@jean-malo
Copy link

@jean-malo jean-malo commented May 28, 2020

I can confirm that this behaviour does not happen on version 2018.11.26.

@mohamedMok You can use pip install 'pipenv==2018.11.26' which is the last version that does not have this breaking change.

@frostming
Copy link
Contributor

@frostming frostming commented May 28, 2020

@gps035 Any chance sending a PR to mention it in the CHANGELOG?

I have filed a PR to address this issue, thanks for everyone.

@kbenchaaboun
Copy link

@kbenchaaboun kbenchaaboun commented May 28, 2020

Not a funny breaking changes, all our dockers are using pipenv install during build :/

@frostming frostming added the Type: Regression label May 28, 2020
@killuazhu
Copy link

@killuazhu killuazhu commented May 28, 2020

Run into the same issue, using fix in #4273 (comment) worked for me.

@techalchemy
Copy link
Member

@techalchemy techalchemy commented May 28, 2020

Our docker builds failed this morning due to a change in behaviour when running pipenv install in the root directory.

can you explain the workflow here -- are you using --system?

@techalchemy
Copy link
Member

@techalchemy techalchemy commented May 28, 2020

As I just mentioned in #4275:

The main reason for the change in the first place is due to locating virtual environments & related python paths -- as far as I was aware, this was a substantial cause of bugs and breakages and basically didn't work. The fact that it is breaking workflows is the first I am hearing of it working at all.

This is not intended to be a breaking change, it's intended to prevent a previously broken interaction -- for anyone for whom this was working, please include the full set of command line arguments you were passing to pipenv (e.g. pipenv install --<whatever> and information about your workflow:

  • were you using docker? some other container infrastructure?
  • what user was invoking the command? what UID (basically, was it a root account)
  • were you passing --system to pipenv, creating your own virtualenv, or allowing pipenv to create one for you?
  • what OS was pipenv running under?
  • What python version?

Thats probably enough for now

@engstrom
Copy link

@engstrom engstrom commented May 28, 2020

@techalchemy This is the relevant portions of our Dockerfile which no longer works.

FROM python:3.8

RUN pip install --no-cache-dir pipenv
RUN pipenv install --system --deploy

@jayaddison
Copy link

@jayaddison jayaddison commented May 28, 2020

@techalchemy Thanks for canvassing use cases. Here's another example:

  • Containerization scenario: yes, although not strictly Docker (OCI images)
  • Pipenv runs within the container build environment as uid=0
  • The --system flag is not being passed
  • Linux
  • Python 3.8

In case it's useful for investigation or reproducing the issue, the build steps are visible in this Make target.

@DylanBohlender
Copy link

@DylanBohlender DylanBohlender commented May 28, 2020

For anyone who wants to use the latest pipenv with --system, adapting your Dockerfile by setting a WORKDIR and copying your Pipfile/lockfile into it as follows may be helpful:

WORKDIR /code
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv && pipenv install --system
COPY . /code/

@killuazhu
Copy link

@killuazhu killuazhu commented May 28, 2020

  • building dockerfile
  • root
  • see code below
  • python:3-slim docker image, comes with Debian GNU/Linux 10
  • 3.8
FROM python:3-slim AS base

ENV PYROOT /pyroot
ENV PYTHONUSERBASE $PYROOT
ENV PATH $PATH:$PYROOT/bin

FROM base AS builder
RUN pip install pipenv
COPY Pipfile* ./
RUN PIP_USER=1 PIP_IGNORE_INSTALLED=1 pipenv install --system --deploy --ignore-pipfile

@ebordon
Copy link

@ebordon ebordon commented May 28, 2020

For anyone who wants to use the latest pipenv with --system, adapting your Dockerfile by setting a WORKDIR and copying your Pipfile/lockfile into it as follows may be helpful:

WORKDIR /code
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv && pipenv install --system
COPY . /code/

This worked for me. Just remember to create the /code dir before.

@micahjsmith
Copy link

@micahjsmith micahjsmith commented May 28, 2020

This worked for me. Just remember to create the /code dir before.

the WORKDIR command already creates the directory if it does not exist

@killuazhu
Copy link

@killuazhu killuazhu commented May 28, 2020

Using WORKDIR did not work for me. I'm getting error

Step 9/9 : RUN PIP_USER=1 PIP_IGNORE_INSTALLED=1 pipenv install --system --deploy --ignore-pipfile
 ---> Running in da6fa387210f
Installing dependencies from Pipfile.lock (387af5)…
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/build-5NmaZ4l5/bin/python: not found

Output: 
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/build-5NmaZ4l5/bin/python: not found

Output: 
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/build-5NmaZ4l5/bin/python: not found

Output: 
^Cmake: *** [build-image-base] Interrupt: 2

when using dockerfile below

FROM python:3-slim AS base

ENV PYROOT /pyroot
ENV PYTHONUSERBASE $PYROOT
ENV PATH $PATH:$PYROOT/bin

FROM base AS builder
WORKDIR /build
RUN pip install pipenv
COPY Pipfile* /build/
RUN PIP_USER=1 PIP_IGNORE_INSTALLED=1 pipenv install --system --deploy --ignore-pipfile

@micahjsmith
Copy link

@micahjsmith micahjsmith commented May 28, 2020

this does not seem to actually be build error, see #4220

@techalchemy
Copy link
Member

@techalchemy techalchemy commented May 28, 2020

As a sidenote, $PIP_USER gets unset by pipenv and I am relatively uncertain about how $PYTHONUSERBASE interacts with it

Also, the --deploy flag would be somewhat pointless with the --ignore-pipfile flag -- --deploy is used to guarantee that your Pipfile and your Pipfile.lock are in alignment, i.e. that your Pipfile.lock was generated from the corresponding Pipfile. If you indicate that you want to ignore your pipfile, this check never happens.

In any case @killuazhu the error in the logs you included are possibly related to the manipulations of your python path, but will require further investigation if you can file a separate issue

@frostming frostming linked a pull request May 29, 2020 that will close this issue
2 tasks
@frostming
Copy link
Contributor

@frostming frostming commented May 29, 2020

For reference, the original issue of #3434 occurs when one tries to pipenv install under / without a Pipfile. And the setup in this ticket is to pipenv install under / with a Pipfile, which used to be working on 2018.11.26. However, #3386 chose a wrong resolution approach, that it prevents the usage from the root directory entirely.

@frostming frostming removed Type: Documentation 📖 good first issue help wanted labels May 29, 2020
@nateg-gladly
Copy link

@nateg-gladly nateg-gladly commented May 29, 2020

Thank you for fixing, do you have an ETA on when the fix will be included in a new release of the pypi package?

@frostming
Copy link
Contributor

@frostming frostming commented May 29, 2020

we need to ensure all regression issues are fixed and the new release will be out in the next week

@nateg-gladly
Copy link

@nateg-gladly nateg-gladly commented May 30, 2020

Awesome thank you, appreciate it!

@frostming frostming added this to the 2020.6.x bugfix release milestone Jun 1, 2020
@roykim98
Copy link

@roykim98 roykim98 commented Jun 7, 2020

I can confirm that this behaviour does not happen on version 2018.11.26.

@mohamedMok You can use pip install 'pipenv==2018.11.26' which is the last version that does not have this breaking change.

I was getting a slightly different error when running python3 -m pipenv install --three --system

Output:
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/app-4PlAip0Q/bin/python: not found

Pinning to the old version also worked for me. Thanks!

@iamWing
Copy link

@iamWing iamWing commented Jun 18, 2020

I can confirm that this behaviour does not happen on version 2018.11.26.
@mohamedMok You can use pip install 'pipenv==2018.11.26' which is the last version that does not have this breaking change.

I was getting a slightly different error when running python3 -m pipenv install --three --system

Output:
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/app-4PlAip0Q/bin/python: not found

Pinning to the old version also worked for me. Thanks!

I’m having the same issue. Now pinning to the old version as a workaround

@mlej8
Copy link

@mlej8 mlej8 commented Jul 5, 2020

we need to ensure all regression issues are fixed and the new release will be out in the next week

This issue is still present in the 2020.6.2 release:

Output:
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/app-lp47FrbD/bin/python: not found

Could you please confirm if this issue was expected to be fixed in 2020.6.2 release?

@gpakosz
Copy link

@gpakosz gpakosz commented Jul 13, 2020

I can confirm I'm hitting this issue with the following Dockerfile

FROM python:3.7-slim

ENV LC_ALL C.UTF-8
ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && \
    apt-get upgrade && \
    apt-get install -y --no-install-recommends libldap2-dev libsasl2-dev libssl-dev && \
    apt-get clean autoclean && rm -rf /var/lib/apt/* /var/cache/apt/* && \
    apt-get autoremove --purge && \
    pip install pipenv --no-cache-dir

WORKDIR /app

COPY Pipfile Pipfile.lock ./
RUN pipenv install --deploy --system --verbose

ENTRYPOINT ["uvicorn", "web.main:app", "--host", "0.0.0.0"]

EXPOSE 8000/tcp

@gpakosz
Copy link

@gpakosz gpakosz commented Jul 15, 2020

@frostming Can you please reopen the issue?

harveyrendell added a commit to harveyrendell/discord-echo that referenced this issue Jul 23, 2020
harveyrendell added a commit to harveyrendell/discord-karma that referenced this issue Jul 23, 2020
@alexwaweru
Copy link

@alexwaweru alexwaweru commented Jul 27, 2020

I can also confirm I'm hitting this issue with the following Dockerfile:

FROM python:3.7.6-slim-stretch
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
WORKDIR /app
COPY .  /app
RUN pip install --upgrade pip
RUN pip install pipenv
RUN pipenv install --system --deploy --ignore-pipfile
CMD ["/bin/bash", "scripts/entrypoint.sh"]

Here is the error:

Step 10/11 : RUN pipenv install --system --deploy --ignore-pipfile
 ---> Running in 00386bcedd89
Installing dependencies from Pipfile.lock (d14b54)…
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/app-4PlAip0Q/bin/python: not found

Output: 
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/app-4PlAip0Q/bin/python: not found

Output: 
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/app-4PlAip0Q/bin/python: not found

@franciscojavierarceo
Copy link

@franciscojavierarceo franciscojavierarceo commented Aug 12, 2020

For folks still having this issue the easiest fix is to configure your Dockerfile as:

FROM python:3.7-slim

# Set environment varibles
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# Set work directory
WORKDIR /code


# Install dependencies
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv==2018.11.26 && pipenv install --system             # <- this is the fix
...

@gnu-lorien
Copy link

@gnu-lorien gnu-lorien commented Mar 30, 2021

This bug manifested slightly differently for me, but #4273 (comment) solved it nonetheless.

Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pipenv/cli/command.py", line 233, in install
    retcode = do_install(
  File "/usr/local/lib/python3.9/site-packages/pipenv/core.py", line 2052, in do_install
    do_init(
  File "/usr/local/lib/python3.9/site-packages/pipenv/core.py", line 1304, in do_init
    do_install_dependencies(
  File "/usr/local/lib/python3.9/site-packages/pipenv/core.py", line 842, in do_install_dependencies
    deps_list = list(lockfile.get_requirements(dev=dev, only=dev_only))
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/lockfile.py", line 273, in get_requirements
    yield Requirement.from_pipfile(k, v)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2747, in from_pipfile
    r = FileRequirement.from_pipfile(name, pipfile)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 1841, in from_pipfile
    arg_dict["parsed_line"] = Line(line)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 171, in __init__
    self.parse()
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 1304, in parse
    self.parse_name()
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 1030, in parse_name
    name = self._parse_name_from_path()
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 993, in _parse_name_from_path
    metadata = get_metadata(self.path)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 567, in get_metadata
    dist = get_distinfo_dist(path, pkg_name=pkg_name)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 535, in get_distinfo_dist
    dist_dir = next(iter(find_distinfo(path, pkg_name=pkg_name)), None)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 527, in find_distinfo
    for dist_dir in dist_dirs:
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 517, in <genexpr>
    dist_dirs = (
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 490, in iter_metadata
    with contextlib.closing(ScandirCloser(p)) as path_iterator:
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 450, in __init__
    self.iterator = scandir(path)
FileNotFoundError: [Errno 2] No such file or directory: '/dev/fd/3'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Bug 🐛 Type: Regression
Projects
None yet
Development

Successfully merging a pull request may close this issue.