pipenv install times out while creating virtualenv #65

jstutters · 2017-01-24T11:13:19Z

Running pipenv --three install fails at Creating a virtualenv for this project with the output:

Creating a virtualenv for this project...
Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/pexpect/expect.py", line 97, in expect_loop
    return self.timeout()
  File "/usr/lib/python3.4/site-packages/pexpect/expect.py", line 70, in timeout
    raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: <pexpect.popen_spawn.PopenSpawn object at 0x7fe6f4e65f28>
searcher: searcher_re:
    0: EOF

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/pipenv", line 9, in <module>
    load_entry_point('pipenv==0.2.8', 'console_scripts', 'pipenv')()
  File "/usr/lib/python3.4/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.4/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.4/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.4/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.4/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.4/site-packages/pipenv/cli.py", line 393, in install
    ensure_project(dev=dev, three=three)
  File "/usr/lib/python3.4/site-packages/pipenv/cli.py", line 76, in ensure_project
    ensure_virtualenv(three=three)
  File "/usr/lib/python3.4/site-packages/pipenv/cli.py", line 57, in ensure_virtualenv
    do_create_virtualenv(three=three)
  File "/usr/lib/python3.4/site-packages/pipenv/cli.py", line 166, in do_create_virtualenv
    click.echo(crayons.blue(c.out))
  File "/usr/lib/python3.4/site-packages/delegator.py", line 78, in out
    self.__out = self._pexpect_out
  File "/usr/lib/python3.4/site-packages/delegator.py", line 66, in _pexpect_out
    result += self.subprocess.read().decode('utf-8')
  File "/usr/lib/python3.4/site-packages/pexpect/spawnbase.py", line 413, in read
    self.expect(self.delimiter)
  File "/usr/lib/python3.4/site-packages/pexpect/spawnbase.py", line 321, in expect
    timeout, searchwindowsize, async)
  File "/usr/lib/python3.4/site-packages/pexpect/spawnbase.py", line 345, in expect_list
    return exp.expect_loop(timeout)
  File "/usr/lib/python3.4/site-packages/pexpect/expect.py", line 107, in expect_loop
    return self.timeout(e)
  File "/usr/lib/python3.4/site-packages/pexpect/expect.py", line 70, in timeout
    raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: <pexpect.popen_spawn.PopenSpawn object at 0x7fe6f4e65f28>
searcher: searcher_re:
    0: EOF
<pexpect.popen_spawn.PopenSpawn object at 0x7fe6f4e65f28>
searcher: searcher_re:
    0: EOF

The .venv directory is created and subsequent use of pipenv install [package] work correctly.

Relevant package versions:

delegator.py==0.0.7
pexpect==4.2.1
pipenv==0.2.8
virtualenv==15.1.0

The text was updated successfully, but these errors were encountered:

kennethreitz · 2017-01-24T17:44:07Z

interesting, cannot reproduce locally

spiliopoulos · 2017-01-25T00:59:31Z

@jstutters did you by any chance have any networking problems at the moment that could potentially lead to packages needed for the initial virtualenv to take too long to download.

How to reproduce:
Modify virtualenv to add a more than 30 second sleep without outputing anything.

pipenv --three install

@kennethreitz Are you using the default timeout of 30 seconds for pexpect in delegator? Does that make sense in this case? Shouldn't it probably have a very larger timeout to accommodate slow connections or even not have a timeout at all and let the user stop it if he thinks its stuck somewhere?

kennethreitz · 2017-01-25T01:14:06Z

I am, but there's no reason creating a virtualenv should take more than 30s. I suspect a setup issue is at hand.

kennethreitz · 2017-01-25T01:14:38Z

@jstutters can you time the creation of a new virtualenv by hand?

jstutters · 2017-01-25T10:04:07Z

@kennethreitz It's on a networked home directory which is certainly a potential source of the slowness (sorry- should have said this in the original report). Execution time for making the virtualenv manually is 49 seconds so I guess a timeout increase would help. I had a look at doing this myself but it didn't seem from the docs that delegator exposes the pexpect timeout argument.

jstutters@host ~/test % /usr/bin/time -f %E virtualenv -p /usr/bin/python3 foo
Running virtualenv with interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/jstutters/test/foo/bin/python3
Also creating executable in /home/jstutters/test/foo/bin/python
Please make sure you remove any previous custom paths from your /home/jstutters/.pydistutils.cfg file.
Installing setuptools, pip, wheel...done.
0:48.59

jessebraham · 2017-01-26T21:24:11Z

Okay, so right off the bat I do think we should leave the timeout by default at 30 seconds, because as @kennethreitz mentioned there's not many cases in which this is not enough time. However, there are use cases when it may be useful (or even necessary).

In the do_create_virtualenv function, we make the call delegator.run(cmd, block=False).

If we go take a look at delegator/delegator.py, it makes the call PopenSpawn(self._popen_args, **self._default_pexpect_kwargs). Viewing the documentation for PopenSpawn, we see the constructor takes an optional timeout parameter

def __init__(self, cmd, timeout=30, maxread=2000, searchwindowsize=None, logfile=None,  
             cwd=None, codec_errors='strict')

So if we're wanting to set a custom timeout for PopenSpawn, then we would need to modify the delegator constructor or delegator.run to take an optional timeout=30 parameter, or something to that effect. Then we would be able to pass in the value through Pipenv in whatever fashion is deemed suitable.

kennethreitz · 2017-01-26T21:25:15Z

I think 30s is reasonable too.

Randati · 2017-03-05T11:14:58Z

It would be awesome if there was a fix for this issue. For me creating a virtualenv takes 40 seconds when I have my antivirus running. Disabling it drops the time to 2 seconds and then pipenv works fine, but that's not really an acceptable solution as disabling antivirus company-wide is not an option.

nateprewitt · 2017-03-05T15:25:21Z

@Randati, can you confirm for me if creating a new environment with the virtualenv command or pew new command takes the same amount of time as pipenv --three?

Randati · 2017-03-05T20:08:46Z

Yes, they all take about the same time: pipenv --three 50 s, virtualenv 40 s, pew new 40 s. For pipenv I increased the timeout manually to be able to measure it. All of them are a couple seconds if antivirus is turned off.

nateprewitt · 2017-03-06T04:31:05Z

So I'm a bit confused what it is about the virtualenv creation that's being delayed so drastically by your AV. This feels like more of an issue with virtualenv, but we could implement a "fix" in Delegator.py. That's a separate project of @kennethreitz, so I'm going to tag him in to see if he has thoughts here.

Bumping the timeout makes for a worse experience for people with a legitimate failure, but will increase tolerance for cases like this. Given that we've only had two users encounter this though, I'm not sure how common it really is.

tpoliaw · 2017-03-09T13:15:35Z

I am having this problem as well, also with a slow network home directory. A way of overriding the default timeout or specifying where the venvs are created would be very useful. It takes around 30s to create the venv normally so it only fails sometimes.

A hugely useful script otherwise. Thanks.

cdetrio · 2017-03-11T22:49:58Z

I ran into this problem too. pipenv times out if I have an intensive process running in the backround, but works fine if I kill the background process. Would prefer an option to override the default timeout.

myoung34 · 2017-04-17T18:26:54Z

This happens for me on AWS t2.x instances types, and likely will continue to happen inside containers as well where limitations are exaggerated. While 30s is reasonable, not allowing configuration through something as easy as an env variable is pretty unflexible

rhymes · 2017-04-19T14:17:52Z

➜ pipenv --two
Creating a virtualenv for this project...
Traceback (most recent call last):
File "/usr/local/bin/pipenv", line 11, in
sys.exit(cli())
File "/usr/local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 1043, in invoke
return Command.invoke(self, ctx)
File "/usr/local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/pipenv/cli.py", line 756, in cli
ensure_project(three=three, python=python)
File "/usr/local/lib/python2.7/site-packages/pipenv/cli.py", line 135, in ensure_project
ensure_virtualenv(three=three, python=python)
File "/usr/local/lib/python2.7/site-packages/pipenv/cli.py", line 115, in ensure_virtualenv
do_create_virtualenv(three=three, python=python)
File "/usr/local/lib/python2.7/site-packages/pipenv/cli.py", line 377, in do_create_virtualenv
click.echo(crayons.blue(c.out), err=True)
File "/usr/local/lib/python2.7/site-packages/pipenv/vendor/delegator.py", line 78, in out
self.__out = self._pexpect_out
File "/usr/local/lib/python2.7/site-packages/pipenv/vendor/delegator.py", line 66, in _pexpect_out
result += self.subprocess.read().decode('utf-8')
File "/usr/local/lib/python2.7/site-packages/pexpect/spawnbase.py", line 413, in read
self.expect(self.delimiter)
File "/usr/local/lib/python2.7/site-packages/pexpect/spawnbase.py", line 321, in expect
timeout, searchwindowsize, async)
File "/usr/local/lib/python2.7/site-packages/pexpect/spawnbase.py", line 345, in expect_list
return exp.expect_loop(timeout)
File "/usr/local/lib/python2.7/site-packages/pexpect/expect.py", line 107, in expect_loop
return self.timeout(e)
File "/usr/local/lib/python2.7/site-packages/pexpect/expect.py", line 70, in timeout
raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: <pexpect.popen_spawn.PopenSpawn object at 0x1052c2a10>
searcher: searcher_re:
0: EOF
<pexpect.popen_spawn.PopenSpawn object at 0x1052c2a10>
searcher: searcher_re:
0: EOF

Don't know why it takes so long normally:

➜ time virtualenv whatever
New python executable in /.../whatever/bin/python2.7
Also creating executable in /.../whatever/bin/python
Installing setuptools, pip, wheel...done.
virtualenv whatever 2.58s user 0.71s system 7% cpu 42.808 total

nateprewitt · 2017-04-19T14:27:32Z

@kennethreitz and I had previously discussed adding a timeout option, or bumping the timeout on delegator.py. If he still feels that's a good way to go, we can look into a patch for this.

liangshinegood · 2018-01-02T16:15:44Z

I am having this problem as well

asumawang · 2018-02-13T23:10:32Z

I'm experiencing the same issue as well but I solved it. I tried switching wifi networks because the problem is somehow related to the firewall setup. Didn't bother to ask IT as it will take time.

awhillas · 2018-04-17T00:13:39Z

It was a networking problem for me. VPN was blocking it somehow. Better error message would be handy

uranusjr · 2018-04-19T07:33:02Z

Contributions on better error messages would be welcomed!

nuno-andre · 2018-05-12T22:42:31Z

It would be good to make a reference to PIPENV_TIMEOUT environment variable in that exception.

Eunoia · 2019-07-28T00:11:42Z

I was on my phone's internet, and when I got back on the good wifi, this error went away.

hminooei · 2019-10-10T21:03:34Z

This solution solved my problem: #2681 (comment) (i.e. "using PIP_NO_CACHE_DIR=off when running pipenv install")

positron96 · 2019-11-10T12:15:15Z

Got similar problem with timeout. It waited for around 20 minutes, then failed:

positron@ubox:~/Documents$ pipenv install mesa
Creating a virtualenv for this project…
Pipfile: /home/positron/Documents/Pipfile
Using /usr/bin/python3 (3.5.2) to create virtualenv…
⠏ Creating virtual environment...Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/positron/.local/share/virtualenvs/Documents-rjeAWyrt/bin/python3
Also creating executable in /home/positron/.local/share/virtualenvs/Documents-rjeAWyrt/bin/python
Installing setuptools, pip, wheel...
done.

✔ Successfully created virtual environment! 
Virtualenv location: /home/positron/.local/share/virtualenvs/Documents-rjeAWyrt
Creating a Pipfile for this project…
Installing mesa…
Adding mesa to Pipfile's [packages]…
✔ Installation Succeeded 
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
⠇ Locking...
⠙ Locking...
⠸ Locking...
Traceback (most recent call last):
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/pexpect/expect.py", line 109, in expect_loop
    return self.timeout()
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/pexpect/expect.py", line 82, in timeout
    raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: <pexpect.popen_spawn.PopenSpawn object at 0x7f870d98f1d0>
searcher: searcher_re:
    0: re.compile('\n')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/positron/.local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/cli/command.py", line 254, in install
    editable_packages=state.installstate.editables,
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/core.py", line 1992, in do_install
    skip_lock=skip_lock,
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/core.py", line 1244, in do_init
    pypi_mirror=pypi_mirror,
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/core.py", line 1068, in do_lock
    lockfile=lockfile
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/utils.py", line 649, in venv_resolve_deps
    c = resolve(cmd, sp)
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/utils.py", line 517, in resolve
    result = c.expect(u"\n", timeout=environments.PIPENV_INSTALL_TIMEOUT)
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/delegator.py", line 215, in expect
    self.subprocess.expect(pattern=pattern, timeout=timeout)
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/pexpect/spawnbase.py", line 341, in expect
    timeout, searchwindowsize, async_)
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/pexpect/spawnbase.py", line 369, in expect_list
    return exp.expect_loop(timeout)
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/pexpect/expect.py", line 119, in expect_loop
    return self.timeout(e)
  File "/home/positron/.local/lib/python3.5/site-packages/pipenv/vendor/pexpect/expect.py", line 82, in timeout
    raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: <pexpect.popen_spawn.PopenSpawn object at 0x7f870d98f1d0>
searcher: searcher_re:
    0: re.compile('\n')
<pexpect.popen_spawn.PopenSpawn object at 0x7f870d98f1d0>
searcher: searcher_re:
    0: re.compile('\n')

Pipenv is 2018.11.26
OS is xubuntu 16.04.3 x64

Atrbear · 2020-03-10T15:48:11Z

@positron96 I had the same problem with u. Could u tell me how to solve the problem?

kennethreitz added the reported bug label Jan 26, 2017

kennethreitz closed this as completed Jan 26, 2017

This was referenced Jun 8, 2017

Pipenv fails on a RaspberryPi Zero W and Python 2.7.13 #401

Closed

Added --timeout to allow custom timeouts for command execution #403

Merged

tpoliaw mentioned this issue Dec 13, 2017

If pipenv times out, it leaves processes running in the background #1203

Closed

bufgix mentioned this issue Jul 24, 2019

Pipenv Timeout grapheneX/grapheneX#72

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pipenv install times out while creating virtualenv #65

pipenv install times out while creating virtualenv #65

jstutters commented Jan 24, 2017

kennethreitz commented Jan 24, 2017

spiliopoulos commented Jan 25, 2017

kennethreitz commented Jan 25, 2017

kennethreitz commented Jan 25, 2017

jstutters commented Jan 25, 2017

jessebraham commented Jan 26, 2017

kennethreitz commented Jan 26, 2017

Randati commented Mar 5, 2017

nateprewitt commented Mar 5, 2017 •

edited

Loading

Randati commented Mar 5, 2017

nateprewitt commented Mar 6, 2017

tpoliaw commented Mar 9, 2017

cdetrio commented Mar 11, 2017

myoung34 commented Apr 17, 2017

rhymes commented Apr 19, 2017

nateprewitt commented Apr 19, 2017

liangshinegood commented Jan 2, 2018

asumawang commented Feb 13, 2018

awhillas commented Apr 17, 2018

uranusjr commented Apr 19, 2018

nuno-andre commented May 12, 2018

Eunoia commented Jul 28, 2019

hminooei commented Oct 10, 2019 •

edited

Loading

positron96 commented Nov 10, 2019

Atrbear commented Mar 10, 2020

pipenv install times out while creating virtualenv #65

pipenv install times out while creating virtualenv #65

Comments

jstutters commented Jan 24, 2017

kennethreitz commented Jan 24, 2017

spiliopoulos commented Jan 25, 2017

kennethreitz commented Jan 25, 2017

kennethreitz commented Jan 25, 2017

jstutters commented Jan 25, 2017

jessebraham commented Jan 26, 2017

kennethreitz commented Jan 26, 2017

Randati commented Mar 5, 2017

nateprewitt commented Mar 5, 2017 • edited Loading

Randati commented Mar 5, 2017

nateprewitt commented Mar 6, 2017

tpoliaw commented Mar 9, 2017

cdetrio commented Mar 11, 2017

myoung34 commented Apr 17, 2017

rhymes commented Apr 19, 2017

nateprewitt commented Apr 19, 2017

liangshinegood commented Jan 2, 2018

asumawang commented Feb 13, 2018

awhillas commented Apr 17, 2018

uranusjr commented Apr 19, 2018

nuno-andre commented May 12, 2018

Eunoia commented Jul 28, 2019

hminooei commented Oct 10, 2019 • edited Loading

positron96 commented Nov 10, 2019

Atrbear commented Mar 10, 2020

nateprewitt commented Mar 5, 2017 •

edited

Loading

hminooei commented Oct 10, 2019 •

edited

Loading