Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide persistent skip-lock option #2200

Closed
hmaarrfk opened this issue May 16, 2018 · 16 comments
Closed

Provide persistent skip-lock option #2200

hmaarrfk opened this issue May 16, 2018 · 16 comments

Comments

@hmaarrfk
Copy link

@hmaarrfk hmaarrfk commented May 16, 2018

I'm always frustrated when pipenv wants to lock things. This happens when I install and uninstall things. The problem is that it takes FOREVER to lock big scientific packages (that probably include sample data).

I would like to be able to specify completely skipping the locking process within the Pipfile itself.

I've tried to specify --skip-lock in many commands that support it, but some commands simply don't support it. Additionally, if I forget to specify --skip-lock then I have to force quit the program and hope that it maintains a usable state.

@uranusjr
Copy link
Member

@uranusjr uranusjr commented May 16, 2018

If this is done it should be a user preference (like the one discussed in #2197), not a project one. Maybe we can implement a PIPENV_SKIP_LOCK or PIPENV_NO_LOCK environ. With that said, I’m not sure why you’d even bother using Pipenv if you don’t lock at all—Pipfile essentially become requirements.txt in this situation.

@hmaarrfk
Copy link
Author

@hmaarrfk hmaarrfk commented May 16, 2018

Pipfile essentially become requirements.txt in this situation.

Pipenv gives you the virtual environment management. Quite nice IMO.

@hmaarrfk
Copy link
Author

@hmaarrfk hmaarrfk commented May 16, 2018

Would it be appropriate to place the option within the lockfile itself?
That way it wouldn't be in the Pipfile, but we could still have it on a per-project basis?

@techalchemy
Copy link
Member

@techalchemy techalchemy commented May 16, 2018

I think if anything we would want an environment variable for this. We've said no in the past, but I think we've heard it raised enough that we could revisit. I wonder about the security concerns, any thoughts there @ncoghlan? If not I'll run it by Kenneth and we can make a judgment call

@hmaarrfk
Copy link
Author

@hmaarrfk hmaarrfk commented May 16, 2018

If it makes any difference, I would vote against an environment variable. Having new users to linux/windows set environment variables is rather difficult.

I really think the choice should be on a per project basis for the two reasons:

  1. Advanced users: work on one project that requires locked dependencies and an other that is in heavy development where the exact version doesn't matter so much until later in the development cycle.
  2. New users: Already have a hard time understanding what python environments are. Since the locking mechanism takes so long on large packages (this doesn't seem like it will change even with caching), it can take 10s of minutes to install something new you find on github. I would prefer asking them to run a command such as: pipenv --three --unlock install which would create a lock file that simply has a "unlocked" flag.
@techalchemy
Copy link
Member

@techalchemy techalchemy commented May 16, 2018

Which packages are taking 10s of minutes? If you look at npm, composer, etc, this type of feature is never specified at the project level and there are a lot of good reasons for this — locking is an important element of the behavior we are providing, so disabling it should be an advanced feature, if we choose to offer that at all.

You’re right to consider new users. Adding extra knobs on the command line interface is the best way to confuse them.

@hmaarrfk
Copy link
Author

@hmaarrfk hmaarrfk commented May 16, 2018

Ok, here is 5 minutes, not 10s, which is an eternity to add when trying to teach people to add new packages to implement new features. I also have an i7 for what it is worth.

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
spyder = "*"
scikit-image = "*"
"pyqt5" = "*"
cython = "*"
matplotlib = "*"
numba = "*"
vispy = "*"

[dev-packages]

[requires]
python_version = "3.6"
$ pipenv --three
$ pipenv install --skip-lock
$ time pipenv lock
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (7d13c5)!
174.69user 18.86system 5:14.28elapsed 61%CPU (0avgtext+0avgdata 687236maxresident)k
2961256inputs+1890392outputs (0major+4989898minor)pagefaults 0swaps
@hmaarrfk
Copy link
Author

@hmaarrfk hmaarrfk commented May 16, 2018

And here is one from my development environment (15:20.04 elapsed)

$ time pipenv lock
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (fd1cef)!
175.93user 25.78system 15:20.04elapsed 21%CPU (0avgtext+0avgdata 706728maxresident)k
676824inputs+4291400outputs (0major+4954121minor)pagefaults 0swaps
$ cat Pipfile
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]
numpy = "*"
matplotlib = "*"
scipy = "*"
imageio = "*"
cython = "*"
wheel = "*"
pip = "*"
numpydoc = "*"
vispy = {git = "https://github.com/vispy/vispy.git"}
"pyqt5" = "*"
jupyter = "*"

[dev-packages]
"pyqt5" = "*"
jupyter = "*"
ipython = "*"
spyder = "*"
pyyaml = "*"
"pep8" = "*"
setuptools = "*"
wheel = "*"
pytest = "*"
sip = "*"

[requires]
python_version = "3.6"
@ncoghlan
Copy link
Member

@ncoghlan ncoghlan commented May 20, 2018

Skipping locking generally isn't a major security concern - OWASP A9 only comes up when you lock your dependencies once, and then never update them (hence why I think figuring out a better way of assessing lock file freshness will let us make --selective-upgrade the default when installing new dependencies).

So I think if folks running a project want to opt out of dependency pinning, and are happy to accept the resulting time-dependent variation in the way different people have their environments configured, that's an entirely reasonable way to go - it's pretty similar to the situation where folks skip checking Pipfile.lock into source control, but goes a step further in not even locking down the local environment.

@kennethreitz42
Copy link
Contributor

@kennethreitz42 kennethreitz42 commented Jul 4, 2018

skip lock can be an environment variable, but not a pipfile feature.

@uranusjr uranusjr changed the title Provide skip-lock feature in the Pipfile itself Provide persistent skip-lock feature Jul 4, 2018
@uranusjr uranusjr changed the title Provide persistent skip-lock feature Provide persistent skip-lock option Jul 4, 2018
@uranusjr
Copy link
Member

@uranusjr uranusjr commented Jul 4, 2018

Title edited to reflect resolution.

@devxpy
Copy link
Contributor

@devxpy devxpy commented Sep 24, 2018

With that said, I’m not sure why you’d even bother using Pipenv if you don’t lock at all

I present another argument for enabling such a feature:

The "locking" part of pipenv slows me down drastically while in development.

There are times when I just want to work, and not have pipenv take enormous amounts of cpu, internet and time to just install something.

I will obviously, lock later when I'm in mood :)

PIPENV_SKIP_LOCK env var would be awesome!

@joaqo
Copy link

@joaqo joaqo commented Oct 19, 2018

Just want to chime in and validate that this is very useful to several people. Makes pipenv install much faster, which is really cool on simple personal projects. And we still can create the lock whenever we please.

@techalchemy
Copy link
Member

@techalchemy techalchemy commented Oct 19, 2018

I think if we have the prefix for automatically seeking out environment variables enabled this may work by default, but I’m not totally sure. @uranusjr, any idea?

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Oct 19, 2018

Yeah the Click auto prefix thing will probably just work. Is it in the current release yet?

@pauloromeira
Copy link

@pauloromeira pauloromeira commented Nov 9, 2018

+1 for having that option inside Pipenv.

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
skip_lock = true
...

It could have both options because some projects doesn't need locking and you don't usually track .env files. Other than that, env vars are only useful when you already have isolated environments outside pipenv, otherwise is just simpler to pass skip-lock as an argument.

techalchemy added a commit that referenced this issue Nov 14, 2018
…191-3196-3209

- Closes #2200
- Closes #3112 (via merge)

Signed-off-by: Dan Ryan <dan@danryan.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants
You can’t perform that action at this time.