Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

git - Should Pipfile.lock be committed to version control? #598

Closed
LeCoupa opened this issue Sep 19, 2017 · 26 comments
Closed

git - Should Pipfile.lock be committed to version control? #598

LeCoupa opened this issue Sep 19, 2017 · 26 comments

Comments

@LeCoupa
Copy link

@LeCoupa LeCoupa commented Sep 19, 2017

When two developers are working on a projet with different operating systems, the Pipfile.lock is different (especially the part inside host-environment-markers).

For Composer, most people recommend to commit composer.lock. Do we have to do the same for Pipenv?

@kennethreitz
Copy link
Contributor

@kennethreitz kennethreitz commented Sep 19, 2017

i recommend it.

@kennethreitz
Copy link
Contributor

@kennethreitz kennethreitz commented Sep 19, 2017

Especially for applications. For libraries, less so.

@LeCoupa
Copy link
Author

@LeCoupa LeCoupa commented Sep 19, 2017

Thanks a lot @kennethreitz

@salty-horse
Copy link
Contributor

@salty-horse salty-horse commented Oct 1, 2017

Should this recommendation be documented? Perhaps in a section similar to Composer's.

@dougireton
Copy link

@dougireton dougireton commented Jan 30, 2018

Are their negative consequences to using a Pipfile.lock across OS types? In other words, if I generate my Pipfile.lock on MacOS, then deploy to Linux will Pipenv "do bad things"?

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Jan 31, 2018

@dougireton This depends on your dependencies. Using a lockfile from a different OS is fine if all your packages are modern and well-behaved (either platform-agnostic or have good cross-platform support). Otherwise Pipenv can do things wrong, like installing dependencies that don’t work, or not installing all needed dependencies.

@kennethreitz
Copy link
Contributor

@kennethreitz kennethreitz commented Feb 17, 2018

Generally, yes you should commit it to version control.

@behrangsa
Copy link

@behrangsa behrangsa commented Feb 28, 2018

How come pipenv's own Pipefile.lock is not added to git then?

@kennethreitz
Copy link
Contributor

@kennethreitz kennethreitz commented Feb 28, 2018

because we are special snowflakes.

@anentropic
Copy link

@anentropic anentropic commented Mar 7, 2018

I'm happy to take this advice as correct but I would like to understand why it is given.

the docs say:

Generally, keep both Pipfile and Pipfile.lock in version control.

but I couldn't find any more reasoning in the docs, can you point me to it?

Pipfile.lock is auto-generated and contents will differ depending on platform. If I'm developing on macOS and deploying to Debian ...already it sounds to me like I don't want the lock file in version control.

Then the comment above says:

Using a lockfile from a different OS is fine if all your packages are modern and well-behaved (either platform-agnostic or have good cross-platform support). Otherwise Pipenv can do things wrong, like installing dependencies that don’t work, or not installing all needed dependencies.

Again it sounds like I would not want the lock file in version control.

Here is someone asking similar question #954

Reading through the responses on that issue I have a clearer idea of why I would want it. I think the docs need more elaboration.

The related question is how explicitly should versions be specified in the Pipfile?

If I do pipenv install <package> during development to get the latest version, it will go in the Pipfile with no version specifier. If everyone else gets my Pipfile.lock from vc and does pipenv install --ignore-pipfile they get pinned to the versions I installed, and no need to be more specific in the Pipfile. All good. If they don't do --ignore-pipfile then the lock file may change if a package updated and they have to decide whether to commit the changes. Ok. At any time devs can pipenv update <package> and the Pipfile won't change but the lock file will, and we would commit that. Ok. Is this the intended workflow?

But if the other dev is on a different platform the lock file contents will change(?) but it's not necessarily due to different package versions and we wouldn't want to commit it. Hmm.

And I would only add version specifier to Pipfile if for example I knew at dev time that I needed to install a non-latest version?

Also I noticed in the example Dockerfile you use pipenv install --deploy rather than pipenv install --ignore-pipfile ...they seem to have similar meaning, I would like to understand the subtlety of one vs the other.

(loving pipenv so far though!)

@anentropic
Copy link

@anentropic anentropic commented Mar 13, 2018

I just found some more opinion from the pipenv tool itself:

requirements.txt found, instead of Pipfile! Converting…
Warning: Your Pipfile now contains pinned versions, if your requirements.txt did.
We recommend updating your Pipfile to specify the "*" version, instead.

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Mar 14, 2018

This thread is quite old, but since it emerges again I guess it is best to make a definite, up-to-date statement. As of March 2018, the answer to this question is yes, you should always commit the lock file. Always.

@mmohaveri
Copy link

@mmohaveri mmohaveri commented Apr 1, 2018

@uranusjr could you please explain why?

what has changed in March 2018 that changed the answer from "generally yes" to "always yes"?

What about the issue you explain earlier, is it fixed?

Using a lockfile from a different OS is fine if all your packages are modern and well-behaved (either platform-agnostic or have good cross-platform support). Otherwise Pipenv can do things wrong, like installing dependencies that don’t work, or not installing all needed dependencies.

@techalchemy
Copy link
Member

@techalchemy techalchemy commented Apr 1, 2018

Nothing is different. You should commit it to version control. If there are os-specific markers they should be included automatically. Because setup.py files are non-deterministic, it is possible to resolve a package on linux and find that it resolves differently on windows, but there is nothing we or anyone else besides the package maintainer can do about that. You won't know that's a problem unless you encounter it. If and when you do, simply re-lock and see the diff.

@gsemet
Copy link
Contributor

@gsemet gsemet commented Apr 1, 2018

For application using pipenv, yes you track the lock files.

My librairies however I write using pipenv (and use PBR to reflect to setup.py and an automatic generation of requirements.txt so PBR is happy once the package is deployed), I do not track the lock file. Pretty simple

Hanse00 added a commit to Hanse00/LecToCal that referenced this issue May 26, 2018
…m/questions/46330327/please-explain-the-usage-of-pipfile-and-pipfile-lock) suggests the Pipenv should be versioned as well.

This is to help others build the tool easily. I've decided to start tracking the Pipenv as such, however as of this commit, the pipenv contains far from all dependencies. I will be slowly working those into the pipenv.
dmd pushed a commit to dmd/nkplay that referenced this issue Sep 13, 2018
sdrogers added a commit to sdrogers/nplinker that referenced this issue Oct 29, 2018
…a couple of things from requirements.txt and pipfile to make it compatible with linux
Wittano added a commit to Wittano/Live that referenced this issue Jan 28, 2021
- In main.py create two new functions: get_args(path) and
get_files(path). get_args(path) create list of args elements, which will
be used to run program. that was given by user. get_files(path) create
list of python files, which will be observed by program.
- Changed reloading logic. Now when program detect changes, firstly
he'll terminate current running program and then run again
diff --git a/.gitignore b/.gitignore
index fb7d283..a5e9408 100644
--- a/.gitignore
+++ b/.gitignore
@@ -85,14 +85,14 @@ ipython_config.py
 # pyenv
 #   For a library or package, you might want to ignore these files since the code is
 #   intended to run in multiple environments; otherwise, check them in:
-# .python-version
+.python-version

 # pipenv
 #   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
 #   However, in case of collaboration, if having platform-specific dependencies or dependencies
 #   having no cross-platform support, pipenv may install dependencies that don't work, or not
 #   install all needed dependencies.
-#Pipfile.lock
+Pipfile.lock

 # PEP 582; used by e.g. github.com/David-OConnor/pyflow
 __pypackages__/
@@ -142,3 +142,6 @@ test_file.py

 # Visual Studio Code
 .vscode/
+
+# Test directory
+test/
\ No newline at end of file
diff --git a/Pipfile b/Pipfile
index 02396e0..8f73d14 100644
--- a/Pipfile
+++ b/Pipfile
@@ -4,7 +4,7 @@ verify_ssl = true
 name = "pypi"

 [scripts]
-dev = "python main.py test_file.py"
+dev = "python main.py test/test_file.py"

 [packages]
 watchdog = "*"
diff --git a/handler/handler.py b/handler/handler.py
index d0eb0a5..800d72a 100644
--- a/handler/handler.py
+++ b/handler/handler.py
@@ -1,32 +1,20 @@
-import re
 import subprocess

 from watchdog.events import FileSystemEvent, FileSystemEventHandler

 class FileHandler(FileSystemEventHandler):
-    _path = ""
+    _subprocess = None

-    @Property
-    def path(self):
-        return self._path
-
-    @path.setter
-    def path(self, val: str):
-        if len(val) == 0:
-            raise ValueError("Path must not be empty or none")
-        elif re.search("py$", val) is None:
-            raise ValueError(f"{val} is wrong path. Path must be keept to python file")
-
-        self._path = val
+    def __init__(self, cmd: list, files: list):
+        self._cmd = cmd
+        self._files = files

     def on_any_event(self, event: FileSystemEvent):
-        if event.event_type == "modified" and not event.is_directory:
-            self.run(["python", self._path])

-    def run(self, cmd: dict):
-        """
-        Parameters:
-        cmd (dict): Command with args, which will be run
-        """
-        subprocess.run(cmd)
+        if event.event_type == "modified" and event.src_path in self._files:
+            if self._subprocess is not None:
+                print("\nTerminate the last process!\n")
+                self._subprocess.terminate()
+
+            self._subprocess = subprocess.Popen(self._cmd)
diff --git a/main.py b/main.py
index 18ae0a0..52d84b2 100644
--- a/main.py
+++ b/main.py
@@ -1,3 +1,4 @@
+import glob
 import sys

 from watchdog.observers import Observer
@@ -5,16 +6,19 @@ from watchdog.observers import Observer
 from handler.handler import FileHandler

-def main():
+def main() -> None:
     if sys.argv[1].endswith(".py"):
         path = sys.argv[1]
     else:
         raise IOError("Expect python file")

-    observer = Observer()
-    handler = FileHandler()
+    directory = path.replace(f'/{path.split("/")[-1]}', "")
+
+    handler = FileHandler(get_args(path), get_files(directory))
     handler.path = path
-    observer.schedule(handler, path)
+
+    observer = Observer()
+    observer.schedule(handler, directory)
     observer.start()

     print(f"Start live reload file on {path}")
@@ -27,5 +31,26 @@ def main():
     observer.join()

+def get_args(path: str) -> list:
+    """
+    Create dict for subprocess.Popen() function
+    """
+    cmd = ["python", path]
+    for arg in sys.argv[2:]:
+        cmd.append(arg)
+
+    return cmd
+
+
+def get_files(path: str) -> dict:
+    """
+    Create files dict, which will be observed by live-reloading program
+
+    Parameters:
+    path (str): Directory, which is observed
+    """
+    return glob.glob(path + "/**/*.py", recursive=True)
+
+
 if __name__ == "__main__":
     main()
diff --git a/runner.py b/runner.py
deleted file mode 100644
index b2642e3..0000000
--- a/runner.py
+++ /dev/null
@@ -1,16 +0,0 @@
-import subprocess
-
-
-class PythonRunner:
-    """
-    Script runner for python script or programs
-    """
-
-    def __init__(self):
-        self.file_target = ""
-
-    def run(self):
-        """
-        Execute python prorgam without any args
-        """
-        subprocess.run(["python", self.file_target])
rievo pushed a commit to wikifactory/import-export that referenced this issue Feb 4, 2021
webcallaghan added a commit to webcallaghan/BankCSVtoQif that referenced this issue Mar 20, 2021
The Pipenv authors recommend that Pipfile.lock be committed to version
control. See pypa/pipenv#598.
webcallaghan added a commit to webcallaghan/BankCSVtoQif that referenced this issue Mar 22, 2021
The Pipenv authors recommend that Pipfile.lock be committed to version
control. See pypa/pipenv#598.
webcallaghan added a commit to webcallaghan/BankCSVtoQif that referenced this issue Mar 22, 2021
The Pipenv authors recommend that Pipfile.lock be committed to version
control. See pypa/pipenv#598.
webcallaghan added a commit to webcallaghan/BankCSVtoQif that referenced this issue Mar 23, 2021
The Pipenv authors recommend that Pipfile.lock be committed to version
control. See pypa/pipenv#598.
kawamataryo added a commit to kawamataryo/8bitdo_zero2 that referenced this issue May 4, 2021
diff --git a/utils/.gitignore b/utils/.gitignore
new file mode 100644
index 0000000..7e86cf0
--- /dev/null
+++ b/utils/.gitignore
@@ -0,0 +1,140 @@
+### Python template
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
wspk added a commit to wspk/food-access-map-data that referenced this issue May 11, 2021
maxachis pushed a commit to CodeForPittsburgh/food-access-map-data that referenced this issue May 11, 2021
* Merged requirements.txt into Pipfile, using the Pipfig the Pipefile version where conflicts existed

* pypa/pipenv#598 recommends including Pipefile.lock in version control

* Added __pycache__ to gitignore

* true -> True, causing trivial test cases to now pass
maxachis added a commit to CodeForPittsburgh/food-access-map-data that referenced this issue May 16, 2021
* Merged requirements.txt into Pipfile, using the Pipfig the Pipefile version where conflicts existed

* pypa/pipenv#598 recommends including Pipefile.lock in version control

* Added __pycache__ to gitignore

* true -> True, causing trivial test cases to now pass

* Added .env to .gitignore, and 'git rm'd the files that should be excluded. What all got removed? Everything in the '.env' folder, some '__pycache__' files, 'tests/.RData' and 'tests/.RHistory'. As far as I can tell, these are all good things to remove from version control.

* Consolidate python package management to use pipenv, updated README and workflows accordingly

* Allow pytest workflow to be dispatched direclty

* Updated python version to match github workflow

* Run pytest inside pipenv

Co-authored-by: maxachis <48846180+maxachis@users.noreply.github.com>
maxachis added a commit to CodeForPittsburgh/food-access-map-data that referenced this issue May 25, 2021
* Merged requirements.txt into Pipfile, using the Pipfig the Pipefile version where conflicts existed

* pypa/pipenv#598 recommends including Pipefile.lock in version control

* Added __pycache__ to gitignore

* true -> True, causing trivial test cases to now pass

* Added .env to .gitignore, and 'git rm'd the files that should be excluded. What all got removed? Everything in the '.env' folder, some '__pycache__' files, 'tests/.RData' and 'tests/.RHistory'. As far as I can tell, these are all good things to remove from version control.

* Consolidate python package management to use pipenv, updated README and workflows accordingly

* Allow pytest workflow to be dispatched direclty

* Updated python version to match github workflow

* Run pytest inside pipenv

* Updated tests/test_id_duplicates.py

Tests are now done against a snapshot of data stored in
`tests/test-data/test_id_duplicates.csv`. Note that changing this file
will likely cause tests to fail as ids are referenced explicitly.

As of this commit, the first test `test_bloomfield` fails. As far as I
can tell, this is a problem with the deduplication process, and not the
test case itself.

* Made pytest script passw

* Changed python 3.7 -> 3.6 in Pipfile

* Reverted workflows/pytest.yml to use requirements.txt

* Restored requirements.txt, added pytest

* Changed python version from 3.6 -> 3.7 in workflow

* cleaned workflows to match upstream

Co-authored-by: maxachis <48846180+maxachis@users.noreply.github.com>
meodora added a commit to meodora/meodora that referenced this issue Jun 19, 2021
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
.python-version

# pipenv
#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
#   However, in case of collaboration, if having platform-specific dependencies or dependencies
#   having no cross-platform support, pipenv may install dependencies that don't work, or not
#   install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/
Desoky12 added a commit to Desoky123/desoky that referenced this issue Jul 10, 2021
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
.python-version

# pipenv
#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
#   However, in case of collaboration, if having platform-specific dependencies or dependencies
#   having no cross-platform support, pipenv may install dependencies that don't work, or not
#   install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/
matttyb80 added a commit to galacticcouncil/HydraDX-simulations that referenced this issue Aug 12, 2021
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
.python-version

# pipenv
#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
#   However, in case of collaboration, if having platform-specific dependencies or dependencies
#   having no cross-platform support, pipenv may install dependencies that don't work, or not
#   install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# ignore pycache
*.pyc

# old models
uniswap/**
hydra_single/**
hydra_multi/**
hydra_multi_class/**
KMcLaurin12 added a commit to KMcLaurin12/web-scraping-challenge that referenced this issue Aug 17, 2021
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
.python-version

# pipenv
#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
#   However, in case of collaboration, if having platform-specific dependencies or dependencies
#   having no cross-platform support, pipenv may install dependencies that don't work, or not
#   install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/
louisgaron added a commit to louisgaron/Graph-Theory that referenced this issue Aug 19, 2021
# History files
.Rhistory
.Rapp.history

# Session Data files
.RData

# User-specific files
.Ruserdata

# Example code in package build process
*-Ex.R

# Output files from R CMD build
/*.tar.gz

# Output files from R CMD check
/*.Rcheck/

# RStudio files
.Rproj.user/

# produced vignettes
vignettes/*.html
vignettes/*.pdf

# OAuth2 token, see https://github.com/hadley/httr/releases/tag/v0.3
.httr-oauth

# knitr and R markdown default cache directories
*_cache/
/cache/

# Temporary files created by R markdown
*.utf8.md
*.knit.md

# R Environment Variables
.Renviron

# pkgdown site
docs/

# translation temp files
po/*~



##############################################################################
# Julia
# https://raw.githubusercontent.com/github/gitignore/master/Julia.gitignore
##############################################################################


# Files generated by invoking Julia with --code-coverage
*.jl.cov
*.jl.*.cov

# Files generated by invoking Julia with --track-allocation
*.jl.mem

# System-specific files and directories generated by the BinaryProvider and BinDeps packages
# They contain absolute paths specific to the host computer, and so should not be committed
deps/deps.jl
deps/build.log
deps/downloads/
deps/usr/
deps/src/

# Build artifacts for creating documentation generated by the Documenter package
docs/build/
docs/site/

# File generated by Pkg, the package manager, based on a corresponding Project.toml
# It records a fixed state of all packages used by the project. As such, it should not be
# committed for packages, but should be committed for applications that require a static
# environment.
Manifest.toml


##############################################################################
# Java
# https://raw.githubusercontent.com/github/gitignore/master/Java.gitignore
##############################################################################

# Compiled class file
*.class

# Log file
*.log

# BlueJ files
*.ctxt

# Mobile Tools for Java (J2ME)
.mtj.tmp/

# Package Files #
*.jar
*.war
*.nar
*.ear
*.zip
*.tar.gz
*.rar

# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
hs_err_pid*

##############################################################################
# C
# https://raw.githubusercontent.com/github/gitignore/master/C.gitignore
##############################################################################


# Prerequisites
*.d

# Object files
*.o
*.ko
*.obj
*.elf

# Linker output
*.ilk
*.map
*.exp

# Precompiled Headers
*.gch
*.pch

# Libraries
*.lib
*.a
*.la
*.lo

# Shared objects (inc. Windows DLLs)
*.dll
*.so
*.so.*
*.dylib

# Executables
*.exe
*.out
*.app
*.i*86
*.x86_64
*.hex

# Debug files
*.dSYM/
*.su
*.idb
*.pdb

# Kernel Module Compile Results
*.mod*
*.cmd
.tmp_versions/
modules.order
Module.symvers
Mkfile.old
dkms.conf



##############################################################################
# Jupyter
# https://raw.githubusercontent.com/github/gitignore/master/community/Python/JupyterNotebooks.gitignore
##############################################################################


# gitignore template for Jupyter Notebooks
# website: http://jupyter.org/

.ipynb_checkpoints
*/.ipynb_checkpoints/*

# IPython
profile_default/
ipython_config.py

# Remove previous ipynb_checkpoints
#   git rm -r .ipynb_checkpoints/


##############################################################################
# AWS
# https://raw.githubusercontent.com/github/gitignore/master/community/AWS/SAM.gitignore
##############################################################################

# gitignore template for AWS Serverless Application Model project
# website: https://docs.aws.amazon.com/serverless-application-model

# Ignore build folder
.aws-sam/



##############################################################################
# Windows
# https://raw.githubusercontent.com/github/gitignore/master/Global/Windows.gitignore
##############################################################################



# Windows thumbnail cache files
Thumbs.db
Thumbs.db:encryptable
ehthumbs.db
ehthumbs_vista.db

# Dump file
*.stackdump

# Folder config file
[Dd]esktop.ini

# Recycle Bin used on file shares
$RECYCLE.BIN/

# Windows Installer files
*.cab
*.msi
*.msix
*.msm
*.msp

# Windows shortcuts
*.lnk



##############################################################################
# MacOS
# https://raw.githubusercontent.com/github/gitignore/master/Global/macOS.gitignore
##############################################################################


# General
.DS_Store
.AppleDouble
.LSOverride

# Icon must end with two \r
Icon


# Thumbnails
._*

# Files that might appear in the root of a volume
.DocumentRevisions-V100
.fseventsd
.Spotlight-V100
.TemporaryItems
.Trashes
.VolumeIcon.icns
.com.apple.timemachine.donotpresent

# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk

##############################################################################
# Python
# https://raw.githubusercontent.com/github/gitignore/master/Python.gitignore
##############################################################################

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
#   For a library or package, you might want to ignore these files since the code is
#   intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
#   However, in case of collaboration, if having platform-specific dependencies or dependencies
#   having no cross-platform support, pipenv may install dependencies that don't work, or not
#   install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/
xia0nan added a commit to xia0nan/Gatech-CS6457 that referenced this issue Aug 23, 2021
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
.python-version

# pipenv
#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
#   However, in case of collaboration, if having platform-specific dependencies or dependencies
#   having no cross-platform support, pipenv may install dependencies that don't work, or not
#   install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/
build/
bin/
Nestor-Ojeda added a commit to danialucasm/proyectoInfo that referenced this issue Aug 25, 2021
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
.python-version

# pipenv
#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
#   However, in case of collaboration, if having platform-specific dependencies or dependencies
#   having no cross-platform support, pipenv may install dependencies that don't work, or not
#   install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/
@fcovatti
Copy link

@fcovatti fcovatti commented Sep 1, 2021

This has been 4 years long. From the security perspective there is to me a trade-off that I would like to understand if it makes sense for you as well:

  1. Commit the Pipfile.lock makes sure that a specific hash is used for the package that will be installed, that is a good security functionality
  2. But not committing the Pipfile.lock and using inclusive or exclusive version comparisons can make the package be always upgraded to latest version that will potentially have security fixes.

Which ones do you consider better?

Here is my reasoning on option 2):
For example, in one project I have in the Pipfile:
Django = "<4.0,>=3.2"

But on the Pipfile.lock it was locked to 3.2.4 hash.

But there is a security vulnerability fixed in 3.2.5 Django version.
So due to committing the Pipfile.lock I was using a package version with a known CVE vulnerability.
If I do not version control the Pipfile.lock my Django version would be bumped to 3.2.7 automatically on the next CI/CD build of my latest docker image.

I came from the basic requirements.txt world and have a preference for option 2), could you please say what I may be missing here in terms of security for the recommendation to commit it?

@gsemet
Copy link
Contributor

@gsemet gsemet commented Sep 1, 2021

From a security point of view you should always take what is in the lock file, never try to update it silently. Remember project might not follow semver correctly. So what if 3.2.7 introduces itself a huge new bug…

so, update only on purpose with pipenv update. Maybe you want something like « pipenv update —bugfix-only », but again, you expect people to follow semver which is sadly not always the case

@fcovatti
Copy link

@fcovatti fcovatti commented Sep 1, 2021

Totally understand, but I am trusting that my unit tests and system tests will catch a potential bug as I version control the build container with the 3.2.7 package and test the release before going into production.

I understand that if the network between my CI/CD server and pypi is compromised as I am not verifying the hash I could potentially install a insecure package.

I am just worry that not updating the versions and let it go to production with a known CVE due to the lock file and trusting developers will handle the pipenv update could be a potential security risk.

It seems the only way to go is to commit the pipfile.lock and avoid to create a release with known CVEs by using security scan of potential vulnerabilities in the pipenv.lock packages via another application. This could be a potential improvement in pipenv to inform that are packages in the pipfile.lock with known vulnerabilities.

@gsemet
Copy link
Contributor

@gsemet gsemet commented Sep 1, 2021

do a pipenv update in your CI only for unit test if you really want it. Would be great to have something like pipenv update --only-cve-updates.

You can give a try to https://github.com/pyupio/safety to check if you have a known vulnerability in your tree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet