How should we identify dependencies?

[b-long]

Hi @ofek

Thanks again for this magnificent project! I have a quick question, which is, how are we suppose to track requirements?

I noticed the requirements.txt file is a little different than I'm used to seeing:

(venv) my-computer:tests blong$ cat requirements.txt 
-e .

I also noticed, I can install pip dependencies using hatch, like so:

(venv) my-computer:tests blong$ hatch install opencv-python
Installing...
Collecting opencv-python
  Using cached opencv_python-3.3.0.10-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Collecting numpy>=1.11.1 (from opencv-python)
  Downloading numpy-1.13.1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (4.5MB)
    100% |████████████████████████████████| 4.6MB 324kB/s 
Installing collected packages: numpy, opencv-python
Successfully installed numpy-1.13.1 opencv-python-3.3.0.10

Does this mean I am supposed to track dependencies (for now) using pip freeze > requirements.txt or is there some better process?

[amjith]

In the future there will be a project file that will track dependencies so you won't have to manually add them to requirements or setup.py. Hopefully in a week or two. I haven't discussed this with Ofek (creator of hatch) yet.

…

On Sep 29, 2017 10:54 AM, "Brian" ***@***.***> wrote: Hi @ofek <https://github.com/ofek> Thanks again for this magnificent project! I have a quick question, which is, how are we suppose to track requirements? I noticed the requirements.txt file is a little different than I'm used to seeing: (venv) my-computer:tests blong$ cat requirements.txt -e . I also noticed, I can install pip dependencies using hatch, like so: (venv) my-computer:tests blong$ hatch install opencv-python Installing... Collecting opencv-python Using cached opencv_python-3.3.0.10-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl Collecting numpy>=1.11.1 (from opencv-python) Downloading numpy-1.13.1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (4.5MB) 100% |████████████████████████████████| 4.6MB 324kB/s Installing collected packages: numpy, opencv-python Successfully installed numpy-1.13.1 opencv-python-3.3.0.10 Does this mean I am supposed to track dependencies (for now) using pip freeze > requirements.txt or is there some better process? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#47>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADAbG60dAGgjKMtzIKu9Sea6WxNfv5nks5snRK-gaJpZM4Po3v4> .

[serathius]

I would propose to integrate with https://github.com/jazzband/pip-tools huge difference in UX

[ofek]

@serathius Yes that is what we will use.

[serathius]

Great. I will start using hatch then.

[winni2k]

It looks like a two-file solution of some kind is coming, although the pipenv format (#14) and a pyproject.toml format (#52) appear to also be in the running?

[malarinv]

Any update on this?

[johnbridstrup]

update?

[brainwane]

@ofek am I right to understand that this is blocked on pypa/pip#988 ?

[ofek]

@brainwane Yes indeed!

[nlhkabu]

As this ticket is blocked by the development of pip's dependency resolver pypa/pip#988, I thought I would mention here that the pip team is looking for help from the community to move forward on that subject.

We need to better understand the circumstances under which the new resolver fails, so are asking for pip users with complex dependencies to:

1. Try the new resolver (use version 20.1, run --unstable-feature=resolver)
2. Break it :P
3. File an issue

You can find more information and more detailed instructions here

Thanks!

[merwok]

It looks like a two-file solution of some kind is coming, although the pipenv format (#14) and a pyproject.toml format (#52) appear to also be in the running?

To avoid any confusion:

• pyproject.toml is what enables a variety of packaging tools (like pip) to work with a variety of build tools (setuptools, flit, poetry, etc.), thanks to the wheel format + PEP 517 + PEP 518. In pyproject.toml the build-requires section only lists the things necessary to call the build tool (example: wheel and setuptools)

• each build tool is free to do what it wants to let the developer define the runtime requirements: a Python script and/or config file for setuptols, a tool-specific metadata section in pyproject.toml for flit, a shell script for your custom tool, a tool-specific metadata section (+ non-standard format for requirements versions) for poetry, pipfile for pipenv, etc.

• in the wheel and source distributions produced by the process, we still get runtime requirements using the requires-dist format of the metadata spec: https://packaging.python.org/specifications/core-metadata/#requires-dist-multiple-use (this has been stable for years, and is not generally seen directly by developers, but runtime inspection tools work with that format)

Some people think it’s time to move the dependencies from a tool-specific format and file into a standardized metadata section of pyproject.toml; other think that tools have different needs and philosophies. This is one long discussion about this: https://discuss.python.org/t/structured-exchangeable-lock-file-format-requirements-txt-2-0/876

[ofek]

FYI, a few things will happen when pip's resolver gets stabilized:

1. I will return to this 🙂
2. there will be a major release with breaking changes like only runs on Python 3.8+, config file will be simplified and in TOML, fewer commands
3. deps will be defined in pyproject.toml along with some extra cool stuff

[brainwane]

Leaving a note here about the beta of the new resolver in pip 20.2 which can be tried now.

[ofek]

@brainwane Yup, thanks! I'm ~60% finished the rewrite, which implements PEP 621 and PEP 631.

[brainwane]

pip 20.3 is now out and has the new dependency resolver on by default; folks following this issue, please see the documentation on how to test and migrate.

[ofek]

Hatch has been rewritten from scratch and now uses PEP 621, see https://ofek.dev/hatch/latest/config/metadata/#dependencies

Waiting on PEP 665: https://ofek.dev/hatch/latest/meta/faq/#libraries-vs-applications