Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support URL and path dependencies without a package name #313

Closed
charliermarsh opened this issue Nov 3, 2023 · 37 comments
Closed

Support URL and path dependencies without a package name #313

charliermarsh opened this issue Nov 3, 2023 · 37 comments
Assignees
Labels
enhancement New feature or request

Comments

@charliermarsh
Copy link
Member

Apparently this works with pip?

git+https://github.com/pallets/flask.git@refs/pull/5313/head

As opposed to:

flask @ git+https://github.com/pallets/flask.git@refs/pull/5313/head

We could decide not to support these. I don't know if they're spec-compliant.

@charliermarsh charliermarsh added the enhancement New feature or request label Nov 3, 2023
@charliermarsh
Copy link
Member Author

\cc @konstin - do you know if these are spec-compliant? Or legacy?

@konstin
Copy link
Member

konstin commented Nov 3, 2023

They are iirc not part of PEP 621 and PEP 508, but they are supported in many places (pip install https://... makes sense)

@charliermarsh
Copy link
Member Author

Sadly, adds a lot of complexity.

@zanieb
Copy link
Member

zanieb commented Nov 4, 2023

Oh this is the syntax I use every time I install a package from git using pip 😬

I'm not sure if we really need to support it in a requirements file? Unless it's common in the wild. Generally it seems reasonable to have a nice error suggesting inclusion of the package name.

@charliermarsh
Copy link
Member Author

If we support it at all, then it’s not too hard to support it in a requirements file too.

@charliermarsh
Copy link
Member Author

Although, I guess it lets us skip supporting it in the parser. Anyway, we probably do need to support this long-term if it’s allowed in pyproject.toml.

@charliermarsh charliermarsh self-assigned this Nov 4, 2023
@charliermarsh charliermarsh added this to the Feature complete milestone Nov 4, 2023
@charliermarsh
Copy link
Member Author

But yeah we should support them :)

@charliermarsh charliermarsh changed the title Support (or decide not to support) URL dependencies without a package name Support URL dependencies without a package name Nov 4, 2023
@charliermarsh
Copy link
Member Author

@konstin - Do you think we should make name optional on Requirement, or convert Requirement to an enum? Names can only be omitted when the version specific is a URL. Similarly, you can't have extras without a name. Feels like all of this could be encoded in the type system.

@konstin
Copy link
Member

konstin commented Nov 6, 2023

We should add a RequirementsTxtRequirement which allows not having a name (or where name can be a url), i'd keep PEP 508 Requirement as-is.

@charliermarsh charliermarsh removed their assignment Nov 6, 2023
@charliermarsh
Copy link
Member Author

I'll give this another try.

@konstin
Copy link
Member

konstin commented Dec 13, 2023

Would you mind waiting until #587 is merged? Editables will profit from this, but doing it now would clash badly.

@charliermarsh
Copy link
Member Author

Will do!

charliermarsh added a commit that referenced this issue Dec 16, 2023
`pip` supports installing packages without names (e.g.,
`git+https://github.com/pallets/flask.git`), but it doesn't adhere to
the PEP grammar, and we don't yet support it (and may never) (#313).

This PR adds a dedicated error path for such cases, to ensure that we
can give meaningful feedback to the user:

```
error: Couldn't parse requirement in requirements.in position 0 to 18
  Caused by: URL requirement is missing a package name; expected: `package_name @ https://google.com`
https://google.com
^^^^^^^^^^^^^^^^^^
```

Closes #650.
@sfc-gh-jcarroll
Copy link

It seems like this is also an issue for installing a local whl file?

% uv pip install ./streamlit-1.31.1-py2.py3-none-any.whl 
error: Failed to parse `./streamlit-1.31.1-py2.py3-none-any.whl`
  Caused by: Expected package name starting with an alphanumeric character, found '.'
./streamlit-1.31.1-py2.py3-none-any.whl
^

% uv pip install "streamlit @ ./streamlit-1.31.1-py2.py3-none-any.whl"
Resolved 40 packages in 680ms
Downloaded 40 packages in 1.74s
Installed 40 packages in 97ms
...

BTW uv seems awesome!! Thank you!!!

@charliermarsh
Copy link
Member Author

Yup, all the same issue! Although we can probably support local wheels "trivially" since the file name is required to be encoded in the wheel.

@edgarrmondragon
Copy link
Contributor

This is probably the biggest blocker to experimenting with uv in Meltano since dependencies of the type git+https://... are very widely used by plugins.

@jeremander
Copy link

Loving uv so far!

Will supporting this also allow the very basic uv pip install . (non-editable) to work?

@charliermarsh
Copy link
Member Author

Yup!

@charliermarsh
Copy link
Member Author

Publicly committing to this one.

@charliermarsh charliermarsh self-assigned this Mar 18, 2024
@dadokkio
Copy link

I've a similar issue with a requirements.txt file that contains a package installed as:

-e git+https://github.com/xxx/package.git@commit_id#egg=package package

at the moment uv pip install fails because -e seems to work only for local files
error: Unsupported URL (expected a `file://` scheme) in requirements.txt

Will this syntax be supported? There is an alternative way to install a specific package in requirements in current folder from a github repo?

Thanks

@charliermarsh
Copy link
Member Author

You can do package @ git+https://github.com/xxx/package.git@commit_id, right?

@dadokkio
Copy link

Yes, but in that case it'll be installed under python path and not under current folder.. also with standard pip the behavior is different

@charliermarsh
Copy link
Member Author

We don't support editable installs for Git and URL dependencies. I'd suggest just cloning the repo and installing it as a file-based editable install (-e ./package).

@charliermarsh
Copy link
Member Author

Okay, this is now in review.

charliermarsh added a commit that referenced this issue Mar 21, 2024
## Summary

First piece of #313. In order to
support unnamed requirements, we need to be able to parse them in
`requirements-txt`, which in turn means that we need to introduce a new
type that's distinct from `pep508::Requirement`, given that these
_aren't_ PEP 508-compatible requirements.

Part of: #313.
charliermarsh added a commit that referenced this issue Mar 21, 2024
## Summary

This PR enables `uv pip install` to accept unnamed requirements, as long
as the requirement ends with the wheel or source distribution archive
name. For example: `cargo run pip install
~/Downloads/anyio-4.3.0.tar.gz`.

In subsequent PRs, I'll expand the scope of supported archives and
patterns.

Part of: #313.
charliermarsh added a commit that referenced this issue Mar 21, 2024
## Summary

For example: `cargo run pip install .`

The strategy taken here is to attempt to extract the package name from
the distribution without executing the PEP 517 build steps. We could
choose to do that in the future if this proves lacking, but it adds
complexity.

Part of: #313.
charliermarsh added a commit that referenced this issue Mar 21, 2024
## Summary

This PR ensures that if a package is already satisfied by the current
environment, we don't bother resolving the named requirement.

Part of: #313.

## Test Plan

- `cargo run pip install ./scripts/editable-installs/black_editable`
- `cargo run pip install black --verbose`
charliermarsh added a commit that referenced this issue Mar 21, 2024
## Summary

In `pip uninstall`, we shouldn't need to resolve unnamed requirements,
since we already index packages in `site-packages` by their URL.

This also changes `uninstall` to ignore editables, which matches pip's
behavior.

Part of: #313.

## Test Plan

Run `cargo run pip install ./scripts/editable-installs/black_editable`,
followed by each of the following:

- `cargo run pip uninstall ./scripts/editable-installs/black_editable`
- `cargo run pip uninstall black`
- `cargo run pip uninstall ./scripts/editable-installs/black_editable
black`
charliermarsh added a commit that referenced this issue Mar 21, 2024
@charliermarsh
Copy link
Member Author

These have all merged. Will be supported in the next release.

@charliermarsh charliermarsh unpinned this issue Mar 22, 2024
@franciscoafonsoo
Copy link

Thank you so much for making this, makes a world of a difference !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests