Skip to content

Commit

Permalink
Support remote https:// requirements files (#1332) (#2081)
Browse files Browse the repository at this point in the history
## Summary

Allow using http(s) urls for constraints and requirements files handed
to the CLI, by handling paths starting with `http://` or `https://`
differently. This allows commands for such as: `uv pip install -c
https://raw.githubusercontent.com/apache/airflow/constraints-2.8.1/constraints-3.8.txt
requests`.

closes #1332

## Test Plan

Testing install using a `constraints.txt` file hosted on github in the
airflow repository:

https://github.com/jannisko/uv/blob/fbdc2eba8e5e8fb4160baae495daff8fb48df13f/crates/uv/tests/pip_install.rs#L1440-L1484

## Advice Needed

- filesystem/http dispatch is implemented at a relatively low level (at
`crates/uv-fs/src/lib.rs#read_to_string`). Should I change some naming
here so it is obvious that the function is able to dispatch?
- I kept the CLI argument for -c and -r as a PathBuf, even though now it
is technically either a path or a url. We could either keep this as is
for now, or implement a new enum for this case? The enum could then
handle dispatch to files/http.
- Using another abstraction layer like
https://docs.rs/object_store/latest/object_store/ for the
files/urls/[s3] could work as well, though I ran into a bug during
testing which I couldn't debug
  • Loading branch information
jannisko committed Mar 6, 2024
1 parent 2ebcef9 commit 71626e8
Show file tree
Hide file tree
Showing 16 changed files with 430 additions and 145 deletions.
17 changes: 17 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ members = ["crates/*"]
exclude = [
"scripts",
# Needs nightly
"crates/uv-trampoline"
"crates/uv-trampoline",
]
resolver = "2"

Expand All @@ -22,6 +22,7 @@ anyhow = { version = "1.0.80" }
async-compression = { version = "0.4.6" }
async-channel = { version = "2.2.0" }
async-trait = { version = "0.1.77" }
async-recursion = { version = "1.0.5" }
async_http_range_reader = { version = "0.7.0" }
async_zip = { git = "https://github.com/charliermarsh/rs-async-zip", rev = "d76801da0943de985254fc6255c0e476b57c5836", features = ["deflate"] }
base64 = { version = "0.21.7" }
Expand Down
6 changes: 6 additions & 0 deletions crates/requirements-txt/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,21 @@ workspace = true
[dependencies]
pep440_rs = { path = "../pep440-rs", features = ["rkyv", "serde"] }
pep508_rs = { path = "../pep508-rs", features = ["rkyv", "serde", "non-pep508-extensions"] }
uv-cache = { path = "../uv-cache" }
uv-client = { path = "../uv-client" }
uv-fs = { path = "../uv-fs" }
uv-normalize = { path = "../uv-normalize" }
uv-warnings = { path = "../uv-warnings" }

async-recursion = { workspace = true }
fs-err = { workspace = true }
once_cell = { workspace = true }
regex = { workspace = true }
reqwest = { workspace = true }
reqwest-middleware = { workspace = true }
serde = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true }
tracing = { workspace = true }
unscanny = { workspace = true }
url = { workspace = true }
Expand Down

0 comments on commit 71626e8

Please sign in to comment.