Skip to content

Support using --index to refer to index names#17455

Draft
EliteTK wants to merge 23 commits intomainfrom
tk/index-by-name
Draft

Support using --index to refer to index names#17455
EliteTK wants to merge 23 commits intomainfrom
tk/index-by-name

Conversation

@EliteTK
Copy link
Contributor

@EliteTK EliteTK commented Jan 13, 2026

Summary

Implement #13974.

Some of the code was inspired by existing code from uv_distribution::metadata::lowering which handles resolving index name references within pyproject.toml.

This PR adds a new IndexArg type to uv_distribution_types which holds information about an index as passed on the CLI. Specifically the current Index type has a non-optional url field. IndexArg solves this by including two variants, one for the Index type and one for an UnresolvedIndex which holds the IndexName. This means that all the CLI handling can use IndexArg and then the standard resolve functions can transform these IndexArgs into Indexes. This allows you to reference indexes by name if they are already defined somewhere else.

Currently support for --default-index by name is not included.

Error handling

The initial code made use of Result to propagate errors regarding resolution (i.e. no index found for that name). But I added a commit which replaces this with an early exit(2) as that seems to be the style used by other resolve style code.

The new Resolve trait

PipOptions required a bunch of changes as it was using the From trait initially but index resolution requires additional metadata for the names which could have been passed as a tuple except for the fact that it's ugly and also breaks the orphan rule so it wasn't actually an option. Another alternative would be to do the resolution earlier but that seemed like an excessive amount of boilerplate, so instead a new trait was used.

Argument cross-references

You can't resolve an index which you passed on the CLI (e.g. --index foo=bar --index foo) as I felt that didn't make sense as a feature even if it could have been supported. Maybe there is a use for it though? Not sure.

To Do

  • Index priority - it's still using the resolution priority which means that indexes can only come from the workspace pyproject.toml in workspaces. I haven't finished figuring out the best way to get this working yet.
  • Index writeback - Once index priority is working, the Workspace origin will be used here and indexes with Project or Workspace origin won't get written back.

uv tool

Passing --index to uv tool even if it's the only index doesn't guarantee that it will use only that index for the tool. Moreover, due to the fact that --index <name> works identically to --index <name>=..., the side effect is that the index will appear twice in the uv-receipt.toml. I think that any changes in this area are probably outside of the scope of this change.

Test Plan

Tests currently need updating and changing to deal with the other changes to this PR.

@EliteTK EliteTK temporarily deployed to uv-test-registries January 13, 2026 23:40 — with GitHub Actions Inactive
@EliteTK EliteTK force-pushed the tk/index-by-name branch 3 times, most recently from 747f88f to 6355b20 Compare January 15, 2026 19:35
Comment on lines 11482 to 11441
uv_snapshot!(filters, context.add().arg("iniconfig").arg("--index").arg("test-index").current_dir(&child_dir), @r"
success: false
exit_code: 2
----- stdout -----

----- stderr -----
error: Request failed after 3 retries
Caused by: Failed to fetch: `http://[LOCALHOST]/iniconfig/`
Caused by: HTTP status server error (503 Service Unavailable) for url (http://[LOCALHOST]/iniconfig/)
");

//let lock = fs_err::read_to_string(context.temp_dir.join("uv.lock"))?;
//insta::with_settings!({
// filters => filters,
//}, {
// assert_snapshot!(lock, @r#"
//"#);
//});
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to investigate what happens if you don't pass --index.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I am going to leave this as is for now.

Although I think there's an argument to be made that if you're adding an index and it matches identically an index in the workspace (regardless of if you used the name, or spelled it out) that maybe we shouldn't mirror it in the child.

But the way things like #17610 work seems like maybe justification for leaving it as is.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remove the commented out code, and/or leave a TODO comment?

Comment on lines +11279 to +11289
[[tool.uv.index]]
name = "test-index"
url = "https://pypi-proxy.fly.dev/simple"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if we want this. But also, need to check what happens normally.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, so index name references in [tool.uv.sources] are resolved in "project" then "workspace" order.

But when it comes to picking an index to use for a package without a specified source, we pick "workspace" then "project".

I'm not really sure what the correct answer is here. I'm going to leave it as is for now...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer workspace root only, to avoid the problem of workspace root and member declaration getting out of sync.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@konstin you'd prefer workspace root only in what sense?

Isn't @EliteTK describing an existing behavior?

@EliteTK EliteTK force-pushed the tk/index-by-name branch 2 times, most recently from 4c9003d to a4070c9 Compare January 19, 2026 15:15
@EliteTK EliteTK marked this pull request as ready for review January 19, 2026 15:16
@EliteTK EliteTK requested a review from konstin January 19, 2026 15:16
let filters: Vec<_> = context
.filters()
.into_iter()
.chain([(r"http://127\.0\.0\.1:\d+", "[PARENT_INDEX]")])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: regex::escape the mock server URL - though that filter doesn't seem to be used?

Comment on lines 11482 to 11441
uv_snapshot!(filters, context.add().arg("iniconfig").arg("--index").arg("test-index").current_dir(&child_dir), @r"
success: false
exit_code: 2
----- stdout -----

----- stderr -----
error: Request failed after 3 retries
Caused by: Failed to fetch: `http://[LOCALHOST]/iniconfig/`
Caused by: HTTP status server error (503 Service Unavailable) for url (http://[LOCALHOST]/iniconfig/)
");

//let lock = fs_err::read_to_string(context.temp_dir.join("uv.lock"))?;
//insta::with_settings!({
// filters => filters,
//}, {
// assert_snapshot!(lock, @r#"
//"#);
//});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remove the commented out code, and/or leave a TODO comment?

@konstin konstin requested a review from zanieb January 20, 2026 09:30
@zanieb zanieb added the breaking A breaking change label Jan 21, 2026
@zanieb zanieb added this to the v0.10.0 milestone Jan 21, 2026
@zanieb
Copy link
Member

zanieb commented Jan 21, 2026

Moreover, due to the fact that --index works identically to --index =..., the side effect is that the index will appear twice in the uv-receipt.toml

This seems problematic?

@zanieb
Copy link
Member

zanieb commented Jan 21, 2026

This means that if you reference a workspace index in a package add command, it will get copied to the package. This behaviour matches what would happen if you pass the full index including a name. I wasn't completely sure what to do here, but I feel as if it might be confusing to have them behave differently.

Is it necessary for it to be copied into the child package? Or if you added it to the child package without doing so would the parent index still be used properly?

@zanieb
Copy link
Member

zanieb commented Jan 21, 2026

When resolving the name of an index within a pyproject.toml, the lowering code resolves it by looking at the current file first, and then the workspace file.

On the other hand, when resolving which index to use for a package which has no specified index, the resolution takes the workspace index first and then the package's indexes next.

So --name resolves to an index in the opposite order in which index priority is applied during resolution?

@EliteTK
Copy link
Contributor Author

EliteTK commented Jan 21, 2026

Moreover, due to the fact that --index works identically to --index =..., the side effect is that the index will appear twice in the uv-receipt.toml

This seems problematic?

This doesn't cause any issues at the moment but we could de-duplicate things such that only the first index for each unique URL is kept in the order they would have been kept before. I don't think the name is itself used for anything? That being said, this seems like an issue with how uv tool install works as you can also replicate the same receipt (with a duplicate entry) in other ways.

I'm happy to fix it though, do you want me to do it as part of this PR?

This means that if you reference a workspace index in a package add command, it will get copied to the package. This behaviour matches what would happen if you pass the full index including a name. I wasn't completely sure what to do here, but I feel as if it might be confusing to have them behave differently.

Is it necessary for it to be copied into the child package? Or if you added it to the child package without doing so would the parent index still be used properly?

It's definitely an option not to add it to the child package. The right index will be picked up as the name resolution order is project then workspace, so if the project doesn't have an index with that name, the workspace one will be picked.

But there is this issue: #17610, which is somewhat related.

I feel that if we were to fix this, we should fix this in a way which is agnostic to whether the index was mentioned by name, or fully written out with a conflicting name. Although we can also just mark the index as having originated by name and then not write it back in those situations. But we have to be careful as you can specify a --default-index by name but the original index may have been a non-default one, in which case it would be wrong to not write it back.

When resolving the name of an index within a pyproject.toml, the lowering code resolves it by looking at the current file first, and then the workspace file.
On the other hand, when resolving which index to use for a package which has no specified index, the resolution takes the workspace index first and then the package's indexes next.

So --name resolves to an index in the opposite order in which index priority is applied during resolution?

No, --index <name> resolves in the order of index priority during resolution, which I felt made sense initially, but the approach used by the lowering code (still uses priority, but it puts package indexes in front of the workspace indexes, which is the reverse) also makes sense.

@zanieb
Copy link
Member

zanieb commented Jan 21, 2026

@EliteTK can we ship this under a preview flag so we don't need to wait until 0.10 to merge?

@zanieb
Copy link
Member

zanieb commented Jan 21, 2026

No, --index resolves in the order of index priority during resolution, which I felt made sense initially, but the approach used by the lowering code (still uses priority, but it puts package indexes in front of the workspace indexes, which is the reverse) also makes sense.

I don't understand how to reconcile this comment with your original one?

Comment on lines 188 to 192
/// Resolve [`IndexArg`]s into [`Index`]es using indexes defined on the
/// filesystem and combine the `default_index` and `index` into one vector.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this could be a bit less descriptive of the code and more of the intent? LIke

Suggested change
/// Resolve [`IndexArg`]s into [`Index`]es using indexes defined on the
/// filesystem and combine the `default_index` and `index` into one vector.
/// Resolve index options into a prioritized list of [`Index`].
///
/// Index name references are resolved to index URLs using values from the file system.
///
/// The default index is placed after all other indexes.

Comment on lines +205 to +230
Err(error) => {
eprintln!("error: {error}");
std::process::exit(2);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know why we do this in the other code? It's pretty janky. It doesn't respect our Printer, it doesn't print the error chain, etc.

@EliteTK
Copy link
Contributor Author

EliteTK commented Jan 21, 2026

@EliteTK can we ship this under a preview flag so we don't need to wait until 0.10 to merge?

Yeah, no objections here.

No, --index resolves in the order of index priority during resolution, which I felt made sense initially, but the approach used by the lowering code (still uses priority, but it puts package indexes in front of the workspace indexes, which is the reverse) also makes sense.

I don't understand how to reconcile this comment with your original one?

I will adjust it for clarity, but the section you referenced mentions that this PR uses the "second strategy" which refers to:

"On the other hand, when resolving which index to use for a package which has no specified index, the resolution takes the workspace index first and then the package's indexes next."

pub struct ResolveIndexArgError(IndexName);

impl IndexArg {
/// Parse an Index passed on the command line
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Parse an Index passed on the command line
/// Parse an index passed on the command line

}
}

pub fn try_resolve<I>(self, indexes: I) -> Result<Index, ResolveIndexArgError>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could probably use a doc comment

})
}

pub fn indexes(&self) -> impl Iterator<Item = Index> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably have a doc comment

self.index
.iter()
.flatten()
.cloned()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to clone? Can the caller clone?

----- stderr -----
warning: Relative paths passed to `--index` or `--default-index` should be disambiguated from index names (use `[PREFIX]test-index`). Support for ambiguous values will be removed in the future
error: Directory not found for index: file://[TEMP_DIR]/test-index
error: Could not find an index named `test-index`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want a follow-up issue to enumerate searched source files here.

Comment on lines -11132 to -11133
warning: Relative paths passed to `--index` or `--default-index` should be disambiguated from index names (use `[PREFIX]test-index`). Support for ambiguous values will be removed in the future
error: Directory not found for index: file://[TEMP_DIR]/test-index
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have we considered continuing to support the current behavior if ./test-index exists and / or if the name test-index doesn't exist?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hadn't, but I would say it should be only if the directory exists and the index doesn't. Otherwise you're going to get a confusing error whenever you mis-type an index or mis-type a directory name.

But honestly I am mildly against this kind of special casing.

Rather simple to implement though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think such special casing generally makes transitioning easier because then the change is breaking for less people? I don't think it's an ideal end state, but the mistake was when we allowed foo to be treated as a relative path without ./foo in the first place.

@zanieb
Copy link
Member

zanieb commented Jan 21, 2026

This will need some documentation updates, which might help explain the behavior?

The code looks fine but I only sort of understand how behaves, tbh.

@EliteTK EliteTK marked this pull request as draft January 30, 2026 14:32
EliteTK and others added 8 commits February 5, 2026 13:16
It's only used by the cli parsing, and involves a silly
destructuring/restructuring dance to set the `default` field.

Making it a function also avoids needing to explicitly set the origin.
Original code by Zanie with the temporary jury-rigging done by Tom.
These aren't used since IndexArg must only originate from the CLI.
We _could_ print the index name before resolution but the goal here
isn't to verify the integrity of the resolve function (to ensure that it
did actually do the equality comparison) but rather just to troubleshoot
which index it resolved to (in which case the name is embedded in the
final index).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking A breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants