-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add local registry overlays #13926
base: master
Are you sure you want to change the base?
Add local registry overlays #13926
Conversation
Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @ehuss (or someone else) some time within the next two weeks. Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (
|
crates/cargo-test-support/src/lib.rs
Outdated
if let Some(ref mut p) = self.process_builder { | ||
let env_value = format!("{}={}", url, path); | ||
p.env( | ||
"__CARGO_TEST_PACKAGE_CONFUSION_VULNERABILITY_DO_NOT_USE_THIS", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: The term of art for this kind of vulnerability is "Dependency Confusion". As with so many names in computer science, it's better to have consistency then trying to find a better name. This term was coined in the blog post https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610 which might be worth linking in comments about how users could easily miss use this.
Catching Eric up on some of the context, Arlo and I discussed the questions from #10948 (comment) at office hours on Thursday. Adding this API, exclusively for internal use, seemed like the least hacky solution. |
Thanks for the review! I think this is ready for another look -- we managed to successfully use this for #10948 as discussed. |
/// affect any newly created `PackageRegistry`s. | ||
/// | ||
/// See [`crate::sources::overlay::OverlayConfusionAttack`] for why you shouldn't use this. | ||
pub fn local_overlays(&self) -> RefMut<'_, HashMap<SourceId, PathBuf>> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd hope we can avoid putting this on GlobalContext but its hard to say without reviewing the follow up at the same time. Should we defer this to the follow up PR?
My motivation is that it gets messy adding and removing from the Global Context and if we can directly instantiate things, the easier it will be to maintain.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can give a little more context before we decide whether or not to defer. cargo package
needs the overlay to take effect during a call to ops::compile
. AFAICT there are three possibilities:
- put the overlay configuration in
GlobalContext
- put the overlay configuration in the ephemeral workspace that
cargo package
creates - have
cargo package
do something lower-level thanops::compile
The workspace option has the advantage that the overlay configuration doesn't need to be removed (because the workspace is discarded). It seems like it would just require a little work in resolve_ws
and resolve_ws_with_opts
to get the overlay configuration out of the workspace and insert it into the PackageRegistry
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The workspace seems the most beneficial. Whichever way we go, still seems like it could be split out into a follow up PR
for (key, path) in gctx.local_overlays().iter() { | ||
let overlay_id = SourceId::for_local_registry(path)?; | ||
base.overlays.insert(*key, overlay_id); | ||
} | ||
|
||
if let Ok(overlay) = | ||
gctx.get_env("__CARGO_TEST_DEPENDENCY_CONFUSION_VULNERABILITY_DO_NOT_USE_THIS") | ||
{ | ||
let (url, path) = overlay.split_once('=').ok_or(anyhow::anyhow!( | ||
"invalid overlay format. I won't tell you why; you shouldn't be using it anyway" | ||
))?; | ||
let overlay_id = SourceId::for_local_registry(path.as_ref())?; | ||
base.overlays.insert(SourceId::from_url(url)?, overlay_id); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we move this hack up into the caller? PackageRegistry
is the immediate caller but maybe a layer above that? That would help with the GlobalContext
. The main downside is it would make it more annoying to support __CARGO_TEST_DEPENDENCY_CONFUSION_VULNERABILITY_DO_NOT_USE_THIS
Thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think if we moved the overlay configuration into Workspace
as discussed below then this could be pushed up into resolve_ws
and resolve_ws_with_opts
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just wanted to check in if there is anything you are waiting on me for this change as you've done other work on this PR but not this part yet.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah sorry, I thought this would go in a follow-up (along with moving the overlay configuration to Workspace
). On Monday I'll clean up the commit history and look into doing this too.
src/cargo/sources/config.rs
Outdated
@@ -125,6 +145,22 @@ impl<'gctx> SourceConfigMap<'gctx> { | |||
self.gctx | |||
} | |||
|
|||
fn load_and_overlay( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I feel like and
in a name is a code smell, that you are making a fucntion do unrelated things. Ideally, we find a name that gets at the heart of what distinguishes this from load
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about load_unreplaced
? Because the difference between load
and load_unreplaced
is that load
does source replacement.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Describing something by what it isn't doing is a bit messy.
Your name hints that pub fn load
is load_replaced
. So maybe this could be load_overlaid
. Still not thrilled but probably the best we can do without blocking this further.
btw feel free to clean up your commit history for how you want to present it for review and merge. |
This PR adds (private to cargo internals) support for local registry overlays, in which you can locally pretend to add packages to remote registries; the local packages will have the same source ids as the remote registry that you're overlaying.
There are two ways to set up these overlays: programmatically using
GlobalContext::local_overlays
and through the__CARGO_TEST_PACKAGE_CONFUSION_VULNERABILITY_DO_NOT_USE_THIS
environment variable. You can't set up these overlays with.cargo/config
.The motivation for this is packaging workspaces. When we're packing a workspace, we'd like to be able to pretend (for lockfile generation and verification) that some workspace packages are already published even though they aren't.