Skip to content

Commit

Permalink
Respect submodule update=none strategy in .gitmodules
Browse files Browse the repository at this point in the history
Git lets users define the default update/checkout strategy for a submodule
by setting the `submodule.<name>.update` key in `.gitmodules` file.

If the update strategy is `none`, the submodule will be skipped during
update. It will not be fetched and checked out:

1. *foo* is a big git repo

```
/tmp $ git init foo
Initialized empty Git repository in /tmp/foo/.git/
/tmp $ dd if=/dev/zero of=foo/big bs=1000M count=1
1+0 records in
1+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.482087 s, 2.2 GB/s
/tmp $ git -C foo add big
/tmp $ git -C foo commit -m 'I am big'
[main (root-commit) 84fb533] I am big
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 big
```

2. *bar* is a repo with a big submodule with `update=none`

```
/tmp $ git init bar
Initialized empty Git repository in /tmp/bar/.git/
/tmp $ git -C bar submodule add file:///tmp/foo foo
Cloning into '/tmp/bar/foo'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 1 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), 995.50 KiB | 338.00 KiB/s, done.
/tmp $ git -C bar config --file .gitmodules submodule.foo.update none
/tmp $ cat bar/.gitmodules
[submodule "foo"]
        path = foo
        url = file:///tmp/foo
        update = none
/tmp $ git -C bar commit --all -m 'I have a big submodule with update=none'
[main (root-commit) 6c355ea] I have a big submodule not updated by default
 2 files changed, 4 insertions(+)
 create mode 100644 .gitmodules
 create mode 160000 foo
```

3. *baz* is a clone of *bar*, notice *foo* submodule gets skipped

```
/tmp $ git clone --recurse-submodules file:///tmp/bar baz
Cloning into 'baz'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), done.
Submodule 'foo' (file:///tmp/foo) registered for path 'foo'
Skipping submodule 'foo'
/tmp $ git -C baz submodule update --init
Skipping submodule 'foo'
/tmp $
```

Cargo, on the other hand, ignores the submodule update strategy set in
`.gitmodules` properties when updating dependencies. Such behavior can
be considered against the wish of the crate publisher.

4. *bar* is now a lib with a big submodule with update disabled

```
/tmp $ cargo init --lib bar
     Created library package
/tmp $ git -C bar add .
/tmp $ git -C bar commit -m 'I am a lib with a big submodule but update=none'
[main eb07cf7] I am a lib with a big submodule but update=none
 3 files changed, 18 insertions(+)
 create mode 100644 .gitignore
 create mode 100644 Cargo.toml
 create mode 100644 src/lib.rs
/tmp $
```

5. *qux* depends on *bar*, notice *bar*'s submodules are fetched

```
/tmp $ cargo init qux && cd qux
     Created binary (application) package
/tmp/qux $ echo -e '[dependencies.bar]\ngit = "file:///tmp/bar"' >> Cargo.toml
/tmp/qux $ time cargo update
    Updating git repository `file:///tmp/bar`
    Updating git submodule `file:///tmp/foo`

real    0m22.182s
user    0m20.402s
sys     0m1.714s
/tmp/qux $
```

Fix it by checking if a Git repository submodule should be updated when
cargo processes dependencies.

6. With the change applied, submodules with `update=none` are skipped

```
/tmp/qux $ cargo cache -a > /dev/null
/tmp/qux $ time ~/src/cargo/target/debug/cargo update
    Updating git repository `file:///tmp/bar`
    Skipping git submodule `file:///tmp/foo`

real    0m0.029s
user    0m0.021s
sys     0m0.008s
/tmp/qux $
```

Fixes rust-lang#4247.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
  • Loading branch information
jsitnicki authored and Hezuikn committed Sep 22, 2022
1 parent dd588b1 commit c380adc
Show file tree
Hide file tree
Showing 3 changed files with 67 additions and 0 deletions.
1 change: 1 addition & 0 deletions crates/cargo-test-support/src/compare.rs
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,7 @@ fn substitute_macros(input: &str) -> String {
("[OWNER]", " Owner"),
("[MIGRATING]", " Migrating"),
("[EXECUTABLE]", " Executable"),
("[SKIPPING]", " Skipping"),
];
let mut result = input.to_owned();
for &(pat, subst) in &macros {
Expand Down
12 changes: 12 additions & 0 deletions src/cargo/sources/git/utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -370,6 +370,18 @@ impl<'a> GitCheckout<'a> {
anyhow::format_err!("non-utf8 url for submodule {:?}?", child.path())
})?;

// Skip the submodule if the config says not to update it.
if child.update_strategy() == git2::SubmoduleUpdate::None {
cargo_config.shell().status(
"Skipping",
format!(
"git submodule `{}` due to update strategy in .gitmodules",
url
),
)?;
return Ok(());
}

// A submodule which is listed in .gitmodules but not actually
// checked out will not have a head id, so we should ignore it.
let head = match child.head_id() {
Expand Down
54 changes: 54 additions & 0 deletions tests/testsuite/git.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1023,6 +1023,60 @@ Caused by:
.run();
}

#[cargo_test]
fn dep_with_skipped_submodule() {
// Ensure we skip dependency submodules if their update strategy is `none`.
let qux = git::new("qux", |project| {
project.no_manifest().file("README", "skip me")
});

let bar = git::new("bar", |project| {
project
.file("Cargo.toml", &basic_manifest("bar", "0.0.0"))
.file("src/lib.rs", "")
});

// `qux` is a submodule of `bar`, but we don't want to update it.
let repo = git2::Repository::open(&bar.root()).unwrap();
git::add_submodule(&repo, qux.url().as_str(), Path::new("qux"));

let mut conf = git2::Config::open(&bar.root().join(".gitmodules")).unwrap();
conf.set_str("submodule.qux.update", "none").unwrap();

git::add(&repo);
git::commit(&repo);

let foo = project()
.file(
"Cargo.toml",
&format!(
r#"
[project]
name = "foo"
version = "0.0.0"
authors = []
[dependencies.bar]
git = "{}"
"#,
bar.url()
),
)
.file("src/main.rs", "fn main() {}")
.build();

foo.cargo("build")
.with_stderr(
"\
[UPDATING] git repository `file://[..]/bar`
[SKIPPING] git submodule `file://[..]/qux` [..]
[COMPILING] bar [..]
[COMPILING] foo [..]
[FINISHED] dev [unoptimized + debuginfo] target(s) in [..]\n",
)
.run();
}

#[cargo_test]
fn dep_ambiguous() {
let project = project();
Expand Down

0 comments on commit c380adc

Please sign in to comment.