Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Noisy warnings from submodule deinit on git >= 2.21.0 #3293

Closed
ajvondrak opened this issue Nov 19, 2019 · 6 comments · Fixed by #3969
Closed

Noisy warnings from submodule deinit on git >= 2.21.0 #3293

ajvondrak opened this issue Nov 19, 2019 · 6 comments · Fixed by #3969

Comments

@ajvondrak
Copy link

Since Bundler v1.13.0, bundler does an automatic git submodule deinit --all --force, per rubygems/bundler#4717. To this day, bundler still just fires the deinit all the time when :submodules is false: https://github.com/bundler/bundler/blob/f6045fcf93cd3076cdd3d5e15622927c87df2db5/lib/bundler/source/git/git_proxy.rb#L146-L150

Git v2.21.0 started maintaining an invariant where the core.worktree configuration gets removed on git submodule deinit, per the release notes. This was implemented by git/git@898c2e6:

void submodule_unset_core_worktree(const struct submodule *sub)
{
	char *config_path = xstrfmt("%s/modules/%s/config",
				    get_git_common_dir(), sub->name);


	if (git_config_set_in_file_gently(config_path, "core.worktree", NULL))
		warning(_("Could not unset core.worktree setting in submodule '%s'"),
			  sub->path);


	free(config_path);
}

With the above implementation, though, git spits out a warning if the submodule has already been deinit'd.

Here's a minimal setup:

$ git --version
git version 2.24.0
$ git init repo
Initialized empty Git repository in /private/tmp/repo/.git/
$ cd repo
$ git submodule add git@github.com:bundler/bundler.git
Cloning into '/private/tmp/repo/bundler'...
remote: Enumerating objects: 26, done.
remote: Counting objects: 100% (26/26), done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 96811 (delta 7), reused 6 (delta 0), pack-reused 96785
Receiving objects: 100% (96811/96811), 62.82 MiB | 6.81 MiB/s, done.
Resolving deltas: 100% (61721/61721), done.
$ git commit -m 'initial commit'
[master (root-commit) e4f11b7] initial commit
 2 files changed, 4 insertions(+)
 create mode 100644 .gitmodules
 create mode 160000 bundler

We deinit once, no problem:

$ git submodule deinit --all --force
Cleared directory 'bundler'
Submodule 'bundler' (git@github.com:bundler/bundler.git) unregistered for path 'bundler'

When we deinit again, git can't open the lock file, so it spits out warnings:

$ git submodule deinit --all --force # already deinit'd, so this causes an error
Cleared directory 'bundler'
warning: Could not unset core.worktree setting in submodule 'bundler'

What's more, the same warning is given if you never init'd in the first place - like in the fresh git clone (sans --recurse-submodules) that Bundler does:

$ bundle --version
Bundler version 2.0.2
$ mkdir example
$ cd example
$ bundle init
Writing new Gemfile to /private/tmp/example/Gemfile
$ echo 'gem "repo", git: "../repo"' >> Gemfile
$ bundle
Fetching ../repo/.git
error: could not lock config file .git/modules/bundler/config: No such file or directory
warning: Could not unset core.worktree setting in submodule 'bundler'
Could not find gem 'repo' in ../repo (at master@da0e26b).
The source does not contain any versions of 'repo'

Note: the fact that the "repo" example doesn't have a gemspec is immaterial to the git behavior. I encountered this error on a git-based gem (which had submodules, even though I still opted for submodules: false) that installed just fine despite issuing the above deinit warning about core.worktree.

Running the above with some more verbose tracing:

$ GIT_TRACE=1 bundle --verbose
Running `bundle install --verbose` with bundler 2.0.2
Fetching ../repo
18:35:32.144383 git.c:439               trace: built-in: git clone ../repo ~/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/cache/bundler/git/repo-487077bff9f7d5e665d6d031e0ee79c997133134 --bare --no-hardlinks --quiet
18:35:32.162513 run-command.c:663       trace: run_command: unset GIT_DIR; 'git-upload-pack '\''/tmp/example/../repo'\'''
18:35:32.179503 git.c:439               trace: built-in: git upload-pack /tmp/example/../repo
18:35:32.202261 git.c:439               trace: built-in: git rev-parse --verify master
18:35:32.235128 git.c:439               trace: built-in: git fetch --force --quiet --tags ~/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/cache/bundler/git/repo-487077bff9f7d5e665d6d031e0ee79c997133134
18:35:32.236902 run-command.c:663       trace: run_command: unset GIT_PREFIX; 'git-upload-pack '\''~/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/cache/bundler/git/repo-487077bff9f7d5e665d6d031e0ee79c997133134'\'''
18:35:32.255099 git.c:439               trace: built-in: git upload-pack ~/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/cache/bundler/git/repo-487077bff9f7d5e665d6d031e0ee79c997133134
18:35:32.257607 run-command.c:663       trace: run_command: git rev-list --objects --stdin --not --all --quiet --alternate-refs
18:35:32.275425 run-command.c:663       trace: run_command: git rev-list --objects --stdin --not --all --quiet --alternate-refs
18:35:32.288280 git.c:439               trace: built-in: git rev-list --objects --stdin --not --all --quiet --alternate-refs
18:35:32.292688 run-command.c:1616      run_processes_parallel: preparing to run up to 1 tasks
18:35:32.292724 run-command.c:1648      run_processes_parallel: done
18:35:32.292745 run-command.c:663       trace: run_command: git gc --auto --quiet
18:35:32.305775 git.c:439               trace: built-in: git gc --auto --quiet
18:35:32.322476 git.c:439               trace: built-in: git reset --hard da0e26b17a162be3fbe435f502d327405f957e03
18:35:32.343167 git.c:439               trace: built-in: git version
18:35:32.358704 git.c:702               trace: exec: git-submodule deinit --all --force
18:35:32.359357 run-command.c:663       trace: run_command: git-submodule deinit --all --force
18:35:32.417265 git.c:702               trace: exec: git-sh-i18n--envsubst --variables 'usage: $dashless $USAGE'
18:35:32.417862 run-command.c:663       trace: run_command: git-sh-i18n--envsubst --variables 'usage: $dashless $USAGE'
18:35:32.444306 git.c:702               trace: exec: git-sh-i18n--envsubst 'usage: $dashless $USAGE'
18:35:32.444920 run-command.c:663       trace: run_command: git-sh-i18n--envsubst 'usage: $dashless $USAGE'
18:35:32.481299 git.c:439               trace: built-in: git rev-parse --git-dir
18:35:32.497948 git.c:439               trace: built-in: git rev-parse --git-path objects
18:35:32.513447 git.c:439               trace: built-in: git rev-parse -q --git-dir
18:35:32.542893 git.c:439               trace: built-in: git rev-parse --show-prefix
18:35:32.556973 git.c:439               trace: built-in: git rev-parse --show-toplevel
18:35:32.583547 git.c:439               trace: built-in: git submodule--helper deinit --force --all --
error: could not lock config file .git/modules/bundler/config: No such file or directory
warning: Could not unset core.worktree setting in submodule 'bundler'
18:35:32.586970 run-command.c:663       trace: run_command: git config --get-regexp 'submodule.bundler\.'
18:35:32.599742 git.c:439               trace: built-in: git config --get-regexp 'submodule.bundler\.'
Found changes from the lockfile, re-resolving dependencies because the list of sources changed, the dependencies in your gemfile changed, you added a new platform to your gemfile
HTTP GET https://index.rubygems.org/versions
HTTP 206 Partial Content https://index.rubygems.org/versions
Fetching gem metadata from https://rubygems.org/
Could not find gem 'repo' in ../repo (at master@da0e26b).
The source does not contain any versions of 'repo'

Solution: I think that Bundler could probably do something to avoid the git submodule deinit --all --force if there are no submodules in the cloned repo anyway. This keeps Git from issuing confusing warnings on newer versions.

@deivid-rodriguez
Copy link
Member

Thanks for the thorough report. I don't know much about git submodules, but the solution you're proposing seems reasonable, would you be able to create a PR?

@ajvondrak
Copy link
Author

Hm. I've been searching, and I don't know if there's a good way to test whether a submodule has been initialized.

There's a way to loop through all initialized submodules using git submodule foreach, but that changes the working directory to the submodule itself. So I have to do something like this:

diff --git a/lib/bundler/source/git/git_proxy.rb b/lib/bundler/source/git/git_proxy.rb
index 2a4d7138a..98bf591ac 100644
--- a/lib/bundler/source/git/git_proxy.rb
+++ b/lib/bundler/source/git/git_proxy.rb
@@ -146,7 +146,7 @@ module Bundler
             if submodules
               git_retry "submodule update --init --recursive"
             elsif Gem::Version.create(version) >= Gem::Version.create("2.9.0")
-              git_retry "submodule deinit --all --force"
+              git_retry "submodule foreach --quiet 'cd $toplevel; git submodule deinit --force $sm_path'"
             end
           end
         end

This fixes the noisy warnings for me locally. But foreach executes a shell command. So is there a concern about this patch not being cross-platform? Also, I'm not sure how to write an rspec test for the problem/solution. 😅

@deivid-rodriguez
Copy link
Member

I think it should be fine, I think the specific command should be ok for all shells? Maybe we could do git_retry "submodule foreach --quiet 'git -C $toplevel submodule deinit --force $sm_path'" instead, but I'm not sure it makes much of a difference.

Regarding the test, I was writing down some tips for you but I was not sure whether I was in the right track, so I ended up writing the test myself :). It would be something like:

diff --git a/spec/install/gemfile/git_spec.rb b/spec/install/gemfile/git_spec.rb
index 00f8e9662..a3b7def6c 100644
--- a/spec/install/gemfile/git_spec.rb
+++ b/spec/install/gemfile/git_spec.rb
@@ -845,6 +845,26 @@ RSpec.describe "bundle install with git sources" do
     expect(the_bundle).not_to include_gems "has_submodule 1.0"
   end
 
+  it "does not warn when deiniting submodules" do
+    build_git "submodule", "1.0"
+    build_git "has_submodule", "1.0"
+
+    Dir.chdir(lib_path("has_submodule-1.0")) do
+      sys_exec "git submodule add #{lib_path("submodule-1.0")} submodule-1.0"
+      `git commit -m "submodulator"`
+    end
+
+    install_gemfile <<-G
+      git "#{lib_path("has_submodule-1.0")}" do
+        gem "has_submodule"
+      end
+    G
+    expect(err).to be_empty
+
+    expect(the_bundle).to include_gems "has_submodule 1.0"
+    expect(the_bundle).to_not include_gems "submodule 1.0"
+  end
+

@hsbt hsbt transferred this issue from rubygems/bundler Mar 14, 2020
@deivid-rodriguez
Copy link
Member

Hi @ajvondrak, are you still interested in fixing this or should we take it over?

@deivid-rodriguez
Copy link
Member

I transformed the discussion here into a PR: #3969.

@ajvondrak
Copy link
Author

@deivid-rodriguez I know I'm very late to reply, but just wanted to say thanks so much for taking care of this. ❤️ If you couldn't tell by my radio silence, I was not in a place to focus on fixing this. 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants