(ruby-modules/gem): (refactor)

[nyarly]

Motivation for this change

Rubygems with a git source use leaveDotGit across the board. The upshot is that many unrelated changes to the upstream git repo (even on a gem that is referred to by ref) will change the fixed output digest and fail builds.

Things done

This PR removes leaveDotGit and replaces it with a series of git commands to recreate the git repo locally during build. The reason for this is that many gems use some variation of git ls-files rather than specifically enumerating their manifests. This is plainly wrong, but that battle is well lost.

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Assured whether relevant documentation is up to date
• Fits CONTRIBUTING.md.

---

I have used this branch to build a relatively complicated gemset.nix that includes git gems. Attempts to use nox-review exhausted the disk on my laptop.

[zimbatm]

I like this idea. Do you know if it's bit-reproducible?

You might be able to reduce a few lines by using --author and also add --date= just in case

[alyssais]

You might be able to reduce a few lines by using --author and also add --date= just in case

Those flags only affect the author date, IIRC. You can’t set the committer through flags, although you can use environment variables.

But, shouldn’t git ls-files report all files tracked by git? In that case, is a commit even necessary?

[alyssais]

I like this idea. Do you know if it's bit-reproducible?

It could definitely change between git versions. There’s a lot of stuff like initial repository config and sample hooks in a brand new repo.

Something like this could help:

nixpkgs/pkgs/build-support/fetchgit/nix-prefetch-git

Lines 245 to 246 in c514728

	rm -rf .git/logs/ .git/hooks/ .git/index .git/FETCH_HEAD .git/ORIG_HEAD \
	.git/refs/remotes/origin/HEAD .git/config

Aside from those, and possibly some other files like info/exclude, I think this would be bit-reproducible, unless the git repository format changed. There’s not really anything we could do about that, and it would already be a big problem for Nixpkgs.

[nyarly]

Excellent notes. I've removed the commit which removes the requirement for config and added that rm. Retesting on my problem repo still works.

[alyssais]

Oh, really, you can remove .git/index? What's left in .git after this?

[nyarly]

:) I copied your reference above! I experimented directly, and removing the index clobbers the effect of git add (which is obvious in hindsight). Reversing the order fixes it.

[zimbatm]

what is the purpose of having the files in the staging area?

[Mic92]

Adding this at build time can be useful for git ls-files to work.

[nyarly]

@alyssais @zimbatm Are we good to merge this, or have I missed a lingering concern?