Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fetchgit ignore deepClone and does a shallow clone anyway #179788

Closed
jbgi opened this issue Jul 1, 2022 · 7 comments · Fixed by #252865
Closed

fetchgit ignore deepClone and does a shallow clone anyway #179788

jbgi opened this issue Jul 1, 2022 · 7 comments · Fixed by #252865

Comments

@jbgi
Copy link
Contributor

jbgi commented Jul 1, 2022

Since #130040 deepClone = true; (for fetchgit and fetchFromGitHub) and nix-prefetch-git --deepClone don't work anymore.
ie, when running

cat $(nix-prefetch-git --quiet --deepClone --leave-dotGit https://github.com/NixOS/rfcs 220af8e4f171425d168334f07163ea3db2eb8161 | jq -r .path)/.git/shallow

Actual result :

220af8e4f171425d168334f07163ea3db2eb8161

Expected:

cat: /nix/store/h1rfypna3gfbxnkhza30yjd792wr0id5-rfcs-220af8e/.git/shallow: No such file or directory

Reverting #130040 fix the issue, but the optimization should be kept when deepClone is not used.

@jbgi jbgi changed the title fetchgit ignore deepClone and do a shallow clone anyway fetchgit ignore deepClone and does a shallow clone anyway Jul 1, 2022
@Atemu
Copy link
Member

Atemu commented Jul 1, 2022

Could you give a real-world scenario for deepClone? All the ones in Nixpkgs are broken, with or without the optimisation.

AFAICT your reproducer merely checks whether a clone is sparse?

@jbgi
Copy link
Contributor Author

jbgi commented Jul 3, 2022

My real-world scenario is for a short-lived, local only derivation where the broken aspect (non-determinism) of deepClone are ok.

My issue is that the doc say:

If deepClone is set to true, the entire repository is cloned as opposing to just creating a shallow clone.

but my reproducer shows that this is not the case: --deepClone produce a shallow clone. This should either be fixed, or --deepClone be removed entirely if we don't want to support it. I don't really care either way, I can use 21.05 for the non (completely) broken deepClone support, and, anyway, my use-case is mostly for some convenience during testing a "merge multiple repos" script, but nothing I can't live without.

@Atemu
Copy link
Member

Atemu commented Jul 4, 2022

What I need is a concrete Nix example snippet that makes use of a property of deepClone and doesn't work without because I don't know what deepClone is useful for.
I need to be able to understand what the script did before my patch, why it's different now and how that affects real-world deepClone usage.

Relying on a stable version is not a good idea as that will be deprecated in half a year or so and will likely break in the future. My change should also not have caused any regressions in the real world.

Given that it's taken this long for someone to find out though, I also question why we even need deepClone. I also don't really understand why you'd need it in the first place. It'd be great if you, as a real-world consumer of that feature, could enlighten me of that

@jbgi
Copy link
Contributor Author

jbgi commented Jul 4, 2022

Well the point of using deepClone is to retain the full git history, my use case is merging multiple repos into one, while preserving history, which is why I needed deepClone to work (though I could work around it by not doing this in a nix derivation). It's here :
https://github.com/input-output-hk/cardano-world/blob/prepare-monorepo/nix/cardano/prepare-mono-repo/default.nix

#130040 force use of --depth=1 (shallow clone), even if a deepClone is asked for.

@wcole3
Copy link

wcole3 commented Dec 5, 2022

Bumping to see if anyone has workarounds for this. Some projects make use of the contents in the .git folder and without deep clones they cannot be built cleanly. An example of a repo that makes use of the .git folder is https://github.com/PX4/PX4-Autopilot

emily-is-my-username added a commit to emily-is-my-username/nixpkgs that referenced this issue Jun 30, 2023
@emily-is-my-username
Copy link
Contributor

emily-is-my-username commented Jun 30, 2023

@wcole3 Disabling the optimized shallow clone when --deepClone is explicitly requested worked for me (patch here)

@infinisil
Copy link
Member

@emily-is-my-username I created a PR from your patch! I hope you don't mind: #252865

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants