Permalink
5 comments
on commit
sign in to comment.
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Add fetchgit builtin
The function builtins.fetchgit fetches Git repositories at evaluation time, similar to builtins.fetchTarball. (Perhaps the name should be changed, being confusing with respect to Nixpkgs's fetchgit function, with works at build time.) Example: (import (builtins.fetchgit git://github.com/NixOS/nixpkgs) {}).hello or (import (builtins.fetchgit { url = git://github.com/NixOS/nixpkgs-channels; rev = "nixos-16.03"; }) {}).hello Note that the result does not contain a .git directory.
- Loading branch information
Showing
with
78 additions
and 1 deletion.
- +77 −0 src/libexpr/primops/fetchgit.cc
- +1 −1 src/libstore/download.cc
@@ -0,0 +1,77 @@ | ||
#include "primops.hh" | ||
#include "eval-inline.hh" | ||
#include "download.hh" | ||
#include "store-api.hh" | ||
|
||
namespace nix { | ||
|
||
static void prim_fetchgit(EvalState & state, const Pos & pos, Value * * args, Value & v) | ||
{ | ||
// FIXME: cut&paste from fetch(). | ||
if (state.restricted) throw Error("‘fetchgit’ is not allowed in restricted mode"); | ||
|
||
std::string url; | ||
std::string rev = "master"; | ||
|
||
state.forceValue(*args[0]); | ||
|
||
if (args[0]->type == tAttrs) { | ||
|
||
state.forceAttrs(*args[0], pos); | ||
|
||
for (auto & attr : *args[0]->attrs) { | ||
string name(attr.name); | ||
if (name == "url") | ||
url = state.forceStringNoCtx(*attr.value, *attr.pos); | ||
else if (name == "rev") | ||
rev = state.forceStringNoCtx(*attr.value, *attr.pos); | ||
else | ||
throw EvalError(format("unsupported argument ‘%1%’ to ‘fetchgit’, at %3%") % attr.name % attr.pos); | ||
} | ||
|
||
if (url.empty()) | ||
throw EvalError(format("‘url’ argument required, at %1%") % pos); | ||
|
||
} else | ||
url = state.forceStringNoCtx(*args[0], pos); | ||
|
||
if (!isUri(url)) | ||
throw EvalError(format("‘%s’ is not a valid URI, at %s") % url % pos); | ||
|
||
Path cacheDir = getCacheDir() + "/nix/git"; | ||
|
||
if (!pathExists(cacheDir)) { | ||
createDirs(cacheDir); | ||
runProgram("git", true, { "init", "--bare", cacheDir }); | ||
} | ||
|
||
Activity act(*logger, lvlInfo, format("fetching Git repository ‘%s’") % url); | ||
|
||
std::string localRef = "pid-" + std::to_string(getpid()); | ||
Path localRefFile = cacheDir + "/refs/heads/" + localRef; | ||
|
||
runProgram("git", true, { "-C", cacheDir, "fetch", url, rev + ":" + localRef }); | ||
|
||
std::string commitHash = chomp(readFile(localRefFile)); | ||
|
||
unlink(localRefFile.c_str()); | ||
|
||
debug(format("got revision ‘%s’") % commitHash); | ||
|
||
// FIXME: should pipe this, or find some better way to extract a | ||
// revision. | ||
auto tar = runProgram("git", true, { "-C", cacheDir, "archive", commitHash }); | ||
|
||
Path tmpDir = createTempDir(); | ||
AutoDelete delTmpDir(tmpDir, true); | ||
|
||
runProgram("tar", true, { "x", "-C", tmpDir }, tar); | ||
|
||
Path storePath = state.store->addToStore("git-export", tmpDir); | ||
|
||
mkString(v, storePath, PathSet({storePath})); | ||
} | ||
|
||
static RegisterPrimOp r("__fetchgit", 1, prim_fetchgit); | ||
|
||
} |
This comment has been minimized.
We really need to be consistent here and pick a name that:
fetchTarball
alternative that just uses a different formatThis comment has been minimized.
nix-exec has a fetchgit builtin we may want to pull from here https://github.com/shlevy/nix-exec/blob/master/src/fetchgit.cc https://github.com/shlevy/nix-exec/blob/master/scripts/fetchgit.sh.in
FWIW I'm a bit worried about using one local cache repo for everything. It doesn't seem to gain much (seems unlikely that a lot of big files will be shared between repos) and just costs us a higher chance of collision. nix-exec has a separate local repo for each repo.
This comment has been minimized.
@shlevy I think significant savings can be made when sharing e.g. nixpkgs and nixpkgs-channels, or different linux kernel repos, for example.
This comment has been minimized.
@lheckemann On the other hand, we see ridiculously huge slowdowns (to the point of unusability) when using builtins.fetchGit on multiple actually-distinct repos. I'm fine with a way to give a common cache name to disparate repos (e.g. "nixpkgs-channels should live with nixpkgs"), but the current situation is effectively broken for the multi-repo usecase.
This comment has been minimized.
Huh, I've never seen that. I suppose I don't use fetchGit enough!