Skip to content
Permalink
Browse files

Merge branch 'ab/fetch-prune'

Clarify how configured fetch refspecs interact with the "--prune"
option of "git fetch", and also add a handy short-hand for getting
rid of stale tags that are locally held.

* ab/fetch-prune:
  fetch: make the --prune-tags work with <url>
  fetch: add a --prune-tags option and fetch.pruneTags config
  fetch tests: add scaffolding for the new fetch.pruneTags
  git-fetch & config doc: link to the new PRUNING section
  git remote doc: correct dangerous lies about what prune does
  git fetch doc: add a new section to explain the ins & outs of pruning
  fetch tests: fetch <url> <spec> as well as fetch [<remote>]
  fetch tests: expand case/esac for later change
  fetch tests: double quote a variable for interpolation
  fetch tests: test --prune and refspec interaction
  fetch tests: add a tag to be deleted to the pruning tests
  fetch tests: re-arrange arguments for future readability
  fetch tests: refactor in preparation for testing tag pruning
  remote: add a macro for "refs/tags/*:refs/tags/*"
  fetch: stop accessing "remote" variable indirectly
  fetch: trivially refactor assignment to ref_nr
  fetch: don't redundantly NULL something calloc() gave us
  • Loading branch information...
gitster committed Mar 6, 2018
2 parents a4ae2e5 + 6317972 commit c1a7902f9adda4c3a64de99d565cf9982f12b1d9
@@ -1398,7 +1398,16 @@ fetch.unpackLimit::

fetch.prune::
If true, fetch will automatically behave as if the `--prune`
option was given on the command line. See also `remote.<name>.prune`.
option was given on the command line. See also `remote.<name>.prune`
and the PRUNING section of linkgit:git-fetch[1].

fetch.pruneTags::
If true, fetch will automatically behave as if the
`refs/tags/*:refs/tags/*` refspec was provided when pruning,
if not set already. This allows for setting both this option
and `fetch.prune` to maintain a 1=1 mapping to upstream
refs. See also `remote.<name>.pruneTags` and the PRUNING
section of linkgit:git-fetch[1].

fetch.output::
Control how ref update status is printed. Valid values are
@@ -2945,6 +2954,15 @@ remote.<name>.prune::
remote (as if the `--prune` option was given on the command line).
Overrides `fetch.prune` settings, if any.

remote.<name>.pruneTags::
When set to true, fetching from this remote by default will also
remove any local tags that no longer exist on the remote if pruning
is activated in general via `remote.<name>.prune`, `fetch.prune` or
`--prune`. Overrides `fetch.pruneTags` settings, if any.
+
See also `remote.<name>.prune` and the PRUNING section of
linkgit:git-fetch[1].

remotes.<group>::
The list of remotes which are fetched by "git remote update
<group>". See linkgit:git-remote[1].
@@ -73,7 +73,22 @@ ifndef::git-pull[]
are fetched due to an explicit refspec (either on the command
line or in the remote configuration, for example if the remote
was cloned with the --mirror option), then they are also
subject to pruning.
subject to pruning. Supplying `--prune-tags` is a shorthand for
providing the tag refspec.
+
See the PRUNING section below for more details.

-P::
--prune-tags::
Before fetching, remove any local tags that no longer exist on
the remote if `--prune` is enabled. This option should be used
more carefully, unlike `--prune` it will remove any local
references (local tags) that have been created. This option is
a shorthand for providing the explicit tag refspec along with
`--prune`, see the discussion about that in its documentation.
+
See the PRUNING section below for more details.

endif::git-pull[]

ifndef::git-pull[]
@@ -99,6 +99,93 @@ The latter use of the `remote.<repository>.fetch` values can be
overridden by giving the `--refmap=<refspec>` parameter(s) on the
command line.

PRUNING
-------

Git has a default disposition of keeping data unless it's explicitly
thrown away; this extends to holding onto local references to branches
on remotes that have themselves deleted those branches.

If left to accumulate, these stale references might make performance
worse on big and busy repos that have a lot of branch churn, and
e.g. make the output of commands like `git branch -a --contains
<commit>` needlessly verbose, as well as impacting anything else
that'll work with the complete set of known references.

These remote-tracking references can be deleted as a one-off with
either of:

------------------------------------------------
# While fetching
$ git fetch --prune <name>

# Only prune, don't fetch
$ git remote prune <name>
------------------------------------------------

To prune references as part of your normal workflow without needing to
remember to run that, set `fetch.prune` globally, or
`remote.<name>.prune` per-remote in the config. See
linkgit:git-config[1].

Here's where things get tricky and more specific. The pruning feature
doesn't actually care about branches, instead it'll prune local <->
remote-references as a function of the refspec of the remote (see
`<refspec>` and <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> above).

Therefore if the refspec for the remote includes
e.g. `refs/tags/*:refs/tags/*`, or you manually run e.g. `git fetch
--prune <name> "refs/tags/*:refs/tags/*"` it won't be stale remote
tracking branches that are deleted, but any local tag that doesn't
exist on the remote.

This might not be what you expect, i.e. you want to prune remote
`<name>`, but also explicitly fetch tags from it, so when you fetch
from it you delete all your local tags, most of which may not have
come from the `<name>` remote in the first place.

So be careful when using this with a refspec like
`refs/tags/*:refs/tags/*`, or any other refspec which might map
references from multiple remotes to the same local namespace.

Since keeping up-to-date with both branches and tags on the remote is
a common use-case the `--prune-tags` option can be supplied along with
`--prune` to prune local tags that don't exist on the remote, and
force-update those tags that differ. Tag pruning can also be enabled
with `fetch.pruneTags` or `remote.<name>.pruneTags` in the config. See
linkgit:git-config[1].

The `--prune-tags` option is equivalent to having
`refs/tags/*:refs/tags/*` declared in the refspecs of the remote. This
can lead to some seemingly strange interactions:

------------------------------------------------
# These both fetch tags
$ git fetch --no-tags origin 'refs/tags/*:refs/tags/*'
$ git fetch --no-tags --prune-tags origin
------------------------------------------------

The reason it doesn't error out when provided without `--prune` or its
config versions is for flexibility of the configured versions, and to
maintain a 1=1 mapping between what the command line flags do, and
what the configuration versions do.

It's reasonable to e.g. configure `fetch.pruneTags=true` in
`~/.gitconfig` to have tags pruned whenever `git fetch --prune` is
run, without making every invocation of `git fetch` without `--prune`
an error.

Pruning tags with `--prune-tags` also works when fetching a URL
instead of a named remote. These will all prune tags not found on
origin:

------------------------------------------------
$ git fetch origin --prune --prune-tags
$ git fetch origin --prune 'refs/tags/*:refs/tags/*'
$ git fetch <url of origin> --prune --prune-tags
$ git fetch <url of origin> --prune 'refs/tags/*:refs/tags/*'
------------------------------------------------

OUTPUT
------

@@ -172,10 +172,14 @@ With `-n` option, the remote heads are not queried first with

'prune'::

Deletes all stale remote-tracking branches under <name>.
These stale branches have already been removed from the remote repository
referenced by <name>, but are still locally available in
"remotes/<name>".
Deletes stale references associated with <name>. By default, stale
remote-tracking branches under <name> are deleted, but depending on
global configuration and the configuration of the remote we might even
prune local tags that haven't been pushed there. Equivalent to `git
fetch --prune <name>`, except that no new references will be fetched.
+
See the PRUNING section of linkgit:git-fetch[1] for what it'll prune
depending on various configuration.
+
With `--dry-run` option, report what branches will be pruned, but do not
actually prune them.
@@ -189,7 +193,7 @@ remotes.default is not defined, all remotes which do not have the
configuration parameter remote.<name>.skipDefaultUpdate set to true will
be updated. (See linkgit:git-config[1]).
+
With `--prune` option, prune all the remotes that are updated.
With `--prune` option, run pruning against all the remotes that are updated.


DISCUSSION
@@ -39,6 +39,10 @@ static int fetch_prune_config = -1; /* unspecified */
static int prune = -1; /* unspecified */
#define PRUNE_BY_DEFAULT 0 /* do we prune by default? */

static int fetch_prune_tags_config = -1; /* unspecified */
static int prune_tags = -1; /* unspecified */
#define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */

static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity, deepen_relative;
static int progress = -1;
static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen;
@@ -66,6 +70,11 @@ static int git_fetch_config(const char *k, const char *v, void *cb)
return 0;
}

if (!strcmp(k, "fetch.prunetags")) {
fetch_prune_tags_config = git_config_bool(k, v);
return 0;
}

if (!strcmp(k, "submodule.recurse")) {
int r = git_config_bool(k, v) ?
RECURSE_SUBMODULES_ON : RECURSE_SUBMODULES_OFF;
@@ -128,6 +137,8 @@ static struct option builtin_fetch_options[] = {
N_("number of submodules fetched in parallel")),
OPT_BOOL('p', "prune", &prune,
N_("prune remote-tracking branches no longer on remote")),
OPT_BOOL('P', "prune-tags", &prune_tags,
N_("prune local tags no longer on remote and clobber changed tags")),
{ OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules, N_("on-demand"),
N_("control recursive fetching of submodules"),
PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules },
@@ -1220,6 +1231,8 @@ static void add_options_to_argv(struct argv_array *argv)
argv_array_push(argv, "--dry-run");
if (prune != -1)
argv_array_push(argv, prune ? "--prune" : "--no-prune");
if (prune_tags != -1)
argv_array_push(argv, prune_tags ? "--prune-tags" : "--no-prune-tags");
if (update_head_ok)
argv_array_push(argv, "--update-head-ok");
if (force)
@@ -1323,12 +1336,15 @@ static inline void fetch_one_setup_partial(struct remote *remote)
return;
}

static int fetch_one(struct remote *remote, int argc, const char **argv)
static int fetch_one(struct remote *remote, int argc, const char **argv, int prune_tags_ok)
{
static const char **refs = NULL;
struct refspec *refspec;
int ref_nr = 0;
int j = 0;
int exit_code;
int maybe_prune_tags;
int remote_via_config = remote_is_configured(remote, 0);

if (!remote)
die(_("No remote repository specified. Please, specify either a URL or a\n"
@@ -1338,18 +1354,39 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)

if (prune < 0) {
/* no command line request */
if (0 <= gtransport->remote->prune)
prune = gtransport->remote->prune;
if (0 <= remote->prune)
prune = remote->prune;
else if (0 <= fetch_prune_config)
prune = fetch_prune_config;
else
prune = PRUNE_BY_DEFAULT;
}

if (prune_tags < 0) {
/* no command line request */
if (0 <= remote->prune_tags)
prune_tags = remote->prune_tags;
else if (0 <= fetch_prune_tags_config)
prune_tags = fetch_prune_tags_config;
else
prune_tags = PRUNE_TAGS_BY_DEFAULT;
}

maybe_prune_tags = prune_tags_ok && prune_tags;
if (maybe_prune_tags && remote_via_config)
add_prune_tags_to_fetch_refspec(remote);

if (argc > 0 || (maybe_prune_tags && !remote_via_config)) {
size_t nr_alloc = st_add3(argc, maybe_prune_tags, 1);
refs = xcalloc(nr_alloc, sizeof(const char *));
if (maybe_prune_tags) {
refs[j++] = xstrdup("refs/tags/*:refs/tags/*");
ref_nr++;
}
}

if (argc > 0) {
int j = 0;
int i;
refs = xcalloc(st_add(argc, 1), sizeof(const char *));
for (i = 0; i < argc; i++) {
if (!strcmp(argv[i], "tag")) {
i++;
@@ -1359,9 +1396,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
argv[i], argv[i]);
} else
refs[j++] = argv[i];
ref_nr++;
}
refs[j] = NULL;
ref_nr = j;
}

sigchain_push_common(unlock_pack_on_signal);
@@ -1380,6 +1416,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
struct string_list list = STRING_LIST_INIT_DUP;
struct remote *remote = NULL;
int result = 0;
int prune_tags_ok = 1;
struct argv_array argv_gc_auto = ARGV_ARRAY_INIT;

packet_trace_identity("fetch");
@@ -1446,6 +1483,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
} else {
/* Zero or one remotes */
remote = remote_get(argv[0]);
prune_tags_ok = (argc == 1);
argc--;
argv++;
}
@@ -1454,7 +1492,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
if (remote) {
if (filter_options.choice || repository_format_partial_clone)
fetch_one_setup_partial(remote);
result = fetch_one(remote, argc, argv);
result = fetch_one(remote, argc, argv, prune_tags_ok);
} else {
if (filter_options.choice)
die(_("--filter can only be used with the remote configured in core.partialClone"));
@@ -1468,7 +1468,7 @@ __git_fetch_recurse_submodules="yes on-demand no"
__git_fetch_options="
--quiet --verbose --append --upload-pack --force --keep --depth=
--tags --no-tags --all --prune --dry-run --recurse-submodules=
--unshallow --update-shallow
--unshallow --update-shallow --prune-tags
"

_git_fetch ()
@@ -22,6 +22,7 @@ static struct refspec s_tag_refspec = {
"refs/tags/*"
};

/* See TAG_REFSPEC for the string version */
const struct refspec *tag_refspec = &s_tag_refspec;

struct counted_string {
@@ -103,6 +104,17 @@ static void add_fetch_refspec(struct remote *remote, const char *ref)
remote->fetch_refspec[remote->fetch_refspec_nr++] = ref;
}

void add_prune_tags_to_fetch_refspec(struct remote *remote)
{
int nr = remote->fetch_refspec_nr;
int bufsize = nr + 1;
int size = sizeof(struct refspec);

remote->fetch = xrealloc(remote->fetch, size * bufsize);
memcpy(&remote->fetch[nr], tag_refspec, size);
add_fetch_refspec(remote, xstrdup(TAG_REFSPEC));
}

static void add_url(struct remote *remote, const char *url)
{
ALLOC_GROW(remote->url, remote->url_nr + 1, remote->url_alloc);
@@ -173,6 +185,7 @@ static struct remote *make_remote(const char *name, int len)

ret = xcalloc(1, sizeof(struct remote));
ret->prune = -1; /* unspecified */
ret->prune_tags = -1; /* unspecified */
ALLOC_GROW(remotes, remotes_nr + 1, remotes_alloc);
remotes[remotes_nr++] = ret;
ret->name = xstrndup(name, len);
@@ -391,6 +404,8 @@ static int handle_config(const char *key, const char *value, void *cb)
remote->skip_default_update = git_config_bool(key, value);
else if (!strcmp(subkey, "prune"))
remote->prune = git_config_bool(key, value);
else if (!strcmp(subkey, "prunetags"))
remote->prune_tags = git_config_bool(key, value);
else if (!strcmp(subkey, "url")) {
const char *v;
if (git_config_string(&v, key, value))
@@ -47,6 +47,7 @@ struct remote {
int skip_default_update;
int mirror;
int prune;
int prune_tags;

const char *receivepack;
const char *uploadpack;
@@ -297,4 +298,8 @@ extern int parseopt_push_cas_option(const struct option *, const char *arg, int
extern int is_empty_cas(const struct push_cas_option *);
void apply_push_cas(struct push_cas_option *, struct remote *, struct ref *);

#define TAG_REFSPEC "refs/tags/*:refs/tags/*"

void add_prune_tags_to_fetch_refspec(struct remote *remote);

#endif

0 comments on commit c1a7902

Please sign in to comment.
You can’t perform that action at this time.