FR: Pushing and fetching arbitrary refs

[ilyagr]

Is your feature request related to a problem? Please describe.
Currently, there is no way to use jj to push or fetch an arbitrary ref from the remote. This causes problems for e.g. Gerrit as discussed on Discord.

On one hand, this reinforces the desire for Gerrit support, #4387, but separately it would be nice to support something for the general case.

Describe the solution you'd like
I can think of 2 possible solutions:

1. Re-design our branches (soon to be called bookmarks) to support tracking an arbitrary ref on a remote. This probably would imply having to support tracking different refs on different remotes, tracking remote branches of a different name, and perhaps tracking multiple refs per remote.
2. Set up simple commands to push and fetch arbitrary refs unsafely, either permanently or until we can implement 1 or think of a better solution.

Here's a possible interface for option 2. It's different enough from regular fetch/push that I think it deserves separate commands:

• jj git fetch-ref [--remote REMOTE] REF_NAME [branch_name] would fetch an arbitrary ref and optionally create a branch at the newly fetched commit for easier reference later. This new local branch wouldn't track anything. It'd just be a bookmark :).
• jj git push-ref <--force|--expected-at COMMIT_ID> [--remote REMOTE] REVISION REF_NAME

Both 1 and 2 have disadvantages. 1 adds complexity to the UI and the model. I'm not sure whether or not 2. is better than telling people to co-locate the repo and use git fetch/git push.

In any case, I'm curious what people think should be done here.

[yuja]

Some previous discussions:

1. Re-design our branches (soon to be called bookmarks) to support tracking an arbitrary ref on a remote.

#3235

2. Set up simple commands to push and fetch arbitrary refs

#2098 (comment)

For 2, we might want to introduce command namespace for plumbing commands so jj git fet[TAB] wouldn't stop at jj git fetch(-refs)?.

[emilazy]

This is also nice for GitHub (refs like pull/123/head).

[ilyagr]

Thanks Yuya, I forgot about those discussions.

For a subcommand, jj git ref fetch could work.

[Minion3665]

Heya! I was the one who originally prompted the discussion on Discord - I have a few thoughts to put here too :)

Re-design our branches (soon to be called bookmarks) to support tracking an arbitrary ref on a remote

This seems interesting to me, although I note that it wouldn't quite solve the gerrit/agit/radicle/idk use-case in a nice way, at least for pushes. That's because so-called "magic refs" do weird things on the server - you always push to refs/for/main, etc., and you can't pull them back down from the same place. I'm not sure how much software uses magic refs like this; my assumption is that a small but certainly nonzero amount does.

That said: my use-case only particularly cares about pulls, and I generally think that they are more widely useful. The refs/pull/#/head mentioned earlier are a great example of this - GitHub is really widely used!

1 adds complexity to the UI and the model

This is probably true, and is a shame. No matter how hard we try to make it so refs aren't exposed unless you're using the feature (hide the refs/foo/... prefix if foo is heads, etc.), probably it at least makes the documentation more confusing if you don't know what a ref is.

I'm not sure whether or not 2. is better than telling people to co-locate the repo and use git fetch/git push

To me it would be

• I'm using nix with flakes, nix has a thing where files that are untracked aren't seen at all, which has left me very confused about why my code isn't doing anything more times that I would like to admit. jj "solves" this by not being git, and therefore not having this oddity. More generally: some tools integrate poorly with Git. It can be a disadvantage to have a colocated repository.

Additionally, if I do want to push to arbitrary refs...

• I want to refer to things in the way I am doing with jj. I often jj edit change_id, leaving my changes in git uncommited. It's a bit more friction to jj show @, copy the commit ID, and go over to git, given there's no HEAD reference for that.

I'm sure that I could write a hacky bash function to workaround these issues (using the underlying git repo and a lot of duct tape) and go on my way... call it jj.git.ref or something, but I'm not particularly enthusiastic...

[ilyagr]

@jennings suggested using --named for this on Discord. I'm not sure I like that, but perhaps --ref would work. Copying the examples he used with this minor modification, it could look like this:

jj git fetch --ref pr123=refs/pull/123/head
jj git push --ref ref/for/main=xyz

(I'm not claiming that this is necessarily better than the syntax old me suggested, or a syntax anyone else suggested. I just wanted to write the fresh idea down before I forget.)

[9999years]

I would like to register my interest in being able to fetch arbitrary commits as well as refs. This is useful for backports or cherry-picks, especially on GitHub where repo contains all of its forks in the backing storage, so you can do git fetch my_github_remote some_commit_hash to fetch a commit from an arbitrary fork (see #7369).

[djmitche]

I started to take a look at the code around fetching. I'd be very interested in implementing a solution (with some assistance), but it seems that there's not yet consensus on what that should look like. Maybe that offer makes it a bit more attractive to pick an approach? The suggestion two comments before this one seems workable. AIUI that would essentially set up tracking for a local name to a remote ref, rather than just to a remote branch.

[Minion3665]

@djmitche as I understand it, that wouldn't set up tracking - rather jj git fetch --ref pr123=refs/pull/123/head would create a new local bookmark, pr123 that was at whatever commit refs/pull/123/head referenced - but jj git fetch would not fetch refs/pull/123/head again if that changed...

...similarly for jj git push --ref refs/for/main=xyz, whatever single commit was referenced by revset xyz would be pushed to refs/for/main but - if xyz was a local bookmark - running jj git push after updating xyz wouldn't make it push to refs/for/main again

Tracking could be cool but (quoting myself from earlier in this thread)

This seems interesting to me, although I note that it wouldn't quite solve the gerrit/agit/radicle/idk use-case in a nice way, at least for pushes. That's because so-called "magic refs" do weird things on the server - you always push to refs/for/main, etc., and you can't pull them back down from the same place. I'm not sure how much software uses magic refs like this; my assumption is that a small but certainly nonzero amount does

(emphasis mine... from now, not then)

---

I do really like the suggestion from 2 3 comments ago too - I'd be very happy if we got something like that