Redesign open/closed concept?

[martinvonz]

Background

Because we automatically commit the working copy when we detect changes, we have the concept of "open" and "closed" commits. If you check out an open commit, any changes you make in the working copy will be automatically amended into it. If you check out a closed commit, jj will create a new open commit (with a new change ID) on top and check out that instead, so any changes in the working copy will always be amended into the current checkout. The way we tell open and closed commits apart is by a flag set in the commit. When closing a commit (by jj close/commit), jj will create a new commit with the open flag cleared.

Problem

Perhaps the open/closed bit should not be embedded in the commit.

• Users may not agree on which commits should be open. For example, if Alice checks out user Bob's open commit, Alice probably doesn't want changes in the working copy to automatically amend the commit. Since we prevent exchange of open commits, this only comes up if you share the repository on disk, e.g. via shared file system. It will also come up if all commits are transparently shared among the users in an organization.
• Closing commits is the main reason for bug jj close (and some others) sometimes crashes on first attempt with Git backend #27. However, I consider this a very weak argument against having the flag in the commit, because there are more general solutions for the bug.

Solutions

I think we want to preserve how changes to the working copy will always result in amending the working copy commit, as opposed to creating a new commit on top when the first changes are made, for example. Given that, I think that means that we need to still have some way of indicating at the time of checkout whether to check out the specified commit or to create a new commit on top of. Note that checkouts can be a result of not only jj checkout, but also of jj abandon or jj git fetch, for example.

Store set of open commits in repo view

We could let the repo's "view" object keep track of which commits are considered open. The view concept didn't exist when I added the open/closed concept, so that's why I didn't do it that way to start with.

This proposal doesn't have much user-visible impact other than fixing the problems described above.

Make only the current checkout "open"

We could remove the per-commit open/closed concept completely by considering it to be a property of the working copy, so only the current checkout is "open". Instead of a having jj checkout behave differently depending on whether the commit is open or closed, we would then have to let the user decide if they wanted to check out the commit in order to edit it or in order to build on top. Perhaps that could be jj edit <commit> to say that you want to check out a commit for editing (ignoring the fact that the command already exists and does something else) and jj checkout <commit> to say that you want to check out a commit for building on top (or just run tests or whatever). Actually, jj new is pretty much already that command (we should probably make it always update to the newly created commit, regardless of what else we decide about open/closed commits).

This model is perhaps more intuitive - the user typically knows whether they're checking out a commit for editing or not, so it's probably not too demanding to have them say which. Removing the concept of open/closed commits seems like a good simplification.

An argument against this model is that knowing that a commit is open is a useful signal that it is unfinished.

If we decide to remove the concept of open/closed commits, we'll want to also rename or remove jj close. While the command no longer changes the open flag on the commit, it will still ask the user for a commit message (if there isn't already one), and create a new working-copy commit on top. So it might be useful as a convenience command for checking that the current working-copy commit has a description and creating a new one on top. Perhaps we should keep the jj commit alias for the command (since that's familiar to git/hg users) and remove the jj close name for it.

If closing a commit is not necessary (or even possible), perhaps users will start using jj new more over time, thus bypassing the check for empty description. I don't know if there's a better place to remind the user to provide a description. Maybe we don't need to remind them at all? We will probably at least want to prevent pushing commits without a description (which we should do regardless of our decision here).

Note that jj commit and jj new will be very similar. The main difference would be that jj commit -m foo sets the description on the old commit (parent) while jj new -m foo sets the description on the new commit (child). Interestingly, if we make jj checkout a synonym for jj new, then jj checkout will also be a near-synonym for jj commit.

The automatic checkout we do when commits get rewritten and abandoned needs some thought. When the current checkout is abandoned, we should create a new commit for it. When the current checkout has been rewritten, we should just update to it without creating a new commit on top. Once we have support for tracking of public/draft commits (like hg's phases), we'll need to make sure that we create a new commit on top if a draft commit was rewritten and became public (actually, maybe the condition is unrelated to rewrites - we also want to create a new commit on top if current checkout became public without a rewrite).

[martinvonz]

We (@hooper, @rdamazio, @spectral54, @torquestomp, and I) talked about this for quite a long time at work today. I think we all agreed that removing the concept of open commits is the best solution (i.e. the second solution above). In case it gets annoying to have to specify that you want to go back to editing a commit after you temporarily switched away, we could implement a hybrid model where we keep the open/editing commits in the repo view (suggested by @torquestomp). It's obviously simpler to not have do that, so I prefer to at least start without that feature and we can see how it goes.

Someone (@spectral54, I think) suggested adding a checkout --rebase, which would behave like hg/git checkout --merge or jj rebase -r @. That could be useful at least for smoothing the transition from git/hg. We could have a config for the default behavior and allow overriding it on the command line. We could have other config options e.g. for preventing checkout if the current working copy commit is not empty (like hg checkout --check).

@torquestomp suggested using checkout --edit for checking out a commit for editing. That sounds good to me, but I wonder if we want something as short as just edit because this will be a common operation. Starting with checkout --edit still seems good and if we realize that it's common that people create aliases for it, then we can create a builtin alias or top-level command for it.

We talked about how the new model (and the current model, actually) would interact with multiple workspaces and with concurrency. It is currently possible to have an open commit that has descendants. If you do, any descendants will be automatically rebased every time you run a command. So it works, but it's admittedly a bit weird to be in that state. The same situation could occur with the new model if the user did checkout --edit <non-head commit>. We agreed to try to prevent that by checking for descendants before checking out for editing. Similar situations can occur if the user is editing a commit in one workspace and creates a child commit in another workspace (including by rebasing some commit on top). In that case, we'll create a new commit for the workspace whose working copy we created a child of (that's possible to do since the repo view knows which commit is checked out in each workspace).

Any thoughts from anyone else? Does the above sound good to you? I know @arxanas had some ideas on Discord about some kind of higher-level (even hierarchical?) commit groups.

[spectral54]

We also mentioned having a (configurable?) prompt when it was ambiguous what to do. If only the current checkout is 'open', then updating to a different commit implies 'closing' the current one. So what should happen:

• error out telling user they need to close it? (hg update --check)
• implicitly close it?
• prompt the user? (This is where the jj checkout --rebase thought came from - we could give options like:
  • cancel
  • close it "as is"
  • edit description and close it
  • rebase it

[martinvonz]

Oh, I forgot the bit about when to check that a commit has a description. We talked about having a config for that too. There would be one config value for prompting the user for a description when checking out away from a commit without a description. There would be another config value for not doing anything about it (but we should still prevent pushing them, as noted earlier). Maybe there would also be a config option for printing a warning.

[arxanas]

Commit evolution

First, a digression on the topic of commit evolution. I'm sure most of this is familiar to you, but I'm not sure if we agree on the ideas around syncing.

Under most current mainstream VCSes, a commit is a combination of some number of edges to parent commits and a snapshot (or patch, depending on the VCS), and some other metadata like the commit message and timestamp, which aren't important here. This handles a notion of history along the codebase dimension, where the functionality of the codebase changes over time.

Mercurial (and also Jujutsu and git-branchless) have a concept of tracking the history of commits themselves as they're modified. In this model, a commit can have another kind of parent edge indicating its historical predecessor along the history dimension.

Under Jujutsu, commits which are linked across the history dimension have the same change ID.

One question is this: now that we're tracking the history of the commit graph, do we want to track one further level of changes, i.e. the history of the history of the commit graph? In practice, yes: we want to track the repository history so that we can undo operations. Jujutsu and git-branchless both accomplish this with an operation log, which track the commit nodes/edges we added. If you want to undo stuff, then you add even more commit nodes/edges which cancel out the effects of the previous operations.

There's also the question of merging repository-level changes. Can you just apply someone else's operations to your repository and get a useful result?

Sync

This model of commits allows the repository to form a natural CRDT: you can combine the states of two repositories by unioning together both their sets of commits (and edges).

Any commits which have incoming history edges should be considered hidden. They've been superseded by a different commit — maybe even someone else's commit!

Furthermore, the repository is in a state of conflict if there are any hidden commits which have non-hidden descendants (along the codebase dimension). In Jujutsu, this is less of a concern, since it's always safe to rebase a descendant onto the new, non-hidden version of the commit. But what if there are multiple non-hidden versions? In particular, what if we've modified an open commit? I'm assuming that Jujutsu doesn't actually properly handle this (correct me if I'm wrong). We'll come back to this shortly.

Illustration

For illustration, here are some relevant scenarios demonstrating the codebase vs. history dimension edges.

To make a new commit, you create a new node and parent commit edge. Starting here:

flowchart RL
    B --> A

Loading

we can create a new commit C by setting its parent commit edge to B (and its parent history edge to 0, not shown):

flowchart RL
    C--> B --> A

Loading

To amend commit B instead you create a new commit C, which preserves the parent commit edge and points to the old one via the history edge (denoted via a dotted line):

flowchart RL
    C -.-> B --> A
    C --> A

Loading

To rebase commit B onto a different parent commit C, you create a new commit D with both a new parent commit edge and a new history edge:

flowchart RL
    D -.-> B --> A
    D --> C

Loading

There's not any situation where you wouldn't want to take an existing commit and preserve both its parent commit edge and parent history edge. Even when modifying commit metadata, we typically create a new commit along the history dimension which supersedes that commit.

To illustrate the above sync problem for Jujutsu, suppose you start with this commit graph:

flowchart RL
    B ---> A

Loading

Then locally, you create a new commit C, while remotely, two other people amend commit B:

flowchart RL
    C ---> B ---> A
    D -.-> B
    E -.-> B

Loading

(AFAICT, Mermaid has no hints about how to lay out these nodes; I would have liked to have drawn the codebase axis horizontally and the history axis vertically, but it instead renders all child commits to the right.)

Logically, what is the resolution? In my opinion, it would be to create two new auto-rebased commits, so that the graph looks like this:

flowchart RL
    C ---> B ---> A
    D -.-> B
    E -.-> B
    F -.-> C
    F ---> D
    G ---> E
    G -.-> C

Loading

I don't know if Jujutsu does this today.

Given the above graph, how do we know that the repository is in a state of conflict, and how do we know what to do to repair it?

Multi-head syncing

Branches

Jujutsu today allows a branch to have multiple associated heads. A branch is considered conflicted if it has more than one associated head. For example, more than one party tries to push the same branch to a remote server, that server can end up with a branch in a conflicted state.

The Monotone VCS also thinks about branches in this way. However, a branch is a set of commits, rather than just a set of heads. The logical value of the branch is the unique head of that set of commits, and a branch is conflicted if there is more than one head. To push to a remote branch, you send all of your commits reachable from your repository's version of that branch.

To resolve a conflicting branch, you have to create a merge commit with all of the previous heads as parents. Thus, commit resolution in Monotone is accomplished by adding more edges, not by removing some of the heads from the branch. This is elegant from a CRDT-style syncing standpoint. If you have another race, then you just end up with a branch (commit set) which still has multiple heads.

Inferred branches

I said above that commit evolution and the history dimension are used to track changes to the commit graph over time. There's another interpretation: under Monotone-style branching, history edges let us infer logical branches, without having to name those branches explicitly. Instead, the branches are determined by the operations you do on the commits.

I didn't figure out the exact criteria, but they're something like this:

• Find all commits for which there is more than one outgoing edge, and at least one edge is a history edge.
• For each commit, take the descendants of that commit, and the heads of those descendants. These are the Monotone-style heads of the branch. If there's more than one head, then it's conflicting.
• All the other heads in the repository which aren't one of the above heads are separate, non-conflicting branches.

If you would push a commit to a remote Git-style branch, you would also push all of the commits reachable from that commit (via both codebase and history edges) and union them into the commit set represented by the remote branch. If people push to the remote repository's local branch such that you end up with this graph from above:

flowchart RL
    C ---> B ---> A
    D -.-> B
    E -.-> B

Loading

then you know that the branch is in conflict because there are multiple heads for this commit set. To resolve this, first restore the abandoned-commit invariant by rebasing C (twice!), to end up with the graph from above:

flowchart RL
    C ---> B ---> A
    D -.-> B
    E -.-> B
    F -.-> C
    F ---> D
    G ---> E
    G -.-> C

Loading

Then merge the heads together in some order:

flowchart RL
    C ---> B ---> A
    D -.-> B
    E -.-> B
    F -.-> C
    F ---> D
    G ---> E
    G -.-> C
    H --> F
    H --> G

Loading

(Maybe the merging edges should be history edges and not parent commit edges?)

Now the conflict has been reduced from a branch-level conflict to a commit-level conflict. (One other question is: can we model commit-level and branch-level conflicts in the same way or using the same algorithms?)

Commit groups

Merge commits are a sham. Practically every workflow in use is essentially a patch-based workflow. For example, the Git repository merges changes through a series of progressively-more-stable branches, but they also cherry-pick individual commits into branches for backporting. They use the three-way diff functionality that merge commits give you, but really, they just want to apply all unapplied patches onto a given branch.

There is no "canonical" form of a merge commit under Git; you can introduce "evil" merges which don't really reflect the underlying composition of both branches. The order of the parents is also significant.

Here's another common complaint: when you merge a branch into another branch, you lose information, such as the historical branch name.

We can generalize the open/closed commit idea into patch-based hierarchical commit groups and get rid of merge commits, while preserving the logical structure of the commit graph.

Illustration

Suppose we start with this commit graph, and that B is currently checked out:

flowchart RL
    B --> A

Loading

There's now two notions of committing. A "closed" commit is as usual:

flowchart RL
    C --> B --> A

Loading

An "open" commit adds an extra internal commit to the list of commits, so the graph would look kind of like this instead. Note that both Bs are actually the same commit:

flowchart RL
    subgraph X
        direction RL
        B2[B]
        C --> B2
    end
    X --> A
    X -.-> B --> A

Loading

A rebase looks like this:

flowchart RL
    subgraph X
        direction RL
        B2[B]
        C --> B2
    end
    X --> D
    X -.-> B --> A

Loading

The advantage is that you can group arbitrary levels of commits and preserve the groups as they move around the commit graph. Suppose you have an offshoot branch foo like this:

flowchart RL
    subgraph foo
        direction RL
        D --> C
    end
    foo --> A
    B --> A

Loading

You can rebase it like this:

flowchart RL
    subgraph foo
        direction RL
        D --> C
    end
    subgraph foo'
        direction RL
        D2[D] --> C2[C]
    end
    foo' -.-> foo
    foo' --> B
    foo --> A
    B --> A

Loading

And the fact that there was a grouping of commits is preserved.

Formalization

Define $P$ to be the set of patches. A patch is an abstract object with an associated abstract operation $+$ which represents composition.

Composition is total, so $p_1 + p_2$ is always defined. (To implement this, you would want a conflict-aware representation, like in Pijul or Jujutsu.)

The $+$ operator is left-associative, so

$$ p_1 + p_2 + p_3 = \left(p_1 + p_2\right) + p_3 $$

There's an identity patch object $0_P$, such that

$$ 0_P + p = p $$

Patches are idempotent:

$$ (p_1 + \ldots + p_i + \ldots + p_n) + p_i = (p_1 + \ldots + p_i + \ldots + p_n) $$

Then define $C$ to be the set of commits. A commit is an object of the following form:

$$ C ::= (C, C, (C | P)^+) $$

The first element is the parent commit (or some object $0$); the second element is the predecessor commit (or some object $0$); and the third element is a non-empty list of commits or patches.

Let $\left[c\right]$ represent the patch evaluation of the commit $c$, defined as follows:

$$ \begin{align} \left[p\right] &= p \\ \left[\left(a, b, c_0 \ldots c_n\right)\right] &= \left[a\right] + \left[c_0\right] + \ldots + \left[c_n\right] \end{align} $$

The interesting operations are something like this:

$$ \begin{align} \text{commit-open}\left(\left(a, b, c_0, \ldots, c_n\right), p\right) &= \left(a, b, c_0, \ldots, c_n, p\right) \\ \text{commit-closed}\left(c_1, c_2\right) &= \left(c_1, 0, c_2\right) \\ \text{amend}\left(\left(a, b, c_0, \ldots, c_n\right), p\right) &= \left(a, \left(a, b, c_0, \ldots, c_n\right), c_0, \ldots, c_{n - 1}, \left(c_{n - 1}, c_n, p\right)\right) \\ \text{rebase}\left(c_1, c_2\right) &= \left(c_1, c_2, c_2\right) \end{align} $$

There are no merge commits; you have to rebase one commit onto another. By patch idempotency, if $a$ and some $c_i$, or some $c_i$ and $c_j$ have a common ancestor, then the patches in that ancestor aren't applied twice.

This structure also means that the commit graph is independent of the conflict-resolution mechanism, so you can change the conflict resolution algorithm later and re-evaluate the commit graph.

So the answer to the question

Perhaps the open/closed bit should not be embedded in the commit.

is that open/closed commits are only leaf nodes in the current implementation, but you should be able to nest commits arbitrarily. Then, to check out a commit, you want to specify whether you want an inner or outer commit (which would have different commit hashes).

Sync

Regarding this question:

Users may not agree on which commits should be open. For example, if Alice checks out user Bob's open commit, Alice probably doesn't want changes in the working copy to automatically amend the commit. Since we prevent exchange of open commits, this only comes up if you share the repository on disk, e.g. via shared file system. It will also come up if all commits are transparently shared among the users in an organization.

The above commentary on commit evolution provides a(n apparently-sound?) semantics for syncing. In particular, it's independent of the type of the snapshot associated with a commit, as long as you can rebase one commit onto another. That means that you can swap in the commit group concept here where we have a list of commits instead of a single snapshot and still have meaningful syncing (except that above, when we merged to resolve conflicts, we should have instead rebased, because commit groups don't support merging).

[martinvonz]

That will be an interesting case when we have support for public commits. If the first jj git push there went to a publishing server, that would mean that the "feature1" commit became public, which presumably would imply that a new commit would get created on top. So I suppose it will work as you were hoping, but only if the remote is a publishing remote (however we end up defining that).

[spoerri]

Hmm. Now I'm scared that it'd be too tricky. Let me suggest something completely different, but which I believe is more faithful to what you have above...

jj new : checkout a new child commit
jj open : checkout the commit itself for amending
and remove the original jj checkout command completely!

jj new would never be automatic. The case in my previous comment would try to rename the one commit from feature1 to feature2.

I hope it wouldn't be too confusing for people to have an open command that has a meaning which is so different from the current meaning of open...

I also hope you'll forgive the bikeshedding; I'm just so excited! 😄

Rationale detail:

• The open command would be heavily used, so wants a nicer name than checkout --edit.
  • Both open and new would be heavily used, and it's nice to use separate names which are appropriately related.
• The open name would be the obvious choice to retrieve historical commits, even just for viewing. Creating a new commit for viewing historical commits would be weird, and editing them is the only alternative.
• Removing checkout would reduce confusion about any single command replacing git checkout, which is used for both use cases.
• We could avoid git's mistake of unfortunate overloading of the checkout command.

[martinvonz]

That proposal does sound very similar to what the conclusion from our internal discussion at work (see above).

I proposed using edit as a short form for checkout --edit for the same reason you suggested open (that it will be used frequently and for different use cases). I like edit better because I worry that having a command called open makes the user wonder how to "close" when they're done. I could be convinced, though. The problem with edit is that it's already taken, but maybe we could rename the existing edit to touchup? I think that matches pretty well what it's for.

The open name would be the obvious choice to retrieve historical commits, even just for viewing. Creating a new commit for viewing historical commits would be weird, and editing them is the only alternative.

I understand the feeling that creating the empty working copy commit when checking out historical commits is weird, but I also don't like the idea of editing by default. I'm worried that would make it too easy to accidentally change historical commits. Always creating the commit (as opposed to doing it only once the user makes some changes) makes the behavior of e.g. jj diff -r @ simple and consistent.

Removing checkout would reduce confusion about any single command replacing git checkout, which is used for both use cases.

If jj checkout is synonymous with jj new, it would behave the same way as git checkout, I think, so it would hopefully not be that confusing. Maybe we could make jj checkout behave just like jj new except that it also prints a message saying something like this:

Use `jj new <target revision>` to create a new working copy on top of the target revision.
Use `jj edit <target revision>` to make the target revision the working copy.

Anyway, I'm happy to hear that it sounds like there's consensus about the idea of moving the decision of whether to auto-amend from being a property of the commit to being a decision the user makes when the "check out" the commit. What we call the commands is less important (but still important!).

I also hope you'll forgive the bikeshedding; I'm just so excited! 😄

No problem, and thanks for your input! Out of curiosity, do you already use jj or are you waiting for it to improve first (or for something else)?

[yuja]

when checking out historical commits is weird, but I also don't like the idea of editing by default. I'm worried that would make it too easy to accidentally change historical commits.

+1 for not editing by default. Draft commit is known-good state, and I would want to see diffs from that state even if I intended to do jj squash the changes. I think auto-closing @ on checking out another revision is safer, and is good tradeoff.

So, my two cents, the jj checkout / jj edit pair sounds good to me.

[spoerri]

do you already use jj

Gave it a spin for implementing a couple of features at work, but haven't started using it exclusively, no good reason, really...

don't like the idea of editing by default. I'm worried that would make it too easy to accidentally change historical commits

Yes, that makes sense. Curious about how to manage all the little dangling commits. Will ask later after using more 😄

[kajmagnus]

What about jj open <revision/branch> for checking out & opening a commit and continue editing it?

And if one types jj open with no revision/branch, then, the current commit gets opened?

(Sorry I didn't read all the discussion above, just skimmed 1/5 of it. And I don't know much about how jj works.)

jj new is pretty much already [...] to check out a commit for building on top (or just run tests or whatever).

Then jj open and jj new would behave a bit like files in the file system? But with commits instead of files. "Open" a file (commit) to continue editing it. Or create a "new" file (commit).

Sorry again for not having read everything. I did search for jj open in this repository and there doesn't seem to be such a command? And jj edit -r ... was mentioned, but that was in use for sth else already.

(jj continue ... is another thought)

Does jj new sound good if one just intends to look at the code or run tests? I wonder if jj get would be more accurate than "new", and simpler to understand than "checkout" for people not already familiar with other VCS:es. — And that last bit, people not already familiar with VCS:es, that could be a discussion topic on its own ... maybe I'll start one (if there isn't already) :- )

[martinvonz]

Hi Magnus, I'm happy to see you here! (Magnus is an old family friend and university classmate.)

What about jj open <revision/branch> for checking out & opening a commit and continue editing it?

That's what @spoerri also suggested above. My problem with jj open is that the user might think a corresponding jj close is required.

And if one types jj open with no revision/branch, then, the current commit gets opened?

The current commit is always open (by definition). I think jj open (which I vote for calling jj edit) would be a no-op if we made it default to @ (the working copy), so it's probably best to require an argument.

jj new is pretty much already [...] to check out a commit for building on top (or just run tests or whatever).

Then jj open and jj new would behave a bit like files in the file system? But with commits instead of files. "Open" a file (commit) to continue editing it. Or create a "new" file (commit).

Good analogy.

Sorry again for not having read everything. I did search for jj open in this repository and there doesn't seem to be such a command? And jj edit -r ... was mentioned, but that was in use for sth else already.

Yes, jj edit is currently for editing the content-changes of a commit without modifying the working copy. I tentatively suggested renaming it o jj touchup to make room for the the new behavior under jj edit. jj open exists by I practically never use it. It's the opposite of the current jj close - it flips the "is_open" flag on a commit from false to true.

(jj continue ... is another thought)

Makes sense, but I think it's too likely to be associated with hg continue/git/hg rebase/merge/etc. --continue.

Does jj new sound good if one just intends to look at the code or run tests? I wonder if jj get would be more accurate than "new", and simpler to understand than "checkout" for people not already familiar with other VCS:es.

I actually think jj checkout for the purpose of just taking a look at the working copy at a commit makes sense, since "check out" also means "take a look at" :) jj new makes more sense for the use case of starting a new change on top of the current one, though. Maybe we should make them very close to synonyms but require jj checkout to be given an argument to make it more similar to the checkout command from other VCSs.

[tp-woven]

A bit late to the discussion, but my 2c as a user:

Though I agree that having to learn the open/closed concept is an extra burden for users, in practice I really like the workflow where I can just check-out a commit, start working on it and, for the most part, jj will do "the right thing". More specifically, I think having a different way to "check-out" a commit for modification vs. viewing would be pretty inconvenient (commit vs. amend is OK because the decision happens after I've made the changes so I know which one I want, if that makes sense... Or maybe I'm just too used to thinking about it in Git terms.)

[martinvonz]

That's a fair point. I think I'm going to try adding support for both workflows to start with. That would let some of us try the jj checkout/edit workflow in practice before we remove the jj close workflow (if that's what we decide).

[martinvonz]

From the first message in this discussion:

The automatic checkout we do when commits get rewritten and abandoned needs some thought. When the current checkout is abandoned, we should create a new commit for it. When the current checkout has been rewritten, we should just update to it without creating a new commit on top.

Here are some case additional cases where we rely on the current default behavior for updating the working copy (creating a new commit on top iff the new commit is closed). In some cases, we will have to change the code to preserve a behavior similar to today, and in some cases we can't do that and we instead get a different behavior.

1. jj close: We will have to manually (in the code) update any working copies to a new commit on top of the closed commit.
2. jj split: The parent commit will no longer be open. When I have split the working copy, I have always wanted the parent commit to be closed (i.e. using it like hg commit -i).
3. jj squash: When squashing the working copy into another open commit, the result is that the squashed commit gets checked out. If we don't have open commits, a new commit will be created on top instead (just like if you squash into closed commit today). That makes it behave like hg amend.
4. jj abandon: When the working copy is on top of another open commit and you abandon the working copy, we currently check out the parent commit (since it was open). We will now create a new commit on top instead.

[martinvonz]

With PR #390 merged, if you want to experiment with the new workflow, add this to your ~/.jjconfig.toml:

[ui]
enable-open-commits = false

That will make jj checkout always create a new commit on top. It will also render open and closed commits the same way (in blue) in e.g. jj log so you can see how it works to not rely on that information. The old jj edit (for editing a commit without touching the working copy) is now called jj touchup and a new jj edit can be used for editing a commit in the working copy (like what checking out an open commit used to do).

I've used this workflow for only a day or two and it's worked fine so far. I haven't wanted to use the new jj edit yet. Let me know how it works for you once you've used it for while (or sooner if you quickly find it annoying).

[mackross]

I kind of like the open/close status, it seems to just always do the right thing for me. I also use it for tracking whether I'm finished the work – just my 2c (working alone with JJ so no team sync issues around what open/closed means).

[ilyagr]

I started using jj a week ago, when open commits were already gone, and I somehow missed the existence of jj edit for a while. By trial and error, I found that jj checkout && jj unamend does approximately what I need, but it was an annoyance.

I was mostly looking for this when I'd be using git unstash, for example if I was working on A, jumped to working on B, and want to go back to working on A. When I looked at the list of jj commands for what I needed, my eye caught uncommit, open, and touchup, which all did something different and somewhat confusing.

I would suggest that, when open commits are disabled, uncommit and open act like jj edit and default to acting on the @- revision. It may also make sense to discourage the use of these commands on a commit that have non-empty descendants. In that case, a warning could be printed recommending a checkout & amend workflow, with an option to override it. (In fact, testing jj edit on a commit with many descendants, I just ran into some sort of race condition between VS code, jj log, and jj undo that put the repo in a confusing state)

Other than this, not having open commits seems fine. Nothing was such an annoyance as to motivate me to enable them and learn how to use them.

[martinvonz]

I somehow missed the existence of jj edit

Oh, we should at the very least make sure we mention it in the tutorial! I would have thought that we did, but we apparently don't. Sorry! I'll see how I can update the tutorial to describe it.

I would suggest that, when open commits are disabled, uncommit and open act like jj edit and default to acting on the @- revision. It may also make sense to discourage the use of these commands on a commit that have non-empty descendants. A warning could be printed in that case, with an option to override it.

I think I would prefer to hide or even disable the commands when open commits are disabled (hiding is easier - just need to add a clap attribute). I'd like to also hide or disable jj close, but jj commit can still be useful as a shortcut for adding a commit message and starting a new commit, so maybe I should make jj commit a proper command first (it's currently an alias for jj close).

Other than this, not having open commits seems fine. Nothing was such an annoyance as to motivate me to enable them and learn how to use them.

Thanks for the feedback. That's a useful datapoint.

[ilyagr]

Adding jj edit to the tutorial would help, I think. Though, the reason I suggested a warning in my previous comment is that I think of it as a relatively dangerous command if used carelessly. It forces a rebase of every descendant commit on every jj log that follows an editor save. As I mentioned in an edit to the previous comment, it's a good way to trigger any bugs jj's conflict resolution, jj git import, or jj git export may have with race conditions. I would also guess it's relatively slow and grows the size of the repository storage.

In terms of naming, uncommit makes a lot of sense as an alias for jj edit @- IMO. In my example of working on A, jumping to B, and wanting to go back to A, my first instinct was to do jj checkout A, which gets me to a state as though I've just done jj commit on A. So, I wanted to "uncommit".

On the other hand, this doesn't take into account whatever other changes to names you might have in mind. Perhaps it's best to wait and reorganize the name and function of all the commands together before 1.0.

[martinvonz]

Though, the reason I suggested a warning in my previous comment is that I think of it as a relatively dangerous command if used carelessly.

We hope to add support for public/draft phases like Mercurial has, or something similar at least. That should remove most of the danger because you wouldn't be allowed to rewrite public commits then.

A warning still seems appropriate. I'll look into adding that.

(In fact, testing jj edit on a commit with many descendants, I just ran into some sort of race condition between VS code, jj log, and jj undo that put the repo in a confusing state)

A bug report for that would be useful, if you remember what you did. Even if you don't remember the details and you're able to share the repo, we might be able to figure out based on the operation log.

In terms of naming, uncommit makes a lot of sense as an alias for jj edit @- IMO. In my example of working on A, jumping to B, and wanting to go back to A, my first instinct was to do jj checkout A, which gets me to a state as though I've just done jj commit on A. So, I wanted to "uncommit".

Make sense.

We'll have to think about how it should behave if the working copy has changes. You would probably want to move such changes into the parent (just like jj squash does). So maybe a jj uncommit command would be like jj squash && jj edit @-.

Also, all current commands work on any commit and without treating the working copy differently. The obvious extrapolation would be to make jj uncommit X = jj squash X && jj edit X- (except that it would actually edit the rewritten X-), with a default of @. Perhaps that could be made an option of jj squash (e.g. jj squash --edit/-e). I suppose we would have the same flags on jj unsquash and jj move for editing the target commit afterwards, mostly for consistency.

I'm not completely sold on that idea and would be happy to hear thoughts from others.

[ilyagr]

Even if you don't remember the details and you're able to share the repo, we might be able to figure out based on the operation log.

The repository should be attached here: lf_repo.tgz. If you extract it and run jj op restore 5366, you should get to the confusing state I mentioned:

@ 2da121558bfc 3c8314e49844 ilyagr@github 2022-09-03 19:42:28.000 -07:00    conflict
| (no description set)
| o c0c143495006 09745de478ae ilyagr@github 2022-09-04 11:49:32.000 -07:00
|/  (no description set)
o 69de62bb8b54 3715a209bf48 ilyagr@github 2022-09-01 19:21:58.000 -07:00 ilya-new   HEAD@git divergent conflict
| Stop focus events when launching programs
o 7e246fa123f6 5f8827b378a9 ilyagr@github 2022-08-30 00:15:27.000 -07:00    divergent conflict
| When losing focus, dim cursors instead of removing them
o-.   30f57e3eaf5e c014cf4a03c9 ilyagr@github 2022-08-30 00:13:16.000 -07:00    divergent
|\ \  Merge pull requests into ilya
o | | 6e240018bcde ccffc1e4ee57 ilyagr@github 2022-08-29 22:05:54.000 -07:00 dim-preview-cursor
| | | For directories in preview column, dim cursor
| | | o c0f2616162aa 2132926dfe6c ilyagr@github 2022-08-26 23:04:30.000 -07:00 list-more-keys
| | | | Ugly addition of cmaps
| o | | dd5dc591c4d4 8caf9f43cd3f ilyagr@github 2022-08-29 23:45:57.000 -07:00 minor-fixes
|/ / /  Minor corrections
| | | o 19c4c64af972 47aab51c79fd ilyagr@github 2022-08-28 20:29:31.000 -07:00 ilya
| | |/  For directories in preview column, dim cursor

My memory of it is that I did jj edit, edited something, and then jj undo, all with watch jj log running in the background and VS Code running as well treating the repo as a git repository. This resulted in confusing state with conflicts and "divergent" commits. Happily, it wasn't that hard to recover from in that all the important commits were still there and I just had to abandon the confusing ones.

[chooglen]

I think we want to preserve how changes to the working copy will always result in amending the working copy commit, as opposed to creating a new commit on top when the first changes are made.

This is unlikely to work well with submodules. To illustrate why, consider a naive implementation of checkout with submodules where we create a working copy commit on every submodule. Since each submodule is represented in the superproject by their commit SHA, we've marked every submodule as dirty relative to the parent! In that situation, it would be much more desirable to lazily create the submodule working copy commit upon a snapshot. An alternative set of checkout and open/closed semantics might be useful here:

• When @ points to an open commit, new changes are amended into @.
• When @ points to a closed commit, a new open commit is created on top of @ and checked out, then new changes are written into the new @.

With this scheme, we can make jj edit and jj checkout more similar while still preserving the amend vs commit distinction:

• jj checkout -r foo will set @ to foo.
• jj edit -r foo will set @ to foo and open it.

And since jj checkout no longer needs to create a working copy commit, checking out a submodule becomes a lot simpler. This is also useful once you start working on the submodule - as long as your WIP commit is open, you will amend it when you check it out.

This does complicate some matters though, since @ is no longer equivalent to the 'working copy'. That is still true if @ is open, but if @ is closed, it can be said that there is no working copy or that the working copy is empty. We would have to figure out what relies on that assumption and change that.

[martinvonz]

This is unlikely to work well with submodules. To illustrate why, consider a naive implementation of checkout with submodules where we create a working copy commit on every submodule. Since each submodule is represented in the superproject by their commit SHA, we've marked every submodule as dirty relative to the parent!

I had been thinking that we would not create working-copy commits for submodules. When snapshotting the working copy and there had been changes in a submodule, we could then create a new commit on top of the current commit. If there are further changes in the working copy and we snapshot again, we could decide that the commit should be amended (instead of creating a new commit on top) by noticing that the submodule commit is already different from what the superproject's parent commit says. Do you think that would work?

[chooglen]

Doesn't this mean that the submodule commit can only be at most 1 commit ahead of what the superproject says? It should work in a lot of cases, but not e.g. if you're developing a stack of commits on the submodule to send as a PR and you only want the superproject to point to the tip of the PR.

[martinvonz]

Hmm, good question. What UX do you have in mind for creating a stack of commits in a submodule? Maybe we have jj new --submodule foo?

[chooglen]

Yeah, I think we want to keep it very close to the existing set of commands and have an option to pick which submodule we want to operate on.