UX design notes

[PoignardAzur]

Hello! I mentioned on the reddit announcement thread that I had recently written down some notes for a "rust git" project idea. I recently found these notes, and after editing them a bit, I thought I'd give you the highlights.

This isn't very structured, but hopefully it should be useful. Feel free to shoot down anything that doesn't match your vision of gitoxide.

UX goals

Start every command with a unique 2-letters prefix

Every gix command should have a unique prefix. In other words, there should be no two subcommands starting with the same two letters (eg gix foobar and gix fofo). There are two reasons for this: to encourage distinctive command names, and to make commands faster to tab-complete.

Git is especially bad with this; you have git status and git stash, and you have rebase, restore, rerere, reset, revert, etc. Git also suffers from the "command names aren't distinctive enough" problem; eg it's not immediately obvious what the difference between git restore and git reset is.

Cover common operations with well-named commands

If you look at Stack Overflow's most common git questions, a lot of them are "I want to do VERY_COMMON_THING. How do I do it?", and the answer turns out to be "use this ambiguously named command with these specific flags". In particular, git reset is a bad offender.

Some specific suggestions:

• There should be a gix unstage command and/or a gix add --unstage option.
• There should be a gix setfile command, that covers features from git restore; eg copying files from other commits, discarding changes.
• Have a gix switch command, same semantics as git. No gix checkout.
• Fold git reset into gix branch or some similarly named command. The idiomatic way to go back to a previous commit should make it clear that you're mutating the branch, not the commits themselves. Something like gix branch --set-commit <commit_hash>.

Ctrl_z command

A common worry for non-expert git users is messing up and putting the repository in an unrecoverable state.

While experienced users know that most operations are non-destructive, that operations like merge and rebase that put the repository in a transition state can trivially be aborted, and that the previous state of the repository can usually be found by reading the reflog, beginner users aren't aware of these things, and end up being very conservative in what they do. If anything goes wrong, they'll usually burn the whole thing down and start over.

Having a general purpose "just undo whatever you just did" command would help users be more confident to try commands and see their effect.

Downsides: such a command would be inherently limited in some cases (for instance, it wouldn't undo file changes from an external editor); and it might be extremely hard to write without being incorrect in corner cases.

Stash handling

Git stashes are a really convenient feature when you want to say "I don't want to care about these changes right now. Just put them in a corner and I'll deal with them later".

Stashes become kind of a pain in the ass in a few cases:

• You want to stash untracked files.
• You want to only stash unstaged files.
• You want to only stash staged files.
• You want to stash both staged and unstaged files.
• The changes you stashed conflict with the changes you've made since.

Proposal: by default, stashing should be a non-destructive equivalent of git checkout -- .. This means:

• Untracked files are stashed by default (equivalent to git stash --include-untracked).
• By default, only unstaged files are stashed.
• Various flags (eg --include-staged, --no-include-staged, etc) to customize this behaviour.

Moreover, stashes should differentiate between staged changes and unstaged changes, and stash them "separately". Currently, if you have both staged and unstaged changes in your repository, doing git stash; git stash pop is equivalent to git restore --staged * (all files are unstaged). Instead it should be a no-op.

Stash merge

Currently, if you stash a change to a file, then stage a conflicting change to the same file, then pop the stash, git will create a merge between the two file versions.

If this wasn't the desired behavior, it's hard for the user to cancel this; git merge --abort doesn't work, and the pop operation had other side effects to the working directory. As far as I can tell, there's no trivial way to restore the working directory to the state it was before the pop (for added confusion, the stash entry is copied instead of being popped).

I'm not sure what the exact semantics should be; stash merging is non-trivial. But at the very least, trying to pop a stash with conflicting changes should fail with a warning; the current behaviour could be accessed with a flag, like --merge-if-necessary.

Local stashes

This is a little more niche, but I think there should be two kinds of stashes: local stashes, which are associated to a branch, and global stashes, which work as before.

By default, gix stash would only look at local stashes. In my experience, it's rare that stashes from a given branch are relevant to another (except when stashes are explicitly used to transfer changes between branches). The existing behavior could be enabled by doing gix stash --global.

Also, I would recommend making it easier to list and browse stashes (maybe with a TUI), but I haven't dug into the question too deeply.

Local gitignore file

Have a file named .gitignore-local that is itself automatically ignored, and whose contents are "added" to the regular gitignore when considering files.

This is useful if you want to have some temporary files (eg MY_NOTES.txt) that you don't want gix to mess with, but you don't want to have to add them to the central .gitignore either.

Diff model

I also had a whole section about the diff model, but it's still WIP. The main points are:

• Gix should do what github does when displaying diffs, and show the differences between two similar lines.
• Gix should be aware of syntax (at least basic stuff like matching parens) and try to group blocks of code together, instead of assuming every line has the same weight.
• Gix should treat indentation as a special case.
• Gix should be able to automatically merge two small changes to the same line.
• Gix should track in-file moves (like copy-pasting an entire function).

(keeping in mind that all of the above are heuristics, and may or may not be used depending on the size of the changes, the available compute time, etc)

[Byron]

Thanks a lot for sharing, these thoughts are super valuable to me as I haven't thought about most of these in detail just yet. As a funny side-note: When reading about these similar sounding subcommands even as a somewhat seasoned user of the git cli my mind was struggling to link these to meaningful operations. Interestingly enough: To this day I just copy-paste the command to unstage something as proposed by git in the git status output as its so wildly unintuitive to me.

As a general idea, for gix tooling I wanted to think along user-journeys that are performed most often, and have one command-invocation to perform it while smartly dealing with edge-cases. The latter can happen with interactive visualizations (in the terminal) of what's going on while showing the user the result of their choice before it is even applied: Think 'git pull' but what to do if it's not a fast-forward? "Rebase? Merge? Here is how each of your choices would affect the commit graph".

I am very inspired by tools like gh (to operate on GitHub PRs) for example which use interactive queries sans visualization to get the information they need and avoid an otherwise complex command-line. The latter is, of course, optional in case you know what you want (as in "always merge please, don't ask").

Another nice side-effect could be that gix stash wouldn't have to be a thing at all, as it's merely a side-effect of another operation, like switching branches. From there all other operations you can apply will deal with (or query about) the stashed changes. Not leaving you in a state with merge-conflicts is a no-brainer, that possibility alone makes me fear each git stash pop.

As you can see for gix there is only one path, the one of outstanding UX and usability for anyone to make it the go-to tool for using git. And yes, it should also be faster, a promise that right now it still delivers on, but I am getting ahead of myself :D.

I hope you will be able to stay around as I would be glad to involve you/ask for feedback on first versions of actual user facing tooling - this will most certainly make them much better from the start.

Thanks a million.

[PoignardAzur]

Some additional comments about diffs:

The official git diff is actually more powerful than most people know. I think some of the suggestions I made can be implemented by passing the right flags to git diff or in the config. For instance, patience diff is a lot more powerful than the default diff. (I have patience diff set up in my config, but I thing editors like Atom use regular diff.)

There have recently been something of a surge of diffing tools. I found these in particular through HN:

• https://news.ycombinator.com/item?id=27768861
• https://fazzone.github.io/autochrome.html
• https://prettydiff.com/

I don't think any of those are appropriate for use in gitoxide, though. Some of them explicitly don't produce patches; but beyond that, they focus on parsing code files in ideal cases (in autochrome's case, Lisp syntax) where the entire file can be parsed into an AST; and they focus on displaying a visual diff, over emitting a machine-readable diff format that can be used by other tools.

I think there's room in the rust ecosystem for a standalone diff crate. That crate would:

• Have both a library and a CLI version.
• Try to provide good performance even with extremely large files; avoid schemes that require storing the entire file as an AST.
• Be able to parse some information from files. (In particular, knowing what is a comment and what is code would be helpful. Having an idea of how lines are grouped semantically would help too)
• Degrade gracefully when facing syntax errors, unknown languages, very large files, etc.

Also, before this project even got started, it would need a huge battery of test cases. These test cases would mainly be pairs of files for the tool to diff, with some metrics for what the best diff would be. (eg keeping blocks of code together, minimal number of diff lines, etc)

[Byron]

Thanks a lot for sharing!

git-diff currently provides the bare minimum of diff functionality, such as the ability to diff trees and see which objects changed and at which path that happened. From there any diff algorithm could be fed.

Since there is a wide variety of ways to do decent diffs, my stance would be to let the application decide which one to use. With a little more experience of what's available, maybe git-diff could provide support for a few standard algorithms based on existing crates, like git-delta. That way, inventing or implementing an entire diff algorithm could be avoided or postponed, it seems like a project of epic proportions on its own depending on the feature set.

[PoignardAzur]

More UX thoughts:

Gix should have first-class support for partial clones.

Git has a complicated history on non-full clones. If you only want to download enough data to get the latest commit, Stack Overflow is likely to recommend git clone --depth=1 <url>, aka shallow clones. However, Github recommends avoiding shallow clones, and recommends partial clones instead, eg: git clone --filter=blob:none <url>.

Problems:

• Good luck remembering what I just told you. Which one are you supposed to avoid, shallow clones or partial clones? This is yet another legacy naming problem.
• The syntax for both shallow clones and partial clones is kind of hard to remember. If you tell someone about shallow clone, they'll expect to use git clone --shallow, but it's actually more complicated.
• In the case of --filter, most of the options aren't even documented in the git-clone documentation, you need to look at the git-rev-list doc instead.

Overall, the --filter option is powerful, but it's overkill for simple cases. Gix should have stronger defaults.

Speaking of defaults... subjectively speaking, I don't really think downloading the entire repository every time is a good default. Collectively, it means millions of developers are sending billions of requests to servers to package, compress, and send gigabytes of data that ends up never being read. It's a tremendous waste of energy and server time. Because build systems almost always use the default git clone, it means they're downloading a ton of data the build doesn't need, needlessly extending build times.

Individually, it makes the average clone slower than it needs to be, because most developers don't want to spend time figuring out git clone options (and probably don't even know about partial clones).

I don't think a last-commit-only partial clone would be a good default either. Some times developers need to compare to previous commits, find who made a change, etc. Trying to make a git diff and getting stalled because Git needs to download blobs would be a poor default experience; especially since developers hate annoyances that don't come from their own decisions. (Also, it can make work impossible if a user finds themselves offline for a prolonged period of time; in practice, I'd expect these cases to be rare)

That said, there's probably a middle-ground to be found. Some suggestions:

• Let the user set the default clone filter in .gitconfig.
• Treat file types differently. For instance, images and the like probably don't need history by default.
• By default, filter blobs changed before a certain date. For instance, users are more likely to diff/checkout/blame changes made less than 12 months ago, than 10-yo changes. A one-year cutoff would make it tremendously easier to clone very large, old repositories, without hampering user's abilities to get diff in most cases.
• Have an easily-remembered flag for partial clones. Something like gix clone --single-commit.

On the other hand, if really don't want to risk users getting blindsided because they thought they'd had a full clone and they can't fetch anymore, you could do full clones by default, but add a warning message that recommends changing the default in .gitconfig.

Anyway, that was my take on the subject. Hope it was helpful =)

[Byron]

Thanks so much for sharing (and please keep doing this as ideas come up :)). When reading the text, my eyes widened for a moment in awe, getting a glimpse of what could be: faster clones made easy. I agree with everything said, and changing the default clone behavior to become a partial clone by default probably isn't going to work initially as it goes against what we have come to expect of git. However, making such things easy and/or obvious with defaults which are suitable for most could certainly help and reduce clone times significantly while retaining the complete history.

Furthermore, the gitoxide program could detect if it runs on CI and clone differently by default, maybe along the lines of depth=1, helping every user, whether robot or not, get the best defaults which are of course all adjustable using the git configuration.

Faster clones could be a real selling point, so I truly love the idea.

Reading this also made me realize that this program really isn't a git anymore, so the name of the binary should reflect that.

Collectively, it means millions of developers are sending billions of requests to servers to package, compress, and send gigabytes of data that ends up never being read.

Besides me agreeing that transmitting less objects is less costly, especially if those would be unlikely to ever be read, let me amend that the major cost of unnecessary objects is in their transmission. Compression doesn't play a role as new (loose) objects have to be recompressed anyway, and older one are likely to be packed already. If it's the latter, git will only do pack-to-pack copies, without the need to decompress and recompress. The counting of objects is accelerated too.

[davidbarsky]

I came across this discussion because I'm really excited by gitoxide and decided to poke around :). On the off-chance that y'all didn't see gitless, I recommend checking it out! They did some neat research to identify where git users struggle with git, both mechanically and conceptually—it could be really useful as gitoxide evolves!

[Byron]

Thanks a lot for sharing, I will make sure to study this as well before starting any work on ein, a name chosen to ideally be the opposite of git. It's a smart tool to assist in doing the right things, without hassle.

I think even though it's going to learn from as many sources as possible it will try to go a step further, my intuition is to not actually give it any sub-commands but to make it predominantly a TUI. At the current pace it's probably years away to actually do so, I will be writing here once new information materializes.

[martinvonz]

I have been working on an alternative Git CLI for the past two years (it's actually its own VCS but it has a lot of Git interop). That’s pretty much ready; I have used it myself for almost a year and I don’t miss much from Git or Mercurial. Using that instead of starting essentially from scratch (AIUI) with ein could be an option. Please check out the features on https://github.com/martinvonz/jj (it has unlimited undo, for example). I haven't documented the architecture there yet, unfortunately.

jj is a bigger departure from Git's UX than most other VCSs are (Pijul being a notable exception). There's no staging area, for example. That means it won't be perfectly smooth to switch between jj and git commands in the same repo. I don't know if you had intended for ein commands to be easily interleaved with git commands in the same worktree and be able to understand staged changes, interrupted rebases, etc.

I will probably try switching from git2-rs/libgit2 to gitoxide once you have support for push, and especially if you add support for partial clones (which libgit2 doesn’t support).

[martinvonz]

Btw, @Byron, jj is ready to be used. Consider trying it for your own development of gitoxide (since my impression is that gix is not ready yet). If nothing else, it might be useful as inspiration for gix.

[Byron]

Thanks for letting me know! As an avid user of Stacked Git I wonder if jj would support a similar workflow or might add to it in meaningful ways. Just now I have added git-absorb to my wishlist - maybe jj is already all of that and I simply wasn't aware.

As a side-note ,gix isn't intended to be a git replacement or be useful for everyday work. If it is, it's a side-effect as its main purpose is to help with using gitoxide implementations of git features in the real world. Thus it's a development tool primarily.

ein, however, might one day be a tool that implements 'my perfect workflow', if I ever get to it, and to do that there is definitely a whole lot of workflows to soak in and understand to be able to find meaningful improvements to what most people might want (or don't know they want). So yes, jj is welcome in my tool-belt, I hope you could give me this one killer-feature (or maybe multiple even) that everybody uses or that you think I might want to use. Thanks for your advice.

[martinvonz]

I've never used Stacked Git, but my impression that jj provides the same functionality, but in different ways. It makes working with stacks of commits much simpler than Git does. You can edit the description of any commit in the stack without having to manually rebase descendants, for example. And you can make changes in the working copy and the run jj move --to <target commit> to move those changes (diffs, not states) into an arbitrary ancestor (or even an unrelated commit). That command replaces many of the uses of git absorb, so I rarely miss having that command (and I more frequently miss having jj move when I use hg absorb).

And yes, sorry, I meant ein when I said gix :)

I hope you could give me this one killer-feature (or maybe multiple even) that everybody uses or that you think I might want to use

• The working copy is a commit that's automatically created, so no commands will complain that the working copy is dirty. This also makes the CLI simpler and more consistent (e.g. git checkout -m is jj rebase -r @ to rebase the working-copy commit).
• I think the automatic rebasing is very useful. It's what makes it easy to edit ancestors without having to worry about rebasing descendants. Branch pointers get automatically updated.
• First-class conflicts means that rebase etc. won't get interrupted by conflicts. It's also what enables the auto-rebase feature to work in every case.
• The entire repo is under source control, and you can undo any operation (jj undo) or go back to an earlier state (jj op restore).

Also, note that you can use jj and git in the same working copy and backed by the same git repo, so if jj is lacking some feature (like blame or tagging), you can use regular git for that.

[Byron]

Thank you! I will definitely try it, but before I do just one last question. Is running jj init --git-repo=. without using it (or switching back and forth between jj and git) safe to do? I found the answer: - probably. Once stg gets thrown in there as well it will be hell of a VCS soup ;).

[martinvonz]

Yes, it mostly works pretty well, I'm told. The main issue is that the export to git can sometimes get confused after you undo operations.

[PoignardAzur]

Another interesting UI pattern to steal: git-absorb (itself taken from mercurial).

[Byron]

That's a great hint and… a tool I would find useful as well! Thanks so much.

Here is the analysis for oxidization - I couldn't resist.