[October 26] - 30% faster object database!

[Sebastian Thiel (Byron)]

This month, just in time, and a lot happened in the past 28 days. For one, I worked 155 hours in open-source, of which 89 went into Gitoxide, 25 into GitPython (yay, it's moving again), 16h into Rustty, and a whopping 15.5 hours into 'various' maintenance, which includes 4h of rust-lang maintenance like for flate2.
Total work time was 216h, which is probably why the month felt particularly intense.
After all these numbers, let's dive right into… more numbers!

Sirocco: another speedup from Sam Reis

It was just September when Sam Reis swung his magic wand for faster collision-detecting SHA-1. This month he is back for some more magic, because what else would one call this? You'd think that something like DEFLATE-handling in Git is already optimised; after all, it gets to run a whole lot around the world, so nobody seems to even dare to have this thought: what… if not?

Meet Sirocco, a pure Rust decompressor that does two things differently:

1. It's optimised for non-streaming operation, which is the typical workload when decompressing pack entries
2. Scanning for the end of a DEFLATEd stream is much faster if one doesn't actually retain the output

All this leads to real-world wins of raising the measured commit-traversal rate from 187k to 259k commits/s via ein t hours -p, so 1.33x faster. Pack indexing was about 1.27x faster, which is a good portion of any clone.

So yes... now the Gitoxide object database is about 33% faster, just like that, and any operation that reads objects, like diffs or merges, is that much faster, too.

If you too feel the need for speed, add the zlib-sirocco feature to your gix dependency.

Create and remove linked worktrees, with many cores

While I had been using this functionality for a long time already, it was waiting for review and refactoring for longer than I had hoped.
But here we are, and gix::Repository can now create worktrees with the usual multi-threaded checkouts, just like when cloning, and the newly added sparse index support. It can also delete them, with multi-threaded file discovery and deletion, a capability that was tried and tested in dua-cli already.
Locking and unlocking were also implemented for good measure, so I think this feature should be pretty usable for the Git clients out there.

Toward a stable gix API

While it hasn't landed yet, I can't wait to review this PR as it represents the final technical prerequisite (as far as I know) to achieve a stable release of the gix crate.

The idea is to put everything that exposes the mostly forever-unstable plumbing in its types directly behind an unstable feature toggle. That way, and probably after stabilising gix-hash and maybe some parts of gix-object, core operations can already be fully stable, while everything else can be stabilised over time.

Stabilisation probably means a wrapping of plumbing types in gix-owned and Repository-connected types, something that typically helps with usability and fluid API use.

Then, over time, everything can be stabilised so eventually we end up with feature parity and stability.

Tix and Kitten

This was a super-exciting revelation based on something I had completely forgotten about: With the Kitten protocol, terminal emulators can show pixel graphics! Not only can you play games in your terminal…

…but tix can also draw real lines for better graph visualisations.

This, along with optional topological cursor movement and lane-highlighting with the mouse, allows you to literally follow where the development is going.
Also, it can visualise dense strands for 'super-fork-points', typically tags that a lot of work is based on.

There is more work to do, of course, to finally conquer repositories like Git or Linux, but now it seems like a possibility.

Git3 planning

Git3 will happen, and whether it's with or without SHA-256 by default, it will most definitely be with reftable. So that's apparently due by March, and it's very much on my radar, and I hope soon there is nothing but this to work on so ideally it's ready way before that deadline.

Wish me luck 😅

What is Open-Source software?

Recently I thought about this, and in the past, to my mind, the answer would have been: Open-Source software is usually started by one person to fulfil their personal needs, and then given to the public for continued maintenance by them or by the community formed by people who also find it useful.

Now it's still the same thing, except that everyone can now have software to fulfil their specific requirements. Some of these will even be maintained as they are cared for by the original 'prompter', and maybe even a community. As this kind of assisted development is so much faster and easier, I expect the open-source world and software world in general to thrive.

I keep doing open-source, of course, for instance with Rustty, Tmuxx, and Rash. Most of these exist only as forks of other repos, so maybe they may be useful to others. But most importantly, they have to be useful to me, and that's all there is to it.

It's also a great pleasure to scratch these itches, and to own more and more of the stack. Software I use every day should fit my needs, and now… there is no reason not to have that software. Exciting times.

Community

As you can imagine, the number of PRs and security advisories went through the roof, but by now most of them are worthy of being merged. Many of them were bugfixes, which are always appreciated and fixed with priority.

Diff compatibility, with consequences for merges and Jujutsu

Tyler Smith struck again, along with Christoph Rüßler, who kept improving the diff-slider-correctness and diff performance. Both led to significantly better diff and blame conformance, and even merges can now be expected to be more faithful, leading to fewer unnecessary conflicts, something that should be noticeable in GitButler, for instance.

Gix in Cargo

I am currently updating the gix dependency in Cargo, but besides that
there is nothing new here, but let's keep the horizon active:

With GitButler slated to have its checkout driven by a tailor-made implementation of reset in gitoxide, this is coincidentally exactly what Cargo would need to greatly speed up its checkouts and make them more compatible with Git, too. We are talking about proper Git filter support and speedups in the realm of ~7x.

Cheers,
Sebastian

PS: The latest timesheets can be found here (2026).

[达瓦里氏 (zippozeng)]

good job

[35V LG84 (35VLG84)]

This is really cool!

Tackler's build system has git benchmark target (just git-bench), and there is real life ~29% speed up for git based storage system just by updating gix - really nice work, and much appreciated!