Conclude `gitoxide` to `git2` transition #1285

Byron · 2024-03-13T10:19:31Z

Fixes #628

I had to wait a long time for this PR to become possible, and finally here it is :)!

Tasks

transition the status computation
remove git2 dependency
Switch to new public release of gix

Review Notes

Deviation: I think it's better to not write the index back if any stat-changes are picked up to make onefetch truly read-only.
If there is disagreement, the index could be updated and written back - it will be valid, but won't write back all extensions that might
have been present before. git2 most certainly also didn't do it in a way that is 100% faithful to what Git would do.
renames_head_to_index() was never functional, as the status was only obtained between the index and the worktree. If this must be present
with the switch, then I have to hold off with this update as gix status doesn't yet support HEAD->index status.
The binary size isn't positively affected by this change, unfortunately. The unstripped binary is now 17710984, up from 17023992, with the build settings of this project.

Performance

In short, onefetch got a little bit faster, and speeds up with the size of the repositories measured by the amount of files in their worktrees.

WebKit

Here the faster status implementation is definitely visible.

❯ hyperfine -w1 -N -r3 onefetch /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch
Benchmark 1: onefetch
  Time (mean ± σ):     22.839 s ±  1.328 s    [User: 80.878 s, System: 17.448 s]
  Range (min … max):   21.898 s … 24.358 s    3 runs

Benchmark 2: /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch
  Time (mean ± σ):     18.506 s ±  0.145 s    [User: 77.921 s, System: 26.274 s]
  Range (min … max):   18.378 s … 18.663 s    3 runs

Summary
  /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch ran
    1.23 ± 0.07 times faster than onefetch

Linux

For linux, the commit-aggregation is the most stressful part, and there is much less of a difference as status is just a small part of the overall runtime.

linux (ffc2532)
❯ hyperfine -w1 -N -r3 onefetch /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch
Benchmark 1: onefetch
  Time (mean ± σ):     12.166 s ±  0.049 s    [User: 23.918 s, System: 2.509 s]
  Range (min … max):   12.110 s … 12.200 s    3 runs

Benchmark 2: /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch
  Time (mean ± σ):     11.828 s ±  0.031 s    [User: 24.007 s, System: 2.904 s]
  Range (min … max):   11.801 s … 11.862 s    3 runs

Summary
  /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch ran
    1.03 ± 0.00 times faster than onefetch

Git

Git is interesting as it doesn't have too many files in the worktree, yet it seems to greatly benefit from the new status implementation. Unfortunately, at ~200ms, the difference isn't really perceivable by a user.

❯ hyperfine -w1 -N -r3 onefetch /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch
Benchmark 1: onefetch
  Time (mean ± σ):     237.5 ms ±   4.5 ms    [User: 883.7 ms, System: 223.4 ms]
  Range (min … max):   234.2 ms … 242.6 ms    3 runs

Benchmark 2: /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch
  Time (mean ± σ):     213.7 ms ±   3.5 ms    [User: 872.6 ms, System: 235.9 ms]
  Range (min … max):   210.6 ms … 217.5 ms    3 runs

Summary
  /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch ran
    1.11 ± 0.03 times faster than onefetch

spenserblack · 2024-03-14T17:39:32Z

src/info/pending.rs

+                Summary::Removed => (added, deleted + 1, modified),
+                Summary::Added => (added + 1, deleted, modified),
+                Summary::Modified | Summary::TypeChange => (added, deleted, modified + 1),
+                Summary::Renamed | Summary::Copied => (added + 1, deleted + 1, modified),


Watching the conclusion of switching from git2 to gix is awesome!

Should deleted really be incremented when the status is Copied? AFAIK a rename is always a delete + add, while a copy just means a new file was created with extremely similar contents to an existing file.

That's a great catch right there! Copied should only count as added file.

This Summary was added just for onefetch and it was interesting to see that the possible states indeed are best represented by an enum, even though libgit2 tries to convince you otherwise.

Byron · 2024-03-14T19:29:22Z

While I prepare the gix release to ready this PR for review, I recommend trying it out on repos with modifications. I noticed that in my case on the linux kernel, the git2 version showed removals where there were none. Thus, gitoxide seems to be (more) inline with Git.

❯ gst
HEAD detached at v6.6
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   include/uapi/linux/netfilter/xt_DSCP.h
        modified:   include/uapi/linux/netfilter/xt_MARK.h
        modified:   include/uapi/linux/netfilter/xt_RATEEST.h
        modified:   include/uapi/linux/netfilter/xt_TCPMSS.h
        modified:   include/uapi/linux/netfilter/xt_connmark.h
        modified:   include/uapi/linux/netfilter_ipv4/ipt_ECN.h
        modified:   include/uapi/linux/netfilter_ipv4/ipt_TTL.h
        modified:   include/uapi/linux/netfilter_ipv6/ip6t_HL.h
        modified:   net/netfilter/xt_DSCP.c
        modified:   net/netfilter/xt_HL.c
        modified:   net/netfilter/xt_RATEEST.c
        modified:   net/netfilter/xt_TCPMSS.c
        modified:   tools/memory-model/litmus-tests/Z6.0+pooncelock+poonceLock+pombonce.litmus

no changes added to commit (use "git add" and/or "git commit -a")

linux (ffc2532) +369 -819 [!]
❯ onefetch && /Users/byron/dev/github.com/o2sh/onefetch/target/release/onefetch
                 ++++++                    Sebastian Thiel ~ git version 2.39.3 (Apple Git-146)
              ++++++++++++                 ----------------------------------------------------
          ++++++++++++++++++++             Project: linux (2 branches, 820 tags)
       ++++++++++++++++++++++++++          HEAD: ffc253263a13
    ++++++++++++++++++++++++++++++++       Pending: 12+- 13-
 +++++++++++++************+++++++++++++    Version: v6.7-rc8
+++++++++++******************++++++++;;;   Created: 18 years ago
+++++++++**********************++;;;;;;;   Languages:
++++++++*********++++++******;;;;;;;;;;;              ● C (98.4 %) ● Shell (0.7 %)
+++++++********++++++++++**;;;;;;;;;;;;;              ● Python (0.3 %) ● Makefile (0.3 %)
+++++++*******+++++++++;;;;;;;;;;;;;;;;;              ● Perl (0.2 %) ● Rust (0.1 %)
+++++++******+++++++;;;;;;;;;;;;;;;;;;;;              ● Other (0.0 %)
+++++++*******+++:::::;;;;;;;;;;;;;;;;;;   Authors: 3% Linus Torvalds 35041
+++++++********::::::::::**;;;;;;;;;;;;;            1% David S. Miller 14200
++++++++*********::::::******;;;;;;;;;;;            1% Arnd Bergmann 10387
++++++:::**********************::;;;;;;;   Last change: 4 months ago
+++::::::::******************::::::::;;;   Contributors: 35615
 :::::::::::::************:::::::::::::    URL: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
    ::::::::::::::::::::::::::::::::       Commits: 1217245
       ::::::::::::::::::::::::::          Churn (491): …/usb/r8152.c 8
          ::::::::::::::::::::                          …/netfilter/nf_tables_api.c 5
              ::::::::::::                              MAINTAINERS 4
                 ::::::                    Lines of code: 17644823
                                           Size: 1.26 GiB (81766 files)



                 ++++++                    Sebastian Thiel ~ git version 2.39.3 (Apple Git-146)
              ++++++++++++                 ----------------------------------------------------
          ++++++++++++++++++++             Project: linux (2 branches, 820 tags)
       ++++++++++++++++++++++++++          HEAD: ffc253263a13
    ++++++++++++++++++++++++++++++++       Pending: 13+-
 +++++++++++++************+++++++++++++    Version: v6.7-rc8
+++++++++++******************++++++++;;;   Created: 18 years ago
+++++++++**********************++;;;;;;;   Languages:
++++++++*********++++++******;;;;;;;;;;;              ● C (98.4 %) ● Shell (0.7 %)
+++++++********++++++++++**;;;;;;;;;;;;;              ● Python (0.3 %) ● Makefile (0.3 %)
+++++++*******+++++++++;;;;;;;;;;;;;;;;;              ● Perl (0.2 %) ● Rust (0.1 %)
+++++++******+++++++;;;;;;;;;;;;;;;;;;;;              ● Other (0.0 %)
+++++++*******+++:::::;;;;;;;;;;;;;;;;;;   Authors: 3% Linus Torvalds 35041
+++++++********::::::::::**;;;;;;;;;;;;;            1% David S. Miller 14200
++++++++*********::::::******;;;;;;;;;;;            1% Arnd Bergmann 10387
++++++:::**********************::;;;;;;;   Last change: 4 months ago
+++::::::::******************::::::::;;;   Contributors: 35615
 :::::::::::::************:::::::::::::    URL: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
    ::::::::::::::::::::::::::::::::       Commits: 1217245
       ::::::::::::::::::::::::::          Churn (445): …/usb/r8152.c 8
          ::::::::::::::::::::                          Makefile 4
              ::::::::::::                              …/misc/fastrpc.c 4
                 ::::::                    Lines of code: 17644823
                                           Size: 1.26 GiB (81766 files)

Byron · 2024-03-14T19:31:56Z

src/info/pending.rs

+                Summary::Removed => (added, deleted + 1, modified),
+                Summary::Added | Summary::Copied => (added + 1, deleted, modified),
+                Summary::Modified | Summary::TypeChange => (added, deleted, modified + 1),
+                Summary::Renamed => (added + 1, deleted + 1, modified),


By the way, it won't run into Renamed and Copied as rename tracking isn't activated by default.

Can you expand on that, what are the implications ?

This would mean the code could call unreachable!() in case of Copied and Renamed and be right about it. The implementation/settings have the same effect as git2 previously.

Does that make more sense?

Are you refering to git config --get diff.renames ? Whether it's enable doesn't change the output for onefetch, right? a rename is always shown as an addition and an deletion.

That's a great point to bring up! I decided not to let it affect renames between index and worktree as Git itself doesn't even expose that functionality to the user (only git2 seems to have it). Everything that is affected by configuration, really only emit_untracked at this time, is overridden though.

o2sh

Finally, onefetch is fully oxidized 🎉

The missing piece finally came through.

Just tested it and #978 seems to be resolved

Can the PR be merged?

Byron · 2024-03-16T21:55:18Z

Great to hear that another issue is fixed :).

This PR is good to go from my side.

vercel bot deployed to Preview March 13, 2024 10:19 View deployment

Byron force-pushed the gitoxide-for-status branch from b6feeb6 to a942922 Compare March 14, 2024 16:50

vercel bot deployed to Preview March 14, 2024 16:50 View deployment

Byron mentioned this pull request Mar 14, 2024

rename-tracking for gix-status Byron/gitoxide#1306

Merged

26 tasks

Byron force-pushed the gitoxide-for-status branch from a942922 to 68b0e42 Compare March 14, 2024 16:53

vercel bot deployed to Preview March 14, 2024 16:54 View deployment

spenserblack reviewed Mar 14, 2024

View reviewed changes

Byron force-pushed the gitoxide-for-status branch from 68b0e42 to dfc0831 Compare March 14, 2024 18:03

vercel bot deployed to Preview March 14, 2024 18:03 View deployment

Byron commented Mar 14, 2024

View reviewed changes

Byron added 3 commits March 14, 2024 21:41

Switch to latest version of gitoxide for status support

626747c

Use gitoxide for obtaining status information

aede58d

Remove git2 as it's not used anymore

9c15105

Byron force-pushed the gitoxide-for-status branch from dfc0831 to 9c15105 Compare March 14, 2024 20:42

vercel bot deployed to Preview March 14, 2024 20:42 View deployment

Byron marked this pull request as ready for review March 14, 2024 20:50

Byron requested a review from o2sh as a code owner March 14, 2024 20:50

o2sh approved these changes Mar 16, 2024

View reviewed changes

o2sh merged commit 8fcaf08 into o2sh:main Mar 16, 2024
12 checks passed

o2sh added the chore label Mar 16, 2024

Byron deleted the gitoxide-for-status branch March 17, 2024 06:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Conclude `gitoxide` to `git2` transition #1285

Conclude `gitoxide` to `git2` transition #1285

Byron commented Mar 13, 2024 •

edited

Loading

spenserblack Mar 14, 2024

Byron Mar 14, 2024

Byron commented Mar 14, 2024

Byron Mar 14, 2024

o2sh Mar 16, 2024

Byron Mar 16, 2024

o2sh Mar 16, 2024

Byron Mar 17, 2024

o2sh left a comment

Byron commented Mar 16, 2024

Conclude gitoxide to git2 transition #1285

Conclude gitoxide to git2 transition #1285

Conversation

Byron commented Mar 13, 2024 • edited Loading

Tasks

Review Notes

Performance

spenserblack Mar 14, 2024

Choose a reason for hiding this comment

Byron Mar 14, 2024

Choose a reason for hiding this comment

Byron commented Mar 14, 2024

Byron Mar 14, 2024

Choose a reason for hiding this comment

o2sh Mar 16, 2024

Choose a reason for hiding this comment

Byron Mar 16, 2024

Choose a reason for hiding this comment

o2sh Mar 16, 2024

Choose a reason for hiding this comment

Byron Mar 17, 2024

Choose a reason for hiding this comment

o2sh left a comment

Choose a reason for hiding this comment

Byron commented Mar 16, 2024

Conclude `gitoxide` to `git2` transition #1285

Conclude `gitoxide` to `git2` transition #1285

Byron commented Mar 13, 2024 •

edited

Loading