Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
NixOS troubleshooting with git bisect #17
NixOS is an amazing Linux distribution. The InfoQ article and thesis are well worth your time to read. Meanwhile, here is a new trick I discovered for debugging Linux distribution upgrades using
I upgraded from NixOS 15.07 to 17.03 and found that the Pharo Virtual Machine had broken. Starting the VM would cause a Segmentation Fault within around one second. There was no obvious cause in the Pharo VM code itself: it seemed to be indirectly caused by a change in some dependency. There had been around 35,000 package updates to NixOS between those two releases, so how do you know which one is the problem?
It turns out that you can use
#!/usr/bin/env bash nix-env -j 10 -f . -iA pkgs.pharo-launcher || exit 125 timeout --preserve-status 20 pharo-launcher | grep '(Segmentation fault)' status=$? if [ "$status" == 0 ]; then echo "SEGFAULT" exit 1 else echo "OK" exit 0 fi
Then once I have this script I can ask
git bisect start master 15.09 git bisect run ./pharo-nix-bisect.sh
Finding the bad commit from a set of 35,000 actually only requires around 15 tests because
This test ran for a few hours, testing many different versions of the whole OS including compiler toolchains, etc, and then finally pointed me in the right direction. It turns out that the problem was introduced by adding "hardening" to the default
Truly, this feels like a small step towards "dependency heaven." Thanks, Nix!
I used the same technique to trace this issue down to a guile upgrade and rapidly fix it. Nix and git are both wonderful tools, and in combination they're amazing. On top of that, nix didn't even have to rebuild weechat every time because not all the commits affected weechat's dependencies, and it took just about 10min to find the problem.