Trouble using Cachix with nix develop

[mtlynch]

I'm a newcomer to Nix, and I'm trying to integrate nix develop into some of my open-source projects.

I have it working alright for one of them, but when I tried to switch my CI environment to use Nix, every job got 2m slower because it had to populate the Nix store.

I thought Cachix might be a good solution to this issue, but I'm having trouble getting up and running.

Here's what my current CircleCI config looks like with Cachix:

https://github.com/mtlynch/whatgotdone/blob/1cbde15946c95df95033180ab287adaa71c64a78/.circleci/config.yml

And as a relatively simple example, my lint_sql job looks like this:

https://github.com/mtlynch/whatgotdone/blob/1cbde15946c95df95033180ab287adaa71c64a78/.circleci/config.yml#L94-L112

lint_sql runs in 10s without Nix, but adding Nix balloons the runtime to 2.5m.

I thought I'd get a significant speedup from integrating Cachix, but it doesn't seem to have any impact on performance. From my dashboard, it seems like the CI config is successfully storing some of the Nix store to Cachix:

Does Cachix work with nix develop?

[domenkozar]

Do you have a log of those runs? I'd also recommend using a docker image with cachix installed to avoid installing cachix during the CI itself.

[mtlynch]

Do you have a log of those runs?

Sure, here they are:

• Without Nix/Cachix: https://app.circleci.com/pipelines/github/mtlynch/whatgotdone/2246/workflows/90b8dffb-d7bb-4e4f-80a8-1a4df7b936cf/jobs/8680
• With Nix/Cachix: https://app.circleci.com/pipelines/github/mtlynch/whatgotdone/2257/workflows/091db4f9-7d82-47b1-a346-0f032c99416a/jobs/8750

[domenkozar]

Most likely this is fixed by 8c7825c

[domenkozar]

I just released 1.6.1, it should be part of the official installation tomorrow or a day after.

[domenkozar]

@mtlynch could you try again now with Cachix 1.6.1?

[mtlynch]

@domenkozar - Sure, thanks for publishing the new version!

It seems to have sped it up quite a bit. It's now 86s as opposed to 146s pre-1.6.1.

https://app.circleci.com/pipelines/github/mtlynch/whatgotdone/2257/workflows/113fd2d4-b575-4f80-9169-b3c82f6bc674/jobs/8790

Is that expected behavior?

[domenkozar]

@mtlynch it's not, there should be no overhead at all.

If you remove cachix watch-exec "${CACHIX_NAME}" -- , how long does it take? I am guessing it's nix develop that takes a long time.

[mtlynch]

it's not, there should be no overhead at all.

Right, I'm not expecting overhead from Cachix, but I was wondering how much I should expect Cachix to reduce the time it takes to initialize the Nix shell from the flake.

Is the current speed about what I should expect? Or is there more headroom to gain from using Cachix in a more effective way than what I'm currently doing?

If you remove cachix watch-exec "${CACHIX_NAME}" -- , how long does it take? I am guessing it's nix develop that takes a long time.

96s, so roughly the same as with cachix watch-exec.

[domenkozar]

it's not, there should be no overhead at all.

Right, I'm not expecting overhead from Cachix, but I was wondering how much I should expect Cachix to reduce the time it takes to initialize the Nix shell from the flake.

Is the current speed about what I should expect? Or is there more headroom to gain from using Cachix in a more effective way than what I'm currently doing?

If you remove cachix watch-exec "${CACHIX_NAME}" -- , how long does it take? I am guessing it's nix develop that takes a long time.

96s, so roughly the same as with cachix watch-exec.

That's mostly because Nix needs to download 4x nixpkgs in your case. I'd try to reduce the number of invocations.

[mtlynch]

Ah, interesting. I consolidated to a single nixpkg, which brings it down to 55s.

Is that about maximum expected performance with Cachix?