Cleaning is broken for stack projects

[quetz]

Using project.shellFor results in multiple (one for each package) copies of whole project directory (including dist-newstyle) in nix store. It happens during clean-source-with-nix evaluation that is part of project {..} evaluation.

Is there any way to avoid it? Growing nix store by several GB on each nix-shell invocation is pretty harsh.

Looks like it is related to sources being needed for cabal setup (but I'm not sure here). In this case, is it possible to manually assemble pkgset and avoid this copying steps?

[hamishmack]

We have changed stuff recently in the way cabal components are clean, so this might be a bug.

Are you using explicit hs-source-dirs for each component in the .cabal file (not the default which is .)? That way the cleaning of the cabal component will know which directories are needed. This will also reduce the number of times haskell.nix will need to rebuild the components (since it will only rebuild if the source for the component changes). Perhaps use hs-source-dirs: src and the component name instead of src for each of the other components.

I would also recommend clean the project src directory before passing it to project. If you use git you can use the src = haskell-nix.haskellLib.cleanGit { name = "project-name"; src = ./.; } which will filter out anything not in the git index. This does mean you have to git add your source files before they will be seen by haskell.nix.

[quetz]

Are you using explicit hs-source-dirs for each component in the .cabal file (not the default which is .)?
Yes.

Using cleanGit helps (in a sense that it does not include full project directory in a nix store) but still there's nix store path for each of project package with exactly same contents (cleanGit results of project directory). Also, noop changes in source files (adding blank line) result in creating fresh new set of those nix store paths on each nix-shell invocation (no project building even required).

Why is it neeed to have full copy of source tree in nix store upon entering nix shell?

[michaelpj]

The source for each package should be filtered further to just be the source of that package.

But it is a bit odd that you have the any source being copied in for each nix-shell invocation. That shouldn't be happening - it only builds the dependencies anyway!

[ocharles]

Is it surprising that the source is in the store? It's not surprising to me - it has to run cabal configure, and ot can only do that on things in the store, no? (This only applies to non-materialised projects)

[michaelpj]

For the configure step we filter it down to only contain cabal.project and the .cabal files. So a) it shouldn't change on changes to source files, and b) it should be really small.

[michaelpj]

Although the way that source filtering step got done was changed recently. @hamishmack do you think we could conceivably be copying the pre-filtering source as well?

[ocharles]

I'm pretty certain this is a regression that came in here:

https://github.com/input-output-hk/haskell.nix/pull/843/files#diff-5176cec122f219c11fae32aa032f2a89ccb02f417f682e169cb52f5b64ec6307R43

This change means that every package now has includeSiblings = true, which means that it ends up having src set to the entire project source tree, rather than just its relative sub-directory. I see @michaelpj said

I do find it tempting to say you should just Not Do That...

and I'm inclined to agree.

[quetz]

What gets created on every nix-shell invocation on noop changes in sources (added empty line):

- <project>-root-<package directory name> (for each package in project)
- <project>-root-<package dir>-lib-<package name>-root (for package that gets its source file modified)
- ... -config.drv (for each executable / library for each package in project)
- ... -ghc-<ghc version>-env.drv (for each executable / library for each package in project)
- ... .drv (for each executable / library for each package in project)

Looks like it all boils down to -root-<dir> store paths that contain whole (filtered) source. And even noop changes are detected by nix producing different hash each time. And then everything else refers to this new path producing different hash also, and so on...

[ocharles]

Nix shouldn't (can't) produce a different hash time, so you must have something changing. If you can find the path (I used inotifytools to watch /nix/store), you can do a recursive diff on them to see what's changing. The hash of a source path is the product of its contents hash + name, and this is stable (if those inputs are stable).

@quetz I suggest reverting the line I linked to in #843, though. Just to reiterate, that's changing true to false here:

haskell.nix/lib/import-and-filter-project.nix

Line 55 in 22ad57c

includeSiblings = true; # Filtering sibling dirs of the package dir is done in the

[quetz]

Nix shouldn't (can't) produce a different hash time, so you must have something changing.

Yeah, but source code is changing (I insert empty line, so this is noop change). It should lead to different hash and if everything else refers to this new source it gets new hash too...

@quetz I suggest reverting the line I linked to in #843, though. Just to reiterate, that's changing true to false here

Will experiment with that, thanks.

[quetz]

@quetz I suggest reverting the line I linked to in #843, though. Just to reiterate, that's changing true to false here:

haskell.nix/lib/import-and-filter-project.nix

Line 55 in 22ad57c

includeSiblings = true; # Filtering sibling dirs of the package dir is done in the

It is much better with this change. Some store paths are still created on each nix-shell invocations, but source code gets reduced to only one component that is changed.

But still, is it possible to get shell in haskell.nix behave as advertised in nix?

The command nix-shell will build the dependencies of the specified derivation, but not the derivation itself.

I want to be dropped into shell where all dependencies of my project are built and registered, but project itself is not touched (do not configure my packages, do not make derivations that can be used to build these packages).

[hamishmack]

How can I reproduce this issue?

[ocharles]

@hamishmack With the following directory structure:

- foo
- bar
|-- bar.cabal
|-- src/.../.hs
- shell.nix
- cabal.project

That is, a directory structure where your .cabal packages are in sub directories. Now make foo a very large file that's not referenced from any .cabal file (i.e., foo is not part of the Haskell build) and observe that it gets copied into the Nix store. If you make multiple .cabal subdirectories, notice that the .drv for building the library component will have a src that contains foo (so foo is being copied for all projects).

[hamishmack]

I can't seem reproduce it that way. See https://github.com/hamishmack/test-1013.

git clone https://github.com/hamishmack/test-1013.git
cd test-1013
nix-shell --run 'echo ok'
echo test >> foo                                                                 
nix-shell --run 'echo ok'

Seems to work as expected for me. No derivations are rebuilt. It works with or without the cleanGit call.

One thing that would not work is if something was triggering the source to be copied to the store before the haskell.nix project function sees it. For instance if the example project was in subDir and we used:

src = haskellLib.cleanGit { src = ./.; } + "/subDir";

Nix will be forced to copy the all of haskellLib.cleanGit { src = ./.; } to the store in order to make a path for src. To avoid this we would have to use:

src = haskellLib.cleanSourceWith { src = haskellLib.cleanGit { src = ./.; }; subDir = "subDir"; };

[ocharles]

@hamishmack I'm not able to check that out right, now but I'd suggest instantiating the derivation for bar.components.library (nix-instantiate -A bar.components.library) and looking at what the input source is.

This isn't about things being rebuild, it's about a huge amount of stuff being copied into the store. It's naturally not noticable with a small test case, which is why I suggest making foo big (e.g., use dd to make a multiple GB file).

In all of my tests with master at the moment, some components have their src point to the entire directory tree, but then have a prePatch phase that just copies a subset during the build. You wouldn't see this in nix-shell, because in a shell no components are actually built.

[ocharles]

Ok, I had a chance to look at indeed hsPkgs.bar.components.exes.bar doesn't include the foo file from the source root. So something else is up. I'll try and get a minimal repro

[phile314]

I think I have also encountered this error just now. We are using callStackToNix/mkStackPkgSet. On Hydra, the builds are working with current master of haskell.nix. However, when using a local checkout it breaks because we have some sockets in the same directory which trip up nix:

error: file 'REPO_ROOT/temp_files/mydata/mysock.sock' has an unsupported type

This socket file is being filtered out using the nixpkgs cleanSourceWith function, so I have no idea why it should be copied. If I put the src in quotes, it works again.

Broken:

  stack-nix-build = with tiko.pkgs-2003.haskell-nix; callStackToNix {
    src = cleanSourceWith { src = ./.; ...};
  };

Works:

  stack-nix-build = with tiko.pkgs-2003.haskell-nix; callStackToNix {
    src = "${ cleanSourceWith { src = ./.; ...} }";
  };

[hamishmack]

I did not realise you were using callStackToNix and mkStackPkgSet directly. It might be easier to use stackProject' or stackProject. If you want to stick to callStackToNix and mkStackPkgSet, make sure you are calling importAndFilterProject as well, like stackProject' does.

[quetz]

I can't seem reproduce it that way. See https://github.com/hamishmack/test-1013.

Try this PR - hamishmack/test-1013#1

[hamishmack]

Doh. Still seems to work correctly.

[hamishmack]

I wonder if it might only affect stack projects?

[hamishmack]

Switching to a stack based project does it! hamishmack/test-1013@f003490

We must be doing something wrong in handling of stack projects.

[hamishmack]

I think the problem is that we never got around to adding code to clean the source to callStackToNix.

In callCabalProjectToNix we:

• identify the sub directory relative to the original source
• filter just the files that are needed (.cabal and package.yaml)
• make a temp dir with just those files
• cd to the sub dir if necessary

I think we can do something similar in callStackToNix, but we might need to parse the stack.yaml file to see if it is a path to a file (since we will need to include that file too).

[quetz]

Doh. Still seems to work correctly.

How do you test it?
If I take your code and then invoke nix-shell several times while changing source code in between I get these store paths created:

/nix/store/0612xilw8k2a3ardnd1gczkahs4rg25g-bar-exe-bar-0.1.0.0.drv
/nix/store/1f4h280516wb0lqzb1glrdzsqfl0pd33-test-1013-root-lib
/nix/store/1wa6k73zk6wj63p32m9xy7qaffiq3kl1-test-1013-root-bar-exe-bar-root
/nix/store/6j3vmyy9aymzrd7idl5ja1760lvvsfa2-test-1013-root-bar
/nix/store/7i3ynrdmx6ni44n7c33jy9a48slxzzhp-test-1013-root-bar
/nix/store/ddlbk32vhq3mjs391iwr4mx462li7s0n-bar-exe-bar-0.1.0.0.drv
/nix/store/dqjih4jz2zy4nplcliyxgml607hz2ky9-git-ls-files.drv
/nix/store/g5xkc116b3xki8c9si73xl0p2x9wnkaq-test-1013-root-bar-exe-bar-root
/nix/store/hfcxlb9g2karm36bz3i6cl0icy5qfq5s-git-ls-files
/nix/store/id0ihnlly9mar1hp2cf8pla92dmahbcq-test-1013-root-lib
/nix/store/j439sgd2s3wb9y9ynfy3pw6xqy5i7c0d-test-1013-root-bar
/nix/store/jqwywdn41nzs63p7sslz3758dkfp5igb-test-1013-root-lib
/nix/store/r0jqijacdswkr5nv8bfgcjf756cl05wj-bar-exe-bar-0.1.0.0.drv
/nix/store/r6918pjdgrr2xiw0jr89v95jjz0mxq5z-test-1013-root-lib
/nix/store/sn2n92jaq5gx93n18s3z7vgdlzsjf64z-test-1013-gitFiles
/nix/store/w4wn1k1l79msadgk0wwwig6ab7a2v4vs-test-1013-root-bar-exe-bar-root
/nix/store/yjnj3wg1zfzzzya7a4gazl0ihb83cpvd-ghc-shell-for-bar.drv
/nix/store/zpsc81kb9zi4wak2xj9pw4x4kb828kp5-test-1013-root-bar

I do not ever invoke nix-build or cabal build from within shell, just wait till I get dropped into shell.

[hamishmack]

How do you test it?

I am testing with:

hamish@Hamishs-MacBook-Air test-1013 % ls -l /nix/store | wc -l                                                         
   19501
hamish@Hamishs-MacBook-Air test-1013 % echo test >> foo                                                                               
hamish@Hamishs-MacBook-Air test-1013 % NIV_OVERRIDE_haskell_nix=/Users/hamish/iohk/haskell.nix nix-shell --run 'echo ok'
trace: Warning: `stdenv.lib` is deprecated and will be removed in the next release. Please use `pkgs.lib` instead. For more information see https://github.com/NixOS/nixpkgs/issues/108938
trace: No index state specified for test-1013, using the latest index state that we know about (2021-02-03T00:00:00Z)!
trace: Warning: `stdenv.lib` is deprecated and will be removed in the next release. Please use `pkgs.lib` instead. For more information see https://github.com/NixOS/nixpkgs/issues/108938
trace: To make project.plan-nix for hoogle a fixed-output derivation but not materialized, set `plan-sha256` to the output of the 'calculateMaterializedSha' script in 'passthru'.
trace: To materialize project.plan-nix for hoogle entirely, pass a writable path as the `materialized` argument and run the 'updateMaterialized' script in 'passthru'.
ok
hamish@Hamishs-MacBook-Air test-1013 % ls -l /nix/store | wc -l                                                         
   19501

[ocharles]

It's definitely not Stack only as this effects us, but we don't use stack! I have a to-do to try and investigate more

[hamishmack]

Ah think I understand now. Its not foo that is the problem. The question is why does this copy to the store:

hamish@Hamishs-MacBook-Air test-1013 % ls -l /nix/store | wc -l                                                         
   19501
hamish@Hamishs-MacBook-Air test-1013 % echo >> bar/Main.hs     
hamish@Hamishs-MacBook-Air test-1013 % NIV_OVERRIDE_haskell_nix=/Users/hamish/iohk/haskell.nix nix-shell --run 'echo ok'
trace: Warning: `stdenv.lib` is deprecated and will be removed in the next release. Please use `pkgs.lib` instead. For more information see https://github.com/NixOS/nixpkgs/issues/108938
trace: No index state specified for test-1013, using the latest index state that we know about (2021-02-03T00:00:00Z)!
trace: Warning: `stdenv.lib` is deprecated and will be removed in the next release. Please use `pkgs.lib` instead. For more information see https://github.com/NixOS/nixpkgs/issues/108938
trace: To make project.plan-nix for hoogle a fixed-output derivation but not materialized, set `plan-sha256` to the output of the 'calculateMaterializedSha' script in 'passthru'.
trace: To materialize project.plan-nix for hoogle entirely, pass a writable path as the `materialized` argument and run the 'updateMaterialized' script in 'passthru'.
ok
hamish@Hamishs-MacBook-Air test-1013 % ls -l /nix/store | wc -l                                                         
   19503

[hamishmack]

That is odd.

[hamishmack]

Hmm maybe not, probably just .drv for bar exe and its setup drv.

[quetz]

That's not only .drv, but source code is copied into store upon entering nix-shell (see, my previous message with example of nix store paths that are actually added).

[hamishmack]

It's definitely not Stack only as this effects us, but we don't use stack! I have a to-do to try and investigate more

@ocharles do you have a package in the root of your project? It looks like that was broken. The test case had both packages in subdirs. I came across filtering issues while updating Leksah to the latest haskell.nix. There were two bugs, one resulted in all the files being copied and the other one (that was masked by the first) caused filtering to when it ran to leave files out. #1041 fixes it for Leksah.

[ocharles]

Do you mean a .cabal file in the root directory? I don't think so, but I will check out #1041

[hamishmack]

A .cabal file in the root of the project (same dir as cabal.project). With packages: . in the cabal.project.

[ocharles]

I'm pretty sure we don't have that but I'll double check tomorrow

[purefn]

Has there been any attempt to fix this issue? We've just tried upgrading haskell.nix on our large, monorepo stack project in order to use a newer LTS and GHC 8.10.4, but copying our entire project source tree for every component means it takes > 30 minutes before anything even gets built.

[purefn]

Update: nevermind. I just figured out that it is the includeSiblings = true here

haskell.nix/lib/import-and-filter-project.nix

Line 55 in 22ad57c

includeSiblings = true; # Filtering sibling dirs of the package dir is done in the

that is causing ths problem for us. Is there a way to override that? For a large monorepo, this is not a good default.

[hamishmack]

Please try out #1074 as it might help.

The includeSiblings = true in import-and-filter-project.nix should only be a problem if per component level filtering is not working (or if you had something like extra-source-files: ../../**.txt in your .cabal file).

Are you using hpack? That currently breaks the component level filtering. We should probably filter just the package dir when that happens, but I think we might include everything (just in case hpack wants to reference files outside the package dir).

[purefn]

The issue I was having was after callStackToNix. Because of the includeSiblings = true in importAndFilter function, every component that was built was causing the entire source tree to be copied into the nix store, once for every component. We have hundreds (maybe 1000+) components in our project, so that meant we were getting that many copies of our entire source tree in the nix store. I created a fork to test disabling that and it is now much much faster and the build takes up a fraction of the disk space.

[hamishmack]

That indicates that the component filter is failing completely (the files should be filtered when the component builds to just the ones referenced in the .cabal file). I would really like to get to the bottom of why that is not working. It could be a problem with the output of callStackToNix (please do try #1074) or it could be something else. If you are using hpack it could be to do with that.

Unfortunately just changing to includeSiblings = false would break the ability to use relative directories in .cabal files to reference files in other packages.

Perhaps try the #1074 branch with this trace enabled. It should tell log for every file in the component the reason it was not filtered (if it prints nothing then the filter is not even running).

[hamishmack]

I have checked #1074 issue I was able to reproduce before (a file foo in the root of the project with no read access does not fail). The filtering also seems to exclude the file correctly (but it did before too).

I used pkgs.srcOnly to extract the source files for the component derivation like this:

git clone https://github.com/hamishmack/test-1013.git
cd test-1013
find $(nix-build -E 'let p = import ./default.nix; in p.pkgs.srcOnly p.lib.components.library' --no-out-link)

Only the files for the component are were included.

[purefn]

@hamishmack I tried it and got the same result. I think the issue is that there are two source trees copied to the nix store, one is filtered down to just the component source, while the other contains all of our source. I build a single package and looked for the source afterwards, and found this

$ ls /nix/store/80ww5zhf9xjj2f0f3v49aqmiiic8s4b8-range-root-devtools-simlint-root /nix/store/qln3qdfyx9nmkl4gilrwr5igqis0xpgh-range-root-devtools-simlint
/nix/store/80ww5zhf9xjj2f0f3v49aqmiiic8s4b8-range-root-devtools-simlint-root:
devtools

/nix/store/qln3qdfyx9nmkl4gilrwr5igqis0xpgh-range-root-devtools-simlint:
administration-client  defender-logs		  jwt-config-simspace	   portal-services-client    simspace-generics
aeson-simspace	      deployment-target-agent	  jwt-middleware-simspace  project		     simspace-h5p-drupal
apitest		      devtools			  jwt-test-simspace	   PULL_REQUEST_TEMPLATE.md  simspace-ids
attack-designer        doc			  keycloak		   range-query		     simspace-immortal
attack-integrations    docker			  lariat-core		   range-server	     simspace-pagination
authz		      docker-compose.yml	  learning		   range-server-api	     simspace-servant-docs
bors.toml	      documentation-generation    lib			   range-server-ts-gen       simspace-snapshot.yaml
brittany	      document-server		  logger-handle	   README.md		     simspace-squeal
BSD3-LICENSE	      event-controller		  Makefile		   reps		     simspace-test
build.sbt	      event-historian		  mattermost		   request-logger	     simspace-tracing
caching		      event-template		  MIT-LICENSE		   request-logger-simspace   simtosca
campaign-status        except-simspace		  monad-logger-extra	   resource-manager	     simyaml
client		      hardhat-expansion	  mto			   s3-simspace		     simyamltest
common-package.yaml    hardhat-server		  nagios		   SAMLSetup.md	     stack.yaml
common-testlib	      hardhat-traffic-config	  network-impact	   sample-data		     team-assessment
common-utilities       hardhat-ts-gen		  nix			   scoring		     time-simspace
content-repo	      hardhat-versioning-library  notification		   servant-convert	     tools
control-point	      hspec-extra		  notification-queue	   servant-range-proxy       tracker-client
datagen		      http-error		  package.json		   servant-simjson	     ts-gen
data-server	      human-readable		  perf-testing		   servant-zoo		     ttx
db-infra	      Jenkinsfile		  persistent-runners	   shared		     uri-utils
db-migration-test      json-logger		  portal-client	   simjson		     user-emulation
db-schema	      json-query-string	  portal-runner	   simspace-amazonka	     warden
default.nix	      json-websocket-client	  portal-server	   simspace-db-seed-user     yarn.lock

When I use my fork with includeSiblings = false, both range-root-devtools-simlint-root and range-root-devtools-simlint contain only the package/component source.

[hamishmack]

I have merged #1074 into master as it at least stops callStackToNix from depending on more than it should. Try chmod -r on a directory that it should not need. Then run nix-build with --show-trace. That might give us a clue as to what is trying to copy the files in that directory to the store (or if we are unlucky it will just skip them).

[hamishmack]

Actually chmod -r on a directory won't work (callStackToNix tries to include all directories). chmod -r on a file instead might work.

[hamishmack]

I think something must be accessing the package src without filtering it for a component first. I can't see it in haskell.nix, but it is hard to check this sort of thing without a type system.

[purefn]

I think something must be accessing the package src without filtering it for a component first. I can't see it in haskell.nix, but it is hard to check this sort of thing without a type system.

I haven't been able to track it down either. But having that includeSiblings = true makes haskell.nix unusable for us. So I guess we'll be sticking with our fork that sets it to false for now.

[purefn]

I changed includeSiblings in importAndFilterProject to throw an exception. Here's the stack trace

error: while evaluating the attribute 'prePatch' of the derivation 'simlint-lib-simlint-0.2.0.0' at /nix/store/8iyvwzjld11rspy3bjq7d3ra50z55agz-nixpkgs-2003-src/pkgs/stdenv/generic/make-derivation.nix:191:11:
while evaluating the attribute 'prePatch' at /home/rwallace/haskell.nix/builder/comp-builder.nix:251:7:
while evaluating 'optionalString' at /nix/store/8iyvwzjld11rspy3bjq7d3ra50z55agz-nixpkgs-2003-src/lib/strings.nix:180:5, called from /home/rwallace/haskell.nix/builder/comp-builder.nix:259:10:
while evaluating the attribute 'subDir' at /home/rwallace/haskell.nix/lib/default.nix:321:5:
while evaluating anonymous function at /nix/store/8iyvwzjld11rspy3bjq7d3ra50z55agz-nixpkgs-2003-src/lib/modules.nix:84:45, called from undefined position:
while evaluating the attribute 'value' at /nix/store/8iyvwzjld11rspy3bjq7d3ra50z55agz-nixpkgs-2003-src/lib/modules.nix:379:9:
while evaluating the option `packages.simlint.src':
while evaluating the attribute 'mergedValue' at /nix/store/8iyvwzjld11rspy3bjq7d3ra50z55agz-nixpkgs-2003-src/lib/modules.nix:411:5:
while evaluating anonymous function at /nix/store/8iyvwzjld11rspy3bjq7d3ra50z55agz-nixpkgs-2003-src/lib/modules.nix:413:22, called from /nix/store/8iyvwzjld11rspy3bjq7d3ra50z55agz-nixpkgs-2003-src/lib/modules.nix:413:9:
while evaluating 'merge' at /nix/store/8iyvwzjld11rspy3bjq7d3ra50z55agz-nixpkgs-2003-src/lib/types.nix:567:20, called from /nix/store/8iyvwzjld11rspy3bjq7d3ra50z55agz-nixpkgs-2003-src/lib/modules.nix:416:12:
while evaluating 'mergeEqualOption' at /nix/store/8iyvwzjld11rspy3bjq7d3ra50z55agz-nixpkgs-2003-src/lib/options.nix:108:27, called from /nix/store/8iyvwzjld11rspy3bjq7d3ra50z55agz-nixpkgs-2003-src/lib/types.nix:572:21:
while evaluating anonymous function at /nix/store/8iyvwzjld11rspy3bjq7d3ra50z55agz-nixpkgs-2003-src/lib/options.nix:110:23, called from /nix/store/8iyvwzjld11rspy3bjq7d3ra50z55agz-nixpkgs-2003-src/lib/options.nix:110:10:
while evaluating 'filter'' at /home/rwallace/haskell.nix/lib/clean-source-with.nix:67:23, called from undefined position:

So the entire source tree is being copied for each component because of the prePatch, which evaluates cleanSrc'.subDir, which evaluates packages.simlint.src. I figured as much as that, so this didn't prove too helpful. So I started adding traces all over the source filtering code. Eventually I added a trace to cleanSourceWith

      outPath = lib.traceValFn (x: "${toString name} ${name'} outPath ${x}") ((builtins.path { filter = filter'; path = origSrc; name = name'; }) + origSubDir);

Now I'm able to see that each of the nix store paths that have a full copy of our repo each have a name = null, and that is the only difference between them. So I looked into why that is null, and it turns out that's because appendSubDir doesn't pass a name to cleanSourceWith. I tried adding one, just using inherit (src) name; for a quick test.

Now I can see that all the nix store paths produced by appendSubDir in importAndFilterProject are pointing to the same place

trace: range range outPath /nix/store/n38d83dnga3gk9s6sblzaf57ijpk3yf0-range
trace:  range outPath /nix/store/34ivvl2fny4rs6c07lky828la2ky3rip-range
trace: range-root range-root outPath /nix/store/vcq6hfby2ynyfzhcpvm3j380ds30ybpa-range-root/except-simspace
trace: range-root-root range-root-root outPath /nix/store/6kqrv6xl4jb8cnid3s18z3chjdw3qggx-range-root-root
trace: range-root range-root outPath /nix/store/vcq6hfby2ynyfzhcpvm3j380ds30ybpa-range-root/lib/simspace-prelude
trace: range-root-root range-root-root outPath /nix/store/y4cj5c8agp0wijvx9inj14fkzinhlayy-range-root-root
trace: range-root range-root outPath /nix/store/vcq6hfby2ynyfzhcpvm3j380ds30ybpa-range-root/simjson
trace: range-root-root range-root-root outPath /nix/store/j8z901jmzf1x93kgj966i3dc139r3097-range-root-root
trace: range-root range-root outPath /nix/store/vcq6hfby2ynyfzhcpvm3j380ds30ybpa-range-root/time-simspace
trace: range-root-root range-root-root outPath /nix/store/v65731b92yca3fjzbgkxr5bfikbbwy07-range-root-root

so they are not creating n duplicates of our source code in the nix store. Now I only have two copies, the one produced by cleanGit and then the one produced by importAndFilterProject. All the other nix store paths, the range-root-root variety (I suspect we can fix them up to include the sub dir path in the name but haven't done that yet), only contain the source for the component being built. 🎉

Unfortunately, this is still painfully slow and unusable. 😭 I don't have any insights as to why that might be.

[domenkozar]

Using https://github.com/hercules-ci/gitignore.nix things work again using the latest master. Thanks!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.