More context in context string

[guibou]

Is your feature request related to a problem? Please describe.

I'm gathering informations about all the dependencies of a project, such as licences, maintainers, kind (i.e. haskell deps, python deps, ...) and count of the dependencies, ...

It is straightforward to walk the nix structures and gather some information. builtins.genericClosure helps. However I'm missing the dependencies which are implicitly attached to string as string context.

I can recover the context using builtins.getContext, however it does returns a path in the nix store to the derivation, not the original attribute set, so the information I care about is lost.

For example, the expression pkgs.runCommand "foo" {} ''${pkgs.blender}'' implicitly depends on pkgs.blender. However I cannot find a reference to the derivation blender other than using builtins.getContext ((pkgs.runCommand "foo" {} ''${pkgs.blender}'').buildCommand) which returns an attribute set where the drv path is the key: { "/nix/store/8vwn856hd9q6413qqnjiv3l4x7wxq31r-blender-2.91.0.drv" = { ... }; }, unfortunately, the derivation attribute set is lost, including the meta information I care about.

Unfortunately, the .drv does not contain the meta information either.

Describe the solution you'd like

The string context could include a reference to the original derivation which was interpolated as a path?

Describe alternatives you've considered

• I considered rebuilding the information based on the .drv path, by looking in all the derivation of my projects (including nixpkgs), but it is slow (i.e. I need to generate the attribute set of ALL the possible outputPath) and it may not be complete, if a derivation does not exists as an explicit nix reference somewhere (e.g. "${pkgs.blender}" will be found when scanning pkgs, but "${pkgs.f x y}" won't, because only f will be found when scanning pkgs.)
• I can just accept that it is not complete
• Adding meta informations in .drv

Additional context

The issue with this approach is that it will cost memory for the new entry in the context as well as keeping a reference to the associated attribute set. I did not estimated the memory overhead introduced.

[stale]

I marked this as stale due to inactivity. → More info

[roberth]

In NixOS/nixpkgs#140792, writeStringReferencesToFile needs to resort to regular expressions to attempt to recover proper output paths from the context. This would be much easier and more accurate if Nix provided the output paths directly.
This is a simpler change than @guibou is requesting here, but also requires a change to the return value of (the successor of) builtins.getContext.

Yet another idea is to allow the context to be extended such that a string can not be written to the store. This could make Nix much safer as a configuration language, allowing secret strings to be marked as such.

it will cost memory for the new entry in the context as well as keeping a reference to the associated attribute set

This will usually be limited to thunks that reference already existing heap objects, so I would estimate that it's only a small number of extra heap objects per instantiated package.

More good news is that it is compatible with evaluation caching. The string context could be modeled the same as an attribute set in this regard.

[roberth]

This will usually be limited to thunks that reference already existing heap objects

This highlights somewhat of a problem with Nixpkgs, but this change is not a regression in this regard. If a package repository can be structured to avoid referencing packages for too long, it can also restrict the information it adds to the context to a small amount and force evaluation of that information using seq before returning the augmented strings.

[roberth]

Related

• Track test derivations and parallelize building and testing #7662