-
-
Notifications
You must be signed in to change notification settings - Fork 121
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dream2nix fully based on nixos modules? #156
Comments
I would have to stew a bit on it and maybe a PoC would be useful to iron out the interface before committing to it, but certainly one thing you sketched here that I was thinking about was project configuration also being incorporated into the module system, not only the subsystem themselves. It seems like a good idea, and will probably make it easier to refer to projects without threading stuff around. That said, I wouldn't forego namespacing of subsystems, eg. I would imagine it should be something like I see you're already doing something similar that with Again, sorry if this is disjointed. |
I've also only just noticed @yusdacra's #155 – while I think an interface similar to what you describe here looks like a nice end-state to be in, I think it might be useful to first have something like #155 done, so it can enable external extensibility sooner and give more time to figure out what exactly the module-based interface should look like. |
I don't think I like using nixos modules for everything, especially for the user interface. While I think it would be okay for builders / translators etc. and overrides, I think we should keep the other interfaces as-is and not use modules. I think it would make it harder to integrate with other stuff in general, and feels too complex instead of just having validation IMO (maybe with something like https://github.com/divnix/yants). So it seems like it doesn't provide much value as opposed to the complexity it would bring. Personally I think it would be best if this module system wasn't directly integrated into dream2nix, but rather built on top of it, as to keep dream2nix itself as simple as possible. dream2nix itself should expose primitives for making this possible, which then everyone can benefit from, and this would allow for people to use stuff other than nixos modules. About #155, I think I like the approach there, and I think it could be even simpler (which I'll work on). I'm of the opinion that the less abstractions we have, the better we can read and debug the code, and nixos modules feel like the opposite of that, while not providing much value. The interface for extending can be changed there and I'm open to any ideas for that (the first change I'll do is probably just taking a list of files instead of subsystem, name, file attrset because the module should already have these). |
Good point. Because a project currently is defined by a source tree and that source tree can contain sub-projects of different subsystems, we should make sure to categorize the project specific overrides and injects by the subsystem. Concerning attrset vs list:
Why would it make the user interface harder to integrate with other stuff? Currently our interface consists of functions which receive an attrset. A nixos module can be an attrset as well. So there wouldn't be a reason why the user interface should look any different by default. We can make the user interface however we want it. Users wouldn't have to know that the function argument is interpreted as a nixos module. Right now there are a lot of uncertainties in our user interface. We have
I feel it is going to be the opposite way. It would reduce the complexity of our code if we would just re-use the existing module system. Right now and especially after #155 A lot of our code is just about extensibility, how we load our own modules, how extra modules can be loaded, how settings are applied etc. All of that will simply vanish if we use the nixos module system. Currently we are about to re-implement our own module system and we are probably going to re-do all the mistakes that have already been solved by nixos modules. Why not just re-use what is already there and is maintained by the community. Less work for us. But I agree, that making the whole framework module based at once might be too big of a step. First we should start using nixos modules where can really benefit (overrides, settings, config, etc.) and see how this works out. I think I might start working on a POC with something simple like the fetchers and see if my hypothesis of the code becoming less complex turns out to be true. |
I agree that nixos modules would give more discoverability and structure which is why I think it makes most sense for overrides (source, package, injecting dependencies too potentially) since it's the most user facing interface. But for builders / translators etc. I think they are really simple in nature (pretty much only I'd like to keep dream2nix's core stuff ideally depending on as less nixpkgs code as possible so maybe one day we can make it portable... So I think instead of integrating nixos modules into dream2nix, we can build on top of it so that it's not a strict requirement (eg. you could skip using it and directly use the underlying interfaces). Even if it is maintained by not us, it still needs to be maintained (meaning eventually we might need to touch it), and it still means we depend on more code ( |
At Zurihac I developed a small demo for using the module system for dream2nix with the feedback of @DavHau, here's the result, mainly see the |
Thanks a lot for this. To add some information, the scope of the demo is to replace the current |
I took a better look at the POC and this could be nice, but I'm not sure if I understand it right. My understanding is that the Based on my understanding I feel like we could make it better if we:
{
overrides.packages.rust.crane.my-override = {
# attribute for deciding which packages to attribute with this
# you could also set it to `_: true` to override all packages
# `pkg` should be an attribute set containing a `name` and a `version`
# so you can select both at the same time.
enableIf = pkg: lib.any (n: pkg.name == n) ["ripgrep" "hello"];
# or for example
enableIf = pkg: pkg.name == "ripgrep" && (builtins.compareVersions pkg.version "13.0.0" >= 0);
# you can also set a simple attribute to enable it for one package only
enableForPackage = "ripgrep";
# set builder config / arguments
config = {
runTests = false;
profile = "debug";
# builders can expose attributes like `overrideAttrs` for overriding derivations directly
overrideAttrs = old: {
MY_ENV_VAR = "my value";
};
# since this is an example for the `crane` builder, it can also expose something like this
# (since it builds two separate derivations, one for deps-only and other for main package)
overrideDepsAttrs = old: {
someEnvVarForADependency = "some value";
};
};
};
}
{
overrides.sources.my-override = {
# pkg contains a `name` and `version` like `overrides.packages`
# pretty much the same as that
enableIf = pkg: pkg.name == "ripgrep" && pkg.version == "13.0.0";
# we can add helper functions like this to make it easier
enableIf = dlib.matchNameVersion "ripgrep" "13.0.0";
changeSource = old: /* patch the old source or something */;
};
}
{
overrides.dependencies.my-override = {
# you get the gist :P
enableIf = pkg: pkg.name == "ripgrep" && pkg.version == "13.0.0";
changeDependencies = old: old ++ [
["foo" "0.1.0"]
["bar" "0.2.0"]
];
};
} I think doing them like this would make the interface way more consistent than how it is currently, which I think is a good thing; instead of all the different interfaces you now have one entrypoint, and similar interfaces for the packages / sources / dependencies. We could still add helpers to make it even easier like so: {
overrides.sources.my-override = dlib.overrides.sources.mkSingle {
name = "ripgrep";
version = "13.0.0";
source = old: /* something */;
};
# or maybe
overrides.sources.my-override = dlib.overrides.sources.mkSingle "ripgrep" "13.0.0" (old: /* something */);
# or maybe even (by-name-version is reserved)
overrides.sources.by-name-version.ripgrep."13.0.0" = old: /* something */;
} that is up to discussion, and I'm not sure what would work better or not. |
Correct.
I agree. There are still some things that need to be re-structured.
Concerning the I'm not sure if overrides should be namespaced by I'm thinking of something like:
Though the drawback of this would be that we won't be able to typecheck overrides before they are actually applied, because we don't know which builders interface they apply to. Maybe an alternative to this could be to have a Concerning the options for |
I think if we have the
I think we should have it be namespaced by
Yes, I think that's okay. To make it easier to debug packages and incrementally move to modules, I think it would be okay if we took an argument like |
I think the idea is very straightforward, and it's only the details that need some thought. Summary as I understand it:
A point of some confusion for me at first was that e.g. translators are both configuration and executable code that takes configuration. Of course, for Nix it's all just lazy evaluation. I'd like to propose these changes:
Using this modular approach also allows having multiple subsystems within a single project, building different packages. It also allows adding entire new subsystems. |
And as for overrides, there's several places where it might be useful to intervene. During translation, after dream.lock generation and then during build. Right now only the latter is possible, and it would be nice to have a single configuration object for all types. |
related:
#155
#151
#153
This is a hypothetical usage example of dream2nix if the framework was fully based on the nixos module system.
This is not only useful for configuring nixos systems. Even if the intention is to just create single packages or a dev shell, dream2nix could still be invoked via a module definition and benefit from properties of the nixos module system.
Benefits I see:
dream2nix-ripgrep.nix
:For usage with simple nix expressions, outside of a nixos configuration, a utility function could be defined, taking care of calling
evalModules
correctly.This for example could be used in a flake:
(
./dream2nix-ripgrep.nix
corresponds to the file above)@yusdacra @jaen
The text was updated successfully, but these errors were encountered: