Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mkShell: add builder #30975

Merged
merged 1 commit into from Dec 20, 2017
Merged

mkShell: add builder #30975

merged 1 commit into from Dec 20, 2017

Conversation

@zimbatm
Copy link
Member

@zimbatm zimbatm commented Oct 30, 2017

Motivation for this change

For nix-shell-only scenarios, this is a small optimization to make it nicer such that:

{ pkgs ? import <nixpkgs> {} }:
pkgs.stdenv.mkDerivation {
  name = "forced-to-add-but-i-dont-care";
  src = null; # irrelevant
}

Is simplified to:

{ pkgs ? import <nixpkgs> {} }:
pkgs.mkShell { }

It also fails when trying to build it directly:

$ nix-build shell.nix
...
This derivation is only meant to be used in nix-shell, aborting
...

And finally, it supports the scenario where one wants to compose the shell with different inputs from various packages:

{ pkgs ? import <nixpkgs> {}}:
pkgs.mkShell {
  mergeInputs = [ pkgs.hello pkgs.gnutar ];
}
@zimbatm zimbatm force-pushed the mkShell branch to a6eb34a Oct 30, 2017
@zimbatm
Copy link
Member Author

@zimbatm zimbatm commented Oct 30, 2017

For now I am looking for feedback on the following things:

  • should the shellHooks be merged as well?
  • where should I put the documentation for that?
  • is there any other kind of feature that you often use in your shells?
@zimbatm
Copy link
Member Author

@zimbatm zimbatm commented Oct 30, 2017

It would be nice if the shellHooks where composable


buildInputs = mergeInputs' "buildInputs";
nativeBuildInputs = mergeInputs' "nativeBuildInputs";
propagatedBuildInputs = mergeInputs' "propagatedBuildInputs";

This comment has been minimized.

@orivej

orivej Oct 30, 2017
Contributor

You are missing propagatedNativeBuildInputs.

This comment has been minimized.

@zimbatm

zimbatm Oct 30, 2017
Author Member

Thanks, I didn't know this existed, but makes sense. Fixed in b111860 which will be squashed later

@gilligan
Copy link
Contributor

@gilligan gilligan commented Oct 30, 2017

@zimbatm Could you also add documentation to the manual for this?

Edit: I see you asked where to document.. good question ;-)

@gilligan
Copy link
Contributor

@gilligan gilligan commented Nov 8, 2017

bump - i would love to see this merged. What's the status?

@@ -301,6 +301,8 @@ with pkgs;
inherit kernel rootModules allowMissing;
};

mkShell = callPackage ../build-supports/shell-builder.nix { };

This comment has been minimized.

@orivej

orivej Nov 8, 2017
Contributor

This should be build-support, not build-supports.

This comment has been minimized.

@orivej

orivej Nov 8, 2017
Contributor

I'd suggest moving it into ../build-support/mkshell/default.nix anyway. Quiet a few derivations there are already single file.

This comment has been minimized.

@zimbatm

zimbatm Nov 25, 2017
Author Member

fixed

buildInputs ? [],
nativeBuildInputs ? [],
propagatedBuildInputs ? [],
...

This comment has been minimized.

@orivej

orivej Nov 8, 2017
Contributor

You are missing propagatedNativeBuildInputs here (or not, and the others are not needed too?)

This comment has been minimized.

@zimbatm

zimbatm Nov 25, 2017
Author Member

fixed

# A special kind of derivation that is only meant to be consumed by the
# nix-shell.
{
mergeInputs ? [], # a list of derivations whose inputs will be merged

This comment has been minimized.

@edolstra

edolstra Nov 8, 2017
Member

This does not explain what it means for an input to be "merged". (And it's not clear to me from your description.)

Also, linguistically, mergeInputs sounds like a Boolean variable, i.e. whether or not to merge inputs, not "list of things to be merged".

This comment has been minimized.

@zimbatm

zimbatm Nov 25, 2017
Author Member

How about inputsFrom? a list of derivations whose inputs will be made available to the environment

@avnik
Copy link
Contributor

@avnik avnik commented Nov 9, 2017

Nice,
can it be a base for myEnvFun replacement?

@zimbatm zimbatm force-pushed the mkShell branch to db6136f Nov 25, 2017
@zimbatm zimbatm force-pushed the mkShell branch from db6136f to a5eabe9 Dec 20, 2017
@zimbatm zimbatm merged commit adc5c9b into master Dec 20, 2017
@zimbatm zimbatm deleted the mkShell branch Dec 20, 2017
@zimbatm
Copy link
Member Author

@zimbatm zimbatm commented Dec 20, 2017

I realize that the doc needs to be improved, will do another round tomorrow.

@vaibhavsagar
Copy link
Member

@vaibhavsagar vaibhavsagar commented Jan 19, 2018

Is this coming to nixos-17.09?

@zimbatm
Copy link
Member Author

@zimbatm zimbatm commented Jan 19, 2018

This is a new feature so it will land in 18.04

@CMCDragonkai
Copy link
Member

@CMCDragonkai CMCDragonkai commented Jul 5, 2018

The inputsFrom is so intriguing.

I'm wondering if one major usecase is say you have multiple projects each with their own shell.nix, and you could create 1 master shell.nix that merges all subproject's shell.nix in case you wanted to work on all the subprojects together.

@zimbatm Could you enumerate what other usecases you envision for inputsFrom?

Also with regards to merging shell hook, this is definitely something that would be cool as well as long as the shell hooks were written in a way that didn't conflict with each other. My project's shell hook tends to be creating relevant environment variables. So they could be automerged. However I'd imagine in some cases you wouldn't want to auto merge them. Maybe some function that can explicitly compose the shell hooks based on the inputsFrom.

@zimbatm
Copy link
Member Author

@zimbatm zimbatm commented Jul 5, 2018

Yes that's exactly the use-case that I had in mind. Imagine you have a monorepo with multiple services, each in their own folder with their own default.nix that you can nix-shell and nix-build. On the top-level you can then add a shell.nix with, mkShell { inputsFrom = [ serviceA serviceB ]; to pull all the dependencies together.

shellHooks are a bit more tricky because they also tend to assume things about $PWD. For example if you have a node package it might add $PWD/node_modules/.bin to the PATH. But that only works if you start the shell within the frontend service for example.

@CMCDragonkai
Copy link
Member

@CMCDragonkai CMCDragonkai commented Jul 7, 2018

How should mkShell be used in comparison to higher order shell functions like haskell.lib.buildStackProject or pythonPackages.buildPythonApplication that already set things up relevant to stack or python build environment? Could potentially mkShell be used to eventually compose a stack and python environment together in case you're working with Python and Haskell at the same time?

@zimbatm
Copy link
Member Author

@zimbatm zimbatm commented Jul 7, 2018

I don't have a satisfying answer and would be interested in figuring this out. Aside from changing the PATH it's going to be hard to prove that two shells are going to compose nicely and that the application is commutative.

One idea is to require each environment to provide a function that takes the project root as an argument. Eg: nodePackage.shellHook could be executed to be setup for the $PWD/frontend. But then it doesn't work if there are two nodejs projects in the same repo.

For now I just use mkShell to pull project-global dependencies like deployment tooling and when the $language build tools can just be imported into the PATH and knows how to do the right thing based on $PWD alone. And if necessary I compose the things by hand by writing my own shellHook. It's more work but also creates less surprises in the event of a nixpkgs update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

7 participants
You can’t perform that action at this time.