Redesign of the nix command line #779

Profpatsch · Jan 21, 2016

Initial writeup by @edolstra: https://gist.github.com/edolstra/efcadfd240c2d8906348

I did a little writeup with my thoughts & proposals:
https://gist.github.com/Profpatsch/d5c8e1ccd68baab0e3f0

In that I address the problem of search path clashes @edolstra mentioned in the General section.

Profpatsch · Jan 21, 2016

I did a little writeup with my thoughts & proposals:
https://gist.github.com/Profpatsch/d5c8e1ccd68baab0e3f0

In that I address the problem of search path clashes @edolstra mentioned in the General section.

(Commenting here as @copumpkin requested)

I strongly agree with @Profpatsch's points. To be fair there, there is a use for imperative package management, as articulated by @lethalman in #684. Since under @Profpatsch's plan the profile is always defined declarative, and since @lethalman's usecases boil down to pinning the exact version of a package, I think we should speak of "pinned" vs "regular" packages.

Start a development shell based on ./default.nix or ./shell.nix:
$ nix shell
(Hm, do we want a special case like this?)

I'm reminded of git changing the default push settings. I suggest we disable this by default, and then have a setting to enable this power users can enable. In the name of orthogonality, I also propose that the setting be that the namespace is queried unless default.nix/shell.nix is present, and have it apply to at least nix build and nix use in addition to nix shell. [I don't think nix install should have this, because that command is more global, and it violates our new attribute path agenda].

Ericson2314 · Jan 22, 2016

(Commenting here as @copumpkin requested)

I strongly agree with @Profpatsch's points. To be fair there, there is a use for imperative package management, as articulated by @lethalman in #684. Since under @Profpatsch's plan the profile is always defined declarative, and since @lethalman's usecases boil down to pinning the exact version of a package, I think we should speak of "pinned" vs "regular" packages.

Start a development shell based on ./default.nix or ./shell.nix:
$ nix shell
(Hm, do we want a special case like this?)

I'm reminded of git changing the default push settings. I suggest we disable this by default, and then have a setting to enable this power users can enable. In the name of orthogonality, I also propose that the setting be that the namespace is queried unless default.nix/shell.nix is present, and have it apply to at least nix build and nix use in addition to nix shell. [I don't think nix install should have this, because that command is more global, and it violates our new attribute path agenda].

More broadly, I see a convergence in the way userland nix and nixos are configured and used. We've already talked about using the nixos module system both for user-mode services (e.g. on OSX), and for configuring packages. Likewise there was recently a thread to the mailing list about decoratively pinning nixpkgs for nixos in https://www.mail-archive.com/nix-dev@lists.science.uu.nl/msg17827.html .

Ultimately It would be great if root and normal users alike can define and enable service and packages in the same module system, and the same CLI tools can assist with querying and editing the config files.

Converging all this will take a great deal of work, but I think a few decisions can be made now to help the process go faster. In the thread I linked, @edolstra mentioned we could use fetchTarball, or move the module system in nix itself (presumably into the corepkgs). We have a similar choice for the UI itself---to use the git terminology, we could have just the minimal plumbing commands in nix, and porcelain commands like nix install kept elsewhere for more rapid iteration. [This also provides a way to make nix repl work without bundling the repl with nix].

Ericson2314 · Jan 22, 2016

More broadly, I see a convergence in the way userland nix and nixos are configured and used. We've already talked about using the nixos module system both for user-mode services (e.g. on OSX), and for configuring packages. Likewise there was recently a thread to the mailing list about decoratively pinning nixpkgs for nixos in https://www.mail-archive.com/nix-dev@lists.science.uu.nl/msg17827.html .

Ultimately It would be great if root and normal users alike can define and enable service and packages in the same module system, and the same CLI tools can assist with querying and editing the config files.

Converging all this will take a great deal of work, but I think a few decisions can be made now to help the process go faster. In the thread I linked, @edolstra mentioned we could use fetchTarball, or move the module system in nix itself (presumably into the corepkgs). We have a similar choice for the UI itself---to use the git terminology, we could have just the minimal plumbing commands in nix, and porcelain commands like nix install kept elsewhere for more rapid iteration. [This also provides a way to make nix repl work without bundling the repl with nix].

Is it possible to cut down the proposal to the essentials ? It's good to show the end-goal but for implementation it might be a bit daunting. How would we go on implementing all of this ? Are we going to port one command at the time and update the docs accordingly ?

One thing that might feel inherent but that needs to be specified is that thenix executable wouldn't be doing much, it would behaves like git and just dispatch sub-commands to nix-<command> executables in it's libexec/ and PATH.

zimbatm · Jan 23, 2016

Is it possible to cut down the proposal to the essentials ? It's good to show the end-goal but for implementation it might be a bit daunting. How would we go on implementing all of this ? Are we going to port one command at the time and update the docs accordingly ?

One thing that might feel inherent but that needs to be specified is that thenix executable wouldn't be doing much, it would behaves like git and just dispatch sub-commands to nix-<command> executables in it's libexec/ and PATH.

First my strongest points:

Very nice ideas overall.
There are too many similar top-level commands, as noted already. I believe it's better to group them more, e.g. create more hierarchy or introduce options switching some behavior changes.
The commands should have short-cuts, even though the long form should be preferred in documentation. Some people like to have short aliases for frequent commands. One way is adding aliases like nix -i bash, nix -q foo, another way is allowing to use a unique prefix of a command like btrfs fi def for btrfs filesystem defragment. Enhanced alternative: allow defining aliases, e.g. like git.
Commands that take attribute names should accept arbitrary nix expressions instead. This could be extended even to package updates, e.g. if I nix install 'foo.override { myOption = bar; }', I want to keep the override even after nix upgrade foo.
Installation actions should have an interactive mode, perhaps even by default if connected to terminal: first show essentially --dry-run and then ask for confirmation whether to proceed.
Implementation: I assume that to simplify #341 we'll only do the new C++ implementation for the new command scheme, and the old commands would require perl and we would remove them anyway after several months of overlap.

Now some replies:

@itkovian: nix build is what adds stuff into the store (and a symlink into the current directory). To most people "installation" is expect to do much more than that, e.g. make $PATH contain the executables, etc.
@madjar: I've got a set of packages that I use daily, and I want those consistent and often updated. Then I have some I rarely use and sometimes huge, so updating them often would be rather wasteful, e.g. LibreOffice.
@CMCDragonkai: that sounds like ctags and similar tools. Perhaps such functionality belongs into the REPL, but note that the "location" is often difficult to pinpoint, e.g. the binding is done in all-packages.nix but most of the code is in another file.
@Profpatsch: the nixpkgs.pkgs. prefix seems redundant. Anyone can introduce extra namespaces via packageOverrides already. (I use it to define personalized easy to update custom package sets.) There's only a single nixpkgs=... in $NIX_PATH and that should be the default namespace, I believe.

More detailed thoughts of mine, ordered according to the original proposal:

nix search --name foo is rather confusing terminology, as we'll likely want the attribute name to be the (unqualified) name
nix install
- --declarative: rather --auto-update or similar. Calling it "declarative" just seems strange to me (from a fresh point of view).
- -f: perhaps replace by -I to handle all $NIX_PATH changes more consistently?
- machine:package : also support the other direction?
nix rebuild: wouldn't that better be somewhere in nix upgrade? For example, nix upgrade by itself working only on auto-upgraded packages and with -a or --all doing the "usual" upgrade?
nix upgrade: maybe I like update better. From some mainstream distro I've fixed upgrade as something big and dangerous... but we could e.g. make them synonymous.
nix list and nix status: these two seem to work too similar to warrant extra (top-level) subcommands. Perhaps call it nix query or nix info instead. (Also noted by @trishume.)
nix uninstall: maybe unlink? (analogy to UNIX* traditions around file deletion)
nix rollback: could be also undo, but rollback seems a more popular term in general. Improvement: generations might remember which one came before, as currently nix-env --rollback; nix-env -i foo; nix-env --rollback does an unintuitive thing.
nix use, nix sandbox, nix shell: very similar, again. Maybe nix use --sandbox or --pure or something...
nix query-options: perhaps this doesn't warrant another subcommand. I'd join this with the above, e.g. nix query --options bash and nix query --plugins firefox.
- Note: perhaps among this the various variants should be handled. I'd think that instead of firefox-esr attribute we should have an option, e.g. variant="esr". Similarly for the barFull packages, etc.
- Note: for options, especially global ones, there's a question whether to propagate them to dependencies. We might consider bringing nixos and nixpkgs closer to each other.
nix build: again the -f considerations.
nix repl and nix eval: similar, maybe nix eval -i or --interactive, though "repl" might be too well remembered already.
nix make-store-derivation: why not nix-build --drv instead? It should do the same thing, only stop after instantiation.
nix fetch-url: maybe have nix fetch instead, accepting multiple schemes: https?, ftp, git, ...
- add --unpack as fetchzip equivalent.
nix fetch-closure, nix send-closure: maybe not needed when we allow nix build --from machine /some/store/path and --to similarly.
nix query-closure, nix closure-size: why not nix query --closure...?
- TBD answer: it should be able find out closure from binary caches, even without fetching the paths.
nix path-info, nix substitutes: why not under nix query?
nix source: perhaps add --rollback to address some concerns.

vcunat · Jan 24, 2016

First my strongest points:

Very nice ideas overall.
There are too many similar top-level commands, as noted already. I believe it's better to group them more, e.g. create more hierarchy or introduce options switching some behavior changes.
The commands should have short-cuts, even though the long form should be preferred in documentation. Some people like to have short aliases for frequent commands. One way is adding aliases like nix -i bash, nix -q foo, another way is allowing to use a unique prefix of a command like btrfs fi def for btrfs filesystem defragment. Enhanced alternative: allow defining aliases, e.g. like git.
Commands that take attribute names should accept arbitrary nix expressions instead. This could be extended even to package updates, e.g. if I nix install 'foo.override { myOption = bar; }', I want to keep the override even after nix upgrade foo.
Installation actions should have an interactive mode, perhaps even by default if connected to terminal: first show essentially --dry-run and then ask for confirmation whether to proceed.
Implementation: I assume that to simplify #341 we'll only do the new C++ implementation for the new command scheme, and the old commands would require perl and we would remove them anyway after several months of overlap.

Now some replies:

@itkovian: nix build is what adds stuff into the store (and a symlink into the current directory). To most people "installation" is expect to do much more than that, e.g. make $PATH contain the executables, etc.
@madjar: I've got a set of packages that I use daily, and I want those consistent and often updated. Then I have some I rarely use and sometimes huge, so updating them often would be rather wasteful, e.g. LibreOffice.
@CMCDragonkai: that sounds like ctags and similar tools. Perhaps such functionality belongs into the REPL, but note that the "location" is often difficult to pinpoint, e.g. the binding is done in all-packages.nix but most of the code is in another file.
@Profpatsch: the nixpkgs.pkgs. prefix seems redundant. Anyone can introduce extra namespaces via packageOverrides already. (I use it to define personalized easy to update custom package sets.) There's only a single nixpkgs=... in $NIX_PATH and that should be the default namespace, I believe.

More detailed thoughts of mine, ordered according to the original proposal:

nix search --name foo is rather confusing terminology, as we'll likely want the attribute name to be the (unqualified) name
nix install
- --declarative: rather --auto-update or similar. Calling it "declarative" just seems strange to me (from a fresh point of view).
- -f: perhaps replace by -I to handle all $NIX_PATH changes more consistently?
- machine:package : also support the other direction?
nix rebuild: wouldn't that better be somewhere in nix upgrade? For example, nix upgrade by itself working only on auto-upgraded packages and with -a or --all doing the "usual" upgrade?
nix upgrade: maybe I like update better. From some mainstream distro I've fixed upgrade as something big and dangerous... but we could e.g. make them synonymous.
nix list and nix status: these two seem to work too similar to warrant extra (top-level) subcommands. Perhaps call it nix query or nix info instead. (Also noted by @trishume.)
nix uninstall: maybe unlink? (analogy to UNIX* traditions around file deletion)
nix rollback: could be also undo, but rollback seems a more popular term in general. Improvement: generations might remember which one came before, as currently nix-env --rollback; nix-env -i foo; nix-env --rollback does an unintuitive thing.
nix use, nix sandbox, nix shell: very similar, again. Maybe nix use --sandbox or --pure or something...
nix query-options: perhaps this doesn't warrant another subcommand. I'd join this with the above, e.g. nix query --options bash and nix query --plugins firefox.
- Note: perhaps among this the various variants should be handled. I'd think that instead of firefox-esr attribute we should have an option, e.g. variant="esr". Similarly for the barFull packages, etc.
- Note: for options, especially global ones, there's a question whether to propagate them to dependencies. We might consider bringing nixos and nixpkgs closer to each other.
nix build: again the -f considerations.
nix repl and nix eval: similar, maybe nix eval -i or --interactive, though "repl" might be too well remembered already.
nix make-store-derivation: why not nix-build --drv instead? It should do the same thing, only stop after instantiation.
nix fetch-url: maybe have nix fetch instead, accepting multiple schemes: https?, ftp, git, ...
- add --unpack as fetchzip equivalent.
nix fetch-closure, nix send-closure: maybe not needed when we allow nix build --from machine /some/store/path and --to similarly.
nix query-closure, nix closure-size: why not nix query --closure...?
- TBD answer: it should be able find out closure from binary caches, even without fetching the paths.
nix path-info, nix substitutes: why not under nix query?
nix source: perhaps add --rollback to address some concerns.

The commands should have short-cuts

+1 for unique substrings of command names, -1 for alias definiton (that’s what the shell is for, and the git folks got it all wrong!)

Commands that take attribute names should accept arbitrary nix expressions instead

What should be in scope by default?

Installation actions should have an interactive mode

I find that absolutely crucial with a “destructive” package manager like e.g. pacman, but what use is it with nix?

Anyone can introduce extra namespaces via packageOverrides already

Yet, that is not useful if one uses two versions of e.g. nixpkgs, e.g. 15.09 and master (which is a pretty standard usecase).

Profpatsch · Jan 25, 2016

The commands should have short-cuts

+1 for unique substrings of command names, -1 for alias definiton (that’s what the shell is for, and the git folks got it all wrong!)

Commands that take attribute names should accept arbitrary nix expressions instead

What should be in scope by default?

Installation actions should have an interactive mode

I find that absolutely crucial with a “destructive” package manager like e.g. pacman, but what use is it with nix?

Anyone can introduce extra namespaces via packageOverrides already

Yet, that is not useful if one uses two versions of e.g. nixpkgs, e.g. 15.09 and master (which is a pretty standard usecase).

@vcunat I notice that wherever you suggested a third level in the hierarchy, you did it as an --option; but couldn't it be turtles all the way down, so that sub-subcommands look and work the same as subcommands, e.g. nix query closure instead of nix query --closure?

glaebhoerl · Jan 25, 2016

@vcunat I notice that wherever you suggested a third level in the hierarchy, you did it as an --option; but couldn't it be turtles all the way down, so that sub-subcommands look and work the same as subcommands, e.g. nix query closure instead of nix query --closure?

-1 for alias definiton (that’s what the shell is for, and the git folks got it all wrong!)

@Profpatsch: how do you simply define aliases for sub-commands in bash?

What should be in scope by default?

I imagine it like this: the default $NIX_PATH points to some channel so I can use nix install bash directly, and if I want to use a different nixpkgs tree (channel), I'll have other ways to do that, among them e.g. modifying $NIX_PATH by -I options.

I find that absolutely crucial with a “destructive” package manager like e.g. pacman, but what use is it with nix?

Personally, I sometimes like to see what would happen beforehand, instead of doing it and the rolling back immediately.

Anyone can introduce extra namespaces via packageOverrides already

Yet, that is not useful if one uses two versions of e.g. nixpkgs, e.g. 15.09 and master (which is a pretty standard usecase).

Even today you should be able to do this:

{
    packageOverrides = pkgs: {
        pkgs-1509 = import /nix/var/nix/profiles/per-user/root/channels/nixos-15.09 {};
    };
}

Other ways are e.g. adding nixpkgs-15.09=... into $NIX_PATH instead and importing that. Of course, we could automatize something like this when adding channels, in some way...

(And even today you can relatively simply get rid of the prefixes; the only question is what should be the default. There was a similar discussion on the mailing-list a couple days ago.)

@glaebhoerl: for clarity reasons, I assume it's better to prefix by -- in cases where it's optional to specify a sub-command. For example, if we allowed nix query git bash to display information about multiple packages, it would seems strange to allow nix query closure bash and instead I feel it would be better to have nix query --closure bash.

vcunat · Jan 26, 2016

-1 for alias definiton (that’s what the shell is for, and the git folks got it all wrong!)

@Profpatsch: how do you simply define aliases for sub-commands in bash?

What should be in scope by default?

I imagine it like this: the default $NIX_PATH points to some channel so I can use nix install bash directly, and if I want to use a different nixpkgs tree (channel), I'll have other ways to do that, among them e.g. modifying $NIX_PATH by -I options.

I find that absolutely crucial with a “destructive” package manager like e.g. pacman, but what use is it with nix?

Personally, I sometimes like to see what would happen beforehand, instead of doing it and the rolling back immediately.

Anyone can introduce extra namespaces via packageOverrides already

Yet, that is not useful if one uses two versions of e.g. nixpkgs, e.g. 15.09 and master (which is a pretty standard usecase).

Even today you should be able to do this:

{
    packageOverrides = pkgs: {
        pkgs-1509 = import /nix/var/nix/profiles/per-user/root/channels/nixos-15.09 {};
    };
}

Other ways are e.g. adding nixpkgs-15.09=... into $NIX_PATH instead and importing that. Of course, we could automatize something like this when adding channels, in some way...

(And even today you can relatively simply get rid of the prefixes; the only question is what should be the default. There was a similar discussion on the mailing-list a couple days ago.)

@glaebhoerl: for clarity reasons, I assume it's better to prefix by -- in cases where it's optional to specify a sub-command. For example, if we allowed nix query git bash to display information about multiple packages, it would seems strange to allow nix query closure bash and instead I feel it would be better to have nix query --closure bash.

@vcunat Yea I think the functionality should be in REPL or in the manual. See for example the Haskell documentation, and how if you look up the source for any particular function, it points to the actual implementation. For the REPL if it could show the type signature (even though nix is untyped, it could just be manually specified, like a help page for that exact command), that would be great.

CMCDragonkai · Jan 27, 2016

@vcunat Yea I think the functionality should be in REPL or in the manual. See for example the Haskell documentation, and how if you look up the source for any particular function, it points to the actual implementation. For the REPL if it could show the type signature (even though nix is untyped, it could just be manually specified, like a help page for that exact command), that would be great.

Once some of the initial discussion settles down, I would suggest creating a new repo to host a design document. PRs would then be issued to get feedback on proposals before updating the doc.

I'm really enjoying the level of discourse here, but I think it's important that we have an authoritative document so we're all on the same page, and have a more directed approach to converging on a final, actionable design.

Ideally, we'd have @edolstra and a couple others that we could trust to review and give feedback on each proposal, merging in the changes if they sound good.

A deadline for the design might also be a good idea, to avoid this work languishing.

Thoughts?

cstrahan · Jan 28, 2016

Once some of the initial discussion settles down, I would suggest creating a new repo to host a design document. PRs would then be issued to get feedback on proposals before updating the doc.

I'm really enjoying the level of discourse here, but I think it's important that we have an authoritative document so we're all on the same page, and have a more directed approach to converging on a final, actionable design.

Ideally, we'd have @edolstra and a couple others that we could trust to review and give feedback on each proposal, merging in the changes if they sound good.

A deadline for the design might also be a good idea, to avoid this work languishing.

Thoughts?

Alternatively, we could just have @edolstra comb through all the commentary, and in true BDFL fashion, develop the final design to his liking :). Whatever works best for us / @edolstra.

cstrahan · Jan 28, 2016

Alternatively, we could just have @edolstra comb through all the commentary, and in true BDFL fashion, develop the final design to his liking :). Whatever works best for us / @edolstra.

Yes, a very good point. Best agree on basics of the UI before implementation (perhaps except for some prototypes).

vcunat · Jan 28, 2016

Yes, a very good point. Best agree on basics of the UI before implementation (perhaps except for some prototypes).

how do you simply define aliases for sub-commands in bash?

alias myalias='nix subcommand'

It’s nice to see the difference between built-in and not. Otherwise you sit at a different system and wonder why the most basic commands are missing …

Even today you should be able to do this:

packageOverrides is dependent on nixpkgs, though. I don’t like the idea that most functionality of nix only works in conjunction with the default package set. In the same manner I don’t think we should hardcode the string nixpkgs anywhere in nix.

Profpatsch · Jan 29, 2016

how do you simply define aliases for sub-commands in bash?

alias myalias='nix subcommand'

It’s nice to see the difference between built-in and not. Otherwise you sit at a different system and wonder why the most basic commands are missing …

Even today you should be able to do this:

packageOverrides is dependent on nixpkgs, though. I don’t like the idea that most functionality of nix only works in conjunction with the default package set. In the same manner I don’t think we should hardcode the string nixpkgs anywhere in nix.

Based on my email:

Just wanted to ask, is this package search enhanced with showing package
parameters planned to be added to the standard command line Nix query
tools? Or if it's already available and I haven't found it.

I would like to suggest that any querying functionality should include the ability to show package parameters.

CMCDragonkai · Mar 21, 2016

Based on my email:

Just wanted to ask, is this package search enhanced with showing package
parameters planned to be added to the standard command line Nix query
tools? Or if it's already available and I haven't found it.

I would like to suggest that any querying functionality should include the ability to show package parameters.

Another suggestion, an inline way of overriding package parameters instead of using packageOverrides inside ~/.nixpkgs/config.nix. Something like nix-env -i firefox -o '{ blah = true; }'. Where the attribute set could be a recursive attribute set evaluated inside a function that takes the old pkgs, like pkgs: rec { blah = true; }.

Oh this already has been suggested. NVM.

CMCDragonkai · Mar 22, 2016

Another suggestion, an inline way of overriding package parameters instead of using packageOverrides inside ~/.nixpkgs/config.nix. Something like nix-env -i firefox -o '{ blah = true; }'. Where the attribute set could be a recursive attribute set evaluated inside a function that takes the old pkgs, like pkgs: rec { blah = true; }.

Oh this already has been suggested. NVM.

Something like nix-env -i firefox -o '{ blah = true; }'.

That is of course only an instance of the more general case of -E 'with import <nixpkgs> {}; firefox.override { blah = true; };'. We will have to do very close evaluation what special cases we want to introduce, since we are going to have to support them for all eternity.

I’m in strong favor of not depending on anything nixpkgs specific in the new tool.

Profpatsch · Mar 22, 2016

Something like nix-env -i firefox -o '{ blah = true; }'.

That is of course only an instance of the more general case of -E 'with import <nixpkgs> {}; firefox.override { blah = true; };'. We will have to do very close evaluation what special cases we want to introduce, since we are going to have to support them for all eternity.

I’m in strong favor of not depending on anything nixpkgs specific in the new tool.

Derivations and Nix sees them do not reflect any configuration options, and I'd like to keep it that way. I earlier proposed that this be developed in nixpkgs. But really there are 3 layers: nix itself, modules system tools and config idioms (other parts of nixpkgs/lib perhaps), and the actual nixpkgs packages. I agree nothing in the Nix repo should depend on nixpkgs at runtime, but whether we move this and module system into nix, or put tools in nixpkgs, we are combining those 3 layers into 2.

Ericson2314 · Mar 22, 2016

Derivations and Nix sees them do not reflect any configuration options, and I'd like to keep it that way. I earlier proposed that this be developed in nixpkgs. But really there are 3 layers: nix itself, modules system tools and config idioms (other parts of nixpkgs/lib perhaps), and the actual nixpkgs packages. I agree nothing in the Nix repo should depend on nixpkgs at runtime, but whether we move this and module system into nix, or put tools in nixpkgs, we are combining those 3 layers into 2.

Disclaimer: just an implementation detail.

You may want to consider docopt. There's a cpp port and some examples of how you may implement a huge multi-command app like git.

Greets

EmmanuelOga · Mar 31, 2016

Disclaimer: just an implementation detail.

You may want to consider docopt. There's a cpp port and some examples of how you may implement a huge multi-command app like git.

Greets

Don't forget to look at guix for inspiration. I've read some of its CLI documentation, it looks very good. Annoyingly good actually... can I haz?

Here is one (out of many) really useful looking guix sub-commands:
http://www.gnu.org/software/guix/manual/guix.html#Invoking-guix-refresh

bjornfor · Apr 30, 2016

Don't forget to look at guix for inspiration. I've read some of its CLI documentation, it looks very good. Annoyingly good actually... can I haz?

Here is one (out of many) really useful looking guix sub-commands:
http://www.gnu.org/software/guix/manual/guix.html#Invoking-guix-refresh

The code now contains nix command with some basic functionality (in src/nix/). I've been using it to see what it feels like. I was missing some flags, e.g. -kKj, so I looked into adding them, but I see they were moved into LegacyArgs which suggests they're not planned to be supported. @edolstra: can you confirm/explain that? I would assume at least some of those would be kept. Another point is that nix build doesn't print results or create those symlinks, but that seems just like not implemented yet.

BTW, working with the code is relatively hard for me as I can see just the code, almost without any explanation of intention etc., which makes it a little problematic to understand, especially as it's WIP. (When trying to understand header definitions, I often had to look into implementation and call sites to see what they're for.)

vcunat · May 26, 2016

The code now contains nix command with some basic functionality (in src/nix/). I've been using it to see what it feels like. I was missing some flags, e.g. -kKj, so I looked into adding them, but I see they were moved into LegacyArgs which suggests they're not planned to be supported. @edolstra: can you confirm/explain that? I would assume at least some of those would be kept. Another point is that nix build doesn't print results or create those symlinks, but that seems just like not implemented yet.

BTW, working with the code is relatively hard for me as I can see just the code, almost without any explanation of intention etc., which makes it a little problematic to understand, especially as it's WIP. (When trying to understand header definitions, I often had to look into implementation and call sites to see what they're for.)

If we're planning a multi command app like nix build and nix shell... etc. There should be an easy to to create nix aliases like how git allows git aliases.

CMCDragonkai · May 26, 2016

If we're planning a multi command app like nix build and nix shell... etc. There should be an easy to to create nix aliases like how git allows git aliases.

That was already discussed above in more detail.

vcunat · May 26, 2016

That was already discussed above in more detail.

When this will be implemented, will it be done in a backwards compatible manner? So that there are scripts around in my PATH which are nix-env -i and map to nix install for example, including printing a big fat "WARNING" or something that the nix UI has changed?

Another idea: Some of you might know my nixos-scripts which are basically scripts around nix-{env, shell, build} to provide some more functionality (like diffing of generations). These things will be covered in the new UI as far as I can see, but maybe you guys can get some more inspiration from my scripts on what to include (for example nix container new --template ~/container-templates/apache-container.template.nix or something like that). Just suggesting!

Edit: Another thing I really like to say that the idea for aliases is really great, but we should take this a small step further: the nix binary should be able to find nix-<foo> binaries in the $PATH and provide them as subcommands nix <foo>. This way, one could for example write a command nix ui to start a ncurses interface or nix gui to start a graphical interface for the nix command or even more tools I cannot think of yet.

matthiasbeyer · Jun 6, 2016

When this will be implemented, will it be done in a backwards compatible manner? So that there are scripts around in my PATH which are nix-env -i and map to nix install for example, including printing a big fat "WARNING" or something that the nix UI has changed?

Another idea: Some of you might know my nixos-scripts which are basically scripts around nix-{env, shell, build} to provide some more functionality (like diffing of generations). These things will be covered in the new UI as far as I can see, but maybe you guys can get some more inspiration from my scripts on what to include (for example nix container new --template ~/container-templates/apache-container.template.nix or something like that). Just suggesting!

Edit: Another thing I really like to say that the idea for aliases is really great, but we should take this a small step further: the nix binary should be able to find nix-<foo> binaries in the $PATH and provide them as subcommands nix <foo>. This way, one could for example write a command nix ui to start a ncurses interface or nix gui to start a graphical interface for the nix command or even more tools I cannot think of yet.

nix-env will coexist with new nix command. They both use the same api while exposing a different layer to the user.

domenkozar · Jun 6, 2016

nix-env will coexist with new nix command. They both use the same api while exposing a different layer to the user.

I thought nix-env's name-centric approach was mostly getting removed in the new UI?

copumpkin · Jun 6, 2016

I thought nix-env's name-centric approach was mostly getting removed in the new UI?

@copumpkin I hope @domenkozar means the same C++ API internally, and the new commands simply won't use the name-centric parts.

Is there a nixos-setting for using nixUnstable so we can try this out more easily?

Ericson2314 · Jun 6, 2016

@copumpkin I hope @domenkozar means the same C++ API internally, and the new commands simply won't use the name-centric parts.

Is there a nixos-setting for using nixUnstable so we can try this out more easily?

I don't see why not add a switch for name-based resolution.

vcunat · Jun 9, 2016

I don't see why not add a switch for name-based resolution.

@vcunat: I find it weird to have two different ways to name packages at install time (by attr or name). What is the use case of being able to install by name? Isn't it just slower?

EDIT: typo

bjornfor · Jun 9, 2016

@vcunat: I find it weird to have two different ways to name packages at install time (by attr or name). What is the use case of being able to install by name? Isn't it just slower?

EDIT: typo

I didn't mean to imply I know about some particular use cases. (And there's probably little sense to add the flag unless/until there's a nontrivial demand for it, especially as the old nix-env interface isn't going away soon.)

vcunat · Jun 9, 2016

I didn't mean to imply I know about some particular use cases. (And there's probably little sense to add the flag unless/until there's a nontrivial demand for it, especially as the old nix-env interface isn't going away soon.)

Is there a checklist somewhere for how much of this is already done and which things still need to be done? I might be able to contribute, but I'm not sure where to start.

solson · Jun 29, 2016

Is there a checklist somewhere for how much of this is already done and which things still need to be done? I might be able to contribute, but I'm not sure where to start.

I couldn't find anything like a checklist, and the current nix command implementation takes quite a while to understand if you come from outside, and getting reactions to PRs in the nix repo isn't easy either (from some of the few with push access).

vcunat · Jun 30, 2016

I couldn't find anything like a checklist, and the current nix command implementation takes quite a while to understand if you come from outside, and getting reactions to PRs in the nix repo isn't easy either (from some of the few with push access).

boy tbh i like every one

damonwayn · Nov 15, 2016

boy tbh i like every one

@vcunat, @domenkozar, how far along is this, and how does the community experiment with the current state of implementation?

deepfire · Dec 28, 2016

@vcunat, @domenkozar, how far along is this, and how does the community experiment with the current state of implementation?

I haven't been watching it. To me it feels very difficult to get any nix-repo changes reviewed/merged, so I focused on nixpkgs instead. My nix PRs commonly wait for weeks for any reaction from someone with merge rights, which is just difficult on my memory and motivation, etc.

vcunat · Dec 28, 2016

I haven't been watching it. To me it feels very difficult to get any nix-repo changes reviewed/merged, so I focused on nixpkgs instead. My nix PRs commonly wait for weeks for any reaction from someone with merge rights, which is just difficult on my memory and motivation, etc.

But note that the experimental nix command has been there for a long time, in unstable nix versions at least; you can try it.

vcunat · Dec 28, 2016

But note that the experimental nix command has been there for a long time, in unstable nix versions at least; you can try it.

@garbas said in his talk at NixCon 2015 that we need color and emojis.

For changing the color you can use ANSI escape codes.

http://stackoverflow.com/a/5947802/2611995

Example: printf "Nix: \033[0;31mError \033[0m25" ('Error' is red)

Emoji is also simple.

Example: printf '😀'

(i would really like to have colorful output but not sure if emoji are professional. brew uses only one and you can disable it https://evanhahn.com/disable-homebrew-emoji/)

davidak · Mar 31, 2017

@garbas said in his talk at NixCon 2015 that we need color and emojis.

For changing the color you can use ANSI escape codes.

http://stackoverflow.com/a/5947802/2611995

Example: printf "Nix: \033[0;31mError \033[0m25" ('Error' is red)

Emoji is also simple.

Example: printf '😀'

(i would really like to have colorful output but not sure if emoji are professional. brew uses only one and you can disable it https://evanhahn.com/disable-homebrew-emoji/)

I opened a separate (CLI-related) issue about (not) paging by default: #1261.

alexeymuranov · Mar 31, 2017

I opened a separate (CLI-related) issue about (not) paging by default: #1261.

Another thing is that when running nix-build, if the current directory already has a result symlink, that nix store path pointed to by the symlink shouldn't be left to rot, and should be garbage collected IMO. Otherwise repeated invocations of nix-build results in a "store leak".

CMCDragonkai · May 31, 2017

Another thing is that when running nix-build, if the current directory already has a result symlink, that nix store path pointed to by the symlink shouldn't be left to rot, and should be garbage collected IMO. Otherwise repeated invocations of nix-build results in a "store leak".

@CMCDragonkai Not sure I understand. Previous builds will be garbage-collected eventually. Or do you mean that they should be GC'ed immediately?

edolstra · May 31, 2017

@CMCDragonkai Not sure I understand. Previous builds will be garbage-collected eventually. Or do you mean that they should be GC'ed immediately?

I think the point is that src should filter out result symlinks.

We do that as part of cleanSourceFilter in https://github.com/NixOS/nixpkgs/blob/master/lib/sources.nix#L24

domenkozar · May 31, 2017

I think the point is that src should filter out result symlinks.

We do that as part of cleanSourceFilter in https://github.com/NixOS/nixpkgs/blob/master/lib/sources.nix#L24

@edolstra Yes I meant gced immediately. I didn't see any use of the existing store path.

CMCDragonkai · May 31, 2017

@edolstra Yes I meant gced immediately. I didn't see any use of the existing store path.

Bump everyone.
At least something?

It is crucial for Nix to evolve CLI before it becomes widely adopted and used in business production.

Than all going to support for 30 years what was while it became popular.

@davidak - I watched video long ago. That garbas jokes about emojis and you didn't get that. There you here: "color output, ascii art, emojis, more emojis, ponies?, more ponies! - the more - the better!"

What unprofessional means? When boss looks at utility you try to promote to roll on 1000 instances and XXXXXXL AWS databases, - and it uses emojis to explain messages.

My strictest best university teacher even tough us why and how to talk really as technical engineer.
No words 'we', 'here we have', 'you have', passive voice without any link to pearson actions.

And it does appeal. It is the most accepted, portable, informative scientific language. Nix in many places adopts diametrically opposite language.

A couple - possible, better in form of occasional small logo, maybe mascot in color ASCII art.

One smile in the end of successful deployment, when your hart otherwise stops.
And when epic fail occurred - it is better not to send any smiles to engineer, who now not going to sleep for a week troubleshooting some Nix related losses of data, in such cases those smile going to be established as a meme to avoid, with Nix altogether.

       .cc    .kkk,  .xk'       
       'cc:    'xkx'.dkk;       
        .cc:.   .dkkxkx'        
    ;cccccccc::::;okkk.   .,    
   .....,lll'......:kko. .cc;   
       .xxx'        'xxl,cc;    
,llllllxkd.          .c:ccc;;;;.
,lllokkko;.          .:c:,,,,,,.
    okkc;::.        .:::.       
   lkk,  ;::,.''''',ccc,''''.   
    c.   .ccc:lxxkkkkkkkkkkc    
        .ccccc:.   .xkk;        
       'cc: .ccc.   .okk;       
       .cc   .ccc.    xk.       

               :ccc.        lkkkkkk,      .kkkd               
              ;cccc:.        lkkkkkx.    .xkkkkl              
              ,cccccc.        :kkkkkk,  ,kkkkkkc              
               .cccccc,        ,kkkkkkcckkkkkk,               
                .:ccccc;        .xkkkkkkkkkkx.                
         ',,,,,,,:cccccc:,,,,,,,,'dkkkkkkkkd.                 
        ;ccccccccccccccccc::::::::'lkkkkkkd        .c;        
       :cccccccccccccccccc:::::::::,ckkkkkx,      .ccc:       
       ..........:ccccc:............ ,xkkkkk:    ,cccccc      
                lxxxxxd.              .xkxxxxl  ;cccccc.      
               okxxxxd.                .dxxxxx,:ccccc:        
             .dkkkkko                    oxxd,:ccccc:         
 oxxxxxxxxxxxxkkkkkc                      :l,ccccccccccccccc: 
okkkkkkkkkkkkkkkkk:                        ,ccccccccccccccccc;
 oxxxxxxxxkkkkkkx;;,                      ,::ccc::::::::::::; 
        .dkkkkkd,;::;                    ;::::::.             
       .xkkkkko'::::::.                .::::::;               
      'xkkkkkc  ,::::::.              .::::::,                
      xkkkkk:    .:::::c' ............;;;;;;;...........      
       okkx'      .:ccccc;:xxxxxxkkkkkkkkkkkkkkkkkkkkko       
        cx.        ;cccccc;;xxxxxkkkkkkkkkkkkkkkkkkkkl        
                  :cccccccc:,ccccccccokkkkkkdccccccc;         
                .:ccccccccccc.        ckkkkkx'                
               'cccccc,'cccccc'        :kkkkkk;               
              ,cccccc.  .cccccc,        'xkkkkkl              
              ,cccc:.    .:ccccc:        .xkkkkl              
               :ccc.      .cccccc:        .kkkd

Color coding - is crusial.
It is the most useful design feature.

This article was created by me: https://wiki.archlinux.org/index.php/Color_output_in_console

Anton-Latukha · Sep 4, 2017

Bump everyone.
At least something?

It is crucial for Nix to evolve CLI before it becomes widely adopted and used in business production.

Than all going to support for 30 years what was while it became popular.

@davidak - I watched video long ago. That garbas jokes about emojis and you didn't get that. There you here: "color output, ascii art, emojis, more emojis, ponies?, more ponies! - the more - the better!"

What unprofessional means? When boss looks at utility you try to promote to roll on 1000 instances and XXXXXXL AWS databases, - and it uses emojis to explain messages.

My strictest best university teacher even tough us why and how to talk really as technical engineer.
No words 'we', 'here we have', 'you have', passive voice without any link to pearson actions.

And it does appeal. It is the most accepted, portable, informative scientific language. Nix in many places adopts diametrically opposite language.

A couple - possible, better in form of occasional small logo, maybe mascot in color ASCII art.

One smile in the end of successful deployment, when your hart otherwise stops.
And when epic fail occurred - it is better not to send any smiles to engineer, who now not going to sleep for a week troubleshooting some Nix related losses of data, in such cases those smile going to be established as a meme to avoid, with Nix altogether.

       .cc    .kkk,  .xk'       
       'cc:    'xkx'.dkk;       
        .cc:.   .dkkxkx'        
    ;cccccccc::::;okkk.   .,    
   .....,lll'......:kko. .cc;   
       .xxx'        'xxl,cc;    
,llllllxkd.          .c:ccc;;;;.
,lllokkko;.          .:c:,,,,,,.
    okkc;::.        .:::.       
   lkk,  ;::,.''''',ccc,''''.   
    c.   .ccc:lxxkkkkkkkkkkc    
        .ccccc:.   .xkk;        
       'cc: .ccc.   .okk;       
       .cc   .ccc.    xk.       

               :ccc.        lkkkkkk,      .kkkd               
              ;cccc:.        lkkkkkx.    .xkkkkl              
              ,cccccc.        :kkkkkk,  ,kkkkkkc              
               .cccccc,        ,kkkkkkcckkkkkk,               
                .:ccccc;        .xkkkkkkkkkkx.                
         ',,,,,,,:cccccc:,,,,,,,,'dkkkkkkkkd.                 
        ;ccccccccccccccccc::::::::'lkkkkkkd        .c;        
       :cccccccccccccccccc:::::::::,ckkkkkx,      .ccc:       
       ..........:ccccc:............ ,xkkkkk:    ,cccccc      
                lxxxxxd.              .xkxxxxl  ;cccccc.      
               okxxxxd.                .dxxxxx,:ccccc:        
             .dkkkkko                    oxxd,:ccccc:         
 oxxxxxxxxxxxxkkkkkc                      :l,ccccccccccccccc: 
okkkkkkkkkkkkkkkkk:                        ,ccccccccccccccccc;
 oxxxxxxxxkkkkkkx;;,                      ,::ccc::::::::::::; 
        .dkkkkkd,;::;                    ;::::::.             
       .xkkkkko'::::::.                .::::::;               
      'xkkkkkc  ,::::::.              .::::::,                
      xkkkkk:    .:::::c' ............;;;;;;;...........      
       okkx'      .:ccccc;:xxxxxxkkkkkkkkkkkkkkkkkkkkko       
        cx.        ;cccccc;;xxxxxkkkkkkkkkkkkkkkkkkkkl        
                  :cccccccc:,ccccccccokkkkkkdccccccc;         
                .:ccccccccccc.        ckkkkkx'                
               'cccccc,'cccccc'        :kkkkkk;               
              ,cccccc.  .cccccc,        'xkkkkkl              
              ,cccc:.    .:ccccc:        .xkkkkl              
               :ccc.      .cccccc:        .kkkd

Color coding - is crusial.
It is the most useful design feature.

This article was created by me: https://wiki.archlinux.org/index.php/Color_output_in_console

Emojis are a useful way to encode ample information which can be seen at a glance, without requiring the reader to read and process text (ala "a picture is worth a thousand words"). I think they should be used way more often in general, even in technical tools, and I fully agree with @garbas.

vyp · Sep 4, 2017

Emojis are a useful way to encode ample information which can be seen at a glance, without requiring the reader to read and process text (ala "a picture is worth a thousand words"). I think they should be used way more often in general, even in technical tools, and I fully agree with @garbas.

Occasional emojis is cool with me. Especially when big complex process finished.
Smiles are great to get moral of person up. And to: additionally to technical information give emotional express.

And as I love you, Nix team, I share a secret that only my parents and girlfriend knows:
Ponies - are my favorite cartoon to relax.

I want to say serious stuff. I am a serious cat and this is serious thread:

There is classic system Syslog protocol (RFC 5424).
I wrote about it here: https://wiki.archlinux.org/index.php/Systemd#Priority_level

It very important for such tools as Nix to understand why it is important to correspond to it while logging/printing messages.

On bleeding-edge people forget about industry standards because they didn't know or remember (but I think most developers at least once seen that gradation), but on Enterprise servers, where still System V/SysV init system and Syslog is very strict and beautiful. systemd journal as you see also corresponds to it.

Why I talk about corresponding to Syslog protocol (RFC 5424) model?

Than - your messages is automatically sorted in logging system. So DevOps don't need to look through emojis of bagillions of log messages on ELK and do custom rules for every new error message.

He just says to report him:

journalctl -p 3..0

And as you can see - they are already color coded in terminal. journalctl does that, because it corresponds to that RFC message model. Color coded by ELK stack on cluster monitoring the same thing.

Color coded stuff can scroll at really fast speed, but you catch suden color change. Smiles are hard to grasp as efficient as that.

And your night allert on:

journalctl -p 1..0

Because it is a data loss happened. Financial dataloss, our worldwide service stopped to work. It is Russian slur here that can't be translated... It's a huge responsibility on shoulders. You probaly know that also.

Emojis not help me then.

As I wrote all that I really think you already know that message level protocol.

I mean.

Sysadmin had 1-3 mainframe 20-30 years ago. When that log message model was created.
It is still true and most usefull thing today. It works across all Linux distributions. And it going to be until developers remember about it and DevOps can be sure that warnings are on 4-th level, errors on 3-d.

Now DevOps is not so respected profession. But somehow we need to manage 1 000 instances, no big deal right?, without seconds of downtime and without errors in infrastructure. With managing/curating processes of development, production releases, scaling, migrations, oh, and security. Did I mentioned products on 10 languages we need to utilize and troubleshoot to make integration smooth. And all products has their perspective of how they want to do things. And you need to figure-out how to tie such products togather to have more stable system with less headache.

I tearfully ask you all, provide as strictly useful incightfull info for user of what is happening, as possible, with details that look like noone can understand them, don't reduse them to smiles. There is going to be a time - it going to play a role.

Anton-Latukha · Sep 4, 2017

Occasional emojis is cool with me. Especially when big complex process finished.
Smiles are great to get moral of person up. And to: additionally to technical information give emotional express.

And as I love you, Nix team, I share a secret that only my parents and girlfriend knows:
Ponies - are my favorite cartoon to relax.

I want to say serious stuff. I am a serious cat and this is serious thread:

There is classic system Syslog protocol (RFC 5424).
I wrote about it here: https://wiki.archlinux.org/index.php/Systemd#Priority_level

It very important for such tools as Nix to understand why it is important to correspond to it while logging/printing messages.

On bleeding-edge people forget about industry standards because they didn't know or remember (but I think most developers at least once seen that gradation), but on Enterprise servers, where still System V/SysV init system and Syslog is very strict and beautiful. systemd journal as you see also corresponds to it.

Why I talk about corresponding to Syslog protocol (RFC 5424) model?

Than - your messages is automatically sorted in logging system. So DevOps don't need to look through emojis of bagillions of log messages on ELK and do custom rules for every new error message.

He just says to report him:

journalctl -p 3..0

And as you can see - they are already color coded in terminal. journalctl does that, because it corresponds to that RFC message model. Color coded by ELK stack on cluster monitoring the same thing.

Color coded stuff can scroll at really fast speed, but you catch suden color change. Smiles are hard to grasp as efficient as that.

And your night allert on:

journalctl -p 1..0

Because it is a data loss happened. Financial dataloss, our worldwide service stopped to work. It is Russian slur here that can't be translated... It's a huge responsibility on shoulders. You probaly know that also.

Emojis not help me then.

As I wrote all that I really think you already know that message level protocol.

I mean.

Sysadmin had 1-3 mainframe 20-30 years ago. When that log message model was created.
It is still true and most usefull thing today. It works across all Linux distributions. And it going to be until developers remember about it and DevOps can be sure that warnings are on 4-th level, errors on 3-d.

Now DevOps is not so respected profession. But somehow we need to manage 1 000 instances, no big deal right?, without seconds of downtime and without errors in infrastructure. With managing/curating processes of development, production releases, scaling, migrations, oh, and security. Did I mentioned products on 10 languages we need to utilize and troubleshoot to make integration smooth. And all products has their perspective of how they want to do things. And you need to figure-out how to tie such products togather to have more stable system with less headache.

I tearfully ask you all, provide as strictly useful incightfull info for user of what is happening, as possible, with details that look like noone can understand them, don't reduse them to smiles. There is going to be a time - it going to play a role.

Color coded stuff can scroll at really fast speed, but you catch suden color change. Smiles are hard to grasp as efficient as that.

This is a good point you bring up. I just felt that seeing emojis as unprofessional seemed a bit short-sighted to me. But I'd agree that colour is probably faster to decipher, and once the reader has recognized the colour, log messages should probably be verbose and not reduced to emojis, as you say.

vyp · Sep 4, 2017

Color coded stuff can scroll at really fast speed, but you catch suden color change. Smiles are hard to grasp as efficient as that.

This is a good point you bring up. I just felt that seeing emojis as unprofessional seemed a bit short-sighted to me. But I'd agree that colour is probably faster to decipher, and once the reader has recognized the colour, log messages should probably be verbose and not reduced to emojis, as you say.

stdout stderr also logs, they logged as an example - in containers.

I try to not be curmudgeon, but whatever I want to say, - it sounds so. Sorry, probably I am.

Anton-Latukha · Sep 4, 2017

stdout stderr also logs, they logged as an example - in containers.

I try to not be curmudgeon, but whatever I want to say, - it sounds so. Sorry, probably I am.

Respect for the new CLI. Great.

I described, why all tools of corporations today use --help:
https://gist.github.com/Profpatsch/d5c8e1ccd68baab0e3f0

And I propose hack:
If on receiving command --help is get parsed - discard all options, legitimate they or not (because if person needs help, he could probably entered not full data or supplied wring keys (optinasa).
He wants help - show him help of current command argument.

command argument --option -o --option2 option.argument.is here (wait .. I forgot)
command argument --option -o --option2 option.argument.is here --help (reading `command argument --help`)
(Arrow up)
command argument --option -o --option2 option.argument.is here --help| (ALT + Backspace)
command argument --option -o --option2 option.argument.is here |
command argument --option -o --option2 option.argument.is here I am finishing command

And promote to use that hack.
I think some Nix commands inevitably going to be long (but it is great, because so much will be done greatly with it), and this hack can ease expirience for users that learn the tool.

Anton-Latukha · Sep 4, 2017

Respect for the new CLI. Great.

I described, why all tools of corporations today use --help:
https://gist.github.com/Profpatsch/d5c8e1ccd68baab0e3f0

And I propose hack:
If on receiving command --help is get parsed - discard all options, legitimate they or not (because if person needs help, he could probably entered not full data or supplied wring keys (optinasa).
He wants help - show him help of current command argument.

command argument --option -o --option2 option.argument.is here (wait .. I forgot)
command argument --option -o --option2 option.argument.is here --help (reading `command argument --help`)
(Arrow up)
command argument --option -o --option2 option.argument.is here --help| (ALT + Backspace)
command argument --option -o --option2 option.argument.is here |
command argument --option -o --option2 option.argument.is here I am finishing command

And promote to use that hack.
I think some Nix commands inevitably going to be long (but it is great, because so much will be done greatly with it), and this hack can ease expirience for users that learn the tool.

Bear in mind that color coding might not help a visually impaired person all that much, the information needs some semantic information as well (read: journal log levels).

Profpatsch · Sep 4, 2017

Bear in mind that color coding might not help a visually impaired person all that much, the information needs some semantic information as well (read: journal log levels).

I haven't read all the previous discussion so I'm not sure if this has already been brought up but I had a quick look and it didn't seem like it. With nix-shell we can use it as a shebang interpreter (this is outlined in the man page for it):

#! /usr/bin/env nix-shell
#! nix-shell -i bash -p bash
echo hello world

$ ./hello
hello world
$

But with nix shell it doesn't work:

#! /usr/bin/env nix shell
#! nix shell -i bash -p bash
echo hello world

$ ./hello
/usr/bin/env: ‘nix shell’: No such file or directory
$

This is because the POSIX script invocation mechanism only allows one argument to appear on the #! line after the path to the executable, and so it reads nix shell as one "nix shell" string (I have nixUnstable installed). So I was wondering if this was already considered somewhere?

Guile uses a 'meta switch' as a workaround for this. So if this hasn't been considered maybe something like that could also be implemented for nix, to make it continue to work as a shebang interpreter?

Edit: I also found https://github.com/shlevy/long-shebang.

vyp · Oct 3, 2017

I haven't read all the previous discussion so I'm not sure if this has already been brought up but I had a quick look and it didn't seem like it. With nix-shell we can use it as a shebang interpreter (this is outlined in the man page for it):

#! /usr/bin/env nix-shell
#! nix-shell -i bash -p bash
echo hello world

$ ./hello
hello world
$

But with nix shell it doesn't work:

#! /usr/bin/env nix shell
#! nix shell -i bash -p bash
echo hello world

$ ./hello
/usr/bin/env: ‘nix shell’: No such file or directory
$

This is because the POSIX script invocation mechanism only allows one argument to appear on the #! line after the path to the executable, and so it reads nix shell as one "nix shell" string (I have nixUnstable installed). So I was wondering if this was already considered somewhere?

Guile uses a 'meta switch' as a workaround for this. So if this hasn't been considered maybe something like that could also be implemented for nix, to make it continue to work as a shebang interpreter?

Edit: I also found https://github.com/shlevy/long-shebang.

Maybe it can be just:

nix-shell
---
#!/bin/sh
nix shell

So manually you can use nix shell seamlessly as all other commands.
And nix-shell for scripting.

Anton-Latukha · Oct 3, 2017

Maybe it can be just:

nix-shell
---
#!/bin/sh
nix shell

So manually you can use nix shell seamlessly as all other commands.
And nix-shell for scripting.

Having a distinction between the two seems very reasonable. It might even be called nix-script in that case? That way there at least is no confusion between the two (with dash, without dash).

bobvanderlinden · Oct 3, 2017

Having a distinction between the two seems very reasonable. It might even be called nix-script in that case? That way there at least is no confusion between the two (with dash, without dash).

@bobvanderlinden It exists already, nix-script

rnhmjoj · Oct 3, 2017

@bobvanderlinden It exists already, nix-script

@Anton-Latukha But that's sort of my point though, we can't use nix-shell if it becomes deprecated.

vyp · Oct 3, 2017

@Anton-Latukha But that's sort of my point though, we can't use nix-shell if it becomes deprecated.

I'm not sure whether this is still active here, but I'd like to propose a hook interface.

The idea behind that is a use-case (of course): I run custom scripts when rebuilding my system. The relevant ones are the channel-update and switch which both have a single purpose: Build the update (channel or system) and tag my configuration with a special git tag (special as in it identifies the generation by number) so I can see in my configuration repository which generation was build from what git commit.

Having a hook executed for each action in the rebuild process would simplify that a lot.

I think about these hook points:

Starting to build a new generation
Rebuiling a generation failed
Rebuiling a generation succeeded

And parameters should give the script the ability to see what gets rebuild, who did the rebuild, how it gets build (switch, boot, build, test, ...) and possibly more I do not think of right now.

matthiasbeyer · Nov 3, 2017

I'm not sure whether this is still active here, but I'd like to propose a hook interface.

The idea behind that is a use-case (of course): I run custom scripts when rebuilding my system. The relevant ones are the channel-update and switch which both have a single purpose: Build the update (channel or system) and tag my configuration with a special git tag (special as in it identifies the generation by number) so I can see in my configuration repository which generation was build from what git commit.

Having a hook executed for each action in the rebuild process would simplify that a lot.

I think about these hook points:

Starting to build a new generation
Rebuiling a generation failed
Rebuiling a generation succeeded

And parameters should give the script the ability to see what gets rebuild, who did the rebuild, how it gets build (switch, boot, build, test, ...) and possibly more I do not think of right now.

Don't know about those use cases, but it would be good to have a garbage collector hook to allow NixOS to remove boot menu entries for GC'ed versions.

edolstra · Nov 3, 2017

Don't know about those use cases, but it would be good to have a garbage collector hook to allow NixOS to remove boot menu entries for GC'ed versions.

@vcunat @Profpatsch @edolstra Really like this new interface! Quick comment on the subject of commands that take attributes being able to work with expression though.

Given that I can tell our users (Canadian HPC center) to do this to get an environment with python3

nix run nixpkgs.python3

it would be very nice if I could tell them to do this to get an environment with python3 and numpy

nix run 'nixpkgs.python3.withPackages (pkgs: [pkgs.numpy])'

instead of having to drop this sort of thing on them

nix run '(let nixpkgs = import <nixpkgs> { }; in nixpkgs.python3.withPackages (pkgs: [pkgs.numpy]))'

twhitehead · Jan 12, 2018

@vcunat @Profpatsch @edolstra Really like this new interface! Quick comment on the subject of commands that take attributes being able to work with expression though.

Given that I can tell our users (Canadian HPC center) to do this to get an environment with python3

nix run nixpkgs.python3

it would be very nice if I could tell them to do this to get an environment with python3 and numpy

nix run 'nixpkgs.python3.withPackages (pkgs: [pkgs.numpy])'

instead of having to drop this sort of thing on them

nix run '(let nixpkgs = import <nixpkgs> { }; in nixpkgs.python3.withPackages (pkgs: [pkgs.numpy]))'

@twhitehead, you can already simplify that to the following, can't you?

nix run '(import <nixpkgs> {}).python3.withPackages (p: [p.numpy])'

Wizek · Jan 12, 2018

@twhitehead, you can already simplify that to the following, can't you?

nix run '(import <nixpkgs> {}).python3.withPackages (p: [p.numpy])'

@Wizek Yes, that is more compact and avoids introducing let ...; in ..., which is nice.

I would still really like to provide colleagues and users with a command that looks like just a small extension of the basic one and avoids exposing any more of the nix language than absolutely required.

I know it may seem silly, but being able to do the basics (e.g., install/use python with a select set of packages) with as uncryptic command as possible is really important for adoption in our organization.

twhitehead · Jan 12, 2018

@Wizek Yes, that is more compact and avoids introducing let ...; in ..., which is nice.

I would still really like to provide colleagues and users with a command that looks like just a small extension of the basic one and avoids exposing any more of the nix language than absolutely required.

I know it may seem silly, but being able to do the basics (e.g., install/use python with a select set of packages) with as uncryptic command as possible is really important for adoption in our organization.

On the subject of being able to do the basics like installing python with a set of packages with as nonthreatening looking command as possible (i.e., gives the impression that I understand what is going on without having to learn a whole bunch), I've been wondering if even this could be improved upon

nix run 'nixpkgs.python3.withPackages (pkgs: [pkgs.numpy pkgs.matplotlib])'

For example (not suggesting this is the best way to do this, but more throwing it out as an example to get the conversation started), as withPackages (pkgs: [ ... ]) seems to have become a fairly standard construct, what if there was some syntatic sugar like so

expr1 | expr2 ... --> expr1 (dict: with dict; [ expr2 ... ])

to enable to following sort of commands

nix run 'nixpkgs.python3.withPackages | numpy matplotlib'

twhitehead · Jan 12, 2018

On the subject of being able to do the basics like installing python with a set of packages with as nonthreatening looking command as possible (i.e., gives the impression that I understand what is going on without having to learn a whole bunch), I've been wondering if even this could be improved upon

nix run 'nixpkgs.python3.withPackages (pkgs: [pkgs.numpy pkgs.matplotlib])'

For example (not suggesting this is the best way to do this, but more throwing it out as an example to get the conversation started), as withPackages (pkgs: [ ... ]) seems to have become a fairly standard construct, what if there was some syntatic sugar like so

expr1 | expr2 ... --> expr1 (dict: with dict; [ expr2 ... ])

to enable to following sort of commands

nix run 'nixpkgs.python3.withPackages | numpy matplotlib'

This could be done by having an option to pass regular arguments like --arg does for attribute arguments. So something like this (not sure what the best option name is though):

nix run nixpkgs.python.withPackages --arg1 'pkgs: [pkgs.numpy pkgs.matplotlib]'

hedning · Jan 12, 2018

This could be done by having an option to pass regular arguments like --arg does for attribute arguments. So something like this (not sure what the best option name is though):

nix run nixpkgs.python.withPackages --arg1 'pkgs: [pkgs.numpy pkgs.matplotlib]'

Or, if the attribute selection is to remain special syntax, then it could just be extended to also be able to take an optional space separated list on the end that is translated like so

attr --> attr
attr val1 ...  --> attr (dict: with dict; [val1 ...])

allowing the fabulously clean and intuitive looking

nix run 'nixpkgs.python3.withPackages numpy matplotlib'

twhitehead · Jan 12, 2018

Or, if the attribute selection is to remain special syntax, then it could just be extended to also be able to take an optional space separated list on the end that is translated like so

attr --> attr
attr val1 ...  --> attr (dict: with dict; [val1 ...])

allowing the fabulously clean and intuitive looking

nix run 'nixpkgs.python3.withPackages numpy matplotlib'

Regardless of how the final syntax is, it would be great if all the language-specific packages had the same style of invocation.

CMCDragonkai · Jan 13, 2018

Regardless of how the final syntax is, it would be great if all the language-specific packages had the same style of invocation.

@hedning @CMCDragonkai with regard to options and the same style of invocation, maybe it would be possible to add command line options that do common invocation things (thus also encourage further packages to implement those invocation styles).

For example, a --with-packages option could be added

nix run nixpkgs.python3 --with-packages 'numpy matplotlib'

that, given the above, would check if nixpkgs.python3 has a withPackages attribute and then invoke it like so

nixpkgs.python3.withPackages (p: with p;  [ numpy matplotlib ])

Likewise, an --override option could be added

nix use nixpkgs.python3 --override x11Support=true'

that would check for an override attribute and then invoke it like so if it exists

nixpkgs.python3.override { x11Support = true; }

This has the advantage of being fairly future proof as it provides a layer of abstraction between the user and the continuously evolving nixpkgs interfaces. Options can be added, deprecated, removed, or even just have their internal implementation details changed to continue providing the same functionality against new nixpkgs interfaces.

One pain I could see is that as nixpkgs evolves, nix would start having to check versions to provide the appropriate glue code. This suggests that possible the majority of the functionality should live in nixpkgs itself. For example, nixpkgs could provide a top-level attribute set that nix can use to extend the options it takes. Combined with a standard way of serializing option data, nix could then accept these options and invoke the corresponding nixpkgs expressions and let them manipulate the underlying nix expression that is ultimately used.

That is, nix would start with an expression for the specified attribute. This would be passed to the option function corresponding to the first option along with the serialized option data. This would give a new expression, which would then be passed to the option function corresponding to the second option along with its serialized option data and so on until there are no more options. Then nix uses the final expression to obtain the required derivation instead of the specified initial attribute.

twhitehead · Jan 13, 2018

@hedning @CMCDragonkai with regard to options and the same style of invocation, maybe it would be possible to add command line options that do common invocation things (thus also encourage further packages to implement those invocation styles).

For example, a --with-packages option could be added

nix run nixpkgs.python3 --with-packages 'numpy matplotlib'

that, given the above, would check if nixpkgs.python3 has a withPackages attribute and then invoke it like so

nixpkgs.python3.withPackages (p: with p;  [ numpy matplotlib ])

Likewise, an --override option could be added

nix use nixpkgs.python3 --override x11Support=true'

that would check for an override attribute and then invoke it like so if it exists

nixpkgs.python3.override { x11Support = true; }

This has the advantage of being fairly future proof as it provides a layer of abstraction between the user and the continuously evolving nixpkgs interfaces. Options can be added, deprecated, removed, or even just have their internal implementation details changed to continue providing the same functionality against new nixpkgs interfaces.

One pain I could see is that as nixpkgs evolves, nix would start having to check versions to provide the appropriate glue code. This suggests that possible the majority of the functionality should live in nixpkgs itself. For example, nixpkgs could provide a top-level attribute set that nix can use to extend the options it takes. Combined with a standard way of serializing option data, nix could then accept these options and invoke the corresponding nixpkgs expressions and let them manipulate the underlying nix expression that is ultimately used.

That is, nix would start with an expression for the specified attribute. This would be passed to the option function corresponding to the first option along with the serialized option data. This would give a new expression, which would then be passed to the option function corresponding to the second option along with its serialized option data and so on until there are no more options. Then nix uses the final expression to obtain the required derivation instead of the specified initial attribute.

I was thinking some more about this, and had another suggestion along these lines. What if individual packages could be passed command line options, so something like this

nix run nixpkgs.python3 --with numpy matplotlib --foo --bar -- nixpkgs.gcc

would be translated into something like this

nixpkgs.python3.cmdline { with = [ "numpy" "matplotlib" ]; foo = []; bar = []; }
nixpkgs.gcc

That is, if a set of options are specified after an attribute selection, they are collected into an attribute set and passed to a cmdline handling attribute of the selection attribute (or whatever name would be appropriate) if it exists, otherwise an x does not support command line options error is generated.

Thanks! -Tyson

twhitehead · Feb 4, 2018

I was thinking some more about this, and had another suggestion along these lines. What if individual packages could be passed command line options, so something like this

nix run nixpkgs.python3 --with numpy matplotlib --foo --bar -- nixpkgs.gcc

would be translated into something like this

nixpkgs.python3.cmdline { with = [ "numpy" "matplotlib" ]; foo = []; bar = []; }
nixpkgs.gcc

That is, if a set of options are specified after an attribute selection, they are collected into an attribute set and passed to a cmdline handling attribute of the selection attribute (or whatever name would be appropriate) if it exists, otherwise an x does not support command line options error is generated.

Thanks! -Tyson

There is an interesting trade off between providing a generic (more power, the ultimate being an arbitrary function transformation) and a specific interface (better error messages and such) for these things. It seems you want to be a specific as possible but no more specific. This allowing everything that needs to be done to be done while still giving the best possible error messages and such.

Most of my suggestions so far have been for generic machinery. Here is one for specific machinery instead. Maybe packages could optionally provide a options (or meta.options) attribute as with NixOS modules. This could specify what options (flags) could be provided, their types (i.e., boolean, string, etc.), and document them. All of which would make for a very good user experience (think help and error messages).

This might ultimately also dovetail nicely with also having a better config system. That is, one where options are declared in a modular manner, documented, and the user's config is verified against them in order to provide a top-level pkgs config attribute, rather than the current case of the top-level config attribute just be sucked in from ~/.config/nixpkgs/config.nix. Deeper operations (i.e., more fundamental modifications to nixpkgs) would then be left to be done as overlays.

In terms of the command line usage, you could imagine the NixOs options/config machinery merging and verifying nixpkgs.config from /etc/nixos/config.nix, ~/.config/nixpkgs/config.nix, and finally a config specification generated from the command line options to provide the top level config attribute to nixpkgs. A high-level package like python could then provide a packages list-of-strings option declaration and access it via the top-level config.python.packages attribute.

nix run nixpkgs.python --packages numpy scipy -- nixpkgs.gcc

twhitehead · Feb 4, 2018

There is an interesting trade off between providing a generic (more power, the ultimate being an arbitrary function transformation) and a specific interface (better error messages and such) for these things. It seems you want to be a specific as possible but no more specific. This allowing everything that needs to be done to be done while still giving the best possible error messages and such.

Most of my suggestions so far have been for generic machinery. Here is one for specific machinery instead. Maybe packages could optionally provide a options (or meta.options) attribute as with NixOS modules. This could specify what options (flags) could be provided, their types (i.e., boolean, string, etc.), and document them. All of which would make for a very good user experience (think help and error messages).

This might ultimately also dovetail nicely with also having a better config system. That is, one where options are declared in a modular manner, documented, and the user's config is verified against them in order to provide a top-level pkgs config attribute, rather than the current case of the top-level config attribute just be sucked in from ~/.config/nixpkgs/config.nix. Deeper operations (i.e., more fundamental modifications to nixpkgs) would then be left to be done as overlays.

In terms of the command line usage, you could imagine the NixOs options/config machinery merging and verifying nixpkgs.config from /etc/nixos/config.nix, ~/.config/nixpkgs/config.nix, and finally a config specification generated from the command line options to provide the top level config attribute to nixpkgs. A high-level package like python could then provide a packages list-of-strings option declaration and access it via the top-level config.python.packages attribute.

nix run nixpkgs.python --packages numpy scipy -- nixpkgs.gcc

Can we close this ticket? It feels to me like it's not terribly useful any more. If anyone has issues with the new UI, then it's probably better to open a new, specific ticket about that.

peti · Apr 21, 2018

Can we close this ticket? It feels to me like it's not terribly useful any more. If anyone has issues with the new UI, then it's probably better to open a new, specific ticket about that.

Did everything mentioned here get addressed? If not, maybe they can be packaged into a feature wish-list wiki?

CMCDragonkai · Apr 27, 2018

Did everything mentioned here get addressed? If not, maybe they can be packaged into a feature wish-list wiki?

Profpatsch changed the title from Redesign of the nix tool to Redesign of the nix command line Jan 21, 2016

ericsagnes referenced this issue Mar 5, 2016
Closed
Proposal: give a dedicated name to Nix Expression Language #835

Ericson2314 referenced this issue May 10, 2016
Open
Inconsistency in -E between nix-env/nix-build #899

joelmo referenced this issue Jun 20, 2016
Closed
zsh/bash/... completion for nix* commands #37

olejorgenb referenced this issue Jun 25, 2016
Merged
Attribute path completion for nix-env, etc. #4

Profpatsch closed this Sep 4, 2017

Profpatsch reopened this Sep 4, 2017

vyp referenced this issue Sep 6, 2017
Closed
GFM #83

bobvanderlinden referenced this issue Oct 2, 2017
Open
nixops depends on nix-channel #736

gnidorah referenced this issue Mar 18, 2018
Closed
nix build is unusable #36489

shlevy added the backlog label Apr 1, 2018

shlevy assigned peti Apr 1, 2018

peti closed this Apr 27, 2018

NixOS/nix

Join GitHub today

Redesign of the nix command line #779

Comments

Profpatsch commented Jan 21, 2016

This comment has been minimized.

Profpatsch commented Jan 21, 2016

Profpatsch changed the title from Redesign of the nix tool to Redesign of the nix command line Jan 21, 2016

This comment has been minimized.

Ericson2314 commented Jan 22, 2016

This comment has been minimized.

Ericson2314 commented Jan 22, 2016

This comment has been minimized.

zimbatm commented Jan 23, 2016

This comment has been minimized.

First my strongest points:

Now some replies:

More detailed thoughts of mine, ordered according to the original proposal:

vcunat commented Jan 24, 2016

First my strongest points:

Now some replies:

More detailed thoughts of mine, ordered according to the original proposal:

This comment has been minimized.

Profpatsch commented Jan 25, 2016

This comment has been minimized.

glaebhoerl commented Jan 25, 2016

This comment has been minimized.

vcunat commented Jan 26, 2016

This comment has been minimized.

CMCDragonkai commented Jan 27, 2016

This comment has been minimized.

cstrahan commented Jan 28, 2016

This comment has been minimized.

cstrahan commented Jan 28, 2016

This comment has been minimized.

vcunat commented Jan 28, 2016

This comment has been minimized.

Profpatsch commented Jan 29, 2016

ericsagnes referenced this issue Mar 5, 2016

This comment has been minimized.

CMCDragonkai commented Mar 21, 2016

This comment has been minimized.

CMCDragonkai commented Mar 22, 2016

This comment has been minimized.

Profpatsch commented Mar 22, 2016

This comment has been minimized.

Ericson2314 commented Mar 22, 2016

This comment has been minimized.

EmmanuelOga commented Mar 31, 2016

This comment has been minimized.

bjornfor commented Apr 30, 2016

Ericson2314 referenced this issue May 10, 2016

This comment has been minimized.

vcunat commented May 26, 2016

This comment has been minimized.

CMCDragonkai commented May 26, 2016

This comment has been minimized.

vcunat commented May 26, 2016

This comment has been minimized.

matthiasbeyer commented Jun 6, 2016 • edited Edited 1 time matthiasbeyer edited Jun 6, 2016 (most recent)

This comment has been minimized.

domenkozar commented Jun 6, 2016

This comment has been minimized.

copumpkin commented Jun 6, 2016

This comment has been minimized.

Ericson2314 commented Jun 6, 2016 • edited Edited 1 time Ericson2314 edited Jun 6, 2016 (most recent)

This comment has been minimized.

vcunat commented Jun 9, 2016

This comment has been minimized.

bjornfor commented Jun 9, 2016 • edited Edited 1 time bjornfor edited Jun 9, 2016 (most recent)

This comment has been minimized.

vcunat commented Jun 9, 2016

joelmo referenced this issue Jun 20, 2016

olejorgenb referenced this issue Jun 25, 2016

This comment has been minimized.

solson commented Jun 29, 2016

This comment has been minimized.

vcunat commented Jun 30, 2016

This comment has been minimized.

damonwayn commented Nov 15, 2016

matthiasbeyer commented Jun 6, 2016 •

edited

Edited 1 time

matthiasbeyer edited Jun 6, 2016 (most recent)

Ericson2314 commented Jun 6, 2016 •

edited

Edited 1 time

Ericson2314 edited Jun 6, 2016 (most recent)

bjornfor commented Jun 9, 2016 •

edited

Edited 1 time

bjornfor edited Jun 9, 2016 (most recent)

vcunat commented Dec 28, 2016 •

edited

Edited 1 time

vcunat edited Dec 28, 2016 (most recent)

CMCDragonkai commented May 31, 2017 •

edited

Edited 1 time

CMCDragonkai edited May 31, 2017 (most recent)

Anton-Latukha commented Sep 4, 2017 •

edited

Edited 1 time

Anton-Latukha edited Sep 4, 2017 (most recent)

vyp commented Sep 4, 2017 •

edited

Edited 1 time

vyp edited Sep 4, 2017 (most recent)