Learning Journey tutorial - "nix-shell -p"

[zmitchell]

The Learning Journey WG identified that the first tutorial in the learning journey should be a tutorial walking the user through creating ephemeral shell environments with nix-shell -p. This is the tracking issue for discussion on this tutorial and related work.

• Compile a survey of nix-shell tutorials (@erooke)
• Determine which ground we want to cover in the tutorial
  • Do we want to cover just nix-shell -p, or do we want to end with creating a shell.nix?
• Prepare an outline of the tutorial
• Prepare a draft of the tutorial
• Submit the draft as a PR

Tutorial contents

• Install Python via nix-shell -p python310
• Illustrate that the version of Python outside nix-shell differs from the version of Python inside nix-shell
• Illustrate that once you leave nix-shell you no longer have access to the Python 3.10 that you had inside the shell
• Demonstrate how to run a single command inside an ephemeral shell with nix-shell -p python310 --run python3 --version
• Explain that using nix-shell -p <pkgs> is useful for experimentation

How

• Where to put the result: A new file in https://github.com/NixOS/nix.dev/tree/master/source/tutorials
• Person to contact for help:
  • Technical: @infinisil
  • Editorial: @zmitchell

[erooke]

Here is a dump of all the notable resources I found on nix-shell last week along with a brief summary trying to categorize it according to the diataxis.

Unofficial

• ghedam
  Introduces nix-shell -p, shows how it modifies the env, builds a shell.nix with a shellHook. Includes examples for python and rust. Pins dependencies and attempts to create a cross platform shell.nix. Somewhere between a tutorial and a how to guide.

• inria
  Uses nix-shell --pure to test compiling a package in a derivation. Moves on to defining a dev environment inside a shell.nix. Tutorial.

• Xebia functional (Video)
  Introduction to nix in general. Installs nix, uses nix-env to install a package, uses nix-shell to run a package not installed. Tutorial.

• Wil taylor (Video)
  Quick nix-shell -p hello example, introduces shell.nix with a shellHook, shows how to add custom scripts to the shell with pkgs.writeScriptBin. How to guide?

Official

• nix pill Explanation.
• manual Reference.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-03-30-learning-journey-working-group-meeting-notes-2/26858/1

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/this-month-in-nix-docs-1-march-2023/26913/1

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-04-06-documentation-team-meeting-notes-39/27048/1

[zmitchell]

We've identified that the tutorial should use language-agnostic tooling to accomplish some kind of task. A reasonable suggestion is to use curl and jq to query an API, or something similar.

Next steps are formulating a more concrete idea for the task the reader will complete and drafting an outline.

[fricklerhandwerk]

Just as a side consideration: currently one learning objective is finding package names for programs you care about – an essential skill.

A mundane example where they are "accidentally" equal loses the effect of teaching the difference between package names and program names by assigning a task where they are not equal but the software is quite well-known. If we can somehow preserve that, it would be great.

We may want to end the first tutorial after the nested shells part, and then provide the curl and jq task as a final exercise for searching packages, and then pick it up in the shell.nix tutorial.

The "towards reproducibility" part is really another topic, and could as well be covered in more depth in a dedicated tutorial which goes over the multiple ways of nailing down package sources (-I, channel:<name> syntax, Nixpkgs commit hashes, ...) using practical examples.

[erooke]

The set of constraints for the programs is a weird set to simultaneously meet. If I understand correctly we want a command line tool which is:

1. useful to developers
2. language agnostic
3. exoctic enough to not be commonly packaged
4. has minimal (no) configuration

The idea of using curl and jq is interesting but fail to meet criteria 3 as curl is ubiquitous and jq is packaged for debian, ubuntu, fedora, suse, and arch. We could add glow as a command which formats markdown for a terminal and is much less commonly packaged. This allows for a command which fetches the most recent issue (or pr) from a repo and prints them like so:

gh api \
  --method GET \
  -F per_page=1 \
  -f status="open" \
  -H "Accept: application/vnd.github+json" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  /repos/nixos/nix.dev/issues | \
jq -r '.[] | "# " + ( .number | tostring) + "\n" + .body ' | \
glow -

This produces the following output:

My only concern with this route is the commands start to get quite lengthy. I have been really struggling to come up with tools which check all 4 boxes that doesn't fall into lengthy commands or lengthy setup.

[fricklerhandwerk]

It should not be available by default. Being packaged at all is not a criterion.

[zmitchell]

Could you elaborate? I’m not sure what you mean by this.

…

It should not be available by default. Being packaged at all is not a criterion.

[zmitchell]

What about installing something like Nginx and a load testing tool like wrk? We could provide a basic Nginx config that serves some static (local) content, and then provide some commands to run wrk (again, locally) against this web server.

Edit: On second thought Nginx isn't really a command line tool that you can quickly run without configuration.

[fricklerhandwerk]

Could you elaborate? I’m not sure what you mean by this.

It should not be available by default. Being packaged at all is not a criterion.

Those packages we use for demonstration should not be installed on a vanilla Ubuntu or Debian or Arch. They of course will be packaged and available for installation on these systems, and that shouldn't bother us.

[henrik-ch]

Could you elaborate? I’m not sure what you mean by this.

It should not be available by default. Being packaged at all is not a criterion.

Those packages we use for demonstration should not be installed on a vanilla Ubuntu or Debian or Arch. They of of course will be packaged and available for installation on these systems, and that shouldn't bother us.

Using 2 packages you would find on vanilla Ubuntu and a third which you don't would meet the criteria, right?

[henrik-ch]

Just adding time stamps to these for ease of determined priority

Unofficial

• ghedam (July 2020)
• inria (Feb 2021)
• Xebia functional (Video) (Apr 2022) 7.9K views 09:32 length
• Wil taylor (Video) (Apr 2021) 5.6K views 10:36 length

Official

• nix pill Explanation. (approx 2020)
• manual Reference. (spring 2023)

[zmitchell]

@roberth assigned to you for feedback

[proofconstruction]

useful to developers
language agnostic
has minimal (no) configuration

Text editors (e.g. sublime*) and IRC clients (e.g. irssi) satisfy this set of criteria, and can be used somewhat non-trivially (so, at least a bit more than the cowsay + lolcat example in ad hoc shell environments):

• open the editor, create and save a file, close the editor, cat the created file
• open the IRC client, connect to a developer-focused server (maybe the NixOS Matrix? I don't use the IRC bridge so I can't comment on how appropriate this would be for a tutorial)

*Sublime is GUI-only as far as I know, but I'm not sure we need to restrict attention to CLI-only applications.

[roberth]

nix-shell is a warty piece of old software that sadly doesn't have a stable alternative yet.
"nix-shell -p to bring a shell command in to scope" in particular is a complex and architecturally unsound way of achieving something that should have been very simple.

So the main caveats are

• nix-shell always produces a build environment that approximates the shell that builds a mkDerivation derivation.
• This build environment, in case of -p is a legacy build environment that's not compatible with strictDeps = true or cross compilation. It also does not support shell tab completion for any of the "new" commands.
• An architectural note: a "nix shell" or "dev shell" or whatever is not an inherent property of a derivation. (It might become a package attribute at some point, but don't mention this)

These points should be kept in mind, as without them, it's easy to accidentally document false or incoherent statements.
Just because a hacky usage - which this is - has become canon doesn't mean that the design has changed.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-07-13-learning-journey-working-group-meeting-notes-17/30393/1

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-07-17-tutorial-series-call-for-feedback/30616/1