Skip to content

Commit

Permalink
simplify, deduplicate
Browse files Browse the repository at this point in the history
- move introduction to the overview
- use nix-shell and nix-instantiate
- be a bit more precise where it seems important
- add section headings
- add more links
- cut redundant parts from the deep dive
  • Loading branch information
fricklerhandwerk committed May 22, 2024
1 parent 30ea522 commit cfe0f78
Show file tree
Hide file tree
Showing 8 changed files with 92 additions and 127 deletions.
1 change: 1 addition & 0 deletions _redirects
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,6 @@
/tutorials/learning-journey/sharing-dependencies /guides/recipes/sharing-dependencies 301
/tutorials/learning-journey/packaging-existing-software /tutorials/packaging-existing-software 301
/tutorials/file-sets /tutorials/working-with-local-files 301
/tutorials/module-system/module-system /tutorials/module-system/deep-dive 301

/permalink/stub-ld /guides/faq#how-to-run-non-nix-executables 301
10 changes: 10 additions & 0 deletions source/tutorials/module-system/a-basic-module/default.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{ pkgs ? import <nixpkgs> { } }:
let
result = pkgs.lib.evalModules {
modules = [
./options.nix
./config.nix
];
};
in
result.config
7 changes: 0 additions & 7 deletions source/tutorials/module-system/a-basic-module/eval.nix

This file was deleted.

1 change: 1 addition & 0 deletions source/tutorials/module-system/a-basic-module/eval.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
nix-shell -p jq --run "nix-instantiate --eval --json | jq"
104 changes: 43 additions & 61 deletions source/tutorials/module-system/a-basic-module/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,107 +2,89 @@

What is a module?

* A module is a function that takes an attrset and returns an attrset.
* It *may* declare options.
* It *may* define option values.
* When evaluated, it produces a configuration based on the declarations and definitions.
* A module is a function that takes an attribute set and returns an attribute set.
* It may declare options, telling which attributes are allowed in the final outcome.
* It may define values, for options declared by itself or other modules.
* When evaluated by the module system, it produces an attribute set based on the declarations and definitions.

The format is like so:
The simplest possible module is a function that takes any attributes and returns an empty attribute set:

```{code-block} nix
:caption: useless.nix
{...}: {
:caption: options.nix
{ ... }:
{
}
```

This, as the filename suggests, is completely useless.
It takes no arguments and returns an empty attrset.
However, it is a valid module.
Let us add to this to make it a bit more useful.

To define any values, the module system first has to know which ones are allowed.
This is done by declaring options that specify which values can be set and used elsewhere.
Options are declared by adding an attribute under the top-level `options` attribute.
The most general way to declare an option is using `lib.mkOption`.
This is done by declaring *options* that specify which attributes can be set and used elsewhere.

## Declaring options

Options are declared under the top-level `options` attribute with [`lib.mkOption`](https://nixos.org/manual/nixpkgs/stable/#function-library-lib.options.mkOption).

```{literalinclude} options.nix
:language: nix
:caption: options.nix
```

While many attributes for customizing options are available,
the most important one is `type`,
which specifies which values are valid for an option.
There are several types available under [`lib.types`][option-types-basic] in the Nixpkgs library.
:::{note}
The `lib` argument is passed automatically by the module system.
This makes [Nixpkgs library functions](https://nixos.org/manual/nixpkgs/stable/#chap-functions) available in each module's function body.

As you can see, we have declared an option `name`.
We have specificied that the `name` option will be of type `str`, so the module system will expect a string when we set a value.
The ellipsis `...` is necessary because the module system can pass arbitrary arguments to modules.

You may have noticed that we also changed the function arguments.
Now the module is a function which takes *at least* one argument, `lib`,
and may accept other arguments (expressed by the ellipsis `...`).
This will make Nixpkgs library functions available within the function body.
We needed this to get access to `mkOption` and `types`.
:::

:::{note}
The ellipsis `...` is necessary because arbitrary arguments can be passed to modules.
Every module should have this.
The attribute `type` in the argument to `lib.mkOption` specifies which values are valid for an option.
There are several types available under [`lib.types`](https://nixos.org/manual/nixos/stable/#sec-option-types-basic).

The `lib` argument is passed automatically by the module system.
It is absolutely vital for modules that have option declarations, as you will need `lib` for defining options and their types.
It is one of several arguments that are automatically provided by the module system.
The full list of arguments is discussed later.
:::
Here we have declared an option `name` of type `str`:
The module system will expect a string when a value is defined.

Now that we have declared an option, we would naturally want to give it a value.
Options can be set or *defined* using another top-level attribute, `config`.

## Defining values

Options are set or *defined* under the top-level `config` attribute:

```{literalinclude} config.nix
:language: nix
:caption: config.nix
```

Previously, in our option declaration, we created an option, `name`, with a string type.
In our option declaration, we created an option `name` with a string type.
Here, in our option definition, we have set that same option to a string.

:::{note}
`options` and `config` and have formal names —
that is ***option declarations*** and ***option definitions*** respectively.
The rest of these lessons will use them interchangeably.
:::
Option declarations and option definitions don't need to be in the same file.
Which modules will contribute to the resulting attribute set is specified when setting up module system evaluation.

:::{note}
Our option declarations and option definitions do not need to exist in the same file.
When we evaluate our modules, we can simply include both files.
As long as every definition has a declaration, we can successfully evaluate our modules.
If there is an option definition that has not been declared, the module system will throw an error.
:::
## Evaluating modules

Modules are evaluated by [`lib.evalModules`](https://nixos.org/manual/nixpkgs/stable/#module-system-lib-evalModules) from the Nixpkgs library.
It takes an attribute set as an argument, where the `modules` attribute is a list of modules to merge and evaluate.

Now that we have our declaration and definition, how do we evaluate them?
There is a function provided by the Nixpkgs library, `evalModules`.
It takes an attrset as an argument and one of the attributes is `modules` which is a list of modules you want to merge and evaluate.
The output of `evalModules` is a rather large attrset with a information about all the modules.
For now, the attribute we care about is `config` which is where the final configuration values appear.
The output of `evalModules` contains information about all evaluated modules, and the final values appear in the attribute `config`.

```{literalinclude} eval.nix
```{literalinclude} default.nix
:language: nix
:caption: eval.nix
:caption: default.nix
```

We can create a helper script to parse and evaluate our `eval.nix` file and print the output in a nice format.
Here's a helper script to parse and evaluate our `default.nix` file with [`nix-instantiate --eval`](https://nixos.org/manual/nix/stable/command-ref/nix-instantiate) and print the output as JSON:

```{literalinclude} run.sh
```{literalinclude} eval.sh
:language: bash
:caption: run.sh
:caption: eval.sh
```

If you execute the run file (`./run.sh`), you should see an output that matches what we have configured.
As long as every definition has a corresponding declaration, evaluation will be successful.
If there is an option definition that has not been declared, or the defined value has the wrong type, the module system will throw an error.

Running the script (`./eval.sh`) should show an output that matches what we have configured:

```{code-block}
{
"name": "Boaty McBoatface"
}
```

[option-types-basic]: https://nixos.org/manual/nixos/stable/#sec-option-types-basic

3 changes: 0 additions & 3 deletions source/tutorials/module-system/a-basic-module/run.sh

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@

Or: *Wrapping the world in modules*

Much of the power in Nixpkgs and NixOS comes from the module system.
It provides mechanisms for conveniently declaring and automatically merging interdependent attribute sets that follow dynamic type constraints, making it easy to express modular configurations.
In this tutorial you will follow an extensive demonstration of how to wrap an existing API with Nix modules.

## Overview

Expand All @@ -14,42 +13,22 @@ It may help playing it alongside this tutorial to better keep track of changes t

### What will you learn?

In this tutorial you'll learn
- what a module is
- how to define one
- what options are
- how to declare them
- how to express dependencies between modules

and follow an extensive demonstration of how to wrap an existing API with Nix modules.

Concretely, you'll write modules to interact with the [Google Maps API](https://developers.google.com/maps/documentation/maps-static), declaring options which represent map geometry, location pins, and more.
You'll write modules to interact with the [Google Maps API](https://developers.google.com/maps/documentation/maps-static), declaring module options which represent map geometry, location pins, and more.

During the tutorial, you will first write some *incorrect* configurations, creating opportunities to discuss the resulting error messages and how to resolve them, particularly when discussing type checking.

### What do you need?

- Familiarity with data types and general programming concepts
- A {ref}`Nix installation <install-nix>` to run the examples
- Intermediate proficiency in reading and writing the Nix language

You will use two helper scripts for this exercise.
Download {download}`map.sh <files/map.sh>` and {download}`geocode.sh <files/geocode.sh>` to your working directory.

:::{warning}
To run the examples in this tutorial, you will need a [Google API key](https://developers.google.com/maps/documentation/maps-static/start#before-you-begin) in `$XDG_DATA_HOME/google-api/key`.
:::

### How long will it take?

This is a very long tutorial.
Prepare for at least 3 hours of work.

## The empty module

We have to start somewhere.
The simplest module is just a function that takes any attributes and returns an empty attribute set.

Write the following into a file called `default.nix`:

```{code-block} nix
Expand All @@ -60,10 +39,9 @@ Write the following into a file called `default.nix`:
}
```

## Module arguments
## Declaring options

We will need some helper functions, which will come from the Nixpkgs library.
Start by changing the first line in `default.nix`:
We will need some helper functions, which will come from the [Nixpkgs library](Nixpkgs library ), which is passed by the module system as `lib`:

```{code-block} diff
:caption: default.nix
Expand All @@ -74,25 +52,7 @@ Start by changing the first line in `default.nix`:
}
```

Now the module is a function which takes *at least* one argument, called `lib`, and may accept other arguments (expressed by the ellipsis `...`).

This will make Nixpkgs library functions available within the function body.
The `lib` argument is passed automatically by the module system.

:::{note}
The ellipsis `...` is necessary because arbitrary arguments can be passed to modules.
:::

## Declaring options

To set any values, the module system first has to know which ones are allowed.

This is done by declaring *options* that specify which values can be set and used elsewhere.
Options are declared by adding an attribute under the top-level `options` attribute, using `lib.mkOption`.

In this section, you will define the `scripts.output` option.

Change `default.nix` to include the following declaration:
Using [`lib.mkOption`](https://nixos.org/manual/nixpkgs/stable/#function-library-lib.options.mkOption), declare the `scripts.output` option to have the type `lines`:

```{code-block} diff
:caption: default.nix
Expand All @@ -107,10 +67,7 @@ Change `default.nix` to include the following declaration:
}
```

While many attributes for customizing options are available, the most important one is `type`, which specifies which values are valid for an option.
There are several types available under [`lib.types`](https://nixos.org/manual/nixos/stable/#sec-option-types-basic) in the Nixpkgs library.

You have just declared `scripts.output` with the `lines` type, which specifies that the only valid values are strings, and that multiple definitions should be joined with newlines.
The `lines` type means that the only valid values are strings, and that multiple definitions should be joined with newlines.

:::{note}
The name and attribute path of the option is arbitrary.
Expand All @@ -119,12 +76,12 @@ Here we use `scripts`, because we will add another script later, and call this o

## Evaluating modules

Write a new file, `eval.nix`, which you will use to evaluate `default.nix`:
Write a new file `eval.nix` to call [`lib.evalModules`](https://nixos.org/manual/nixpkgs/unstable/#module-system-lib-evalModules) and evaluate the module in `default.nix`:

```{code-block} nix
:caption: eval.nix
let
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-22.11";
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-23.11";
pkgs = import nixpkgs { config = {}; overlays = []; };
in
pkgs.lib.evalModules {
Expand All @@ -134,9 +91,6 @@ pkgs.lib.evalModules {
}
```

[`evalModules`](https://nixos.org/manual/nixpkgs/unstable/#module-system-lib-evalModules) is the function that evaluates modules, applies type checking, and merges values into the final attribute set.
It expects a `modules` attribute whose value is a list, where each element can be a path to a module or an expression that follows the [module schema](https://nixos.org/manual/nixos/stable/#sec-writing-modules).

Run the following command:

:::{warning}
Expand All @@ -147,9 +101,11 @@ This will result in an error.
nix-instantiate --eval eval.nix -A config.scripts.output
```

:::{dropdown} Detailed explanation
[`nix-instantiate --eval`](https://nixos.org/manual/nix/stable/command-ref/nix-instantiate) parses and evaluates the Nix file at the specified path, and prints the result.
`evalModules` produces an attribute set where the final configuration values appear in the `config` attribute.
Therefore we evaluate the Nix expression in `eval.nix` at the [attribute path](https://nixos.org/manual/nix/stable/language/operators#attribute-selection) `config.scripts.output`.
:::

The error message indicates that the `scripts.output` option is used but not defined: a value must be set for the option before accessing it.
You will do this in the next steps.
Expand Down
29 changes: 27 additions & 2 deletions source/tutorials/module-system/index.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,34 @@
# Module system

Learn about the module system.
Much of the power in Nixpkgs and NixOS comes from the module system.

The module system is a Nix language library that enables you to
- Declare one attribute set using many separate Nix expressions.
- Impose dynamic type constraints on values in that attribute set.
- Define values for the same attribute in different Nix expressions and merge these values automatically according to their type.

These Nix expressions are called modules and must have a particular structure.

In this tutorial series you'll learn
- What a module is and how to create one.
- What options are and how to declare them.
- How to express dependencies between modules.

## What do you need?

- Familiarity with data types and general programming concepts
- A {ref}`Nix installation <install-nix>` to run the examples
- Intermediate proficiency in reading and writing the {ref}`Nix language <reading-nix-language>`

## How long will it take?

This is a very long tutorial.
Prepare for at least 3 hours of work.

```{toctree}
:maxdepth: 1
:caption: Lessons
:numbered:
a-basic-module/index.md
module-system.md
deep-dive.md
```

0 comments on commit cfe0f78

Please sign in to comment.