Nixpkgs documentation additions for 19.03 #54659

matthewbauer · 2019-01-27T03:36:09Z

Motivation for this change

These are a bunch of documentation work I have done. They each have an open issue associated with them that I am hoping to close for 19.03.

I put these together in a rush, so proofreading would be great if you are able to! There may also be some things, especially functions, that I have left out.

Apparently this is a little slower but much safer & not prone to potential argument errors.

Fixes NixOS#32439.

Fixes NixOS#34857.

Fixes NixOS#25507.

Mic92 · 2019-01-27T07:24:55Z

doc/functions/fetchers.xml

+
+ <para>
+   <function>fetchpatch</function> works very similarly to
+   <function>fetchurl</function> with the same arguments expected.


Suggested change

<function>fetchurl</function> with the same arguments expected.

<function>fetchurl</function> with the same arguments expected.

It expects patch files as a source and and performs normalization

on them before computing the checksum. For example it will remove

comments or other unstable parts that are sometimes added

by version control systems and can change over time.

danbst · 2019-01-27T07:25:22Z

pkgs/top-level/all-packages.nix

@@ -261,78 +261,15 @@ in

  fetchCrate = callPackage ../build-support/rust/fetchcrate.nix { };

-  fetchFromGitHub = {


Mic92 · 2019-01-27T07:30:13Z

doc/functions/fetchers.xml

+     <listitem>
+      <para>
+       Used with Git. Expects <literal>url</literal> to a Git repo,
+       <literal>rev</literal>, and <literal>sha256</literal>.


Suggested change

<literal>rev</literal>, and <literal>sha256</literal>.

<literal>rev</literal>, and <literal>sha256</literal>.

<literal>rev</literal> in this case can be full the git commit id (SHA1 hash)

or a tag name like <literal>refs/tags/v1.0</literal>.

danbst · 2019-01-27T07:31:37Z

doc/functions/trivial-builders.xml

+       <literal>env</literal>, and <literal>buildCommand</literal>.
+       <literal>name</literal> is just the name that Nix will use to
+       refer to the derivation. <literal>env</literal> is an attribute
+       set specifying environment variables that will be set for this


probably mention that env is propagated to underlying mkDerivation?

Mic92 · 2019-01-27T07:32:30Z

doc/functions/fetchers.xml

+        GitHub HTML page as
+        <literal>owner</literal>/<literal>repo</literal>.
+        <literal>rev</literal> corresponds to the Git commit hash or
+        tag that will be downloaded from Git. Finally,


Suggested change

tag that will be downloaded from Git. Finally,

tag (e.g <literal>v1.0</literal>) that will be downloaded from Git. Finally,

danbst · 2019-01-27T07:32:35Z

doc/functions/trivial-builders.xml

+       you will need to create <literal>$out</literal> for Nix to
+       register the command as successful.
+    </para>
+   </listitem>


example is missing

Mic92 · 2019-01-27T07:34:48Z

doc/stdenv.xml

+       to the shebang interpreter. A shebang interpreter is the first
+       commented line of a script telling the operating system
+       what to use to run this script. In Nix, we want an exact path
+       to that interpreter to be used. This often replcaes


Suggested change

to that interpreter to be used. This often replcaes

to that interpreter to be used. This often replaces

Mic92 · 2019-01-27T07:37:35Z

doc/stdenv.xml

+       This setup hook patches installed scripts to use the full path
+       to the shebang interpreter. A shebang interpreter is the first
+       commented line of a script telling the operating system
+       what to use to run this script. In Nix, we want an exact path


Suggested change

what to use to run this script. In Nix, we want an exact path

what to use to run this script (e.g <literal>#!/bin/bash</literal>). In Nix, we want an exact path

Suggested change

what to use to run this script. In Nix, we want an exact path

which program will run the script (e.g <literal>#!/bin/bash</literal>). In Nix, we want an exact path

Mic92 · 2019-01-27T07:38:27Z

doc/stdenv.xml

+       This verifies that no references are left from the install
+       binaries to the directory used to build those binaries. This
+       ensures that the binaries do not need things outside the Nix
+       store. This currently Linux only.


Suggested change

store. This currently Linux only.

store. This is currently supported in Linux only.

danbst · 2019-01-27T07:39:55Z

doc/functions/trivial-builders.xml

+  </varlistentry>
+  <varlistentry>
+   <term>
+    <literal>writeTextFile</literal>


maybe mention writeText, writeTextDir, writeScript, writeScriptBin and say those are specialization of makeTextFile.

I see those have doc and examples attached in trivial-builders.nix, maybe it should be possible to use some autodocs here?

I missed this thread - sorry!

@danbst It should be possible to use nixdoc for this, but the current manual structure only includes doc generation for functions in the lib folder. It would be nice to do it for functions in the package set (such as the trivial builders), too. Ping me here or on IRC if you want some pointers!

roberth · 2019-01-27T11:25:33Z

doc/functions/fetchers.xml

+
+ <para>
+   When using Nix, you will frequently need to download source code
+   and other file from the internet. Nixpkgs comes with a few helper


Suggested change

and other file from the internet. Nixpkgs comes with a few helper

and other files from the internet. Nixpkgs comes with a few helper

roberth · 2019-01-27T11:26:05Z

doc/functions/fetchers.xml

+   When using Nix, you will frequently need to download source code
+   and other file from the internet. Nixpkgs comes with a few helper
+   functions that allow you to fetch fixed-output derivations in
+   structured way.


Suggested change

structured way.

a structured way.

roberth · 2019-01-27T11:33:22Z

doc/functions/trivial-builders.xml

+ <title>Trivial builders</title>
+
+ <para>
+   There are a couple of functions provide in Nixpkgs that help with


Suggested change

There are a couple of functions provide in Nixpkgs that help with

Nixpkgs provides a couple of functions that help with

roberth · 2019-01-27T11:34:16Z

doc/functions/trivial-builders.xml

+   There are a couple of functions provide in Nixpkgs that help with
+   building derivations. The most important one,
+   <function>stdenv.mkDerivation</function>, has already been
+   documented above. These wrap


Suggested change

documented above. These wrap

documented above. The following functions wrap

roberth · 2019-01-27T11:36:02Z

doc/functions/trivial-builders.xml

+       This takes three arguments, <literal>name</literal>,
+       <literal>env</literal>, and <literal>buildCommand</literal>.
+       <literal>name</literal> is just the name that Nix will use to
+       refer to the derivation. <literal>env</literal> is an attribute


the name that Nix will append to the store path.

roberth · 2019-01-27T11:47:23Z

doc/stdenv.xml

+     <listitem>
+       <para>
+       This runs the strip command on installed binaries and
+       libraries. This removed things like debug symbols when they are


Suggested change

libraries. This removed things like debug symbols when they are

libraries. This removes unnecessary information like debug symbols when they are

roberth · 2019-01-27T11:47:35Z

doc/stdenv.xml

+       <para>
+       This runs the strip command on installed binaries and
+       libraries. This removed things like debug symbols when they are
+       not needed. This also helps to reduce installed size of


Suggested change

not needed. This also helps to reduce installed size of

not needed. This also helps to reduce the installed size of

roberth · 2019-01-27T11:49:59Z

doc/stdenv.xml

+       This setup hook patches installed scripts to use the full path
+       to the shebang interpreter. A shebang interpreter is the first
+       commented line of a script telling the operating system
+       what to use to run this script. In Nix, we want an exact path


Suggested change

what to use to run this script. In Nix, we want an exact path

which program will run the script (e.g <literal>#!/bin/bash</literal>). In Nix, we want an exact path

roberth · 2019-01-27T11:52:19Z

doc/stdenv.xml

+
+  <para>
+   Here are some more packages that provide a setup hook. Since the
+   mechanism is modular, this probably isn't an exhaustive list. Then


Suggested change

mechanism is modular, this probably isn't an exhaustive list. Then

list of hooks is extensible, this is not an exhaustive list. Then

roberth · 2019-01-27T11:53:08Z

doc/stdenv.xml

+   Here are some more packages that provide a setup hook. Since the
+   mechanism is modular, this probably isn't an exhaustive list. Then
+   again, since the mechanism is only to be used as a last resort, it
+   might be.


Suggested change

might be.

might cover most uses.

Mostly taken from requested changes exactly as recommended.

matthewbauer · 2019-01-27T16:58:37Z

Thanks for the review comments! I have integrated all of them mentioned so far.

matthewbauer requested review from grahamc and roberth January 27, 2019 03:36

GrahamcOfBorg added 6.topic: fetch Fetchers (e.g. fetchgit, fetchsvn, ...) 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: documentation This PR adds or changes documentation labels Jan 27, 2019

matthewbauer added 5 commits January 26, 2019 22:47

all-packages: move fetch* to pkgs/build-support/

d16e0f8

nixos/manual: use default bs value

6604240

Apparently this is a little slower but much safer & not prone to potential argument errors.

nixpkgs/manual: document fetcher functions

17ec7f3

Fixes NixOS#32439.

nixpkgs/manual: document default setup hooks

adb717a

Fixes NixOS#34857.

nixpkgs/manual: add trivial builders section

498a242

Fixes NixOS#25507.

matthewbauer force-pushed the doc-fixes branch from 5e8c0d3 to 498a242 Compare January 27, 2019 03:49

GrahamcOfBorg added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 labels Jan 27, 2019

matthewbauer changed the title ~~Nixpkgs documentation fixes for 19.03~~ Nixpkgs documentation additions for 19.03 Jan 27, 2019

Mic92 reviewed Jan 27, 2019

View reviewed changes

danbst reviewed Jan 27, 2019

View reviewed changes

Mic92 reviewed Jan 27, 2019

View reviewed changes

danbst reviewed Jan 27, 2019

View reviewed changes

Mic92 reviewed Jan 27, 2019

View reviewed changes

danbst reviewed Jan 27, 2019

View reviewed changes

Mic92 reviewed Jan 27, 2019

View reviewed changes

danbst reviewed Jan 27, 2019

View reviewed changes

roberth reviewed Jan 27, 2019

View reviewed changes

nixpkgs/manual: address review comments

d7b62cb

Mostly taken from requested changes exactly as recommended.

nixpkgs/manual: add one more fix for a missed review

a376d62

matthewbauer merged commit 14ecdbe into NixOS:master Jan 27, 2019

matthewbauer deleted the doc-fixes branch February 22, 2019 04:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nixpkgs documentation additions for 19.03 #54659

Nixpkgs documentation additions for 19.03 #54659

matthewbauer commented Jan 27, 2019

Mic92 Jan 27, 2019 •

edited

Loading

danbst Jan 27, 2019

Mic92 Jan 27, 2019

danbst Jan 27, 2019

Mic92 Jan 27, 2019

danbst Jan 27, 2019

Mic92 Jan 27, 2019

Mic92 Jan 27, 2019 •

edited

Loading

roberth Jan 27, 2019

Mic92 Jan 27, 2019 •

edited

Loading

danbst Jan 27, 2019

Mic92 Jan 27, 2019

tazjin Feb 1, 2019

roberth Jan 27, 2019

roberth Jan 27, 2019

roberth Jan 27, 2019

roberth Jan 27, 2019

roberth Jan 27, 2019

roberth Jan 27, 2019

roberth Jan 27, 2019

roberth Jan 27, 2019

roberth Jan 27, 2019

roberth Jan 27, 2019

matthewbauer commented Jan 27, 2019

-   <function>fetchurl</function> with the same arguments expected.
+   <function>fetchurl</function> with the same arguments expected.
+   It expects patch files as a source and and performs normalization
+   on them before computing the checksum. For example it will remove
+   comments or other unstable parts that are sometimes added
+   by version control systems and can change over time.

		@@ -261,78 +261,15 @@ in

		fetchCrate = callPackage ../build-support/rust/fetchcrate.nix { };

		fetchFromGitHub = {

	tag that will be downloaded from Git. Finally,
	tag (e.g <literal>v1.0</literal>) that will be downloaded from Git. Finally,

	to that interpreter to be used. This often replcaes
	to that interpreter to be used. This often replaces

	what to use to run this script. In Nix, we want an exact path
	what to use to run this script (e.g <literal>#!/bin/bash</literal>). In Nix, we want an exact path

	what to use to run this script. In Nix, we want an exact path
	which program will run the script (e.g <literal>#!/bin/bash</literal>). In Nix, we want an exact path

	store. This currently Linux only.
	store. This is currently supported in Linux only.

	and other file from the internet. Nixpkgs comes with a few helper
	and other files from the internet. Nixpkgs comes with a few helper

	There are a couple of functions provide in Nixpkgs that help with
	Nixpkgs provides a couple of functions that help with

	documented above. These wrap
	documented above. The following functions wrap

	libraries. This removed things like debug symbols when they are
	libraries. This removes unnecessary information like debug symbols when they are

	not needed. This also helps to reduce installed size of
	not needed. This also helps to reduce the installed size of

	mechanism is modular, this probably isn't an exhaustive list. Then
	list of hooks is extensible, this is not an exhaustive list. Then

Nixpkgs documentation additions for 19.03 #54659

Nixpkgs documentation additions for 19.03 #54659

Conversation

matthewbauer commented Jan 27, 2019

Motivation for this change

Mic92 Jan 27, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mic92 Jan 27, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mic92 Jan 27, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matthewbauer commented Jan 27, 2019

Mic92 Jan 27, 2019 •

edited

Loading

Mic92 Jan 27, 2019 •

edited

Loading

Mic92 Jan 27, 2019 •

edited

Loading