Skip to content
This repository was archived by the owner on Apr 29, 2024. It is now read-only.

Commit 0402b27

Browse files
thoughtpolicethoughtpolice
committed
docs: overhaul
Signed-off-by: Austin Seipp <aseipp@pobox.com> Change-Id: I55cc7463fbb2774b673ea53c5cfa7120
1 parent 1d91f14 commit 0402b27

File tree

11 files changed

+280
-30
lines changed

11 files changed

+280
-30
lines changed

‎docs/README.md‎

Lines changed: 5 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,8 @@
11
# Documentation
22

3-
This contains all the needed documentation to (hopefully) help you explicate and
4-
understand what Nix is, how to use it wisely, how to play with PostgreSQL, and
5-
probably other stuff to help educate you.
3+
This directory contains most of the "runbooks" and documentation on how to use
4+
this repository.
65

7-
## Table of Contents
8-
9-
- **[00 &mdash; Starting Line](./00-START-HERE.md)** &mdash; we'll install Nix
10-
and play around with it a little.
11-
- **[01 &mdash; Use this Repository](./01-USE-THIS-REPO.md)** &mdash; We'll
12-
build a PostreSQL distribution, and also use the magical `nix develop`
13-
- **[02 &mdash; Install `direnv`](./02-INSTALL-DIRENV.md)** &mdash; Using
14-
`direnv`, you can activate your development shell "transparently" on demand.
6+
You probably want to start with the [starting guide](./start-here.md). Then,
7+
learn how to play with `postgres` in the [build guide](./build-postgres.md).
8+
After that, you can probe around a bit.

‎docs/adding-tests.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
There are basically two types of tests you can add:
2+
3+
- pgTAP based tests, and
4+
- Migration tests.
5+
6+
In both cases, a number of extensions may be installed into the database for
7+
use; you can see those in both [postgresql.conf.in](../tests/postgresql.conf.in)
8+
and [prime.sql](../tests/prime.sql) (extensions may be enabled in either place.)
9+
10+
## pgTAP tests
11+
12+
These are super easy: simply add `.sql` files to the
13+
[tests/smoke](./../tests/smoke/) directory, then:
14+
15+
```
16+
nix flake check
17+
```
18+
19+
These files are run using `pg_prove`; they pretty much behave exactly like how
20+
you expect; you can read
21+
[the pgTAP documentation](https://pgtap.org/documentation.html) for more.
22+
23+
For a good example of a pgTAP test as a pull request, check out
24+
[pull request #4](https://github.com/supabase/nix-postgres/pull/4/files).
25+
26+
## Migration tests
27+
28+
> **NOTE**: Currently, migration tests _do not happen in CI_. They can only be
29+
> run manually.
30+
31+
Migration tests are pretty simple in the sense they follow a very simple
32+
principle:
33+
34+
- You put data in the database
35+
- Run the migration procedure
36+
- It should probably not fail
37+
38+
Step 1 and 2 are easy, and for various reasons (e.g. mistakes from upstream
39+
extension authors), step 3 isn't guaranteed, so that's what the whole idea is
40+
designed to test.
41+
42+
To add data into the database, modify the
43+
[data.sql](../tests/migrations/data.sql) script and add whatever you want into
44+
it. This script gets loaded into the old version of the database at startup, and
45+
it's expected that the new version of the database can handle it.
46+
47+
To run the `migration-test` tool, check out the documentation on
48+
[migration-tests](./migration-tests.md).

docs/01-USE-THIS-REPO.md renamed to docs/build-postgres.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -178,7 +178,3 @@ shell goes away, `just` will be removed from your `$PATH` as well.
178178

179179
There's an even easier way to do this that is completely transparent to you, as
180180
well...
181-
182-
## Next page
183-
184-
Next up: **[02 &mdash; Install `direnv`](./02-INSTALL-DIRENV.md)**

‎docs/migration-tests.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
Migration tests are run similar to running the client and server; see
2+
[more on that here](./start-client-server.md).
3+
4+
Instead, you use the following format to specify the upgrade:
5+
6+
```
7+
nix run .#migration-test <from> <to> [pg_dumpall|pg_upgrade]
8+
```
9+
10+
The arguments are:
11+
12+
- The version to upgrade from
13+
- The version to upgrade to
14+
- The upgrade mechanism: either `pg_dumpall` or `pg_upgrade`
15+
16+
## Specifying the version
17+
18+
The versions for upgrading can be one of two forms:
19+
20+
- A major version number, e.g. `14` or `15`
21+
- A path to `/nix/store`, which points to _any_ version of PostgreSQL, as long
22+
as it has the "expected" layout and is a postgresql install.
23+
24+
## Always use the latest version of the migration tool
25+
26+
Unlike the method for starting the client or server, you probably always want to
27+
use the latest version of the `migration-test` tool from the repository. This is
28+
because it can ensure forwards and backwards compatibility if necessary.
29+
30+
## Upgrading between arbitrary `/nix/store` versions
31+
32+
If you want to test migrations from arbitrary versions built by the repository,
33+
you can combine `nix build` and `nix run` to do so. You can use the syntax from
34+
the runbook on [running the server & client](./start-client-server.md) to refer
35+
to arbitrary git revisions.
36+
37+
For example, if you updated an extension in this repository, and you want to
38+
test a migration from PostgreSQL 14 to PostgreSQL 14 + (updated extension),
39+
using `pg_upgrade` &mdash; simply record the two git commits you want to
40+
compare, and you could do something like the following:
41+
42+
```
43+
OLD_GIT_VERSION=...
44+
NEW_GIT_VERSION=...
45+
46+
nix run github:supabase/nix-postgres#migration-test \
47+
$(nix build "github:supabase/nix-postgres/$OLD_GIT_VERSION#psql_14/bin") \
48+
$(nix build "github:supabase/nix-postgres/$NEW_GIT_VERSION#psql_14/bin") \
49+
pg_upgrade
50+
```

‎docs/new-major-postgres.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
PostgreSQL versions are managed in upstream nixpkgs.
2+
3+
See this example PR to add a new version of PostgreSQL; this version is for 16
4+
beta3, but any version is roughly the same. In short, you need to:
5+
6+
- Add a new version and hash
7+
- Possibly patch the source code for minor refactorings
8+
- In this example, an old patch had to be rewritten because a function was
9+
split into two different functions; the patch is functionally equivalent but
10+
textually different
11+
- Add the changes to `all-packages.nix`
12+
- Integrate inside the CI and get code review
13+
- Run `nix flake update` to get a new version, once it's ready
14+
15+
https://github.com/NixOS/nixpkgs/pull/249030
16+
17+
## Adding the major version to this repository
18+
19+
It isn't well abstracted, unfortunately. In short: look for the strings `14` and
20+
`15` under `flake.nix` and `tools/`. More specifically:
21+
22+
- Add `psql_XX` to `basePackages` in `flake.nix`
23+
- Ditto with `checks` in `flake.nix`
24+
- Modify the tools under `tools/` to understand the new major version
25+
- Make sure the CI is integrated under the GitHub Actions.
26+
27+
The third step and fourth steps are the most annoying, really. The first two are
28+
easy and by that point you can run `nix flake check` in order to test the build,
29+
at least.
30+
31+
## Other notes
32+
33+
See also issue [#6](https://github.com/supabase/nix-postgres/issues/6), which
34+
would make it possible to define PostgreSQL versions inside this repository.

‎docs/references.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
Nix references and other useful tools:
2+
3+
- **Zero to Nix**: Start here to get your feet wet with how Nix works, and how
4+
to use Nixpkgs: https://zero-to-nix.com/
5+
- `nix-installer`: My recommended way to install Nix
6+
- https://github.com/DeterminateSystems/nix-installer
7+
- Nix manual https://nixos.org/manual/nix/stable/
8+
- Useful primarily for option and command references
9+
- Flake schema reference https://nixos.wiki/wiki/Flakes
10+
- Useful to know what `flake.nix` is referring to
11+
- Example pull requests for this repo:
12+
- Adding smoke tests for an extension:
13+
https://github.com/supabase/nix-postgres/pull/2
14+
- Extension smoke tests, part 2:
15+
https://github.com/supabase/nix-postgres/pull/3
16+
- Adding an extension and a smoke test at once:
17+
https://github.com/supabase/nix-postgres/pull/4/files
18+
- Updating an extension to trunk:
19+
https://github.com/supabase/nix-postgres/pull/7
20+
- Updating an extension to the latest release:
21+
https://github.com/supabase/nix-postgres/pull/9
22+
- Contributing to [nixpkgs](https://github.com/nixos/nixpkgs)
23+
- Adding a PGRX-powered extension:
24+
https://github.com/NixOS/nixpkgs/pull/246803
25+
- Adding a normal extension: https://github.com/NixOS/nixpkgs/pull/249000
26+
- Adding new PostgreSQL versions: https://github.com/NixOS/nixpkgs/pull/249030
27+
- NixOS Discourse: https://discourse.nixos.org/
28+
- Useful for community feedback, guidance, and help
29+
- `nix-update`: https://github.com/Mic92/nix-update
30+
- Used in this repository to help update extensions
31+
- pgTAP for testing: https://pgtap.org/documentation.html

‎docs/start-client-server.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
If you want to run a postgres server, just do this from the root of the
2+
repository:
3+
4+
```
5+
nix run .#start-server 14
6+
```
7+
8+
Replace the `14` with a `15`, and you'll be using a different version.
9+
10+
This always uses port 5432.
11+
12+
Actually, you don't even need the repository. You can do this from arbitrary
13+
directories, if the left-hand side of the hash character (`.` in this case) is a
14+
valid "flake reference":
15+
16+
```
17+
# from any arbitrary directory
18+
nix run github:supabase/nix-postgres#start-server 14
19+
```
20+
21+
## Arbitrary versions at arbitrary git revisions
22+
23+
Let's say you want to use a PostgreSQL build from a specific version of the
24+
repository. You can change the syntax of the above to use _any_ version of the
25+
repository, at any time, by adding the commit hash after the repository name:
26+
27+
```
28+
# use postgresql 15 build at commit a9989d4800dd6038827afed27456f19ba4b2ae0a
29+
nix run github:supabase/nix-postgres/a9989d4800dd6038827afed27456f19ba4b2ae0a#start-server 15
30+
```
31+
32+
## Running the client
33+
34+
All of the same rules apply, but try using `start-client` on the right-hand side
35+
of the hash character, instead. For example:
36+
37+
```
38+
nix run github:supabase/nix-postgres#start-server 14 &
39+
sleep 5
40+
nix run github:supabase/nix-postgres#start-client 15
41+
```

docs/00-START-HERE.md renamed to docs/start-here.md

Lines changed: 0 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -70,12 +70,3 @@ examples:
7070
- **A.1**: `nix profile install nixpkgs#ripgrep`
7171
- **A.2**: `nix profile install nixpkgs#bat`
7272
- **A.3**: And yes, you also have exa, fd, hyperfine, and more!
73-
74-
<!-- break bulletpoints -->
75-
76-
- **Q**: What does '`nixpkgs`' mean? And how can I find out more?
77-
- **A**: Please keep reading...
78-
79-
## Next page
80-
81-
Next up: **[01 &mdash; Use this repository](./01-USE-THIS-REPO.md)**

‎docs/update-extension.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
Extensions are managed in two places: upstream, and this repository. Therefore
2+
there are two update strategies available.
3+
4+
## Updating extensions in this repository
5+
6+
Assuming that you run `nix develop` to get a development shell, there is a tool
7+
named `nix-update` that is available:
8+
9+
```
10+
austin@GANON:~/work/nix-postgres$ which nix-update
11+
/nix/store/2jyq6h0ln3f5vlgz2had80l2crdkjmdy-nix-update-0.19.2/bin/nix-update
12+
```
13+
14+
Run something like this to update the extension `pg_foobar`:
15+
16+
```bash
17+
nix-update --flake psql_15/exts/pg_foobar
18+
git commit -asm "pg_foobar: update to latest release"
19+
```
20+
21+
It doesn't matter if you use `psql_14` or `psql_15` here, because `nix-update`
22+
will look at the _file_ that extension is defined in, in order to update the
23+
source code.
24+
25+
## Updating extensions upstream
26+
27+
You can use the same tool, `nix-update`, to do this. If you're sitting in the
28+
root of the nixpkgs repository, try this:
29+
30+
```
31+
nix run nixpkgs#nix-update -- postgresqlPackages.pg_foobar
32+
git commit -asm "pg_foobar: update to latest release"
33+
```
34+
35+
Because the tool may not be in your shell by default, we use `nix run` to run it
36+
for us.
37+
38+
The full list of available names to substitute for `pg_foobar` is available in
39+
the file
40+
[pkgs/servers/sql/postgresql/packages.nix](https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/sql/postgresql/packages.nix)
41+
42+
### Updating the Nixpkgs snapshot
43+
44+
Now that your change is merged upstream, you need to update the version of
45+
`nixpkgs` used in this repository:
46+
47+
- Check the `nixpkgs-unstable` branch:
48+
https://github.com/nixos/nixpkgs/tree/nixpkgs-unstable
49+
- Wait until your commit is fast-forwarded and part of that branch
50+
- Run `nix flake update`
51+
52+
## Release tags versus latest trunk
53+
54+
By default, `nix-update` will update an expression to the latest tagged release.
55+
No extra arguments are necessary. You can specify an exact release tag using the
56+
`--version=<xyz>` flag. Using the syntax `--version=branch` means "update to the
57+
latest version on the default branch."
58+
59+
## Example PRs
60+
61+
- https://github.com/supabase/nix-postgres/pull/9 updates `pg_net` to the latest
62+
release
63+
- https://github.com/supabase/nix-postgres/pull/7 updates `pg_hashids` to the
64+
latest `trunk` tip
65+
66+
## Other notes
67+
68+
See issue [#5](https://github.com/supabase/nix-postgres/issues/5) for more
69+
information about extension management.

docs/02-INSTALL-DIRENV.md renamed to docs/use-direnv.md

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,3 @@
1-
# 02 &mdash; Install `direnv`
2-
31
Have you ever used a tool like `pip`'s `bin/activate` script, or `rbenv`? These
42
tools populate your shell environment with the right tools and scripts and
53
dependencies (e.g. `PYTHONPATH`) to run your software.
@@ -150,7 +148,3 @@ This is the power of `direnv`: your projects always, on demand, will have the
150148
right tools configured and available, no matter if you last worked on them a day
151149
ago or a year ago, or it was done by your teammate, or you have a brand new
152150
computer that you've never programmed on.
153-
154-
## Next page
155-
156-
Not Invented Here.

0 commit comments

Comments
 (0)