diff --git a/nix/docs/README.md b/nix/docs/README.md index 66d63ef4c..0d7b080ee 100644 --- a/nix/docs/README.md +++ b/nix/docs/README.md @@ -11,6 +11,8 @@ learn how to play with `postgres` in the [build guide](./build-postgres.md). ## Development - **[Nix tree structure](./nix-directory-structure.md)** - Overview of the Nix directory structure +- **[Flake-Parts Architecture](./flake-parts-architecture.md)** - Deep dive into the flake-parts module system +- **[Flake-Parts and nixpkgs lib](./flake-parts-nixpkgs-lib.md)** - How flake-parts uses nixpkgs lib foundations - **[Development Workflow](./development-workflow.md)** - Complete development and testing workflow - **[Build PostgreSQL](./build-postgres.md)** - Building PostgreSQL from source - **[Receipt Files](./receipt-files.md)** - Understanding build receipts diff --git a/nix/docs/adding-new-package.md b/nix/docs/adding-new-package.md index 83d546d21..7d9fbdae9 100644 --- a/nix/docs/adding-new-package.md +++ b/nix/docs/adding-new-package.md @@ -1,5 +1,10 @@ # Adding a new extension package +!!! tip "Understanding the Module System" + To better understand how packages are organized and how `ourExtensions` works with flake-parts, see: + + - **[Flake-Parts Architecture](./flake-parts-architecture.md)** - Module structure overview + - **[Flake-Parts and nixpkgs lib](./flake-parts-nixpkgs-lib.md)** - Extension composition patterns ## Pre-packaging steps 1. Make sure you have nix installed [Nix installer](https://github.com/DeterminateSystems/nix-installer) diff --git a/nix/docs/flake-parts-architecture.md b/nix/docs/flake-parts-architecture.md new file mode 100644 index 000000000..e3d4dffc2 --- /dev/null +++ b/nix/docs/flake-parts-architecture.md @@ -0,0 +1,652 @@ +# Flake-Parts Architecture + +This document explains how this repository uses [flake-parts](https://flake.parts/) to organize its Nix flake into maintainable, composable modules. + +!!! info "Deep Dive into nixpkgs lib" + For a detailed explanation of how flake-parts leverages nixpkgs lib functions and the module system, see **[Flake-Parts and nixpkgs lib](./flake-parts-nixpkgs-lib.md)**. + +## Overview + +Flake-parts is a module system for Nix flakes that allows splitting a monolithic `flake.nix` into specialized modules. Instead of one large file with all outputs, we have multiple focused modules that each handle a specific concern. + +## Why Flake-Parts? + +Traditional flakes can become unwieldy as they grow: + +```nix +{ + outputs = { nixpkgs, ... }: { + packages.x86_64-linux = { ... }; + packages.aarch64-linux = { ... }; + packages.aarch64-darwin = { ... }; + devShells.x86_64-linux = { ... }; + devShells.aarch64-linux = { ... }; + # ... hundreds of lines of repetitive code + }; +} +``` + +Flake-parts solves this by: + +1. **Per-system evaluation**: Write code once, evaluate it for each system automatically +2. **Module composition**: Split concerns into separate files +3. **Type safety**: Define typed configuration options +4. **Module system**: Import third-party modules (treefmt, git-hooks, etc.) + +## Entry Point + +The root `flake.nix` is minimal and delegates to modules: + +```nix +{ + outputs = { flake-utils, ... }@inputs: + inputs.flake-parts.lib.mkFlake { inherit inputs; } (_: { + systems = [ + "x86_64-linux" + "aarch64-linux" + "aarch64-darwin" + ]; + imports = [ + nix/apps.nix + nix/checks.nix + nix/config.nix + nix/devShells.nix + nix/fmt.nix + nix/hooks.nix + nix/nixpkgs.nix + nix/packages + nix/overlays + ]; + }); +} +``` + +**Key function**: `mkFlake` accepts inputs and a configuration. Each imported module can define outputs. + +## Module Scopes + +Flake-parts modules operate at two scopes: + +### perSystem Scope + +Defines outputs for each system (x86_64-linux, aarch64-darwin, etc.): + +```nix +{ ... }: +{ + perSystem = { self', pkgs, system, lib, config, inputs', ... }: + { + # System-specific outputs + packages = { ... }; + apps = { ... }; + devShells = { ... }; + checks = { ... }; + }; +} +``` + +**Available arguments**: + +| Argument | Description | +|----------|-------------| +| `self'` | Outputs from the current system (e.g., `self'.packages.foo`) | +| `pkgs` | nixpkgs for current system | +| `system` | Current system string (e.g., `"x86_64-linux"`) | +| `lib` | nixpkgs library functions | +| `config` | Module configuration (from `config.nix`) | +| `inputs'` | Flake inputs for current system | + +### Flake Scope + +Defines system-independent, flake-wide configuration: + +```nix +{ lib, ... }: +{ + flake = { + options = { ... }; # Module options + config = { ... }; # Configuration values + overlays = { ... }; # System-independent overlays + }; +} +``` + +## Module Breakdown + +### Infrastructure Modules + +#### nix/nixpkgs.nix + +**Purpose**: Configure the `pkgs` argument for all perSystem modules. + +```nix +{ self, inputs, ... }: +{ + perSystem = { system, ... }: { + _module.args.pkgs = import inputs.nixpkgs { + inherit system; + config.allowUnfree = true; + permittedInsecurePackages = [ "v8-9.7.106.18" ]; + overlays = [ + (import inputs.rust-overlay) + self.overlays.default + ]; + }; + }; +} +``` + +**Critical role**: Instantiates nixpkgs once per system with: +- Unfree packages enabled +- Custom overlays applied +- System-specific configuration + +This runs first to provide `pkgs` to all other modules. + +#### nix/config.nix + +**Purpose**: Define typed, flake-wide configuration options. + +Uses the NixOS module system for type-safe configuration: + +```nix +{ lib, ... }: +let + postgresqlDefaults = lib.types.submodule { + options = { + port = lib.mkOption { + type = lib.types.str; + default = "5435"; + }; + host = lib.mkOption { + type = lib.types.str; + default = "localhost"; + }; + superuser = lib.mkOption { + type = lib.types.str; + default = "supabase_admin"; + }; + }; + }; +in +{ + flake = { + options = { + supabase = lib.mkOption { type = postgresqlDefaults; }; + }; + config.supabase = { + defaults = { }; + supportedPostgresVersions = { + postgres = { + "15" = { version = "15.14"; hash = "sha256-..."; }; + "17" = { version = "17.6"; hash = "sha256-..."; }; + }; + }; + }; + }; +} +``` + +**Access pattern**: Other modules access via `self.supabase.defaults`. + +### Output Modules + +#### nix/packages/default.nix + +**Purpose**: Combine all packages from various sources. + +```nix +{ self, inputs, ... }: +{ + perSystem = { pkgs, self', lib, ... }: + { + packages = ( + { + # Individual hand-written packages + dbmate-tool = pkgs.callPackage ./dbmate-tool.nix { + inherit (self.supabase) defaults; + }; + start-server = pkgs-lib.makePostgresDevSetup { ... }; + } + // lib.filterAttrs (...) ( + # Generated PostgreSQL packages + pkgs.callPackage ../postgresql/default.nix { ... } + ) + ); + }; +} +``` + +**Pattern**: Uses `//` (attribute set merge) to combine: +- Hand-written utility packages +- Auto-generated PostgreSQL packages with extensions + +#### nix/packages/postgres.nix + +**Purpose**: Generate PostgreSQL packages with extensions. + +Demonstrates advanced functional composition: + +```nix +{ inputs, ... }: +{ + perSystem = { pkgs, ... }: + let + # List of extension definitions + ourExtensions = [ + ../ext/rum.nix + ../ext/timescaledb.nix + ../ext/pgsodium.nix + # ... 40+ extensions + ]; + + # Filter extensions for specific versions + orioleFilteredExtensions = builtins.filter ( + x: x != ../ext/timescaledb.nix && x != ../ext/plv8 + ) ourExtensions; + + # Build all extensions for a version + makeOurPostgresPkgs = version: + map (path: pkgs.callPackage path { inherit postgresql; }) + extensionsToUse; + + # Create full PostgreSQL distribution + makePostgres = version: { + bin = makePostgresBin version; # postgres + extensions + exts = makeOurPostgresPkgsSet version; # individual extensions + recurseForDerivations = true; + }; + + basePackages = { + psql_15 = makePostgres "15"; + psql_17 = makePostgres "17"; + psql_orioledb-17 = makePostgres "orioledb-17"; + }; + in + { + # Flatten nested structure into dot-separated names + packages = inputs.flake-utils.lib.flattenTree basePackages; + }; +} +``` + +**flattenTree transformation**: +```nix +# Input: +{ psql_15.bin = ; psql_15.exts.rum = ; } + +# Output: +{ "psql_15/bin" = ; "psql_15/exts/rum" = ; } +``` + +This allows `nix build .#psql_15/bin` or `nix build .#psql_15/exts/rum`. + +#### nix/apps.nix + +**Purpose**: Define runnable applications. + +Maps packages to app definitions: + +```nix +{ ... }: +{ + perSystem = { self', ... }: + let + mkApp = attrName: binName: { + type = "app"; + program = "${self'.packages."${attrName}"}/bin/${binName}"; + }; + in + { + apps = { + start-server = mkApp "start-server" "start-postgres-server"; + start-client = mkApp "start-client" "start-postgres-client"; + dbmate-tool = mkApp "dbmate-tool" "dbmate-tool"; + }; + }; +} +``` + +**Usage**: `nix run .#start-server` executes the app. + +#### nix/overlays/default.nix + +**Purpose**: Define nixpkgs overlays. + +Overlays are flake-level (not per-system): + +```nix +{ self, ... }: +{ + flake.overlays.default = final: prev: { + # Re-export packages from current system + inherit (self.packages.${final.system}) + postgresql_15 + postgresql_17 + supabase-groonga; + + # Define new packages in terms of final/prev + cargo-pgrx = final.callPackage ../cargo-pgrx/default.nix { + inherit (final) lib darwin fetchCrate openssl; + }; + + # Override existing packages + buildPgrxExtension = final.callPackage ../cargo-pgrx/buildPgrxExtension.nix { + inherit (final) cargo-pgrx lib; + }; + }; +} +``` + +**Pattern**: The overlay is applied in `nix/nixpkgs.nix` to all systems. + +#### nix/checks.nix + +**Purpose**: Define build checks and tests. + +```nix +{ self, ... }: +{ + perSystem = { lib, pkgs, self', system, ... }: + { + checks = { + psql_15 = pkgs.runCommand "run-check-harness-psql-15" { } + (lib.getExe (makeCheckHarness self'.packages."psql_15/bin")); + psql_17 = pkgs.runCommand "run-check-harness-psql-17" { } + (lib.getExe (makeCheckHarness self'.packages."psql_17/bin")); + } + // pkgs.lib.optionalAttrs (system == "x86_64-linux") { + devShell = self'.devShells.default; + }; + }; +} +``` + +**Conditional outputs**: `optionalAttrs` includes checks only on x86_64-linux. + +**Usage**: `nix flake check` runs all checks. + +#### nix/devShells.nix + +**Purpose**: Define development environments. + +```nix +{ ... }: +{ + perSystem = { pkgs, self', config, ... }: + { + devShells = { + default = pkgs.mkShell { + packages = with pkgs; [ + coreutils + just + nix-update + shellcheck + self'.packages.start-server + config.treefmt.build.wrapper + ]; + shellHook = '' + export HISTFILE=.history + ${config.pre-commit.installationScript} + ''; + }; + }; + }; +} +``` + +**Cross-module references**: +- `self'.packages.start-server` - from packages module +- `config.treefmt.build.wrapper` - from fmt module +- `config.pre-commit.installationScript` - from hooks module + +### Integration Modules + +#### nix/fmt.nix + +**Purpose**: Configure code formatting via treefmt-nix. + +```nix +{ inputs, ... }: +{ + imports = [ inputs.treefmt-nix.flakeModule ]; + perSystem = { pkgs, ... }: + { + treefmt.programs = { + deadnix.enable = true; + nixfmt = { + enable = true; + package = pkgs.nixfmt-rfc-style; + }; + ruff-format.enable = true; + }; + }; +} +``` + +**Module import**: `inputs.treefmt-nix.flakeModule` provides the `treefmt` option namespace. + +**Exposed outputs**: +- `config.treefmt.build.wrapper` - available to other modules +- `formatter` - automatic flake output for `nix fmt` + +#### nix/hooks.nix + +**Purpose**: Configure git pre-commit hooks. + +```nix +{ inputs, ... }: +{ + imports = [ inputs.git-hooks.flakeModule ]; + perSystem = { config, ... }: + { + pre-commit = { + check.enable = true; + settings.hooks = { + treefmt = { + enable = true; + package = config.treefmt.build.wrapper; + }; + }; + }; + }; +} +``` + +**Cross-module dependency**: Uses `config.treefmt.build.wrapper` from `nix/fmt.nix`. + +## Common Patterns + +### Self-Reference + +Modules reference outputs from other modules: + +```nix +# Reference current system's packages +self'.packages."psql_15/bin" + +# Reference flake-level config +self.supabase.defaults + +# Reference flake-level overlay +self.overlays.default + +# Reference inputs for current system +inputs'.nix-editor.packages.default +``` + +### Dependency Injection via callPackage + +`pkgs.callPackage` automatically injects function arguments: + +```nix +# File: nix/packages/dbmate-tool.nix +{ writeShellApplication, dbmate, ... }: +writeShellApplication { + name = "dbmate-tool"; + # ... +} + +# Usage in nix/packages/default.nix +dbmate-tool = pkgs.callPackage ./dbmate-tool.nix { + # Override specific arguments + inherit (self.supabase) defaults; +}; +``` + +Arguments from `pkgs` are auto-injected; explicit args override. + +### Recursive Package Sets + +Allow building nested package paths: + +```nix +{ + psql_15 = { + bin = ; + exts = { + rum = ; + pgsodium = ; + recurseForDerivations = true; + }; + recurseForDerivations = true; + }; +} +``` + +**Effect**: Enables `nix build .#psql_15/exts/rum`. + +### Attribute Set Merging + +Combine multiple package sources: + +```nix +packages = ( + { manually-defined = ...; } + // generatedPackages + // lib.optionalAttrs (system == "x86_64-linux") { + linux-only = ...; + } +); +``` + +### Typed Configuration + +Define schemas with `lib.types`: + +```nix +let + configType = lib.types.submodule { + options = { + port = lib.mkOption { + type = lib.types.str; + default = "5435"; + description = "PostgreSQL port"; + }; + }; + }; +in { + flake.options.supabase = lib.mkOption { type = configType; }; +} +``` + +## Nixpkgs Library Functions + +!!! tip "In-Depth Coverage" + For detailed examples of how these functions work together with flake-parts, including the module system foundations and composition patterns, see **[Flake-Parts and nixpkgs lib](./flake-parts-nixpkgs-lib.md)**. + +Common utilities from `pkgs.lib`: + +| Function | Purpose | Example | +|----------|---------|---------| +| `lib.types.*` | Type definitions | `lib.types.str`, `lib.types.submodule` | +| `lib.mkOption` | Define typed options | `lib.mkOption { type = lib.types.str; }` | +| `lib.filterAttrs` | Filter attribute sets | `lib.filterAttrs (n: v: n != "override") pkgs` | +| `lib.mapAttrsToList` | Convert attrs to list | `lib.mapAttrsToList (n: v: { name = n; }) attrs` | +| `lib.optionalAttrs` | Conditional attributes | `lib.optionalAttrs (system == "x86_64-linux") {...}` | +| `lib.hasSuffix` | String suffix check | `lib.hasSuffix ".sql" filename` | +| `lib.makeBinPath` | Create PATH string | `lib.makeBinPath [ pkg1 pkg2 ]` | +| `lib.getExe` | Extract executable | `lib.getExe pkgs.hello` → `/nix/store/.../bin/hello` | +| `pkgs.callPackage` | Dependency injection | `pkgs.callPackage ./pkg.nix { extra = value; }` | +| `pkgs.symlinkJoin` | Merge package outputs | `pkgs.symlinkJoin { paths = [ pkg1 pkg2 ]; }` | + +## Module Evaluation Order + +1. **System selection**: Flake-parts evaluates modules once per declared system +2. **nixpkgs.nix**: Instantiates `pkgs` for current system with overlays +3. **config.nix**: Defines flake-wide configuration options +4. **overlays/**: Overlay definition (referenced by nixpkgs.nix) +5. **packages/**: Package definitions (uses `pkgs` and `self.supabase.config`) +6. **apps.nix**: App definitions (references `self'.packages`) +7. **devShells.nix**: Dev shells (references `self'.packages` and `config.*`) +8. **checks.nix**: Tests (references `self'.packages`) +9. **fmt.nix**, **hooks.nix**: Integration modules (expose `config.*`) + +**Key insight**: `pkgs` is available to all modules because `nixpkgs.nix` sets `_module.args.pkgs`. + +## Extending the Flake + +### Adding a New Module + +1. Create file in `nix/` directory: + +```nix +# nix/my-module.nix +{ ... }: +{ + perSystem = { pkgs, self', ... }: + { + packages = { + my-package = pkgs.writeShellScriptBin "my-script" '' + echo "Hello from my module" + ''; + }; + }; +} +``` + +2. Import in `flake.nix`: + +```nix +imports = [ + nix/apps.nix + nix/my-module.nix # Add here + # ... +]; +``` + +### Adding Flake-Level Configuration + +Add to `nix/config.nix`: + +```nix +flake.config.myConfig = { + someOption = "value"; +}; +``` + +Access in other modules: + +```nix +perSystem = { ... }: +{ + packages.example = pkgs.writeText "config.txt" + self.myConfig.someOption; +}; +``` + +## Benefits of This Architecture + +1. **Modularity**: Each concern (packages, apps, checks) in separate files +2. **DRY**: Write per-system code once, evaluated for all systems +3. **Type Safety**: Typed configuration catches errors early +4. **Composability**: Import third-party modules (treefmt, git-hooks) +5. **Maintainability**: Easy to locate and modify specific functionality +6. **Clarity**: Self-documenting structure with clear separation of concerns + +## Further Reading + +- [Flake-parts documentation](https://flake.parts/) +- [NixOS module system](https://nixos.org/manual/nixos/stable/#sec-writing-modules) +- [Nixpkgs lib reference](https://nixos.org/manual/nixpkgs/stable/#chap-functions) +- [Nix flakes](https://nixos.wiki/wiki/Flakes) diff --git a/nix/docs/flake-parts-nixpkgs-lib.md b/nix/docs/flake-parts-nixpkgs-lib.md new file mode 100644 index 000000000..f590123dd --- /dev/null +++ b/nix/docs/flake-parts-nixpkgs-lib.md @@ -0,0 +1,699 @@ +# Flake-Parts and nixpkgs lib Foundations + +This document explains how flake-parts leverages nixpkgs lib to provide a powerful module system for organizing Nix flakes, with examples from this repository's implementation. + +!!! note "Related Documentation" + This page focuses on the nixpkgs lib foundations and how flake-parts uses them. For a high-level overview of the module structure and practical usage patterns, see **[Flake-Parts Architecture](./flake-parts-architecture.md)**. + +## Overview + +Flake-parts is built on top of the **nixpkgs module system** (`lib.modules`), which is the same foundation used by NixOS configuration. Understanding this relationship helps you reason about how flake-parts works and why it's designed the way it is. + +## The nixpkgs lib Module System + +### Core Components + +Flake-parts leverages these fundamental nixpkgs lib components: + +| Component | Purpose | Example Usage | +|-----------|---------|---------------| +| `lib.mkOption` | Declare configuration options | Define typed module options | +| `lib.types.*` | Type checking and validation | Ensure configuration correctness | +| `lib.mkIf` | Conditional configuration | Include config based on conditions | +| `lib.mkMerge` | Merge multiple configurations | Combine attribute sets | +| `lib.mkDefault` | Default values with priority | Set overridable defaults | +| `lib.evalModules` | Evaluate module system | Process module imports | + +### How flake-parts Uses lib.modules + +When you call `mkFlake`, flake-parts internally uses `lib.evalModules` to: + +1. **Evaluate all imported modules** - Process each module in the `imports` list +2. **Merge configurations** - Combine options from all modules +3. **Type-check values** - Validate configuration against option types +4. **Generate outputs** - Transform module config into flake outputs + +```nix +# Simplified internal implementation +mkFlake = { inputs }: moduleArgs: + let + evaluated = lib.evalModules { + modules = [ baseModule ] ++ moduleArgs.imports; + specialArgs = { inherit inputs; }; + }; + in + evaluated.config.flake; +``` + +## Key nixpkgs lib Functions in Action + +### 1. lib.genAttrs - System Generation + +Flake-parts uses `lib.genAttrs` to generate per-system outputs: + +```nix +# Internal flake-parts logic +systems = [ "x86_64-linux" "aarch64-linux" "aarch64-darwin" ]; +packages = lib.genAttrs systems (system: + # Your perSystem config evaluated for each system + perSystemConfig.packages +); +``` + +**In our flake:** +```nix +systems = with flake-utils.lib; [ + system.x86_64-linux + system.aarch64-linux + system.aarch64-darwin +]; +``` + +This generates outputs for all three systems automatically. + +### 2. lib.mapAttrs - Attribute Transformation + +Used to transform outputs across systems: + +```nix +# Transform all packages to add metadata +packages = lib.mapAttrs (name: drv: + drv.overrideAttrs (old: { + meta = old.meta or {} // { platforms = [ system ]; }; + }) +) perSystemPackages; +``` + +**In our postgres.nix:** +```nix +makeOurPostgresPkgsSet = version: + (builtins.listToAttrs ( + map (drv: { + name = drv.pname; + value = drv; + }) (makeOurPostgresPkgs version) + )) + // { recurseForDerivations = true; }; +``` + +### 3. lib.recursiveUpdate - Deep Merging + +Merges nested attribute sets: + +```nix +packages = lib.recursiveUpdate { + default = myPackage; +} { + tools = { cli = cliTool; }; +}; +# Result: { default = ...; tools.cli = ...; } +``` + +**In our checks.nix:** +```nix +checks = { + psql_15 = ...; + psql_17 = ...; +} +// pkgs.lib.optionalAttrs (system == "x86_64-linux") { + devShell = self'.devShells.default; +} +// pkgs.lib.optionalAttrs (system == "x86_64-linux") ( + import ./ext/tests { ... } +); +``` + +### 4. lib.filterAttrs - Selective Inclusion + +Filters attribute sets by predicate: + +```nix +# Remove internal attributes +publicPackages = lib.filterAttrs + (n: _: n != "override" && n != "overrideAttrs") + allPackages; +``` + +**In our packages/default.nix:** +```nix +packages = ( + { /* hand-written packages */ } + // lib.filterAttrs + (n: _v: n != "override" && n != "overrideAttrs" && n != "overrideDerivation") + (pkgs.callPackage ../postgresql/default.nix { ... }) +); +``` + +## The perSystem Abstraction + +### Standard Flake Pattern (Verbose) + +Without flake-parts, you'd write: + +```nix +outputs = { self, nixpkgs }: { + packages.x86_64-linux.hello = + nixpkgs.legacyPackages.x86_64-linux.hello; + packages.aarch64-linux.hello = + nixpkgs.legacyPackages.aarch64-linux.hello; + packages.aarch64-darwin.hello = + nixpkgs.legacyPackages.aarch64-darwin.hello; + + devShells.x86_64-linux.default = + nixpkgs.legacyPackages.x86_64-linux.mkShell { ... }; + devShells.aarch64-linux.default = + nixpkgs.legacyPackages.aarch64-linux.mkShell { ... }; + devShells.aarch64-darwin.default = + nixpkgs.legacyPackages.aarch64-darwin.mkShell { ... }; +}; +``` + +### flake-parts Pattern (DRY) + +With flake-parts, you write once: + +```nix +perSystem = { pkgs, system, ... }: { + packages.hello = pkgs.hello; + devShells.default = pkgs.mkShell { ... }; +}; +``` + +Flake-parts expands this using `lib.genAttrs` under the hood: + +```nix +# What flake-parts generates +let + perSystemOutputs = system: + let pkgs = import nixpkgs { inherit system; }; + in { + packages.hello = pkgs.hello; + devShells.default = pkgs.mkShell { ... }; + }; +in { + packages = lib.genAttrs systems (system: + (perSystemOutputs system).packages + ); + devShells = lib.genAttrs systems (system: + (perSystemOutputs system).devShells + ); +} +``` + +## Module Composition with lib + +### Import System + +Flake-parts uses the nixpkgs module `imports` mechanism: + +```nix +imports = [ + ./nix/apps.nix # Custom module + ./nix/checks.nix # Custom module + inputs.treefmt-nix.flakeModule # Third-party module + inputs.git-hooks.flakeModule # Third-party module +]; +``` + +Each imported module can: +- Define options with `lib.mkOption` +- Set configuration values +- Import other modules +- Access shared state + +### Module Pattern + +**Basic module structure:** + +```nix +{ lib, ... }: +{ + options = { + myProject.enable = lib.mkOption { + type = lib.types.bool; + default = false; + description = "Enable my project"; + }; + }; + + config = lib.mkIf config.myProject.enable { + perSystem = { pkgs, ... }: { + packages.myPackage = pkgs.hello; + }; + }; +} +``` + +**In our config.nix:** + +```nix +{ lib, ... }: +let + postgresqlDefaults = lib.types.submodule { + options = { + port = lib.mkOption { + type = lib.types.str; + default = "5435"; + }; + host = lib.mkOption { + type = lib.types.str; + default = "localhost"; + }; + superuser = lib.mkOption { + type = lib.types.str; + default = "supabase_admin"; + }; + }; + }; +in +{ + flake.options.supabase = lib.mkOption { + type = lib.types.submodule { + options.defaults = lib.mkOption { + type = postgresqlDefaults; + }; + }; + }; + flake.config.supabase = { defaults = { }; }; +} +``` + +## Type Safety with lib.types + +Flake-parts leverages nixpkgs type system for validation: + +### Common Types + +```nix +lib.types.str # String +lib.types.int # Integer +lib.types.bool # Boolean +lib.types.path # File system path +lib.types.package # Nix derivation +lib.types.listOf T # List of type T +lib.types.attrsOf T # Attribute set with values of type T +lib.types.enum [...] # Enumeration +lib.types.submodule # Nested module +lib.types.nullOr T # T or null +``` + +### Submodules for Structure + +**In our config.nix:** + +```nix +postgresqlVersion = lib.types.submodule { + options = { + version = lib.mkOption { type = lib.types.str; }; + hash = lib.mkOption { type = lib.types.str; }; + revision = lib.mkOption { + type = lib.types.nullOr lib.types.str; + default = null; + }; + }; +}; + +supabaseSubmodule = lib.types.submodule { + options = { + defaults = lib.mkOption { type = postgresqlDefaults; }; + supportedPostgresVersions = lib.mkOption { + type = lib.types.attrsOf (lib.types.attrsOf postgresqlVersion); + default = { }; + }; + }; +}; +``` + +This provides: +- **Compile-time validation** - Catches type errors early +- **Auto-documentation** - Types serve as documentation +- **IDE support** - Better completion and hints + +## Advanced Patterns + +### 1. Conditional Configuration with lib.mkIf + +```nix +perSystem = { pkgs, system, config, ... }: { + packages = lib.mkIf (system == "x86_64-linux") { + linux-only-tool = pkgs.callPackage ./tool.nix { }; + }; +}; +``` + +**In our checks.nix:** + +```nix +checks = { ... } + // pkgs.lib.optionalAttrs (system == "x86_64-linux") { + inherit (self'.packages) + postgresql_15_debug + postgresql_17_debug; + }; +``` + +### 2. Priority System with lib.mkDefault + +```nix +options.port = lib.mkOption { + type = lib.types.str; + default = lib.mkDefault "5432"; # Low priority +}; + +config.port = "5435"; # Higher priority, overrides +``` + +### 3. List Merging with lib.mkMerge + +```nix +config.packages = lib.mkMerge [ + { base = basePackage; } + (lib.mkIf enableExtras { extra = extraPackage; }) + { tools = toolsPackage; } +]; +``` + +### 4. Cross-System References with withSystem + +```nix +flake.nixosModules.postgres = { config, ... }: { + options.services.supabase-postgres.package = lib.mkOption { + type = lib.types.package; + default = inputs.self.packages.${config.nixpkgs.system}.psql_17; + }; +}; +``` + +## Composition Patterns in This Project + +### Extension Composition + +**postgres.nix demonstrates functional composition:** + +```nix +let + # Base extensions + ourExtensions = [ + ../ext/rum.nix + ../ext/timescaledb.nix + ../ext/pgsodium.nix + # ... 40+ extensions + ]; + + # Filtered for specific versions + orioleFilteredExtensions = builtins.filter ( + x: x != ../ext/timescaledb.nix && x != ../ext/plv8 + ) ourExtensions; + + orioledbExtensions = orioleFilteredExtensions ++ [ ../ext/orioledb.nix ]; + + # Select extensions based on version + extensionsForVersion = version: + if version == "orioledb-17" then orioledbExtensions + else if version == "17" then dbExtensions17 + else ourExtensions; + + # Build extensions + makeOurPostgresPkgs = version: + map (path: pkgs.callPackage path { inherit postgresql; }) + (extensionsForVersion version); +in +{ + packages = { + psql_15 = makePostgres "15"; + psql_17 = makePostgres "17"; + psql_orioledb-17 = makePostgres "orioledb-17"; + }; +} +``` + +### Package Set Flattening + +**Using flake-utils.lib.flattenTree:** + +```nix +# Input structure +basePackages = { + psql_15 = { + bin = ; + exts = { + rum = ; + pgsodium = ; + }; + }; +}; + +# Flatten to dot notation +packages = inputs.flake-utils.lib.flattenTree basePackages; + +# Result +{ + "psql_15/bin" = ; + "psql_15/exts/rum" = ; + "psql_15/exts/pgsodium" = ; +} +``` + +### Attribute Set Merging + +**checks.nix merges multiple package sources:** + +```nix +checks = + { + # Explicit checks + psql_15 = makeCheckHarness self'.packages."psql_15/bin"; + psql_17 = makeCheckHarness self'.packages."psql_17/bin"; + } + // pkgs.lib.optionalAttrs (system == "x86_64-linux") { + # Debug packages (Linux only) + inherit (self'.packages) + postgresql_15_debug + postgresql_17_debug; + } + // pkgs.lib.optionalAttrs (system == "x86_64-linux") ( + # Extension tests (Linux only) + import ./ext/tests { inherit self pkgs; } + ); +``` + +## Advantages Over Standard Flakes + +### 1. Reduced Boilerplate + +**Before (standard flake):** +```nix +{ + packages.x86_64-linux.postgres = ...; + packages.aarch64-linux.postgres = ...; + packages.aarch64-darwin.postgres = ...; + apps.x86_64-linux.server = ...; + apps.aarch64-linux.server = ...; + apps.aarch64-darwin.server = ...; + # Repeat for every output type +} +``` + +**After (flake-parts):** +```nix +perSystem = { pkgs, ... }: { + packages.postgres = ...; + apps.server = ...; +}; +``` + +### 2. Module Reusability + +Extract common logic to modules: + +```nix +# modules/postgres-common.nix +{ lib, ... }: +{ + options.postgresDefaults = lib.mkOption { + type = lib.types.submodule { + options = { + port = lib.mkOption { type = lib.types.str; }; + superuser = lib.mkOption { type = lib.types.str; }; + }; + }; + }; +} + +# Import in multiple projects +imports = [ ./modules/postgres-common.nix ]; +``` + +### 3. Type-Safe Configuration + +```nix +# Define schema +options.postgresVersion = lib.mkOption { + type = lib.types.enum ["15" "17"]; + default = "17"; +}; + +# Type error caught at evaluation +config.postgresVersion = "16"; # Error: value "16" is not in enum +``` + +### 4. Third-Party Integration + +Seamlessly integrate external modules: + +```nix +imports = [ + inputs.treefmt-nix.flakeModule # Adds 'treefmt' options + inputs.git-hooks.flakeModule # Adds 'pre-commit' options +]; + +perSystem = { config, ... }: { + treefmt.programs.nixfmt.enable = true; + pre-commit.settings.hooks.treefmt = { + enable = true; + package = config.treefmt.build.wrapper; # Cross-module reference + }; +}; +``` + +## Navigation Strategy + +When working with this flake: + +### 1. Start at the Entry Point + +``` +flake.nix → inputs.flake-parts.lib.mkFlake + ├── systems (which architectures) + ├── imports (feature modules) + └── perSystem outputs +``` + +### 2. Follow Module Imports + +```nix +imports = [ + nix/apps.nix # Runnable commands + nix/checks.nix # Tests and validation + nix/config.nix # Configuration options + nix/devShells.nix # Development environments + nix/nixpkgs.nix # nixpkgs configuration + nix/packages # Package definitions + nix/overlays # Package overlays +]; +``` + +### 3. Understand perSystem Context + +Each perSystem block has access to: + +```nix +perSystem = { + # Special arguments + self', # Current system's outputs + inputs', # Current system's inputs + pkgs, # nixpkgs for current system + system, # System string + lib, # nixpkgs lib + config, # Module config + ... +}: { + # Your outputs +} +``` + +### 4. Trace Helper Functions + +**In postgres.nix:** +``` +makePostgres + └── makePostgresBin + ├── makeOurPostgresPkgs + │ └── extensionsForVersion + └── makeReceipt +``` + +## Practical Examples + +### Adding a New PostgreSQL Version + +```nix +# nix/config.nix +flake.config.supabase.supportedPostgresVersions.postgres."18" = { + version = "18.0"; + hash = "sha256-..."; +}; + +# nix/packages/postgres.nix +basePackages = { + psql_15 = makePostgres "15"; + psql_17 = makePostgres "17"; + psql_18 = makePostgres "18"; # Add here +}; +``` + +### Adding a Custom Extension + +```nix +# nix/ext/my_extension.nix +{ postgresql, stdenv, fetchFromGitHub }: +stdenv.mkDerivation { + pname = "my_extension"; + version = "1.0.0"; + + src = fetchFromGitHub { ... }; + + buildInputs = [ postgresql ]; + + installPhase = '' + install -D -t $out/lib *.so + install -D -t $out/share/postgresql/extension *.sql + install -D -t $out/share/postgresql/extension *.control + ''; +} + +# nix/packages/postgres.nix +ourExtensions = [ + # ... existing extensions + ../ext/my_extension.nix +]; +``` + +### Adding Development Tools + +```nix +# nix/devShells.nix +perSystem = { pkgs, self', config, ... }: { + devShells.default = pkgs.mkShell { + packages = [ + pkgs.postgresql + self'.packages.start-server + config.treefmt.build.wrapper + ]; + shellHook = '' + echo "PostgreSQL development environment" + echo "Run: start-server 15" + ''; + }; +}; +``` + +## Key Insights + +The Supabase Postgres project demonstrates how flake-parts and nixpkgs lib work together to create: + +1. **Systematic organization** - Clear separation of concerns via modules +2. **Type safety** - Configuration validated at evaluation time +3. **Reusability** - Helper functions eliminate duplication +4. **Extensibility** - New versions/extensions slot in easily +5. **Integration** - Third-party modules compose seamlessly +6. **Maintainability** - Changes localized to relevant modules + +This architecture scales well for complex multi-version software builds, making it ideal for a PostgreSQL distribution with 40+ extensions across multiple versions. + +## Further Reading + +- [Nixpkgs lib reference](https://nixos.org/manual/nixpkgs/stable/#chap-functions) +- [NixOS module system](https://nixos.org/manual/nixos/stable/#sec-writing-modules) +- [Flake-parts documentation](https://flake.parts/) +- [Flake-parts module options](https://flake.parts/options/flake-parts.html) diff --git a/nix/docs/nix-directory-structure.md b/nix/docs/nix-directory-structure.md index 24ca49943..c58441b4b 100644 --- a/nix/docs/nix-directory-structure.md +++ b/nix/docs/nix-directory-structure.md @@ -2,6 +2,12 @@ This document explains the Nix structure used in this repository. The project uses [flake-parts](https://flake.parts/) to split a `flake.nix` into specialized, maintainable modules. +!!! tip "Deep Dive" + For a comprehensive explanation of how flake-parts works in this repository, including module scopes, evaluation order, and common patterns, see: + + - **[Flake-Parts Architecture](./flake-parts-architecture.md)** - Module structure and patterns + - **[Flake-Parts and nixpkgs lib](./flake-parts-nixpkgs-lib.md)** - nixpkgs lib foundations + The root `flake.nix` serves only as an entry point that references specialized modules in the `nix/` directory: ``` diff --git a/nix/mkdocs.yml b/nix/mkdocs.yml index f472b51c4..a6e2bc7ad 100644 --- a/nix/mkdocs.yml +++ b/nix/mkdocs.yml @@ -12,6 +12,8 @@ nav: - Development: - Getting Started: start-here.md - Nix tree structure: nix-directory-structure.md + - Flake-Parts Architecture: flake-parts-architecture.md + - Flake-Parts and nixpkgs lib: flake-parts-nixpkgs-lib.md - Workflow: development-workflow.md - Build PostgreSQL: build-postgres.md - Start Client/Server: start-client-server.md