Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
253 changes: 240 additions & 13 deletions .github/workflows/ci.yml

Large diffs are not rendered by default.

7 changes: 7 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -721,6 +721,13 @@ jobs:
--job Builds \
--artifact liboliphaunt-native-release-assets

- name: Prove native and WASIX ICU data identity
if: ${{ steps.release_plan.outputs.has_release_changes == 'true' && contains(fromJson(steps.release_plan.outputs.products_json), 'liboliphaunt-native') && contains(fromJson(steps.release_plan.outputs.products_json), 'liboliphaunt-wasix') }}
run: |
tools/dev/bun.sh tools/release/check-cross-family-icu-data.mjs \
target/liboliphaunt/release-assets \
target/oliphaunt-wasix/release-assets

- name: Set up Bun for TypeScript npm consumer checks
if: ${{ steps.release_plan.outputs.has_release_changes == 'true' && contains(fromJson(steps.release_plan.outputs.products_json), 'oliphaunt-js') }}
uses: ./.github/actions/setup-bun
Expand Down
425 changes: 425 additions & 0 deletions docs/architecture/cluster-seeds-and-icu.md

Large diffs are not rendered by default.

25 changes: 17 additions & 8 deletions docs/architecture/database-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ does not require relearning what storage means.
## Public vocabulary

`storage` answers one question: where PostgreSQL's mutable database files live.
It does not select the runtime, extension artifacts, packaged PGDATA template,
It does not select the runtime, extension artifacts, packaged cluster seed,
or initialization policy.

| Storage | Lifetime | Owner | Default |
Expand Down Expand Up @@ -76,11 +76,19 @@ to the same opaque format. Portable `@oliphaunt/wasix-ts` and native

## Initialization and restore

Opening a new or empty store initializes it with the product's packaged
template. Reopening a non-empty persistent store never silently reinitializes
it. Restore is a separate static operation into a new or empty destination.
Tooling that needs `initdb` invokes the packaged tool directly; ordinary SDKs do
not expose an initialization-mode abstraction.
Every ordinary package-managed new or empty embedded store uses its runtime
product's packaged cluster seed. Native targets and WASIX use separately
qualified physical compatibility identities, and ICU selection resolves ICU
data with the matching `icu` seed. Explicit locally built native runtimes and
new native server roots use the internal `initdb` path with the fixed
`postgres` bootstrap role. Public usernames always select existing connection
roles. The decisions and recurring release gates are recorded in
[Cluster seeds and ICU](cluster-seeds-and-icu.md).

Reopening a non-empty persistent store never silently reinitializes it. Restore
is a separate static operation into a new or empty destination. Tooling that
needs `initdb` invokes the packaged tool directly; ordinary SDKs do not expose
an initialization-mode abstraction.

A published non-empty persistent cluster is never silently reinitialized. A
provider may discard and rebuild only its explicitly unpublished first-open
Expand Down Expand Up @@ -151,8 +159,9 @@ build physically interchangeable. Native roots are shared among compatible
native SDKs. Rust and WASIX TypeScript use the same root descriptor and WASIX
physical-format value, but cross-binding root handoff is not a supported or
qualified workflow. The managed-root descriptor is
written once when the root is created. Source fingerprints remain asset-graph coherence identities
used to reject mixed runtime, template, AOT, and extension build outputs; they
written once when the root is created. WASIX source fingerprints remain
asset-graph coherence identities used to reject mixed runtime, cluster-seed,
AOT, and extension build outputs; they
are not a physical-reopen key or binding identity in the root. Both runtime
families validate either exact descriptor shape. Opening another family's root
is not a supported transfer path, so the SDKs add no cross-family rejection
Expand Down
2 changes: 1 addition & 1 deletion docs/architecture/ios.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Apple platforms delegates to the same Swift implementation.

- One PostgreSQL backend is resident in the app process.
- One physical PostgreSQL session is serialized by the SDK.
- Native runtime, template, and exact selected-extension resources are packaged
- Native runtime, cluster-seed, and exact selected-extension resources are packaged
into the app.
- App-owned storage is a managed root containing exactly `pgdata` and the
`.oliphaunt.json` descriptor. PostgreSQL owns `pg_wal` inside `pgdata`.
Expand Down
4 changes: 2 additions & 2 deletions docs/architecture/native-liboliphaunt.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ backup/restore, cancellation, raw protocol execution, and static extension
registration.

Language SDKs own typed query values, builders, paths, async adapters, package
resolution, template hydration, server and broker process orchestration, and
resolution, cluster-seed hydration, server and broker process orchestration, and
language-native errors. They do not reimplement the C runtime's direct
lifecycle, archive parser, or root validation.

Expand Down Expand Up @@ -45,7 +45,7 @@ than multiple direct handles.
## Managed roots

The configured path is always `<root>/pgdata`. `oliphaunt_init` validates an
already-prepared root and never runs `initdb`, hydrates a template, creates a
already-prepared root and never runs `initdb`, hydrates a cluster seed, creates a
descriptor, or adopts raw PGDATA. Before backend startup it requires:

- a real root, `pgdata`, and `pgdata/global` directory, with no symbolic-link,
Expand Down
14 changes: 7 additions & 7 deletions docs/internal/IMPLEMENTATION_CHECKLIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,8 @@ intentionally not maintained here.
- [x] Stable CI job names are derived from Moon task `ci-*` tags. Evidence:
`tools/graph/ci_plan.mjs` and `tools/policy/check-moon-product-graph.mjs`.
- [x] Runtime target fan-out is metadata-driven, not hardcoded in mobile jobs.
Evidence: focused mobile planner output narrows native runtime and native
extension matrices by surface, and `tools/policy/check-release-policy.py`
Evidence: focused mobile planner output selects complete native runtime and
native extension compatibility domains by surface, and `tools/policy/check-release-policy.py`
asserts Android mobile builds request only `android-arm64-v8a` and
`android-x86_64` extension artifacts while iOS mobile builds request only
`ios-xcframework`.
Expand Down Expand Up @@ -682,11 +682,11 @@ Run before claiming this architecture complete:
- [x] Focused mobile builder plans are target-consistent:
`GITHUB_EVENT_NAME=workflow_dispatch NATIVE_TARGET=android-arm64-v8a
WASM_TARGET=all MOBILE_TARGET=android tools/dev/bun.sh tools/graph/ci_plan.mjs`
emits one Android exact-extension row, one Android app row, and
`mobile_extension_package_native_targets=["android-arm64-v8a"]`; the matching
iOS probe emits only `ios-xcframework`. Incompatible focused inputs such as
`MOBILE_TARGET=android NATIVE_TARGET=ios-xcframework` now fail closed in the
planner.
expands to both Android ABI receipt/extension producers and one representative
`android-x86_64` emulator app; the matching iOS probe emits only
`ios-xcframework`. Incompatible focused inputs such as
`MOBILE_TARGET=android NATIVE_TARGET=ios-xcframework` fail closed in the
planner instead of producing a partial compatibility-domain proof.
- [x] Android SDK provisioning is shared and reproducible. Evidence:
`.github/actions/setup-android` calls `tools/dev/setup-android-sdk.sh`; the
script bootstraps Android command-line tools when `sdkmanager` is absent,
Expand Down
10 changes: 6 additions & 4 deletions docs/internal/OLIPHAUNT_PATCH_STACK.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ src/runtimes/liboliphaunt/native/tools/check-patch-stack.mjs --write
| 13 | `0013-liboliphaunt-fix-embedded-backend-main-return-contract.patch` | liboliphaunt <liboliphaunt@example.invalid> | liboliphaunt: fix embedded BackendMain return contract |
| 14 | `0014-liboliphaunt-use-portable-embedded-socketpair.patch` | liboliphaunt <liboliphaunt@example.invalid> | liboliphaunt: use portable embedded socketpair |
| 15 | `0015-liboliphaunt-add-embedded-meson-option.patch` | liboliphaunt <liboliphaunt@example.invalid> | liboliphaunt: add embedded meson option |
| 16 | `0016-liboliphaunt-skip-icu-collation-version-without-icu-data.patch` | liboliphaunt <liboliphaunt@example.invalid> | liboliphaunt: skip ICU collation setup without ICU data |
| 16 | `0016-liboliphaunt-control-initdb-collation-discovery.patch` | liboliphaunt <liboliphaunt@example.invalid> | liboliphaunt: control initdb collation discovery |
| 17 | `0017-liboliphaunt-namespace-dynahash-host-collisions.patch` | liboliphaunt <liboliphaunt@example.invalid> | liboliphaunt: namespace Apple dynahash host collisions |
| 18 | `0018-liboliphaunt-contain-embedded-proc-signals.patch` | liboliphaunt <liboliphaunt@example.invalid> | liboliphaunt: contain embedded process signals |
| 19 | `0019-liboliphaunt-link-windows-embedded-modules-to-host.patch` | liboliphaunt <liboliphaunt@example.invalid> | liboliphaunt: link Windows embedded modules to host |
Expand All @@ -47,6 +47,7 @@ src/runtimes/liboliphaunt/native/tools/check-patch-stack.mjs --write
- `meson_options.txt` (`0015-liboliphaunt-add-embedded-meson-option.patch`, `0019-liboliphaunt-link-windows-embedded-modules-to-host.patch`)
- `src/backend/access/transam/xlogarchive.c` (`0007-liboliphaunt-disable-shell-commands-on-apple-mobile.patch`)
- `src/backend/archive/shell_archive.c` (`0007-liboliphaunt-disable-shell-commands-on-apple-mobile.patch`)
- `src/backend/commands/collationcmds.c` (`0016-liboliphaunt-control-initdb-collation-discovery.patch`)
- `src/backend/commands/event_trigger.c` (`0012-liboliphaunt-enable-event-triggers-in-embedded-backend.patch`)
- `src/backend/libpq/be-secure.c` (`0001-liboliphaunt-add-backend-host-io.patch`)
- `src/backend/libpq/pqcomm.c` (`0001-liboliphaunt-add-backend-host-io.patch`)
Expand All @@ -59,7 +60,7 @@ src/runtimes/liboliphaunt/native/tools/check-patch-stack.mjs --write
- `src/backend/storage/ipc/procsignal.c` (`0018-liboliphaunt-contain-embedded-proc-signals.patch`)
- `src/backend/tcop/postgres.c` (`0002-liboliphaunt-add-embedded-entrypoint.patch`, `0003-liboliphaunt-return-from-embedded-frontend-terminate.patch`, `0004-liboliphaunt-run-embedded-exit-cleanup.patch`, `0005-liboliphaunt-restore-host-cwd.patch`, `0009-liboliphaunt-guard-embedded-proc-exit.patch`, `0010-liboliphaunt-use-host-runtime-paths.patch`, `0014-liboliphaunt-use-portable-embedded-socketpair.patch`, `0018-liboliphaunt-contain-embedded-proc-signals.patch`)
- `src/backend/utils/fmgr/dfmgr.c` (`0006-liboliphaunt-add-static-extension-loader.patch`, `0008-liboliphaunt-clean-embedded-symbols.patch`)
- `src/bin/initdb/initdb.c` (`0016-liboliphaunt-skip-icu-collation-version-without-icu-data.patch`)
- `src/bin/initdb/initdb.c` (`0016-liboliphaunt-control-initdb-collation-discovery.patch`)
- `src/include/libpq/libpq-be.h` (`0001-liboliphaunt-add-backend-host-io.patch`)
- `src/include/port.h` (`0011-liboliphaunt-add-android-embedded-shared-memory.patch`, `0020-liboliphaunt-enforce-embedded-signal-boundary.patch`)
- `src/include/storage/dsm_impl.h` (`0011-liboliphaunt-add-android-embedded-shared-memory.patch`)
Expand All @@ -78,6 +79,7 @@ src/runtimes/liboliphaunt/native/tools/check-patch-stack.mjs --write
| `meson_options.txt` | Meson-hosted embedded builds declare opt-in backend and Windows module-provider options without changing default PostgreSQL builds. |
| `src/backend/access/transam/xlogarchive.c` | Apple mobile embedded builds compile out optional archive shell commands. |
| `src/backend/archive/shell_archive.c` | Apple mobile embedded builds compile out optional archive shell commands. |
| `src/backend/commands/collationcmds.c` | System-collation import preserves host providers except during deliberate deterministic distributed-seed production; verified ICU readiness independently gates only the ICU provider. |
| `src/backend/commands/event_trigger.c` | Embedded FE/BE protocol sessions can run event triggers without changing standalone recovery behavior. |
| `src/backend/libpq/be-secure.c` | Backend secure read/write path delegates to a host I/O vtable only when OLIPHAUNT_EMBEDDED is set. |
| `src/backend/libpq/pqcomm.c` | Standalone embedded sessions avoid waiting on a non-existent postmaster death latch. |
Expand All @@ -90,7 +92,7 @@ src/runtimes/liboliphaunt/native/tools/check-patch-stack.mjs --write
| `src/backend/storage/ipc/procsignal.c` | The one-backend embedded runtime dispatches ProcSignal flags without sending process-directed host signals. |
| `src/backend/tcop/postgres.c` | Embedded backend entrypoint, protocol lifecycle, cwd restoration, host runtime paths, and host-owned SIGUSR1 disposition. |
| `src/backend/utils/fmgr/dfmgr.c` | Static extension lookup reuses PostgreSQL dynamic function manager semantics. |
| `src/bin/initdb/initdb.c` | Base runtimes skip ICU-backed collation setup until optional ICU data is present. |
| `src/bin/initdb/initdb.c` | Controlled seed production selects standard or verified ICU collation discovery without changing ordinary initdb semantics. |
| `src/include/libpq/libpq-be.h` | Host I/O vtable is attached to PostgreSQL Port state under OLIPHAUNT_EMBEDDED. |
| `src/include/port.h` | Embedded mobile builds avoid POSIX shared memory declarations and route embedded backend signal calls through the host-safe provider boundary. |
| `src/include/storage/dsm_impl.h` | Embedded mobile builds keep DSM on mmap instead of POSIX or SysV shared memory. |
Expand Down Expand Up @@ -136,7 +138,7 @@ src/runtimes/liboliphaunt/native/tools/check-patch-stack.mjs --write
| Embedded mobile shared memory and semaphores are process-local | `0011-liboliphaunt-add-android-embedded-shared-memory.patch` | `oliphaunt_embedded_shmem.c`, `oliphaunt_embedded_sema.c`, `OLIPHAUNT_EMBEDDED_MOBILE_SHMEM` | Android and Apple mobile builds avoid unavailable SysV shared memory and semaphores while direct mode remains one backend per process. |
| Event triggers run in embedded protocol sessions | `0012-liboliphaunt-enable-event-triggers-in-embedded-backend.patch` | `EventTriggersHaveRunnableBackend`, `OLIPHAUNT_EMBEDDED`, `event_triggers` | Keeps upstream single-user escape hatch outside OLIPHAUNT_EMBEDDED but treats embedded protocol sessions as runnable backends. |
| Meson builds expose an explicit embedded backend option | `0015-liboliphaunt-add-embedded-meson-option.patch` | `oliphaunt_embedded`, `add_project_arguments`, `-DOLIPHAUNT_EMBEDDED` | Windows and other Meson-hosted embedded builds enable the backend entrypoint through PostgreSQL build configuration while default server builds remain unchanged. |
| Optional ICU data stays optional during initdb | `0016-liboliphaunt-skip-icu-collation-version-without-icu-data.patch` | `getenv("ICU_DATA")`, `pg_collation_actual_version`, `pg_import_system_collations` | Base liboliphaunt runtimes can bootstrap non-ICU databases without bundling optional ICU data; ICU packages keep upstream collation setup by setting ICU_DATA. |
| Optional ICU data stays optional during initdb | `0016-liboliphaunt-control-initdb-collation-discovery.patch` | `OLIPHAUNT_INTERNAL_ICU_READY`, `OLIPHAUNT_INTERNAL_SKIP_SYSTEM_COLLATION_DISCOVERY`, `OLIPHAUNT_INTERNAL_SKIP_ICU_DISCOVERY`, `strcmp`, `pg_collation_actual_version`, `pg_import_system_collations` | Ordinary initdb and public collation import retain PostgreSQL host discovery. Distributed standard seeds suppress OS and ICU discovery; ICU seeds suppress only OS discovery and verify ICU readiness for initdb's unicode-version probe. |
| Apple builds namespace PostgreSQL dynahash symbols that collide with libSystem | `0017-liboliphaunt-namespace-dynahash-host-collisions.patch` | `#ifdef __APPLE__`, `oliphaunt_pg_hash_create`, `oliphaunt_pg_hash_destroy`, `oliphaunt_pg_hash_search` | Apple backend and extension objects share collision-free dynahash names; non-Apple PostgreSQL binary names remain unchanged. |
| Embedded ProcSignal delivery cannot escape into the host process | `0018-liboliphaunt-contain-embedded-proc-signals.patch` | `oliphaunt_send_proc_signal`, `pid != MyProcPid`, `procsignal_sigusr1_handler(SIGUSR1)`, `host owns SIGUSR1` | The one-backend embedded runtime dispatches ProcSignal flags synchronously, rejects foreign PIDs, and leaves the host SIGUSR1 disposition untouched; normal PostgreSQL server builds retain upstream signal delivery. |
| Windows embedded extension modules link to the host DLL provider | `0019-liboliphaunt-link-windows-embedded-modules-to-host.patch` | `oliphaunt_embedded_module_provider`, `requires an embedded MSVC Windows build`, `pg_mod_link_args += oliphaunt_embedded_module_provider`, `oliphaunt_embedded_module_provider == ''` | Embedded MSVC extension modules resolve PostgreSQL backend symbols from the oliphaunt host import library; ordinary PostgreSQL modules retain the upstream postgres executable link contract. |
Expand Down
Loading
Loading