feat(packages): extract fn-types; make fn-runtime/fn-app publishable (Wave 1 of portable-functions toolkit)#41
Conversation
First wave of the portable-functions toolkit (Starship-style split). Establishes a single source-of-truth types package that the rest of the toolkit (fn-generator, fn-client, fn-cli — landing in later waves) will depend on. - New @constructive-io/fn-types@0.1.0 — runtime contract types (FunctionHandler, FunctionContext, ServerOptions), HandlerManifest, FnRegistry/FnRegistryEntry, FnConfig + defineConfig() helper for typed fn.config.ts files. No logic, no shell-out. - @constructive-io/fn-runtime@1.2.0 — drops private:true; now depends on fn-types and re-exports the runtime types from there. Source files import from @constructive-io/fn-types instead of local ./types.ts (which is removed). - @constructive-io/knative-job-fn@1.6.0 — drops private:true; ready to publish (rename to @constructive-io/fn-app deferred to a later wave to avoid churning internal imports). Verified: pnpm generate && pnpm install && pnpm build passes for all three existing functions (example, simple-email, send-email-link).
There was a problem hiding this comment.
Pull request overview
This PR lays the first foundation for a portable-functions package split by introducing a shared @constructive-io/fn-types package and updating the existing runtime/app packages so they can be published independently. In the broader codebase, it centralizes shared handler/config/registry types while moving fn-runtime off its local type definitions and adding public package metadata to fn-runtime and knative-job-fn.
Changes:
- Adds a new
@constructive-io/fn-typesworkspace package with shared runtime, manifest, registry, and config types plusdefineConfig(). - Updates
@constructive-io/fn-runtimeto import and re-export types from@constructive-io/fn-typesinstead of a localtypes.ts. - Makes
@constructive-io/fn-runtimeand@constructive-io/knative-job-fnpublishable by adding package metadata and READMEs.
Reviewed changes
Copilot reviewed 15 out of 16 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
pnpm-lock.yaml |
Adds lockfile entries for the new fn-types package and runtime dependency updates. |
packages/fn-types/tsconfig.json |
Configures TypeScript output for the new shared types package. |
packages/fn-types/src/runtime.ts |
Defines shared runtime handler/context/logger/server option types. |
packages/fn-types/src/registry.ts |
Adds shared function registry interfaces. |
packages/fn-types/src/manifest.ts |
Adds shared handler manifest interface. |
packages/fn-types/src/index.ts |
Re-exports the public surface of the new types package. |
packages/fn-types/src/config.ts |
Adds shared config types and defineConfig(). |
packages/fn-types/README.md |
Documents the new shared types package. |
packages/fn-types/package.json |
Adds publish metadata, scripts, and dependencies for fn-types. |
packages/fn-runtime/src/server.ts |
Switches runtime server types to import from fn-types. |
packages/fn-runtime/src/index.ts |
Re-exports shared runtime types from fn-types. |
packages/fn-runtime/src/context.ts |
Switches context typing to the shared fn-types package. |
packages/fn-runtime/README.md |
Adds public-facing runtime package documentation. |
packages/fn-runtime/package.json |
Makes fn-runtime publishable and adds fn-types as a dependency. |
packages/fn-app/README.md |
Adds public-facing documentation for knative-job-fn. |
packages/fn-app/package.json |
Makes knative-job-fn publishable and adds package metadata. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "license": "SEE LICENSE IN LICENSE", | ||
| "main": "dist/index.js", | ||
| "types": "dist/index.d.ts", | ||
| "files": [ | ||
| "dist", | ||
| "README.md" | ||
| ], | ||
| "publishConfig": { | ||
| "access": "public" |
| "license": "SEE LICENSE IN LICENSE", | ||
| "main": "dist/index.js", | ||
| "types": "dist/index.d.ts", | ||
| "files": [ | ||
| "dist", | ||
| "README.md" | ||
| ], | ||
| "publishConfig": { | ||
| "access": "public" |
| "types": "dist/index.d.ts", | ||
| "files": [ | ||
| "dist", | ||
| "README.md" |
| - **Runtime** — `FunctionHandler`, `FunctionContext`, `ServerOptions` (used by `@constructive-io/fn-runtime` and handler authors). | ||
| - **Manifest** — `HandlerManifest` (the shape of `functions/<name>/handler.json`). | ||
| - **Config** — `FnConfig`, `FnPreset`, `K8sOptions`, `DockerOptions`, plus a `defineConfig()` helper for `fn.config.ts` files. | ||
| - **Registry** — `FnRegistry`, `FnRegistryEntry` (manifest format consumed by `@constructive-io/fn-job-service`). |
|
|
||
| Runtime contract for Constructive Functions: wraps a typed handler in an Express app with a built-in GraphQL client, structured logger, and Knative job-callback support. | ||
|
|
||
| This is the package handler authors import directly. The `@constructive-io/fn-cli` toolchain stamps out function packages that depend on this runtime. |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Wave 2 of the portable-functions toolkit. Introduces the programmatic
generator that fn-client and fn-cli (next waves) will compose.
@constructive-io/fn-generator@0.1.0
- FnGenerator class with discover()/buildPackages()/buildManifest()/
buildConfigMaps()/buildSkaffold()/apply()/generate() methods.
- Pure builder layer (returns Manifest[]) + a single apply() boundary
that does idempotent file I/O via writeIfChanged() and ensureSymlink().
- Builders: per-function package files (templates + shared + handler
symlinks), functions-manifest.json, per-function and aggregate
functions-configmap.yaml, root skaffold.yaml.
- Honours --only and --packages-only modes (skip k8s/skaffold and
per-function/aggregate configmaps respectively).
- Auto-assigns ports starting at 8081, validates 8080 is reserved
for the job-service, detects port conflicts.
Verification: snapshot test runs FnGenerator against the brasilia
repo's own functions/+templates/ into a tmp dir and asserts byte-
identical output (file contents, symlink targets, skaffold.yaml) vs
scripts/generate.ts. All three assertions pass.
Also updates @constructive-io/fn-types FnRegistry to match the actual
on-disk format ({name, dir, port, type}) and adds optional moduleName
and url for the upcoming job-service rewrite (Wave 4).
scripts/generate.ts is unchanged in this PR — Wave 4 will collapse it
into a one-line shim that delegates to FnGenerator.
Wave 3 of the portable-functions toolkit. Composes fn-generator into a
user-facing layer; closes the loop on the Starship-style split:
fn-types → fn-generator → fn-client → fn-cli.
@constructive-io/fn-client@0.1.0
- FnClient class wraps FnGenerator and adds:
- JSON config loading (fn.config.json or .fnconfig.json) — .ts/.js
loading deferred until we add an esbuild/jiti loader.
- loadManifest() — reads generated/functions-manifest.json.
- defaultProcessDefs() — derives DevProcessDef[] from the manifest.
- build({ only? }) — runs `pnpm -r build` with optional filter.
- dev({ only?, env?, jobService? }) — spawns Node child processes,
returns a DevHandle with pids and a SIGTERM-based stop().
- Job-service is opt-in (jobs-bundle preset is wired by passing
`dev({ jobService: ... })`); functions-only consumers omit it.
- Smoke tests cover discover/generate/manifest round-trip and
defaultProcessDefs derivation.
@constructive-io/fn-cli@0.1.0
- bin/fn executable (chmod +x in build) using minimist for argparse.
- Subcommands: generate (--only, --packages-only), build, dev,
manifest, verify, help.
- Each command is ~10 lines: parse → FnClient method → format output.
- Verified end-to-end against the brasilia repo:
fn generate ⇒ idempotent (0 file changes on rerun)
fn manifest ⇒ prints functions-manifest.json
fn verify ⇒ "OK: 4 function(s) in sync."
The brasilia scripts/generate.ts, scripts/dev.ts, scripts/docker-build.ts
remain in place; Wave 4 will collapse them into one-line shims.
…loader
Wave 4b of the portable-functions toolkit. The hardcoded registry at
job/service/src/index.ts:34-43 was the biggest portability blocker —
adding a function required editing this map. Now the registry is
loaded at runtime from one of three sources, in priority order:
1. FUNCTIONS_REGISTRY env var
Format: "name:moduleName:port,..." (moduleName + port optional;
missing moduleName falls back to @constructive-io/<name>-fn).
2. FUNCTIONS_MANIFEST_PATH env var pointing to a JSON file with the
existing functions-manifest.json shape. Manifest entries can carry
an optional moduleName field; otherwise convention applies.
3. Default: <cwd>/generated/functions-manifest.json (the file the
toolkit's fn-generator produces).
If no source resolves, the registry is empty; lookups still throw the
legacy "Unknown function X" error to preserve existing behaviour.
Implementation lives in job/service/src/registry.ts (new), exporting
loadFunctionRegistry(env, cwd) for testability. job/service/src/index.ts
imports the loader and replaces the const at the top of the file; the
rest of the file is unchanged.
types.ts: FunctionName widened from a literal union to `string` since
names are dynamic. All existing call sites continue to compile.
Tests: tests/integration/job-registry.test.ts covers the three sources,
override behaviour, and the moduleName convention. All 6 cases pass.
Existing unit (4 suites, 19 tests) and integration (runtime.test.ts, 3
tests) still pass.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 51 out of 53 changed files in this pull request and generated 4 comments.
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const rootDir = opts.rootDir ?? process.cwd(); | ||
| const templatesDir = opts.templatesDir ?? path.resolve(rootDir, 'templates'); | ||
| return { | ||
| rootDir, | ||
| functionsDir: opts.functionsDir ?? path.resolve(rootDir, 'functions'), | ||
| outputDir: opts.outputDir ?? path.resolve(rootDir, 'generated'), | ||
| templatesDir, | ||
| sharedDir: path.resolve(templatesDir, 'shared'), |
| if (!opts.packagesOnly) { | ||
| manifests.push(this.buildManifest(fns)); | ||
| // Per-function and aggregate configmaps + skaffold are skipped in --only mode | ||
| // (matches legacy generator: those files reflect the whole repo). | ||
| if (!opts.only) { | ||
| manifests.push(...this.buildConfigMaps(fns)); |
| const BASELINE_GENERATED = path.join(ROOT, 'generated'); | ||
| const BASELINE_SKAFFOLD = path.join(ROOT, 'skaffold.yaml'); |
| - **Runtime** — `FunctionHandler`, `FunctionContext`, `ServerOptions` (used by `@constructive-io/fn-runtime` and handler authors). | ||
| - **Manifest** — `HandlerManifest` (the shape of `functions/<name>/handler.json`). | ||
| - **Config** — `FnConfig`, `FnPreset`, `K8sOptions`, `DockerOptions`, plus a `defineConfig()` helper for `fn.config.ts` files. | ||
| - **Registry** — `FnRegistry`, `FnRegistryEntry` (manifest format consumed by `@constructive-io/fn-job-service`). |
Wave 5a of the portable-functions toolkit. CI now publishes the six @constructive-io/fn-* packages to npm with provenance when an `fn-v*` tag is pushed. The workflow can also be triggered via workflow_dispatch with dry_run=true for pre-release verification. Publish order (deps first): fn-types → fn-app (knative-job-fn) → fn-runtime → fn-generator → fn-client → fn-cli Each package already has publishConfig.access=public and files[] set from its respective Wave 1/2/3 commit. Also adds docs/portable-functions-toolkit.md: package map, customer- repo flow, registry loader behaviour, release procedure, and a manual verification checklist for the first release. Documents the deferred follow-ups (Wave 4c k8s manifest migration, .ts config loader, fn init/dockerfile/k8s standalone subcommands, fn-templates packaging).
Wave 6 of the portable-functions toolkit. Adds the user-facing way
to scaffold a new handler — bundled templates + minimal glue.
@constructive-io/fn-cli@0.1.0:
- New runtime dep on genomic@^5.3.11 (same engine pgpm init uses).
- Bundled templates at packages/fn-cli/templates/handler/:
- node-graphql/ (.boilerplate.json, handler.json, handler.ts)
- python/ (.boilerplate.json, handler.json, handler.py)
- New `fn init <name>` subcommand:
- Positional name OR --name flag
- --type=node-graphql|python (default: node-graphql)
- --description=<d> (optional)
- --force overwrites existing dir
- --no-tty / CI=true detection (matches pgpm init's pattern)
- Refuses to overwrite without --force; clean error
- Honors functionsDir from fn.config.json
- Tarball includes templates/ via files: ["dist", "templates", "README.md"]
so the bundled handler templates ship with the published package.
- Help text and README updated.
Tests: packages/fn-cli/__tests__/init.test.ts — 7 unit cases covering
node-graphql + python scaffolding, dir-exists guard, --force,
unknown-type rejection, missing-name error, and custom functionsDir
from fn.config.json. All pass.
Verified end-to-end against /tmp:
$ fn init hello --no-tty → functions/hello/{handler.json,ts}
$ fn init pyfn --type python --no-tty → functions/pyfn/{handler.json,py}
$ fn init hello --no-tty → refuses (exit 1)
$ fn init hello --no-tty --force --description "second pass" → overwrites
Wave 7 of the portable-functions toolkit. The existing test workflow ran pnpm test:unit and pnpm test:integration only, which skipped the package-local jest suites (fn-generator's snapshot, fn-client's API tests, fn-cli's init tests). That gap is now closed. Two new jobs in .github/workflows/test.yaml: 1. **toolkit** — installs, builds all six fn-* packages in dependency order, then runs jest in fn-generator, fn-client, and fn-cli. This catches regressions in the toolkit code paths that the existing tests:unit/integration jobs don't reach. 2. **fn-init-e2e** — builds fn-cli, then runs the new binary integration suite at tests/integration/fn-init.test.ts. The suite spawns the compiled `dist/bin/fn.js` against a tmpdir and asserts: - node-graphql scaffolding produces handler.json + handler.ts - python scaffolding produces handler.py with type=python - duplicate scaffold without --force fails with exit 1 - --force overwrites and updates description - fn generate finds the just-scaffolded function This proves the bundled templates resolve correctly when fn is invoked from a non-package cwd — the load-bearing scenario for end-user installs. All 5 binary tests pass locally. Existing CI jobs (build/lint, unit, integration) untouched — these run in addition.
Wave 8 of the portable-functions toolkit. Updates user-facing docs so that `fn init` is the first thing a new user sees, not the last. - docs/portable-functions-toolkit.md: replace the "Customer repo experience" section with a "Quick start" that begins with `fn init send-welcome --no-tty` rather than assuming the user has hand-authored a handler.json. Add the full CLI surface table including `init` and its flags. Note the genomic-via-pgpm-init alignment so users who know `pgpm init` carry their knowledge over. - README.md: lead with the in-another-repo flow. The brasilia-as- dogfood instructions stay below as the second quick-start. New paragraph at top points at the toolkit guide. The starter-kit / pgpm init scaffold-the-whole-project flow is deliberately left as future work — the addendum in the plan file documents it but it lands in its own follow-on.
| // Positional name first, then --name flag. | ||
| const positional = typeof args._[1] === 'string' ? args._[1] : undefined; | ||
| const name = positional ?? (typeof args.name === 'string' ? args.name : ''); | ||
| if (!name) { | ||
| process.stderr.write( | ||
| 'fn init: function name is required (positional or --name=<name>)\n' | ||
| ); | ||
| return 1; | ||
| } | ||
|
|
||
| const type = typeof args.type === 'string' ? args.type : 'node-graphql'; | ||
| if (!isHandlerType(type)) { | ||
| process.stderr.write( | ||
| `Unknown type "${type}". Available: ${KNOWN_HANDLER_TYPES.join(', ')}\n` | ||
| ); | ||
| return 1; | ||
| } | ||
|
|
||
| const templateDir = path.join(TEMPLATES_ROOT, type); | ||
| if (!fs.existsSync(templateDir)) { | ||
| process.stderr.write( | ||
| `Bundled template missing at ${templateDir}. Reinstall @constructive-io/fn-cli.\n` | ||
| ); | ||
| return 1; | ||
| } | ||
|
|
||
| const client = buildClient(args); | ||
| const functionsDir = client.config.functionsDir | ||
| ? path.resolve(client.rootDir, client.config.functionsDir) | ||
| : path.resolve(client.rootDir, 'functions'); | ||
| const outDir = path.join(functionsDir, name); | ||
|
|
| // Genomic strips the ____ wrapping when matching argv keys, so plain | ||
| // names ('name', 'description', …) are the right shape. version defaults | ||
| // to 0.1.0 from the .boilerplate.json; users can edit handler.json after. | ||
| const argv: Record<string, string> = { | ||
| name, | ||
| version: '0.1.0', | ||
| description: typeof args.description === 'string' ? args.description : '', | ||
| }; | ||
|
|
||
| const templatizer = new Templatizer(); | ||
| await templatizer.process(templateDir, outDir, { | ||
| argv, | ||
| noTty: detectNoTty(args), | ||
| }); |
| it('fn generate finds the just-scaffolded function', () => { | ||
| runFn(['init', 'discoverme', '--no-tty'], tmpRoot); | ||
| // fn generate needs a templates/ dir to do its full pipeline. For | ||
| // this binary test we only verify discovery — the scaffolded | ||
| // handler.json is enough for the scanner to enumerate it. | ||
| fs.mkdirSync(path.join(tmpRoot, 'templates', 'node-graphql'), { | ||
| recursive: true, | ||
| }); | ||
| fs.mkdirSync(path.join(tmpRoot, 'templates', 'shared'), { | ||
| recursive: true, | ||
| }); | ||
| // Empty templates → generator runs but produces no per-fn files; | ||
| // it still emits the manifest. Use --packages-only to skip k8s. | ||
| const result = runFn( | ||
| ['generate', '--only', 'discoverme', '--packages-only'], | ||
| tmpRoot |
| }); | ||
|
|
||
| // One-shot | ||
| generator.generate(); // all functions | ||
| generator.generate({ only: 'simple-email' }); // single | ||
| generator.generate({ packagesOnly: true }); // skip k8s/skaffold | ||
|
|
| This repo is also the source of the **Portable Functions Toolkit**: a set of `@constructive-io/fn-*` npm packages that any external repo can `pnpm add` to get the same code-gen + Docker + k8s pipeline against its own `functions/` directory. See [docs/portable-functions-toolkit.md](docs/portable-functions-toolkit.md) for the full toolkit guide. | ||
|
|
||
| ## Quick start (in another repo) | ||
|
|
||
| ```bash | ||
| pnpm add -D @constructive-io/fn-cli | ||
| pnpm add @constructive-io/fn-runtime | ||
|
|
||
| pnpm fn init send-welcome --no-tty # scaffold functions/send-welcome/ | ||
| pnpm fn generate # stamp out generated/<name>/ packages | ||
| pnpm install # link the new workspaces | ||
| pnpm fn build # compile | ||
| pnpm fn dev # run functions as local Node processes | ||
| ``` |
| ## Repo layout the toolkit expects | ||
|
|
||
| ``` | ||
| my-app/ | ||
| ├── functions/ | ||
| │ └── send-welcome/ | ||
| │ ├── handler.json # {"name":"send-welcome","version":"0.1.0","type":"node-graphql"} | ||
| │ └── handler.ts # default-exported FunctionHandler | ||
| ├── fn.config.json # FnConfig (typed via fn-types) — optional | ||
| └── package.json | ||
| ``` | ||
|
|
The Build <function>-fn matrix jobs were failing in "Post Setup
Node.js" with:
Path Validation Error: Path(s) specified in the action for
caching do(es) not exist, hence no cache is being saved.
Root cause: pnpm/action-setup@v6 ran before actions/setup-node@v5,
exposing PNPM_HOME. setup-node@v5 then auto-detected pnpm via the
packageManager field in root package.json and enabled pnpm-store
caching — but these jobs never run pnpm install (they only invoke
node --experimental-strip-types scripts/generate.ts). The store
path never gets created, and v5 promotes the missing-path warning
to an error during cache save.
Fix: remove pnpm/action-setup from docker.yaml since the workflow
doesn't use pnpm on the runner. The actual function builds run
inside Docker, which installs pnpm itself via
"npm install -g pnpm@<v>". simple-email had been passing only by
accident (cache hit from a previous warm run); the other three were
deterministic failures.
Previous commit removed pnpm/action-setup, but that broke a different
thing: setup-node@v5 still tries to invoke pnpm at setup time when it
auto-detects the packageManager field in package.json, and now no
pnpm exists on PATH ("Unable to locate executable file: pnpm").
Restoring pnpm setup AND pinning setup-node to v4. v4 doesn't have
the aggressive auto-cache behavior that caused the original "Path
Validation Error" failure. The other workflows (ci.yaml, test.yaml,
publish.yaml) keep v5 because they actually run `pnpm install` so
the store path exists when v5 tries to cache it.
| const parseEnvRegistry = (raw: string): FunctionRegistry => { | ||
| const out: FunctionRegistry = {}; | ||
| for (const pair of raw.split(',')) { | ||
| const trimmed = pair.trim(); | ||
| if (!trimmed) continue; | ||
| const [name, moduleName, portStr] = trimmed.split(':').map((s) => s.trim()); | ||
| if (!name) continue; | ||
| const portNumber = portStr ? Number(portStr) : NaN; | ||
| out[name] = { | ||
| moduleName: moduleName || conventionalModuleName(name), | ||
| defaultPort: Number.isFinite(portNumber) ? portNumber : 0, | ||
| }; | ||
| } |
| /** Identity helper for editor autocomplete in fn.config.ts files. */ | ||
| export const defineConfig = (config: FnConfig): FnConfig => config; |
| export const assignAndValidatePorts = ( | ||
| manifests: HandlerManifest[], | ||
| defaultTemplate: string | ||
| ): void => { | ||
| const usedPorts = new Set<number>( | ||
| manifests.filter((m) => m.port).map((m) => m.port as number) | ||
| ); | ||
| let nextPort = usedPorts.size > 0 ? Math.max(...usedPorts) + 1 : 8081; | ||
| for (const m of manifests) { | ||
| if (!m.port) { | ||
| while (usedPorts.has(nextPort)) nextPort++; | ||
| m.port = nextPort; | ||
| usedPorts.add(nextPort); | ||
| nextPort++; | ||
| } | ||
| } | ||
|
|
||
| const portToFunction = new Map<number, string>(); | ||
| for (const m of manifests) { | ||
| if (m.port === 8080) { | ||
| throw new Error( | ||
| `Function "${m.name}" uses port 8080 which is reserved for job-service.` | ||
| ); | ||
| } | ||
| if (portToFunction.has(m.port as number)) { | ||
| throw new Error( | ||
| `Port ${m.port} conflict: "${m.name}" and "${portToFunction.get(m.port as number)}".` | ||
| ); | ||
| } | ||
| portToFunction.set(m.port as number, m.name); | ||
| } | ||
| // suppress unused var lint by referencing defaultTemplate consumer side | ||
| void defaultTemplate; | ||
| }; |
Summary
First wave of the portable-functions toolkit, modeled on Starship's V2 split (types → generator → client → cli). This PR establishes the types layer that every subsequent package depends on, and unblocks the runtime libraries for npm publishing.
@constructive-io/fn-types@0.1.0— single source-of-truth types:FunctionHandler,FunctionContext,FunctionLogger,ServerOptions,HandlerManifest,FnRegistry/FnRegistryEntry,FnConfig+defineConfig()helper for typedfn.config.ts. No logic, no shell-out.@constructive-io/fn-runtime@1.2.0— dropsprivate:true; depends onfn-types; source files import types from@constructive-io/fn-typesinstead of the localsrc/types.ts(now removed).@constructive-io/knative-job-fn@1.6.0— dropsprivate:true; addspublishConfig. (Rename to@constructive-io/fn-appdeferred to a later wave to avoid churning internal imports.)The full design lives in the approved plan; this PR is intentionally small (foundation only). Follow-up waves will add
fn-generator,fn-client,fn-cli, and replace the hardcoded registry injob/service/src/index.ts:34-43and the hand-writtenk8s/base/functions/*.yamlwith builder-class output.Test plan
pnpm --filter @constructive-io/fn-types build— cleanpnpm --filter @constructive-io/fn-runtime build— cleanpnpm generate && pnpm install && pnpm build— all three existing functions (example,simple-email,send-email-link) still build end-to-end with no regressionspnpm publish --dry-runfrom each newly-public package shows no leakedworkspace:references