Skip to content

DigitalArsenal/sdn-flow

Repository files navigation

sdn-flow

sdn-flow composes signed modules into typed flow graphs, compiles those graphs into deployable WebAssembly runtimes, and provides host surfaces for loading, starting, and serving those flows.

It builds on space-data-module-sdk, which defines the module artifact, single-file bundle, capability vocabulary, and module ABI. sdn-flow owns the flow graph, compiled flow descriptors, hosted runtime planning, deployment client, and installed-flow startup layers.

Project Scope

This repo covers the full flow path:

  • normalized flow programs and runtime descriptors
  • designer-facing graph inspection and requirement summaries
  • a hosted flow editor runtime with embeddable fetch/HTTP surfaces
  • compilation from flow graph to one generated C++ translation unit
  • emception-backed compilation to one deployable .wasm runtime artifact
  • deployment packaging built on canonical module auth and transport records
  • hosted runtime planning for browser, JS server, Go-oriented, and vendor runtime profiles
  • installed-flow startup surfaces for browser, Deno, Bun, and Node hosts

Flow Artifact

A compiled flow is one runtime artifact with:

  • WebAssembly bytes
  • an embedded FlatBuffer manifest
  • exported runtime, invocation, and descriptor symbols
  • embedded signed module dependencies
  • deployment metadata for local or remote rollout

The compiler path uses a native C++ source generator compiled to WebAssembly plus an emception-compatible compiler adapter, so the flow graph always converges on the same runtime artifact format.

Runtime Portability

Flows generated by sdn-flow are runtime-neutral compiled WebAssembly artifacts. A host can run the output anywhere it can:

  • instantiate WebAssembly
  • read FlatBuffers
  • implement the canonical module and flow ABI surface

That is the same runtime family already exercised in the companion flatbuffers/wasm work:

  • browser
  • Node.js
  • C#
  • Go
  • Java
  • Kotlin
  • Python
  • Rust
  • Swift

This repo currently ships turnkey startup surfaces for browser, Deno, Bun, and Node, and it includes environment profiles for go-sdn and WasmEdge-style hosts. Non-JS runtimes consume the same compiled artifact through the runtime, invocation, and descriptor ABIs exposed from src/host.

For runtimes that cannot embed WasmEdge directly, sdn-flow now ships only the thin compatibility wrappers that are actually needed:

  • sdn-flow/wrappers/browser
  • sdn-flow/wrappers/bun
  • sdn-flow/wrappers/deno

Go, Rust, Java, Python, Node, and C/C++ stay on direct WasmEdge SDK paths. Kotlin stays on the JVM path through Java interop, and C# / Swift stay on native interop against the WasmEdge C API. See WasmEdge-Like Wrappers.

Relationship To Module SDK

space-data-module-sdk is the canonical module contract:

  • manifest schema and embedded manifest exports
  • module capability vocabulary
  • compliance rules
  • module signing and appended SDS publication records (REC, PNM, ENC)
  • sds.bundle single-file packaging
  • module hostcall/import ABI

sdn-flow layers flow composition on top of that contract:

  • program structure and graph normalization
  • compiled flow runtime descriptor and invocation ABI
  • dependency composition for multi-module runtimes
  • deployment clients and hosted runtime plans
  • startup surfaces for hosts that launch compiled flows

Distribution

sdn-flow is currently consumed from source checkouts and submodules. This repo is not currently published to npm, so do not rely on npm install sdn-flow from the public registry.

Use one of these source-based dependency paths instead:

npm install ../sdn-flow
{
  "dependencies": {
    "sdn-flow": "file:../sdn-flow"
  }
}

If you need a remote dependency outside the sibling-workspace flow, use a git reference rather than an npm registry version.

The editor runtime is also available through the dedicated export surface:

import { createSdnFlowEditorFetchHandler } from "sdn-flow/editor";

The WasmEdge-like compatibility wrappers are available as explicit installable subpath exports:

import { startBrowserWasmEdgeLikeRuntime } from "sdn-flow/wrappers/browser";
import { startBunWasmEdgeLikeRuntime } from "sdn-flow/wrappers/bun";
import { startDenoWasmEdgeLikeRuntime } from "sdn-flow/wrappers/deno";

Hosted Editor

sdn-flow includes a packaged editor runtime that serves the flow canvas, inspector, generated-runtime preview, and debug sidebar from embedded assets. That runtime has two intended deployment modes:

  • embed it behind any fetch-compatible host with createSdnFlowEditorFetchHandler(...)
  • launch it directly with a host wrapper such as sdn-flow-editor or the Deno single-file entrypoint in tools/sdn-flow-editor.ts

Example embed:

import { createSdnFlowEditorFetchHandler } from "sdn-flow/editor";

const handler = createSdnFlowEditorFetchHandler({
  basePath: "/editor",
  title: "Mission Control",
  engineLabel: "Embedded runtime",
  initialFlow: flowJson,
  async onExport(flow) {
    await saveFlow(flow);
    return { saved: true };
  },
  async onDeploy(deployment) {
    await queueDeployment(deployment);
    return { queued: true };
  },
});

Node-oriented local startup:

sdn-flow-editor --flow ./examples/flows/single-plugin-flow.json
sdn-flow-editor --port 9090 --base-path /editor

Executable app startup from the repo:

npm run start -- --flow ./examples/flows/single-plugin-flow.json

That command builds one Deno executable into generated-tools/sdn-flow-editor, launches it, and opens the editor URL in your browser. Use npm run build:editor-executable when you want the standalone binary without launching it.

Direct Deno single-file build:

deno compile --allow-net --allow-read --output sdn-flow-editor ./tools/sdn-flow-editor.ts
./sdn-flow-editor --flow ./examples/flows/single-plugin-flow.json

The Deno launcher compiles into one executable while serving the same embedded editor asset set used by the embeddable fetch handler and the Node host.

Compile A Flow

import {
  EmceptionCompilerAdapter,
  SignedArtifactCatalog,
  normalizeProgram,
} from "sdn-flow";

const program = normalizeProgram(flowJson);

const compiler = new EmceptionCompilerAdapter({
  emception,
  artifactCatalog: new SignedArtifactCatalog(),
  manifestBuilder: async () => manifestBytes,
});

const artifact = await compiler.compile({
  program,
});

The compiled artifact carries the exported symbols needed by hosts to inspect and run the flow without reinterpreting the source graph.

Start A Host

For JS-family hosts, the quickest path is a workspace file plus the environment-neutral auto-host entrypoint:

import { startInstalledFlowAutoHost } from "sdn-flow";

await startInstalledFlowAutoHost({
  workspacePath: "./workspace.json",
});

For Node-based startup from the command line:

sdn-flow-host --workspace ./workspace.json
sdn-flow-host --workspace ./workspace.json --engine node

That CLI is a thin wrapper over startInstalledFlowAutoHost(...) with startup summary output and SIGINT / SIGTERM shutdown handling.

The editor runtime has its own CLI alongside it:

sdn-flow-editor --flow ./examples/flows/single-plugin-flow.json

Host Surfaces

The host package is organized around a few layers:

  • createInstalledFlowHost(...) Discover or register plugin packages, load a program, and start the runtime.
  • createInstalledFlowService(...) Keep timer and HTTP-trigger services running for a loaded flow.
  • createInstalledFlowApp(...) Boot from workspace.json, persist startup state, and refresh the runtime against workspace changes.
  • startInstalledFlowAutoHost(...) Select the matching startup surface for browser, Deno, Bun, or Node from workspace metadata.
  • createNpmPackageManager(...) Tie workspace package references to real install, update, and remove flows.

For hosts that load compiled artifacts directly, bindCompiledRuntimeAbi(...), bindCompiledInvocationAbi(...), and bindCompiledDescriptorAbi(...) expose the low-level contract used to inspect and execute a compiled flow artifact.

Runtime Families

Layer Coverage
Compiled flow artifact Browser, Node.js, C#, Go, Java, Kotlin, Python, Rust, Swift
Turnkey startup surfaces in this repo Browser, Deno, Bun, Node
Checked-in environment profiles sdn-js, go-sdn, WasmEdge-style vendor profile

The built-in host planners focus on the engines exercised by the current examples and startup helpers. The compiled flow artifact itself stays portable across the broader runtime set above.

Examples

Notable environment profiles:

Package Surface

  • sdn-flow
  • sdn-flow/editor
  • sdn-flow/runtime
  • sdn-flow/designer
  • sdn-flow/host
  • sdn-flow/auth
  • sdn-flow/transport
  • sdn-flow/deploy
  • sdn-flow/compiler
  • sdn-flow/compliance

Documentation

Development

npm install
npm test

Node.js >=20 is required. The compile path uses ../emception for generated C++ to WebAssembly builds.

License

MIT

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors