Skip to content

phips-git/dynamic-pipeline-processing-prototype

Repository files navigation

Brand Insights Prototype

Bun-based server that handles HTTP requests, async item pipeline processing & serves a simple React frontend.

Prerequisites

  • Bun 1.3+
  • No external database needed (uses local SQLite database via Bun)
  • Optional: Devbox for a pinned Bun toolchain (run devbox shell)

Install

bun install

Run (development)

bun dev
  • Automatically applies pending migrations before starting
  • Serves on http://localhost:3000
  • Check server logs for pipeline processing output

Run (production)

bun run build
bun run start
  • Sets NODE_ENV=production
  • Outputs to dist/
  • Automatically applies pending migrations before starting

Database

  • bun run db:generate — Generates Drizzle migration files based on current schema changes.
  • bun run db:migrate — Applies all pending migrations to the local SQLite database.
  • Auto-migrations: bun dev and bun run start automatically run migrations before starting the server (via predev and prestart scripts).
  • Storage: The database is stored locally as a SQLite file on disk (database/local.db).

Extension

Pipelines are designed to be extensible via custom processors.

To add a new processor:

  1. Implement PipelineItemProcessor interface

    • Create a new class (e.g., CustomProcessor) that complies with PipelineItemProcessor interface.
    • Ensure it exposes the required methods (e.g., a process method and any lifecycle hooks expected by the pipeline).
    • Processors are able to stop the pipeline execution by returning {stop: true} from their process method.
  2. Register it in the processor registry

    • Add your processor to processor-registry.ts so the system can resolve it by class name.
  3. Reference it from a pipeline-specific config

    • In the pipeline’s configuration file (like item-pipeline-config.ts), reference your processor by its class name.
    • Include any settings the processor needs under its config entry.

At runtime, the pipeline builder will resolve processors from the registry based on the pipeline config and construct the pipeline dynamically.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published