Skip to content

Conversation

@jumski
Copy link
Contributor

@jumski jumski commented Dec 3, 2025

TL;DR

Add automatic flow compilation at worker startup with the ability to opt-out.

What changed?

  • Added a new ensureCompiledOnStartup configuration option to FlowWorkerConfig (defaults to true)
  • When enabled (default), workers call pgflow.ensure_flow_compiled() at startup to verify flows are up-to-date
  • In development mode, mismatched flows are automatically recompiled
  • In production mode, mismatches cause errors
  • Added comprehensive tests for the new configuration option

How to test?

  1. Create a flow worker with default settings to see automatic compilation:

    const worker = createFlowWorker(MyFlow, { /* config */ });
  2. Opt-out of automatic compilation:

    const worker = createFlowWorker(MyFlow, {
      ensureCompiledOnStartup: false,
      // other config
    });
  3. Run tests to verify behavior:

    deno test pkgs/edge-worker/tests/integration/flow/compilationAtStartup.test.ts
    deno test pkgs/edge-worker/tests/unit/FlowWorkerLifecycle.compilation.test.ts
    

Why make this change?

This change improves the developer experience by ensuring flows are properly compiled at worker startup. It helps catch flow definition mismatches early, automatically recompiling in development environments while failing fast in production. The opt-out option provides flexibility for environments where flows are pre-compiled via CLI or other means.

@changeset-bot
Copy link

changeset-bot bot commented Dec 3, 2025

🦋 Changeset detected

Latest commit: 0b84bb0

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 6 packages
Name Type
@pgflow/core Patch
@pgflow/dsl Patch
@pgflow/edge-worker Patch
pgflow Patch
@pgflow/client Patch
@pgflow/example-flows Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link
Contributor Author

jumski commented Dec 3, 2025


How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

  • merge:queue - adds this PR to the back of the merge queue
  • hotfix:queue - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

@nx-cloud
Copy link

nx-cloud bot commented Dec 3, 2025

View your CI Pipeline Execution ↗ for commit 0b84bb0

Command Status Duration Result
nx run edge-worker:test:integration ✅ Succeeded 3m 45s View ↗
nx run client:e2e ✅ Succeeded 1m 8s View ↗
nx affected -t verify-exports --base=origin/mai... ✅ Succeeded 3s View ↗
nx affected -t build --configuration=production... ✅ Succeeded 25s View ↗
nx affected -t lint typecheck test --parallel -... ✅ Succeeded 23s View ↗
nx run cli:e2e ✅ Succeeded 4s View ↗
nx run core:pgtap ✅ Succeeded <1s View ↗
nx run edge-worker:e2e ✅ Succeeded 46s View ↗

☁️ Nx Cloud last updated this comment at 2025-12-04 11:28:27 UTC

@graphite-app
Copy link
Contributor

graphite-app bot commented Dec 4, 2025

Merge activity

#499)

### TL;DR

Add automatic flow compilation at worker startup with the ability to opt-out.

### What changed?

- Added a new `ensureCompiledOnStartup` configuration option to `FlowWorkerConfig` (defaults to `true`)
- When enabled (default), workers call `pgflow.ensure_flow_compiled()` at startup to verify flows are up-to-date
- In development mode, mismatched flows are automatically recompiled
- In production mode, mismatches cause errors
- Added comprehensive tests for the new configuration option

### How to test?

1. Create a flow worker with default settings to see automatic compilation:
   ```typescript
   const worker = createFlowWorker(MyFlow, { /* config */ });
   ```

2. Opt-out of automatic compilation:
   ```typescript
   const worker = createFlowWorker(MyFlow, {
     ensureCompiledOnStartup: false,
     // other config
   });
   ```

3. Run tests to verify behavior:
   ```
   deno test pkgs/edge-worker/tests/integration/flow/compilationAtStartup.test.ts
   deno test pkgs/edge-worker/tests/unit/FlowWorkerLifecycle.compilation.test.ts
   ```

### Why make this change?

This change improves the developer experience by ensuring flows are properly compiled at worker startup. It helps catch flow definition mismatches early, automatically recompiling in development environments while failing fast in production. The opt-out option provides flexibility for environments where flows are pre-compiled via CLI or other means.
@graphite-app graphite-app bot force-pushed the 12-01-add_advisory_lock_to_ensure_flow_compiled_to_prevent_race_conditions branch from bcc4f76 to cf8811b Compare December 4, 2025 11:15
@graphite-app graphite-app bot force-pushed the 12-01-add_ensurecompiledonstartup_config_flag_to_opt-out_of_auto-compilation branch from e4cea89 to 0b84bb0 Compare December 4, 2025 11:16
@graphite-app graphite-app bot changed the base branch from 12-01-add_advisory_lock_to_ensure_flow_compiled_to_prevent_race_conditions to main December 4, 2025 11:32
@graphite-app graphite-app bot merged commit 0b84bb0 into main Dec 4, 2025
12 checks passed
@graphite-app graphite-app bot deployed to production December 4, 2025 11:33 Active
@github-actions
Copy link
Contributor

github-actions bot commented Dec 4, 2025

🚀 Production Deployment: Website

Successfully deployed to production!

🔗 Production URL: https://pgflow.dev

📝 Details:

  • Commit: 0b84bb0503d49ff1d7a8ca54e1beb3fd8776bc58
  • View Logs

Deployed at: 2025-12-04T11:09:29Z

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants