Send final state to lightning

[taylordowns2000]

Short Description

Changes RUN_COMPLETE event to send actual final state (using the nice engine implementation that the CLI already uses) instead of the unused dataclip ID reference. This enables Lightning to receive the complete workflow result directly, supporting branching workflows with multiple leaf nodes.

Implementation Details

Type Definitions

• lexicon/lightning.d.ts: Replace final_dataclip_id?: string with final_state?: any in RunCompletePayload

Worker Implementation

• ws-worker/src/events/run-complete.ts: Send final_state: event.state (from runtime) instead of final_dataclip_id
• ws-worker/src/events/run-error.ts: Remove final_dataclip_id field from error case

Mock/Test Infrastructure

• ws-worker/src/mock/runtime-engine.ts: Include state in workflow-complete event dispatch
• ws-worker/test/events/run-complete.test.ts: Update tests + add coverage for linear/branching workflows, serialization, and binary data
• ws-worker/test/lightning.test.ts: Assert final_state is object instead of final_dataclip_id is string

Lightning Mock

• lightning-mock/src/api-dev.ts: Read final_state from payload instead of looking up dataclip by ID
• lightning-mock/src/api-sockets.ts: Store final_state directly instead of dataclip lookup
• lightning-mock/test/channels/run.test.ts: Send final_state in test payloads instead of final_dataclip_id

QA Notes

This doesn't do anything interesting unless you're connecting the worker to lightning on the sync-mode branch: OpenFn/lightning#3785

AI Usage

Please disclose how you've used AI in this work (it's cool, we just want to know!):

• Code generation (copilot but not intellisense)
• Learning or fact checking
• Strategy / design
• Optimisation / refactoring
• Translation / spellchecking / doc gen
• Other
• I have not used AI

You can read more details in our Responsible AI Policy

Release branch checklist

Delete this section if this is not a release PR.

If this IS a release branch:

• Run pnpm changeset version from root to bump versions
• Run pnpm install
• Commit the new version numbers
• Run pnpm changeset tag to generate tags
• Push tags git push --tags

Tags may need updating if commits come in after the tags are first generated.

[taylordowns2000]

@josephjclark , would love your eyes here. i don't understand mock at all. this isn't super urgent, but i'm hoping it's not actually a big change. there should be no impact on lighting, until this gets merged: OpenFn/lightning#3785

[josephjclark]

QA notes:

• main runs absolutely fine with these changes
• main with multiple outputs isn't very interesting, as our man @taylordowns2000 says. But nothing breaks.

I'll leave it up to lightning to decide what to do with the new final_state. Everything here looks rosy to me