Skip to content

fix(kernel): handle messages in queue after kernel restart#611

Merged
sirtimid merged 3 commits intomainfrom
sirtmid/kernel-queue-initialization
Aug 28, 2025
Merged

fix(kernel): handle messages in queue after kernel restart#611
sirtimid merged 3 commits intomainfrom
sirtmid/kernel-queue-initialization

Conversation

@sirtimid
Copy link
Copy Markdown
Contributor

This PR addresses a race condition in kernel initialization where the run queue would start processing messages before all vats were fully initialized, potentially causing messages to fail when sent to vats that weren't ready yet.

The key change reorders the initialization sequence:

  1. Start all vats that were previously running
  2. Then start the kernel queue processing

This prevents the queue processor from attempting to deliver messages to vats that haven't been fully initialized yet, which was causing intermittent failures in scenarios involving kernel restarts with pending messages.

Added a new test that verifies the kernel correctly processes messages that were enqueued in the database before a restart. The test manually injects a message into the run queue storage, restarts the kernel, and confirms the message is processed correctly with the expected counter increment.

@sirtimid sirtimid requested a review from a team as a code owner August 28, 2025 19:32
await kernel1.stop();
// verify that the message is in the database
expect(kernelStore.kv.get('queue.run.3')).toBeDefined();
await kernel1.stop();
Copy link
Copy Markdown
Contributor

@grypez grypez Aug 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does reading this queue item from the kernel store not work after the kernel stops?

Copy link
Copy Markdown
Contributor Author

@sirtimid sirtimid Aug 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does work actually. I just wanted to show the state after the kernel has stopped.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet

Copy link
Copy Markdown
Contributor

@grypez grypez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Order must be maintained.

@sirtimid sirtimid merged commit 6dd91c1 into main Aug 28, 2025
23 checks passed
@sirtimid sirtimid deleted the sirtmid/kernel-queue-initialization branch August 28, 2025 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants