fix(pegboard): convert ws manager socket to unix socket

[MasterPtato]

Changes

[MasterPtato]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• fix: get pb manager test working e2e again #2721
• fix: convert runner protocol to protobuf #2702
• fix: even more actor api changes #2714
• chore: flatten actor api spec #2698
• fix: increase default actor kill timeout to 5s #2701
• fix: dont clear ingress ports on reschedule #2700
• fix: add cpu constraints to oci config #2697
• fix: api changes #2692
• fix: fix logs for new actors #2649
• feat: add actor queue #2644
• feat: add actor kv to runners #2638
• feat: get multi actors working e2e on docker compose #2637
• fix: add udp ports to docker compose #2635
• chore: fix merge conflicts #2634
• fix: get new actors working on docker compose #2623
• fix: figure out actor log shipping #2622
• fix(pegboard): convert ws manager socket to unix socket #2478 👈 (View in Graphite)
• fix(pb): get new actor ids working e2e #2471
• feat: implement new actor ids #2467
• feat(pb): get multi-actors working e2e, add resources for builds #2465
• feat(pb): add runner logs #2447
• feat: add runners to pb protocol #2433
• chore(pegboard): add exit code log from container runner #2477
• fix(workflows): fix branch version bug #2475
• chore(pegboard): replace computing image size manually with using tar bytes read #2625
• fix: make pb image downloads random, remove prewarm from pb protocol #2784
• fix: clean up worker threads in the event of an error #2783
• fix: make query_as macros retry #2773
• main

---

How to use the Graphite Merge Queue

Add the label merge-queue to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

[cloudflare-workers-and-pages]

Deploying rivet with    Cloudflare Pages

Latest commit:	d0f06e4
Status:	🚫  Build failed.

View logs

[cloudflare-workers-and-pages]

Deploying rivet-studio with    Cloudflare Pages

Latest commit:	d0f06e4
Status:	🚫  Build failed.

View logs

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Explain this complex logic.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai explain this code block.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read src/utils.ts and explain its main purpose.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai generate docstrings to generate docstrings for this PR.
• @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[claude]

Claude finished @MasterPtato's task —— View job

---

PR Review: WebSocket to Unix Socket Conversion

Reviewing this pull request that converts WebSocket manager sockets to Unix sockets in the pegboard system.

Review Tasks:

• Read repository guidelines (CLAUDE.md)
• Analyze core architecture changes (WebSocket → Unix socket)
• Review security implications of socket conversion
• Examine error handling and resource management
• Check performance considerations
• Address existing automated review comments
• Assess test coverage impact
• Provide comprehensive review feedback

---

Summary

This PR implements a significant architectural change in the pegboard system, converting from WebSocket-based communication to Unix domain sockets for runner processes. The manager still connects to pegboard via WebSocket, but runners now communicate with the manager through Unix sockets with a length-delimited protocol.

Architecture Analysis ✅

Key Changes:

• Echo client now uses UnixStream instead of WebSocket connections (echo/src/main.rs)
• Runner communication migrated from WebSocket to Unix socket protocol (runner_protocol.rs)
• Manager handles Unix socket connections with frame-based communication (runner/mod.rs)
• JWT authentication removed (claims.rs deleted) - simplifies auth but removes token-based security
• Configuration changes to support Unix socket paths instead of WebSocket URLs

Impact: This change reduces network overhead and improves security by moving from network-based to local-only communication for runner processes.

Critical Issues 🔴

1. Unchecked write() Result (echo/src/main.rs:146)

cursor.write(&[0u8; 4]); // header (currently unused)

Problem: The write() method returns a Result that's being ignored, potentially causing silent failures.
Fix: Use cursor.write_all(&[0u8; 4])?; instead to properly handle errors and ensure all bytes are written.

2. Thread/Connection Leak (runner/mod.rs:146)

// TODO: need to kill old handler thread
// Spawn a new thread to handle incoming messages

Problem: When a new socket connection replaces an old one, the old handler thread is not cleaned up, causing resource leaks.
Impact: Over time, this will accumulate zombie threads and consume memory.

3. Dead Code in Usage Metrics (usage-metrics-publish/src/lib.rs:149)

let resources = match build.allocation_type { // Variable never used

Problem: The resources variable is declared but never assigned a value. The match expression performs side effects but doesn't assign to the variable.

Security Analysis 🟡

Positive Changes:

• Reduced attack surface: Unix sockets are local-only, eliminating network-based attack vectors
• Simplified authentication: Removal of JWT system reduces complexity

Security Concerns:

1. Socket permissions too permissive (runner/setup.rs:247-249):

   fs::set_permissions(&socket_path, std::fs::Permissions::from_mode(0o666))

   • 0o666 allows any local user to read/write to the socket
   • Should be more restrictive (e.g., 0o600 for owner-only access)

2. Missing authentication: Unlike WebSocket with JWT, Unix sockets have no built-in authentication

3. Socket path validation timing (runner/setup.rs:237-241): Validation happens after directory creation, potentially leaving incomplete cleanup on failure

Performance Considerations ✅

Improvements:

• Lower latency: Unix sockets have less overhead than WebSocket connections
• Reduced memory usage: No WebSocket frame parsing overhead
• Better resource efficiency: Direct kernel communication vs network stack

Potential Concerns:

• File descriptor usage: More Unix sockets may increase FD consumption
• Socket buffer management: Length-delimited codec adds some processing overhead

Error Handling & Resource Management ⚠️

Issues Identified:

1. Inconsistent error handling: Some socket operations use .context() while others don't
2. Missing connection timeout configuration: Hardcoded 5-second ping timeout may be too aggressive
3. Socket cleanup: Proper cleanup appears implemented but old threads aren't being terminated

Good Practices:

• Proper async cleanup in runner lifecycle management
• Graceful socket connection handling with timeouts
• Database transaction management

Test Coverage Assessment ❌

Missing Coverage:

• Unix socket communication: Tests still use WebSocket connections to pegboard
• Socket protocol: No tests for the new length-delimited frame encoding/decoding
• Connection scenarios: Missing tests for socket file permissions, connection failures, etc.
• Concurrent connections: No tests for multiple runners connecting simultaneously

Existing Coverage:

• Manager lifecycle tests work (test WebSocket to pegboard)
• Container execution and actor state management well tested

Recommendations

Immediate Fixes Required:

1. Fix unchecked write() in echo client (echo/src/main.rs:146)
2. Implement thread cleanup for replaced socket handlers (runner/mod.rs:146)
3. Fix dead code in usage metrics (usage-metrics-publish/src/lib.rs:149)
4. Restrict socket permissions to 0o600 or 0o660 (runner/setup.rs:247-249)

Suggested Improvements:

1. Add Unix socket tests covering the new communication protocol
2. Implement connection retry logic for runners
3. Add socket validation with proper error handling sequence
4. Consider protocol versioning for future compatibility
5. Add structured logging for socket operations
6. Document the hybrid architecture (WebSocket to pegboard, Unix sockets to runners)

Code Quality

Strengths:

• Clean separation of concerns between manager and runner protocols
• Proper use of async/await patterns
• Good error context in most places

Areas for Improvement:

• Consistent error handling throughout socket operations
• Remove magic numbers (timeouts, buffer sizes)
• Add more comprehensive logging for debugging socket issues

Overall Assessment

This is a solid architectural improvement that reduces network complexity and improves security posture. The core implementation is sound, but there are several critical issues that must be addressed before merging:

1. The unchecked write() result could cause silent failures
2. Thread leaks will accumulate over time
3. Socket permissions are too permissive for production

Recommendation: Request changes to address the critical issues, then approve with the suggested improvements for follow-up work.

---

[claude]

Claude finished @MasterPtato's task —— View job

---

PR Review: WebSocket to Unix Socket Conversion

Reviewing this pull request that converts WebSocket manager sockets to Unix sockets in the pegboard system.

Review Checklist:

• Read repository guidelines (CLAUDE.md)
• Analyze core architecture changes (WebSocket → Unix socket)
• Review security implications of socket conversion
• Examine error handling and resource management
• Check performance considerations
• Address existing automated review comments
• Assess test coverage impact
• Provide comprehensive review feedback

---

Summary

This PR implements a significant architectural change in the pegboard system, converting from WebSocket-based communication to Unix domain sockets for runner processes. The manager still connects to pegboard via WebSocket, but runners now communicate with the manager through Unix sockets with a length-delimited protocol.

Architecture Analysis ✅

Key Changes:

• Echo client now uses UnixStream instead of WebSocket connections (echo/src/main.rs)
• Runner communication migrated from WebSocket to Unix socket protocol (runner_protocol.rs)
• Manager handles Unix socket connections with frame-based communication (runner/mod.rs)
• JWT authentication removed (claims.rs deleted) - simplifies auth but removes token-based security
• Configuration changes to support Unix socket paths instead of WebSocket URLs

Impact: This change reduces network overhead and improves security by moving from network-based to local-only communication for runner processes.

Critical Issues 🔴

1. Unchecked write() Result (echo/src/main.rs:146)

cursor.write(&[0u8; 4]); // header (currently unused)

Problem: The write() method returns a Result that's being ignored, potentially causing silent failures.
Fix: Use cursor.write_all(&[0u8; 4])?; instead to properly handle errors and ensure all bytes are written.

2. Thread/Connection Leak (runner/mod.rs:146)

// TODO: need to kill old handler thread
// Spawn a new thread to handle incoming messages

Problem: When a new socket connection replaces an old one, the old handler thread is not cleaned up, causing resource leaks.
Impact: Over time, this will accumulate zombie threads and consume memory.

3. Dead Code in Usage Metrics (usage-metrics-publish/src/lib.rs:149)

let resources = match build.allocation_type { // Variable never used

Problem: The resources variable is declared but never assigned a value. The match expression performs side effects but doesn't assign to the variable.

Security Analysis 🟡

Positive Changes:

• Reduced attack surface: Unix sockets are local-only, eliminating network-based attack vectors
• Simplified authentication: Removal of JWT system reduces complexity

Security Concerns:

1. Socket permissions too permissive (runner/setup.rs:247-249):

   fs::set_permissions(&socket_path, std::fs::Permissions::from_mode(0o666))

   • 0o666 allows any local user to read/write to the socket
   • Should be more restrictive (e.g., 0o600 for owner-only access)

2. Missing authentication: Unlike WebSocket with JWT, Unix sockets have no built-in authentication

3. Socket path validation timing (runner/setup.rs:237-241): Validation happens after directory creation, potentially leaving incomplete cleanup on failure

Performance Considerations ✅

Improvements:

• Lower latency: Unix sockets have less overhead than WebSocket connections
• Reduced memory usage: No WebSocket frame parsing overhead
• Better resource efficiency: Direct kernel communication vs network stack

Potential Concerns:

• File descriptor usage: More Unix sockets may increase FD consumption
• Socket buffer management: Length-delimited codec adds some processing overhead

Error Handling & Resource Management ⚠️

Issues Identified:

1. Inconsistent error handling: Some socket operations use .context() while others don't
2. Missing connection timeout configuration: Hardcoded 5-second ping timeout may be too aggressive
3. Socket cleanup: Proper cleanup appears implemented but old threads aren't being terminated

Good Practices:

• Proper async cleanup in runner lifecycle management
• Graceful socket connection handling with timeouts
• Database transaction management

Test Coverage Assessment ❌

Missing Coverage:

• Unix socket communication: Tests still use WebSocket connections to pegboard
• Socket protocol: No tests for the new length-delimited frame encoding/decoding
• Connection scenarios: Missing tests for socket file permissions, connection failures, etc.
• Concurrent connections: No tests for multiple runners connecting simultaneously

Existing Coverage:

• Manager lifecycle tests work (test WebSocket to pegboard)
• Container execution and actor state management well tested

Recommendations

Immediate Fixes Required:

1. Fix unchecked write() in echo client (echo/src/main.rs:146)
2. Implement thread cleanup for replaced socket handlers (runner/mod.rs:146)
3. Fix dead code in usage metrics (usage-metrics-publish/src/lib.rs:149)
4. Restrict socket permissions to 0o600 or 0o660 (runner/setup.rs:247-249)

Suggested Improvements:

1. Add Unix socket tests covering the new communication protocol
2. Implement connection retry logic for runners
3. Add socket validation with proper error handling sequence
4. Consider protocol versioning for future compatibility
5. Add structured logging for socket operations
6. Document the hybrid architecture (WebSocket to pegboard, Unix sockets to runners)

Code Quality

Strengths:

• Clean separation of concerns between manager and runner protocols
• Proper use of async/await patterns
• Good error context in most places

Areas for Improvement:

• Consistent error handling throughout socket operations
• Remove magic numbers (timeouts, buffer sizes)
• Add more comprehensive logging for debugging socket issues

Overall Assessment

This is a solid architectural improvement that reduces network complexity and improves security posture. The core implementation is sound, but there are several critical issues that must be addressed before merging:

1. The unchecked write() result could cause silent failures
2. Thread leaks will accumulate over time
3. Socket permissions are too permissive for production

Recommendation: Request changes to address the critical issues, then approve with the suggested improvements for follow-up work.

---

[claude]

Claude finished @MasterPtato's task —— View job

---

PR Review: WebSocket to Unix Socket Conversion

Reviewing this pull request that converts WebSocket manager sockets to Unix sockets in the pegboard system.

Review Checklist:

• Read repository guidelines (CLAUDE.md)
• Analyze core architecture changes (WebSocket → Unix socket)
• Review security implications of socket conversion
• Examine error handling and resource management
• Check performance considerations
• Address existing automated review comments
• Assess test coverage impact
• Provide comprehensive review feedback

---

Summary

This PR implements a significant architectural change in the pegboard system, converting from WebSocket-based communication to Unix domain sockets for runner processes. The manager still connects to pegboard via WebSocket, but runners now communicate with the manager through Unix sockets with a length-delimited protocol.

Architecture Analysis ✅

Key Changes:

• Echo client now uses UnixStream instead of WebSocket connections (echo/src/main.rs)
• Runner communication migrated from WebSocket to Unix socket protocol (runner_protocol.rs)
• Manager handles Unix socket connections with frame-based communication (runner/mod.rs)
• JWT authentication removed (claims.rs deleted) - simplifies auth but removes token-based security
• Configuration changes to support Unix socket paths instead of WebSocket URLs

Impact: This change reduces network overhead and improves security by moving from network-based to local-only communication for runner processes.

Critical Issues 🔴

1. Unchecked write() Result (echo/src/main.rs:146)

cursor.write(&[0u8; 4]); // header (currently unused)

Problem: The write() method returns a Result that's being ignored, potentially causing silent failures.
Fix: Use cursor.write_all(&[0u8; 4])?; instead to properly handle errors and ensure all bytes are written.

2. Thread/Connection Leak (runner/mod.rs:146)

// TODO: need to kill old handler thread
// Spawn a new thread to handle incoming messages

Problem: When a new socket connection replaces an old one, the old handler thread is not cleaned up, causing resource leaks.
Impact: Over time, this will accumulate zombie threads and consume memory.

3. Dead Code in Usage Metrics (usage-metrics-publish/src/lib.rs:149)

let resources = match build.allocation_type { // Variable never used

Problem: The resources variable is declared but never assigned a value. The match expression performs side effects but doesn't assign to the variable.

Security Analysis 🟡

Positive Changes:

• Reduced attack surface: Unix sockets are local-only, eliminating network-based attack vectors
• Simplified authentication: Removal of JWT system reduces complexity

Security Concerns:

1. Socket permissions too permissive (runner/setup.rs:247-249):

   fs::set_permissions(&socket_path, std::fs::Permissions::from_mode(0o666))

   • 0o666 allows any local user to read/write to the socket
   • Should be more restrictive (e.g., 0o600 for owner-only access)

2. Missing authentication: Unlike WebSocket with JWT, Unix sockets have no built-in authentication

3. Socket path validation timing (runner/setup.rs:237-241): Validation happens after directory creation, potentially leaving incomplete cleanup on failure

Performance Considerations ✅

Improvements:

• Lower latency: Unix sockets have less overhead than WebSocket connections
• Reduced memory usage: No WebSocket frame parsing overhead
• Better resource efficiency: Direct kernel communication vs network stack

Potential Concerns:

• File descriptor usage: More Unix sockets may increase FD consumption
• Socket buffer management: Length-delimited codec adds some processing overhead

Error Handling & Resource Management ⚠️

Issues Identified:

1. Inconsistent error handling: Some socket operations use .context() while others don't
2. Missing connection timeout configuration: Hardcoded 5-second ping timeout may be too aggressive
3. Socket cleanup: Proper cleanup appears implemented but old threads aren't being terminated

Good Practices:

• Proper async cleanup in runner lifecycle management
• Graceful socket connection handling with timeouts
• Database transaction management

Test Coverage Assessment ❌

Missing Coverage:

• Unix socket communication: Tests still use WebSocket connections to pegboard
• Socket protocol: No tests for the new length-delimited frame encoding/decoding
• Connection scenarios: Missing tests for socket file permissions, connection failures, etc.
• Concurrent connections: No tests for multiple runners connecting simultaneously

Existing Coverage:

• Manager lifecycle tests work (test WebSocket to pegboard)
• Container execution and actor state management well tested

Recommendations

Immediate Fixes Required:

1. Fix unchecked write() in echo client (echo/src/main.rs:146)
2. Implement thread cleanup for replaced socket handlers (runner/mod.rs:146)
3. Fix dead code in usage metrics (usage-metrics-publish/src/lib.rs:149)
4. Restrict socket permissions to 0o600 or 0o660 (runner/setup.rs:247-249)

Suggested Improvements:

1. Add Unix socket tests covering the new communication protocol
2. Implement connection retry logic for runners
3. Add socket validation with proper error handling sequence
4. Consider protocol versioning for future compatibility
5. Add structured logging for socket operations
6. Document the hybrid architecture (WebSocket to pegboard, Unix sockets to runners)

Code Quality

Strengths:

• Clean separation of concerns between manager and runner protocols
• Proper use of async/await patterns
• Good error context in most places

Areas for Improvement:

• Consistent error handling throughout socket operations
• Remove magic numbers (timeouts, buffer sizes)
• Add more comprehensive logging for debugging socket issues

Overall Assessment

This is a solid architectural improvement that reduces network complexity and improves security posture. The core implementation is sound, but there are several critical issues that must be addressed before merging:

1. The unchecked write() result could cause silent failures
2. Thread leaks will accumulate over time
3. Socket permissions are too permissive for production

Recommendation: Request changes to address the critical issues, then approve with the suggested improvements for follow-up work.

---

[claude]

Claude finished @MasterPtato's task —— View job

---

PR Review: WebSocket to Unix Socket Conversion

Reviewing this pull request that converts WebSocket manager sockets to Unix sockets in the pegboard system.

Review Checklist:

• Read repository guidelines (CLAUDE.md)
• Analyze core architecture changes (WebSocket → Unix socket)
• Review security implications of socket conversion
• Examine error handling and resource management
• Check performance considerations
• Address existing automated review comments
• Assess test coverage impact
• Provide comprehensive review feedback

---

Summary

This PR implements a significant architectural change in the pegboard system, converting from WebSocket-based communication to Unix domain sockets for runner processes. The manager still connects to pegboard via WebSocket, but runners now communicate with the manager through Unix sockets with a length-delimited protocol.

Architecture Analysis ✅

Key Changes:

• Echo client now uses UnixStream instead of WebSocket connections (echo/src/main.rs)
• Runner communication migrated from WebSocket to Unix socket protocol (runner_protocol.rs)
• Manager handles Unix socket connections with frame-based communication (runner/mod.rs)
• JWT authentication removed (claims.rs deleted) - simplifies auth but removes token-based security
• Configuration changes to support Unix socket paths instead of WebSocket URLs

Impact: This change reduces network overhead and improves security by moving from network-based to local-only communication for runner processes.

Critical Issues 🔴

1. Unchecked write() Result (echo/src/main.rs:146)

cursor.write(&[0u8; 4]); // header (currently unused)

Problem: The write() method returns a Result that's being ignored, potentially causing silent failures.
Fix: Use cursor.write_all(&[0u8; 4])?; instead to properly handle errors and ensure all bytes are written.

2. Thread/Connection Leak (runner/mod.rs:146)

// TODO: need to kill old handler thread
// Spawn a new thread to handle incoming messages

Problem: When a new socket connection replaces an old one, the old handler thread is not cleaned up, causing resource leaks.
Impact: Over time, this will accumulate zombie threads and consume memory.

3. Dead Code in Usage Metrics (usage-metrics-publish/src/lib.rs:149)

let resources = match build.allocation_type { // Variable never used

Problem: The resources variable is declared but never assigned a value. The match expression performs side effects but doesn't assign to the variable.

Security Analysis 🟡

Positive Changes:

• Reduced attack surface: Unix sockets are local-only, eliminating network-based attack vectors
• Simplified authentication: Removal of JWT system reduces complexity

Security Concerns:

1. Socket permissions too permissive (runner/setup.rs:247-249):

   fs::set_permissions(&socket_path, std::fs::Permissions::from_mode(0o666))

   • 0o666 allows any local user to read/write to the socket
   • Should be more restrictive (e.g., 0o600 for owner-only access)

2. Missing authentication: Unlike WebSocket with JWT, Unix sockets have no built-in authentication

3. Socket path validation timing (runner/setup.rs:237-241): Validation happens after directory creation, potentially leaving incomplete cleanup on failure

Performance Considerations ✅

Improvements:

• Lower latency: Unix sockets have less overhead than WebSocket connections
• Reduced memory usage: No WebSocket frame parsing overhead
• Better resource efficiency: Direct kernel communication vs network stack

Potential Concerns:

• File descriptor usage: More Unix sockets may increase FD consumption
• Socket buffer management: Length-delimited codec adds some processing overhead

Error Handling & Resource Management ⚠️

Issues Identified:

1. Inconsistent error handling: Some socket operations use .context() while others don't
2. Missing connection timeout configuration: Hardcoded 5-second ping timeout may be too aggressive
3. Socket cleanup: Proper cleanup appears implemented but old threads aren't being terminated

Good Practices:

• Proper async cleanup in runner lifecycle management
• Graceful socket connection handling with timeouts
• Database transaction management

Test Coverage Assessment ❌

Missing Coverage:

• Unix socket communication: Tests still use WebSocket connections to pegboard
• Socket protocol: No tests for the new length-delimited frame encoding/decoding
• Connection scenarios: Missing tests for socket file permissions, connection failures, etc.
• Concurrent connections: No tests for multiple runners connecting simultaneously

Existing Coverage:

• Manager lifecycle tests work (test WebSocket to pegboard)
• Container execution and actor state management well tested

Recommendations

Immediate Fixes Required:

1. Fix unchecked write() in echo client (echo/src/main.rs:146)
2. Implement thread cleanup for replaced socket handlers (runner/mod.rs:146)
3. Fix dead code in usage metrics (usage-metrics-publish/src/lib.rs:149)
4. Restrict socket permissions to 0o600 or 0o660 (runner/setup.rs:247-249)

Suggested Improvements:

1. Add Unix socket tests covering the new communication protocol
2. Implement connection retry logic for runners
3. Add socket validation with proper error handling sequence
4. Consider protocol versioning for future compatibility
5. Add structured logging for socket operations
6. Document the hybrid architecture (WebSocket to pegboard, Unix sockets to runners)

Code Quality

Strengths:

• Clean separation of concerns between manager and runner protocols
• Proper use of async/await patterns
• Good error context in most places

Areas for Improvement:

• Consistent error handling throughout socket operations
• Remove magic numbers (timeouts, buffer sizes)
• Add more comprehensive logging for debugging socket issues

Overall Assessment

This is a solid architectural improvement that reduces network complexity and improves security posture. The core implementation is sound, but there are several critical issues that must be addressed before merging:

1. The unchecked write() result could cause silent failures
2. Thread leaks will accumulate over time
3. Socket permissions are too permissive for production

Recommendation: Request changes to address the critical issues, then approve with the suggested improvements for follow-up work.

---

[claude]

Claude finished @MasterPtato's task —— View job

---

PR Review: WebSocket to Unix Socket Conversion

Reviewing this pull request that converts WebSocket manager sockets to Unix sockets in the pegboard system.

Review Checklist:

• Read repository guidelines (CLAUDE.md)
• Analyze core architecture changes (WebSocket → Unix socket)
• Review security implications of socket conversion
• Examine error handling and resource management
• Check performance considerations
• Address existing automated review comments
• Assess test coverage impact
• Provide comprehensive review feedback

---

Summary

This PR implements a significant architectural change in the pegboard system, converting from WebSocket-based communication to Unix domain sockets for runner processes. The manager still connects to pegboard via WebSocket, but runners now communicate with the manager through Unix sockets with a length-delimited protocol.

Architecture Analysis ✅

Key Changes:

• Echo client now uses UnixStream instead of WebSocket connections (echo/src/main.rs)
• Runner communication migrated from WebSocket to Unix socket protocol (runner_protocol.rs)
• Manager handles Unix socket connections with frame-based communication (runner/mod.rs)
• JWT authentication removed (claims.rs deleted) - simplifies auth but removes token-based security
• Configuration changes to support Unix socket paths instead of WebSocket URLs

Impact: This change reduces network overhead and improves security by moving from network-based to local-only communication for runner processes.

Critical Issues 🔴

1. Unchecked write() Result (echo/src/main.rs:146)

cursor.write(&[0u8; 4]); // header (currently unused)

Problem: The write() method returns a Result that's being ignored, potentially causing silent failures.
Fix: Use cursor.write_all(&[0u8; 4])?; instead to properly handle errors and ensure all bytes are written.

2. Thread/Connection Leak (runner/mod.rs:146)

// TODO: need to kill old handler thread
// Spawn a new thread to handle incoming messages

Problem: When a new socket connection replaces an old one, the old handler thread is not cleaned up, causing resource leaks.
Impact: Over time, this will accumulate zombie threads and consume memory.

3. Dead Code in Usage Metrics (usage-metrics-publish/src/lib.rs:149)

let resources = match build.allocation_type { // Variable never used

Problem: The resources variable is declared but never assigned a value. The match expression performs side effects but doesn't assign to the variable.

Security Analysis 🟡

Positive Changes:

• Reduced attack surface: Unix sockets are local-only, eliminating network-based attack vectors
• Simplified authentication: Removal of JWT system reduces complexity

Security Concerns:

1. Socket permissions too permissive (runner/setup.rs:247-249):

   fs::set_permissions(&socket_path, std::fs::Permissions::from_mode(0o666))

   • 0o666 allows any local user to read/write to the socket
   • Should be more restrictive (e.g., 0o600 for owner-only access)

2. Missing authentication: Unlike WebSocket with JWT, Unix sockets have no built-in authentication

3. Socket path validation timing (runner/setup.rs:237-241): Validation happens after directory creation, potentially leaving incomplete cleanup on failure

Performance Considerations ✅

Improvements:

• Lower latency: Unix sockets have less overhead than WebSocket connections
• Reduced memory usage: No WebSocket frame parsing overhead
• Better resource efficiency: Direct kernel communication vs network stack

Potential Concerns:

• File descriptor usage: More Unix sockets may increase FD consumption
• Socket buffer management: Length-delimited codec adds some processing overhead

Error Handling & Resource Management ⚠️

Issues Identified:

1. Inconsistent error handling: Some socket operations use .context() while others don't
2. Missing connection timeout configuration: Hardcoded 5-second ping timeout may be too aggressive
3. Socket cleanup: Proper cleanup appears implemented but old threads aren't being terminated

Good Practices:

• Proper async cleanup in runner lifecycle management
• Graceful socket connection handling with timeouts
• Database transaction management

Test Coverage Assessment ❌

Missing Coverage:

• Unix socket communication: Tests still use WebSocket connections to pegboard
• Socket protocol: No tests for the new length-delimited frame encoding/decoding
• Connection scenarios: Missing tests for socket file permissions, connection failures, etc.
• Concurrent connections: No tests for multiple runners connecting simultaneously

Existing Coverage:

• Manager lifecycle tests work (test WebSocket to pegboard)
• Container execution and actor state management well tested

Recommendations

Immediate Fixes Required:

1. Fix unchecked write() in echo client (echo/src/main.rs:146)
2. Implement thread cleanup for replaced socket handlers (runner/mod.rs:146)
3. Fix dead code in usage metrics (usage-metrics-publish/src/lib.rs:149)
4. Restrict socket permissions to 0o600 or 0o660 (runner/setup.rs:247-249)

Suggested Improvements:

1. Add Unix socket tests covering the new communication protocol
2. Implement connection retry logic for runners
3. Add socket validation with proper error handling sequence
4. Consider protocol versioning for future compatibility
5. Add structured logging for socket operations
6. Document the hybrid architecture (WebSocket to pegboard, Unix sockets to runners)

Code Quality

Strengths:

• Clean separation of concerns between manager and runner protocols
• Proper use of async/await patterns
• Good error context in most places

Areas for Improvement:

• Consistent error handling throughout socket operations
• Remove magic numbers (timeouts, buffer sizes)
• Add more comprehensive logging for debugging socket issues

Overall Assessment

This is a solid architectural improvement that reduces network complexity and improves security posture. The core implementation is sound, but there are several critical issues that must be addressed before merging:

1. The unchecked write() result could cause silent failures
2. Thread leaks will accumulate over time
3. Socket permissions are too permissive for production

Recommendation: Request changes to address the critical issues, then approve with the suggested improvements for follow-up work.

---

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
rivet-site	Error			Aug 21, 2025 8:21pm

[claude]

Claude finished @MasterPtato's task —— View job

---

PR Review: WebSocket to Unix Socket Conversion

Reviewing this pull request that converts WebSocket manager sockets to Unix sockets in the pegboard system.

Review Tasks:

• Read repository guidelines (CLAUDE.md)
• Analyze core architecture changes (WebSocket → Unix socket)
• Review security implications of socket conversion
• Examine error handling and resource management
• Check performance considerations
• Address existing automated review comments
• Assess test coverage impact
• Provide comprehensive review feedback

---

Summary

This PR implements a significant architectural change in the pegboard system, converting from WebSocket-based communication to Unix domain sockets for runner processes. The manager still connects to pegboard via WebSocket, but runners now communicate with the manager through Unix sockets with a length-delimited protocol.

Architecture Analysis ✅

Key Changes:

• Echo client now uses UnixStream instead of WebSocket connections (echo/src/main.rs)
• Runner communication migrated from WebSocket to Unix socket protocol (runner_protocol.rs)
• Manager handles Unix socket connections with frame-based communication (runner/mod.rs)
• JWT authentication removed (claims.rs deleted) - simplifies auth but removes token-based security
• Configuration changes to support Unix socket paths instead of WebSocket URLs

Impact: This change reduces network overhead and improves security by moving from network-based to local-only communication for runner processes.

Critical Issues 🔴

1. Unchecked write() Result (echo/src/main.rs:146)

cursor.write(&[0u8; 4]); // header (currently unused)

Problem: The write() method returns a Result that's being ignored, potentially causing silent failures.
Fix: Use cursor.write_all(&[0u8; 4])?; instead to properly handle errors and ensure all bytes are written.

2. Thread/Connection Leak (runner/mod.rs:146)

// TODO: need to kill old handler thread
// Spawn a new thread to handle incoming messages

Problem: When a new socket connection replaces an old one, the old handler thread is not cleaned up, causing resource leaks.
Impact: Over time, this will accumulate zombie threads and consume memory.

3. Dead Code in Usage Metrics (usage-metrics-publish/src/lib.rs:149)

let resources = match build.allocation_type { // Variable never used

Problem: The resources variable is declared but never assigned a value. The match expression performs side effects but doesn't assign to the variable.

Security Analysis 🟡

Positive Changes:

• Reduced attack surface: Unix sockets are local-only, eliminating network-based attack vectors
• Simplified authentication: Removal of JWT system reduces complexity

Security Concerns:

1. Socket permissions too permissive (runner/setup.rs:247-249):

   fs::set_permissions(&socket_path, std::fs::Permissions::from_mode(0o666))

   • 0o666 allows any local user to read/write to the socket
   • Should be more restrictive (e.g., 0o600 for owner-only access)

2. Missing authentication: Unlike WebSocket with JWT, Unix sockets have no built-in authentication

3. Socket path validation timing (runner/setup.rs:237-241): Validation happens after directory creation, potentially leaving incomplete cleanup on failure

Performance Considerations ✅

Improvements:

• Lower latency: Unix sockets have less overhead than WebSocket connections
• Reduced memory usage: No WebSocket frame parsing overhead
• Better resource efficiency: Direct kernel communication vs network stack

Potential Concerns:

• File descriptor usage: More Unix sockets may increase FD consumption
• Socket buffer management: Length-delimited codec adds some processing overhead

Error Handling & Resource Management ⚠️

Issues Identified:

1. Inconsistent error handling: Some socket operations use .context() while others don't
2. Missing connection timeout configuration: Hardcoded 5-second ping timeout may be too aggressive
3. Socket cleanup: Proper cleanup appears implemented but old threads aren't being terminated

Good Practices:

• Proper async cleanup in runner lifecycle management
• Graceful socket connection handling with timeouts
• Database transaction management

Test Coverage Assessment ❌

Missing Coverage:

• Unix socket communication: Tests still use WebSocket connections to pegboard
• Socket protocol: No tests for the new length-delimited frame encoding/decoding
• Connection scenarios: Missing tests for socket file permissions, connection failures, etc.
• Concurrent connections: No tests for multiple runners connecting simultaneously

Existing Coverage:

• Manager lifecycle tests work (test WebSocket to pegboard)
• Container execution and actor state management well tested

Recommendations

Immediate Fixes Required:

1. Fix unchecked write() in echo client (echo/src/main.rs:146)
2. Implement thread cleanup for replaced socket handlers (runner/mod.rs:146)
3. Fix dead code in usage metrics (usage-metrics-publish/src/lib.rs:149)
4. Restrict socket permissions to 0o600 or 0o660 (runner/setup.rs:247-249)

Suggested Improvements:

1. Add Unix socket tests covering the new communication protocol
2. Implement connection retry logic for runners
3. Add socket validation with proper error handling sequence
4. Consider protocol versioning for future compatibility
5. Add structured logging for socket operations
6. Document the hybrid architecture (WebSocket to pegboard, Unix sockets to runners)

Code Quality

Strengths:

• Clean separation of concerns between manager and runner protocols
• Proper use of async/await patterns
• Good error context in most places

Areas for Improvement:

• Consistent error handling throughout socket operations
• Remove magic numbers (timeouts, buffer sizes)
• Add more comprehensive logging for debugging socket issues

Overall Assessment

This is a solid architectural improvement that reduces network complexity and improves security posture. The core implementation is sound, but there are several critical issues that must be addressed before merging:

1. The unchecked write() result could cause silent failures
2. Thread leaks will accumulate over time
3. Socket permissions are too permissive for production

Recommendation: Request changes to address the critical issues, then approve with the suggested improvements for follow-up work.

---

[claude]

Claude finished @MasterPtato's task —— View job

---

PR Review: WebSocket to Unix Socket Conversion

Reviewing this pull request that converts WebSocket manager sockets to Unix sockets in the pegboard system.

Review Checklist:

• Read repository guidelines (CLAUDE.md)
• Analyze core architecture changes (WebSocket → Unix socket)
• Review security implications of socket conversion
• Examine error handling and resource management
• Check performance considerations
• Address existing automated review comments
• Assess test coverage impact
• Provide comprehensive review feedback

---

Summary

This PR implements a significant architectural change in the pegboard system, converting from WebSocket-based communication to Unix domain sockets for runner processes. The manager still connects to pegboard via WebSocket, but runners now communicate with the manager through Unix sockets with a length-delimited protocol.

Architecture Analysis ✅

Key Changes:

• Echo client now uses UnixStream instead of WebSocket connections (echo/src/main.rs)
• Runner communication migrated from WebSocket to Unix socket protocol (runner_protocol.rs)
• Manager handles Unix socket connections with frame-based communication (runner/mod.rs)
• JWT authentication removed (claims.rs deleted) - simplifies auth but removes token-based security
• Configuration changes to support Unix socket paths instead of WebSocket URLs

Impact: This change reduces network overhead and improves security by moving from network-based to local-only communication for runner processes.

Critical Issues 🔴

1. Unchecked write() Result (echo/src/main.rs:146)

cursor.write(&[0u8; 4]); // header (currently unused)

Problem: The write() method returns a Result that's being ignored, potentially causing silent failures.
Fix: Use cursor.write_all(&[0u8; 4])?; instead to properly handle errors and ensure all bytes are written.

2. Thread/Connection Leak (runner/mod.rs:146)

// TODO: need to kill old handler thread
// Spawn a new thread to handle incoming messages

Problem: When a new socket connection replaces an old one, the old handler thread is not cleaned up, causing resource leaks.
Impact: Over time, this will accumulate zombie threads and consume memory.

3. Dead Code in Usage Metrics (usage-metrics-publish/src/lib.rs:149)

let resources = match build.allocation_type { // Variable never used

Problem: The resources variable is declared but never assigned a value. The match expression performs side effects but doesn't assign to the variable.

Security Analysis 🟡

Positive Changes:

• Reduced attack surface: Unix sockets are local-only, eliminating network-based attack vectors
• Simplified authentication: Removal of JWT system reduces complexity

Security Concerns:

1. Socket permissions too permissive (runner/setup.rs:247-249):

   fs::set_permissions(&socket_path, std::fs::Permissions::from_mode(0o666))

   • 0o666 allows any local user to read/write to the socket
   • Should be more restrictive (e.g., 0o600 for owner-only access)

2. Missing authentication: Unlike WebSocket with JWT, Unix sockets have no built-in authentication

3. Socket path validation timing (runner/setup.rs:237-241): Validation happens after directory creation, potentially leaving incomplete cleanup on failure

Performance Considerations ✅

Improvements:

• Lower latency: Unix sockets have less overhead than WebSocket connections
• Reduced memory usage: No WebSocket frame parsing overhead
• Better resource efficiency: Direct kernel communication vs network stack

Potential Concerns:

• File descriptor usage: More Unix sockets may increase FD consumption
• Socket buffer management: Length-delimited codec adds some processing overhead

Error Handling & Resource Management ⚠️

Issues Identified:

1. Inconsistent error handling: Some socket operations use .context() while others don't
2. Missing connection timeout configuration: Hardcoded 5-second ping timeout may be too aggressive
3. Socket cleanup: Proper cleanup appears implemented but old threads aren't being terminated

Good Practices:

• Proper async cleanup in runner lifecycle management
• Graceful socket connection handling with timeouts
• Database transaction management

Test Coverage Assessment ❌

Missing Coverage:

• Unix socket communication: Tests still use WebSocket connections to pegboard
• Socket protocol: No tests for the new length-delimited frame encoding/decoding
• Connection scenarios: Missing tests for socket file permissions, connection failures, etc.
• Concurrent connections: No tests for multiple runners connecting simultaneously

Existing Coverage:

• Manager lifecycle tests work (test WebSocket to pegboard)
• Container execution and actor state management well tested

Recommendations

Immediate Fixes Required:

1. Fix unchecked write() in echo client (echo/src/main.rs:146)
2. Implement thread cleanup for replaced socket handlers (runner/mod.rs:146)
3. Fix dead code in usage metrics (usage-metrics-publish/src/lib.rs:149)
4. Restrict socket permissions to 0o600 or 0o660 (runner/setup.rs:247-249)

Suggested Improvements:

1. Add Unix socket tests covering the new communication protocol
2. Implement connection retry logic for runners
3. Add socket validation with proper error handling sequence
4. Consider protocol versioning for future compatibility
5. Add structured logging for socket operations
6. Document the hybrid architecture (WebSocket to pegboard, Unix sockets to runners)

Code Quality

Strengths:

• Clean separation of concerns between manager and runner protocols
• Proper use of async/await patterns
• Good error context in most places

Areas for Improvement:

• Consistent error handling throughout socket operations
• Remove magic numbers (timeouts, buffer sizes)
• Add more comprehensive logging for debugging socket issues

Overall Assessment

This is a solid architectural improvement that reduces network complexity and improves security posture. The core implementation is sound, but there are several critical issues that must be addressed before merging:

1. The unchecked write() result could cause silent failures
2. Thread leaks will accumulate over time
3. Socket permissions are too permissive for production

Recommendation: Request changes to address the critical issues, then approve with the suggested improvements for follow-up work.

---

[claude]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.

[claude]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.