Workers: Adds framework for outbound / external worker requests (starting with HTTP)

[rdaum]

No description provided.

[Copilot]

Pull Request Overview

This PR introduces a framework for outbound/external worker requests (starting with HTTP) by adding worker-specific modules and integrating worker handling in the scheduler and daemon, as well as a new dedicated worker process for HTTP requests.

• Added new worker modules (worker, worker_loop, worker_rpc_client) in the async RPC client crate
• Extended the kernel’s task suspension and scheduling to support worker requests
• Integrated a new HTTP worker (curl-worker) and added corresponding CLI and dependency updates

Reviewed Changes

Copilot reviewed 35 out of 35 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
crates/rpc/rpc-async-client/src/lib.rs	Removes host-only functionality and re-exports new worker functions
crates/rpc/rpc-async-client/src/host.rs	Moves host functionality into its own module; no functional changes aside from code reorganization
crates/kernel/src/vm/mod.rs	Adds a new TaskSuspend variant for worker requests
crates/kernel/src/tasks/*	Introduces new modules and updates (workers, suspension, scheduler) to handle worker requests
crates/daemon/src/*	Integrates worker server functionality into the daemon
crates/curl-worker/*	Adds a new worker application for outbound HTTP requests
Cargo.toml files	Update dependencies to support new functionality

Comments suppressed due to low confidence (1)

crates/kernel/testsuite/moot_suite.rs:168

• Consider adding tests to specifically validate the new worker request functionality, covering both successful responses and error cases.

        None,

[Copilot]

Pull Request Overview

This PR implements a new framework for outbound/external worker requests starting with HTTP. Key changes include refactoring the rpc-async-client by moving host functionality into a dedicated module and adding new worker modules, extending the kernel and scheduler for worker request/response support, and integrating a new workers server and Curl worker for external HTTP requests.

Reviewed Changes

Copilot reviewed 35 out of 35 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
crates/rpc/rpc-async-client/src/lib.rs	Removes redundant host functions and re-exports functions and new worker modules.
crates/rpc/rpc-async-client/src/host.rs	Contains the moved host functions (make_host_token, process_hosts_events, etc.).
crates/kernel/src/tasks/*	Adds WorkerRequest support and handles worker responses in task scheduler.
crates/daemon/src/*	Integrates workers server setup and updates RPC request handling.
crates/curl-worker/src/main.rs	Implements the outbound HTTP worker using the new worker framework.
Cargo.toml files	Update dependencies and workspace settings to support the new worker functionality.
crates/compiler/src/builtins.rs	Adds a new built-in function for worker requests.

Comments suppressed due to low confidence (1)

crates/rpc/rpc-async-client/src/lib.rs:16

• The re-export name 'proces_hosts_events' does not match the actual function name 'process_hosts_events' defined in host.rs. Please update the re-export to ensure consistency.

pub use host::{make_host_token, proces_hosts_events, send_host_to_daemon_msg, start_host_session};

[Copilot]

Pull Request Overview

This PR introduces a new framework to support outbound/external worker requests, starting with HTTP. The changes include new worker modules and updated API integration in the RPC client, kernel task scheduler, daemon, and a new curl-worker crate.

Reviewed Changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
crates/rpc/rpc-async-client/src/lib.rs	Removed host-specific functionality and re-exported worker APIs.
crates/rpc/rpc-async-client/src/host.rs	Moved host functions to a dedicated module.
crates/kernel/*	Introduced new TaskSuspend variant and updated workers handling in the scheduler and suspension logic.
crates/daemon/*	Added workers server integration and updated RPC server handling.
crates/curl-worker/*	New worker crate implementing outbound HTTP request logic.
crates/compiler/src/builtins.rs	Added builtin registration for worker_request.
crates/common/src/tasks/errors.rs	Extended task errors with WorkerError definitions.

Comments suppressed due to low confidence (1)

crates/compiler/src/builtins.rs:1225

• The builtin 'worker_request' is defined to accept exactly 2 arguments, yet the implementation in bf_worker_request supports a variable number of arguments. Consider updating 'max_args' to allow additional arguments or changing the builtin's interface to explicitly accept only two arguments, matching the documented behavior.

Builtin { name: Symbol::mk("worker_request"), min_args: Q(2), max_args: Q(2), types: vec![Typed(TYPE_SYMBOL), Typed(TYPE_LIST)], implemented: true, },

[Copilot]

Pull Request Overview

This PR implements a new worker framework to support outbound/external worker requests (starting with HTTP) by refactoring the RPC client and host modules and integrating new worker functionality into the scheduler and task suspension mechanisms.

• Migrates host token and messaging functions to a dedicated host module and introduces worker-specific modules in the RPC async client.
• Updates the task VM, scheduler, and builtins to support suspending tasks awaiting worker responses.
• Integrates a new workers server (e.g. the curl-worker) with corresponding CLI arguments and daemon configuration.

Reviewed Changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
crates/rpc/rpc-async-client/src/lib.rs	Splits host functionality and exports new worker functions.
crates/rpc/rpc-async-client/src/host.rs	Moves host token and message handling to its own module.
crates/rpc/rpc-async-client/Cargo.toml	Adds dependency on moor-common.
crates/kernel/testsuite/moot_suite.rs	Updates scheduler invocation with additional parameters.
crates/kernel/src/vm/mod.rs	Adds new task suspend variant for worker requests.
crates/kernel/src/tasks/workers.rs	Introduces worker request and response enums.
crates/kernel/src/tasks/suspension.rs	Adds Worker wake condition handling to the suspension queue.
crates/kernel/src/tasks/scheduler.rs	Integrates worker request sending and response handling into the scheduler.
crates/kernel/src/tasks/mod.rs	Exposes the new workers module.
crates/kernel/src/builtins/bf_server.rs	Registers a new built-in for issuing worker requests.
crates/daemon/src/rpc_server.rs	Tweaks RPC server response formatting by removing redundant Uuid conversion.
crates/daemon/src/main.rs	Adds workers server startup and wires worker channel integration.
crates/daemon/src/args.rs	Introduces new command-line arguments for worker server configuration.
crates/daemon/Cargo.toml	Adds crossbeam-channel workspace dependency.
crates/curl-worker/src/main.rs	Implements a new worker that processes HTTP requests using the new worker framework.
crates/curl-worker/Cargo.toml	Sets up the package dependencies for the new curl-worker.
crates/compiler/src/builtins.rs	Registers the new worker_request built-in.
crates/common/src/tasks/mod.rs & errors.rs	Updates common tasks errors to include a new WorkerError variant.
Cargo.toml	Adds curl-worker to the workspace and adjusts dependency versions.