Skip to content

Add Instance resource agent, dashboards, and resource reporting integration#108

Closed
zhy1658858023 wants to merge 1 commit into
mainfrom
codex/instance
Closed

Add Instance resource agent, dashboards, and resource reporting integration#108
zhy1658858023 wants to merge 1 commit into
mainfrom
codex/instance

Conversation

@zhy1658858023

Copy link
Copy Markdown
Collaborator

Motivation

  • Provide real-time resource telemetry from Instance hosts and report snapshots to the Proxy control plane for richer scheduling and pool admission decisions.
  • Add convenient local monitoring tools (browser dashboard and Tkinter app) and a small reporter to push the agent snapshots into the Proxy control plane.
  • Enrich Proxy and Scheduler control planes with aggregated pool-level resource awareness so scheduling can consider CPU/memory/GPU/network states.

Description

  • Add a lightweight Rust resource agent (instance/resource_agent/src/main.rs) that samples CPU/memory/network/GPU and exposes HTTP endpoints GET /healthz and GET /v1/resource/snapshot, plus Cargo.toml/Cargo.lock for the agent.
  • Add a reporter script instance/resource_agent/proxy_reporter.py that fetches the local agent snapshot and POSTs it to the Proxy control plane at POST /v1/instance/resource_snapshot.
  • Add a browser dashboard server (instance/resource_dashboard/dashboard_server.py), a Tkinter desktop app (instance/resource_dashboard/dashboard_app.py), and static frontend assets (static/index.html, static/app.js, static/style.css) for visualizing instance snapshots and controlling a managed agent process.
  • Extend the Proxy control-plane and supporting modules: add POST /v1/instance/resource_snapshot and debug endpoints in proxy/resource/p_control_plane.py, store per-instance InstanceResource and compute build_pool_resource_snapshot in proxy/resource/instance_pool.py, and propagate pool_resource in proxy register/heartbeat flows via proxy/sclient/scheduler_client.py.
  • Extend the Scheduler control plane and ProxyPool to accept and expose pool_resource on proxy registration/heartbeat and add debug APIs in scheduler/resource/control_plane.py and scheduler/resource/proxy_pool.py to surface reported pool resources.

Testing

  • No automated tests were executed for these changes.

Codex Task

@zhy1658858023 zhy1658858023 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not recommend merging this PR in its current form.

Main blockers:

  1. The branch is stale and diverged from current main.

    • codex/instance is behind_by=101 relative to main and mergeable=false.
    • Please rebase/update from current main and make the diff a clean increment for Issue #107 only.
  2. Scope is much larger than Issue #107.

    • Issue #107 asks for Proxy pool-level resource aggregation and Proxy -> Scheduler reporting.
    • This PR also adds a Rust Instance Resource Agent, Instance dashboards, and a standalone reporter. Those are outside the requested scope and appear to duplicate earlier resource-monitoring/dashboard work.
    • Please remove these out-of-scope additions from this PR unless there is a separate issue explicitly asking for them.
  3. proxy/resource/p_control_plane.py appears to reintroduce stale liveness behavior.

    • The added instance-list serialization still uses "is_alive": True if not include_dead else None.
    • For include_dead=true, the UI/debug path needs real TTL-derived liveness, not None.
  4. The resource snapshot API shape is narrower than the existing/desired reporting path.

    • The reporter sends only {instance_id, snapshot}.
    • Please preserve any existing metadata fields and avoid regressing resource-report observability.
  5. The core #107 path should remain lightweight and backward compatible:

    • Proxy builds a compact pool_resource from InstancePool.
    • Proxy sends it in register/heartbeat payloads.
    • Scheduler stores it per Proxy.
    • Scheduler exposes /debug/proxy_pool_resources.
    • No Scheduler routing policy changes, no Proxy selection changes, no KDN/KVCache/IWS/Instance forwarding changes.

Please redo this as a clean branch from latest main, with only the pool-resource aggregation/reporting changes plus minimal docs/validation notes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant