Authentication and Authorization Design Proposal

[mahil-2040]

What type of PR is this?

/kind documentation

What this PR does / why we need it:

This PR introduces a design proposal for implementing authentication and authorization in Agentcube.

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a comprehensive design proposal to overhaul AgentCube's authentication and authorization mechanisms. It aims to establish a robust, unified security model by implementing zero-trust mTLS for internal component communication using SPIFFE/SPIRE, and integrating Keycloak for external user authentication and role-based access control. This strategic shift addresses existing security vulnerabilities, enhances system integrity, and provides a scalable, industry-standard foundation for managing identities and permissions across the platform.

Highlights

• Internal mTLS Communication: Established zero-trust, mutually authenticated communication between all AgentCube internal components (Router, WorkloadManager, PicoD) using X.509 mTLS via SPIFFE/SPIRE.
• External User Authentication: Provided external client/SDK authentication at the Router level using Keycloak (OIDC/OAuth2) for identity verification.
• Role-Based Access Control (RBAC): Implemented role-based access control for external users leveraging Keycloak's built-in authorization capabilities.
• Deprecation of Legacy Auth: Superseded the existing PicoD-Plain-Authentication key distribution mechanism with automated certificate lifecycle management provided by SPIRE, with a legacy flag for transition.
• Opt-in Features: Ensured all new authentication and authorization features are opt-in behind configuration flags to avoid affecting existing deployments.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• docs/design/auth-proposal.md
  • Detailed the motivation and goals for a unified security model.
  • Outlined a four-layered design covering internal workload authentication (SPIRE), external user authentication (Keycloak), authorization (Keycloak RBAC), and cloud provider federation (stretch goal).
  • Provided in-depth specifications for SPIRE deployment, workload registration, and mTLS integration within AgentCube components.
  • Described Keycloak deployment, JWT validation, SDK changes, and role hierarchy for authorization.
  • Included architecture diagrams and code snippets to illustrate the proposed solutions.
  • Discussed the impact on existing authentication mechanisms and future enhancements like OPA integration.

Activity

• No significant activity has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Warning

Gemini is experiencing higher than usual traffic and was unable to create the review. Please try again in a few hours by commenting /gemini review.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 43.35%. Comparing base (845b798) to head (8de1f54).
⚠️ Report is 133 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #241      +/-   ##
==========================================
+ Coverage   35.60%   43.35%   +7.74%     
==========================================
  Files          29       30       +1     
  Lines        2533     2611      +78     
==========================================
+ Hits          902     1132     +230     
+ Misses       1505     1358     -147     
+ Partials      126      121       -5     

Flag	Coverage Δ
unittests	43.35% <ø> (+7.74%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

Adds a new design document proposing a unified authentication + authorization model for AgentCube, covering internal component identity and external user access at the Router boundary.

Changes:

• Introduces an SPIFFE/SPIRE mTLS proposal for Router/WorkloadManager/PicoD internal traffic.
• Proposes Keycloak-based external authentication (OIDC) and role-based authorization (RBAC) at the Router.
• Describes migration away from the current PicoD plain-auth/JWT approach toward workload identity.

---

You can also share your feedback on Copilot code review. Take the survey.

[hzxuzhonghu]

Posting the main design consistency issues I found in the auth proposal.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.

---

You can also share your feedback on Copilot code review. Take the survey.

[hzxuzhonghu]

Posted a few design-consistency comments on the current PR head. The main issues are the missing explanation of how end-user identity/audit context survives the Router -> WorkloadManager hop once that channel becomes pure workload mTLS, and the security implications of using a label-only SPIRE selector for PicoD. I also left two smaller documentation comments around OAuth token refresh behavior and the OPA comparison wording.

[hzxuzhonghu]

From above section, does all the picod share same spiffe id?

Previsouly we donot use tls between router and picod, one consideration is latency.
The other concern here is we also allow user build their customized code interperet.

[mahil-2040]

Regarding custom code interpreters, could you clarify the concern here?

Is the concern about the added complexity for users building custom interpreters? because with mTLS enforced on Router→PicoD, custom interpreters would need to handle TLS certificates.

And can you suggest a approach for supporting custom interpreters alongside mTLS?

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

[mahil-2040]

@hzxuzhonghu PTAL, i have made the changes according to your feedback.

[hzxuzhonghu]

We should consider also bring the user's identity in the token

[mahil-2040]

So instead of having the Router extract the user identity and pass it blindly via X-AgentCube-User-ID headers to the WorkloadManager, the Router should perform an OAuth 2.0 Token Exchange (RFC 8693) with Keycloak.

The Router would exchange the incoming user token for a new downstream token (where the subject is the user, and the actor claim is the Router) and pass that nested token to the WorkloadManager.

is my understanding correct?

If yes, then Should I update the proposal to replace the custom HTTP header approach with this Token Exchange flow as the core requirement, or should we keep the header approach for now and mark Token Exchange as a Stretch goal in the proposal?

[hzxuzhonghu]

Right, i think both X-AgentCube-User-ID and token exchange make sense

[hzxuzhonghu]

This proposal not cover router -> agent runtime authentication. how could we delegate user's identity to the agent?

[hzxuzhonghu]

Can limit only the creator or the group can access a sandbox

[mahil-2040]

I’ve updated the proposal to make the tenant isolation explicit using Resource-Level Access Control (RLAC):

Creation: When a sandbox is created, the WorkloadManager tags all related resources (CRDs/Pods) with immutable labels like agentcube.io/owner: user-123, based on JWT claims (sub, groups).
Invocation: On each request, the Router checks the caller’s JWT against the sandbox’s labels. If they don’t match (not owner or allowed group), it returns 403 Forbidden.

This ensures strict tenant isolation , no user can access another user’s sandbox, even within the same namespace.

[hzxuzhonghu]

Reading through the whole section 1, i find it is only talking about x509 certs with spire. Seems not align with the desc below

[mahil-2040]

I added a new section (1.6 File-Based Provisioning with cert-manager) that shows how to achieve the same zero-trust mTLS setup with File-based provisioning. It includes a sample Certificate CRD (mapping uris to SPIFFE IDs) and how the generated Secret plugs into our --mtls-*-file args.

[hzxuzhonghu]

so by ignoring namespace here, does it mean all picod across namespaces would receive svid?

And if yes, are they same?

[mahil-2040]

Yes, to both questions.

1. Because sandboxes are provisioned dynamically into arbitrary, user-specified namespaces, we must omit the namespaceSelector. This allows the SPIRE Agent to attest PicoD pods globally across the cluster, provided they possess the required ServiceAccount and label.
2. Yes, every sandbox receives the exact same SPIFFE ID (spiffe://cluster.local/sa/agentcube-sandbox).

The shared SPIFFE ID purely provides machine-to-machine authentication (proving to the Router that the target is a legitimate, WorkloadManager provisioned PicoD pod).

The actual tenant isolation and authorization (ensuring User A cannot access User B's sandbox, even if they share the same machine identity) is strictly enforced at the Application Layer via the Resource-Level Access Control (RLAC) JWT claim checks detailed in Section 3.6. This prevents the SPIRE Server from being overwhelmed by creating/destroying unique cryptographic identities for every short-lived sandbox instance!

[hzxuzhonghu]

I am concerned the delay here, in agencube its low latency on sandbox provision should be highly prioritized. Especially for agentic rl scenario.

[hzxuzhonghu]

This is a critical issue, maybe we need also jwt based SVID

[mahil-2040]

I've looked into this carefully and The key insight is that JWT-SVIDs in SPIRE still go through the exact same Workload Attestation flow (Agent → kubelet query → selector matching), so the attestation latency is identical regardless of SVID format.

For PicoD sandboxes specifically, the spiffe-helper sidecar fetches the SVID concurrently with PicoD's own container initialization, so attestation (~200-500ms) overlaps with pod startup time rather than adding to it sequentially.

However, for high-throughput scenarios like agentic RL training loops where even this concurrent overhead is unacceptable, our architecture already has a built-in zero-latency alternative: the file-based certificate mode (Section 1.6). In this mode, certificates are pre-provisioned as Kubernetes Secrets and mounted as volumes — they are available the instant the container starts with absolutely no attestation step needed. This makes file-based mode the recommended choice for latency-critical sandbox provisioning.

I've added a dedicated "Latency Considerations for Sandbox Provisioning" note to the proposal to document this tradeoff explicitly.

[hzxuzhonghu]

you are right, the signing is similar flow as x509, the delay sometimes is un acceptable. In code interpreter, we usually need 100ms bootstrap latency. And in reality, tls handshake takes some time maybe tens of milliseconds.

With jwt, the handshake latency can be eliminated. if we have warmpool, i guess the latency is totally from runtime, not from auth system.

[mahil-2040]

Thank you for clarifying. You're absolutely right that the real bottleneck for latency-critical scenarios like Code Interpreters isn't attestation but the TLS handshake overhead itself (~20-50ms per new connection). With JWT, the Router just sends a Bearer token over plain HTTP and PicoD does a ~1ms signature check - the handshake cost is completely eliminated.

Based on your feedback, I've updated the proposal to support two configurable Router→PicoD auth modes via a --picod-auth-mode flag:

1. mtls (default): Full mutual TLS for security-hardened production deployments.
2. jwt: Application-layer JWT authentication for latency-critical scenarios (Code Interpreters, agentic RL). Combined with warm pools, this ensures the auth system adds near-zero overhead.

Router↔WorkloadManager always uses mTLS since both are long-lived and the handshake cost is amortized. The existing PicoD-Plain-Auth code path is retained and improved as the JWT mode implementation rather than being deprecated.

[hzxuzhonghu]

would you implement this way?

[mahil-2040]

we are not implementing any integration code for these four mechanisms inside the AgentCube components.

The core of this design is that the AgentCube Go binaries will only read standard certificate files from the local disk using the new --mtls-cert-file, --mtls-key-file, and --mtls-ca-file flags.

Because AgentCube only cares about reading a file, we automatically support all four of those mechanisms out-of-the-box, as they all ultimately just write PEM files to a Kubernetes volume mount. Our implementation will strictly focus on adding the file-reading logic to the Go codebase, and providing example Kubernetes YAML manifests (like the spiffe-helper sidecar) to show users how to mount those files seamlessly.

[hzxuzhonghu]

Yeah, i was asking whether you would provide such integration guide

[hzxuzhonghu]

A dedicated realm calledagentcube``, I would suggest letting users register to support multi tenants

[mahil-2040]

The phrasing in the proposal made it sound like the architecture was hardcoded to a single realm name - that was definitely my mistake!

But the Go implementation actually accepts the Keycloak Issuer URL and Realm dynamically through CLI configuration flags (e.g., --keycloak-realm). The agentcube realm is purely just the default name created by the "getting started" Helm chart. Cluster administrators can easily alter the deployment flags to point the Router at whatever custom or multiple distinct realms they need to achieve full multi-tenant isolation.

I have updated the proposal text in Section 2.3 to explicitly state this capability so there's no confusion!

[mahil-2040]

This proposal not cover router -> agent runtime authentication. how could we delegate user's identity to the agent?
You're exactly right that relying solely on mTLS for the Router $\rightarrow$ Agent Runtime proxy connection drops the user context for the executing agent.

To solve this, we can use the exact same OAuth 2.0 Token Exchange (RFC 8693) pattern that I have mentioned in for the workloadmanager.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

[Copilot]

The mTLS code snippet overwrites err and never checks the errors from tls.LoadX509KeyPair / os.ReadFile, and it ignores the boolean return from AppendCertsFromPEM. Since this is intended to be copy/paste-able guidance, it should show proper error handling and a failure case when the CA bundle can’t be parsed.

Suggested change

	// Load CA bundle for peer verification
	caCert, err := os.ReadFile(caFile)
	caPool := x509.NewCertPool()
	caPool.AppendCertsFromPEM(caCert)
	if err != nil {
	log.Fatalf("failed to load mTLS certificate/key pair (%s, %s): %v", certFile, keyFile, err)
	}
	// Load CA bundle for peer verification
	caCert, err := os.ReadFile(caFile)
	if err != nil {
	log.Fatalf("failed to read mTLS CA bundle %s: %v", caFile, err)
	}
	caPool := x509.NewCertPool()
	if ok := caPool.AppendCertsFromPEM(caCert); !ok {
	log.Fatalf("failed to parse any CA certificates from %s", caFile)
	}

[Copilot]

In the client-side VerifyPeerCertificate example, peerCert.Verify(x509.VerifyOptions{Roots: caPool}) verifies only the leaf cert and doesn’t populate Intermediates from rawCerts[1:]. This will fail for real-world chains that include intermediates, and it also doesn’t constrain key usage (serverAuth). Consider building an intermediates pool from the remaining certs and setting appropriate KeyUsages in VerifyOptions.

Suggested change

	peerCert, err := x509.ParseCertificate(rawCerts[0])
	if err != nil { return err }
	// 1. Manually verify against our CA pool
	opts := x509.VerifyOptions{ Roots: caPool }
	if _, err := peerCert.Verify(opts); err != nil { return err }
	if len(rawCerts) == 0 {
	return fmt.Errorf("no server certificates provided")
	}
	peerCert, err := x509.ParseCertificate(rawCerts[0])
	if err != nil {
	return err
	}
	// Build intermediates pool from remaining certificates in the chain, if any.
	intermediates := x509.NewCertPool()
	for _, certDER := range rawCerts[1:] {
	cert, err := x509.ParseCertificate(certDER)
	if err != nil {
	return err
	}
	intermediates.AddCert(cert)
	}
	// 1. Manually verify against our CA pool with intermediates and serverAuth key usage.
	opts := x509.VerifyOptions{
	Roots: caPool,
	Intermediates: intermediates,
	KeyUsages: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
	}
	if _, err := peerCert.Verify(opts); err != nil {
	return err
	}

[YaoZengzeng]

what does Inherits means? Like sandbox:manage include permissions of sandbox:invoke plus its own permissions?

[mahil-2040]

Yes, exactly! In Keycloak this uses "Composite Roles. Assigning a user the higher-level sandbox:manage role automatically grants them the sandbox:invoke permissions as well, preventing us from having to assign both roles to administrators.

[YaoZengzeng]

Seems we create AgentRuntime and CodeInterpreter by running kubectl manually, not from SDK.

[mahil-2040]

You're right. currently AgentRuntime and CodeInterpreter CRDs are created via kubectl or the AgentCube CLI (agentcube publish), both of which go directly to the Kubernetes API and are governed by Kubernetes RBAC, not Keycloak.

The sandbox:manage role is defined as a forward-looking placeholder for when CRD lifecycle operations are eventually exposed through the Router. For now, only sandbox:invoke is actively enforced by the Keycloak middleware. I've added a clarifying note to the proposal to make this distinction explicit.

[hzxuzhonghu]

Assume this is all installed beforehand, right?

[mahil-2040]

Not quite beforehand. In the spire-controller-manager we can bundle these ClusterSPIFFEID manifests directly into the main AgentCube Helm chart/release bundle.
When the admin runs helm install agentcube, the WorkloadManager, the Router, and these SPIFFE ID registrations are all applied to the cluster at the same time.

[hzxuzhonghu]

got it, So please add a install option

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 6 comments.

[Copilot]

The ClusterSPIFFEID example for PicoD only matches on app: picod, but earlier in this document you state PicoD identity should also be constrained by a dedicated ServiceAccount (and the manual CLI example includes k8s:sa:agentcube-sandbox). As written, this example would allow any pod that can set app: picod to obtain the sandbox SPIFFE ID; please add the ServiceAccount constraint (and any other required selectors) or note that this snippet is incomplete.

Suggested change

	spiffeIDTemplate: "spiffe://cluster.local/sa/{{ .PodSpec.ServiceAccountName }}"
	spiffeIDTemplate: "spiffe://cluster.local/sa/{{ .PodSpec.ServiceAccountName }}"
	# NOTE: This example is intentionally minimal for illustration. In production,
	# you MUST also constrain this identity by a dedicated ServiceAccount
	# (for example, `agentcube-sandbox`) and any other required selectors so that
	# only trusted PicoD pods can obtain the sandbox SPIFFE ID.

[Copilot]

In the client-side TLS verification example, peerCert.Verify is called with only Roots: caPool. If the server cert chain includes intermediates (common with many PKIs/cert-manager issuers), verification will fail unless the intermediates from rawCerts[1:] are added to VerifyOptions.Intermediates. Also, when using InsecureSkipVerify: true, it’s important that the custom callback fully replicates normal TLS verification semantics.

Suggested change

	peerCert, err := x509.ParseCertificate(rawCerts[0])
	if err != nil { return err }
	// 1. Manually verify against our CA pool
	opts := x509.VerifyOptions{ Roots: caPool }
	if _, err := peerCert.Verify(opts); err != nil { return err }
	if len(rawCerts) == 0 {
	return fmt.Errorf("no server certificates provided")
	}
	// Parse the leaf (peer) certificate
	peerCert, err := x509.ParseCertificate(rawCerts[0])
	if err != nil {
	return err
	}
	// Parse and add any intermediate certificates presented by the server
	intermediates := x509.NewCertPool()
	for _, raw := range rawCerts[1:] {
	cert, err := x509.ParseCertificate(raw)
	if err != nil {
	return err
	}
	intermediates.AddCert(cert)
	}
	// 1. Manually verify the certificate chain against our CA pool
	opts := x509.VerifyOptions{
	Roots: caPool,
	Intermediates: intermediates,
	}
	if _, err := peerCert.Verify(opts); err != nil {
	return err
	}

[Copilot]

The JWKS caching snippet appears to use a jwk.NewCache API, but this repo doesn’t currently depend on a JWKS/JWK cache library and the snippet doesn’t show how the cached keys are actually used for JWT verification (and it also ignores the error return from Register if present). Please either mark this as pseudocode and name the intended library, or update to a concrete, correct example (e.g., using an OIDC/JWKS client that returns a keyfunc for github.com/golang-jwt/jwt/v5).

Suggested change

	type KeycloakConfig struct {
	IssuerURL string // e.g., "https://keycloak.agentcube-system.svc:8443"
	Realm string // Default: "agentcube"
	Audience string // expected audience claim
	JWKSCacheTTL time.Duration // how often to refresh JWKS keys
	}
	func NewKeycloakValidator(cfg KeycloakConfig) (*KeycloakValidator, error) {
	jwksURL := fmt.Sprintf("%s/realms/%s/protocol/openid-connect/certs",
	cfg.IssuerURL, cfg.Realm)
	// JWKS keys are cached locally - no per-request calls to Keycloak
	keySet := jwk.NewCache(context.Background())
	keySet.Register(jwksURL, jwk.WithRefreshInterval(cfg.JWKSCacheTTL))
	return &KeycloakValidator{
	issuer: fmt.Sprintf("%s/realms/%s", cfg.IssuerURL, cfg.Realm),
	audience: cfg.Audience,
	keySet: keySet,
	}, nil
	}
	package auth
	import (
	"context"
	"fmt"
	"time"
	"github.com/MicahParks/keyfunc"
	"github.com/golang-jwt/jwt/v5"
	)
	// KeycloakConfig configures JWT validation against a Keycloak realm.
	type KeycloakConfig struct {
	IssuerURL string // e.g., "https://keycloak.agentcube-system.svc:8443"
	Realm string // default: "agentcube"
	Audience string // expected audience claim
	JWKSCacheTTL time.Duration // how often to refresh JWKS keys
	}
	// KeycloakValidator validates JWTs using a cached JWKS from Keycloak.
	type KeycloakValidator struct {
	issuer string
	audience string
	keyFunc jwt.Keyfunc
	}
	func NewKeycloakValidator(cfg KeycloakConfig) (*KeycloakValidator, error) {
	jwksURL := fmt.Sprintf("%s/realms/%s/protocol/openid-connect/certs",
	cfg.IssuerURL, cfg.Realm)
	// JWKS keys are cached locally by the keyfunc library - no per-request calls to Keycloak.
	options := keyfunc.Options{
	RefreshInterval: cfg.JWKSCacheTTL,
	RefreshUnknownKID: true,
	// In production code, wire this to your logger instead of ignoring the error.
	RefreshErrorHandler: func(err error) {
	// log refresh error
	_ = err
	},
	}
	jwks, err := keyfunc.Get(jwksURL, options)
	if err != nil {
	return nil, fmt.Errorf("create JWKS from %q: %w", jwksURL, err)
	}
	return &KeycloakValidator{
	issuer: fmt.Sprintf("%s/realms/%s", cfg.IssuerURL, cfg.Realm),
	audience: cfg.Audience,
	keyFunc: jwks.Keyfunc,
	}, nil
	}
	// ValidateToken verifies the signature and standard claims of a Keycloak-issued JWT.
	func (v *KeycloakValidator) ValidateToken(ctx context.Context, rawToken string) (*jwt.RegisteredClaims, error) {
	claims := &jwt.RegisteredClaims{}
	token, err := jwt.ParseWithClaims(rawToken, claims, v.keyFunc,
	jwt.WithIssuer(v.issuer),
	jwt.WithAudience(v.audience),
	)
	if err != nil {
	return nil, fmt.Errorf("parse token: %w", err)
	}
	if !token.Valid {
	return nil, fmt.Errorf("invalid token")
	}
	return claims, nil
	}

[Copilot]

In the sequence diagram, the token exchange call is shown as POST /token, but elsewhere you use Keycloak’s token endpoint under /realms/<realm>/protocol/openid-connect/token. For Keycloak, token exchange uses the same token endpoint, so the diagram should use the full OIDC token URL to avoid misimplementation.

Suggested change

	Router->>KC: POST /token<br/>grant_type=urn:ietf:params:oauth:grant-type:token-exchange<br/>subject_token=<sdk_jwt><br/>audience=workloadmanager,picod
	Router->>KC: POST /realms/<realm>/protocol/openid-connect/token<br/>grant_type=urn:ietf:params:oauth:grant-type:token-exchange<br/>subject_token=<sdk_jwt><br/>audience=workloadmanager,picod

[Copilot]

This states “When WorkloadManager creates an AgentRuntime or CodeInterpreter…”, but in the current architecture WorkloadManager creates Sandbox/SandboxClaim resources based on existing AgentRuntime/CodeInterpreter CRDs (it doesn’t create those CRDs). Suggest rewording to reflect labeling the sandbox (and/or its PodTemplate) created for a given AgentRuntime/CodeInterpreter invocation.

Suggested change

1. **Creation Tagging:** When WorkloadManager creates an `AgentRuntime` or `CodeInterpreter`, it reads the `sub` (user ID) and optionally the `groups` claim from the exchanged Keycloak JWT. It applies these as immutable labels on the generated Kubernetes CRDs and Pods (e.g., `agentcube.io/owner: user-123`, `agentcube.io/group: engineering`).

1. **Creation Tagging:** When WorkloadManager provisions a sandbox (e.g., `Sandbox` / `SandboxClaim`) for an invocation of an existing `AgentRuntime` or `CodeInterpreter` CRD, it reads the `sub` (user ID) and optionally the `groups` claim from the exchanged Keycloak JWT. It applies these as immutable labels on the sandbox resources and their underlying Pods/PodTemplates (e.g., `agentcube.io/owner: user-123`, `agentcube.io/group: engineering`).

[Copilot]

The table claims JWT mode has “Automatic (Router key rotation)”, but the current Router JWT implementation stores a long-lived RSA keypair in the picod-router-identity Secret and doesn’t rotate it automatically. If the proposal includes adding rotation, it would help to call that out explicitly (how/when rotation happens and how PicoD gets the new public key) rather than implying it already exists.

Suggested change

	| **Key rotation** | Automatic (SPIRE 1h TTL or cert-manager) | Automatic (Router key rotation) |
	| **Key rotation** | Automatic (SPIRE 1h TTL or cert-manager) | Manual (long-lived Router keypair in `picod-router-identity` Secret; no automatic rotation yet) |

[hzxuzhonghu]

generally LG, @YaoZengzeng to have a another look

[volcano-sh-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hzxuzhonghu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• docs/OWNERS [hzxuzhonghu]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[YaoZengzeng]

/lgtm