Add runtime api design proposal#44
Conversation
Summary of ChangesHello @hzxuzhonghu, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a comprehensive design proposal for a declarative API to manage Agent and CodeInterpreter runtimes within the AgentCube platform. The core objective is to provide developers with a robust mechanism to define and control the desired state of their sandboxes, facilitating automatic provisioning upon initial invocation. This new API aims to overcome the limitations of generic sandbox templates by offering specialized configurations and lifecycle management tailored to the unique requirements of agent and code interpreter environments. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a design proposal for AgentRuntime and CodeInterpreterRuntime CRDs, along with their Go type definitions. The proposal is well-detailed and provides a solid foundation for the new APIs. My review focuses on addressing several inconsistencies between the design document and the Go code, correcting API definitions, and suggesting improvements for clarity and robustness. Key feedback points include removing non-standard JSON tags, aligning field requirements and naming conventions across the documentation and code, and ensuring that critical fields have appropriate validation.
| metav1.TypeMeta `json:",inline"` | ||
| // metadata is a standard object metadata | ||
| // +optional | ||
| metav1.ObjectMeta `json:"metadata,omitempty,omitzero" protobuf:"bytes,1,opt,name=metadata"` |
There was a problem hiding this comment.
The omitzero tag is not a standard Go json tag and will be ignored by encoding/json. It should be removed to avoid confusion and adhere to standard practices.
| metav1.ObjectMeta `json:"metadata,omitempty,omitzero" protobuf:"bytes,1,opt,name=metadata"` | |
| metav1.ObjectMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"` |
| RuntimeClassName *string `json:"runtimeClassName,omitempty" protobuf:"bytes,3,opt,name=runtimeClassName"` | ||
|
|
||
| // Image indicates the container image to use for the code interpreter runtime. | ||
| Image string `json:"image,omitempty" protobuf:"bytes,4,opt,name=image"` |
There was a problem hiding this comment.
The Image field is fundamental for a container runtime, yet it's marked with omitempty and lacks a +kubebuilder:validation:Required annotation. This field should be mandatory to prevent invalid configurations.
| Image string `json:"image,omitempty" protobuf:"bytes,4,opt,name=image"` | |
| Image string `json:"image" protobuf:"bytes,4,opt,name=image"` |
| metav1.TypeMeta `json:",inline"` | ||
| // metadata is a standard object metadata | ||
| // +optional | ||
| metav1.ObjectMeta `json:"metadata,omitempty,omitzero" protobuf:"bytes,1,opt,name=metadata"` |
There was a problem hiding this comment.
The omitzero tag is not a standard Go json tag and will be ignored by encoding/json. It should be removed to avoid confusion. This comment applies to all CRD definitions in this proposal where omitzero is used.
| metav1.ObjectMeta `json:"metadata,omitempty,omitzero" protobuf:"bytes,1,opt,name=metadata"` | |
| metav1.ObjectMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"` |
| DefaultVersion string `json:"defaultVersion"` | ||
| Versions map[string]string `json:"versions"` |
There was a problem hiding this comment.
The DefaultVersion and Versions fields in AgentRuntimeStatus are missing the omitempty JSON tag. This is inconsistent with CodeInterpreterRuntimeStatus (lines 301 and 307). It's a good practice to use omitempty for status fields to prevent them from being serialized when empty.
| DefaultVersion string `json:"defaultVersion"` | |
| Versions map[string]string `json:"versions"` | |
| DefaultVersion string `json:"defaultVersion,omitempty"` | |
| Versions map[string]string `json:"versions,omitempty"` |
|
|
||
| #### Why not we use existing SandboxTemplate? | ||
|
|
||
| There is a existing [`SandboxTemplate`](http://github.com/kubernetes-sigs/agent-sandbox/blob/main/extensions/api/v1alpha1/sandboxtemplate_types.go#L57) in the kubernetes-sigs/agent-sandbox project. However, it is designed to be a generic template for various sandbox types and may not cater specifically to the unique requirements of Agent and CodeInterpreter runtimes. |
There was a problem hiding this comment.
There's a minor grammatical error in this sentence.
| There is a existing [`SandboxTemplate`](http://github.com/kubernetes-sigs/agent-sandbox/blob/main/extensions/api/v1alpha1/sandboxtemplate_types.go#L57) in the kubernetes-sigs/agent-sandbox project. However, it is designed to be a generic template for various sandbox types and may not cater specifically to the unique requirements of Agent and CodeInterpreter runtimes. | |
| There is an existing [`SandboxTemplate`](http://github.com/kubernetes-sigs/agent-sandbox/blob/main/extensions/api/v1alpha1/sandboxtemplate_types.go#L57) in the kubernetes-sigs/agent-sandbox project. However, it is designed to be a generic template for various sandbox types and may not cater specifically to the unique requirements of Agent and CodeInterpreter runtimes. |
| // Languages declares the set of languages/runtimes that this code interpreter | ||
| // environment supports (e.g. ["python", "bash"]). This is metadata to help | ||
| // routers / UIs select a compatible runtime. | ||
| // +optional | ||
| Languages []string `json:"languages,omitempty" protobuf:"bytes,6,rep,name=languages"` |
There was a problem hiding this comment.
The Languages field is defined here in CodeInterpreterRuntimeSpec, but it is missing from the corresponding struct definition in the design document (docs/design/runtime-template-proposal.md). Please update the design document to include this field for consistency.
|
|
||
| type AgentRuntimeSpec struct { | ||
| // Ports is a list of ports that the agent runtime will expose. | ||
| Ports []TargetPort |
There was a problem hiding this comment.
The Ports field is missing a json tag. Based on the design document's YAML example, the tag should be ports. Adding omitempty is also recommended for slice fields.
| Ports []TargetPort | |
| Ports []TargetPort `json:"ports,omitempty"` |
| There is a existing [`SandboxTemplate`](http://github.com/kubernetes-sigs/agent-sandbox/blob/main/extensions/api/v1alpha1/sandboxtemplate_types.go#L57) in the kubernetes-sigs/agent-sandbox project. However, it is designed to be a generic template for various sandbox types and may not cater specifically to the unique requirements of Agent and CodeInterpreter runtimes. | ||
|
|
||
| - SandboxTemplate simply reuses pod template, making it hard to express multi-version runtimes. As it is common to have multiple versions of Agent or CodeInterpreter runtimes, a more specialized template is needed to manage these variations effectively. | ||
| - Different runtimes may have distinct configuration needs that are not adequately addressed by a generic pod template. Likely we need to support warmpool for code interpreter runtime, but not for agent runtime. Because code interpreter needs very lower latency for cold start, while agent runtime can afford longer cold start time. |
There was a problem hiding this comment.
The phrase "very lower" is grammatically incorrect. Consider using "very low" or "much lower" instead for better clarity.
| - Different runtimes may have distinct configuration needs that are not adequately addressed by a generic pod template. Likely we need to support warmpool for code interpreter runtime, but not for agent runtime. Because code interpreter needs very lower latency for cold start, while agent runtime can afford longer cold start time. | |
| - Different runtimes may have distinct configuration needs that are not adequately addressed by a generic pod template. Likely we need to support warmpool for code interpreter runtime, but not for agent runtime. Because code interpreter needs very low latency for cold start, while agent runtime can afford longer cold start time. |
| In order to expose the code interpreter internally or externally, we also need to define a `CodeInterpreterEndpoint` CRD, it can be used to bind to a specific `CodeInterpreterRuntime` version. | ||
| With both `CodeInterpreterRuntime` and `CodeInterpreterEndpoint` published, users can access the code interpreter runtime through the endpoint `https://<frontend>:<frontend-port>/v1/namespaces/{codeInterpreterNamespace}/codeinterpreters/{codeInterpreterName}/<code-interpreter specific path>?endpoint=<code-interpreter-endpoint>`. | ||
|
|
||
| `CodeInterpreterEndpoint` is similar as `AgentEndpoint`, it provides a way to access a specific version of that runtime. If no `CodeInterpreterEndpoint` is specified, the system will default to using the `DefaultVersion` defined in the `CodeInterpreterRuntime` status. For example, users can publish a python code interpreter runtime for different versions like `3.10`, `3.11`, `3.12`, then create different `CodeInterpreterEndpoint` to bind to different versions. |
There was a problem hiding this comment.
There's a minor grammatical error here. "similar as" should be "similar to".
| `CodeInterpreterEndpoint` is similar as `AgentEndpoint`, it provides a way to access a specific version of that runtime. If no `CodeInterpreterEndpoint` is specified, the system will default to using the `DefaultVersion` defined in the `CodeInterpreterRuntime` status. For example, users can publish a python code interpreter runtime for different versions like `3.10`, `3.11`, `3.12`, then create different `CodeInterpreterEndpoint` to bind to different versions. | |
| `CodeInterpreterEndpoint` is similar to `AgentEndpoint`, it provides a way to access a specific version of that runtime. If no `CodeInterpreterEndpoint` is specified, the system will default to using the `DefaultVersion` defined in the `CodeInterpreterRuntime` status. For example, users can publish a python code interpreter runtime for different versions like `3.10`, `3.11`, `3.12`, then create different `CodeInterpreterEndpoint` to bind to different versions. |
There was a problem hiding this comment.
Pull request overview
This PR introduces a declarative API design for managing Agent and CodeInterpreter runtimes in AgentCube. The proposal enables users to define runtime templates that automatically create sandboxes on first invocation, with support for multi-version management, session lifecycle controls, and specialized security configurations for code execution environments.
Key changes:
- Adds
AgentRuntimeandAgentEndpointCRDs for managing agent runtime environments - Adds
CodeInterpreterRuntimeandCodeInterpreterEndpointCRDs for managing code interpreter sandboxes with enhanced security controls - Provides comprehensive design documentation explaining the rationale and usage patterns
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 8 comments.
| File | Description |
|---|---|
pkg/apis/runtime/agent_type.go |
Defines API types for AgentRuntime and AgentEndpoint, including SandboxTemplate and TargetPort configuration |
pkg/apis/runtime/code_interpreter_type.go |
Defines API types for CodeInterpreterRuntime and CodeInterpreterEndpoint with specialized security-focused sandbox template |
docs/design/runtime-template-proposal.md |
Comprehensive design proposal documenting motivation, API design, usage examples, and alternatives |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| type AgentRuntimeSpec struct { | ||
| // Ports is a list of ports that the agent runtime will expose. | ||
| Ports []TargetPort |
There was a problem hiding this comment.
The Ports field is missing JSON and protobuf struct tags, which are present on all other fields in the spec. This inconsistency could cause serialization issues. Add the appropriate tags: json:\"ports,omitempty\" protobuf:\"bytes,5,rep,name=ports\"
| Ports []TargetPort | |
| Ports []TargetPort `json:"ports,omitempty" protobuf:"bytes,5,rep,name=ports"` |
|
|
||
| // PodTemplate describes the template that will be used to create an agent sandbox. | ||
| // +kubebuilder:validation:Required | ||
| Template *SandboxTemplate `json:"podTemplate" protobuf:"bytes,1,opt,name=podTemplate"` |
There was a problem hiding this comment.
The JSON field name podTemplate doesn't match the Go field name Template. This inconsistency can cause confusion. Consider either renaming the Go field to PodTemplate or changing the JSON field to template for consistency with the CodeInterpreterRuntime API which uses template for both.
| Template *SandboxTemplate `json:"podTemplate" protobuf:"bytes,1,opt,name=podTemplate"` | |
| Template *SandboxTemplate `json:"template" protobuf:"bytes,1,opt,name=podTemplate"` |
pkg/apis/runtime/agent_type.go
Outdated
| DefaultVersion string `json:"defaultVersion"` | ||
| Versions map[string]string `json:"versions"` |
There was a problem hiding this comment.
The AgentRuntimeStatus fields are missing the omitempty tag and protobuf annotations that are consistently used in CodeInterpreterRuntimeStatus. For consistency and to allow empty status values, add json:\"defaultVersion,omitempty\" protobuf:\"bytes,1,opt,name=defaultVersion\" and json:\"versions,omitempty\" protobuf:\"bytes,2,rep,name=versions\".
| DefaultVersion string `json:"defaultVersion"` | |
| Versions map[string]string `json:"versions"` | |
| DefaultVersion string `json:"defaultVersion,omitempty" protobuf:"bytes,1,opt,name=defaultVersion"` | |
| Versions map[string]string `json:"versions,omitempty" protobuf:"bytes,2,rep,name=versions"` |
pkg/apis/runtime/agent_type.go
Outdated
| // Target specifies the target agent runtime. | ||
| // +kubebuilder:validation:Required | ||
| Target string `json:"target" protobuf:"bytes,1,opt,name=target"` | ||
| // Version specifies the version of the agent runtime. |
There was a problem hiding this comment.
The Version field in AgentEndpointSpec lacks the +kubebuilder:validation:Required annotation, while the design document (line 235) indicates it should be required. This inconsistency with CodeInterpreterEndpointSpec (which also doesn't mark it as required) suggests unclear API semantics. If version is optional and defaults to DefaultVersion, document this behavior; otherwise, add the validation annotation.
| // Version specifies the version of the agent runtime. | |
| // Version specifies the version of the agent runtime. | |
| // +kubebuilder:validation:Required |
| // Languages declares the set of languages/runtimes that this code interpreter | ||
| // environment supports (e.g. ["python", "bash"]). This is metadata to help | ||
| // routers / UIs select a compatible runtime. | ||
| // +optional | ||
| Languages []string `json:"languages,omitempty" protobuf:"bytes,6,rep,name=languages"` | ||
|
|
||
| // WarmPoolSize specifies the number of pre-warmed sandboxes to maintain | ||
| // for this code interpreter runtime. Pre-warmed sandboxes can reduce startup | ||
| // latency for new sessions at the cost of additional resource usage. | ||
| // +optional | ||
| WarmPoolSize *int32 `json:"warmPoolSize,omitempty" protobuf:"varint,5,opt,name=warmPoolSize"` |
There was a problem hiding this comment.
[nitpick] The protobuf field number 6 comes after field number 5 (WarmPoolSize at line 63), but in the struct definition Languages appears before WarmPoolSize. This ordering mismatch between struct field order and protobuf field numbers can make the code harder to maintain. Consider reordering the struct fields to match the protobuf numbering sequence.
| // Languages declares the set of languages/runtimes that this code interpreter | |
| // environment supports (e.g. ["python", "bash"]). This is metadata to help | |
| // routers / UIs select a compatible runtime. | |
| // +optional | |
| Languages []string `json:"languages,omitempty" protobuf:"bytes,6,rep,name=languages"` | |
| // WarmPoolSize specifies the number of pre-warmed sandboxes to maintain | |
| // for this code interpreter runtime. Pre-warmed sandboxes can reduce startup | |
| // latency for new sessions at the cost of additional resource usage. | |
| // +optional | |
| WarmPoolSize *int32 `json:"warmPoolSize,omitempty" protobuf:"varint,5,opt,name=warmPoolSize"` | |
| // WarmPoolSize specifies the number of pre-warmed sandboxes to maintain | |
| // for this code interpreter runtime. Pre-warmed sandboxes can reduce startup | |
| // latency for new sessions at the cost of additional resource usage. | |
| // +optional | |
| WarmPoolSize *int32 `json:"warmPoolSize,omitempty" protobuf:"varint,5,opt,name=warmPoolSize"` | |
| // Languages declares the set of languages/runtimes that this code interpreter | |
| // environment supports (e.g. ["python", "bash"]). This is metadata to help | |
| // routers / UIs select a compatible runtime. | |
| // +optional | |
| Languages []string `json:"languages,omitempty" protobuf:"bytes,6,rep,name=languages"` |
| // Target specifies the target code interpreter runtime. | ||
| // +kubebuilder:validation:Required | ||
| Target string `json:"target" protobuf:"bytes,1,opt,name=target"` | ||
| // Version specifies the version of the code interpreter runtime. |
There was a problem hiding this comment.
The Version field in CodeInterpreterEndpointSpec is missing the +kubebuilder:validation:Required annotation. Based on the design document and API semantics, if this field is optional and defaults to DefaultVersion from the runtime status, this should be explicitly documented in a comment. Otherwise, add the validation annotation for consistency.
| // Version specifies the version of the code interpreter runtime. | |
| // Version specifies the version of the code interpreter runtime. | |
| // +kubebuilder:validation:Required |
| below is an example of how to define an `AgentRuntime` CRD for an agent runtime environment: | ||
| ```yaml | ||
| apiVersion: runtime.agentcube.io/v1alpha1 # adjust to your actual group/version |
There was a problem hiding this comment.
[nitpick] The example uses runtime.agentcube.io/v1alpha1 as the API group, but this doesn't match the actual package path (pkg/apis/runtime). For clarity, either update the example to use a consistent API group name or add a note explaining that the actual API group will be determined during implementation.
| below is an example of how to define an `AgentRuntime` CRD for an agent runtime environment: | |
| ```yaml | |
| apiVersion: runtime.agentcube.io/v1alpha1 # adjust to your actual group/version | |
| Below is an example of how to define an `AgentRuntime` CRD for an agent runtime environment. | |
| > **Note:** The `apiVersion` field below uses a placeholder value. The actual API group and version will be determined during implementation and should match your project's configuration (e.g., it may be derived from the Go package path such as `pkg/apis/runtime`). | |
| ```yaml | |
| apiVersion: <your-api-group>/v1alpha1 # replace with your actual group/version |
|
|
||
| type AgentRuntimeSpec struct { | ||
| // Ports is a list of ports that the agent runtime will expose. | ||
| Ports []TargetPort |
There was a problem hiding this comment.
The code example in the design document shows Ports []TargetPort without struct tags, which doesn't match the actual implementation in agent_type.go (line 22). While the implementation also lacks tags (which is a separate issue), the documentation should show the correct/intended API design with proper tags.
| Ports []TargetPort | |
| Ports []TargetPort `json:"ports,omitempty" protobuf:"bytes,1,opt,name=ports"` |
| Annotations map[string]string `json:"annotations,omitempty" protobuf:"bytes,2,rep,name=annotations"` | ||
| // Spec is the Pod's spec | ||
| // +kubebuilder:validation:Required | ||
| Spec corev1.PodSpec `json:"spec" protobuf:"bytes,3,opt,name=spec"` |
There was a problem hiding this comment.
shall we use PodTemplate instead of PodSpec here?
|
|
||
| // PodTemplate describes the template that will be used to create an agent sandbox. | ||
| // +kubebuilder:validation:Required | ||
| Template *SandboxTemplate `json:"podTemplate" protobuf:"bytes,1,opt,name=podTemplate"` |
There was a problem hiding this comment.
| Template *SandboxTemplate `json:"podTemplate" protobuf:"bytes,1,opt,name=podTemplate"` | |
| Template *SandboxTemplate `json:"Template" protobuf:"bytes,1,opt,name=Template"` |
Signed-off-by: Zhonghu Xu <xuzhonghu@huawei.com>
hzxuzhonghu
force-pushed
the
agent-code-runtime
branch
from
62abbc4 to
8dd121b
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| // Template describes the template that will be used to create an agent sandbox. | ||
| // +kubebuilder:validation:Required | ||
| Template *SandboxTemplate `json:"template" protobuf:"bytes,1,opt,name=template"` |
There was a problem hiding this comment.
The documentation example on line 82 references Template field but uses JSON tag template, while the actual code in agent_type.go line 26 uses JSON tag podTemplate. This creates an inconsistency between the documentation and the implementation. The JSON tag in the code should be changed to template to match the documentation and the field name.
| Template *SandboxTemplate `json:"template" protobuf:"bytes,1,opt,name=template"` | |
| Template *SandboxTemplate `json:"podTemplate" protobuf:"bytes,1,opt,name=podTemplate"` |
| There is a existing [`SandboxTemplate`](http://github.com/kubernetes-sigs/agent-sandbox/blob/main/extensions/api/v1alpha1/sandboxtemplate_types.go#L57) in the kubernetes-sigs/agent-sandbox project. However, it is designed to be a generic template for various sandbox types and may not cater specifically to the unique requirements of Agent and CodeInterpreter runtimes. | ||
|
|
||
| - SandboxTemplate simply reuses pod template, making it hard to express multi-version runtimes. As it is common to have multiple versions of Agent or CodeInterpreter runtimes, a more specialized template is needed to manage these variations effectively. | ||
| - Different runtimes may have distinct configuration needs that are not adequately addressed by a generic pod template. Likely we need to support warmpool for code interpreter runtime, but not for agent runtime. Because code interpreter needs very lower latency for cold start, while agent runtime can afford longer cold start time. |
There was a problem hiding this comment.
Typo in line 55: "lower" should be "low". The phrase should read "very low latency" instead of "very lower latency".
| - Different runtimes may have distinct configuration needs that are not adequately addressed by a generic pod template. Likely we need to support warmpool for code interpreter runtime, but not for agent runtime. Because code interpreter needs very lower latency for cold start, while agent runtime can afford longer cold start time. | |
| - Different runtimes may have distinct configuration needs that are not adequately addressed by a generic pod template. Likely we need to support warmpool for code interpreter runtime, but not for agent runtime. Because code interpreter needs very low latency for cold start, while agent runtime can afford longer cold start time. |
|
|
||
| ### Summary | ||
|
|
||
| This proposal outlines a design for introducing a declarative API to manage runtime specifically for Agent and CodeInterpreter runtimes in AgentCube. The goal is to enable users to define desired states for their sandboxes, allowing the system to automatically handle creation on the arrival of first invocation. The whole workflow out of the scope of this proposal. So we will focus on the declarative API design and about the whole system architecture, there will be another proposal. |
There was a problem hiding this comment.
Missing article in line 19: "to manage runtime specifically" should be "to manage runtimes specifically" or "to manage the runtime specifically".
| This proposal outlines a design for introducing a declarative API to manage runtime specifically for Agent and CodeInterpreter runtimes in AgentCube. The goal is to enable users to define desired states for their sandboxes, allowing the system to automatically handle creation on the arrival of first invocation. The whole workflow out of the scope of this proposal. So we will focus on the declarative API design and about the whole system architecture, there will be another proposal. | |
| This proposal outlines a design for introducing a declarative API to manage runtimes specifically for Agent and CodeInterpreter runtimes in AgentCube. The goal is to enable users to define desired states for their sandboxes, allowing the system to automatically handle creation on the arrival of first invocation. The whole workflow out of the scope of this proposal. So we will focus on the declarative API design and about the whole system architecture, there will be another proposal. |
| There is a existing [`SandboxTemplate`](http://github.com/kubernetes-sigs/agent-sandbox/blob/main/extensions/api/v1alpha1/sandboxtemplate_types.go#L57) in the kubernetes-sigs/agent-sandbox project. However, it is designed to be a generic template for various sandbox types and may not cater specifically to the unique requirements of Agent and CodeInterpreter runtimes. | ||
|
|
||
| - SandboxTemplate simply reuses pod template, making it hard to express multi-version runtimes. As it is common to have multiple versions of Agent or CodeInterpreter runtimes, a more specialized template is needed to manage these variations effectively. | ||
| - Different runtimes may have distinct configuration needs that are not adequately addressed by a generic pod template. Likely we need to support warmpool for code interpreter runtime, but not for agent runtime. Because code interpreter needs very lower latency for cold start, while agent runtime can afford longer cold start time. |
There was a problem hiding this comment.
Missing word on line 55: "warmpool for code interpreter runtime" should be "warmpool for code interpreter runtimes" or "a warmpool for code interpreter runtime".
| - Different runtimes may have distinct configuration needs that are not adequately addressed by a generic pod template. Likely we need to support warmpool for code interpreter runtime, but not for agent runtime. Because code interpreter needs very lower latency for cold start, while agent runtime can afford longer cold start time. | |
| - Different runtimes may have distinct configuration needs that are not adequately addressed by a generic pod template. Likely we need to support a warmpool for code interpreter runtime, but not for agent runtime. Because code interpreter needs very lower latency for cold start, while agent runtime can afford longer cold start time. |
|
|
||
| ### Alternatives | ||
|
|
||
| We can also design a restful api server to manage the lifecycle of agent runtimes, that will make the runtime management more flexible. However, it will introduce additional complexity in terms of deployment, scaling, and maintenance of the restful api server. So the first stage, we can make use of kubernetes CRD and operator to manage different kinds of runtime. |
There was a problem hiding this comment.
Article missing in line 333: "we can make use of kubernetes CRD" should be "we can make use of the Kubernetes CRD" or "we can make use of Kubernetes CRDs".
| We can also design a restful api server to manage the lifecycle of agent runtimes, that will make the runtime management more flexible. However, it will introduce additional complexity in terms of deployment, scaling, and maintenance of the restful api server. So the first stage, we can make use of kubernetes CRD and operator to manage different kinds of runtime. | |
| We can also design a restful api server to manage the lifecycle of agent runtimes, that will make the runtime management more flexible. However, it will introduce additional complexity in terms of deployment, scaling, and maintenance of the restful api server. So the first stage, we can make use of the Kubernetes CRD and operator to manage different kinds of runtime. |
|
|
||
| ### Alternatives | ||
|
|
||
| We can also design a restful api server to manage the lifecycle of agent runtimes, that will make the runtime management more flexible. However, it will introduce additional complexity in terms of deployment, scaling, and maintenance of the restful api server. So the first stage, we can make use of kubernetes CRD and operator to manage different kinds of runtime. |
There was a problem hiding this comment.
Grammar issue on line 333: "So the first stage" should be "So in the first stage" or "So for the first stage".
| We can also design a restful api server to manage the lifecycle of agent runtimes, that will make the runtime management more flexible. However, it will introduce additional complexity in terms of deployment, scaling, and maintenance of the restful api server. So the first stage, we can make use of kubernetes CRD and operator to manage different kinds of runtime. | |
| We can also design a restful api server to manage the lifecycle of agent runtimes, that will make the runtime management more flexible. However, it will introduce additional complexity in terms of deployment, scaling, and maintenance of the restful api server. So in the first stage, we can make use of kubernetes CRD and operator to manage different kinds of runtime. |
|
|
||
| ### Summary | ||
|
|
||
| This proposal outlines a design for introducing a declarative API to manage runtime specifically for Agent and CodeInterpreter runtimes in AgentCube. The goal is to enable users to define desired states for their sandboxes, allowing the system to automatically handle creation on the arrival of first invocation. The whole workflow out of the scope of this proposal. So we will focus on the declarative API design and about the whole system architecture, there will be another proposal. |
There was a problem hiding this comment.
Grammar issue on line 19: "The whole workflow out of the scope" should be "The whole workflow is out of the scope" or "The whole workflow is outside the scope".
| This proposal outlines a design for introducing a declarative API to manage runtime specifically for Agent and CodeInterpreter runtimes in AgentCube. The goal is to enable users to define desired states for their sandboxes, allowing the system to automatically handle creation on the arrival of first invocation. The whole workflow out of the scope of this proposal. So we will focus on the declarative API design and about the whole system architecture, there will be another proposal. | |
| This proposal outlines a design for introducing a declarative API to manage runtime specifically for Agent and CodeInterpreter runtimes in AgentCube. The goal is to enable users to define desired states for their sandboxes, allowing the system to automatically handle creation on the arrival of first invocation. The whole workflow is out of the scope of this proposal. So we will focus on the declarative API design and about the whole system architecture, there will be another proposal. |
|
|
||
| ### Design Details | ||
|
|
||
| #### Why not we use existing SandboxTemplate? |
There was a problem hiding this comment.
But we use its SandboxWarmPool which reference its existing SanboxTemplate?
There was a problem hiding this comment.
I mean why do we design a api instead of using SanboxTemplate
|
[APPROVALNOTIFIER] This PR is APPROVED Approval requirements bypassed by manually added approval. This pull-request has been approved by: The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
No description provided.