Problem Statement
NemoClaw has a requirement to support multiple sandboxes with different inference provided (for example: OpenAI and Antropic))
See NemoClaw issue: NVIDIA/NemoClaw#1248
NemoClaw policy requires to direct the inferences via the inference router (inference.local). That means, provider endpoints should not be allowed to be accessed directly via relaxing the networking policies.
Proposed Design
OpenShell team is looking into the "provider v2" system, as discussed over slack.
OpenShell inference route is a cluster-wide singleton. OpenShell should support multiple inference routes (i.e. inference1.local, inference2.local and so on).
Alternatives Considered
NemoClaw policy requires to direct the inferences via the inference router (inference.local). That means, provider endpoints should not be allowed to be accessed directly via relaxing the networking policies.
So, this is not a feasible alternative.
Agent Investigation
No response
Checklist
Problem Statement
NemoClaw has a requirement to support multiple sandboxes with different inference provided (for example: OpenAI and Antropic))
See NemoClaw issue: NVIDIA/NemoClaw#1248
NemoClaw policy requires to direct the inferences via the inference router (inference.local). That means, provider endpoints should not be allowed to be accessed directly via relaxing the networking policies.
Proposed Design
OpenShell team is looking into the "provider v2" system, as discussed over slack.
OpenShell inference route is a cluster-wide singleton. OpenShell should support multiple inference routes (i.e. inference1.local, inference2.local and so on).
Alternatives Considered
NemoClaw policy requires to direct the inferences via the inference router (inference.local). That means, provider endpoints should not be allowed to be accessed directly via relaxing the networking policies.
So, this is not a feasible alternative.
Agent Investigation
No response
Checklist