Skip to content

Route only explicit Codex and Claude invocations through smart routing - #502

Merged
lilly-luo merged 8 commits into
mainfrom
stack/lilly/only-do-smart-routing-for-2-cases-codex
Sep 8, 2026
Merged

Route only explicit Codex and Claude invocations through smart routing#502
lilly-luo merged 8 commits into
mainfrom
stack/lilly/only-do-smart-routing-for-2-cases-codex

Conversation

@lilly-luo

@lilly-luo lilly-luo commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

https://databricks.atlassian.net/browse/AIGTWY-4551

explanation

currently, we are invoking smart routing for /every/ codex and claude command. in reality, we only want to invoke smart routing in 2 cases:

  • claude / codex empty - this opens a tui and we route on the first prompt
  • claude -- 'hi' / codex -- 'hi' - this opens a tui with 'hi' as the first prompt.;

claude --model xx or codex --model yy should not invoke smart routing.

this PR makes 3 changes so that ucode codex upgrade and ucode codex app-server will work:

  1. makes it so that we detect whether there's a prompt ('hi') or if there's no subcommand. if 1 of those is true, we invoke smart routing. otherwise, don't. this ensures things like ucode codex update and ucode codex app-server continue working without having to maintain a hardcoded subcommand list.
  2. after (1), we'd get this error on ucode codex update. This is because we, by default, pass the ucode config as argument --profile to all codex commands. however, only codex and codex mcp take --profile. instead, the other commands take --config. so i change codex to pass in --config instead
  Error: --profile only applies to runtime commands and `codex mcp`: `codex`, `codex exec`, `codex review`, `codex resume`, `codex queue`, `codex
  archive`, `codex delete`, `codex unarchive`, `codex fork`, `codex mcp`, `codex sandbox`, and `codex debug prompt-input`.
  ! ucode's `--profile` isn't accepted here (error above). Retrying without it: Codex will resolve /home/lilly.luo/.codex/config.toml and any
  OS-managed settings instead of the ucode profile.
  1. even after (2), ucode codex upgrade would return the error below. this is because when we pass --config instead of --profile to codex, we need to serialize the toml file. the serialization logic had to be updated as a result of tomlkit upgrading to 15.0.

finally after 1, 2, and 3 - ucode codex upgrade and ucode codex app-server work.

  ValueError: Inline tables cannot contain a table

test

  • ENABLE_SMART_ROUTING_V2=1 SMART_ROUTER_NAME=task_v2 uv run ucode claude
  • ENABLE_SMART_ROUTING_V2=1 SMART_ROUTER_NAME=task_v2 uv run ucode codex
  • ENABLE_SMART_ROUTING_V2=1 SMART_ROUTER_NAME=task_v2 uv run ucode claude -- 'hi'
  • ENABLE_SMART_ROUTING_V2=1 SMART_ROUTER_NAME=task_v2 uv run ucode codex -- 'hi'
  • ENABLE_SMART_ROUTING_V2=1 SMART_ROUTER_NAME=task_v2 uv run ucode claude --model system.ai.glm-5-2
  • ENABLE_SMART_ROUTING_V2=1 SMART_ROUTER_NAME=task_v2 uv run ucode codex --model system.ai.glm-5-2
  • ENABLE_SMART_ROUTING_V2=1 SMART_ROUTER_NAME=task_v2 uv run ucode codex update
  • ENABLE_SMART_ROUTING_V2=1 SMART_ROUTER_NAME=task_v2 uv run ucode claude upgrade
  • ENABLE_SMART_ROUTING_V2=1 SMART_ROUTER_NAME=task_v2 uv run ucode codex app-server

the below cmds should be unchanged, i tested to ensure no regression

  • ENABLE_SMART_ROUTING_V2=0 uv run ucode claude
  • ENABLE_SMART_ROUTING_V2=0 uv run ucode codex
  • ENABLE_SMART_ROUTING_V2=0 uv run ucode claude --model system.ai.glm-5-2
  • ENABLE_SMART_ROUTING_V2=0 uv run ucode codex --model system.ai.glm-5-2
  • ENABLE_SMART_ROUTING_V2=0 uv run ucode claude -- 'hi'
  • ENABLE_SMART_ROUTING_V2=0 uv run ucode codex -- 'hi'
  • ENABLE_SMART_ROUTING_V2=0 uv run ucode codex update
  • ENABLE_SMART_ROUTING_V2=0 uv run ucode claude upgrade
  • ENABLE_SMART_ROUTING_V2=0 uv run ucode codex app-server

before (confirming i repro'd the issue) :

# version of ucode currently in isaac 
$ uv tool install \
  --default-index https://pypi-proxy.cloud.databricks.com/simple \
  'ucode==0.1.0+f7b4b97'

$ ENABLE_SMART_ROUTING_V2=1 ucode codex update    

╭────────────────────────────────────╮
│ Launching Codex with Unity Gateway │
╰────────────────────────────────────╯
✔ Starting Codex
Error: `--remote ws://127.0.0.1:40091` is only supported for interactive TUI commands, not `codex update`

$ ENABLE_SMART_ROUTING_V2=1 ucode codex app-server

╭────────────────────────────────────╮
│ Launching Codex with Unity Gateway │
╰────────────────────────────────────╯
✔ Starting Codex
Error: `--remote ws://127.0.0.1:32895` is only supported for interactive TUI commands, not `codex app-server`

after (it's fixed)

$ ENABLE_SMART_ROUTING_V2=1 uv run ucode codex update

╭────────────────────────────────────╮
│ Launching Codex with Unity Gateway │
╰────────────────────────────────────╯
✔ Starting Codex

Updating Codex via `bun install -g @openai/codex`...
bun add v1.3.14 (0d9b296a)

installed @openai/codex@0.153.4 with binaries:
 - codex

[1239.00ms] done

🎉 Update ran successfully! Please restart Codex.

$ ENABLE_SMART_ROUTING_V2=1 uv run ucode codex app-server (the 404 is unrelated and still happens with normal codex) 

╭────────────────────────────────────╮
│ Launching Codex with Unity Gateway │
╰────────────────────────────────────╯
✔ Starting Codex
2026-09-04T23:54:14.795144Z ERROR codex_models_manager::manager: failed to refresh available models: unexpected status 404 Not Found: {"error_code":"ENDPOINT_NOT_FOUND","message":"codex/v1/models is not enabled for this workspace."}, url: https://dbc-a5d4177a-49dc.cloud.databricks.com/ai-gateway/codex/v1/models?client_version=0.153.4, request id: fde50f25-d7db-4145-a011-9a49e892670b

@lilly-luo
lilly-luo force-pushed the stack/lilly/only-do-smart-routing-for-2-cases-codex branch from 6dd7576 to dcda6c4 Compare September 4, 2026 22:58
@lilly-luo
lilly-luo force-pushed the stack/lilly/only-do-smart-routing-for-2-cases-codex branch from dcda6c4 to 586eeef Compare September 4, 2026 23:10
@lilly-luo lilly-luo changed the title Route only explicit Codex TUI invocations through smart routing only route codex / claude (no prompt) and "codex -- '<>'" and "claude -- '<>'" to smart router Sep 4, 2026
@lilly-luo lilly-luo changed the title only route codex / claude (no prompt) and "codex -- '<>'" and "claude -- '<>'" to smart router only route codex / claude (no prompt) and codex -- '<>' and claude -- '<>' to smart router Sep 4, 2026
@lilly-luo
lilly-luo force-pushed the stack/lilly/only-do-smart-routing-for-2-cases-codex branch 2 times, most recently from c5e15dc to c8ec0f0 Compare September 4, 2026 23:47
@lilly-luo
lilly-luo marked this pull request as ready for review September 4, 2026 23:54
@lilly-luo
lilly-luo force-pushed the stack/lilly/only-do-smart-routing-for-2-cases-codex branch from c8ec0f0 to 354c435 Compare September 5, 2026 00:04
@lilly-luo lilly-luo changed the title only route codex / claude (no prompt) and codex -- '<>' and claude -- '<>' to smart router Route only explicit Codex and Claude invocations through smart routing Sep 5, 2026
Comment thread src/ucode/agents/codex.py Outdated
@lilly-luo
lilly-luo force-pushed the stack/lilly/only-do-smart-routing-for-2-cases-codex branch 3 times, most recently from 245a4a5 to 28bcd18 Compare September 5, 2026 00:17
@lilly-luo
lilly-luo force-pushed the stack/lilly/only-do-smart-routing-for-2-cases-codex branch from 28bcd18 to 9cd4c7e Compare September 5, 2026 00:20
Comment thread src/ucode/agents/claude.py
Comment thread src/ucode/agents/codex.py
Comment thread src/ucode/codex_config.py
@lilly-luo
lilly-luo requested a review from rohita5l September 5, 2026 01:52
@david-siqi-liu

Copy link
Copy Markdown
Collaborator

Do we show some indication in the CLI that smart routing has been enabled, after an user launches a session?

Comment thread src/ucode/agents/claude.py
@lilly-luo

Copy link
Copy Markdown
Collaborator Author

Do we show some indication in the CLI that smart routing has been enabled, after an user launches a session?

yes!
Screenshot 2026-09-08 at 11 19 04 AM

…s-codex

# Conflicts:
#	src/ucode/agents/codex.py
#	tests/test_codex_smart_routing_v2.py
@lilly-luo
lilly-luo enabled auto-merge (squash) September 8, 2026 15:35
@lilly-luo
lilly-luo disabled auto-merge September 8, 2026 15:41
@lilly-luo
lilly-luo merged commit 321dc54 into main Sep 8, 2026
2 checks passed
@lilly-luo
lilly-luo deleted the stack/lilly/only-do-smart-routing-for-2-cases-codex branch September 8, 2026 15:41
rohita5l added a commit that referenced this pull request Sep 11, 2026
Codex versions below 0.134.0 enter a configure/launch loop: configure
writes `[profiles.ucode]` into `~/.codex/config.toml` and validation
succeeds, but launch requires the newer `ucode.config.toml` file. This
restores compatibility lost in #502 by using the same version check as
configure and launching older Codex with `--profile ucode`.

Older versions also receive a warning on stderr to upgrade to 0.134.0 or
newer and verify the active installation with `codex --version`. Modern
and unknown versions retain the existing `--config` behavior for
runtime, utility, and server commands.

### Validation

- 105 focused tests pass across `test_agent_codex.py`,
`test_codex_config.py`, and `test_codex_smart_routing_v2.py`, covering
legacy versions, stale profile files, the version boundary, unknown
versions, and upgrade warnings.
- Ruff lint, formatting, and `git diff --check` pass.
- Tested the official Linux ARM64 Codex 0.129.0 binary in isolated
containers against a local Responses API test server. With UG e7553f4,
validation succeeds and both launch attempts fail with the reported
missing-file error. With this patch, validation and both real `codex
exec` launches exit 0, use the `ucode-databricks` provider, and show the
upgrade warning. Gateway authentication and responses use local test
equivalents.
lilly-luo added a commit that referenced this pull request Sep 11, 2026
only 2 cmds should invoke smart routing: 
- `ug codex` or `ug claude` which launches a tui that will smart route
the first prompt
- `ug codex -- <prompt>` or `ug claude -- <prompt>` which will launch a
tui with prompt prepopulated and then smart route it.

no other subcommands should trigger smart routing, even if the
environment variable is set. this came up because `isaac codex app`
which automatically has `enable_smart_routing_v2=1` set tried to invoke
smart routing...

this was missed in #502
because i didn't explicitly test `ug codex app` which apparently
functions diff from `ug codex app-server`

testing: 
- [x] `ENABLE_SMART_ROUTING_V2=1 uv run ug codex` -> invokes smart
routing
- [x] `ENABLE_SMART_ROUTING_V2=1 uv run ug claude` -> invokes smart
routing
- [x] `ENABLE_SMART_ROUTING_V2=1 uv run ug codex -- 'hi'` -> invokes
smart routing
- [x] `ENABLE_SMART_ROUTING_V2=1 uv run ug claude -- 'hi'` -> invokes
smart routing
- [x] `ENABLE_SMART_ROUTING_V2=1 uv run ug codex app` -> does NOT invoke
smart routing
- [x] `ENABLE_SMART_ROUTING_V2=1 uv run ug codex app-server` -> does NOT
invoke smart routing
- [x] `ENABLE_SMART_ROUTING_V2=1 uv run ug codex update` -> does NOT
invoke smart routing
- [x] `ENABLE_SMART_ROUTING_V2=1 uv run ug codex --model
system.ai.glm-5-2` -> does NOT invoke smart routing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants