Switch model on demand

[junglebells]

By default any request goes to the orchestrator first and then routed to the subagent as needed.

Given the orchestrator is usually a more expensive model, for certain requests I would like to send requests directly to the subagent. Is it possible to switch models on demand instead of changing the config file and relaunching.

Secondly, it would be nice to even have a 3rd level of subagent similar to what Claude offers

[mlhher]

for certain requests I would like to send requests directly to the subagent.

Do you mean e.g. having a command /subagent "Do xyz", so that you manually schedule a fresh subagent (irrespective from the orchestrator)? If so I think this could introduce issues since the subagents usually rely on the orchestrator doing the research first and giving it context, otherwise the subagent has to re-do all the research.

Is it possible to switch models on demand instead of changing the config file and relaunching.

I was generally thinking about having a model array inside the config.json so that a user can select them inside the TUI. Will need to take into account whether it's for orchestrator/subagent. Potentially with the researcher might also need to have model by subagent type.

Secondly, it would be nice to even have a 3rd level of subagent similar to what Claude offers

Can you explain what this third level would be for I am not sure I understand. Is it for like a different type of subagent (e.g. researcher gets model A, coder gets model B, orchestrator gets model C; if so would this be solved by the above configuration?) or something else?

[junglebells]

@mlhher - I should have given you some more context for this question.

I really like late-cli, and would love to extend it for more tasks beyond just planning/ coding, such as research, brainstorming ideas, and more. Currently, I use a number of LLM models, some free, others paid with varying price tiers, and would like to optimize which model to use for what task.

So, in the TUI it would be good to have a command: /agent_name "do xyz" and also: /switch_agent agent_name that changes the agent for subsequent interactions. The agent_name could be "orchestrator", "subagent", or one of the custom ones defined by the user, e.g. "researcher", "chatbot", "intern", "something".

For the models, I like your idea to have an array defined in the config.json file, something like this:
agent_name: orchestrator
base_url: <url>
api_key: <key>
model: <model name>
prompt: <instruction-planning.md>

agent_name: subagent
base_url: <url>
api_key: <key>
model: <model name>
prompt: <instruction-coding.md>

agent_name: researcher
base_url: <url>
api_key: <key>
model: <model name>
prompt: <instruction-researcher.md>

TUI picks the first agent by default, so the current functionality stays intact. However, it gives an option for a user to extend to new agents if needed. Perhaps you can also consider to provide ability to rename orchestrator/ subagent, so it is not hard-coded.

I am assuming it is possible to create new or modify the two seeded prompts (instruction-planning.md, instruction-coding.md). I am also assuming this is where one provides the ability to spawn another agent. So for example, if a "researcher" agent wants to spawn a "intern" agent, the instruction-researcher.md is where we specify that. (my example use case: After brainstorming ideas with a senior "researcher" agent, I could either `/switch_agent' or the researcher agent spawns junior "intern" agent to summarize and document key points)

As I am writing this, I am realizing that this might be bigger than what it seemed earlier. However, It would be great if you can consider adding these features. Happy to answer any questions and provide more clarifications. As always, thank you for creating late-cli - this is an excellent product. I am a big fan.

[mlhher]

Thank you for the clarification (and again the nice words). I am understanding it better now.
So the config.json addition (including TUI selection) will very likely come that is also something I had on my mind.

I am also assuming this is where one provides the ability to spawn another agent. So for example, if a "researcher" agent wants to spawn a "intern" agent, the instruction-researcher.md is where we specify that.

I do see the reasoning, though there are two things I do want to clarify:

First, we intentionally do not allow recursive subagents. A subagent should solve a single focused task, they should not have their identity muddled with the orchestrator. The goal is always to get the most out of the models capabilities and this is achieved, among other things, by clearly compartmentalizing identities and objectives.

Second, we cannot have the orchestrator prompt be changed. There is too much instructions to it (for example the fact that the orchestrator does intentionally not have the capabilities to write/edit files). If we allow changing it the user themselves have to take care to also include specific notes ("You cannot write files", "{{CWD}}" and potentially others) which I think is not user-friendly. Note that you may change the system prompt with the flags/env var but this is intentionally undocumented in the Quickstart.md (while --append-system-prompt is documented).

I do like the idea though of allowing users to specify custom subagents. For that we can internally manipulate the system prompt and tools definition; the only thing the user would need to provide is a config object (just like your example showed) and a tiny bit of metadata ("Testing Agent", "Use this agent to test the codebase"). Another advantage I see here is that we could share these for everyone to use.

Thank you again for the detailed explanation.

[junglebells]

So the config.json addition (including TUI selection) will very likely come that is also something I had on my mind.

Excellent, thank you, thank you!

we intentionally do not allow recursive subagents.

By recursive do you mean: orchestrator -> subagent -> sub-subagent. If so, completely agree. I would not want that either.

However, orchestrator -> coding subagent, and orchestrator -> testing subagent - that should be technically possible, no? Where coding and testing subagents are linked to different LLM models in the config file.

we cannot have the orchestrator prompt be changed.

I understand and accept. Would it be possible to create a new custom orchestrator/subagent pair though? As an example, "scientist"/"intern", where scientist is for brainstorming and intern is for scribing and summarizing.

Thanks again for everything, I'm excited.

[mlhher]

By recursive do you mean: orchestrator -> subagent -> sub-subagent. If so, completely agree. I would not want that either.

Exactly, Late will likely not support this unless there is some clear provable benefit/usecase for it (which I do not see whatsoever so far).

However, orchestrator -> coding subagent, and orchestrator -> testing subagent - that should be technically possible, no? Where coding and testing subagents are linked to different LLM models in the config file.

This is what I mean as well. The user provides some metadata and the Orchestrator is now able to also spawn the testing subagent.

I understand and accept. Would it be possible to create a new custom orchestrator/subagent pair though? As an example, "scientist"/"intern", where scientist is for brainstorming and intern is for scribing and summarizing.

The issue I see here is as already noted the Orchestrator has too much in its prompt that is mandatory to its behaviour. I will look into this.

[junglebells]

Thank you, looking forward to it. Best regards