Repository navigation
Replies: 9 comments 2 replies
|
I agree with these limitations of the agent. Personally, I see issue 3 (Agents are slow and expensive, can loop, and can't always be stopped) the most critical one. I saw users get frustrated when agent get stuck on a loop, sometimes they just leave it and it cost us a lot of money. I am not sure how we want to address these issues. One concern is that how much effort it take to fix these issues? I guess it may take a few months, so it has some conflict with the bigger vision (chatbot as the core of Texera). |
|
I checked the items against the agent-service code on |
|
I agree that it is important to stabilize this feature, even if we do plan to change it substantially in the future. The first 3 issues seem especially severe. Are there filed issues yet, for the ones described here? If not we should have them. |
|
I agree with the list, and with @aicam that 3 is the big one. It might be better to split it though, the quick stuff (lower step limit, a cost cap, Stop actually stopping, deleting an agent killing its work) seems doable before the 1.4 cutoff and covers the "user walks away and it burns money" case. The caching work could come after so it doesn't hold that up. For 2, I agree each user should only see their own agents, and not "locked" agents that other people have made. Persistence is definitely needed, but I agree it might not be feasible to finish before the cutoff. For 4, loading the full operator catalog likely makes every request bigger, which kind of works against 3. Maybe the agent could look up operators as it needs them instead if that is possible. Does the newer agent already fix any of these problems (namely 3)? If so it might be easier to bring that over than patch the current one. |
|
I agree on 1, 2, and 3. For 4, I remember users have to go to agent setting to enable other operators from the catalog. As @zyratlo said, loading the entire catalog makes every request bigger so we intentionally didn't let all the operators be available to the agent. But it would be nice if there is a way to optimize and minimize the request for loading the full operator catalog if possible. |
|
I agree with that 1,2,3,4 should be solved ASAP and should be included in release 1.4. @sshiv012 can you raise issues for them ? I want to take them |
|
I agree with the idea to stabilize the current agent before doing more development. @bobbai00 Can you give a very rough estimation of the time needed for each of the tasks? |
Uh oh!
There was an error while loading. Please reload this page.
The AI Agent panel has a few user-facing problems that make it hard to rely on. Below are the five we'd like to address in the next release (about 10 days out). Please share thoughts, pick-ups, or anything we've missed.
1. Agent chats can't be reopened, and model selection doesn't respond
What users see: Existing agent tabs show a lock icon and won't open. On the "+ Agent" tab, clicking a model card does nothing, so "Create Agent" stays greyed out.
Expected: Users can always reopen their own chats. If a chat belongs to a different workflow, show a clear "Open that workflow" option instead of a dead lock. Clicking a model selects it, and the agent can then be created. Add a neat UX feature that guides the user, some users found it confusing that they have to first click the model for the create agent button to be enabled, it is not super intuitive.
2. Agent panel shows agents that don't belong to you
What users see: The panel lists agents created by other people (shown as locked tabs), and the agent count includes them. Agents also vanish when the service restarts.
Expected: Each user sees only their own agents, and conversations aren't lost on a restart. The persistence part might be difficult to achieve so we can plan this for future release.
3. Agents are slow and expensive, can loop, and can't always be stopped
What users see: Sometimes an agent keeps adding or editing operators round after round without finishing. Pressing Stop doesn't always halt it and it can keep running after the tab is closed or the agent is deleted.
Even normal requests cost far more than they should. At every step the agent resends the whole conversation and workflow, rebuilt from scratch. Because of that, the model provider almost never reuses what it has already processed (a low prompt cache hit rate), so each step is billed as if it were the first. On top of that, the agent re-checks and re-runs the workflow after every edit. Cost and wait time grow quickly with each step, and a looping agent multiplies all of it.
Expected:
4. Agents can only use a small set of operators
What users see: The agent only builds workflows from a limited set of basic operators (file scan, filter, join, aggregate, a few charts, Python UDF). It can't use the rest of Texera's operator catalog, even when a better operator exists.
Expected: The agent can use all operators available in Texera. Users or admins can still narrow the list if they want to.
5. Adding new models requires a redeploy (optional for this release)
What users see: The model list in "Select Model Type" is fixed. Making a new model available requires an admin to change the deployment and restart litellm service. Picking a model that isn't actually usable only fails later, mid-conversation.
Expected: Admins can add or remove models from within Texera without a restart, and agent creation fails right away with a clear message if the chosen model isn't available.
Ask: Please comment if you've seen other agent problems we should add.
All reactions