Skip to content

ui: CWD for agent - #26518

Merged
allozaur merged 29 commits into
ggml-org:masterfrom
allozaur:feat/agent-cwd
Aug 4, 2026
Merged

ui: CWD for agent#26518
allozaur merged 29 commits into
ggml-org:masterfrom
allozaur:feat/agent-cwd

Conversation

@allozaur

@allozaur allozaur commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Overview

Additional information

Requirements

Comment thread tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessage.svelte Outdated
Comment thread tools/ui/src/lib/hooks/use-auto-scroll.svelte.ts
Comment thread tools/ui/src/lib/services/database.service.ts Outdated
Comment thread tools/ui/src/lib/services/tools.service.ts Outdated
Comment thread tools/ui/src/lib/stores/chat.svelte.ts
Comment thread tools/ui/src/lib/utils/path-display.ts
Comment thread tools/ui/src/lib/components/app/chat/ChatForm/ChatForm.svelte
Comment thread tools/ui/src/lib/utils/working-directory.ts
Comment thread tools/ui/src/lib/utils/path-display.ts
@allozaur allozaur changed the title Feat/agent cwd ui: CWD for agent Aug 4, 2026
@allozaur
allozaur marked this pull request as ready for review August 4, 2026 09:39
@allozaur
allozaur requested review from a team as code owners August 4, 2026 09:39
Copilot AI lite review requested due to automatic review settings August 4, 2026 09:39
@allozaur
allozaur requested a review from ngxson August 4, 2026 09:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds working-directory (CWD) support to the web UI’s agent/tooling flow: users can pick a server-side working directory, tool calls execute relative paths against it via a request header, and the UI records/display CWD changes as synthetic chat rows. On the server side, file_glob_search gains directory listing + depth/limit controls and a new get_info tool is surfaced in the UI.

Changes:

  • UI: adds a working-directory picker (popover + results list) and synthetic CWD-change message rendering; persists CWD on conversations and on tool-result messages.
  • UI/Tools: threads cwd into built-in tool executions via x-tool-cwd and adds helpers for path display/parsing and picker search.
  • Server: extends file_glob_search (type/max_depth/limit + structured {entries, base} response) and adds/extends unit tests around cwd behavior and glob search.

Reviewed changes

Copilot reviewed 38 out of 42 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
tools/ui/tests/unit/working-directory.test.ts New unit tests for working-directory picker/search helper utilities.
tools/ui/tests/unit/tool-calls.test.ts Adds tests for new path-display + cwd message format/parser helpers.
tools/ui/src/lib/utils/working-directory.ts New pure helpers for working-directory picker search and ranking.
tools/ui/src/lib/utils/path-display.ts New helpers for path abbreviation and synthetic cwd message format/parse.
tools/ui/src/lib/utils/index.ts Re-exports new working-directory/path-display helpers from $lib/utils.
tools/ui/src/lib/utils/agentic.ts Extends AgenticSection with toolCwd sourced from tool result messages.
tools/ui/src/lib/types/database.d.ts Adds persisted conversation cwd, plus per-message toolCwd/isSynthetic.
tools/ui/src/lib/types/chat.d.ts Extends tool-result message callback signature to accept toolCwd.
tools/ui/src/lib/types/agentic.d.ts Extends agentic callbacks to accept toolCwd on tool result creation.
tools/ui/src/lib/stores/tools.svelte.ts Adds server home resolution via file_glob_search and caches it for abbreviations.
tools/ui/src/lib/stores/conversations.svelte.ts Persists conversation cwd; adds pendingCwd for new-chat screen; adds setCwd().
tools/ui/src/lib/stores/chat.svelte.ts Records cwd changes as synthetic messages; injects pending cwd before first user turn.
tools/ui/src/lib/stores/agentic.svelte.ts Threads conversation cwd into tool execution/streaming calls and tool-result creation.
tools/ui/src/lib/services/tools.service.ts Adds x-tool-cwd header support; adds executeToolRaw(); threads cwd into streaming.
tools/ui/src/lib/services/database.service.ts Ensures forked conversations preserve cwd.
tools/ui/src/lib/hooks/use-auto-scroll.svelte.ts Adjusts auto-scroll behavior for discrete synthetic row injection.
tools/ui/src/lib/enums/ui.enums.ts Adds file: to UrlProtocol enum.
tools/ui/src/lib/enums/tools.enums.ts Adds GlobSearchType enum and GET_INFO built-in tool identifier.
tools/ui/src/lib/enums/index.ts Re-exports GlobSearchType.
tools/ui/src/lib/constants/working-directory.ts Adds constants/tuning for working-directory picker glob search behavior.
tools/ui/src/lib/constants/tools.ts Defines X_TOOL_CWD_HEADER constant.
tools/ui/src/lib/constants/path-display.ts Adds constants and regexes for synthetic cwd message encoding/parsing.
tools/ui/src/lib/constants/index.ts Re-exports new constants modules.
tools/ui/src/lib/constants/built-in-tools.ts Adds UI metadata for GET_INFO tool block.
tools/ui/src/lib/components/app/chat/index.ts Exports new working-directory selector and synthetic message components.
tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockWriteFile.svelte Abbreviates file paths under server home in tool-call UI.
tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockGrepSearch.svelte Abbreviates grep path under server home in tool-call UI.
tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockGetInfo.svelte New UI block for GET_INFO tool results.
tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockFileGlobSearch.svelte Abbreviates glob-search base path under server home in tool-call UI.
tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockExecShellCommand.svelte Displays per-call working directory for exec_shell_command results.
tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockEditFile.svelte Abbreviates edit target path under server home in tool-call UI.
tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlock.svelte Routes GET_INFO tool calls to the new tool-call block component.
tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageSynthetic.svelte New synthetic-message wrapper that routes cwd changes to a dedicated renderer.
tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageCwdChange.svelte New synthetic cwd-change row renderer (folder row UI).
tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessage.svelte Detects synthetic/cwd messages and renders via synthetic component; adjusts intrinsic sizing.
tools/ui/src/lib/components/app/chat/ChatForm/ChatFormWorkingDirectoryResultsList.svelte New picker results list UI with match highlighting.
tools/ui/src/lib/components/app/chat/ChatForm/ChatFormWorkingDirectoryChip.svelte New chip UI for showing/clearing the active cwd.
tools/ui/src/lib/components/app/chat/ChatForm/ChatFormWorkingDirectory.svelte New working-directory picker component wired to file_glob_search.
tools/ui/src/lib/components/app/chat/ChatForm/ChatForm.svelte Integrates the working-directory selector into the chat form.
tools/ui/src/app.d.ts Adds Window.showDirectoryPicker typing for File System Access API usage.
tools/server/tests/unit/test_tools_builtin.py Adds unit tests for file_glob_search type/depth/limit and cwd header path resolution.
tools/server/server-tools.cpp Implements file_glob_search enhancements + structured response; adds get_info output sizing constants; refactors listing internals.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tools/ui/src/lib/utils/working-directory.ts
Comment thread tools/ui/src/lib/constants/path-display.ts Outdated
Comment thread tools/ui/src/lib/stores/chat.svelte.ts
Comment thread tools/server/server-tools.cpp Outdated
Comment thread tools/ui/src/lib/components/app/chat/index.ts

@ServeurpersoCom ServeurpersoCom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM -> Linux pod tests

@allozaur

allozaur commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

@ngxson can u take a look at the server-side code please?

@ServeurpersoCom

ServeurpersoCom commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Opus 5 backend only review inside my testing context :

server-tools.cpp, list_entries_fallback

  • if (ec) break; inside the range-for is dead. ec is only written by the directory_iterator constructor, and when that fails the range is empty so the body never runs. Check it before the loop or drop it.
  • The range-for uses the throwing operator++, so an iteration error (a directory removed mid walk, a mount going away) escapes invoke as fs::filesystem_error instead of becoming a tool error. Use the explicit it.increment(ec) loop.
  • std::error_code ec is declared once outside the while and reused for every directory. It happens to work because the constructor resets it, but it reads like shared state. Declare it inside.
  • bool is_symlink = entry.is_symlink(tec); is computed for regular files too, where it is never read. Move it into the directory branch.
  • tec is passed to is_symlink, is_directory and is_regular_file and never checked in any of the three. Either check it or drop the overload.
  • auto [dir, rel_dir, depth] = stack.back(); copies two fs::path on every pop before pop_back(). This is the hot loop, move out of back() instead.
  • The deadline is only checked per entry inside a directory, so a tree of empty directories never reaches it. Check it at the top of the while too.

junk_dir_names()

  • The junk filter hides build, node_modules and dist from directory listings, not just from file search. Verified on this repo: type=dir, max_depth=1, include=build at the llama.cpp root returns 0 matches, so build/ cannot be selected as a working directory. That is a sane default for file_glob_search, much less so for a picker. Consider applying it only to list_kind::files, or exposing a parameter.

tools_io_basic::resolve

  • No normalization, so .. and . survive into the returned base and into git -C. A path like tools/../tools comes back to the UI verbatim. lexically_normal() on the result.

file_glob_search::invoke

  • params.at("path") is read twice, once for base and once to feed caller_path. Hoist it into a local.
  • list_entries now takes 6 parameters, two of them out, and caller_path exists only to build an error string. Either return a small result struct, or return the bare error and let the caller add the path.
  • if (limit < 1) limit = MAX; limit = std::min(limit, MAX); is two statements for one clamp, and it silently coerces a bad value while type returns an error for one. Pick a single policy. SERVER_TOOL_FILE_SEARCH_MAX_RESULTS is size_t and gets cast to int three times, just make it int.

list_entries

  • Worst case latency stacks: list_kind::files can spend SERVER_TOOL_GIT_LS_FILES_TIMEOUT (15 s) in git, fail, then spend SERVER_TOOL_FILE_SEARCH_TIMEOUT (5 s) in the walker. One deadline for the whole call would be more honest than two.
  • entry_depth counts separators on the git path while the walker carries its own depth counter. Verified they agree, but two implementations of the same notion deserve a comment saying so.

// into absolute paths. `plain_text_response` is what the model sees;
// `entries` is the same data as structured JSON for the UI picker,
// which reads `entries`/`base` instead of re-parsing the text.
return {{"plain_text_response", output_text.str()}, {"entries", entries_json}, {"base", base}};

This comment was marked as outdated.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nevermind, let's keep both responses for now to simply the handling logic on web ui

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For that one we'd need to change the client and the server together, since the WebUI falls back to dumping the raw JSON into the model's context when plain_text_response is missing.

@ngxson ngxson left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll let you and @ServeurpersoCom discuss if these issues are worth addressing in this PR, or better to be a follow-up

@ggerganov ggerganov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested briefly and it works nicely

@ServeurpersoCom

Copy link
Copy Markdown
Contributor

I've distilled my AI review this down to the essentials here, it stops the walk from crashing on a directory that disappears, makes build/ and node_modules/ selectable as a working directory without letting searches descend into them, and gives git and the walker a single 15s budget to share instead of one each. I can commit this:

server : harden the file_glob_search directory walk

step the directory iterator explicitly, so an entry that goes away mid
walk surfaces as a tool error instead of escaping as an exception

list junk directories without walking them, so they stay selectable as a
working directory while searches keep ignoring their contents

share a single time budget between the git and walker paths, instead of
one timeout each
diff --git a/tools/server/server-tools.cpp b/tools/server/server-tools.cpp
index 31c24cf8b..dbfd1b9f1 100644
--- a/tools/server/server-tools.cpp
+++ b/tools/server/server-tools.cpp
@@ -36,8 +36,8 @@ json server_tool::to_json() const {
 }

 static constexpr size_t SERVER_TOOL_GIT_LS_FILES_MAX_OUTPUT = 8 * 1024 * 1024; // 8 MB
-static constexpr int SERVER_TOOL_GIT_LS_FILES_TIMEOUT = 15; // seconds
-static constexpr int SERVER_TOOL_FILE_SEARCH_TIMEOUT = 5; // seconds
+// budget for one listing call, shared by the git and walker paths
+static constexpr int SERVER_TOOL_LIST_ENTRIES_TIMEOUT = 15; // seconds

 // entry kinds a directory listing may return
 enum class list_kind {
@@ -174,11 +174,13 @@ public:
             return {};
         }

+        const auto deadline = std::chrono::steady_clock::now() + std::chrono::seconds(SERVER_TOOL_LIST_ENTRIES_TIMEOUT);
+
         // git ls-files cannot list directories; use the walker when they are requested
         if (kind == list_kind::files) {
             auto res = run(
                 {"git", "-C", base, "ls-files", "--cached", "--others", "--exclude-standard"},
-                SERVER_TOOL_GIT_LS_FILES_MAX_OUTPUT, SERVER_TOOL_GIT_LS_FILES_TIMEOUT);
+                SERVER_TOOL_GIT_LS_FILES_MAX_OUTPUT, SERVER_TOOL_LIST_ENTRIES_TIMEOUT);

             if (res.exit_code == 0 && !res.timed_out) {
                 std::vector<list_entry> result;
@@ -197,7 +199,7 @@ public:
             }
         }

-        return list_entries_fallback(base, max_depth, kind, truncated);
+        return list_entries_fallback(base, max_depth, kind, deadline, truncated);
     }

     exec_result run(
@@ -284,12 +286,11 @@ private:
         return names;
     }

-    std::vector<list_entry> list_entries_fallback(const std::string & base, int max_depth, list_kind kind, bool & truncated) const {
+    std::vector<list_entry> list_entries_fallback(const std::string & base, int max_depth, list_kind kind,
+                                                  std::chrono::steady_clock::time_point deadline, bool & truncated) const {
         std::vector<list_entry> result;
         std::error_code ec;

-        auto deadline = std::chrono::steady_clock::now() + std::chrono::seconds(SERVER_TOOL_FILE_SEARCH_TIMEOUT);
-
         std::vector<std::tuple<fs::path, fs::path, int>> stack;
         stack.emplace_back(fs::path(base), fs::path(), 0);

@@ -297,22 +298,28 @@ private:
             auto [dir, rel_dir, depth] = stack.back();
             stack.pop_back();

-            for (const auto & entry : fs::directory_iterator(dir, fs::directory_options::skip_permission_denied, ec)) {
+            // the throwing increment would escape the tool on a directory that
+            // goes away mid walk, so step the iterator explicitly
+            fs::directory_iterator it(dir, fs::directory_options::skip_permission_denied, ec);
+            for (const fs::directory_iterator end; it != end; it.increment(ec)) {
                 if (ec) break;
                 if (std::chrono::steady_clock::now() >= deadline) {
                     truncated = true;
                     return result;
                 }
+                const fs::directory_entry & entry = *it;
                 std::string fname = entry.path().filename().string();
                 std::error_code tec;
                 bool is_symlink = entry.is_symlink(tec);
                 if (entry.is_directory(tec)) {
-                    if (junk_dir_names().count(fname) > 0) continue;
                     std::string rel = (rel_dir / fname).string();
                     std::replace(rel.begin(), rel.end(), '\\', '/');
                     if (kind == list_kind::dirs || kind == list_kind::all) {
                         result.push_back({rel, true});
                     }
+                    // junk directories stay selectable but are never walked: they
+                    // hold nothing worth searching and can be enormous
+                    if (junk_dir_names().count(fname) > 0) continue;
                     // do not descend into symlinks: a link can point back to an
                     // ancestor and loop forever
                     if (!is_symlink && (max_depth == 0 || depth + 1 < max_depth)) {

@ServeurpersoCom

ServeurpersoCom commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

That leaves only Windows letter support, so that you can enter "X:\...." I push a small UI only commit

recognize a drive root (C:) and a UNC share (//host/share) as path
navigation, alongside the POSIX root and ~, so a query like D:\repos
lists that directory instead of glob-matching it under the home dir

split below the root, so a bare drive resolves to its root rather than
to a drive-relative prefix

rewrite backslashes into forward slashes only when the query carries a
Windows root, since a backslash is a legal POSIX filename character

paths keep travelling with forward slashes, which is what the server
returns and what Windows accepts
@allozaur allozaur added the merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. label Aug 4, 2026
@allozaur
allozaur merged commit 2f56fc3 into ggml-org:master Aug 4, 2026
27 of 31 checks passed
smalinin pushed a commit to smalinin/llama.cpp that referenced this pull request Aug 4, 2026
* server : extend file_glob_search for UI pickers

* ui : add per-conversation working directory with picker

* ui : add path navigation and search scope to cwd picker

Treat path-like queries (starting with / or ~) as directory navigation
instead of glob-matching the whole query: search the parent for the last
segment, and descend into an exactly-typed directory by listing its
children. Show the effective search scope in the footer and auto-search
on open so the current directory and its siblings appear immediately.

Assisted-by: Claude

* db : persist per-call tool cwd on tool result messages

* ui : abbreviate tool paths under home with a tilde

* ui : show the per-call cwd on exec shell rows

* ui : clarify the synthetic cwd message for the model

* ui : reuse the trailing cwd row on a repeated pick

* ui : don't jump when a cwd row is injected mid-chat

* chore: Formatting

* refactor: Cleanup comments

* ui : unify working directory naming and add a synthetic-message flag

* ui : render synthetic cwd rows without a scroll jump

* ui : decouple the working directory picker into utils and sub-components

* ui : add get_info tool call block

* chore: Formatting

* refactor: Cleanup

* refactor: Cleanup

* refactor: Cleanup

* fix: UI

* server : harden file_glob_search listing (kind enum, timeout, symlink guard, absolute base)

* ui : use persisted isSynthetic flag for cwd rows, drop legacy formats

* ui : cache picker search, fail visibly on native resolve

* ui : escape glob metacharacters in picker search glob

* ui : simplify auto-scroll pin

* chore: Format

* fix: Use `SvelteMap`

* refactor: Post-review fixes

* ui: accept Windows roots in the working directory picker

recognize a drive root (C:) and a UNC share (//host/share) as path
navigation, alongside the POSIX root and ~, so a query like D:\repos
lists that directory instead of glob-matching it under the home dir

split below the root, so a bare drive resolves to its root rather than
to a drive-relative prefix

rewrite backslashes into forward slashes only when the query carries a
Windows root, since a backslash is a legal POSIX filename character

paths keep travelling with forward slashes, which is what the server
returns and what Windows accepts

---------

Co-authored-by: Pascal <admin@serveurperso.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. server/ui server

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants