v1.109: Draw the classification banner on the loading screen (#432)

Change log through v1.108 (#430)

Regenerated with node scripts/release-notes.mjs --changelog after the v1.108 release.

Query across libraries as one gufi_vt statement, and show the GUFI commands (#431)

A query over all libraries now runs as one gufi_vt statement where the Studio host's GUFI has gufi_vt: a virtual table per library, unioned, with the overall order and limit in the outer SELECT. A library on a cluster is a remote location whose command is the Studio's wrapper (gufiVtRemote.js). gufi_vt hands it the gufi_query command line, the wrapper splits it without a shell and passes it back to the server over loopback, and the server runs gufi_query on the cluster through the user's session. The server accepts only gufi_query with gufi_vt's stages on that location's own index, under a per-process secret and a per-query token.

Each location is checked first (the index exists, and on a cluster GUFI is installed), and one that cannot answer is left out and named beside the rows, since gufi_vt fails the whole statement when a location exits nonzero with no rows. Without gufi_vt on the host, for SQL containing a double quote, or when the statement fails, the per-library merge answers as before. The response's engine says which one answered.

Search, the Query page, and the chat's search_kb and query_corpus tool blocks now show the GUFI commands behind their results, as a shell takes them. Commands on the Studio host are shown as run. A cluster's command is shown under its name, to run there after pw ssh <cluster>. A gufi_vt statement is shown as a here-document in gufi_vt's usual remote form (remote_cmd='pw', remote_arg='ssh', remote_arg='<cluster>', with the cluster's GUFI added to the PATH there), so it runs from a terminal with GUFI and the pw CLI. The chat model does not receive the commands.

Tests: server/test/gufiVt.test.mjs covers the wrapper's splitting, the endpoint's checks, the generated script, and the command formatting. With GUFI installed, it indexes two synthetic trees, one local and one behind a fake remote session, and checks that gufi_vt returns the same rows as the merge for a ranked query, grouped totals, a builder filter, and raw SQL, with a missing library left out and named.

Draw the classification banner on the loading screen (#432)

In an installed app with the window controls overlay, the classification banner is the title bar. The loading screen drew before the app had its configuration, so the strip between the window controls showed the page's ground until the app mounted.

The app now caches the banner's text and colors beside the theme color, and the loading screen draws them from that cache: in the title bar area in that display mode, across the top otherwise, and not inside the platform frame unless the deployment shows the banner there. The first launch after this change still shows the gap, since the cache is written once the app has loaded.

v1.108

Change log through v1.107 (#428)

Regenerated with node scripts/release-notes.mjs --changelog after the v1.107 release.

One library or all: search, queries, statistics, the tree, and the chat (#429)

Search, the Query page, statistics, the Library tree, and the assistant's tools can now cover one library or all of them, whether read and write or read only, on this host or on a user's cluster. Until now the Query page and the chat used only the knowledge base, and nothing covered every library.

The Library switcher gains All libraries. The tree, Search, Query, Stats, and the index health popup follow it, and Search, Query, and Stats now show the switcher at the top, so the scope is visible where it applies.

What library=all does:

• The tree has one top-level folder per library, named by its id, with that library's own tree below. A path in it is <library id>/<path>. Files open and download from their library, and a cluster library's file comes from the cluster.
• Search runs in each library what that library supports: full text and semantic search where its index has them, and names always. It blends the hits and tags each with its library. A library that fails adds nothing, and the search goes on. Labels belong to the knowledge base, so a label filter keeps knowledge-base hits.
• The Query page has each library answer on its own, and the rows gain a leading library column.
  • For canned queries and the builder, whose sort order the Studio knows, rows are re-ranked into the overall top rows. Grouped totals, such as per extension, are added up.
  • Raw SQL rows are listed library by library.
  • A library that cannot answer is named beside the rows.
  • The Query page now works on any single library too, including a cluster library, with gufi_query run there.
• Statistics show a total and a line per library.

The All entry is read only.

The chat has a Library control beside Persona, which sets the library the assistant searches and queries in that conversation. The web client now sends that choice, which the server already accepted. The assistant's tools take a library argument: search_kb and query_corpus accept all, read_kb_file and list_kb_dir take one library's id, and a new list_libraries tool says which ids exist. A cluster library's files are read and listed on the cluster.

Fixes in the same area:

• Stale statistics: the Stats page and the footer loaded statistics once, so deleting a directory or pressing sync now left them stale until a full reload. Both now reload when the index changes and when the library changes, and deletes in the Library announce the change.
• Cluster file dates: a file opened from a cluster library showed 1970-01-01, because the preview did not read the file's size and modified time. It now reads them on the cluster in the same call.
• Cluster downloads: a cluster library's file now downloads through the session, up to 50 MB.
• Library paths: Your libraries rows show the index and source paths on the cluster, as the host libraries do.

Tests cover scope resolution, re-ranking and summed totals across two cluster libraries, a library that cannot answer, a single library on its own, tagged search hits, and the combined tree. The clusters are faked, through the remote session and a fake gufi_query.

A query across libraries as one gufi_vt statement is next. The Studio's own merge above stays as the fallback and as the check on its answers.

v1.107

Change log through v1.106 (#426)

Regenerated with node scripts/release-notes.mjs --changelog after the v1.106 release.

Your libraries: directories on your clusters, indexed and searched there (#427)

Users can add a directory on one of their clusters as their own library. The Studio indexes it there with the index workflow (#409), then browses and searches it on the cluster, as that user.

Adding. Settings, Libraries, "Your libraries" takes four inputs:

• a cluster from the user's active clusters;
• a directory, $WORKDIR by default;
• a label;
• optionally, a Slurm partition and account.

The Studio registers the workflow in the user's account the first time, named after its content hash so a changed workflow registers afresh. It runs the workflow with the user's own key, the one their chat uses. It follows the run while Settings is open, and when the run completes it reads the index summary from the run log, resolves the index and GUFI paths on the cluster, and marks the library ready.

Using. A ready library joins that user's Library switcher, labeled with its cluster, and no one else's. It is read only. The tree lists each folder from the index on the cluster, name search runs there, and text files open as a preview read from the cluster. Refresh reruns the workflow in update mode. Remove drops the library from the Studio and leaves the index on the cluster.

Remote commands. One long-lived pw ssh session per user and cluster carries the commands (server/src/remoteExec.ts). Each request is the command's argv, every argument base64-encoded, and the cluster decodes and runs it without a shell, so SQL and paths never meet shell quoting. On a customer cluster the first request took about 5 seconds for the ssh setup and later ones about 0.3 seconds.

Wiring. A user library's index root is an opaque reference (remote:<owner>:<id>), and gufi.ts sends any query on one to the cluster, so search and statistics work unchanged. The bundle and the container image now carry deploy/gufi-index.yaml for the Studio to register.

Live test on a customer cluster. The test used a synthetic directory of 6 folders and 6 files, and was removed afterwards. Adding it took 73 seconds to reach ready. The root listing then took 343 ms, a subfolder 308 ms, a name search 309 ms, and a text preview worked. The first try showed that this GUFI build names its depth flags --min-level and --max-level, and the tree uses those.

Not yet. Scheduled refresh, which needs the user's key when it falls due. Full-text and semantic search, which need enrichment. Searching several libraries at once through gufi_vt.

Tests cover the summary parsing, the workflow registration and its run inputs, a finished run making the library ready, its place in the library list, the tree listing, a name search through the remote path, and path checks. The fakes are pw, the remote session, and gufi_query.

v1.106

Change log through v1.105 (#425)

Regenerated with node scripts/release-notes.mjs --changelog after the v1.105 release.

A workflow that builds or updates a GUFI index on a cluster (#409)

deploy/gufi-index.yaml is an ACTIVATE workflow that builds or updates a GUFI index of a directory on a cluster, for remote libraries. It runs as the user who launches it, so the index covers what that user can read. The Studio does not launch it yet. It is written so the Studio can register it under each user's account and run it with that user's platform key, the pattern the agent runner uses.

It has four jobs. prepare checks the inputs and writes the index script. install_gufi builds GUFI's core tools from source on the login node into ~/.activate-studio/gufi/<commit>, once per commit, without the AI extensions. index hands the script to the platform's script_submitter subworkflow, which runs it on the login node or submits it to Slurm or PBS. record checks the script's exit status and prints the manifest the script wrote, .studio-index.json, between STUDIO_INDEX_BEGIN and STUDIO_INDEX_END in the run log.

The directory defaults to $WORKDIR, resolved on the cluster, or the home directory where $WORKDIR is not set. The index defaults to ~/.activate-studio/indexes/<name>-<hash> and must be outside the directory it indexes. update mode rescans the directories changed since the previous run with gufi_incremental_update; build writes a new index beside the old one and swaps it in. GUFI's incremental update creates a temporary file in the directory that holds the indexed one, so where that directory is not writable, as for a work directory under a site-owned parent, an update rebuilds instead. The GUFI source is its GitHub repository at a pinned commit, or a pw:// tarball for clusters without GitHub access.

The install step handles three differences found on a test cluster. The site's default modules set CC to a vendor compiler, so the build uses the system gcc. GUFI's configure needs Python 3.9 or later and the system python3 was 3.6, so it picks an installed python3.X that is new enough. setfattr and getfattr are required at configure time but used only by GUFI's tests, and were missing, so /bin/false stands in for them.

Tested on a Slurm cluster at an HPC center: the install from GitHub (about three minutes) and from a pw:// tarball; runs on the login node and as Slurm jobs; an update after files were added and removed and a directory created, which matched find's counts; a rebuild over an existing index; and a work directory of 3,510 directories and 21,764 files, indexed in 4 seconds on a compute node into a 180 MB index. The YAML validates against the platform's workflow schema. docs/LIBRARIES.md lists the inputs.

v1.105

Change log through v1.104 (#424)

Regenerated with node scripts/release-notes.mjs --changelog after the v1.104 release.

Inside the platform's page, a session opens on its ACTIVATE page (#423)

Sessions did not open when the Studio was itself open inside the platform's page. The Studio opened a scripted window, and a window opened from inside the platform's sandboxed frame inherits that frame's restrictions. When the browser refused the window, the Workflows list did nothing.

• Inside the platform's page: Open now reads Open in ACTIVATE and opens the session's page on the platform (/sessions/<user>/<name>, from the session record's internalHref) in a new tab, where the platform shows the session embedded. The tab opens through a link, which a sandboxed frame allows.
• Elsewhere: the session gets a window of its own as before, and now a new tab when the browser blocks the window, so a click always opens something.

Session records carry the new platformUrl field. docs/WORKFLOWS.md explains why the platform can embed sessions and the Studio cannot yet: the platform reads its own login on its own page and gets a one-time sign-in pass through /me/embed. The browser does not send that login to the route from the Studio's site, and the route takes no API key or CLI token.

v1.104

Change log through v1.103 (#421)

Regenerated with node scripts/release-notes.mjs --changelog after the v1.103 release.

pw code agents on the user's clusters, with goals (#422)

The assistant can run pw code agents on the user's clusters and follow, steer, and bound them with pw code's /goal.

Transport. A cluster's pw code daemon is reached the way pw code attach reaches it. The Studio server runs pw ssh <cluster> pw code daemon proxy --ensure, which bridges the ssh session to that machine's daemon socket and starts the daemon if none runs. The Studio then speaks the daemon's HTTP API over the pipe (server/src/pwcodeBridge.ts). One keep-alive bridge per user and cluster carries every request. On a customer cluster, the first request took 5.4 seconds for the ssh setup and later ones about 0.24 seconds. The bridge closes after 10 idle minutes. It needs only pw ssh and a pw CLI with pw code daemon proxy on the cluster, so it works where the platform's internal agent routes are not served. It also reaches the whole daemon API, goals included.

Credentials. A user's agents act with the key their chat uses, so each connection is that user's and runs as them on the cluster. The deployment's key stands in only when sign-in is off and the Studio has one user. On a shared Studio, a viewer without their own key gets no remote agent. Over the bridge the Studio has the owner's authority on that daemon, so it sets each session's permission mode: accept-edits by default, or read-only or plan.

Goals. start_agent creates a session in a workspace and sends the objective. The workspace is a new directory under ~/studio-agents unless one is named. When a goal is given, start_agent sets it through the daemon's /goal route, and pw code then checks it after every turn and continues until it is met. pw code puts no limit on a goal, so the Studio writes the bound into the condition, 20 turns by default. agent_status reports the goal's state, the last check, waiting approvals, and the end of the transcript.

Tools. agent_sites, agent_sessions, start_agent, agent_status, agent_steer, agent_goal, agent_approve, and agent_stop, with matching routes under /api/agent-sites for a page to come. Steering reaches a working turn, or becomes the next message when none is working. Approvals are answered only on the user's decision.

The local site. This host's daemon belongs to the account the Studio runs as. It is offered only when sign-in is off, or when AGENT_LOCAL_SITE=1.

The daemon client now takes a transport, and its request timeout is a timer, since a pipe has no socket timeout.

Tests. A fake daemon speaks HTTP over the bridge's stdio. The tests cover one bridge reused across calls, the site list, a start with a bounded goal, the status summary, steering fallback, goal controls, input checks, and the key rule.

Live run on a customer cluster. The bridge, session creation, the objective turn, and the goal check all ran. The check then reported the provider's monthly token limit, and the status showed it.

v1.103

Change log through v1.102 (#419)

Regenerated with node scripts/release-notes.mjs --changelog after the v1.102 release.

Workflow forms keep the declared input order; tests cannot reach the network (#420)

Workflow forms in the Studio listed their inputs in a different order from the platform's form. The Studio read each workflow's definition with pw workflows get -o json, and the CLI returns the YAML as a Go map, so every level of inputs comes back sorted by name. On one workflow the top-level groups arrived as cluster, doe, postprocessing, resource, software, study, while the workflow declares resource, cluster, doe, study, postprocessing, software. The platform API (GET /api/workflows/<name>) returns the declared order. Compared against the workflow's YAML file, it matched at all 39 levels of nesting.

The Studio now reads workflow records through the API with the viewer's key, and uses the CLI only when the API cannot answer. This covers:

• the workflow form;
• Add to my workflows, which writes the copy into the viewer's account. A copy written from the sorted map also showed its form alphabetically on the platform.
• the assistant's workflow tools, and model serving's input listing;
• composed workflows, so a composed form lists each source's inputs in their own order.

The shared reader is server/src/platformRecords.ts. The workflow graph view already read the API first.

Server tests now run with server/test/setup.mjs loaded first. It makes any fetch to a non-loopback host fail, so a test that forgets to stub fetch cannot reach a platform with the host's credentials. The full suite passes with it. A new form test serves inputs in declared order from the API and sorted from the CLI, and checks that the form follows the API. It fails without the change.

v1.102

Change log through v1.101 (#418)

Regenerated with node scripts/release-notes.mjs --changelog after the v1.101 release.

Sessions open in a window of their own; Workflows in the sidebar returns to its main page (#417)

Sessions now open in a window of their own instead of a frame inside the Studio. The frame did not hold up on a customer platform: a session the browser had just signed in to still refused its frame. Meanwhile the sign-in window loaded the session immediately, and that window is what users wanted.

• Opening: clicking a session under Workflows, Sessions, or in the runs table opens it in a window sized for work. A second click focuses the window the session already has. The window has no opener, so the session gets no handle back to the Studio, and the Studio keeps its own reference to reuse it.
• In chat: a session embed is a compact card with the session's state, the run that opened it, Open, and Open in new tab. The card reports its height to the chat, so the reply shows no empty box, and the full-screen button is hidden for embeds that report a height.
• Removed: the framed view, the sign-in step, and the browser-side framing rules (web/src/sessionFrame.ts and its tests). The assistant's session markdown is the card for every session.

Clicking Workflows in the sidebar now always returns to the Workflows main page, including from an open workflow form.

The user guide, docs/WORKFLOWS.md, and the README describe sessions this way.

v1.101

Change log through v1.100 (#416)

Regenerated with node scripts/release-notes.mjs --changelog after the v1.100 release.

Workflow form tooltips show their text (#414)

The help icons on the workflow form showed nothing on hover. The UI package's tooltips only mark their anchors with data-tooltip-id and the text. The text is shown by one shared tooltip, GlobalTooltip, which the host app renders once at its root, and the Studio never rendered it. It now renders inside UIProvider in web/src/main.tsx. This also covers the package's other anchored tooltips, such as the full text of a truncated label.

Checked in a browser against a workflow whose inputs carry tooltip: text. Before the change, hovering a help icon showed no tooltip element. After it, the input's text appears above the icon.

v1.100

Change log through v1.99 (#415)

Regenerated with node scripts/release-notes.mjs --changelog after the v1.99 release.

A workflow's README beside its form (#413)

The workflow page now shows the workflow's README beside the form, or below it on a narrow window. READMEs usually say how a workflow works and what each input changes. Hide README puts it away, and the choice is remembered.

Each kind of entry names where its README lives:

• An account workflow from a repository, including an account copy of a marketplace workflow, names its repository, branch, and README path in its record's remote block.
• A marketplace entry names them for each version; the Studio uses the version tagged latest.
• A GitHub entry reads the README.md in its workflow's directory.

The README is read through the platform's repository route with the viewer's key, which reaches private GitHub repositories and projects on registered GitLab servers. A GitHub README falls back to the raw file host. A workflow defined only on the platform has no README, and the page shows its description as before.

Relative links are rewritten so they work outside the repository. Images go through a new route, GET /api/workflows/item/asset, which reads image files from the same repository with the same key and nothing else. Other relative links go to the file's page on the repository's site. Code blocks and code spans are left as written. Only a definite answer is cached, either a README or a workflow with none, so a failed platform read is retried on the next view.

The user guide and docs/WORKFLOWS.md describe it. Tests cover the address forms, the README location for each kind of entry, the link rewriting, the routes, and the retry.