An advanced GNOME Shell extension primarily for seamless communication with Hera (Heuristic Reasoning Agent) through the system's Quick Settings.
This project facilitates real-time monitoring and interaction with autonomous agents, providing a discreet "drop window" that glides out from the screen edge for focused dialogue without breaking your workflow.
The core idea has been an iteration around how to interact with an AI that might (or more likely might not) reply in time. It is a kind of pseudo-synchronous interaction, where the replies are sequentially linked to the prompts, but they might be delayed, although usually not out of order. This is somewhat similar to another project I was involved in back in mid 90s, although not identical.
In its original 2019 version, the system included an experimental component. This was a dedicated observation system designed to manifest the agent’s internal cognitive state. Rather than treating the AI as a "black box," it tracked the latent resonance and shifting mental landscape of the agent as it processed information — effectively providing a visual window into the machine's internal reasoning process before a single token was ever externalized.
The agent as such is an instance of a model, and it is running in a daemon that acts as a runtime. The daemon is controlled through signals, and the agent accessed through pipes. Although possible, it is deemed unsafe to connect this kind of AI to the Dbus through the extension.
When controlling free-running AI's that run for a very long time, there are some inherently dangerous aspects. Such bots must be regarded as a possible threat in the same way as running random virus-infected code. A number of safeguards should be put in place, in particular it should not be allowed to roam free on the computer. That is why it won't get access to Dbus.
The extension can be seen as a lighter frontend to the bot, and the real sandboxing is done in the hera framework.
- Quick Settings Integration: Monitor agent health and switch between active contexts directly from the GNOME system menu.
- Drop Window Dialogue: A non-intrusive, collapsible chat interface designed for episodic memory exploration.
- Process Control: Send signals (SIGINT, SIGTERM, SIGKILL) and manage agent execution (STOP/CONT) directly from the UI.
- Context Injection: Attach plain-text files as context for the agent using the XDG Desktop Portal.
- Custom Status Mapping: Map arbitrary agent status strings to visual signal colors (Green/Yellow/Red) through preferences.
- Session Restoration: Automatically reloads previous dialogue history from agent logs.
To test the extension without a full Hera backend, follow this sequence:
The extension requires the run directory to exist. This must be created manually if Hera is not installed:
mkdir -p ${XDG_RUNTIME_DIR:-/run/user/$(id -u)}/heraThen the files are installed:
make installImportant: You must log out and back in (or restart GNOME Shell) now for the system to load the new code.
Once you are in the new session and the extension is enabled, you can generate data:
- Create agents:
make testagents(Creates JSON files and pipes in the run directory). - Create history:
make testlog(Creates simulated chat logs).
To simulate an agent that actually replies to messages and reacts to signals (with a 2-second delay to see the spinner in action), run:
make run-mock-daemonThis script monitors pipes and updates the JSON metadata when it receives signals such as SIGUSR1 (Pause) or SIGUSR2 (Start).
Automated unit tests are available for the core logic and utility functions:
make testThe testing strategy is intentionally lightweight, focusing on internal consistency and protocol adherence.
Agent Interaction acts as a frontend for agents adhering to the simple communication protocol used by hera, that is Heuristic Reasoning Agent. It expects agents to be present in:
/run/hera/(System-wide agents)/run/user/$UID/hera/(User-specific agents)
The protocol is fairly lightweight. Communication is handled via named pipes (.in and .out), control signals are used for urgent out-of-band actions, and the state of the agent is tracked through .json metadata files. Those files act as pid-files with additional data, in particular status (the state), identification, and pid code. Signals sent to the agents are assumed to trigger state changes that can be monitored in those files, but due to load issues there might be a delay before the change is visible, and thus we use tri-state buttons for control flows.
This is an experimental project with roots dating back to late 2019 (AgentInspector). It is designed for developers exploring agentic workflows.
Licensed under the GNU General Public License v3.0 only.
See the LICENSE file for the full text.
Copyright © 2019-2024 John Erling Blad
This extension is a continuation of work started in December 2019 (romjula), inheriting concepts and code from earlier agent monitoring experiments dating back to the summer of 2019.