Add robust unified config system, memory retention with cleanup, expanded CLIs and CI-enabled test harness by akbad · Pull Request #1 · akbad/Bureau

akbad · 2025-12-10T14:51:15Z

Summary

Major additions:

Unified configuration system ➔ layered YAML config architecture for all of Beehive (agents, tools, paths, retention policies). Supports team defaults with local overrides.
Memory auto-cleanup ➔ retention-based cleanup for AI memory backends with soft-delete to trash before permanent removal.
PAL MCP integration ➔ dynamic per-CLI configuration generation for PAL (Gemini, Codex, Claude) with role-based prompt routing.
Developer experience ➔ Makefile for local CI commands, improved documentation, mypy type checking fixes.

Unified configuration system

A general-purpose config layer for Beehive dealing with:

Enabled agents (Claude, Gemini, Codex, OpenCode)
Tool server ports and paths
Retention periods per storage backend
File paths (expandable with ~ and env vars)

Config hierarchy

env vars
↑
local.yml (personal overrides)
↑
queen.yml (team settings)
↑
comb.yml (fixed/upstream)

Components (`lib/pollen/`)

config_loader.py ➔ central config loader with typed accessors, path expansion, duration parsing
config_cli.py ➔ shell interface (get-config agents, get-config --check agent claude)
validate_config.py ➔ config validation with type checking
CONFIGURATION.md ➔ full reference documentation

Memory auto-cleanup

How it works

Old memories are soft-deleted to .wax/trash/ with a 30-day grace period (configurable), then permanently removed. Runs automatically on beehive start or manually via CLI.

Handlers (`lib/pollen/cleanup/handlers/`)

Backend	Default retention period	Method
claude-mem	30d	SQLite DELETE + VACUUM
serena	90d	Move `.md` files to trash
qdrant	180d	REST API scroll + delete
memory-mcp	365d	JSONL rewrite

CLI scripts (`bin/`)

beehive-prune ➔ runs cleanup of stale memories based on configured retention periods
beehive-empty-trash ➔ purges trash (purgatory for stale memories) immediately
beehive-wipe ➔ complete data erasure

PAL MCP integration

Replaces static per-CLI JSON configs with dynamically generated configurations, allowing users to tweak exactly which models and roles to use with each CLI.

Components

configs/pal/settings.yaml ➔ base PAL settings and role definitions
configs/scripts/generate-pal-configs.py ➔ generates CLI-specific configs from templates
queen.yml ➔ pal: section controls model and role settings per CLI

Infrastructure

CI workflow (.github/workflows/ci.yml) ➔ pytest + mypy on push/PR
Makefile ➔ make test, make typecheck, make ci for local dev
Python packaging ➔ pyproject.toml with uv, dev dependencies
Type safety ➔ mypy-clean with cast() for TypedDict accessors, proper handler typing
Test suite ➔ fully covers handlers, trash, and state management

…o ignore uv-related metadata

…uv tool` and `check-prereqs` helper script instead of `python3`/`pip`/`brew`

…ory creation where necessary

…ning soon)

…trypoints, dependency groups and pytest config

…inted output

…ays` (from `never`)

…ntuitiveness

…ce `stale`/`valid` naming for memories that are past and within their configured retention period, respectively

…idance doc for agents

…te HTTP Zen MCP setup)

…to cleanup test suite's README

…implicitly rely on `ubuntu-latest` runner's Python version

…venience commands/scripts

akbad · 2025-12-26T11:10:58Z

Absolutely ridiculous scope creep. But the code is pristine. Goodnight

…nded CLIs, MCP fixes and CI-enabled test harness (#1)

Ranks all 7 platforms: Hermes Agent (#1), OpenHands (#2), Letta (#3), CoPaw (#4), OpenFang (#5), Memoh (#6), OpenClaw (#7). Recommends Hermes + OpenHands as optimal integration stack. Includes head-to-head matrix, top 10 feature merge ideas, tradeoff analysis, and phased next steps. https://claude.ai/code/session_01Ta99Gdof2dBsruDBdNRFZz

* Add agent framework evaluation: Bureau vs OpenClaw, CoPaw, Hermes, Memoh, LettaBot, LocalAGI Compares Bureau's competitive position against six always-on multi-channel assistant frameworks. Documents two-track development plan: Track A (multi-channel expansion) and Track B (dev tool improvements) with concrete sprint ordering. https://claude.ai/code/session_01Ta99Gdof2dBsruDBdNRFZz * Add integration analysis reports for CoPaw, Letta/LettaBot, Memoh, OpenFang Four detailed reports comparing Bureau integration fit across memory architecture, security model, channel support, SWE features, and autonomous capabilities. Hermes, OpenClaw, and OpenHands reports still in progress. https://claude.ai/code/session_01Ta99Gdof2dBsruDBdNRFZz * Add high-impact feature merge brainstorms to CoPaw, Letta, Memoh reports Each report now includes a Section 12 with 8-12 creative, technically specific ideas for Bureau integration that leverage the unique combination of both platforms. https://claude.ai/code/session_01Ta99Gdof2dBsruDBdNRFZz * Add high-impact feature merge brainstorms to Memoh and OpenFang reports Memoh: 12 ideas including per-role container isolation, parallel debugging swarms, and container-isolated MCP server vetting. OpenFang: feature merge section with autonomous Hands + Bureau coding role combinations. https://claude.ai/code/session_01Ta99Gdof2dBsruDBdNRFZz * Add OpenHands integration analysis report Covers Docker sandbox architecture, 77.6% SWE-bench performance, event stream state management, and integration proposal with Bureau as orchestrator and OpenHands as sandboxed execution backend. https://claude.ai/code/session_01Ta99Gdof2dBsruDBdNRFZz * Add high-impact feature merge brainstorms to OpenHands report 11 ideas including verified code reviews via sandbox execution, step-gated editing with test rollback, multi-model tournament execution, and real attack testing in Scrimmage Mode sandboxes. https://claude.ai/code/session_01Ta99Gdof2dBsruDBdNRFZz * Add Hermes Agent and OpenClaw integration analysis reports Hermes: 5-layer memory architecture, autonomous learning loop, 6 channels, 6 execution backends. Rated strongest integration candidate for Bureau. OpenClaw: 23+ channels, 346k stars, companion apps, voice/canvas. Maximalist option with highest reward/risk. 9 CVEs in March 2026. https://claude.ai/code/session_01Ta99Gdof2dBsruDBdNRFZz * Add high-impact feature merge brainstorms to Hermes Agent report 12 ideas including role-scoped evolving memory, Assess Mode with episodic recall, self-improving skills via feedback loop, zero- clarification dispatch, and cross-backend execution routing. https://claude.ai/code/session_01Ta99Gdof2dBsruDBdNRFZz * Add high-impact feature merge brainstorms to OpenClaw report 10 ideas including voice-driven code review via Talk Mode, Live Canvas diff theater, companion app Micro Mode approvals, ClawHub skill transpiler, and iMessage CI/CD war room. https://claude.ai/code/session_01Ta99Gdof2dBsruDBdNRFZz * Add synthesis README with ranked integration recommendations Ranks all 7 platforms: Hermes Agent (#1), OpenHands (#2), Letta (#3), CoPaw (#4), OpenFang (#5), Memoh (#6), OpenClaw (#7). Recommends Hermes + OpenHands as optimal integration stack. Includes head-to-head matrix, top 10 feature merge ideas, tradeoff analysis, and phased next steps. https://claude.ai/code/session_01Ta99Gdof2dBsruDBdNRFZz --------- Co-authored-by: Claude <noreply@anthropic.com>

akbad added 21 commits December 7, 2025 07:35

Add scaffolding and key functionality for memory auto-cleanup feature

e18aca9

Add minor formatting/comment fixes

8ac79ce

Add YML config files (for planned implementation) and update gitignore

79d4896

Add Python project files + prereq verif script and update gitignore t…

c949820

…o ignore uv-related metadata

Port script invocations and shebangs to use the prescribed uv run/`…

b4db151

…uv tool` and `check-prereqs` helper script instead of `python3`/`pip`/`brew`

Streamline config files (corresponding config code updates wip)

a81a5c7

Update agent context files to ensure time metadata is included in mem…

5b5b619

…ory creation where necessary

Update pyproject version tag to 0.1.0 (since will use semantic versio…

1657641

…ning soon)

Refactor config loading and validation for DRYness

67ecf7a

Correct comment in base handler

5edce10

Move guides under docs/ and add config reference

f8d1bf1

Update project files to add sweep-hive and validate-config CLI en…

bf09bd5

…trypoints, dependency groups and pytest config

Refactor prerequisite check script for clarity and standardize its pr…

6b7cf68

…inted output

Make setup/boot scripts config-driven and align tool deps

c0fc9ee

Refine file-level docstrings in scripts

f5553e0

Add SQLite coverage database to gitignore

7e4add4

Add cleanup scripts scaffolded on top of sweep-hive

59629d4

Fix minor style issue in agent selection utils

5d43d15

Add cleanup implementation details to config doc

e9a9009

Add partially-complete READMEs for cleanup module and its test suite

3df209e

Refine inline docs in queen.yml config file

f81a641

akbad changed the title ~~Add robust, unified config system, memory retention with cleanup, expanded CLIs and test harness~~ Add robust, unified config system, memory retention with cleanup, expanded CLIs and CI-enabled test harness Dec 11, 2025

akbad changed the title ~~Add robust, unified config system, memory retention with cleanup, expanded CLIs and CI-enabled test harness~~ Add robust unified config system, memory retention with cleanup, expanded CLIs and CI-enabled test harness Dec 11, 2025

akbad added 7 commits December 11, 2025 03:52

Refactor to change infinite retention period to be specified via `alw…

9b5b8bc

…ays` (from `never`)

Add claude-mem handler description to cleanup module README

e18c8db

Add completed cleanup module README

b4a0d94

Rename and restructure config settings for consistency, clarity and i…

430acdf

…ntuitiveness

Add fixtures + related README content in cleanup test suite and enfor…

1dcb56b

…ce `stale`/`valid` naming for memories that are past and within their configured retention period, respectively

Add dummy scaffolding for lib package

fa17e4f

Remove unused imports from fixtures file

e7ce955

akbad added 17 commits December 20, 2025 04:58

Prefix pytest fixtures not explicitly used with underscores for clarity

b40102d

Add tests for claude-mem cleanup handler

3e32d15

Standardize formatting and streamline file ops instructions in MCP gu…

4a1d90a

…idance doc for agents

Add backend-specific cleanup handler for Memory MCP

cda7619

Update README to reflect new fixture naming

a793f3c

Add test suite for Memory MCP cleanup handler

9370803

Rename cleanup module's test fixtures for consistency

5402cff

Add CI workflows and documentation

59ceb43

Add Qdrant backend-specific cleanup handler

5165470

Add Serena backend-specific cleanup handler

294a3fd

Add Playwright as a browsing fallback in MCP tools guidance for agents

e2f68bd

Merge in latest fixes from main

09c8504

Add GitHub content retrieval guidelines for agents

d6f9a5f

Port beehive to configure PAL MCP per-CLI via stdio (replacing obsole…

0ed4819

…te HTTP Zen MCP setup)

Make PAL CLI-specific settings dynamically generated and configurable

2d56f49

Add remaining test files to cleanup module and add remaining updates …

46f9440

…to cleanup test suite's README

Explicitly specify Python version to properly enable caching and not …

eb7c267

…implicitly rely on `ubuntu-latest` runner's Python version

akbad force-pushed the feature/auto-delete-retention branch from 4f6fea2 to eb7c267 Compare December 26, 2025 09:19

akbad added 6 commits December 26, 2025 04:23

Enable manual triggering for CI

368c786

Fix version-file setting in uv-run ci workflow

b701460

Test CI without explicit Python caching

130fcf1

Add convenience commands for mirroring CI checks and document all con…

d23cd8a

…venience commands/scripts

Document PAL-related config additions/changes

74c3046

Fix miscellaneous minor typing issues and warnings

f2b359f

akbad merged commit bac158e into main Dec 26, 2025
2 checks passed

akbad deleted the feature/auto-delete-retention branch December 26, 2025 11:11

akbad added a commit that referenced this pull request Mar 12, 2026

Add robust unified config system, memory retention with cleanup, expa…

ddfc3ed

…nded CLIs, MCP fixes and CI-enabled test harness (#1)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add robust unified config system, memory retention with cleanup, expanded CLIs and CI-enabled test harness#1

Add robust unified config system, memory retention with cleanup, expanded CLIs and CI-enabled test harness#1
akbad merged 65 commits intomainfrom
feature/auto-delete-retention

akbad commented Dec 10, 2025 •

edited

Loading

Uh oh!

akbad commented Dec 26, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

akbad commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Unified configuration system

Config hierarchy

Components (lib/pollen/)

Memory auto-cleanup

How it works

Handlers (lib/pollen/cleanup/handlers/)

CLI scripts (bin/)

PAL MCP integration

Components

Infrastructure

Uh oh!

akbad commented Dec 26, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

akbad commented Dec 10, 2025 •

edited

Loading

Components (`lib/pollen/`)

Handlers (`lib/pollen/cleanup/handlers/`)

CLI scripts (`bin/`)