关于 2.1.x 中的 HarnessAgent 事件仅存单kv 表的问题。

[hoojunev]

你好，我需要做一个多用户使用的平台级的 agent，打算使用最新的预发布版本，HarnessAgent 默认使用 eventLog模式，并且使用关系型数据库作为事件存储，我发现全部事件都会存在 agentscope-store的一个 kv 表中。

因為必然会增长很快，其次我没有发现框架有提供扩展支持业务接管单 kv 表这个设定。我想问这个模式真的推荐多用户或者多租户平台级 agent 的场景吗？

或者对于我的场景官方会比较推荐推荐使用 LEGACY 模式？

[Medamineelkhattabi]

Short answer: EVENT_LOG can work for a multi-tenant platform, and the 2.1 session features (AgentSession, HITL, resume after restart) are built on it. But in 2.1.0-BETA1 the framework doesn't scope, limit or clean the journal for you. The single agentscope_store table is only the default wiring. You can move the journal to its own table, put the tenant into the key, and delete old sessions yourself. I'd only pick LEGACY if you never need AgentSession.

I read the code at the v2.1.0-BETA1 tag.

1. What's in the table

The store table is (namespace_path, item_key, value_json, version, updated_at) with PRIMARY KEY (namespace_path, item_key) (MysqlDialect#L65-L86). Agent, user and session are already in item_key, which starts with SessionKey.storagePath():

agents/s_<b64(agentId)>/sessions/s_<b64(userId)>/s_<b64(sessionId)>/
  head.json                overwritten (CAS) on every commit
  commits/<sha256>.json    insert only, one row per flush
  blobs/<sha256>           payloads over 64 KiB
  inbox/...                same layout, for AgentSession commands

Commit rows are append-only (JournalSessionLog#L192-L272), the recorder flushes about once a second during a run, and each streamed model chunk is its own event. So you're right about growth: a user in #3816 measured 14.35 MB and 15,252 events after 12 turns.

2. Taking over the storage

• HarnessAgent.Builder.sessionLogStore(SessionLogStore) moves only the journal. Workspace files stay where they are (HarnessAgent#L1265-L1269).
• The JDBC dialect builder takes storeTableName(...), tablePrefix(...) and autoCreateTable(false) with any DataSource (AbstractJdbcDialectBuilder#L77-L87). With autoCreateTable(false) you create the table yourself (partitioning, extra indexes). The validator only checks that the columns exist.
• RemoteFilesystem(BaseStore, NamespaceFactory) computes namespace_path per call from the RuntimeContext.
• RedisStore and MongoBaseStore also work. For something fully custom (say, a table per tenant), implement SessionLogStore (open + list) and pick the backing store by tenant inside it (session-log.md).

What I'd do is give the journal its own table, with tenant and user in the namespace:

AbstractJdbcDialect logDialect = AbstractJdbcDialect.from(logDataSource)
        .storeTableName("agentscope_session_log")
        .build();
BaseStore logStore = JdbcStore.builder(logDataSource).dialect(logDialect).build();

NamespaceFactory tenantUser = rc -> List.of(
        "t", Objects.requireNonNull(rc.get("tenantId", String.class), "tenantId"), // your app sets this
        "u", rc.getUserId() == null ? "_anon" : rc.getUserId());

HarnessAgent agent = HarnessAgent.builder()
        .name("assistant").agentId("assistant").model(model)
        .distributedStore(JdbcDistributedStore.create(dataSource))   // files stay in agentscope_store
        .filesystem(new RemoteFilesystemSpec().isolationScope(IsolationScope.USER))
        .sessionLogStore(new WorkspaceSessionLogStore(new RemoteFilesystem(logStore, tenantUser)))
        .build();

The user goes in the namespace too because SessionLogStore.list() (behind session_list) pages through every row of a namespace and filters in Java (StoreSessionStorage#L43-L52). With one shared namespace, that means the whole table. Since the primary key starts with namespace_path, deleting a tenant is also a range delete. Pick the namespace before go-live: if you change it later, existing sessions are no longer found. There's no built-in tenant_id yet. #3356 collects use cases for one, so add yours there.

3. Retention

There is none: no TTL, no compaction, no delete method on SessionLogStore (memory.md#L295). A delete API is requested in #3612 (open). You can't trim old commits, because scan() walks the chain back from head.json and fails on a missing commit (JournalSessionLog#L320-L341). So delete whole idle sessions (no run in progress; the writer lease is 2 minutes), using the framework's own key encoding:

String ns = String.join("\u001F", "t", tenantId, "u", userId, "__agentscope_session_log_v1__") + "\u001F";
String prefix = new SessionKey(userId, agentId, sessionId).storagePath() + "/";
String pattern = prefix.replace("!", "!!").replace("%", "!%").replace("_", "!_") + "%";

DELETE FROM agentscope_session_log
WHERE namespace_path = ? AND item_key LIKE ? ESCAPE '!';

The escape matters, because the s_ markers and Base64URL both contain _. I rebuilt the key layout in SQLite to test this query: without the escape, the pattern for one idle session also matched another, active session. To find idle sessions, head.json is rewritten on every commit, so its updated_at is the last write (check inbox/head.json too and take the newer one). For very long sessions, SessionMigration.fork(...) starts a new session from the last checkpoint (SessionMigration#L100-L117), and then you delete the old one.

4. LEGACY

.legacySessionHistory(true) is described as a compatibility mode for backends without atomic storage (HarnessAgent#L1271-L1278). It saves the whole AgentState per userId:sessionId in agentscope_sessions when a call ends, so that row tracks state size, not event count, and delete(userId, sessionId) exists. It still writes two action_* rows per tool call that nothing prunes. What you lose: agent.session(ctx) (submit, steer, respond, resume), sessionLog and sessionTranscript throw "Native session history is disabled" (ReActAgent#L363-L371), the session_* tools aren't registered, and progress inside a run is lost if the process dies.

So: with background tasks, HITL, or a UI that reconnects, keep EVENT_LOG with the setup above plus a cleanup job. For plain request/response chat where you only need the latest state, LEGACY is simpler, but you still need cleanup.

One trap to know early: item_key is VARCHAR(255). With UUID user and session ids, an agentId longer than about 38 characters pushes inbox/commits/... keys past 255, and PostgreSQL rejects the insert (#4010; fix PR #3464 still open). Keep agent ids short, or create the table yourself with a wider item_key.

[hoojunev]

你说的把journallog 存到另外的表，意思只是指定自己的单表吗？那依然面临单表膨胀过快的问题。但是我没有看到能够支持自定义存储设计的扩展。框架依然依赖日志重放。我暂时只想到唯一可以减缓问题的办法是物理分表，例如建立agentscope_store_001，agentscope_store_002 .... 。然後做路由映射，保证同一用户的会话都在一个单表中。又或者直接使用分布式数据库。不知道我有没有理解错你前面说的，如果有请指正。

[Medamineelkhattabi]

You understood it right: a separate table on its own doesn't stop the growth. I meant two things together: give the journal its own table so it doesn't share agentscope_store with workspace files, and delete idle sessions on a schedule. Rows only pile up because nothing in 2.1.0-BETA1 removes them.

There is an extension point for custom storage, though. SessionLogStore is a public interface with just open(SessionKey, RuntimeContext) and an optional list(RuntimeContext) (SessionLogStore.java), and the docs cover it under 自定义后端与备份 (session-log.md). So your sharding plan works without touching the framework. Route by user id:

final class ShardedSessionLogStore implements SessionLogStore {
    private final List<SessionLogStore> shards;

    ShardedSessionLogStore(DataSource ds, NamespaceFactory ns, int count) {
        var list = new ArrayList<SessionLogStore>();
        for (int i = 0; i < count; i++) {
            var dialect = AbstractJdbcDialect.from(ds)
                    .storeTableName(String.format("agentscope_store_%03d", i))
                    .build();
            var store = JdbcStore.builder(ds).dialect(dialect).build();
            list.add(new WorkspaceSessionLogStore(new RemoteFilesystem(store, ns)));
        }
        shards = List.copyOf(list);
    }

    private SessionLogStore shard(String userId) {
        int h = Objects.requireNonNull(userId, "userId").hashCode();
        return shards.get(Math.floorMod(h, shards.size()));
    }

    @Override
    public SessionLog open(SessionKey key, RuntimeContext ctx) {
        return shard(key.userId()).open(key, ctx);
    }

    @Override
    public List<SessionKey> list(RuntimeContext ctx) {
        return shard(ctx.getUserId()).list(ctx);
    }
}

String.hashCode() is fixed by the Java spec, so every node routes the same way. Choose the shard count once (16 or 64), since changing it sends users to a table without their sessions. If you ever need to move a session, SessionMigration.fork(source, destination, ...) takes two SessionLogs, so they can come from different shards (SessionMigration.java#L100-L117).

Before writing a router, I'd look at native partitioning on one table. The only unique key is PRIMARY KEY (namespace_path, item_key) (MysqlDialect.java#L65-L86), so MySQL accepts PARTITION BY KEY(namespace_path) PARTITIONS 32. Create the table yourself and set autoCreateTable(false). With tenant and user in the namespace, each user's rows stay in one partition and lookups only touch that partition. A distributed MySQL-compatible database works the same way with that key.

Either way, sharding only spreads the rows out. Total size still comes down to deleting idle sessions, and for very long ones, forking into a new session (it starts from a baseline snapshot, so replay is short again) and deleting the old one.