Skip to content

fix(workflow): isolate LLM provider from shared singleton#316

Merged
duguwanglong merged 1 commit into
devfrom
fix/workflow-llm-provider-isolation
May 25, 2026
Merged

fix(workflow): isolate LLM provider from shared singleton#316
duguwanglong merged 1 commit into
devfrom
fix/workflow-llm-provider-isolation

Conversation

@xiami762
Copy link
Copy Markdown
Contributor

@xiami762 xiami762 commented May 25, 2026 •
edited
Loading

Summary

  • Workflow LLMClient._prepare_provider now clones a workflow-local provider from the shared singleton instead of mutating it under per-provider locks and event-loop markers.
  • Prevents cross-loop httpx.AsyncClient reuse and session/workflow races on _config / _client during llm.ask() calls.
  • Workflow overrides (api_key, base_url, trust_env) apply only to the isolated instance; the shared provider used by session/agent is left untouched.

@cursoragent
fix(workflow): isolate LLM provider from shared singleton
f325984 
Clone a workflow-local provider instead of mutating the shared instance
with locks and event-loop markers, preventing cross-loop client reuse
and session config races during workflow llm.ask() calls.

Co-authored-by: Cursor <cursoragent@cursor.com>
@xiami762 xiami762 requested a review from duguwanglong May 25, 2026 02:02
@duguwanglong duguwanglong merged commit bff62db into dev May 25, 2026
duguwanglong pushed a commit that referenced this pull request May 27, 2026
@stephamie7
v2026.05.27 (#333)
ef68167 
* fix(skill): reduce watcher inotify usage

* fix(workflow): isolate LLM provider from shared singleton (#316)

Clone a workflow-local provider instead of mutating the shared instance
with locks and event-loop markers, preventing cross-loop client reuse
and session config races during workflow llm.ask() calls.

* feat(web2cli,agent): remove agent-browser from web2cli, add planner agent (#315)

* fix(chat): stabilize upload paths and dedupe document attachments

Overwrite duplicate chat uploads instead of auto-renaming so workspace
paths stay consistent. Dedupe composer document attachments by path,
reposition the user avatar in SessionChat, and enable rex_junior delegation.

* feat(agent): consolidate planning into Prometheus subagent

Replace metis and momus with prometheus for interview-style planning and
verified plan output under .flocks/plans/. Route /plan and delegate_task
session permissions through the new agent, preserve YAML permission rules
when resolving tool lists, and show structured todowrite summaries in SessionChat.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(tool): relocate task and skill_load to logical modules

Move task to tool/agent and skill_load to tool/skill, add an enabled flag
to register_function, clarify flocks_skills vs skill_load tool guidance,
limit browser setup to one retry, and update prometheus planning description.

* fix(webui): narrow uploaded document attachment type

Ensure the uploaded document attachment type guard preserves the generic item shape so listUploadedDocumentPaths narrows workspacePath to string during filtering.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(skills): migrate browser workflows to flocks browser (#320)

Align security product skills with browser-use cdp-direct, rename
skyeye-sensor-data-fetch to skyeye-sensor-use, and polish browser doctor,
provider credential delete response, and edit tool mismatch hints

* fix(storage): prevent and recover from SQLite "file is not a database" (#319)

* fix(storage): prevent and recover from SQLite "file is not a database"

Root-cause fixes for the recurring `sqlite3.DatabaseError: file is not a
database` crash that brought down server startup and disabled session
features.

Application-level corruption vectors closed:

* No code path ever ran `PRAGMA wal_checkpoint`. The WAL grew up to the
  default 1000-page (~4 MB) threshold, so every kill -9 / power loss left
  a non-trivial WAL that had to be replayed on the next start - and
  replay rewrites main-DB page 1 (the header). `Storage.shutdown()` now
  runs `wal_checkpoint(TRUNCATE)` at the very end of the FastAPI
  lifespan, and `Storage.init()` does the same on startup to drain any
  residual WAL left by an earlier crash before a second one can land
  during recovery.
* Auto-checkpoint threshold lowered to 200 pages (~800 KB) via
  `PRAGMA wal_autocheckpoint=200`, shrinking the un-persisted window 5x.
* `synchronous=NORMAL` is now set explicitly so the WAL durability
  contract cannot drift to `OFF` via a stray pragma.
* Long-lived SQLite connections in `Storage`, `TaskStore`, and
  `session_binding` now record their owning PID and rebuild after a
  detected `fork()` (uvicorn --reload / multi-worker). Sharing a
  connection across fork is the documented SQLite corruption vector.

Safety net for irreducible external causes (power loss, NFS, AV,
disk-full):

* Pre-flight SQLite magic-header probe before opening so that a corrupt
  file is quarantined *before* `aiosqlite` can delete its `-wal`/`-shm`
  sidecars - the offline `scripts/recover_raw_flocks_db.py` needs them.
* If `Storage.init()` still trips a `NOTADB`/`SQLITE_CORRUPT` /
  "database disk image is malformed" error, the main DB and its sidecars
  are renamed to `<name>.corrupt.<UTC-timestamp>` and a fresh empty DB
  is created so the server can keep booting; a loud log explains how to
  run the recovery script offline.

Tests added/updated:

* corruption quarantine (fast path via magic header, slow path via
  `PRAGMA` failure),
* `_is_db_corruption_error` and `_file_has_invalid_sqlite_header`
  classifiers,
* shutdown TRUNCATE, startup TRUNCATE of residual WAL,
* fork-detection re-init,
* `synchronous=NORMAL` contract assertion on both async and sync paths.

Regression: 346 tests across `tests/storage/`, `tests/server/concurrency/`,
`tests/task/`, `tests/integration/test_task_queue_integration.py`, and
`tests/channel/` all pass.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(storage): surface wal_checkpoint busy + abort init on quarantine failure

Addresses two review findings on the previous commit:

1. **HIGH — `Storage._checkpoint` ignored the PRAGMA result row.** SQLite
   reports contention via the return row ``(busy, log_pages,
   checkpointed_pages)`` rather than a SQL exception, so a TRUNCATE
   blocked by an active reader/writer would return ``(1, n, 0)`` and our
   code would still log ``storage.shutdown.checkpoint.done`` even though
   the WAL was not actually drained — defeating the core goal of "next
   startup must not need WAL recovery".

   `_checkpoint` now fetches that row, returns the full tuple, and
   raises the new ``CheckpointBusyError`` when ``busy=1``.
   ``Storage.shutdown()`` retries TRUNCATE a few times with a short
   backoff, then logs a structured ``checkpoint.unfinished`` warning
   (never ``done``) when the WAL is still occupied — so operators can
   spot the residual-WAL risk in logs.

2. **MEDIUM — fast-path quarantine return value was ignored.** When the
   magic-header pre-flight detects a non-SQLite file, the recovery flow
   depends on quarantining the file *before* SQLite touches its WAL/SHM
   sidecars.  The previous code ignored a possible ``None`` (rename
   failure) and continued to ``_bootstrap_schema``, which would let
   SQLite open the bad file and delete the very sidecars we wanted to
   preserve.

   `Storage.init()` now raises ``StorageError`` when the fast-path
   quarantine fails, so the operator can move the file aside manually
   instead of losing recovery data.

Tests:
* ``test_storage_checkpoint_raises_when_sqlite_reports_busy`` — holds an
  active reader transaction across a TRUNCATE checkpoint and asserts
  ``CheckpointBusyError`` is raised (exposes the original silent
  failure mode).
* ``test_storage_shutdown_reports_unfinished_on_persistent_busy`` —
  spies on the logger to confirm shutdown logs ``unfinished`` (not
  ``done``) when every retry is busy, while still clearing
  ``_initialized`` because the process is exiting anyway.
* ``test_storage_init_raises_when_quarantine_fails_on_invalid_header`` —
  patches the quarantine to return ``None`` and verifies init aborts
  loudly and leaves the WAL/SHM sidecars untouched.

Regression: 349 tests across `tests/storage/`,
`tests/server/concurrency/`, `tests/task/`,
`tests/integration/test_task_queue_integration.py`, and `tests/channel/`
all pass.

* fix(mcp): accept legacy env alias for local server config (#317)

Normalize stdio/local MCP configs so legacy ``env`` maps to the canonical
``environment`` field at load time and when connecting servers.

* feat(compaction): overhaul context compaction with hermes-style pre-pruning and bug fixes

Rewrites the context compaction pipeline to be faster, more accurate, and
available in channel (IM) sessions. Key changes:

Core algorithm (aligning with hermes-agent approach):
- Replace tiktoken with chars/4 estimation; remove system-prompt and
  tool-schema overhead fields from CompactionPolicy
- Fix overflow threshold to a fixed 85% × context_window
- Switch to single-pass LLM summarisation (drop chunked/iterative paths)
- Add hermes-style pre-pruning before summarisation: MD5 dedup of
  identical content, semantic one-line compression of large old messages
  (>200 chars), token-budget tail protection (20% of overflow threshold),
  and stripping of multimodal content with text placeholders
- Add per-message content truncation (head 4000 + tail 1500 chars) before
  feeding messages to the summariser
- Implement error-type-based cooldown for summary failures (60 s for auth
  errors, 30 s for rate-limit, 10 s for transient errors)

Bug fixes:
- Fix overflow detection never using provider-reported token counts:
  last_finished.tokens is a Pydantic TokenUsage model, not a dict, so the
  isinstance(…, dict) guard always fell through to the chars/4 estimate.
  Now normalises TokenUsage → dict via model_dump() before comparison.
- Fix target_chars conversion factor from ×2 to ×4 (chars/4 ↔ tokens)
  in both compaction.py and memory/flush.py
- Return "continue" (skip archive) instead of falling back to a stub
  summary when the summariser is in cooldown, preventing history loss
- Pass original (un-pruned) messages to memory flush to preserve
  full content density for memory extraction

Channel support:
- Expose /compact command in channel surfaces (visible_surfaces + channel_safe)
- Add InboundDispatcher._handle_compact_command: resolves model via
  SessionLoop._resolve_model, runs run_compaction, delivers status reply;
  supports /compact <focus> to bias what the summariser preserves

UI:
- Align the compaction progress indicator with regular assistant message
  bubbles (avatar + "Rex" header row) instead of a bare amber box

Observability:
- Promote loop.tokens_decision from debug to info level so channel-session
  token decisions appear in production logs
- Differentiate "provider temporarily unavailable" from "context genuinely
  too large" in the overflow-exhausted user-facing error message

Tests:
- Remove test_compaction_chunked_strategy.py (chunked path retired)
- Update test_compaction_policy, test_prompt_tokens for new thresholds
- Add overflow_ratio override tests (5 new cases)

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(channel): reliably parse and dispatch slash commands from group-mention messages

WeCom (and other IM platforms) prefix the bot's display name to group
messages before delivering them, so "/compact" arrives as "- test /compact"
rather than starting with "/".  This caused two separate failures:

1. _parse_slash_command used a strict startswith("/") check and returned
   (None, ""), bypassing the slash-command path entirely and letting the
   message fall through to the LLM.

2. Even after adding a regex fallback that detected the command, the
   UserInputEvent was constructed with the raw "- test /compact" text.
   dispatch_user_input then re-parsed event.text with the same strict
   parser (input/dispatcher.py parse_slash_command), got None, and
   called sink.run_llm() — producing the confusing error
   "命令 `- test /compact` 暂不支持在当前渠道中以 slash 形式执行。"

Fix:
- Extend _parse_slash_command with a channel fallback: scan for the last
  "/<word> [args]" token in the text and accept it only when <word>
  resolves in the command registry, preventing false positives on paths
  like "/tmp/foo.log".  Emit a log line (dispatcher.slash_command.
  fallback_matched) when the fallback activates.
- After parsing, normalise event.text to the canonical "/cmd [args]" form
  before constructing UserInputEvent, so the strict re-parse inside
  dispatch_user_input always succeeds.  The original raw text is
  preserved in event.display_text and event.metadata["original_text"]
  for error messages and audit logging.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: remove docs/design directory

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(tool): make read tool output limits configurable via flocks.json

Hard-coded constants in read.py (MAX_LINES, MAX_BYTES, MAX_LINE_LENGTH)
are now overridable through a new `toolOutput` section in flocks.json,
mirroring hermes-agent's tool_output_limits design.

- Add ToolOutputConfig model and ConfigInfo.toolOutput field in config.py
- Add flocks/tool/tool_output_limits.py with sync cache-first config read
- Update read.py to resolve limits at call time via tool_output_limits
- Add toolOutput defaults to .flocks/flocks.json.example

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(device): support host+port providers in connectivity test

The device connectivity probe (POST /devices/{id}/test) previously
required a ``base_url`` field on every device. Providers like Sangfor
SIP store ``host`` + ``port`` instead (e.g. ``192.168.1.100`` + ``7443``)
and never set ``base_url``, so every test attempt returned the misleading
error "未配置设备地址（base_url），请先填写" even when the IP was
correctly filled in.

Backend (server/routes/device.py):
  * When ``base_url`` is empty, fall back to ``host`` + ``port`` from
    the resolved credentials to build ``https://{host}:{port}``.
  * Respect an already-present scheme on ``host`` (e.g. operator typed
    ``http://10.1.2.3``) instead of double-prefixing into
    ``https://http://...``.

Frontend (DeviceIntegration/index.tsx):
  * The probe button now applies the same fallback against the form's
    current values so unsaved edits can be tested before clicking save.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(tool): restore tools when their API service is re-enabled

``ToolRegistry._sync_api_service_states`` used to be a one-way switch:
when an API service was disabled it forced every tool of that provider
to ``enabled=False``, but when the service later became enabled again
the tools stayed off forever. The visible symptom: deleting the last
device of a given provider and then re-adding it left every related
tool greyed out — the only recovery was to toggle each tool by hand.

Changes: * tool/registry.py: make ``_sync_api_service_states`` bi-directional.
    When a service flips to enabled, restore each owned tool to its
    factory default captured at register time (``_enabled_defaults``).
    Already-enabled tools are left untouched to avoid spurious writes.
  * tool/device/sync.py: call ``_apply_tool_settings`` after the sync
    so user-level overrides (``tool_settings[<name>]``) are re-applied
    on top of the restored defaults. This keeps tools the user
    explicitly disabled off, even after the bounce-back path runs.
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(mcp): connect on first enable when server is absent from runtime

``PUT /api/mcp/{name}`` decided whether to reconnect the server based
solely on ``was_connected``. When the user installed a catalog entry
with ``enabled=false`` and then later flipped it to ``enabled=true``
via this endpoint, the runtime status dict had no entry for the server
at all, so ``was_connected`` was False and the reconnect step was
skipped. The result: the server's tools never registered into
``ToolRegistry`` and stayed invisible until the next process restart.

The new ``should_reconnect`` condition reconnects in two cases:
  1. Was already connected (existing behaviour — config change).
  2. Was not present in the runtime status at all AND the new config
     has ``enabled != False`` AND ``get_connect_block_reason`` reports
     no credential/config gap. This covers the first-enable flow
     without surprising operators by auto-connecting servers that the
     handler intentionally left in a non-running state.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(device-startup): skip api-type integrations during _sync_all

``device_startup._sync_all`` swept every storage_key it saw under
``api_services``, including pure-API integrations such as
``tdp_api_v3_3_10`` whose ``_provider.yaml`` declares
``integration_type: api``. Those services never have rows in the
``device_integrations`` table, so ``sync_service_tool_state`` always
counted zero enabled devices and flipped
``api_services[<sk>].enabled = false`` on every restart, silently
disabling the tools. Operators saw the tools come back when they
toggled them manually, only to disappear again on the next restart.

Fix: introduce ``_device_type_storage_keys()`` which scans descriptor
``_provider.yaml`` files once per call and returns the set of
``storage_key`` values whose ``integration_type`` is ``"device"``.
``_sync_all`` consults this set when sweeping the config so pure-API
services are no longer touched by the device subsystem at all.

Notes: * The scan is O(N) over discovered plugins and runs once per startup
    sync; previous draft used O(N²) per-key lookups.
  * Broken or unparseable ``_provider.yaml`` files are tolerated — they
    are simply excluded from the device set rather than aborting the
    whole sync.
Co-authored-by: Cursor <cursoragent@cursor.com>

* test(tool): cover bi-directional sync and api-type exclusion

Locks in the contracts established by the preceding fixes so they
can't silently regress.

tests/tool/test_apply_tool_settings.py (4 new cases):
  * sync_restores_tool_when_service_re_enabled — the headline
    regression: a tool whose YAML default is True bounces back to True
    after its service flips disabled → enabled.
  * sync_does_not_resurrect_user_disabled_tool — a user's explicit
    disable in ``tool_settings`` wins over the bounce-back path.
  * sync_does_not_flip_factory_disabled_tool — tools whose YAML
    default is ``enabled: false`` stay off when the service is enabled;
    only an explicit overlay can open them.
  * sync_leaves_already_enabled_tool_alone — sync is a true no-op when
    tool and service are both already enabled.

tests/tool/test_device_startup_sync.py (new file, 5 cases):
  * TestDeviceTypeStorageKeys covers ``_device_type_storage_keys()``:
    empty plugins dir, device/api separation, YAML parse failures,
    and unknown ``integration_type`` values.
  * TestSyncAllScope.test_skips_pure_api_services_with_no_db_rows
    drives ``_sync_all`` end-to-end with stubbed Storage/ConfigWriter
    and asserts that ``sync_service_tool_state`` is invoked for
    ``integration_type=device`` services and never for
    ``integration_type=api`` services.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(pr321): address three pre-merge issues from code review

R1 — skipped_no_summary no longer masquerades as successful compaction
  process() now returns "skipped" (instead of "continue") for both
  anti-thrashing cooldown and summary-provider cooldown paths.
  session_loop only updates ctx.last_compaction_step and publishes the
  context.compacted event when the result is "continue" (real success);
  "skipped" is logged and the loop continues without touching cooldown state.

R2 — channel fallback slash parser tightened to bot-mention-only prefix
  The regex fallback in dispatcher._parse_slash_command now validates that
  the text before the matched /<command> is a bot-mention prefix of the
  form "- Name" or "@name".  Natural-language sentences such as
  "请解释一下 /help" or "prefix /new thanks" are rejected so they are
  handled by the LLM instead of being dispatched as slash commands.

R3 — ToolOutputConfig fields gain camelCase aliases
  Added alias="readMaxLines" / "readMaxBytes" / "readMaxLineLength" and
  populate_by_name=True so the flocks.json.example entries actually parse
  into the model fields instead of being silently discarded.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(pr321): follow-up cleanup after self-review

R1 follow-ups identified in self-review:
  - Update SessionCompaction.process return type to
    Literal["continue", "stop", "skipped"] to match the new third state.
  - Update run_compaction orchestrator return type and document the three
    states so callers know "skipped" must not be treated as success.
  - Channel /compact handler now delivers a distinct "本轮压缩被跳过" text
    when the result is "skipped" — previously it reported "压缩完成"
    even when nothing was archived.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(pr321): cover the three review fixes with focused unit tests

R1 — anti-thrashing & summary cooldown returns "skipped"
  tests/session/test_compaction_skipped_return.py (4 tests)
    * cooldown_remaining > 0 returns "skipped" (not "continue")
    * total_skipped / total_attempts / cooldown_remaining counters move
    * skip branch must NOT invoke _archive_and_write_summary
    * summary_cooldown_until in the future returns "skipped" and
      archive is not called either

R2 — channel slash fallback bot-mention guard
  tests/channel/test_channel.py::TestParseSlashCommand (12 tests)
    * strict /command  + arg path still works
    * "- BotName /cmd" WeCom prefix accepted
    * "@botName /cmd"  Feishu  prefix accepted
    * Unicode (CJK) bot names accepted
    * 6 negative sentence cases rejected (Chinese, English, /tmp/foo.log,
      bare-word prefixes, multi-word leading text)
    * Unknown command rejected even with valid mention prefix
    * Empty / whitespace-only inputs rejected

R3 — ToolOutputConfig alias acceptance
  tests/config/test_tool_output_config.py (12 tests)
    * camelCase keys populate snake_case fields
    * snake_case keys also accepted (populate_by_name)
    * Partial overrides leave others as None
    * Zero / negative values rejected by gt=0 validator
    * ConfigInfo round-trip from toolOutput / tool_output keys
    * Runtime helpers fall back to defaults without cached config
    * Cached config overrides defaults
    * Sync flocks.json fallback path for CLI one-shot mode
    * Section loader swallows internal errors defensively

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(mcp): reconnect after previous FAILED/DISCONNECTED state too

PR #323 review pointed out that ``should_reconnect`` only fired for the
"first enable with no runtime status" path. Once a server had been
touched in this process — even if the previous attempt ended in
``FAILED`` or ``DISCONNECTED`` — the route would silently skip the
reconnect, forcing users to click Connect by hand after fixing
credentials.

Simplify the condition to: reconnect whenever the new config requests
``enabled`` AND ``get_connect_block_reason`` reports no pending-
credentials issue. ``MCP.remove`` runs unconditionally beforehand
when a previous status existed, so we always start from a clean
runtime slot.

Tests (tests/server/routes/test_mcp_routes.py, +3):
  * connects_on_first_enable_without_prior_status — no runtime entry,
    a local server flips enabled=true → MCP.connect is invoked.
  * reconnects_after_previous_failure — runtime status was FAILED,
    user saved a corrected command → reconnect runs without an extra
    click and the old state is removed first.
  * skips_connect_when_credentials_blank — auth.value="" marks the
    config as pending credentials → connect must NOT run.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(device): cover host+port fallback for connectivity probe

PR #323 review flagged two gaps in the new host+port path:

  1. Error text still only mentioned ``base_url``; users on host+port
     providers (Sangfor SIP) who left both blank were told to fill in
     a field that doesn't exist on their form.
  2. There were no route-level tests for the fallback logic.

Changes: * route_test_device: error message now says "未配置设备地址（base_url
    或 host），请先填写" so the prompt matches the actual provider
    fields.
  * sangfor_sip_v92/_provider.yaml: notes section explains that
    ``host`` defaults to https:// and tells operators how to force
    http:// by typing the scheme into the host field itself.
  * tests/server/routes/test_device_routes.py (new, 6 cases):
      - host+port → https://host:port
      - host only → https://host (no dangling colon)
      - host carries scheme (http://...) → no double prefix
      - body.base_url override beats persisted host
      - empty fields → error message mentions both base_url AND host,
        and the probe never runs
      - unknown device → 404
Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(tool): document the pair-with-apply contract for sync helper

PR #323 review noted that ``_sync_api_service_states`` silently
overwrites ``tool.info.enabled`` with the factory default whenever a
service becomes enabled — any caller that fails to follow up with
``_apply_tool_settings`` would clobber the user overlay (tools the
user explicitly disabled would pop back on).

The two production call sites (plugin bootstrap and device sync)
already pair the calls; add the contract to the docstring so the
next contributor can't accidentally introduce a new call site that
drops the user overlay.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(skills): require device_context lookup before calling device tools

Before invoking any device-specific tool (tdp_*, onesec_*, onesig_*,
qingteng_*, skyeye_*, sangfor_xdr_*), the agent must first call
device_context to list all registered devices, match the user-supplied
device name to the correct device_id, and pass that id on every
subsequent tool call.

Without this step the agent could silently hit the wrong device when
multiple instances of the same product are configured (e.g. "TDP v4"
vs "TDP v6"), or omit device_id entirely when the parameter is marked
optional in the schema.

A dedicated "设备定位（首要步骤，不可跳过）" section has been added at
the top of the API-mode guide in each affected skill, covering:
- mandatory device_context call before the first tool invocation
- name-based matching and device_id extraction
- ask-user-to-confirm when multiple devices match or none match
- sangfor-edr-use (browser-only) adapted to resolve the access URL
  from device_context instead of always prompting for it

Co-authored-by: Cursor <cursoragent@cursor.com>

* perf(webui,session): code-split routes/modals; per-message parts persistence (#322)

* perf(webui): lazy-load modals and heavy routes, memoize sidebar nav

Shrink the initial bundle by code-splitting Layout modals and Session/Agent/auth pages, and stabilize sidebar navigation with useMemo to reduce route-switch re-renders.

* fix(session): persist message parts per message key

Write new sessions to message_parts:<session_id>:<message_id> so tool-call hot paths avoid rewriting the full session blob; keep legacy aggregated blob reads/writes for existing data. Align CLI import and add persistence tests.

* feat(web2cli,browser): improve cookie scoping and payload handling (#324)

* feat(web2cli,browser): improve cookie scoping and payload handling

Align web2cli CLI generation with domain/path-aware Cookie headers,
support json/form/raw payload modes, and harden fetch hook capture.
Filter browser auth-state export by request URL instead of site-root heuristics.

* fix(web2cli,browser): tighten cookie/header edge cases

Preserve empty cookie values, prefer longer-path cookies in header order,
resolve cookie-sourced auth headers without base-url path filtering, and
avoid consuming fetch Request bodies without clone(). Restore legacy browser
hostname helpers for external compatibility.

* docs(readme): increase Docker shared memory to 4gb

Raise the documented --shm-size for browser workloads in Docker run examples.

* feat(web2cli,session): multipart payloads, manual auth headers, tool-loop exit fix (#326)

Extend web2cli spec/CLI generation for multipart uploads, manual HEADER auth,
and time-range params; keep the session loop running when assistant messages
still contain tool parts so results can be fed back to the model.

* refactor(session): use offline chars/4 token estimate (#327)

* refactor(session): use offline chars/4 token estimate

Remove tiktoken cache bootstrap and bundled encoding assets so memory
sync and token counting work without network access or tokenizer files.

* chore(session): drop 4-line concise reply rule from prompts

Remove the hard cap on assistant reply length from general and MiniMax
system prompts so agents can give fuller answers when appropriate.

* fix(session): persist text placeholder on start to preserve part order (#328)

When a tool starts after text streaming begins but before text-end, the
stored part order could become reasoning -> tool -> text. Persist an empty
text part immediately on text-start so completion refetches match stream UI.

* feat(tdp_alert_triage): replace HTTP analysis nodes with unified 5-status attack prompt

Remove the old analyze_payload + analyze_response nodes (each with a
simple, non-standardised prompt) and replace them with a single
attack_analysis_result node that uses the same rigorous 5-category
HTTP attack-state prompt used across the project:

  攻击成功 / 攻击失败 / 攻击 / 未知 / 安全

Also update receive_alert to emit a unified log_text string so the
new node has a single, well-formatted input. The parallel structure
is simplified to three branches:

  receive_alert → [query_threat_intel, query_vuln, attack_analysis_result]
                → join_results → generate_report

No survey / CVE-info LLM nodes are added; tool-based intel lookups
(threatbook) are retained unchanged.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(device): improve tool targeting, discovery, and WebUI source labeling (#329)

* feat(device): improve tool targeting, discovery, and WebUI source labeling

Require explicit device_id when multiple devices share a tool set, auto-load
device_context when enabled devices exist, enrich tool_search with candidate
metadata, and surface device tools separately in catalog/UI grouping.

* refactor(device): refresh device hint on revision and simplify context output

Cache the device asset hint by device_revision, list enabled devices with
vendor in the system prompt, and trim device_context to device list plus
tool-set names/descriptions.

* docs(skills): drop duplicated device targeting sections

Device resolution is now enforced centrally via system prompts, device_context,
and registry validation, so per-vendor skills no longer repeat the same steps.

* Update version to v2026.5.27 in pyproject.toml and uv.lock (#332)
duguwanglong added a commit that referenced this pull request Jun 3, 2026
@stephamie7 @cursoragent
@xiami762 @duguwanglong @chenjie-booker @JohnYin-hub @magicmagicspider
v2026.06.03 (#366)
6dab96a 
* fix(skill): reduce watcher inotify usage

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(workflow): isolate LLM provider from shared singleton (#316)

Clone a workflow-local provider instead of mutating the shared instance
with locks and event-loop markers, preventing cross-loop client reuse
and session config races during workflow llm.ask() calls.

* feat(web2cli,agent): remove agent-browser from web2cli, add planner agent (#315)

* fix(chat): stabilize upload paths and dedupe document attachments

Overwrite duplicate chat uploads instead of auto-renaming so workspace
paths stay consistent. Dedupe composer document attachments by path,
reposition the user avatar in SessionChat, and enable rex_junior delegation.

* feat(agent): consolidate planning into Prometheus subagent

Replace metis and momus with prometheus for interview-style planning and
verified plan output under .flocks/plans/. Route /plan and delegate_task
session permissions through the new agent, preserve YAML permission rules
when resolving tool lists, and show structured todowrite summaries in SessionChat.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(tool): relocate task and skill_load to logical modules

Move task to tool/agent and skill_load to tool/skill, add an enabled flag
to register_function, clarify flocks_skills vs skill_load tool guidance,
limit browser setup to one retry, and update prometheus planning description.

* fix(webui): narrow uploaded document attachment type

Ensure the uploaded document attachment type guard preserves the generic item shape so listUploadedDocumentPaths narrows workspacePath to string during filtering.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(skills): migrate browser workflows to flocks browser (#320)

Align security product skills with browser-use cdp-direct, rename
skyeye-sensor-data-fetch to skyeye-sensor-use, and polish browser doctor,
provider credential delete response, and edit tool mismatch hints

* fix(storage): prevent and recover from SQLite "file is not a database" (#319)

* fix(storage): prevent and recover from SQLite "file is not a database"

Root-cause fixes for the recurring `sqlite3.DatabaseError: file is not a
database` crash that brought down server startup and disabled session
features.

Application-level corruption vectors closed:

* No code path ever ran `PRAGMA wal_checkpoint`. The WAL grew up to the
  default 1000-page (~4 MB) threshold, so every kill -9 / power loss left
  a non-trivial WAL that had to be replayed on the next start - and
  replay rewrites main-DB page 1 (the header). `Storage.shutdown()` now
  runs `wal_checkpoint(TRUNCATE)` at the very end of the FastAPI
  lifespan, and `Storage.init()` does the same on startup to drain any
  residual WAL left by an earlier crash before a second one can land
  during recovery.
* Auto-checkpoint threshold lowered to 200 pages (~800 KB) via
  `PRAGMA wal_autocheckpoint=200`, shrinking the un-persisted window 5x.
* `synchronous=NORMAL` is now set explicitly so the WAL durability
  contract cannot drift to `OFF` via a stray pragma.
* Long-lived SQLite connections in `Storage`, `TaskStore`, and
  `session_binding` now record their owning PID and rebuild after a
  detected `fork()` (uvicorn --reload / multi-worker). Sharing a
  connection across fork is the documented SQLite corruption vector.

Safety net for irreducible external causes (power loss, NFS, AV,
disk-full):

* Pre-flight SQLite magic-header probe before opening so that a corrupt
  file is quarantined *before* `aiosqlite` can delete its `-wal`/`-shm`
  sidecars - the offline `scripts/recover_raw_flocks_db.py` needs them.
* If `Storage.init()` still trips a `NOTADB`/`SQLITE_CORRUPT` /
  "database disk image is malformed" error, the main DB and its sidecars
  are renamed to `<name>.corrupt.<UTC-timestamp>` and a fresh empty DB
  is created so the server can keep booting; a loud log explains how to
  run the recovery script offline.

Tests added/updated:

* corruption quarantine (fast path via magic header, slow path via
  `PRAGMA` failure),
* `_is_db_corruption_error` and `_file_has_invalid_sqlite_header`
  classifiers,
* shutdown TRUNCATE, startup TRUNCATE of residual WAL,
* fork-detection re-init,
* `synchronous=NORMAL` contract assertion on both async and sync paths.

Regression: 346 tests across `tests/storage/`, `tests/server/concurrency/`,
`tests/task/`, `tests/integration/test_task_queue_integration.py`, and
`tests/channel/` all pass.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(storage): surface wal_checkpoint busy + abort init on quarantine failure

Addresses two review findings on the previous commit:

1. **HIGH — `Storage._checkpoint` ignored the PRAGMA result row.** SQLite
   reports contention via the return row ``(busy, log_pages,
   checkpointed_pages)`` rather than a SQL exception, so a TRUNCATE
   blocked by an active reader/writer would return ``(1, n, 0)`` and our
   code would still log ``storage.shutdown.checkpoint.done`` even though
   the WAL was not actually drained — defeating the core goal of "next
   startup must not need WAL recovery".

   `_checkpoint` now fetches that row, returns the full tuple, and
   raises the new ``CheckpointBusyError`` when ``busy=1``.
   ``Storage.shutdown()`` retries TRUNCATE a few times with a short
   backoff, then logs a structured ``checkpoint.unfinished`` warning
   (never ``done``) when the WAL is still occupied — so operators can
   spot the residual-WAL risk in logs.

2. **MEDIUM — fast-path quarantine return value was ignored.** When the
   magic-header pre-flight detects a non-SQLite file, the recovery flow
   depends on quarantining the file *before* SQLite touches its WAL/SHM
   sidecars.  The previous code ignored a possible ``None`` (rename
   failure) and continued to ``_bootstrap_schema``, which would let
   SQLite open the bad file and delete the very sidecars we wanted to
   preserve.

   `Storage.init()` now raises ``StorageError`` when the fast-path
   quarantine fails, so the operator can move the file aside manually
   instead of losing recovery data.

Tests:
* ``test_storage_checkpoint_raises_when_sqlite_reports_busy`` — holds an
  active reader transaction across a TRUNCATE checkpoint and asserts
  ``CheckpointBusyError`` is raised (exposes the original silent
  failure mode).
* ``test_storage_shutdown_reports_unfinished_on_persistent_busy`` —
  spies on the logger to confirm shutdown logs ``unfinished`` (not
  ``done``) when every retry is busy, while still clearing
  ``_initialized`` because the process is exiting anyway.
* ``test_storage_init_raises_when_quarantine_fails_on_invalid_header`` —
  patches the quarantine to return ``None`` and verifies init aborts
  loudly and leaves the WAL/SHM sidecars untouched.

Regression: 349 tests across `tests/storage/`,
`tests/server/concurrency/`, `tests/task/`,
`tests/integration/test_task_queue_integration.py`, and `tests/channel/`
all pass.

* fix(mcp): accept legacy env alias for local server config (#317)

Normalize stdio/local MCP configs so legacy ``env`` maps to the canonical
``environment`` field at load time and when connecting servers.

* feat(compaction): overhaul context compaction with hermes-style pre-pruning and bug fixes

Rewrites the context compaction pipeline to be faster, more accurate, and
available in channel (IM) sessions. Key changes:

Core algorithm (aligning with hermes-agent approach):
- Replace tiktoken with chars/4 estimation; remove system-prompt and
  tool-schema overhead fields from CompactionPolicy
- Fix overflow threshold to a fixed 85% × context_window
- Switch to single-pass LLM summarisation (drop chunked/iterative paths)
- Add hermes-style pre-pruning before summarisation: MD5 dedup of
  identical content, semantic one-line compression of large old messages
  (>200 chars), token-budget tail protection (20% of overflow threshold),
  and stripping of multimodal content with text placeholders
- Add per-message content truncation (head 4000 + tail 1500 chars) before
  feeding messages to the summariser
- Implement error-type-based cooldown for summary failures (60 s for auth
  errors, 30 s for rate-limit, 10 s for transient errors)

Bug fixes:
- Fix overflow detection never using provider-reported token counts:
  last_finished.tokens is a Pydantic TokenUsage model, not a dict, so the
  isinstance(…, dict) guard always fell through to the chars/4 estimate.
  Now normalises TokenUsage → dict via model_dump() before comparison.
- Fix target_chars conversion factor from ×2 to ×4 (chars/4 ↔ tokens)
  in both compaction.py and memory/flush.py
- Return "continue" (skip archive) instead of falling back to a stub
  summary when the summariser is in cooldown, preventing history loss
- Pass original (un-pruned) messages to memory flush to preserve
  full content density for memory extraction

Channel support:
- Expose /compact command in channel surfaces (visible_surfaces + channel_safe)
- Add InboundDispatcher._handle_compact_command: resolves model via
  SessionLoop._resolve_model, runs run_compaction, delivers status reply;
  supports /compact <focus> to bias what the summariser preserves

UI:
- Align the compaction progress indicator with regular assistant message
  bubbles (avatar + "Rex" header row) instead of a bare amber box

Observability:
- Promote loop.tokens_decision from debug to info level so channel-session
  token decisions appear in production logs
- Differentiate "provider temporarily unavailable" from "context genuinely
  too large" in the overflow-exhausted user-facing error message

Tests:
- Remove test_compaction_chunked_strategy.py (chunked path retired)
- Update test_compaction_policy, test_prompt_tokens for new thresholds
- Add overflow_ratio override tests (5 new cases)

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(channel): reliably parse and dispatch slash commands from group-mention messages

WeCom (and other IM platforms) prefix the bot's display name to group
messages before delivering them, so "/compact" arrives as "- test /compact"
rather than starting with "/".  This caused two separate failures:

1. _parse_slash_command used a strict startswith("/") check and returned
   (None, ""), bypassing the slash-command path entirely and letting the
   message fall through to the LLM.

2. Even after adding a regex fallback that detected the command, the
   UserInputEvent was constructed with the raw "- test /compact" text.
   dispatch_user_input then re-parsed event.text with the same strict
   parser (input/dispatcher.py parse_slash_command), got None, and
   called sink.run_llm() — producing the confusing error
   "命令 `- test /compact` 暂不支持在当前渠道中以 slash 形式执行。"

Fix:
- Extend _parse_slash_command with a channel fallback: scan for the last
  "/<word> [args]" token in the text and accept it only when <word>
  resolves in the command registry, preventing false positives on paths
  like "/tmp/foo.log".  Emit a log line (dispatcher.slash_command.
  fallback_matched) when the fallback activates.
- After parsing, normalise event.text to the canonical "/cmd [args]" form
  before constructing UserInputEvent, so the strict re-parse inside
  dispatch_user_input always succeeds.  The original raw text is
  preserved in event.display_text and event.metadata["original_text"]
  for error messages and audit logging.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: remove docs/design directory

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(tool): make read tool output limits configurable via flocks.json

Hard-coded constants in read.py (MAX_LINES, MAX_BYTES, MAX_LINE_LENGTH)
are now overridable through a new `toolOutput` section in flocks.json,
mirroring hermes-agent's tool_output_limits design.

- Add ToolOutputConfig model and ConfigInfo.toolOutput field in config.py
- Add flocks/tool/tool_output_limits.py with sync cache-first config read
- Update read.py to resolve limits at call time via tool_output_limits
- Add toolOutput defaults to .flocks/flocks.json.example

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(device): support host+port providers in connectivity test

The device connectivity probe (POST /devices/{id}/test) previously
required a ``base_url`` field on every device. Providers like Sangfor
SIP store ``host`` + ``port`` instead (e.g. ``192.168.1.100`` + ``7443``)
and never set ``base_url``, so every test attempt returned the misleading
error "未配置设备地址（base_url），请先填写" even when the IP was
correctly filled in.

Backend (server/routes/device.py):
  * When ``base_url`` is empty, fall back to ``host`` + ``port`` from
    the resolved credentials to build ``https://{host}:{port}``.
  * Respect an already-present scheme on ``host`` (e.g. operator typed
    ``http://10.1.2.3``) instead of double-prefixing into
    ``https://http://...``.

Frontend (DeviceIntegration/index.tsx):
  * The probe button now applies the same fallback against the form's
    current values so unsaved edits can be tested before clicking save.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(tool): restore tools when their API service is re-enabled

``ToolRegistry._sync_api_service_states`` used to be a one-way switch:
when an API service was disabled it forced every tool of that provider
to ``enabled=False``, but when the service later became enabled again
the tools stayed off forever. The visible symptom: deleting the last
device of a given provider and then re-adding it left every related
tool greyed out — the only recovery was to toggle each tool by hand.

Changes: * tool/registry.py: make ``_sync_api_service_states`` bi-directional.
    When a service flips to enabled, restore each owned tool to its
    factory default captured at register time (``_enabled_defaults``).
    Already-enabled tools are left untouched to avoid spurious writes.
  * tool/device/sync.py: call ``_apply_tool_settings`` after the sync
    so user-level overrides (``tool_settings[<name>]``) are re-applied
    on top of the restored defaults. This keeps tools the user
    explicitly disabled off, even after the bounce-back path runs.
Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(mcp): connect on first enable when server is absent from runtime

``PUT /api/mcp/{name}`` decided whether to reconnect the server based
solely on ``was_connected``. When the user installed a catalog entry
with ``enabled=false`` and then later flipped it to ``enabled=true``
via this endpoint, the runtime status dict had no entry for the server
at all, so ``was_connected`` was False and the reconnect step was
skipped. The result: the server's tools never registered into
``ToolRegistry`` and stayed invisible until the next process restart.

The new ``should_reconnect`` condition reconnects in two cases:
  1. Was already connected (existing behaviour — config change).
  2. Was not present in the runtime status at all AND the new config
     has ``enabled != False`` AND ``get_connect_block_reason`` reports
     no credential/config gap. This covers the first-enable flow
     without surprising operators by auto-connecting servers that the
     handler intentionally left in a non-running state.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(device-startup): skip api-type integrations during _sync_all

``device_startup._sync_all`` swept every storage_key it saw under
``api_services``, including pure-API integrations such as
``tdp_api_v3_3_10`` whose ``_provider.yaml`` declares
``integration_type: api``. Those services never have rows in the
``device_integrations`` table, so ``sync_service_tool_state`` always
counted zero enabled devices and flipped
``api_services[<sk>].enabled = false`` on every restart, silently
disabling the tools. Operators saw the tools come back when they
toggled them manually, only to disappear again on the next restart.

Fix: introduce ``_device_type_storage_keys()`` which scans descriptor
``_provider.yaml`` files once per call and returns the set of
``storage_key`` values whose ``integration_type`` is ``"device"``.
``_sync_all`` consults this set when sweeping the config so pure-API
services are no longer touched by the device subsystem at all.

Notes: * The scan is O(N) over discovered plugins and runs once per startup
    sync; previous draft used O(N²) per-key lookups.
  * Broken or unparseable ``_provider.yaml`` files are tolerated — they
    are simply excluded from the device set rather than aborting the
    whole sync.
Co-authored-by: Cursor <cursoragent@cursor.com>

* test(tool): cover bi-directional sync and api-type exclusion

Locks in the contracts established by the preceding fixes so they
can't silently regress.

tests/tool/test_apply_tool_settings.py (4 new cases):
  * sync_restores_tool_when_service_re_enabled — the headline
    regression: a tool whose YAML default is True bounces back to True
    after its service flips disabled → enabled.
  * sync_does_not_resurrect_user_disabled_tool — a user's explicit
    disable in ``tool_settings`` wins over the bounce-back path.
  * sync_does_not_flip_factory_disabled_tool — tools whose YAML
    default is ``enabled: false`` stay off when the service is enabled;
    only an explicit overlay can open them.
  * sync_leaves_already_enabled_tool_alone — sync is a true no-op when
    tool and service are both already enabled.

tests/tool/test_device_startup_sync.py (new file, 5 cases):
  * TestDeviceTypeStorageKeys covers ``_device_type_storage_keys()``:
    empty plugins dir, device/api separation, YAML parse failures,
    and unknown ``integration_type`` values.
  * TestSyncAllScope.test_skips_pure_api_services_with_no_db_rows
    drives ``_sync_all`` end-to-end with stubbed Storage/ConfigWriter
    and asserts that ``sync_service_tool_state`` is invoked for
    ``integration_type=device`` services and never for
    ``integration_type=api`` services.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(pr321): address three pre-merge issues from code review

R1 — skipped_no_summary no longer masquerades as successful compaction
  process() now returns "skipped" (instead of "continue") for both
  anti-thrashing cooldown and summary-provider cooldown paths.
  session_loop only updates ctx.last_compaction_step and publishes the
  context.compacted event when the result is "continue" (real success);
  "skipped" is logged and the loop continues without touching cooldown state.

R2 — channel fallback slash parser tightened to bot-mention-only prefix
  The regex fallback in dispatcher._parse_slash_command now validates that
  the text before the matched /<command> is a bot-mention prefix of the
  form "- Name" or "@Name".  Natural-language sentences such as
  "请解释一下 /help" or "prefix /new thanks" are rejected so they are
  handled by the LLM instead of being dispatched as slash commands.

R3 — ToolOutputConfig fields gain camelCase aliases
  Added alias="readMaxLines" / "readMaxBytes" / "readMaxLineLength" and
  populate_by_name=True so the flocks.json.example entries actually parse
  into the model fields instead of being silently discarded.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(pr321): follow-up cleanup after self-review

R1 follow-ups identified in self-review:
  - Update SessionCompaction.process return type to
    Literal["continue", "stop", "skipped"] to match the new third state.
  - Update run_compaction orchestrator return type and document the three
    states so callers know "skipped" must not be treated as success.
  - Channel /compact handler now delivers a distinct "本轮压缩被跳过" text
    when the result is "skipped" — previously it reported "压缩完成"
    even when nothing was archived.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(pr321): cover the three review fixes with focused unit tests

R1 — anti-thrashing & summary cooldown returns "skipped"
  tests/session/test_compaction_skipped_return.py (4 tests)
    * cooldown_remaining > 0 returns "skipped" (not "continue")
    * total_skipped / total_attempts / cooldown_remaining counters move
    * skip branch must NOT invoke _archive_and_write_summary
    * summary_cooldown_until in the future returns "skipped" and
      archive is not called either

R2 — channel slash fallback bot-mention guard
  tests/channel/test_channel.py::TestParseSlashCommand (12 tests)
    * strict /command  + arg path still works
    * "- BotName /cmd" WeCom prefix accepted
    * "@BotName /cmd"  Feishu  prefix accepted
    * Unicode (CJK) bot names accepted
    * 6 negative sentence cases rejected (Chinese, English, /tmp/foo.log,
      bare-word prefixes, multi-word leading text)
    * Unknown command rejected even with valid mention prefix
    * Empty / whitespace-only inputs rejected

R3 — ToolOutputConfig alias acceptance
  tests/config/test_tool_output_config.py (12 tests)
    * camelCase keys populate snake_case fields
    * snake_case keys also accepted (populate_by_name)
    * Partial overrides leave others as None
    * Zero / negative values rejected by gt=0 validator
    * ConfigInfo round-trip from toolOutput / tool_output keys
    * Runtime helpers fall back to defaults without cached config
    * Cached config overrides defaults
    * Sync flocks.json fallback path for CLI one-shot mode
    * Section loader swallows internal errors defensively

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(mcp): reconnect after previous FAILED/DISCONNECTED state too

PR #323 review pointed out that ``should_reconnect`` only fired for the
"first enable with no runtime status" path. Once a server had been
touched in this process — even if the previous attempt ended in
``FAILED`` or ``DISCONNECTED`` — the route would silently skip the
reconnect, forcing users to click Connect by hand after fixing
credentials.

Simplify the condition to: reconnect whenever the new config requests
``enabled`` AND ``get_connect_block_reason`` reports no pending-
credentials issue. ``MCP.remove`` runs unconditionally beforehand
when a previous status existed, so we always start from a clean
runtime slot.

Tests (tests/server/routes/test_mcp_routes.py, +3):
  * connects_on_first_enable_without_prior_status — no runtime entry,
    a local server flips enabled=true → MCP.connect is invoked.
  * reconnects_after_previous_failure — runtime status was FAILED,
    user saved a corrected command → reconnect runs without an extra
    click and the old state is removed first.
  * skips_connect_when_credentials_blank — auth.value="" marks the
    config as pending credentials → connect must NOT run.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(device): cover host+port fallback for connectivity probe

PR #323 review flagged two gaps in the new host+port path:

  1. Error text still only mentioned ``base_url``; users on host+port
     providers (Sangfor SIP) who left both blank were told to fill in
     a field that doesn't exist on their form.
  2. There were no route-level tests for the fallback logic.

Changes: * route_test_device: error message now says "未配置设备地址（base_url
    或 host），请先填写" so the prompt matches the actual provider
    fields.
  * sangfor_sip_v92/_provider.yaml: notes section explains that
    ``host`` defaults to https:// and tells operators how to force
    http:// by typing the scheme into the host field itself.
  * tests/server/routes/test_device_routes.py (new, 6 cases):
      - host+port → https://host:port
      - host only → https://host (no dangling colon)
      - host carries scheme (http://...) → no double prefix
      - body.base_url override beats persisted host
      - empty fields → error message mentions both base_url AND host,
        and the probe never runs
      - unknown device → 404
Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(tool): document the pair-with-apply contract for sync helper

PR #323 review noted that ``_sync_api_service_states`` silently
overwrites ``tool.info.enabled`` with the factory default whenever a
service becomes enabled — any caller that fails to follow up with
``_apply_tool_settings`` would clobber the user overlay (tools the
user explicitly disabled would pop back on).

The two production call sites (plugin bootstrap and device sync)
already pair the calls; add the contract to the docstring so the
next contributor can't accidentally introduce a new call site that
drops the user overlay.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(skills): require device_context lookup before calling device tools

Before invoking any device-specific tool (tdp_*, onesec_*, onesig_*,
qingteng_*, skyeye_*, sangfor_xdr_*), the agent must first call
device_context to list all registered devices, match the user-supplied
device name to the correct device_id, and pass that id on every
subsequent tool call.

Without this step the agent could silently hit the wrong device when
multiple instances of the same product are configured (e.g. "TDP v4"
vs "TDP v6"), or omit device_id entirely when the parameter is marked
optional in the schema.

A dedicated "设备定位（首要步骤，不可跳过）" section has been added at
the top of the API-mode guide in each affected skill, covering:
- mandatory device_context call before the first tool invocation
- name-based matching and device_id extraction
- ask-user-to-confirm when multiple devices match or none match
- sangfor-edr-use (browser-only) adapted to resolve the access URL
  from device_context instead of always prompting for it

Co-authored-by: Cursor <cursoragent@cursor.com>

* perf(webui,session): code-split routes/modals; per-message parts persistence (#322)

* perf(webui): lazy-load modals and heavy routes, memoize sidebar nav

Shrink the initial bundle by code-splitting Layout modals and Session/Agent/auth pages, and stabilize sidebar navigation with useMemo to reduce route-switch re-renders.

* fix(session): persist message parts per message key

Write new sessions to message_parts:<session_id>:<message_id> so tool-call hot paths avoid rewriting the full session blob; keep legacy aggregated blob reads/writes for existing data. Align CLI import and add persistence tests.

* feat(web2cli,browser): improve cookie scoping and payload handling (#324)

* feat(web2cli,browser): improve cookie scoping and payload handling

Align web2cli CLI generation with domain/path-aware Cookie headers,
support json/form/raw payload modes, and harden fetch hook capture.
Filter browser auth-state export by request URL instead of site-root heuristics.

* fix(web2cli,browser): tighten cookie/header edge cases

Preserve empty cookie values, prefer longer-path cookies in header order,
resolve cookie-sourced auth headers without base-url path filtering, and
avoid consuming fetch Request bodies without clone(). Restore legacy browser
hostname helpers for external compatibility.

* docs(readme): increase Docker shared memory to 4gb

Raise the documented --shm-size for browser workloads in Docker run examples.

* feat(web2cli,session): multipart payloads, manual auth headers, tool-loop exit fix (#326)

Extend web2cli spec/CLI generation for multipart uploads, manual HEADER auth,
and time-range params; keep the session loop running when assistant messages
still contain tool parts so results can be fed back to the model.

* refactor(session): use offline chars/4 token estimate (#327)

* refactor(session): use offline chars/4 token estimate

Remove tiktoken cache bootstrap and bundled encoding assets so memory
sync and token counting work without network access or tokenizer files.

* chore(session): drop 4-line concise reply rule from prompts

Remove the hard cap on assistant reply length from general and MiniMax
system prompts so agents can give fuller answers when appropriate.

* fix(session): persist text placeholder on start to preserve part order (#328)

When a tool starts after text streaming begins but before text-end, the
stored part order could become reasoning -> tool -> text. Persist an empty
text part immediately on text-start so completion refetches match stream UI.

* feat(tdp_alert_triage): replace HTTP analysis nodes with unified 5-status attack prompt

Remove the old analyze_payload + analyze_response nodes (each with a
simple, non-standardised prompt) and replace them with a single
attack_analysis_result node that uses the same rigorous 5-category
HTTP attack-state prompt used across the project:

  攻击成功 / 攻击失败 / 攻击 / 未知 / 安全

Also update receive_alert to emit a unified log_text string so the
new node has a single, well-formatted input. The parallel structure
is simplified to three branches:

  receive_alert → [query_threat_intel, query_vuln, attack_analysis_result]
                → join_results → generate_report

No survey / CVE-info LLM nodes are added; tool-based intel lookups
(threatbook) are retained unchanged.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(device): improve tool targeting, discovery, and WebUI source labeling (#329)

* feat(device): improve tool targeting, discovery, and WebUI source labeling

Require explicit device_id when multiple devices share a tool set, auto-load
device_context when enabled devices exist, enrich tool_search with candidate
metadata, and surface device tools separately in catalog/UI grouping.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(device): refresh device hint on revision and simplify context output

Cache the device asset hint by device_revision, list enabled devices with
vendor in the system prompt, and trim device_context to device list plus
tool-set names/descriptions.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(skills): drop duplicated device targeting sections

Device resolution is now enforced centrally via system prompts, device_context,
and registry validation, so per-vendor skills no longer repeat the same steps.

* Update version to v2026.5.27 in pyproject.toml and uv.lock (#332)

* fix(session): reject tool-call title payloads

Prevent malformed title-generation responses from persisting tool-call payloads as session titles by falling back to the user prompt summary.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(agent,webui): device-inspector agent and workflow session navigation (#337)

* feat(agent,webui): add device-inspector and workflow session navigation

Introduce a delegatable device-inspector subagent, surface uv Python hints in
the system env block, fix SessionChat memoization when text streams before tool
parts, and link workflow detail chat to the filtered sessions list.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(agent,webui): subagent delegatable toggle and streaming markdown smoothing

Expose delegatable on agent APIs with subagent defaults, resolve project plugin
YAML for edits, add card-level subagent enable switches, and drain streamed
markdown progressively to avoid post-stall content bursts.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(agent,webui): tighten device-inspector and workflow session linking

Harden device-inspector as read-only, persist delegatable through registry
reloads, compare legacy tool payloads in SessionChat memoization, speed up
streaming markdown catch-up, and resolve workflow session links from stored
history when Chat tab is inactive.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(agent): persist delegatable toggles in sidecar settings

Store built-in subagent enable/disable overrides in ~/.flocks/config instead
of rewriting plugin YAML, add PATCH /api/agent/{name}/delegatable, apply
overrides in registry with cross-process cache invalidation, and wire WebUI
footer toggles to the new endpoint.

* fix(agent): refresh registry after delegatable PATCH

Use Agent.refresh() so delegate_task sees updated is_delegatable immediately
without waiting for a follow-up list call.

* fix(webui): enforce workflow session cap on all localStorage writes (#338)

Centralize session list persistence in setStoredSessions so bulk
overwrites and validation paths respect MAX_STORED_SESSIONS.

* fix(device): per-device tool enable/disable isolation

Root cause: tool_settings in flocks.json was keyed globally by tool name,
so toggling a tool for Device A silently affected Device B when both shared
the same storage_key (same product version, different instance names).

Solution: introduce a dedicated `device_tool_settings` SQLite table that
stores per-device tool overrides independently of the shared global overlay.

Schema:
  PRIMARY KEY (device_id, tool_name)
  FOREIGN KEY device_id REFERENCES device_integrations(id) ON DELETE CASCADE

Changes:
- models.py: register DDL via Storage.register_ddl(); CASCADE handles cleanup
  automatically when a device row is deleted
- store.py: add get/set/delete/list/list_all async CRUD; set and delete bump
  _device_revision() so the session runner's system-prompt cache invalidates
- registry.py: ToolRegistry.execute() checks per-device DB override before
  activating credentials; gate is in try/except so a DB hiccup never blocks
  a tool that is globally enabled
- routes/tool.py: PATCH /api/tools/{name}?device_id= routes per-device
  toggles to the new DB table instead of flocks.json
- routes/device.py: GET /api/devices/{id}/tools and PATCH
  /api/devices/{id}/tools/{name} endpoints; DELETE /api/devices/{id} no
  longer needs manual cleanup (ON DELETE CASCADE)
- prompt.py: build_device_context_section() loads all per-device overrides
  in one SQL query (list_all_device_tool_settings) instead of N+1 calls;
  disabled tools are annotated with device name and device_id so the Agent
  knows not to attempt calling them
- webui: DeviceIntegration page loads per-device enabled state from
  GET /api/devices/{id}/tools and routes toggle to PATCH
  /api/devices/{id}/tools/{name}; wizard mode hides the tool tab entirely
  (no device_id exists yet, so a per-device toggle is impossible)
- tests: 52 passing (18 new); covers CRUD, CASCADE delete, cache-revision
  bumping, batch query correctness, Agent prompt display accuracy

Fixes: toggling a tool for one device no longer affects other devices of
the same product version.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(webui): add agent picker and mentions

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(tool): force-close HTTP tool sockets to prevent CLOSE_WAIT buildup (#345)

Declarative HTTP tools create a fresh ClientSession per call; enable force_close
and conditional enable_cleanup_closed on CPython <3.12.7 so rapid workflow-driven
tool calls don't accumulate half-closed sockets and starve the event loop.

* fix(channel): archive previous session on IM /new command (#344)

Repeated /new left multiple active IM sessions with the same title
prefix, breaking scheduled-task target resolution via session_list.

* fix(logs): bound local log growth (#331)

* fix(logs): bound local log growth

Add size-based rotation and payload truncation so long-running local services cannot grow log files without limit.

* fix(logs): harden local log rotation

Serialize rotating writes and clean up rotated timestamp backups to avoid concurrency failures and stale log files.

* feat(workflow): add Kafka ingest consumer with WebUI integration (#342)

* feat(workflow): add Kafka ingest consumer with WebUI integration

Enable per-workflow Kafka input/output via aiokafka, lifecycle-managed
consumers, runtime status API, and Integration tab controls with connection
error surfacing.

* feat(workflow): add Kafka output-only publishing with outputEnabled toggle

Allow publishing successful execution results without a consumer, gate the producer on outputEnabled, and expose the setting in the Integration tab.

* refactor(webui): remove view sessions shortcut from workflow detail

Drop the redundant right-panel navigation to sessions and related helpers now that session access is handled elsewhere in the workflow UI.

* fix(kafka): reuse output producers and clean up failed starts

Pool per-workflow Kafka producers, tear down workers/producers on connect failures, and document auto-commit semantics. Remove experimental badges from workflow integration UI.

* fix(webui): remove stale workflow detail session sync (#346)

Remove an orphaned session sync effect from WorkflowDetail so the WebUI builds cleanly again after the chat session state was moved out of the page.

* feat(session): add non-blocking prompt queue (#334)

* Squashed commit of the following:

commit bdee8a01224bca53583b1d28f492b9b94a6c6414
Author: chenjie <chenjie@threatbook.cn>
Date:   Thu May 21 22:21:25 2026 +0800

    fix: restore console revocation sync workflow

    Re-enable the legacy sync-revocations API path and wire the upgrade page back to it so revoked licenses propagate again after recent auth-related regressions.

    Co-authored-by: Cursor <cursoragent@cursor.com>

commit 06dee85af690aed75f2c6a1e32c25409f691d2a1
Author: chenjie <chenjie@threatbook.cn>
Date:   Thu May 21 21:28:59 2026 +0800

    chore: trim release bundle workflow config

    Remove obsolete release-bundle workflow lines to keep the pipeline aligned with the current packaging and release process.

    Co-authored-by: Cursor <cursoragent@cursor.com>

commit e9a450a62c90336118c9b2352009c65b0fff7ef7
Author: chenjie <chenjie@threatbook.cn>
Date:   Thu May 21 19:04:27 2026 +0800

    wip: check vulns

commit 54a24ec3c1bbc59ba3ce3067e1499a82f517096f
Author: chenjie <chenjie@threatbook.cn>
Date:   Sat May 16 00:15:46 2026 +0800

    fix: harden console license activation sync

    Use signed activation receipts and safer fallback state handling so Console sync works correctly with installed and uninstalled Pro components.

    Co-authored-by: Cursor <cursoragent@cursor.com>

commit 8eac9c161b25ea3aaaa1033062ee9153c3819f1f
Author: chenjie <chenjie@threatbook.cn>
Date:   Fri May 15 19:59:07 2026 +0800

    feat: gate Flocks Pro features by license capability

    Co-authored-by: Cursor <cursoragent@cursor.com>

commit 42c35c403dcd99dd83880781d844493e3d0f3d3d
Author: chenjie <chenjie@threatbook.cn>
Date:   Fri May 15 19:08:49 2026 +0800

    modify Licence display

commit 42d88987492d4266cbbdf3bedf2a86413a70229d
Author: chenjie <chenjie@threatbook.cn>
Date:   Fri May 15 15:58:59 2026 +0800

    feat: enhance Flocks Pro license lifecycle

    Co-authored-by: Cursor <cursoragent@cursor.com>

commit 523cffea842a86ae2f19a437a6143a763fa83d5c
Author: chenjie <chenjie@threatbook.cn>
Date:   Thu May 14 19:55:38 2026 +0800

    wip: license display

commit 795a7f1abbea518e2dbfd5e4f2047135b64d386e
Author: chenjie <chenjie@threatbook.cn>
Date:   Thu May 14 19:37:26 2026 +0800

    WIP：down+upgrade

commit ee54a88509a48a2538b02ee956e17ce4359e4a4e
Author: chenjie <chenjie@threatbook.cn>
Date:   Thu May 14 18:54:23 2026 +0800

    fix: applicaiton submit

commit 2039cf17b9a4f36490c5057a58632e0430840aee
Author: chenjie <chenjie@threatbook.cn>
Date:   Thu May 14 14:11:01 2026 +0800

    format artifact name

commit 4f524ef7879d16983b1bc3407837a33147711992
Author: chenjie <chenjie@threatbook.cn>
Date:   Thu May 14 13:28:55 2026 +0800

    wip: push flocsk core artifact

commit e622f5ee4599e60995bddaf25b93ecd6956af44d
Author: chenjie <chenjie@threatbook.cn>
Date:   Thu May 14 13:16:26 2026 +0800

    WIP：console push

commit b78e09d5ebe74547188c7f24857dfe3708e6fe18
Author: chenjie <chenjie@threatbook.cn>
Date:   Mon May 11 19:47:19 2026 +0800

    fix Pro component loading at startup

commit 1437faf63b34a9e746623cd0d2f7eca15211cb65
Author: chenjie <chenjie@threatbook.cn>
Date:   Mon May 11 18:48:02 2026 +0800

    feat(console): add cloud account login flow

    Co-authored-by: Cursor <cursoragent@cursor.com>

commit 0f847453fc5455cd944f63ede1054bc98112dce6
Author: chenjie <chenjie@threatbook.cn>
Date:   Mon May 11 11:25:07 2026 +0800

    feat(updater): add console-owned pro bundle upgrade

    Co-authored-by: Cursor <cursoragent@cursor.com>

commit 19e609ecb61a6c09ea5153074c6ddcf49fb46211
Author: chenjie <chenjie@threatbook.cn>
Date:   Sat May 9 21:16:24 2026 +0800

    feat(flockspro): add audit log viewer

    Add Pro audit event capture and a gated audit log UI so admins can review and export account/session audit activity.

    Co-authored-by: Cursor <cursoragent@cursor.com>

commit ec6de7d738ae8bd16baf464497c6d8fa7f6f9265
Author: chenjie <chenjie@threatbook.cn>
Date:   Sat May 9 17:57:30 2026 +0800

    feat(flockspro): add pro extension integration hooks

    Co-authored-by: Cursor <cursoragent@cursor.com>

commit 4b876804bb02aafefcb5f2f6d715f9776f15a634
Author: chenjie <chenjie@threatbook.cn>
Date:   Sat May 9 17:31:33 2026 +0800

    feat(isolation): add workspace/write/session user isolation flow

    Unify default outputs resolution for OSS and Pro username layout, enforce stable filename writes into scoped outputs, and add local shared-session read-only UX/policy with regression tests.

    Co-authored-by: Cursor <cursoragent@cursor.com>

commit a6194ba07dbe00b14ec0d81c9c9471625bc63f11
Author: chenjie <chenjie@threatbook.cn>
Date:   Sat May 9 17:25:14 2026 +0800

    fix(session): enforce owner-only write access across session actions

    Close permission bypasses in init/fork/revert/unrevert/summarize/shell and replace unfiltered session lookup full scans with a constant-time path to avoid performance regressions under larger session sets.

    Co-authored-by: Cursor <cursoragent@cursor.com>

commit b55e02ac4ad3f49a01054eb7b330b743c85982d8
Author: chenjie <chenjie@threatbook.cn>
Date:   Fri May 8 17:43:39 2026 +0800

    fix(cloud): require binding for upgrade requests

    Ensure OSS upgrade requests are tied to an active cloud binding and surface upstream ACT failures as stable API errors.

    Co-authored-by: Cursor <cursoragent@cursor.com>

* removed github push artifacts to console.

* feat: support console pro bundle updates

Co-authored-by: Cursor <cursoragent@cursor.com>

* debug license display

* fix pro bundle update checks

Restore Pro version checks through the Console manifest and route Pro bundle installs through the combined core/component updater.

Co-authored-by: Cursor <cursoragent@cursor.com>

* add linux tar packaging workflow

Trigger a minimal tar packaging workflow on feat/auth_hook2 pushes for branch validation.

Co-authored-by: Cursor <cursoragent@cursor.com>

* minor change

* upload linux tar workflow artifact

Keep branch packaging output downloadable from the workflow run artifacts.

Co-authored-by: Cursor <cursoragent@cursor.com>

* modify tar name

* fix pro bundle update version checks

Use the bundle compare version for Console manifest updates so patch releases are detected independently of component versions.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix pro bundle version display

Prefer the installed bundle version for Pro UI labels and keep license cards focused on license state rather than release versions.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: refine pro bundle upgrade handling

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: remove trial license upgrade paths

Co-authored-by: Cursor <cursoragent@cursor.com>

* debug pro-download

* add process bar for downloading

* set default portal env vars for service startup

Ensure flocks start/restart injects portal WebUI defaults when users do not provide them, while keeping explicit environment overrides intact.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: set default console base url for backend startup

Ensure flocks start/restart injects FLOCKS_CONSOLE_BASE_URL into backend service environment by default, while preserving explicit env overrides.

Co-authored-by: Cursor <cursoragent@cursor.com>

* pro application form includes sales name

* fix license quota sync for account management

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix license sync timestamp display

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix pro branding and upgrade permissions

Co-authored-by: Cursor <cursoragent@cursor.com>

* update default console portal host

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix http middleware hook registration

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(workflow): add generic background poller with API and WebUI (#347)

* feat(workflow): add generic background poller with API and WebUI

Introduce WorkflowPollerManager for scheduled workflow runs, REST
endpoints for config/status/run-once, server startup integration, and
Integration tab controls with tests.

* fix(workflow): persist poller runs and improve stop lifecycle

Record poller executions via execution_store, normalize business-failure
outcomes, and let in-flight runs finish during stop instead of cancelling.

* fix(webui): dedupe uploaded document attachments by path

Export helpers to keep only the latest successful non-image upload per
workspace path when building attachment blocks and after batch upload.

* Fix/provider base url required (#348)

* fix(model): require base URL for openai-compatible providers

Align the add-provider UI with backend validation by making Base URL required for openai-compatible provider creation, and surface localized provider descriptions in the catalog flow.

* fix(model): remove unintended provider description changes

Keep this branch focused on the requested Base URL validation fix by reverting the extra provider description code path changes that were added by mistake.

* feat(device): add 360 WAF v5.5 adapter (#343)

* feat(device): add 360 WAF v5.5 integration

* refactor(device): use built-in confirmations for 360 WAF

* feat(provider): add MiniMax M3 catalog support (#351)

* feat(provider): add MiniMax M3 catalog support

Add MiniMax M3 to the built-in provider catalog and apply the same MiniMax runtime handling to any minimax model ID so newer variants work without extra per-model logic.

* fix(provider): align MiniMax M3 catalog tests

Keep the MiniMax catalog test expectations in sync with the configured 512k context and output limits so the provider suite reflects the intended catalog values.

* fix(session): clear history and surface Feishu websocket disconnects (#350)

* fix(session): clear history and surface Feishu websocket disconnects

Make /clear remove stored session messages across CLI and WebUI so the UI stays in sync. Fail fast when Feishu websocket workers disconnect so supervisors can restart them instead of hanging silently.

* fix(channel): keep Feishu websocket siblings running

Observe disconnects in both Feishu websocket client paths without treating a single account failure as a global stop. Clear queued prompts before deleting session history so /clear leaves the session fully reset.

* fix(channel): restart failed Feishu websocket accounts

Retry failed Feishu websocket accounts with backoff so one dropped connection can recover without interrupting healthy siblings. Cover the restart path with regression tests.

* fix: eliminate OOM in execution history trim under syslog load

Under sustained syslog throughput, _trim_execution_history was called as
a fire-and-forget background task every 5 messages per workflow.  Each
invocation called Storage.list_entries("workflow_execution/") which
json.loads every execution record in the table into Python objects.
Execution records included full alert payloads (resp_body, req_header,
etc.) written by the dedup_and_write node, making each record several
hundred KB.  As the table grew to 3+ GB, a single trim scan materialised
gigabytes of transient objects — py-spy confirmed 100% GIL time in
json.raw_decode attributed to _trim_execution_history → list_entries.
Because multiple trim tasks could be in flight simultaneously (one spawned
per 5 messages, each taking tens of seconds to complete), the transient
allocations multiplied, driving RSS to 20 GB without bound.

Fix:
- Storage.list_raw(): new method that returns (key, raw_value_str) pairs
  without any Python-side JSON parsing, compatible with all SQLite versions.
- _trim_execution_history: replaced list_entries + json.loads with
  list_raw + regex extraction of workflowId/startedAt from the first 400
  bytes of each value string.  Avoids constructing large Python objects
  entirely; regex on a 400-byte prefix is ~100x cheaper than full parse.
- _trim_in_flight: Set[str] guard ensures at most one trim task per
  workflow runs at a time, preventing concurrent scans from multiplying
  peak memory usage.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(kafka): tighten ingest backpressure and compact execution storage (#353)

* fix(kafka): tighten ingest backpressure and compact execution storage

Reduce concurrent Kafka workflow runs and fetch buffering to limit memory
on large payloads. Summarize raw inputs and execution history for storage,
and remove experimental Kafka output producer from API and WebUI.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(workflow): add summary history mode for high-throughput ingest

Summarize step inputs/outputs in-engine for Kafka runs instead of retaining
full payloads in memory. Skip dedup hashing in summary mode and propagate
final workflow outputs on ExecutionResult.

* fix(workflow): summarize final outputs in summary history mode

Keep RunWorkflowResult.outputs bounded for Kafka and other high-throughput
runs by applying the same observability summarization used for step history.

* fix(workflow): clear REPL globals after each node in summary mode

Prevent large node-local variables from accumulating across steps when
Kafka and other high-throughput runs use history_mode=summary.

* update pro upgrade form fields and validation

Align the OSS Pro upgrade application dialog with the new business requirements by removing sales-rep collection, requiring applicant contact info, and validating email/phone formats consistently in frontend and backend.

Co-authored-by: Cursor <cursoragent@cursor.com>

* remove eslint disable comment for exhaustive-deps in AuditLogsPage useEffect

* fix: secure pro upgrade status reporting

* fix(session): normalize legacy stored messages and parts on cache load (#352)

Backfill missing assistant/user fields and tool state timestamps so old
sessions deserialize without dropping the whole cache; skip invalid entries.

* fix(channel): support /clear slash command in channel sessions (#356)

Allow channel surfaces to accept /clear and execute the existing session history reset flow so IM conversations can clear state consistently with WebUI.

* docs(readme): update remote access and proxy guidance (#359)

Align EN/ZH README with WebUI same-origin /api proxy defaults, simplify
remote start flags, and expand reverse-proxy and auth recovery notes.

* fix(session): keep mention agent routing in queued sends

Ensure @mention-selected agents are preserved when messages are queued during streaming, and align agent picker copy with the default-Rex new-session behavior.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(provider): add minimax-m3 and update model limits in catalog

- Add minimax-m3 (1M context, 128K output) to threatbook-cn-llm,
  threatbook-io-llm, and minimax providers
- Update deepseek-v4-flash context window 200K→1M, max output 128K→384K
  in both threatbook-cn-llm and threatbook-io-llm
- Reorder ThreatBook provider models: minimax group (m3/m2.7/m2.5) first
- Fix minimax provider: align minimax-m3 family field and add pricing

Co-authored-by: Cursor <cursoragent@cursor.com>

* Feat/device huorong hwwaf (#362)

* feat(device): add Huorong EDR and Huawei Cloud WAF device plugins

- Add huorong_edr_v1_0: HMAC-SHA1 signed API integration for Huorong
  endpoint security platform, covering group management (group_list/
  create/rename/delete), client management (online/list/info/rename/
  group/leak), and task management (virus scan task creation)

- Add huaweicloud_waf_v1: AK/SK (SDK-HMAC-SHA256) and Token dual-auth
  integration for Huawei Cloud WAF, covering protected domain management
  (cloud mode + dedicated mode), policy and rule management (CC rules,
  custom rules, blacklist/whitelist, geo-IP), attack event queries, and
  security overview statistics

Both plugins follow the standard device plugin layout with _provider.yaml,
_test.yaml, handler.py, and grouped tool YAML files.

* refactor(device): rename huaweicloud_waf to v39 and bump version

Rename huaweicloud_waf_v1 → huaweicloud_waf_v39 to match the official
WAF API reference document version (v39, 2026-04-08), and update
version/product_version fields in _provider.yaml from "1.0" to "39".

* feat(workflow): merge Kafka configured inputs with consumed messages (#361)

* feat(workflow): merge Kafka configured inputs with consumed messages

Persist extra workflow inputs on Kafka consumer config and apply them
when triggering runs, with WebUI JSON editing aligned to the poller UX.

* fix(workflow): strip _comment keys from Kafka configured inputs

Strip execution-only comment fields when saving Kafka inputs in the
WebUI and when persisting or applying configured inputs at runtime.

* fix(workflow): prefer processed cache size in poller status

Surface processed_cache_size_after in poller run summaries when present
and rename the WebUI label to reflect total processed count.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: update project version to v2026.6.3 in pyproject.toml and uv.lock (#365)

---------

Co-authored-by: John Yin <10972267+john-yin2333@user.noreply.gitee.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: xiami <gongyanzh97@gmail.com>
Co-authored-by: xiami762 <>
Co-authored-by: duguwanglong <duguwanglong@163.com>
Co-authored-by: chenjie <chenjie@threatbook.cn>
Co-authored-by: JohnYin <mryin1104@163.com>
Co-authored-by: 香蕉味魔法蜘蛛 <50224719+magicmagicspider@users.noreply.github.com>
Co-authored-by: shangguanhongxin <48522006+duguwanglong@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@duguwanglong duguwanglong duguwanglong approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@xiami762 @duguwanglong