feat(sensor): add opencode support and complete the platform matrix - #30
Conversation
Brings the open-source Sensor up to date with the agents and operating systems it now needs to cover. All changes are additive: no source key, session-id format or event uuid changes, so existing detection pipelines are unaffected. New source: opencode (github.com/sst/opencode) Reads both storage backends — the current SQLite database (opencode.db, or opencode-<channel>.db on non-stable channels, opened read-only) and the older JSON file tree in both its project-scoped and legacy layouts. Honors $XDG_DATA_HOME and $OPENCODE_DB. Tools are classified against the built-in registry, so anything else with an underscore is recorded as an MCP tool with its server_name split out. Windows support for the parsers that were missing it cursor ~/AppData/Roaming/Cursor/User/globalStorage/state.vscdb cline ~/AppData/Roaming/Cursor/User/globalStorage/saoudrizwan.claude-dev/tasks warp ~/AppData/Local/warp/Warp/data/warp.sqlite desktop ~/AppData/Roaming/Claude/local-agent-mode-sessions Each parser now resolves against an ordered candidate list instead of an either/or pair. Warp also gains the sandboxed macOS group-container path (~/Library/Group Containers/2BBY89MBSN.dev.warp/...), which is where the database actually lives on current builds — it was previously unreachable. Claude Desktop agent mode: Dispatch sessions Delegated background agents are stored one level deeper, under agent/local_ditto_<uuid>/, and were not discovered at all. They now surface under the existing claude_desktop source with a claude_desktop_dispatch_ session-id prefix and an is_dispatch flag, so unattended runs can be scored separately from interactive ones. Interactive session ids are unchanged. Session context also picks up plugins, skills, claude_code_version, the memory/skills/plugins toggles and the available slash commands. Observer: ingest_all() iterates a SOURCES table and resolves each parser as self.<source>_parser, replacing seven near-identical branches. Platform-only sources are declared in PLATFORM_RESTRICTED_SOURCES and the CLI derives its --source choices from SOURCES, so adding an agent touches neither. Also: 44 new tests, a Python 3.9-3.13 CI matrix for the Sensor job (the package already declared >=3.9 but only 3.12 was exercised), OS and Python classifiers in pyproject, and docs for the new source, platform matrix, session_context and environment variables.
pengyuzhang
left a comment
There was a problem hiding this comment.
Strong PR — the opencode parser, the Warp Group Container path, and the CI matrix are all clearly the product of testing against real installs. Verified locally: uv run pytest tests/ -q → 108 passed (up from 64, 38 new tests), uv run ruff check adr_sensor/ clean, and the changes are purely additive with existing paths retained as fallbacks.
Three things before merge — one of which is a community question rather than a code one.
1. This overlaps #25, and #25 is more correct on the line they both touch.
@krishnesh1 opened #25 on Aug 5 (addressing @Rahul-s-007's #21) with the same Cursor/Cline Windows fix. Functionally #30 fully subsumes it — plus Warp, Claude Desktop, opencode, and 38 tests that #25 doesn't have. But #25 resolves the Windows app-data root from the environment, and this PR hardcodes it:
# #25
Path(os.getenv("APPDATA", Path.home() / "AppData/Roaming")) / "Cursor/User/globalStorage/state.vscdb"
# #30
Path.home() / "AppData/Roaming" / _CURSOR_STORAGE_SUFFIXOn roaming profiles and redirected-folder setups — common in exactly the enterprise Windows fleets this parser targets — %APPDATA% is not under ~/AppData/Roaming, so the parser silently finds nothing. That's the same failure mode #21 reports. Same applies to AppData/Local in WarpParser, which should read %LOCALAPPDATA%.
Suggested:
_APPDATA = Path(os.environ.get("APPDATA") or Path.home() / "AppData/Roaming")
_LOCALAPPDATA = Path(os.environ.get("LOCALAPPDATA") or Path.home() / "AppData/Local")The parsers already read XDG_DATA_HOME, XDG_CACHE_HOME, and OPENCODE_DB from the environment, so this is also more consistent with the surrounding code. Please credit #25 when you adopt it — I'd like the external contribution acknowledged rather than quietly superseded.
2. The CI matrix skips 3.10, but pyproject.toml now advertises it.
This PR adds the Programming Language :: Python :: 3.10 and :: 3.13 classifiers; the matrix runs 3.9/3.11/3.12/3.13. With adr-sensor heading to PyPI, we shouldn't claim a version we don't test — either add "3.10" to the matrix or drop the classifier.
3. license/cla is still pending on this PR.
Points 2 and 3 are quick. Point 1 is the one I'd hold on: the current form is a small regression against #25 on the specific bug being fixed, and I'd rather not merge a Windows fix that misses redirected app-data roots.
…t 3.10 Addresses review on #30. %APPDATA% and %LOCALAPPDATA% point outside the user profile on roaming-profile and redirected-folder setups, so the profile-relative AppData paths added for Cursor, Cline, Warp and Claude Desktop would silently find nothing on exactly the managed Windows fleets this sensor targets. The new utils/platform_paths.py consults the environment first and keeps the profile-relative location only as a fallback, matching how the parsers already honor XDG_DATA_HOME / XDG_CACHE_HOME / OPENCODE_DB. Adopts the env-first approach from #25 by @krishnesh1, and extends it beyond Cursor/Cline to Warp (%LOCALAPPDATA%) and Claude Desktop. Closes the failure mode reported in #21 by @Rahul-s-007. Also adds Python 3.10 to the Sensor CI matrix so every version advertised in the package classifiers is exercised, documents APPDATA/LOCALAPPDATA in the Sensor README, and adds six tests covering env-set/unset/empty resolution plus an end-to-end redirected-%APPDATA% discovery test. Co-authored-by: krishnesh1 <krishnesh1@users.noreply.github.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Thanks for the careful review — all three points addressed in 9e52e31. 1. Env-resolved app-data roots, adopting #25. Agreed — the profile-relative paths were a regression against #25 on redirected/roaming profiles, which is the exact failure mode #21 reports. All four Windows-touching parsers now resolve their roots from the environment first, via a shared def windows_appdata() -> Path:
return Path(os.environ.get("APPDATA") or Path.home() / "AppData/Roaming")
def windows_local_appdata() -> Path:
return Path(os.environ.get("LOCALAPPDATA") or Path.home() / "AppData/Local")Cursor, Cline and Claude Desktop use 2. 3.10 added to the Sensor CI matrix — all five advertised versions (3.9–3.13) now run; the 3. CLA — signed; Suite is now 114 passing (6 new for the env resolution), |
Branch protection requires a status context named exactly "Sensor tests" - the job name before it became a per-version matrix. The matrix legs report as "Sensor tests (py3.X)", so the required context was never reported and the PR sat at "Expected - Waiting for status to be reported" with every real check green. Add a gate job with the original name that needs the matrix and fails unless every leg succeeded. Runs under if: always() because a skipped job would otherwise satisfy the required check even when a leg fails. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
pengyuzhang
left a comment
There was a problem hiding this comment.
Approving — the ingestion logic is sound and the verification work behind this is unusually thorough.
Re-verified at bac303d: 114 tests pass locally, ruff check adr_sensor/ tests/test_parsers.py examples/demo.py clean, and the whole matrix (py3.9–3.13) plus the new test-sensor-gate aggregate check is green. I also confirmed the central claim independently: AgentEvent.__post_init__ hashes device/user/timestamp/source/session_id/message-count/tool-count/content-prefix and not session_context, so enriching session_context genuinely cannot move existing uuids. The if: always() guard on the aggregate gate is right — skipped jobs otherwise satisfy required checks.
Merging as-is. Two items from my earlier review are still open; neither is a correctness risk to existing telemetry, so they're fine as follow-ups rather than blockers:
-
Windows app-data roots are captured at import time.
CursorParser.DB_PATHS,ClineParser.BASE_PATHS,WarpParser.DB_PATHSandDEFAULT_BASE_PATHSevaluatewindows_appdata()/Path.home()at module import, so a process that sets%APPDATA%after importingadr_sensorstill gets the profile-relative guess — the #21 failure mode.OpencodeParser._candidate_base_dirs()already does this correctly (resolved per-construction); the four Windows parsers should match it. Worth noting #25 got this part right by keeping resolution in__init__. Fixing it also letstest_redirected_appdata_end_to_enddrop theimportlib.reloadworkaround. -
opencode_parser.py:175interpolates the path into a SQLite URI unencoded.f"file:{db_path}?mode=ro"fails withunable to open database fileif the path contains a%(verified locally); the exception is caught at line 194 and the source silently yields zero events."file:" + pathname2url(str(db_path)) + "?mode=ro"fixes it. Matters more than it looks sinceopencodeisn't inPLATFORM_RESTRICTED_SOURCESand so runs on Windows, where backslash paths go into the URI too.
Smaller follow-ups: test_first_candidate_used_when_none_exist asserts against the developer's real $HOME without isolating it (fails on a Linux box that actually has Cursor installed); PLATFORM_RESTRICTED_SOURCES covers only claude_desktop while the README promises automatic skipping more broadly, and a skipped source continues silently with no explanation to the user; and _classify_tool's "underscore ⇒ MCP" heuristic will attach a fabricated server_name to snake_case non-builtin tools, which is a precision loss in a field detection rules key on.
Also: no Windows runner. The headline feature is Windows support and every Windows assertion is a simulation on ubuntu. One windows-latest leg would exercise real Path.home(), path separators, and the sqlite URI above — I'd trade two Python versions for it.
Thanks for crediting @krishnesh1 as co-author. #25 is fully subsumed by this and should be closed as superseded; it also links no issue, whereas this correctly closes #21.
PR Summary —
feat/sensor-agent-and-platform-supportScope:
Sensor/(+ the Sensor CI job and one line of the root README) · Closes #21 · Windows env-root approach adopted from #25 (@krishnesh1)TL;DR
Three gaps closed, all purely additive:
No source key, session-id format, or event
uuidchanges. Verified by diffing output againstmainon identical input: every pre-existing event is byte-identical.1. New source:
opencodeSensor/adr_sensor/parsers/opencode_parser.py(new, 567 lines)opencode had no parser at all. The new one reads both on-disk backends:
opencode.db, oropencode-<channel>.dbon non-stable channels. Openedfile:...?mode=roso a running opencode process is never disturbed and no-wal/-shmside files are created.storage/directory of per-session/message/part JSON, in both the project-scoped and the legacysession/infolayouts (including messages with embedded parts).~/.local/share/opencodeon Linux and macOS.$XDG_DATA_HOMEand$OPENCODE_DBare honored; a macOSLibrary/Application Supportfallback is probed defensively.tool_type: "mcp_tool"withserver_namesplit out, matching how opencode namespaces MCP tools as<server>_<tool>.step-start,snapshot,patch,retry,compaction, …) are skipped.ses_prefix, so you getopencode_abc123rather thanopencode_ses_abc123.2. Windows support — the platform matrix was incomplete
Closes #21. Four parsers silently returned nothing on Windows because they only probed macOS and Linux locations. Each now resolves against an ordered candidate list rather than an either/or pair:
cursor%APPDATA%Cursor/User/globalStorage/state.vscdbcline%APPDATA%Cursor/User/globalStorage/saoudrizwan.claude-dev/taskswarp%LOCALAPPDATA%warp/Warp/data/warp.sqliteclaude_desktop%APPDATA%Claude/local-agent-mode-sessionsThe Windows roots are resolved from the environment first (
utils/platform_paths.py), with the profile-relative~/AppData/...location only as fallback — on roaming-profile and redirected-folder setups%APPDATA%/%LOCALAPPDATA%live outside the user profile, and a hardcoded guess finds nothing. This adopts the env-first approach from #25 by @krishnesh1 (credited as commit co-author) and extends it to Warp and Claude Desktop, which #25 didn't cover. It also matches how the parsers already honorXDG_DATA_HOME/XDG_CACHE_HOME/OPENCODE_DB.Warp also gains the sandboxed macOS group-container path —
~/Library/Group Containers/2BBY89MBSN.dev.warp/Library/Application Support/dev.warp.Warp-Stable/warp.sqlite. This is where the database actually lives on current Warp builds, so the previous single path meant Warp produced zero events for most macOS users. This shows up in the regression diff below as a genuinely new source.claudeandcodexneeded no change. Both use a single dotfile path (~/.claude/projects,~/.codex/sessions) with no OS-specific branching — that convention already covers Windows, sincePath.home()resolves correctly there viaUSERPROFILE/HOMEDRIVE+HOMEPATH. The platform column for both now readsmacOS, Linux, Windows, same as the four parsers above.3. Claude Desktop agent mode: Dispatch sessions
Dispatch (delegated background agents) stores sessions one level deeper, under
agent/local_ditto_<uuid>/, and the old discovery walk never looked there. They now surface under the existingclaude_desktopsource:claude_desktop_dispatch_(interactive ids unchanged)is_dispatch: trueinsession_contextThat split matters for detection: these are agents running unattended, which is exactly the population you least want a blind spot in, and they should be scored differently from interactive sessions.
session_contextalso now carriesplugins,skillsandclaude_code_versionfrom thesystem:initevent, plussession_type,cli_session_id, the memory/skills/plugins toggles and the available slash commands.4. Observer / CLI cleanup
ingest_all()now iterates aSOURCEStable and resolves each parser asself.<source>_parser, replacing seven near-identicalif source_filter in [...]blocks with one loop. Platform-only agents are declared inPLATFORM_RESTRICTED_SOURCESinstead of inlineplatform.system()checks, and the CLI derives its--sourcechoices fromSOURCES.This is not cosmetic: adding these sources under the old structure meant three more copies of the same eight-line block. It is also what lets
CONTRIBUTING.mdhonestly describe adding a parser as a two-line registration.5. Environment
requires-python = ">=3.9"but only 3.12 was ever exercised.uv buildstill runs once, on 3.12.pyproject.toml: added the 3.10 and 3.13 classifiers (3.10 was supported but unlisted) and OS classifiers for macOS, Linux and Windows.6. Docs
Supported-agent table now carries source keys and an accurate per-agent platform column; new sections for Dispatch and opencode; a documented
session_contextpayload; an Environment section covering supported runtimes and the three environment variables the Sensor reads (XDG_CACHE_HOME,XDG_DATA_HOME,OPENCODE_DB); and a rewritten "Adding a New Parser" guide in both README andCONTRIBUTING.md.examples/demo.pygained opencode and Dispatch sample events.CONTRIBUTING.mdnow states explicitly that source keys are part of the Sensor's public contract and that renaming one is a breaking change to be avoided.Verification
Unit tests — 114 passed (64 pre-existing, 50 new)
No existing test needed modification. New coverage:
$OPENCODE_DB, channel-suffixed DBs, SQLite-over-JSON priority), SQLite parsing with tool calls / MCP classification / error states / reasoning-subtask-file parts / age filtering / truncation, both JSON layouts, tool classification.%APPDATA%/%LOCALAPPDATA%set, unset, and empty-string cases, plus an end-to-end test that places a Cursor DB under a redirected%APPDATA%outside the user profile and asserts discovery — the CursorParser and ClineParser have no Windows path support, silently returning zero telemetry on Windows #21 scenario.Real executions
Against this machine's actual agent logs (
~/.claude/projects,~/.codex/sessions):Against a synthetic home with every agent at its real default path, driven through the installed
adr-sensorCLI (not pytest), posing as macOS:Output payload assertions, all passing: Dispatch prefix present · interactive prefix unchanged · thinking blocks excluded · opencode MCP tool classified with
server_name· opencode built-in not misclassified ·ses_prefix stripped · tool results back-filled onto their calls.Windows-shaped home: all four parsers resolved to their
AppDatalocations and ingested successfully.Other CLI modes:
--save-sessionswrote 6 session files and correctly filtered all 6 as already-seen on a second run;--output-format jsonlproduced 6 valid JSONL lines; opencode's legacy JSON backend, channel-suffixed DB auto-detection and$OPENCODE_DBoverride each verified through the CLI;opencode.dbmd5 unchanged after parsing, with no-wal/-shmside files.Regression check against
mainInstalled
mainin a separate venv and ran both versions against the same synthetic home:The
uuidis a SHA-256 content hash, so identical uuids mean downstream dedup keys are stable.Detection component
Detection/has no coupling to the Sensor — it never importsadr_sensorand never reads its output. (The"claude"string literals there are the Claude CLI binary name insubprocesscalls, andsensor_data_processoris an unrelated IoT benchmark MCP server.) Nothing in this PR can affect it.Static checks
ruff checkpasses clean onadr_sensor/,tests/test_parsers.pyandexamples/demo.py. All files parse under Python 3.9 grammar (ast.parse(..., feature_version=(3, 9))); annotations usetyping.List/Dict/Optionalthroughout.Risks