Add status and index diagnostics

AGENT_GUIDE.md

@@ -131,7 +131,7 @@ CI watching must be bounded. Do not loop indefinitely.
 
 ## Status Contract
 
-- `status --json` and related JSON/MCP payloads currently expose the trust fields documented in `README.md` and `DEVELOPER_GUIDE.md`, including `fold_ready`, `fold_ready_reason`, `graph_table_available`, `issues_table_available`, `file_issues_data_current`, `migration_in_progress`, `sql_graph_contract_ready`, `sql_graph_contract_degraded_reason`, `hotspot_family_ready`, `hotspot_family_degraded_reason`, `csharp_symbol_name_ready`, `csharp_metadata_target_ready`, `csharp_metadata_target_degraded_reason`, `indexed_head_commit`, `worktree_head_changed`, `index_writer_version`, `index_newer_than_reader`, `index_newer_than_reader_reason`, `unknown_extension_file_count`, `path_case_sensitive`, `data_dir`, `data_dir_source`, `data_dir_mode`, `mac_profile`, `db_pragma_settings` (`journal_mode`, `synchronous`, `wal_autocheckpoint`, `page_count`, `freelist_count`, `page_size`), `hooks`, MCP-only `mcp_session`, and the `status --check`-only `stale_after_seconds` / `index_age_seconds` threshold audit fields.
+- `status --json` and related JSON/MCP payloads currently expose the trust fields documented in `README.md` and `DEVELOPER_GUIDE.md`, including `fold_ready`, `fold_ready_reason`, `graph_table_available`, `issues_table_available`, `file_issues_data_current`, `migration_in_progress`, `sql_graph_contract_ready`, `sql_graph_contract_degraded_reason`, `hotspot_family_ready`, `hotspot_family_degraded_reason`, `csharp_symbol_name_ready`, `csharp_metadata_target_ready`, `csharp_metadata_target_degraded_reason`, `indexed_head_commit`, `worktree_head_changed`, `index_writer_version`, `index_newer_than_reader`, `index_newer_than_reader_reason`, `unknown_extension_file_count`, `path_case_sensitive`, `data_dir`, `data_dir_source`, `data_dir_mode`, `mac_profile`, `db_size_bytes`, `wal_size_bytes`, `db_pragma_settings` (`journal_mode`, `synchronous`, `wal_autocheckpoint`, `page_count`, `freelist_count`, `page_size`), `symbols_by_language`, `process`, `last_index_run`, `hooks`, MCP-only `mcp_session`, and the `status --check`-only `stale_after_seconds` / `index_age_seconds` threshold audit fields.
 - When any readiness field is degraded, the CLI adds `degraded_root_cause`, `degraded_reason`, `recommended_action`, `alternative_action`, and `readiness_degradations[]`. `degraded_root_cause` is the primary stable machine code; `readiness_degradations[]` lists every degraded field with `root_cause`, human reason, and remediation strings.
 - `issues_table_available` reports physical `file_issues` table presence only. `file_issues_data_current` reports whether the table is also stamped current for the active index generation.
 - `index_writer_version` records the `cdidx` version that last wrote to the DB (stamped into `codeindex_meta` as `cdidx_writer_version` on every full scan, update, and MCP index). `index_newer_than_reader` flips to `true` whenever any persisted numeric contract stamp in `codeindex_meta` (or unknown `PRAGMA user_version` readiness bits) exceeds the current binary's compiled maximum, so an older CLI re-opening a DB written by a newer CLI degrades loudly with an audit trail instead of silently dropping back to text-search fallbacks. `index_newer_than_reader_reason` enumerates the specific newer-than-reader stamps.
@@ -140,6 +140,7 @@ CI watching must be bounded. Do not loop indefinitely.
 - `status` also surfaces `.cdidx` data-directory permissions via `data_dir_mode` on POSIX filesystems. New `.cdidx` data directories are forced to `0700`; the field is omitted on Windows, URI DBs, or when the directory mode cannot be inspected.
 - `status` also surfaces filesystem case-sensitivity via `path_case_sensitive`, stamped on every successful `cdidx index` run (full scan AND partial update, plus MCP-driven indexes) from `core.ignorecase` + a live filesystem probe. `true` means the volume is case-sensitive (`Foo.cs` and `foo.cs` are distinct); `false` means case-insensitive. Omitted on legacy DBs that predate the stamp. Use it to audit path-equality decisions on case-sensitive APFS, WSL NTFS / dev-drive, and ReFS mounts where the prior OS-keyed heuristic could mis-classify the workspace (#1546).
 - `status` also surfaces Linux mandatory-access-control context via `mac_profile` when `/proc/self/attr/current` or `/proc/self/attr/exec` indicates an AppArmor or SELinux profile. It is omitted on non-Linux hosts, unconstrained processes, or unreadable proc attributes (#1768).
+- `status` also surfaces DB/WAL size, per-language symbol-kind histograms, current process heap/GC/working-set metrics, and the last successful index run metadata. `process` is captured at status-call time; `last_index_run` is persisted at the end of successful CLI and MCP index runs and can include a peak-memory summary when CLI `--memory-trace` was used.
 - MCP `status` also surfaces session diagnostics via `mcp_session`. It is not persisted DB state; it includes the current `log_level`, captured `roots`, optional `client_info`, and optional `client_capabilities`.
 - Keep `README.md`, `DEVELOPER_GUIDE.md`, and this file synchronized if this contract changes.
 

DEVELOPER_GUIDE.md

@@ -317,6 +317,8 @@ Index write batches also stamp `codeindex_meta.batch_in_progress=true` before st
 
 Read-only fallback uses an immutable SQLite URI when the normal writable open cannot create or lock journal/WAL side files, so query commands can still read a DB from read-only or sandboxed storage. That fallback intentionally skips writable pragmas, migrations, and WAL recovery writes; if a WAL is present and must be observed, copy the `.db`, `.db-wal`, and `.db-shm` files together to a writable location or use a SQLite backup from an environment that can open the full WAL set. `status --json` exposes the resolved connection values under `db_pragma_settings` (`journal_mode`, `synchronous`, `wal_autocheckpoint`, `page_count`, `freelist_count`, `page_size`) for automation and support diagnostics. `cdidx vacuum` runs `PRAGMA incremental_vacuum` against writable incremental-auto-vacuum DBs, and performs a one-time `PRAGMA auto_vacuum=INCREMENTAL` plus full `VACUUM` conversion for legacy no-autovacuum DBs.
 
+`status --json` also reports `db_size_bytes`, `wal_size_bytes`, `symbols_by_language`, current `process` heap/GC/working-set metrics, and `last_index_run` metadata from successful CLI and MCP index runs. `index --json --memory-trace` adds a `memory_timeline` block to the CLI index result and persists peak working-set MB into `last_index_run`; `CDIDX_MEM_WARN_MB=<mb>` prints a warning when the sampled working set crosses that threshold.
+
 Writable opens also reject databases whose `PRAGMA user_version` contains readiness bits outside the current binary's `CurrentSchemaVersion` mask. Read-only status/query paths may still surface `index_newer_than_reader=true` as a degraded audit signal, but write-capable paths must fail with `E003_SCHEMA_TOO_NEW` so an older cdidx cannot silently rewrite a DB stamped by a newer one.
 
 ### Data directory resolution
@@ -1983,6 +1985,8 @@ WAL では `NORMAL` により 500 行単位の indexing batch ごとの fsync 
 
 通常の writable open が journal/WAL side file を作成または lock できない場合、read-only fallback は immutable SQLite URI を使うため、query command は read-only / sandboxed storage 上の DB でも読み取りを継続できる。この fallback は意図的に writable pragma、migration、WAL recovery write を skip する。WAL が存在し、その内容を観測する必要がある場合は、`.db` / `.db-wal` / `.db-shm` をまとめて writable location に copy するか、full WAL set を open できる環境で SQLite backup を使う。`status --json` は automation / support diagnostics 用に、解決済みの接続値を `db_pragma_settings` (`journal_mode`, `synchronous`, `wal_autocheckpoint`, `page_count`, `freelist_count`, `page_size`) で公開する。`cdidx vacuum` は incremental-auto-vacuum DB では `PRAGMA incremental_vacuum` を実行し、legacy no-autovacuum DB では初回のみ `PRAGMA auto_vacuum=INCREMENTAL` と full `VACUUM` で変換する。
 
+`status --json` は `db_size_bytes`、`wal_size_bytes`、`symbols_by_language`、現在の `process` heap / GC / working-set metrics、成功した CLI / MCP index 実行由来の `last_index_run` metadata も公開する。`index --json --memory-trace` は CLI index 結果に `memory_timeline` block を追加し、peak working-set MB を `last_index_run` に保存する。`CDIDX_MEM_WARN_MB=<mb>` は sampled working set がしきい値を超えたときに warning を出す。
+
 ## データベーススキーマ
 
 ### テーブル

README.md

@@ -165,16 +165,19 @@ The documented `status --json` trust contract covers these fields:
 <tr><td><code>worktree_head_changed</code></td><td><code>indexed_head_sha</code></td><td><code>indexed_head_branch</code></td><td><code>indexed_head_timestamp</code></td></tr>
 <tr><td><code>commits_ahead_of_indexed_head</code></td><td><code>index_writer_version</code></td><td><code>index_newer_than_reader</code></td><td><code>index_newer_than_reader_reason</code></td></tr>
 <tr><td><code>unknown_extension_file_count</code></td><td><code>path_case_sensitive</code></td><td><code>data_dir</code></td><td><code>data_dir_source</code></td></tr>
-<tr><td><code>data_dir_mode</code></td><td><code>mac_profile</code></td><td><code>db_pragma_settings</code></td><td><code>hooks</code></td></tr>
-<tr><td><code>stale_after_seconds</code></td><td><code>index_age_seconds</code></td><td><code>degraded_reason</code></td><td><code>recommended_action</code></td></tr>
-<tr><td><code>alternative_action</code></td><td><code>mcp_session</code></td><td></td><td></td></tr>
+<tr><td><code>data_dir_mode</code></td><td><code>mac_profile</code></td><td><code>db_size_bytes</code></td><td><code>wal_size_bytes</code></td></tr>
+<tr><td><code>db_pragma_settings</code></td><td><code>symbols_by_language</code></td><td><code>process</code></td><td><code>last_index_run</code></td></tr>
+<tr><td><code>hooks</code></td><td><code>stale_after_seconds</code></td><td><code>index_age_seconds</code></td><td><code>degraded_reason</code></td></tr>
+<tr><td><code>recommended_action</code></td><td><code>alternative_action</code></td><td><code>mcp_session</code></td><td></td></tr>
 </tbody>
 </table>
 
 When any readiness field is degraded, `degraded_root_cause` identifies the primary stable code and `readiness_degradations[]` lists every degraded field with `root_cause`, human `degraded_reason`, `recommended_action`, and `alternative_action`. `issues_table_available` reports physical table presence; use `file_issues_data_current` to decide whether `file_issues` rows are current for the index generation.
 
 For MCP `status`, `mcp_session` is session-scoped diagnostic data rather than persisted index state. It includes `log_level`, `roots`, optional `client_info`, and optional `client_capabilities`.
 
+`process` is captured at status-call time and includes heap, GC collection, and working-set counters. `last_index_run` is persisted by successful CLI and MCP index runs with the run mode, duration, file counts, byte count, row-change counts, and optional peak-memory summary from CLI `--memory-trace`.
+
 `hotspot_family_degraded_reason` uses these values:
 
 | Value | Meaning |
@@ -378,16 +381,19 @@ upgrade / downgrade 後はインストール済み補完 script を再生成し
 <tr><td><code>worktree_head_changed</code></td><td><code>indexed_head_sha</code></td><td><code>indexed_head_branch</code></td><td><code>indexed_head_timestamp</code></td></tr>
 <tr><td><code>commits_ahead_of_indexed_head</code></td><td><code>index_writer_version</code></td><td><code>index_newer_than_reader</code></td><td><code>index_newer_than_reader_reason</code></td></tr>
 <tr><td><code>unknown_extension_file_count</code></td><td><code>path_case_sensitive</code></td><td><code>data_dir</code></td><td><code>data_dir_source</code></td></tr>
-<tr><td><code>data_dir_mode</code></td><td><code>mac_profile</code></td><td><code>db_pragma_settings</code></td><td><code>hooks</code></td></tr>
-<tr><td><code>stale_after_seconds</code></td><td><code>index_age_seconds</code></td><td><code>degraded_reason</code></td><td><code>recommended_action</code></td></tr>
-<tr><td><code>alternative_action</code></td><td><code>mcp_session</code></td><td></td><td></td></tr>
+<tr><td><code>data_dir_mode</code></td><td><code>mac_profile</code></td><td><code>db_size_bytes</code></td><td><code>wal_size_bytes</code></td></tr>
+<tr><td><code>db_pragma_settings</code></td><td><code>symbols_by_language</code></td><td><code>process</code></td><td><code>last_index_run</code></td></tr>
+<tr><td><code>hooks</code></td><td><code>stale_after_seconds</code></td><td><code>index_age_seconds</code></td><td><code>degraded_reason</code></td></tr>
+<tr><td><code>recommended_action</code></td><td><code>alternative_action</code></td><td><code>mcp_session</code></td><td></td></tr>
 </tbody>
 </table>
 
 readiness field のいずれかが degraded の場合、`degraded_root_cause` は primary の安定コードを示し、`readiness_degradations[]` は degraded な各 field と `root_cause`、人間向け `degraded_reason`、`recommended_action`、`alternative_action` を列挙します。`issues_table_available` は物理 table の有無を表し、`file_issues` 行が現在の index generation に対して current かどうかは `file_issues_data_current` を使って判定します。
 
 MCP `status` の `mcp_session` は永続化された index 状態ではなく、セッション単位の診断情報です。`log_level`、`roots`、任意の `client_info`、任意の `client_capabilities` を含みます。
 
+`process` は status 呼び出し時点の heap、GC collection、working-set counters です。`last_index_run` は成功した CLI / MCP index 実行が永続化し、run mode、duration、file counts、byte count、row-change counts、CLI `--memory-trace` 由来の任意の peak-memory summary を含みます。
+
 `hotspot_family_degraded_reason` は次の値を使います。
 
 | 値 | 意味 |

changelog.d/unreleased/1637.added.md

@@ -0,0 +1,24 @@
+---
+category: added
+issues:
+  - 1637
+  - 1680
+  - 1792
+  - 1817
+affected:
+  - src/CodeIndex/Models/QueryResults.cs
+  - src/CodeIndex/Database/DbReader.FilesStatus.cs
+  - src/CodeIndex/Cli/IndexCommandRunner.FullScan.cs
+  - src/CodeIndex/Cli/IndexCommandRunner.Update.cs
+  - README.md
+  - DEVELOPER_GUIDE.md
+  - AGENT_GUIDE.md
+---
+
+## English
+
+- **Expanded status and index diagnostics (#1637, #1680, #1792, #1817)** — `status --json` now reports DB/WAL size, per-language symbol-kind counts, process heap/GC/working-set metrics, and last-index-run metadata, while `index --json --memory-trace` emits memory samples and persists the peak working set.
+
+## 日本語
+
+- **status と index diagnostics を拡張しました (#1637, #1680, #1792, #1817)** — `status --json` は DB/WAL サイズ、言語別 symbol-kind 件数、process heap / GC / working-set metrics、直近 index 実行 metadata を返し、`index --json --memory-trace` は memory samples を出力して peak working set を保存するようになりました。

src/CodeIndex/Cli/CliFlagSchema.cs

@@ -276,6 +276,7 @@ private static IReadOnlyList<CliFlag> BuildAll()
             new() { Name = "--duration-format", ValuePlaceholder = "<auto|seconds|hms>", Description = "Index elapsed time display format", Commands = Set("index") },
             new() { Name = "--max-file-bytes", ValuePlaceholder = "<bytes>", Description = "Override the per-file indexing size limit", Commands = Set("index") },
             new() { Name = "--parallelism", ValuePlaceholder = "<n>", Description = "Full-scan extraction worker count (default: CPU count capped at 16; also honors CDIDX_INDEX_PARALLELISM)", Commands = Set("index") },
+            new() { Name = "--memory-trace", Description = "Include phase memory samples in index JSON output", Commands = Set("index") },
             new() { Name = "--commits", ValuePlaceholder = "<id>", Description = "Update files changed in given git commits", Commands = Set("index") },
             new() { Name = "--changed-between", ValuePlaceholder = "<old-ref> <new-ref>", Description = "Update files changed between two git refs", Commands = Set("index") },
             new() { Name = "--files", ValuePlaceholder = "<path>", Description = "Update only the specified files", Commands = Set("index") },