diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index 29f2f390a..d528d1378 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -875,6 +875,7 @@ Current stable codes and triggers: | Writable open pragmas | `DbContext` opens writable indexes in WAL mode, applies connection performance pragmas through `DbPragmaPolicy`, sets `PRAGMA auto_vacuum=INCREMENTAL` before schema creation for new empty databases, sets `PRAGMA application_id=0x43444958` (`CDIX`), sets `PRAGMA synchronous=NORMAL`, and pins `PRAGMA wal_autocheckpoint=1000`. | | Open intent | Every `DbContext` caller declares `QueryOnly`, `WriteIndex`, `Migration`, or `Repair`. `QueryOnly` uses unpooled connections and skips persistent pragmas, migrations, metadata writes, and repair work. WAL databases are read from private temporary snapshots: checkpointed databases copy the main file and open that copy through an immutable read-only URI, while non-empty WAL databases copy the stable main/WAL pair so committed WAL content remains visible. Before and after copying, database-header and WAL-generation fingerprints must match; otherwise the bounded copy retries and ultimately refuses the open instead of returning stale data or modifying source `-wal` / `-shm`. Attached cross-database snapshots are cleaned up only after the attaching connection closes. Missing source/WAL files may retry as generation churn, while persistent copy I/O failures report `query_only_snapshot_copy_failed` with temporary-storage remediation. Long-lived MCP and LSP sessions refresh a detached snapshot when its source generation changes. `RepairIncompleteBatchReadiness` is guarded by `Repair` intent. | | Application id | The application id lets file-type detection tools distinguish cdidx databases from generic SQLite databases. | +| Maintenance error contract | `vacuum`, `backfill-fold`, `optimize` / `index --optimize`, and `db integrity` route failures through `MaintenanceDatabaseErrorClassifier` version `1` and one JSON/human writer. SQLite primary codes `5`/`6`, `8`, `11`, and `26` classify locked/busy, not-writable, corrupt, and not-a-database failures without inspecting exception wording. The shared response carries a stable error code/category, conditional recovery hint, redacted path metadata, and optional primary/extended SQLite codes. Absolute paths are redacted by default; `--show-paths` is the explicit diagnostic opt-in. | | Durable WAL file set | When WAL is active, the durable SQLite index is the `.db` file plus sibling `.db-wal` and `.db-shm` files. Backups, diagnostics bundles, and manual copies must include all three files when the siblings exist, or use SQLite's `.backup` command/API from a live connection. Copying only `codeindex.db` can produce a stale snapshot because committed pages may still live in `codeindex.db-wal`. | | `synchronous=NORMAL` | Under WAL, `NORMAL` avoids per-commit fsync pressure during 500-row indexing batches while preserving database consistency after crashes. | | Checkpointing | `DbWriter` runs `PRAGMA wal_checkpoint(PASSIVE)` after each outer transaction commit, and SQLite may also checkpoint automatically after the configured 1000-page threshold. Both checkpoint paths are opportunistic: active readers are not blocked, and an uncheckpointed WAL is expected state rather than corruption. | @@ -4124,6 +4125,7 @@ apply 時は `PRAGMA optimize` を実行します。 | writable open pragma | `DbContext` は writable な index を WAL mode で開き、connection performance pragma を `DbPragmaPolicy` 経由で適用し、新規の空 DB では schema 作成前に `PRAGMA auto_vacuum=INCREMENTAL` を設定し、`PRAGMA application_id=0x43444958` (`CDIX`)、`PRAGMA synchronous=NORMAL`、`PRAGMA wal_autocheckpoint=1000` を固定します。 | | open intent | すべての `DbContext` caller は `QueryOnly`、`WriteIndex`、`Migration`、`Repair` のいずれかを宣言します。`QueryOnly` は unpooled connection を使い、永続的な pragma、migration、metadata write、repair を skip します。WAL database は private temporary snapshot から読みます。checkpoint 済み database は main file を copy してその copy を immutable read-only URI で開き、non-empty WAL database は安定した main/WAL pair を copy して committed WAL content を維持します。copy 前後の database header / WAL generation fingerprint が一致しなければ bounded copy を retry し、最終的には stale data を返したり source `-wal` / `-shm` を変更したりせず open を拒否します。cross-database で attach した snapshot は、attach 元 connection を閉じた後に cleanup します。source/WAL file の消失は generation churn として retry できますが、永続的な copy I/O failure は temporary storage の remediation を含む `query_only_snapshot_copy_failed` を返します。長時間動作する MCP / LSP session は source generation が変わると detached snapshot を refresh します。`RepairIncompleteBatchReadiness` は `Repair` intent の場合に限り実行できます。 | | application id | application id は file-type detection tool が cdidx database と generic SQLite database を区別するための印です。 | +| maintenance error contract | `vacuum`、`backfill-fold`、`optimize` / `index --optimize`、`db integrity` の失敗は `MaintenanceDatabaseErrorClassifier` version `1` と単一の JSON / human writer を通ります。SQLite primary code `5` / `6`、`8`、`11`、`26` から locked / busy、not-writable、corrupt、not-a-database を分類し、例外 message は判定に使いません。共有 response は stable error code / category、条件別 recovery hint、redaction 済み path metadata、任意の primary / extended SQLite code を返します。absolute path は既定で redaction し、`--show-paths` を明示的な diagnostic opt-in とします。 | | durable WAL file set | WAL が有効な場合、永続化された SQLite index は `.db` file と sibling の `.db-wal` / `.db-shm` file の組です。backup、diagnostics bundle、手動 copy では sibling が存在する場合に 3 file すべてを含めるか、live connection から SQLite の `.backup` command/API を使う必要があります。`codeindex.db` だけを copy すると、committed page がまだ `codeindex.db-wal` に残っているため stale snapshot になる可能性があります。 | | `synchronous=NORMAL` | WAL では `NORMAL` により 500 row 単位の indexing batch ごとの fsync 負荷を避けつつ、crash 後の database consistency を保ちます。 | | checkpoint | `DbWriter` は outer transaction commit 後に `PRAGMA wal_checkpoint(PASSIVE)` を実行し、SQLite も設定済みの 1000 page threshold を超えると自動 checkpoint する場合があります。どちらの checkpoint path も opportunistic で、active reader は block されず、未 checkpoint の WAL は corruption ではなく期待される状態です。 | diff --git a/TESTING_GUIDE.md b/TESTING_GUIDE.md index 2841accfb..cd1c1c701 100644 --- a/TESTING_GUIDE.md +++ b/TESTING_GUIDE.md @@ -350,6 +350,8 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding - `IndexCommandRunnerTests.RunOptimizeFts_DryRunPreviewsWithoutWritingThenOptimizeMutates_Issue4577`, `RunOptimizeFts_LockHeld_ReportsDbLocked`, and `RunOptimizeFts_ReadOnlyUri_ReturnsDbNotWritable` share seeded databases to verify that optimize dry-run reports sizes, readiness, lock state, recommendations, and planned work while preserving the source bytes and creating no lock artifacts; the same fixtures retain the writable optimize and lock/read-only URI mutation guards. `ConsoleUiTests.PrintUsage_WithoutBanner_HidesAsciiArtAndEasterEggFlags` keeps the optimize dry-run flag visible in full CLI usage. +- Issue #4856 maintenance-error coverage spans `CommandErrorWriterTests`, `QueryCommandRunnerTests`, `IndexCommandRunnerTests`, and `DbCommandRunnerTests`. + Keep the SQLite primary-code matrix (`5`, `6`, `8`, `11`, `26`), caller-spelled relative paths, whole-value Unix/Windows/file-URI redaction, missing and invalid-header files, directory/inaccessible preflight classification, integrity corruption rows, lock-holder metadata, the `index --optimize` human preamble, valid JSON, and category-specific recovery hints aligned with classifier version `1`. - `IndexCommandRunnerTests.Run_DryRunWithRebuildAndMemoryTrace_SkipsConfirmationAndPreservesWorkspace_Issue4580` runs a rebuild preview as redirected input without `--yes`, verifies live memory phases and samples, and byte-compares the complete project file set before and after so source, DB, WAL, SHM, and other workspace artifacts cannot be changed silently. - `IndexCommandRunnerTests.Run_MemoryTrace_ReportsFullScanAndUpdatePhaseBoundaries` @@ -1258,6 +1260,8 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests" - `IndexCommandRunnerTests.RunOptimizeFts_DryRunPreviewsWithoutWritingThenOptimizeMutates_Issue4577`、`RunOptimizeFts_LockHeld_ReportsDbLocked`、`RunOptimizeFts_ReadOnlyUri_ReturnsDbNotWritable` seed 済み database を共有し、optimize dry-run が size、readiness、lock state、推奨、planned work を報告しつつ source byte を保持し、lock artifact を作成しないことを検証する。同じ fixture で、書き込み版 optimize と lock/read-only URI の mutation guard も維持する。 `ConsoleUiTests.PrintUsage_WithoutBanner_HidesAsciiArtAndEasterEggFlags` は、full CLI usage に optimize dry-run flag が表示され続けることを固定します。 +- Issue #4856 の maintenance-error coverage は `CommandErrorWriterTests`、`QueryCommandRunnerTests`、`IndexCommandRunnerTests`、`DbCommandRunnerTests` にまたがります。 + SQLite primary-code matrix (`5`、`6`、`8`、`11`、`26`)、呼び出し側表記を維持する relative path、Unix / Windows / file URI の値全体 redaction、missing / invalid-header file、directory / inaccessible preflight classification、integrity corruption row、lock-holder metadata、`index --optimize` の human preamble、有効な JSON、category 別 recovery hint を classifier version `1` と同期させてください。 - `IndexCommandRunnerTests.Run_DryRunWithRebuildAndMemoryTrace_SkipsConfirmationAndPreservesWorkspace_Issue4580` redirect input で `--yes` なしの rebuild preview を実行し、live memory phase/sample を検証します。実行前後の project file set 全体を byte 比較し、source、DB、WAL、SHM、その他 workspace artifact が暗黙に変更されないことも固定します。 - `IndexCommandRunnerTests.Run_MemoryTrace_ReportsFullScanAndUpdatePhaseBoundaries` diff --git a/USER_GUIDE.md b/USER_GUIDE.md index 829ada0bb..df47f47d5 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -1192,13 +1192,16 @@ cdidx db integrity # run PRAGMA integrity_c cdidx db --integrity-check # same legacy spelling cdidx db integrity --db ./.cdidx/codeindex.db # point at a specific DB cdidx db integrity --json # machine-readable result +cdidx db integrity --json --show-paths # explicitly retain the DB path in an error diagnostic cdidx db schema --summary-only --json # counts without full SQL payloads cdidx db schema --type table --name files --json # exact schema object projection cdidx db schema --limit 20 --max-sql-chars 4000 --exclude-internal --json cdidx db checkpoint before-prune --dry-run --json # preview snapshot files and bytes ``` -This opens the database read-only, runs SQLite's `PRAGMA integrity_check`, and prints whether the file is `ok` or lists the failures. Exit codes are stable for scripting: `0` clean, `2` (NotFound) when the file does not exist, `3` (DatabaseError) when corruption is detected. SQLite does not offer a general-purpose repair primitive — if the check fails, recover by rebuilding with `cdidx index --rebuild`. +This opens the database read-only, runs SQLite's `PRAGMA integrity_check`, and prints whether the file is `ok` or lists the failures. Exit codes are stable for scripting: `0` clean, `2` (NotFound) when the file does not exist, `3` (DatabaseError) when corruption or an invalid database is detected, and the transient-database exit code for lock/busy contention. SQLite does not offer a general-purpose repair primitive — if the check fails, recover by rebuilding with `cdidx index --rebuild`. + +Failures from `vacuum`, `backfill-fold`, `optimize` (including `index --optimize`), and `db integrity` share database-error classifier version `1`. The classifier uses SQLite primary result codes for locked/busy, read-only, corrupt, and not-a-database failures rather than exception wording, and emits stable `error_code`, `category`, `hint`, `path`, `path_redacted`, and optional SQLite result-code fields in JSON. Absolute database paths and file URIs are replaced wholesale with `` by default, while relative paths retain the caller's spelling; pass `--show-paths` only when an explicit diagnostic needs the full path. Directory or inaccessible preflight targets use `E008_DB_ERROR` / `database_inaccessible` with regular-file and permission guidance instead of being guessed as missing or invalid databases. Optimize lock failures include bounded holder PID/start-time details when lock metadata is available. A successful `db integrity --json` response keeps the existing `integrity_ok` shape, while failures use this shared error response. Recovery hints are category-specific, so corruption does not recommend lock retries and lock contention does not recommend rebuilding a healthy database. `db schema` keeps the current full schema dump by default for support bundles. Add `--summary-only` to return only object counts, combine `--type ` and `--name ` for an exact projection, and use `--limit`, `--max-sql-chars`, and `--exclude-internal` to keep schema diagnostics bounded. @@ -2100,6 +2103,7 @@ For scripts and AI agents that need to classify failures without substring-match | `E024_CONFIG_INVALID` | `validate-config` discovered a configuration file that failed validation | | `E025_HOOK_OPERATION_FAILED` | A Git hook operation failed at a platform or filesystem boundary | | `E026_NOT_GIT_REPOSITORY` | `hooks` was run outside a Git worktree and no valid `--project` was supplied | +| `E027_DB_NOT_DATABASE` | SQLite rejected the target as not being a database, or maintenance validation rejected it as not being a CodeIndex database | ### Debugging reader errors @@ -4396,13 +4400,16 @@ cdidx db integrity # PRAGMA integrity_check cdidx db --integrity-check # 従来の同義表記 cdidx db integrity --db ./.cdidx/codeindex.db # 特定 DB を指定 cdidx db integrity --json # 機械可読な結果 +cdidx db integrity --json --show-paths # error diagnostic に DB path を明示的に残す cdidx db schema --summary-only --json # SQL 本文なしで件数だけ確認 cdidx db schema --type table --name files --json # schema object を exact に絞り込み cdidx db schema --limit 20 --max-sql-chars 4000 --exclude-internal --json cdidx db checkpoint before-prune --dry-run --json # snapshot 対象 file と byte 数を preview ``` -DB を read-only で開いて SQLite の `PRAGMA integrity_check` を実行し、`ok` か、検出された破損行の一覧を出力します。終了コードは安定しており、`0` = 健全、`2` (NotFound) = ファイル無し、`3` (DatabaseError) = 破損検出です。SQLite には汎用的な修復プリミティブが無いため、チェックが失敗した場合は `cdidx index --rebuild` で再構築するのが推奨復旧手段です。 +DB を read-only で開いて SQLite の `PRAGMA integrity_check` を実行し、`ok` か、検出された破損行の一覧を出力します。終了コードは安定しており、`0` = 健全、`2` (NotFound) = ファイル無し、`3` (DatabaseError) = 破損または不正な database、lock / busy 競合は transient-database exit code です。SQLite には汎用的な修復プリミティブが無いため、チェックが失敗した場合は `cdidx index --rebuild` で再構築するのが推奨復旧手段です。 + +`vacuum`、`backfill-fold`、`optimize` (`index --optimize` を含む)、`db integrity` の失敗は database-error classifier version `1` を共有します。classifier は例外 message ではなく SQLite primary result code から locked / busy、read-only、corrupt、not-a-database を分類し、JSON では安定した `error_code`、`category`、`hint`、`path`、`path_redacted` と任意の SQLite result-code field を返します。absolute database path と file URI は既定で値全体を `` に置換し、relative path は呼び出し側の表記を維持します。明示的な diagnostic で full path が必要な場合だけ `--show-paths` を指定してください。directory またはアクセス不能な preflight target は missing / invalid database と推測せず、regular file と permission の案内を伴う `E008_DB_ERROR` / `database_inaccessible` として扱います。lock metadata が利用できる場合、optimize の lock failure には上限付きの holder PID / start-time detail も含まれます。成功した `db integrity --json` は従来の `integrity_ok` shape を維持し、失敗時はこの共有 error response を使います。recovery hint は category ごとに選ばれるため、corruption に lock retry を案内したり、lock 競合で健全な database の rebuild を案内したりしません。 `db schema` は support bundle 向けに、既定では従来どおり full schema dump を維持します。`--summary-only` を付けると object 件数だけを返し、`--type ` と `--name ` を組み合わせると exact projection を適用できます。schema diagnostics を小さく保つには `--limit`、`--max-sql-chars`、`--exclude-internal` を使います。 @@ -4410,7 +4417,7 @@ DB を read-only で開いて SQLite の `PRAGMA integrity_check` を実行し `cdidx optimize --dry-run --json` は index lock を取得せず、source DB/WAL/SHM file も変更せずに FTS5 maintenance を preview します。結果には DB/core table/FTS の size、page と freelist の指標、incremental write に基づく推奨、現在の lock/readiness 状態、利用可能な場合は前回所要時間に基づく見積もり、repair mode での schema 初期化または migration の確認を含む、実際の optimize が行う操作が含まれます。`object_sizes_measurement` は、正確な `dbstat` page byte と、SQLite が `dbstat` を提供しない場合の logical-payload fallback を区別します。 -`--json` の診断出力は自動化向けに安定した `severity` と `diagnostic_code` を含みます。`db --integrity-check --json` は `integrity_ok` / `integrity_failed` を返し、`db schema --json` は `schema_ok` / `schema_truncated` に加えて `object_type_counts` と `object_type_omitted_counts` で SQLite の table / index / trigger / view 件数と省略数を返します。 +成功時の `--json` 診断出力は自動化向けに安定した `severity` と `diagnostic_code` を含みます。`db --integrity-check --json` の成功時は `integrity_ok` を返し、失敗時は上記の共有 database-error response を返します。`db schema --json` は `schema_ok` / `schema_truncated` に加えて `object_type_counts` と `object_type_omitted_counts` で SQLite の table / index / trigger / view 件数と省略数を返します。 DB / WAL の肥大や空き page を確認したい場合は `status --json` の `maintenance_guidance` を見ます。既定では WAL が 64 MiB 以上で `checkpoint_recommended`、`freelist_count / page_count` が 0.20 以上で `vacuum_recommended` になり、`recommended_command` と `post_maintenance_follow_up` が返ります。しきい値は `CDIDX_MAINTENANCE_WAL_WARN_BYTES` と `CDIDX_MAINTENANCE_FREELIST_WARN_RATIO` で調整できます。 @@ -5265,6 +5272,7 @@ raw match density を正確に測る、といった理由で全 raw chunk hit | `E024_CONFIG_INVALID` | `validate-config` が検出した設定 file の validation に失敗した | | `E025_HOOK_OPERATION_FAILED` | Git hook 操作が platform または filesystem boundary で失敗した | | `E026_NOT_GIT_REPOSITORY` | Git worktree 外で `hooks` を実行し、有効な `--project` も指定されていなかった | +| `E027_DB_NOT_DATABASE` | SQLite が対象を database ではないと拒否した、または maintenance validation が CodeIndex database ではないと判定した | ### reader エラーのデバッグ diff --git a/changelog.d/unreleased/4856.fixed.md b/changelog.d/unreleased/4856.fixed.md new file mode 100644 index 000000000..7f82a83ed --- /dev/null +++ b/changelog.d/unreleased/4856.fixed.md @@ -0,0 +1,20 @@ +--- +category: fixed +issues: + - 4856 +affected: + - src/CodeIndex/Cli + - src/CodeIndex/Database/DbContext.cs + - tests/CodeIndex.Tests + - USER_GUIDE.md + - DEVELOPER_GUIDE.md + - TESTING_GUIDE.md +--- + +## English + +- **Database maintenance failures now share one versioned classifier and error contract (#4856)** — `vacuum`, `backfill-fold`, `optimize`, and `db integrity` classify SQLite result codes consistently, replace absolute paths and file URIs wholesale unless `--show-paths` is explicit, preserve caller-spelled relative paths, keep JSON valid, and provide failure-specific recovery guidance. Inaccessible targets and lock-holder metadata also retain actionable diagnostics. This preserves the recovery intent of #1526 and #4338 without treating corruption as lock contention. + +## 日本語 + +- **database maintenance の失敗を単一の versioned classifier と error contract に統一しました (#4856)** — `vacuum`、`backfill-fold`、`optimize`、`db integrity` は SQLite result code を一貫して分類し、`--show-paths` を明示しない限り absolute path と file URI の値全体を置換し、呼び出し側が指定した relative path は維持します。有効な JSON、failure 別の recovery guidance、アクセス不能 target と lock-holder metadata の actionable diagnostic も返します。corruption を lock contention と誤分類せず、#1526 と #4338 の recovery intent も維持します。 diff --git a/src/CodeIndex/Cli/CliContractManifest.cs b/src/CodeIndex/Cli/CliContractManifest.cs index 79799490f..8be4d68e1 100644 --- a/src/CodeIndex/Cli/CliContractManifest.cs +++ b/src/CodeIndex/Cli/CliContractManifest.cs @@ -63,6 +63,7 @@ internal static class CliContractManifest new(nameof(CommandErrorCodes.ConfigInvalid), CommandErrorCodes.ConfigInvalid, null), new(nameof(CommandErrorCodes.HookOperationFailed), CommandErrorCodes.HookOperationFailed, null), new(nameof(CommandErrorCodes.NotGitRepository), CommandErrorCodes.NotGitRepository, null), + new(nameof(CommandErrorCodes.DbNotDatabase), CommandErrorCodes.DbNotDatabase, CommandExitCodes.DatabaseError), ]; public static IReadOnlyList CliJsonRootTypes { get; } = LoadCliJsonRootTypes(); diff --git a/src/CodeIndex/Cli/CliFlagSchema.cs b/src/CodeIndex/Cli/CliFlagSchema.cs index 881c5d1d0..a29bdc682 100644 --- a/src/CodeIndex/Cli/CliFlagSchema.cs +++ b/src/CodeIndex/Cli/CliFlagSchema.cs @@ -422,6 +422,7 @@ private static IReadOnlyList BuildAll() new() { Name = "--optimize", Description = "Optimize the existing FTS5 table without scanning files", PrimaryCommands = Set("index") }, new() { Name = "--symbols-only", Description = "Build chunks and symbols while skipping reference graph extraction", PrimaryCommands = Set("index") }, new() { Name = "--dry-run", Description = "Preview without writing", PrimaryCommands = Set("index", "backfill-fold", "optimize", "vacuum") }, + new() { Name = "--show-paths", Description = "Show full database paths in maintenance diagnostics; paths are redacted by default", PrimaryCommands = Set("index", "backfill-fold", "optimize", "vacuum", "db") }, new() { Name = "--dry-run-path-limit", ValuePlaceholder = "", Description = "Dry run only: candidate path processing limit before truncated lower-bound estimates", PrimaryCommands = Set("index") }, new() { Name = "--no-checkpoint", Description = "Skip the automatic DB checkpoint before maintenance", PrimaryCommands = Set("backfill-fold") }, new() { Name = "--force", Description = "Bypass the per-database index lock", PrimaryCommands = Set("index") }, diff --git a/src/CodeIndex/Cli/CommandErrorCodes.cs b/src/CodeIndex/Cli/CommandErrorCodes.cs index 9ef0ce15b..8d072ddd9 100644 --- a/src/CodeIndex/Cli/CommandErrorCodes.cs +++ b/src/CodeIndex/Cli/CommandErrorCodes.cs @@ -92,4 +92,7 @@ internal static class CommandErrorCodes /// The requested Git hook operation did not target a Git repository. public const string NotGitRepository = "E026_NOT_GIT_REPOSITORY"; + + /// SQLite rejected the file as not being a database, or CodeIndex validation rejected its format. + public const string DbNotDatabase = "E027_DB_NOT_DATABASE"; } diff --git a/src/CodeIndex/Cli/ConsoleUi.cs b/src/CodeIndex/Cli/ConsoleUi.cs index 3555ddc6d..95b171e99 100644 --- a/src/CodeIndex/Cli/ConsoleUi.cs +++ b/src/CodeIndex/Cli/ConsoleUi.cs @@ -75,11 +75,11 @@ public static partial class ConsoleUi private static readonly (string Command, string Usage)[] CommandUsageLines = [ - ("index", "cdidx index [--db ] [--rebuild [--yes]] [--optimize] [--symbols-only] [--verbose] [--dry-run [--dry-run-path-limit ]] [--force] [--quiet] [--json] [--allow-partial] [--memory-trace] [--duration-format ] [--notify ] [--max-file-bytes ] [--max-symbols-per-file ] [--max-references-per-file ] [--follow-symlinks ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]] [--watch [--debounce ] [--watch-pending-path-limit ]]"), + ("index", "cdidx index [--db ] [--rebuild [--yes]] [--optimize [--show-paths]] [--symbols-only] [--verbose] [--dry-run [--dry-run-path-limit ]] [--force] [--quiet] [--json] [--allow-partial] [--memory-trace] [--duration-format ] [--notify ] [--max-file-bytes ] [--max-symbols-per-file ] [--max-references-per-file ] [--follow-symlinks ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]] [--watch [--debounce ] [--watch-pending-path-limit ]]"), ("hooks", "cdidx hooks [--project ] [--force] [--json]"), - ("backfill-fold", "cdidx backfill-fold [--db ] [--dry-run] [--no-checkpoint] [--json]"), - ("optimize", "cdidx optimize [--db ] [--dry-run] [--json]"), - ("vacuum", "cdidx vacuum [--db ] [--dry-run] [--json]"), + ("backfill-fold", "cdidx backfill-fold [--db ] [--dry-run] [--no-checkpoint] [--show-paths] [--json]"), + ("optimize", "cdidx optimize [--db ] [--dry-run] [--show-paths] [--json]"), + ("vacuum", "cdidx vacuum [--db ] [--dry-run] [--show-paths] [--json]"), ("index-commits", "cdidx index --commits [commit-ref ...] [--db ] [--verbose] [--dry-run [--dry-run-path-limit ]] [--json] [--allow-partial] [--memory-trace] [--duration-format ] [--max-file-bytes ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]]"), ("index-changed-between", "cdidx index --changed-between [--db ] [--verbose] [--dry-run [--dry-run-path-limit ]] [--json] [--allow-partial] [--memory-trace] [--duration-format ] [--max-file-bytes ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]]"), ("index-files", "cdidx index --files [path ...] [--db ] [--verbose] [--dry-run [--dry-run-path-limit ]] [--json] [--allow-partial] [--memory-trace] [--duration-format ] [--max-file-bytes ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]]"), @@ -112,14 +112,14 @@ private static readonly (string Command, string Usage)[] CommandUsageLines = ("config-show", "cdidx config show [--json] [--show-paths]"), ("validate-config", "cdidx validate-config [--json]"), ("doctor", "cdidx doctor [--json] [--redact-paths|--show-paths] [--env-inventory[=compact|full]] [--env-domain ] [--env-category ] [--env-sensitivity ] [--max-json-bytes ]"), - ("db", "cdidx db integrity|--integrity-check [--db ] [--json]"), + ("db", "cdidx db integrity|--integrity-check [--db ] [--show-paths] [--json]"), ("db", $"cdidx db schema [--type ] [--name ] [--limit ] [--max-sql-chars ] [--summary-only] [--include-internal|--exclude-internal] [--db ] [--json]"), ("db", "cdidx db prune --dry-run|--apply [--db ] [--json]"), ("db", "cdidx db checkpoint [name<=128] [--dry-run] [--db ] [--json]"), ("db", "cdidx db checkpoints --list|--delete |--prune [--keep ] [--dry-run] [--db ] [--json]"), ("db", "cdidx db restore [--dry-run] [--db ] [--json]"), ("db", "cdidx db restore-backups --list|--prune [--keep ] [--dry-run] [--db ] [--json]"), - ("db-integrity", "cdidx db integrity|--integrity-check [--db ] [--json]"), + ("db-integrity", "cdidx db integrity|--integrity-check [--db ] [--show-paths] [--json]"), ("db-schema", $"cdidx db schema [--type ] [--name ] [--limit ] [--max-sql-chars ] [--summary-only] [--include-internal|--exclude-internal] [--db ] [--json]"), ("db-prune", "cdidx db prune --dry-run|--apply [--db ] [--json]"), ("db-checkpoint", "cdidx db checkpoint [name<=128] [--dry-run] [--db ] [--json]"), diff --git a/src/CodeIndex/Cli/DbCommandRunner.ArgumentParser.cs b/src/CodeIndex/Cli/DbCommandRunner.ArgumentParser.cs index b8a1e4d67..5e12bb8ed 100644 --- a/src/CodeIndex/Cli/DbCommandRunner.ArgumentParser.cs +++ b/src/CodeIndex/Cli/DbCommandRunner.ArgumentParser.cs @@ -14,6 +14,7 @@ private sealed class DbArgumentParser { private string dbPath = Path.Combine(".cdidx", "codeindex.db"); private bool json; + private bool showPaths; private bool integrityCheck; private bool schema; private bool prune; @@ -68,6 +69,9 @@ internal DbCommandOptions Parse(string[] args) case "--json": json = true; break; + case "--show-paths": + showPaths = true; + break; case "--integrity-check": integrityCheck = true; break; @@ -226,7 +230,7 @@ internal DbCommandOptions Parse(string[] args) parseError = "--list is only valid with `cdidx db checkpoints --list`"; break; case "--help" or "-h": - return new DbCommandOptions { ShowHelp = true, DbPath = dbPath, Json = json }; + return new DbCommandOptions { ShowHelp = true, DbPath = dbPath, Json = json, ShowPaths = showPaths }; default: if (args[i].StartsWith('-')) parseError = $"db does not support option: '{args[i]}'"; @@ -252,6 +256,12 @@ private void ValidateOptionCombinations() parseError = "--dry-run is only valid with a supported preview operation."; if (parseError is null && pruneApply && !prune) parseError = "--apply is only valid with `cdidx db prune --apply`."; + if (parseError is null + && showPaths + && (schema || prune || checkpoint || listCheckpoints || restore || restoreBackups)) + { + parseError = "--show-paths is only valid with `cdidx db integrity` or `cdidx db --integrity-check`."; + } } private DbCommandOptions BuildOptions() @@ -260,6 +270,7 @@ private DbCommandOptions BuildOptions() { DbPath = dbPath, Json = json, + ShowPaths = showPaths, IntegrityCheck = integrityCheck, Schema = schema, Prune = prune, diff --git a/src/CodeIndex/Cli/DbCommandRunner.cs b/src/CodeIndex/Cli/DbCommandRunner.cs index 41d893865..5e586e762 100644 --- a/src/CodeIndex/Cli/DbCommandRunner.cs +++ b/src/CodeIndex/Cli/DbCommandRunner.cs @@ -113,6 +113,19 @@ public static int Run(string[] cmdArgs, JsonSerializerOptions jsonOptions, Cance CommandErrorCodes.DbError); if (!isUri && !File.Exists(LongPath.EnsureWindowsPrefix(dbPath))) + { + if (options.IntegrityCheck) + { + return MaintenanceDatabaseErrorWriter.Write( + options.Json, + jsonOptions, + MaintenanceDatabaseErrorClassifier.Create( + "db integrity", + dbPath, + options.ShowPaths, + MaintenanceDatabaseFailureKind.Missing)); + } + return WriteCommandError( options.Json, jsonOptions, @@ -120,6 +133,7 @@ public static int Run(string[] cmdArgs, JsonSerializerOptions jsonOptions, Cance CommandExitCodes.NotFound, "Point `--db` at an existing `codeindex.db`, or run `cdidx index ` first to create one.", CommandErrorCodes.DbNotFound); + } try { @@ -172,16 +186,29 @@ private static int RunIntegrityCheck(DbCommandOptions options, JsonSerializerOpt var jsonContext = CliJsonSerializerContextFactory.Create(jsonOptions); var displayDbPath = DbPathResolver.FormatDbPathForDisplay(dbPath); + if (!ok) + { + return MaintenanceDatabaseErrorWriter.Write( + options.Json, + jsonOptions, + MaintenanceDatabaseErrorClassifier.Create( + "db integrity", + dbPath, + options.ShowPaths, + MaintenanceDatabaseFailureKind.Corrupt, + details: issues, + detailsTruncated: result.Truncated)); + } + if (options.Json) { - var severity = ok ? "ok" : "error"; Console.WriteLine(JsonSerializer.Serialize( new DbIntegrityCheckJsonResult( displayDbPath, - ok, - severity, - ok ? "integrity_ok" : "integrity_failed", - ok ? new List() : issues, + true, + "ok", + "integrity_ok", + new List(), result.Truncated, result.RowsTruncated, result.TextTruncated, @@ -193,21 +220,10 @@ private static int RunIntegrityCheck(DbCommandOptions options, JsonSerializerOpt { Console.WriteLine("Integrity check"); Console.WriteLine($" database: {displayDbPath}"); - Console.WriteLine($" result : {(ok ? "ok" : "corrupted")}"); - if (!ok) - { - Console.WriteLine($" issues : {ConsoleUi.Counted(issues.Count, "row")}{(result.RowsTruncated ? " (truncated)" : string.Empty)}"); - if (result.Truncated) - Console.WriteLine($" truncated: yes (row limit {IntegrityCheckRowLimit:N0}, text limit {IntegrityCheckTextLimit:N0} chars)"); - foreach (var line in issues) - Console.WriteLine($" - {line}"); - Console.WriteLine(); - CommandErrorWriter.WriteStderr($"Error [{CommandErrorCodes.DbIntegrityFailed}]: SQLite reported integrity_check failures."); - CommandErrorWriter.WriteStderr("Hint: rebuild with `cdidx index --rebuild` to discard the corrupted DB and start fresh."); - } + Console.WriteLine(" result : ok"); } - return ok ? CommandExitCodes.Success : CommandExitCodes.DatabaseError; + return CommandExitCodes.Success; } catch (OperationCanceledException) { @@ -218,13 +234,14 @@ private static int RunIntegrityCheck(DbCommandOptions options, JsonSerializerOpt if (JsonOutputFailure.TryHandle(ex, out var exitCode)) return exitCode; - return WriteCommandError( + return MaintenanceDatabaseErrorWriter.Write( options.Json, jsonOptions, - $"failed to run integrity check: {CommandErrorWriter.FormatSanitizedExceptionMessage(ex)}", - CommandExitCodes.DatabaseError, - "Retry `cdidx db --integrity-check`. If this persists, the DB may be unreadable; rebuild with `cdidx index --rebuild`.", - CommandErrorCodes.DbError); + MaintenanceDatabaseErrorClassifier.FromException( + "db integrity", + dbPath, + options.ShowPaths, + ex)); } } @@ -364,6 +381,7 @@ internal sealed class DbCommandOptions public string DbPath { get; init; } = string.Empty; public bool Json { get; init; } public bool ShowHelp { get; init; } + public bool ShowPaths { get; init; } public bool IntegrityCheck { get; init; } public bool Schema { get; init; } public bool Prune { get; init; } diff --git a/src/CodeIndex/Cli/IndexCommandRunner.Maintenance.cs b/src/CodeIndex/Cli/IndexCommandRunner.Maintenance.cs index fe0a43d56..b517b1929 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.Maintenance.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.Maintenance.cs @@ -11,7 +11,7 @@ public static partial class IndexCommandRunner private static readonly string[] AcceptedBackfillFoldFlags = [ "--db", "--json", "--dry-run", "--help", - "--no-checkpoint", + "--no-checkpoint", "--show-paths", ]; private static readonly (string Name, string[] Columns)[] OptimizeObjectDefinitions = [ @@ -72,13 +72,14 @@ internal static int RunOptimizeFts( return dbUriValidationExitCode.Value; if (!options.DryRun && DbPathResolver.UriRequestsReadOnly(options.DbPath)) - return WriteCommandError( + return MaintenanceDatabaseErrorWriter.Write( options.Json, jsonOptions, - $"database must be writable for optimize: {options.DbPath}", - CommandExitCodes.DatabaseError, - "Point `--db` at a writable filesystem path, or omit read-only URI parameters such as `immutable=1` / `mode=ro`.", - CommandErrorCodes.DbNotWritable); + MaintenanceDatabaseErrorClassifier.Create( + "optimize", + options.DbPath, + options.ShowPaths, + MaintenanceDatabaseFailureKind.NotWritable)); return RunOptimizeFtsForDb( Path.GetFullPath(DbPathResolver.NormalizeDbPath(options.DbPath)), @@ -86,7 +87,9 @@ internal static int RunOptimizeFts( jsonOptions, projectPath: null, options.DryRun, - forceLogicalObjectSizeFallbackForTesting); + forceLogicalObjectSizeFallbackForTesting, + options.ShowPaths, + diagnosticDbPath: options.DbPath); } private static int RunOptimizeFtsForDb( @@ -95,25 +98,38 @@ private static int RunOptimizeFtsForDb( JsonSerializerOptions jsonOptions, string? projectPath, bool dryRun = false, - bool forceLogicalObjectSizeFallbackForTesting = false) + bool forceLogicalObjectSizeFallbackForTesting = false, + bool showPaths = false, + string? diagnosticDbPath = null) { + var errorDbPath = diagnosticDbPath ?? dbPath; if (dryRun) return RunOptimizeFtsPreviewForDb( dbPath, json, jsonOptions, - forceLogicalObjectSizeFallbackForTesting); + forceLogicalObjectSizeFallbackForTesting, + showPaths, + errorDbPath); - if (!DbContext.TryValidateExistingCodeIndexDb(dbPath, out var validationMessage, out var isNotFound)) - return WriteCommandError( + if (!DbContext.TryValidateExistingCodeIndexDb( + dbPath, + requireWritable: true, + requireSupportedUserVersion: false, + out _, + out _, + out _, + out var validationFailure, + out var validationException)) + return MaintenanceDatabaseErrorWriter.Write( json, jsonOptions, - validationMessage, - isNotFound ? CommandExitCodes.NotFound : CommandExitCodes.DatabaseError, - isNotFound - ? "Point `--db` at an existing `codeindex.db`, or run `cdidx index ` first to create one." - : "Point `--db` at an existing CodeIndex database created by `cdidx index`, then retry `cdidx optimize`.", - isNotFound ? CommandErrorCodes.DbNotFound : CommandErrorCodes.DbError); + MaintenanceDatabaseErrorClassifier.FromValidation( + "optimize", + errorDbPath, + showPaths, + validationFailure, + validationException)); var stopwatch = Stopwatch.StartNew(); try @@ -149,29 +165,31 @@ private static int RunOptimizeFtsForDb( catch (IndexLockConflictException ex) { var holderDescription = DescribeLockHolder(ex.Holder); - var message = string.IsNullOrEmpty(holderDescription) - ? "another cdidx index is already running on this database" - : $"another cdidx index is already running on this database ({holderDescription})"; - return WriteCommandError( + return MaintenanceDatabaseErrorWriter.Write( json, jsonOptions, - message, - CommandExitCodes.DatabaseError, - "Wait for the running index to finish, then retry `cdidx optimize`.", - CommandErrorCodes.DbLocked); + MaintenanceDatabaseErrorClassifier.Create( + "optimize", + errorDbPath, + showPaths, + MaintenanceDatabaseFailureKind.Locked, + details: string.IsNullOrEmpty(holderDescription) + ? null + : [holderDescription])); } catch (Exception ex) { if (JsonOutputFailure.TryHandle(ex, out var exitCode)) return exitCode; - return WriteCommandError( + return MaintenanceDatabaseErrorWriter.Write( json, jsonOptions, - $"failed to optimize FTS5 index: {CommandErrorWriter.FormatSanitizedExceptionMessage(ex)}", - CommandExitCodes.DatabaseError, - "Ensure no other writer is holding the database lock, then retry `cdidx optimize`.", - CommandErrorCodes.DbError); + MaintenanceDatabaseErrorClassifier.FromException( + "optimize", + errorDbPath, + showPaths, + ex)); } } @@ -179,35 +197,37 @@ private static int RunOptimizeFtsPreviewForDb( string dbPath, bool json, JsonSerializerOptions jsonOptions, - bool forceLogicalObjectSizeFallbackForTesting) + bool forceLogicalObjectSizeFallbackForTesting, + bool showPaths, + string errorDbPath) { var stopwatch = Stopwatch.StartNew(); if (!File.Exists(LongPath.EnsureWindowsPrefix(dbPath))) { - return WriteCommandError( + return MaintenanceDatabaseErrorWriter.Write( json, jsonOptions, - $"database not found: {dbPath}", - CommandExitCodes.NotFound, - "Point `--db` at an existing `codeindex.db`, or run `cdidx index ` first to create one.", - CommandErrorCodes.DbNotFound); + MaintenanceDatabaseErrorClassifier.Create( + "optimize", + errorDbPath, + showPaths, + MaintenanceDatabaseFailureKind.Missing)); } try { var (lockState, lockHolder) = IndexLock.ProbeReadOnly(IndexLock.GetLockPath(dbPath)); using var db = new DbContext(DbOpenIntent.QueryOnly, dbPath); - if (!db.TryValidateIsCodeIndexDb(out var validationReason)) + if (!db.TryValidateIsCodeIndexDb(out _)) { - return WriteCommandError( + return MaintenanceDatabaseErrorWriter.Write( json, jsonOptions, - $"database is not an existing CodeIndex DB: {dbPath}", - CommandExitCodes.DatabaseError, - string.IsNullOrWhiteSpace(validationReason) - ? "Point `--db` at an existing CodeIndex database created by `cdidx index`, then retry `cdidx optimize --dry-run`." - : "The database failed CodeIndex validation; rebuild it with `cdidx index --rebuild` before optimizing.", - CommandErrorCodes.DbError); + MaintenanceDatabaseErrorClassifier.Create( + "optimize", + errorDbPath, + showPaths, + MaintenanceDatabaseFailureKind.NotDatabase)); } var status = new DbReader(db).GetStatus(); @@ -329,13 +349,14 @@ result.EstimatedDurationMs is { } durationEstimateMs if (JsonOutputFailure.TryHandle(ex, out var exitCode)) return exitCode; - return WriteCommandError( + return MaintenanceDatabaseErrorWriter.Write( json, jsonOptions, - $"failed to preview FTS5 optimization: {CommandErrorWriter.FormatSanitizedExceptionMessage(ex)}", - CommandExitCodes.DatabaseError, - "Ensure the database and temporary snapshot storage are readable, then retry `cdidx optimize --dry-run`.", - CommandErrorCodes.DbError); + MaintenanceDatabaseErrorClassifier.FromException( + "optimize", + errorDbPath, + showPaths, + ex)); } } @@ -510,22 +531,36 @@ internal static int RunBackfillFold( "Run `cdidx backfill-fold --help` to see the supported command shape.", CommandErrorCodes.UsageError); + if (!options.DryRun && DbPathResolver.UriRequestsReadOnly(options.DbPath)) + { + return MaintenanceDatabaseErrorWriter.Write( + options.Json, + jsonOptions, + MaintenanceDatabaseErrorClassifier.Create( + "backfill-fold", + options.DbPath, + options.ShowPaths, + MaintenanceDatabaseFailureKind.NotWritable)); + } + if (!DbContext.TryValidateExistingCodeIndexDb( options.DbPath, requireWritable: !options.DryRun, requireSupportedUserVersion: false, - out var validationMessage, - out var isNotFound, - out _)) - return WriteCommandError( + out _, + out _, + out _, + out var validationFailure, + out var validationException)) + return MaintenanceDatabaseErrorWriter.Write( options.Json, jsonOptions, - validationMessage, - isNotFound ? CommandExitCodes.NotFound : CommandExitCodes.DatabaseError, - isNotFound - ? "Point `--db` at an existing `codeindex.db`, or run `cdidx index ` first to create one." - : "Point `--db` at an existing CodeIndex database created by `cdidx index`, then retry `cdidx backfill-fold`.", - isNotFound ? CommandErrorCodes.DbNotFound : CommandErrorCodes.DbError); + MaintenanceDatabaseErrorClassifier.FromValidation( + "backfill-fold", + options.DbPath, + options.ShowPaths, + validationFailure, + validationException)); try { @@ -644,13 +679,14 @@ internal static int RunBackfillFold( if (JsonOutputFailure.TryHandle(ex, out var exitCode)) return exitCode; - return WriteCommandError( + return MaintenanceDatabaseErrorWriter.Write( options.Json, jsonOptions, - $"failed to backfill folded-name columns: {CommandErrorWriter.FormatSanitizedExceptionMessage(ex)}", - CommandExitCodes.DatabaseError, - "Retry `cdidx backfill-fold`. If this persists, rebuild the index with `cdidx index --rebuild`.", - CommandErrorCodes.DbError); + MaintenanceDatabaseErrorClassifier.FromException( + "backfill-fold", + options.DbPath, + options.ShowPaths, + ex)); } } @@ -659,6 +695,7 @@ private static OptimizeFtsCommandOptions ParseOptimizeFtsArgs(string[] args) var dbPath = Path.Combine(".cdidx", "codeindex.db"); bool json = false; bool dryRun = false; + bool showPaths = false; string? parseError = null; for (int i = 0; i < args.Length; i++) @@ -674,6 +711,9 @@ private static OptimizeFtsCommandOptions ParseOptimizeFtsArgs(string[] args) case "--dry-run": dryRun = true; break; + case "--show-paths": + showPaths = true; + break; case "--help" or "-h": return new OptimizeFtsCommandOptions { ShowHelp = true }; default: @@ -690,6 +730,7 @@ private static OptimizeFtsCommandOptions ParseOptimizeFtsArgs(string[] args) DbPath = dbPath, Json = json, DryRun = dryRun, + ShowPaths = showPaths, ParseError = parseError, }; } @@ -700,6 +741,7 @@ private static BackfillFoldCommandOptions ParseBackfillFoldArgs(string[] args) var json = false; var dryRun = false; var noCheckpoint = false; + var showPaths = false; string? parseError = null; for (int i = 0; i < args.Length; i++) @@ -718,8 +760,11 @@ private static BackfillFoldCommandOptions ParseBackfillFoldArgs(string[] args) case "--no-checkpoint": noCheckpoint = true; break; + case "--show-paths": + showPaths = true; + break; case "--help" or "-h": - return new BackfillFoldCommandOptions { ShowHelp = true, DbPath = dbPath, Json = json, DryRun = dryRun, NoCheckpoint = noCheckpoint }; + return new BackfillFoldCommandOptions { ShowHelp = true, DbPath = dbPath, Json = json, DryRun = dryRun, NoCheckpoint = noCheckpoint, ShowPaths = showPaths }; default: if (args[i].StartsWith("-", StringComparison.Ordinal)) { @@ -739,6 +784,7 @@ private static BackfillFoldCommandOptions ParseBackfillFoldArgs(string[] args) Json = json, DryRun = dryRun, NoCheckpoint = noCheckpoint, + ShowPaths = showPaths, ParseError = parseError, }; } diff --git a/src/CodeIndex/Cli/IndexCommandRunner.Parse.cs b/src/CodeIndex/Cli/IndexCommandRunner.Parse.cs index 1c4d0eb31..7789f2141 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.Parse.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.Parse.cs @@ -46,6 +46,7 @@ public static IndexCommandOptions ParseArgs(string[] args) bool yes = false; bool watch = false; bool optimizeOnly = false; + bool showPaths = false; bool symbolsOnly = false; bool memoryTrace = false; int? watchDebounceMs = null; @@ -141,6 +142,9 @@ public static IndexCommandOptions ParseArgs(string[] args) case "--optimize": optimizeOnly = true; break; + case "--show-paths": + showPaths = true; + break; case "--symbols-only": symbolsOnly = true; break; @@ -321,6 +325,8 @@ public static IndexCommandOptions ParseArgs(string[] args) var themes = new[] { "--sushi", "--coffee", "--ramen", "--wine", "--beer", "--matcha", "--whisky" }; easterEgg = themes[Random.Shared.Next(themes.Length)]; } + if (showPaths && !optimizeOnly) + parseError ??= "--show-paths is only valid with `cdidx index --optimize`."; return new IndexCommandOptions { @@ -352,6 +358,7 @@ public static IndexCommandOptions ParseArgs(string[] args) Yes = yes, Watch = watch, OptimizeOnly = optimizeOnly, + ShowPaths = showPaths, SymbolsOnly = symbolsOnly, MemoryTrace = memoryTrace, WatchDebounceMs = watchDebounceMs, diff --git a/src/CodeIndex/Cli/IndexCommandRunner.cs b/src/CodeIndex/Cli/IndexCommandRunner.cs index 582d8f5d1..487743c6e 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.cs @@ -198,16 +198,32 @@ internal static int Run( if (!options.Json && !options.Quiet) { + var projectDisplayPath = options.OptimizeOnly + ? MaintenanceDatabaseErrorClassifier.FormatPathForOutput( + Path.GetFullPath(options.ProjectPath!), + options.ShowPaths) + : Path.GetFullPath(options.ProjectPath!); + var databaseDisplayPath = options.OptimizeOnly + ? MaintenanceDatabaseErrorClassifier.FormatPathForOutput( + resolvedDbPath, + options.ShowPaths) + : resolvedDbPath; ConsoleUi.PrintBanner(); CommandOutputWriter.WriteLine(); - CommandOutputWriter.WriteLine($" Project : {Path.GetFullPath(options.ProjectPath!)}"); - CommandOutputWriter.WriteLine($" Output : {resolvedDbPath}"); + CommandOutputWriter.WriteLine($" Project : {projectDisplayPath}"); + CommandOutputWriter.WriteLine($" Output : {databaseDisplayPath}"); CommandOutputWriter.WriteLine($" Mode : {(options.OptimizeOnly ? "optimize" : mode)}"); CommandOutputWriter.WriteLine(); } if (options.OptimizeOnly) - return RunOptimizeFtsForDb(resolvedDbPath, options.Json, jsonOptions, options.ProjectPath, options.DryRun); + return RunOptimizeFtsForDb( + resolvedDbPath, + options.Json, + jsonOptions, + options.ProjectPath, + options.DryRun, + showPaths: options.ShowPaths); bool ignoreCase; string ignoreRuleRoot; @@ -597,6 +613,7 @@ public sealed class IndexCommandOptions public bool Yes { get; init; } public bool Watch { get; init; } public bool OptimizeOnly { get; init; } + public bool ShowPaths { get; init; } public bool SymbolsOnly { get; init; } public int? WatchDebounceMs { get; init; } public int WatchPendingPathLimit { get; init; } = IndexWatchRunner.DefaultWatchPendingPathLimit; @@ -726,6 +743,7 @@ public sealed class BackfillFoldCommandOptions public string DbPath { get; init; } = Path.Combine(".cdidx", "codeindex.db"); public bool Json { get; init; } public bool DryRun { get; init; } + public bool ShowPaths { get; init; } public bool NoCheckpoint { get; init; } public string? ParseError { get; init; } } @@ -736,5 +754,6 @@ public sealed class OptimizeFtsCommandOptions public string DbPath { get; init; } = Path.Combine(".cdidx", "codeindex.db"); public bool Json { get; init; } public bool DryRun { get; init; } + public bool ShowPaths { get; init; } public string? ParseError { get; init; } } diff --git a/src/CodeIndex/Cli/JsonOutputContracts.cs b/src/CodeIndex/Cli/JsonOutputContracts.cs index 577ab877a..1978f5558 100644 --- a/src/CodeIndex/Cli/JsonOutputContracts.cs +++ b/src/CodeIndex/Cli/JsonOutputContracts.cs @@ -92,6 +92,22 @@ internal sealed record CommandErrorJsonResult( [property: JsonPropertyName("usage")] string? Usage = null, [property: JsonPropertyName("api_version")] string ApiVersion = JsonOutputContract.ApiVersion) : IVersionedJsonResult; +internal sealed record MaintenanceDatabaseErrorJsonResult( + [property: JsonPropertyName("status")] string Status, + [property: JsonPropertyName("message")] string Message, + [property: JsonPropertyName("hint")] string Hint, + [property: JsonPropertyName("error_code")] string ErrorCode, + [property: JsonPropertyName("category")] string Category, + [property: JsonPropertyName("database_error_classifier_version")] string DatabaseErrorClassifierVersion, + [property: JsonPropertyName("operation")] string Operation, + [property: JsonPropertyName("path")] string Path, + [property: JsonPropertyName("path_redacted")] bool PathRedacted, + [property: JsonPropertyName("sqlite_error_code")] int? SqliteErrorCode, + [property: JsonPropertyName("sqlite_extended_error_code")] int? SqliteExtendedErrorCode, + [property: JsonPropertyName("details")] IReadOnlyList? Details, + [property: JsonPropertyName("details_truncated")] bool? DetailsTruncated, + [property: JsonPropertyName("api_version")] string ApiVersion = JsonOutputContract.ApiVersion) : IVersionedJsonResult; + internal sealed record DoctorJsonResult( [property: JsonPropertyName("api_version")] string ApiVersion, [property: JsonPropertyName("version")] string Version, @@ -997,6 +1013,7 @@ internal sealed record ValidateConfigJsonResult( [JsonSerializable(typeof(CompactSearchResult))] [JsonSerializable(typeof(CompactSearchResult[]))] [JsonSerializable(typeof(CommandErrorJsonResult))] +[JsonSerializable(typeof(MaintenanceDatabaseErrorJsonResult))] [JsonSerializable(typeof(ActiveWorkspaceStatusJsonResult))] [JsonSerializable(typeof(ConfigShowJsonResult))] [JsonSerializable(typeof(ConfigFileStatusJsonResult))] diff --git a/src/CodeIndex/Cli/MaintenanceDatabaseError.cs b/src/CodeIndex/Cli/MaintenanceDatabaseError.cs new file mode 100644 index 000000000..06c82957f --- /dev/null +++ b/src/CodeIndex/Cli/MaintenanceDatabaseError.cs @@ -0,0 +1,405 @@ +using System.Text.Json; +using CodeIndex.Database; +using CodeIndex.Diagnostics; +using CodeIndex.Indexer; +using Microsoft.Data.Sqlite; + +namespace CodeIndex.Cli; + +internal enum MaintenanceDatabaseFailureKind +{ + Missing, + Locked, + SchemaTooNew, + NotWritable, + Inaccessible, + Corrupt, + NotDatabase, + Error, +} + +internal sealed record MaintenanceDatabaseError( + string Operation, + string Message, + string Hint, + string ErrorCode, + string Category, + int ExitCode, + string Path, + bool PathRedacted, + int? SqliteErrorCode = null, + int? SqliteExtendedErrorCode = null, + IReadOnlyList? Details = null, + bool? DetailsTruncated = null); + +internal static class MaintenanceDatabaseErrorClassifier +{ + internal const string Version = "1"; + private const int PathDiagnosticLimit = 512; + private static ReadOnlySpan SqliteHeader => "SQLite format 3\0"u8; + + internal static MaintenanceDatabaseError FromValidation( + string operation, + string dbPath, + bool showPaths, + ExistingCodeIndexDbValidationFailure validationFailure, + Exception? validationException) + { + if (validationException != null) + return FromException(operation, dbPath, showPaths, validationException); + + if (validationFailure != ExistingCodeIndexDbValidationFailure.None) + { + var kind = validationFailure switch + { + ExistingCodeIndexDbValidationFailure.Missing => + MaintenanceDatabaseFailureKind.Missing, + ExistingCodeIndexDbValidationFailure.InvalidDatabase => + MaintenanceDatabaseFailureKind.NotDatabase, + ExistingCodeIndexDbValidationFailure.SchemaTooNew => + MaintenanceDatabaseFailureKind.SchemaTooNew, + ExistingCodeIndexDbValidationFailure.Inaccessible + or ExistingCodeIndexDbValidationFailure.InvalidTarget => + MaintenanceDatabaseFailureKind.Inaccessible, + _ => MaintenanceDatabaseFailureKind.Error, + }; + return Create(operation, dbPath, showPaths, kind); + } + + return ProbeFileState(dbPath) switch + { + MaintenanceDatabaseFileState.Missing => + Create(operation, dbPath, showPaths, MaintenanceDatabaseFailureKind.Missing), + MaintenanceDatabaseFileState.InvalidHeader => + Create(operation, dbPath, showPaths, MaintenanceDatabaseFailureKind.NotDatabase), + _ => Create(operation, dbPath, showPaths, MaintenanceDatabaseFailureKind.NotDatabase), + }; + } + + internal static MaintenanceDatabaseError FromException( + string operation, + string dbPath, + bool showPaths, + Exception exception) + { + ArgumentNullException.ThrowIfNull(exception); + + var structured = FindCodeIndexException(exception); + var structuredKind = structured?.Code switch + { + CommandErrorCodes.DbNotFound => MaintenanceDatabaseFailureKind.Missing, + CommandErrorCodes.DbLocked => MaintenanceDatabaseFailureKind.Locked, + CommandErrorCodes.SchemaTooNew => MaintenanceDatabaseFailureKind.SchemaTooNew, + CommandErrorCodes.DbNotWritable => MaintenanceDatabaseFailureKind.NotWritable, + CommandErrorCodes.DbIntegrityFailed => MaintenanceDatabaseFailureKind.Corrupt, + CommandErrorCodes.DbNotDatabase => MaintenanceDatabaseFailureKind.NotDatabase, + _ => (MaintenanceDatabaseFailureKind?)null, + }; + var sqlite = FindSqliteException(exception); + if (structuredKind is { } knownKind) + { + return Create( + operation, + dbPath, + showPaths, + knownKind, + sqlite?.SqliteErrorCode, + sqlite?.SqliteExtendedErrorCode); + } + + if (sqlite != null) + { + var primaryCode = sqlite.SqliteErrorCode != 0 + ? sqlite.SqliteErrorCode + : sqlite.SqliteExtendedErrorCode & 0xff; + var kind = primaryCode switch + { + 5 or 6 => MaintenanceDatabaseFailureKind.Locked, + 8 => MaintenanceDatabaseFailureKind.NotWritable, + 11 => MaintenanceDatabaseFailureKind.Corrupt, + 26 => MaintenanceDatabaseFailureKind.NotDatabase, + 14 when ProbeFileState(dbPath) == MaintenanceDatabaseFileState.Missing => + MaintenanceDatabaseFailureKind.Missing, + _ => MaintenanceDatabaseFailureKind.Error, + }; + return Create( + operation, + dbPath, + showPaths, + kind, + sqlite.SqliteErrorCode, + sqlite.SqliteExtendedErrorCode); + } + + var fileState = ProbeFileState(dbPath); + if (fileState == MaintenanceDatabaseFileState.Missing) + return Create(operation, dbPath, showPaths, MaintenanceDatabaseFailureKind.Missing); + if (fileState == MaintenanceDatabaseFileState.InvalidHeader) + return Create(operation, dbPath, showPaths, MaintenanceDatabaseFailureKind.NotDatabase); + + return Create(operation, dbPath, showPaths, MaintenanceDatabaseFailureKind.Error); + } + + private static CodeIndexException? FindCodeIndexException(Exception exception) + { + if (exception is CodeIndexException structured) + return structured; + if (exception is AggregateException aggregate) + { + foreach (var inner in aggregate.InnerExceptions) + { + var nested = FindCodeIndexException(inner); + if (nested != null) + return nested; + } + } + + return exception.InnerException == null + ? null + : FindCodeIndexException(exception.InnerException); + } + + internal static MaintenanceDatabaseError Create( + string operation, + string dbPath, + bool showPaths, + MaintenanceDatabaseFailureKind kind, + int? sqliteErrorCode = null, + int? sqliteExtendedErrorCode = null, + IReadOnlyList? details = null, + bool? detailsTruncated = null) + { + var path = FormatPathForOutput(dbPath, showPaths, out var pathRedacted); + var safeDetails = FormatDetailsForOutput(details, showPaths); + var (message, hint, errorCode, category, exitCode) = kind switch + { + MaintenanceDatabaseFailureKind.Missing => ( + "database file was not found", + "Create or refresh the index with `cdidx index `, or import a known-good index archive, then retry the maintenance command.", + CommandErrorCodes.DbNotFound, + "database_missing", + CommandExitCodes.NotFound), + MaintenanceDatabaseFailureKind.Locked => ( + "database is locked or busy", + "Wait for the active writer to finish, inspect the index-lock owner diagnostics when available, then retry with backoff.", + CommandErrorCodes.DbLocked, + "database_locked", + CommandExitCodes.TransientDatabaseError), + MaintenanceDatabaseFailureKind.SchemaTooNew => ( + "database was written by a newer cdidx schema", + "Use a current cdidx binary, or rebuild the index with this version only after confirming that replacing the newer index is safe.", + CommandErrorCodes.SchemaTooNew, + "database_schema_too_new", + CommandExitCodes.DatabaseError), + MaintenanceDatabaseFailureKind.NotWritable => ( + "database is not writable", + "Point `--db` at a writable database and directory, or use a read-only maintenance preview when the command supports one.", + CommandErrorCodes.DbNotWritable, + "database_not_writable", + CommandExitCodes.DatabaseError), + MaintenanceDatabaseFailureKind.Inaccessible => ( + "database target could not be accessed", + "Point `--db` at a readable regular database file, check filesystem permissions, and retry the maintenance command.", + CommandErrorCodes.DbError, + "database_inaccessible", + CommandExitCodes.DatabaseError), + MaintenanceDatabaseFailureKind.Corrupt => ( + "database is corrupt", + "Run `cdidx db integrity` if the file remains readable, then rebuild with `cdidx index --rebuild` or import a known-good index archive.", + CommandErrorCodes.DbIntegrityFailed, + "database_corrupt", + CommandExitCodes.DatabaseError), + MaintenanceDatabaseFailureKind.NotDatabase => ( + "file is not a valid SQLite CodeIndex database", + "Point `--db` at an index created by cdidx, rebuild it with `cdidx index --rebuild`, or import a known-good index archive.", + CommandErrorCodes.DbNotDatabase, + "database_not_a_database", + CommandExitCodes.DatabaseError), + _ => ( + "database maintenance operation failed", + "Check database access and run `cdidx db integrity`; rebuild or import a known-good index if the file is damaged.", + CommandErrorCodes.DbError, + "database_error", + CommandExitCodes.DatabaseError), + }; + + return new MaintenanceDatabaseError( + operation, + message, + hint, + errorCode, + category, + exitCode, + path, + pathRedacted, + sqliteErrorCode, + sqliteExtendedErrorCode, + safeDetails, + detailsTruncated); + } + + internal static string FormatPathForOutput(string dbPath, bool showPaths) + => FormatPathForOutput(dbPath, showPaths, out _); + + private static string FormatPathForOutput(string dbPath, bool showPaths, out bool pathRedacted) + { + var bounded = DiagnosticRedactor.BoundDiagnosticText( + SqliteFileUri.TruncateDiagnosticValue(dbPath), + PathDiagnosticLimit); + if (!showPaths && IsAbsolutePathOrFileUri(bounded)) + { + pathRedacted = true; + return DiagnosticRedactor.AngleRedacted; + } + + var formatted = DiagnosticRedactor.RedactSensitiveText( + bounded, + redactPaths: false); + pathRedacted = !string.Equals(formatted, bounded, StringComparison.Ordinal); + return formatted; + } + + private static bool IsAbsolutePathOrFileUri(string path) + { + if (SqliteFileUri.StartsWithFileScheme(path) + || Path.IsPathFullyQualified(path) + || (path.Length > 0 && path[0] is '/' or '\\')) + { + return true; + } + + if (path.Length >= 3 + && char.IsAsciiLetter(path[0]) + && path[1] == ':' + && path[2] is '\\' or '/') + { + return true; + } + + return false; + } + + private static IReadOnlyList? FormatDetailsForOutput( + IReadOnlyList? details, + bool showPaths) + { + if (details == null) + return null; + + return details + .Select(detail => DiagnosticRedactor.BoundDiagnosticText( + DiagnosticRedactor.RedactSensitiveText(detail, redactPaths: !showPaths), + maxChars: 8192)) + .ToArray(); + } + + private static SqliteException? FindSqliteException(Exception exception) + { + if (exception is SqliteException sqlite) + return sqlite; + if (exception is AggregateException aggregate) + { + foreach (var inner in aggregate.InnerExceptions) + { + var nested = FindSqliteException(inner); + if (nested != null) + return nested; + } + } + + return exception.InnerException == null + ? null + : FindSqliteException(exception.InnerException); + } + + private static MaintenanceDatabaseFileState ProbeFileState(string dbPath) + { + try + { + var localPath = dbPath; + if (SqliteFileUri.StartsWithFileScheme(dbPath) + && (!DbPathResolver.TryNormalizeDbPath(dbPath, out localPath, out _) + || string.IsNullOrWhiteSpace(localPath))) + { + return MaintenanceDatabaseFileState.Unknown; + } + + var longPath = LongPath.EnsureWindowsPrefix(localPath); + if (!File.Exists(longPath)) + return MaintenanceDatabaseFileState.Missing; + + Span header = stackalloc byte[16]; + using var stream = new FileStream( + longPath, + FileMode.Open, + FileAccess.Read, + FileShare.ReadWrite | FileShare.Delete); + var bytesRead = stream.Read(header); + return bytesRead == header.Length && header.SequenceEqual(SqliteHeader) + ? MaintenanceDatabaseFileState.SqliteHeader + : MaintenanceDatabaseFileState.InvalidHeader; + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or ArgumentException or NotSupportedException) + { + return MaintenanceDatabaseFileState.Unknown; + } + } + + private enum MaintenanceDatabaseFileState + { + Unknown, + Missing, + InvalidHeader, + SqliteHeader, + } +} + +internal static class MaintenanceDatabaseErrorWriter +{ + internal static int Write( + bool json, + JsonSerializerOptions jsonOptions, + MaintenanceDatabaseError error) + { + if (json) + { + var payload = new MaintenanceDatabaseErrorJsonResult( + "error", + error.Message, + error.Hint, + error.ErrorCode, + error.Category, + MaintenanceDatabaseErrorClassifier.Version, + error.Operation, + error.Path, + error.PathRedacted, + error.SqliteErrorCode, + error.SqliteExtendedErrorCode, + error.Details, + error.DetailsTruncated); + CommandOutputWriter.WriteLine(JsonSerializer.Serialize( + payload, + CliJsonSerializerContextFactory.Create(jsonOptions).MaintenanceDatabaseErrorJsonResult)); + return error.ExitCode; + } + + CommandErrorWriter.WriteStderr($"Error [{error.ErrorCode}]: {error.Message}."); + CommandErrorWriter.WriteStderr($"Database: {error.Path}"); + CommandErrorWriter.WriteStderr($"Category: {error.Category} (classifier v{MaintenanceDatabaseErrorClassifier.Version})"); + if (error.SqliteErrorCode is { } sqliteCode) + { + CommandErrorWriter.WriteStderr( + $"SQLite result: {sqliteCode}; extended: {error.SqliteExtendedErrorCode ?? sqliteCode}"); + } + if (error.Details is { Count: > 0 }) + { + CommandErrorWriter.WriteStderr("Details:"); + foreach (var detail in error.Details) + CommandErrorWriter.WriteStderr($" - {detail}"); + if (error.DetailsTruncated == true) + CommandErrorWriter.WriteStderr(" - "); + } + CommandErrorWriter.WriteStderr($"Hint: {error.Hint}"); + return error.ExitCode; + } +} diff --git a/src/CodeIndex/Cli/ProgramRunner.ErrorHandling.cs b/src/CodeIndex/Cli/ProgramRunner.ErrorHandling.cs index 39e6b87e3..e1a4ab31d 100644 --- a/src/CodeIndex/Cli/ProgramRunner.ErrorHandling.cs +++ b/src/CodeIndex/Cli/ProgramRunner.ErrorHandling.cs @@ -42,6 +42,7 @@ private static bool IsTruthyEnvironmentVariable(string name) CommandErrorCodes.SchemaTooNew => CommandExitCodes.DatabaseError, CommandErrorCodes.TempStoreExhausted => CommandExitCodes.DatabaseError, CommandErrorCodes.DbError => CommandExitCodes.DatabaseError, + CommandErrorCodes.DbNotDatabase => CommandExitCodes.DatabaseError, CommandErrorCodes.DirectoryNotFound => CommandExitCodes.NotFound, CommandErrorCodes.FeatureUnavailable => CommandExitCodes.FeatureUnavailable, CommandErrorCodes.UsageError => CommandExitCodes.InvalidArgument, diff --git a/src/CodeIndex/Cli/QueryCommandRunner.ArgumentParser.General.cs b/src/CodeIndex/Cli/QueryCommandRunner.ArgumentParser.General.cs index c57bca380..018845479 100644 --- a/src/CodeIndex/Cli/QueryCommandRunner.ArgumentParser.General.cs +++ b/src/CodeIndex/Cli/QueryCommandRunner.ArgumentParser.General.cs @@ -43,6 +43,9 @@ private bool TryParseGeneralOption(string normalizedArg, string currentArg, stri case "--dry-run": dryRun = true; break; + case "--show-paths": + showPaths = true; + break; case "--pretty": break; case "--compact": diff --git a/src/CodeIndex/Cli/QueryCommandRunner.ArgumentParser.cs b/src/CodeIndex/Cli/QueryCommandRunner.ArgumentParser.cs index 209022d41..a4fff7aab 100644 --- a/src/CodeIndex/Cli/QueryCommandRunner.ArgumentParser.cs +++ b/src/CodeIndex/Cli/QueryCommandRunner.ArgumentParser.cs @@ -91,6 +91,7 @@ private sealed partial class QueryArgumentParser private bool dbPathExplicit; private bool readOnly; private bool dryRun; + private bool showPaths; private bool checkWorkspace; private bool statusCheckExplicit; private TimeSpan? staleAfter; @@ -375,6 +376,7 @@ private QueryCommandOptions BuildOptions(DbPathResolution dbResolution, string r DbPathExplicit = dbPathExplicit, ReadOnly = readOnly, DryRun = dryRun, + ShowPaths = showPaths, DataDir = dbResolution.DataDir, DataDirSource = dbResolution.DataDirSource, Json = json ?? jsonDefault, diff --git a/src/CodeIndex/Cli/QueryCommandRunner.Maintenance.cs b/src/CodeIndex/Cli/QueryCommandRunner.Maintenance.cs index 95950dd21..ba623ad94 100644 --- a/src/CodeIndex/Cli/QueryCommandRunner.Maintenance.cs +++ b/src/CodeIndex/Cli/QueryCommandRunner.Maintenance.cs @@ -1,5 +1,6 @@ using System.Text.Json; using CodeIndex.Database; +using CodeIndex.Indexer; namespace CodeIndex.Cli; @@ -18,12 +19,20 @@ public static int RunVacuum(string[] cmdArgs, JsonSerializerOptions jsonOptions, validateDefaultMaxLineWidth: false); if (TryWriteUnsupportedOptionError("vacuum", cmdArgs, CliFlagSchema.GetAcceptedFlagNamesForCommand("vacuum"))) return CommandExitCodes.UsageError; - var explicitDbPathError = BuildExplicitDbPathParseError(options); - if (explicitDbPathError != null && explicitDbPathError.Contains(CommandErrorCodes.DbNotFound, StringComparison.Ordinal)) + if (options.ParseError == null + && options.DbPathExplicit + && !string.IsNullOrWhiteSpace(options.DbPath) + && !SqliteFileUri.StartsWithFileScheme(options.DbPath) + && !File.Exists(LongPath.EnsureWindowsPrefix(options.DbPath))) { - CommandErrorWriter.WriteStderr(explicitDbPathError); - CommandErrorWriter.WriteStderr("Hint: point `--db` at an existing `codeindex.db`, or run `cdidx index ` first to create one."); - return CommandExitCodes.NotFound; + return MaintenanceDatabaseErrorWriter.Write( + options.Json, + jsonOptions, + MaintenanceDatabaseErrorClassifier.Create( + "vacuum", + options.DbPath, + options.ShowPaths, + MaintenanceDatabaseFailureKind.Missing)); } if (TryWriteParseError(options, "vacuum")) return CommandExitCodes.UsageError; @@ -33,20 +42,38 @@ public static int RunVacuum(string[] cmdArgs, JsonSerializerOptions jsonOptions, try { cancellationToken.ThrowIfCancellationRequested(); + if (!options.DryRun && DbPathResolver.UriRequestsReadOnly(options.DbPath)) + { + return MaintenanceDatabaseErrorWriter.Write( + options.Json, + jsonOptions, + MaintenanceDatabaseErrorClassifier.Create( + "vacuum", + options.DbPath, + options.ShowPaths, + MaintenanceDatabaseFailureKind.NotWritable)); + } + if (!DbContext.TryValidateExistingCodeIndexDb( options.DbPath, requireWritable: !options.DryRun, requireSupportedUserVersion: false, - out var validationMessage, - out var isNotFound, out _, + out _, + out _, + out var validationFailure, + out var validationException, cancellationToken)) { - CommandErrorWriter.WriteStderr($"Error [{(isNotFound ? CommandErrorCodes.DbNotFound : CommandErrorCodes.DbError)}]: {validationMessage}"); - CommandErrorWriter.WriteStderr(isNotFound - ? "Hint: point `--db` at an existing `codeindex.db`, or run `cdidx index ` first to create one." - : "Hint: point `--db` at an existing CodeIndex database created by `cdidx index`, then retry `cdidx vacuum`."); - return isNotFound ? CommandExitCodes.NotFound : CommandExitCodes.DatabaseError; + return MaintenanceDatabaseErrorWriter.Write( + options.Json, + jsonOptions, + MaintenanceDatabaseErrorClassifier.FromValidation( + "vacuum", + options.DbPath, + options.ShowPaths, + validationFailure, + validationException)); } using var db = new DbContext( @@ -87,5 +114,19 @@ public static int RunVacuum(string[] cmdArgs, JsonSerializerOptions jsonOptions, "Retry `cdidx vacuum` after the cancelling operation completes.", errorCode: CommandErrorCodes.Interrupted); } + catch (Exception ex) + { + if (JsonOutputFailure.TryHandle(ex, out var exitCode)) + return exitCode; + + return MaintenanceDatabaseErrorWriter.Write( + options.Json, + jsonOptions, + MaintenanceDatabaseErrorClassifier.FromException( + "vacuum", + options.DbPath, + options.ShowPaths, + ex)); + } } } diff --git a/src/CodeIndex/Cli/QueryCommandRunner.cs b/src/CodeIndex/Cli/QueryCommandRunner.cs index 85e3c1967..941b24f17 100644 --- a/src/CodeIndex/Cli/QueryCommandRunner.cs +++ b/src/CodeIndex/Cli/QueryCommandRunner.cs @@ -72,6 +72,7 @@ public sealed class QueryCommandOptions public bool DbPathExplicit { get; init; } public bool ReadOnly { get; init; } public bool DryRun { get; init; } + public bool ShowPaths { get; init; } public string? DataDir { get; init; } public string? DataDirSource { get; init; } public bool Json { get; init; } diff --git a/src/CodeIndex/Database/DbContext.cs b/src/CodeIndex/Database/DbContext.cs index beb44cabc..0c0f41b9f 100644 --- a/src/CodeIndex/Database/DbContext.cs +++ b/src/CodeIndex/Database/DbContext.cs @@ -8,6 +8,17 @@ namespace CodeIndex.Database; +internal enum ExistingCodeIndexDbValidationFailure +{ + None, + Missing, + Inaccessible, + InvalidTarget, + InvalidDatabase, + SchemaTooNew, + Exception, +} + /// /// Manages SQLite connection and schema initialization. /// SQLite接続とスキーマ初期化を管理する。 @@ -446,6 +457,27 @@ internal static bool TryValidateExistingCodeIndexDb( out bool isNotFound, out bool isSchemaTooNew, CancellationToken cancellationToken = default) + => TryValidateExistingCodeIndexDb( + dbPath, + requireWritable, + requireSupportedUserVersion, + out message, + out isNotFound, + out isSchemaTooNew, + out _, + out _, + cancellationToken); + + internal static bool TryValidateExistingCodeIndexDb( + string dbPath, + bool requireWritable, + bool requireSupportedUserVersion, + out string message, + out bool isNotFound, + out bool isSchemaTooNew, + out ExistingCodeIndexDbValidationFailure validationFailure, + out Exception? validationException, + CancellationToken cancellationToken = default) => TryValidateExistingCodeIndexDb(dbPath, openTarget => { var mode = requireWritable ? SqliteConnectionPolicyMode.ReadWrite : SqliteConnectionPolicyMode.ReadOnly; @@ -458,6 +490,8 @@ internal static bool TryValidateExistingCodeIndexDb( out message, out isNotFound, out isSchemaTooNew, + out validationFailure, + out validationException, cancellationToken); internal static bool TryValidateExistingCodeIndexDb( @@ -478,6 +512,31 @@ internal static bool TryValidateExistingCodeIndexDb( out message, out isNotFound, out _, + out _, + out _, + cancellationToken); + + internal static bool TryValidateExistingCodeIndexDb( + string dbPath, + Func createConnection, + Action openConnection, + Action? sleep, + out string message, + out bool isNotFound, + out Exception? validationException, + CancellationToken cancellationToken = default) + => TryValidateExistingCodeIndexDb( + dbPath, + createConnection, + openConnection, + sleep, + requireWritable: true, + requireSupportedUserVersion: false, + out message, + out isNotFound, + out _, + out _, + out validationException, cancellationToken); private static bool TryValidateExistingCodeIndexDb( @@ -490,15 +549,20 @@ private static bool TryValidateExistingCodeIndexDb( out string message, out bool isNotFound, out bool isSchemaTooNew, + out ExistingCodeIndexDbValidationFailure validationFailure, + out Exception? validationException, CancellationToken cancellationToken = default) { message = string.Empty; isNotFound = false; isSchemaTooNew = false; + validationFailure = ExistingCodeIndexDbValidationFailure.None; + validationException = null; cancellationToken.ThrowIfCancellationRequested(); if (SqliteFileUri.StartsWithFileScheme(dbPath) && !SqliteFileUri.TryValidateBounds(dbPath, out var boundsError)) { + validationFailure = ExistingCodeIndexDbValidationFailure.InvalidTarget; message = FormatDatabaseOpenFailure( DatabaseOpenInvalidUriCategory, dbPath, @@ -508,6 +572,7 @@ private static bool TryValidateExistingCodeIndexDb( if (requireWritable && SqliteFileUri.StartsWithFileScheme(dbPath) && SqliteFileUri.RequestsReadOnly(dbPath)) { + validationFailure = ExistingCodeIndexDbValidationFailure.Inaccessible; message = $"database must be writable: {dbPath}"; return false; } @@ -518,6 +583,7 @@ private static bool TryValidateExistingCodeIndexDb( if (!TryGetLocalPath(dbPath, out var normalized, out var pathFailureReason) || normalized == null) { + validationFailure = ExistingCodeIndexDbValidationFailure.InvalidTarget; message = FormatDatabaseOpenFailure( DatabaseOpenInvalidUriCategory, dbPath, @@ -539,6 +605,12 @@ private static bool TryValidateExistingCodeIndexDb( }; message = FormatDatabaseOpenFailure(category, dbPath); isNotFound = category == DatabaseOpenMissingCategory; + validationFailure = preflight switch + { + DatabasePathProbe.Missing => ExistingCodeIndexDbValidationFailure.Missing, + DatabasePathProbe.PermissionDenied => ExistingCodeIndexDbValidationFailure.Inaccessible, + _ => ExistingCodeIndexDbValidationFailure.InvalidTarget, + }; return false; } @@ -557,6 +629,7 @@ private static bool TryValidateExistingCodeIndexDb( cmd.CommandText = SqliteCommandPolicy.PragmaSql("application_id"); if (SqliteCommandPolicy.ReadInt64Scalar(cmd, "pragma application_id") != ApplicationId) { + validationFailure = ExistingCodeIndexDbValidationFailure.InvalidDatabase; message = $"database is not an existing CodeIndex DB: {dbPath}"; return false; } @@ -569,6 +642,7 @@ private static bool TryValidateExistingCodeIndexDb( if (unknownBits != 0) { isSchemaTooNew = true; + validationFailure = ExistingCodeIndexDbValidationFailure.SchemaTooNew; message = $"database was written by a newer cdidx schema stamp (user_version {userVersion}); this binary supports up to {CurrentSchemaVersion}: {dbPath}"; return false; } @@ -583,23 +657,30 @@ private static bool TryValidateExistingCodeIndexDb( if (RequiredCodeIndexTables.All(tables.Contains)) return true; + validationFailure = ExistingCodeIndexDbValidationFailure.InvalidDatabase; message = $"database is not an existing CodeIndex DB: {dbPath}"; return false; } catch (SqliteException ex) when (ex.SqliteErrorCode is 14) { + validationFailure = ExistingCodeIndexDbValidationFailure.Exception; + validationException = ex; var category = ClassifyCantOpenFailure(openTarget, ex.SqliteExtendedErrorCode); message = FormatDatabaseOpenFailure(category, dbPath); isNotFound = category == DatabaseOpenMissingCategory; return false; } - catch (SqliteException) + catch (SqliteException ex) { + validationFailure = ExistingCodeIndexDbValidationFailure.Exception; + validationException = ex; message = $"database is not an existing CodeIndex DB: {dbPath}"; return false; } catch (CodeIndexException ex) { + validationFailure = ExistingCodeIndexDbValidationFailure.Exception; + validationException = ex; message = ex.Message; return false; } diff --git a/tests/CodeIndex.Tests/CommandErrorCodesTests.cs b/tests/CodeIndex.Tests/CommandErrorCodesTests.cs index db3ac4f9b..eaba0acbd 100644 --- a/tests/CodeIndex.Tests/CommandErrorCodesTests.cs +++ b/tests/CodeIndex.Tests/CommandErrorCodesTests.cs @@ -50,15 +50,15 @@ public void DbIntegrityCheck_NoModeFlag_JsonIncludesUsageErrorCode() } [Fact] - public void DbIntegrityCheck_CorruptDb_StderrIncludesIntegrityFailedCode() + public void DbIntegrityCheck_InvalidDatabase_StderrIncludesNotDatabaseCode_Issue4856() { var dbPath = Path.Combine(Path.GetTempPath(), $"cdidx_codes_corrupt_{Guid.NewGuid():N}.db"); try { - // Write a file that opens as SQLite but fails `PRAGMA integrity_check` so the - // human-output corruption branch runs. The same fixture is used in - // DbCommandRunnerTests; we only assert the new bracketed-code prefix. - // SQLite として開けるが integrity_check で破損が出るファイルを作る fixture。 + // Keep a valid SQLite magic header but an invalid page layout. SQLite reports + // primary result code 26, which the maintenance classifier maps without + // inspecting exception wording. + // SQLite magic header は保ちつつ page layout を不正にし、primary code 26 の分類を固定する。 var header = System.Text.Encoding.ASCII.GetBytes("SQLite format 3\0"); var bytes = new byte[4096]; Array.Copy(header, bytes, header.Length); @@ -69,12 +69,7 @@ public void DbIntegrityCheck_CorruptDb_StderrIncludesIntegrityFailedCode() var (exitCode, _, stderr) = RunDbIntegrityCheckCapturingStreams(["--integrity-check", "--db", dbPath]); Assert.Equal(CommandExitCodes.DatabaseError, exitCode); - // Either the integrity-failed branch (E005) or the open-failure branch (E008) ran; - // both must carry a bracketed code so scripts can classify the failure without - // substring-matching the prose. - Assert.True( - stderr.Contains("[E005_DB_INTEGRITY_FAILED]") || stderr.Contains("[E008_DB_ERROR]"), - $"expected bracketed integrity code in stderr, got: {stderr}"); + Assert.Contains("[E027_DB_NOT_DATABASE]", stderr); } finally { diff --git a/tests/CodeIndex.Tests/CommandErrorWriterTests.cs b/tests/CodeIndex.Tests/CommandErrorWriterTests.cs index 44d5c541f..1a318fbdb 100644 --- a/tests/CodeIndex.Tests/CommandErrorWriterTests.cs +++ b/tests/CodeIndex.Tests/CommandErrorWriterTests.cs @@ -1,10 +1,119 @@ using CodeIndex.Cli; +using CodeIndex.Database; +using Microsoft.Data.Sqlite; namespace CodeIndex.Tests; [Collection("Console sensitive")] public class CommandErrorWriterTests { + [Theory] + [InlineData(5, CommandErrorCodes.DbLocked, "database_locked", CommandExitCodes.TransientDatabaseError)] + [InlineData(6, CommandErrorCodes.DbLocked, "database_locked", CommandExitCodes.TransientDatabaseError)] + [InlineData(8, CommandErrorCodes.DbNotWritable, "database_not_writable", CommandExitCodes.DatabaseError)] + [InlineData(11, CommandErrorCodes.DbIntegrityFailed, "database_corrupt", CommandExitCodes.DatabaseError)] + [InlineData(26, CommandErrorCodes.DbNotDatabase, "database_not_a_database", CommandExitCodes.DatabaseError)] + public void MaintenanceClassifier_UsesSqlitePrimaryCodes_Issue4856( + int sqliteCode, + string expectedErrorCode, + string expectedCategory, + int expectedExitCode) + { + var error = MaintenanceDatabaseErrorClassifier.FromException( + "test maintenance", + "/Users/alice/private/codeindex.db", + showPaths: false, + new SqliteException("message wording must not affect classification", sqliteCode)); + + Assert.Equal(expectedErrorCode, error.ErrorCode); + Assert.Equal(expectedCategory, error.Category); + Assert.Equal(expectedExitCode, error.ExitCode); + Assert.Equal(sqliteCode, error.SqliteErrorCode); + Assert.DoesNotContain("alice", error.Path, StringComparison.Ordinal); + if (sqliteCode == 11) + Assert.DoesNotContain("backoff", error.Hint, StringComparison.OrdinalIgnoreCase); + } + + [Theory] + [InlineData(CommandErrorCodes.SchemaTooNew, CommandErrorCodes.SchemaTooNew, "database_schema_too_new", CommandExitCodes.DatabaseError)] + [InlineData(CommandErrorCodes.DbLocked, CommandErrorCodes.DbLocked, "database_locked", CommandExitCodes.TransientDatabaseError)] + public void MaintenanceClassifier_PreservesStructuredDatabaseFailures_Issue4856( + string structuredCode, + string expectedErrorCode, + string expectedCategory, + int expectedExitCode) + { + var sqlite = structuredCode == CommandErrorCodes.DbLocked + ? new SqliteException("message wording must not affect classification", 5) + : null; + var exception = new CodeIndexException( + structuredCode, + CodeIndexExceptionCategory.Database, + "structured database failure", + path: "/Users/alice/private/codeindex.db", + innerException: sqlite); + + var error = MaintenanceDatabaseErrorClassifier.FromException( + "test maintenance", + "/Users/alice/private/codeindex.db", + showPaths: false, + exception); + + Assert.Equal(expectedErrorCode, error.ErrorCode); + Assert.Equal(expectedCategory, error.Category); + Assert.Equal(expectedExitCode, error.ExitCode); + Assert.DoesNotContain("alice", error.Path, StringComparison.Ordinal); + if (structuredCode == CommandErrorCodes.DbLocked) + Assert.Equal(5, error.SqliteErrorCode); + } + + [Theory] + [InlineData("/Users/alice/private/codeindex.db")] + [InlineData("/Users/alice/private/review-missing.db")] + [InlineData(@"C:\Users\alice\private\codeindex.db")] + [InlineData("file:///Users/alice/private/codeindex.db?immutable=1")] + public void MaintenanceClassifier_RedactsPlatformAbsolutePathsUnlessExplicitlyEnabled_Issue4856( + string dbPath) + { + var redacted = MaintenanceDatabaseErrorClassifier.FormatPathForOutput(dbPath, showPaths: false); + var diagnostic = MaintenanceDatabaseErrorClassifier.FormatPathForOutput(dbPath, showPaths: true); + + Assert.Equal("", redacted); + Assert.Equal(dbPath, diagnostic); + } + + [Theory] + [InlineData(nameof(ExistingCodeIndexDbValidationFailure.Inaccessible))] + [InlineData(nameof(ExistingCodeIndexDbValidationFailure.InvalidTarget))] + public void MaintenanceClassifier_UsesInaccessibleClassificationForPreflightFailures_Issue4856( + string validationFailureName) + { + var validationFailure = Enum.Parse(validationFailureName); + var error = MaintenanceDatabaseErrorClassifier.FromValidation( + "vacuum", + "/Users/alice/private/codeindex.db", + showPaths: false, + validationFailure, + validationException: null); + + Assert.Equal(CommandErrorCodes.DbError, error.ErrorCode); + Assert.Equal("database_inaccessible", error.Category); + Assert.Equal(CommandExitCodes.DatabaseError, error.ExitCode); + Assert.Equal("", error.Path); + Assert.Contains("readable regular database file", error.Hint); + Assert.DoesNotContain("--rebuild", error.Hint, StringComparison.Ordinal); + } + + [Fact] + public void MaintenanceClassifier_KeepsRelativePathInDefaultDiagnostics_Issue4856() + { + const string relativePath = ".cdidx/codeindex.db"; + + Assert.Equal( + relativePath, + MaintenanceDatabaseErrorClassifier.FormatPathForOutput(relativePath, showPaths: false)); + } + [Fact] public void ResolveMachineContract_NotFoundUsesGenericCodeWithoutDomainContext_Issue4855() { diff --git a/tests/CodeIndex.Tests/ConsoleUiTests.cs b/tests/CodeIndex.Tests/ConsoleUiTests.cs index 97a21dc64..6c1909885 100644 --- a/tests/CodeIndex.Tests/ConsoleUiTests.cs +++ b/tests/CodeIndex.Tests/ConsoleUiTests.cs @@ -388,13 +388,13 @@ public void PrintUsage_WithoutBanner_HidesAsciiArtAndEasterEggFlags() Assert.DoesNotContain("██████╗", output); Assert.Contains("Usage:", output); - Assert.Contains("cdidx index [--db ] [--rebuild [--yes]] [--optimize] [--symbols-only] [--verbose] [--dry-run [--dry-run-path-limit ]] [--force] [--quiet] [--json] [--allow-partial] [--memory-trace] [--duration-format ] [--notify ] [--max-file-bytes ] [--max-symbols-per-file ] [--max-references-per-file ] [--follow-symlinks ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]] [--watch [--debounce ] [--watch-pending-path-limit ]]", output); + Assert.Contains("cdidx index [--db ] [--rebuild [--yes]] [--optimize [--show-paths]] [--symbols-only] [--verbose] [--dry-run [--dry-run-path-limit ]] [--force] [--quiet] [--json] [--allow-partial] [--memory-trace] [--duration-format ] [--notify ] [--max-file-bytes ] [--max-symbols-per-file ] [--max-references-per-file ] [--follow-symlinks ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]] [--watch [--debounce ] [--watch-pending-path-limit ]]", output); Assert.Contains("cdidx hooks [--project ] [--force] [--json]", output); Assert.Contains("cdidx index --commits [commit-ref ...] [--db ] [--verbose] [--dry-run [--dry-run-path-limit ]] [--json] [--allow-partial] [--memory-trace] [--duration-format ] [--max-file-bytes ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]]", output); Assert.Contains("cdidx index --changed-between [--db ] [--verbose] [--dry-run [--dry-run-path-limit ]] [--json] [--allow-partial] [--memory-trace] [--duration-format ] [--max-file-bytes ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]]", output); Assert.Contains("cdidx index --files [path ...] [--db ] [--verbose] [--dry-run [--dry-run-path-limit ]] [--json] [--allow-partial] [--memory-trace] [--duration-format ] [--max-file-bytes ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]]", output); - Assert.Contains("cdidx backfill-fold [--db ] [--dry-run] [--no-checkpoint] [--json]", output); - Assert.Contains("cdidx optimize [--db ] [--dry-run] [--json]", output); + Assert.Contains("cdidx backfill-fold [--db ] [--dry-run] [--no-checkpoint] [--show-paths] [--json]", output); + Assert.Contains("cdidx optimize [--db ] [--dry-run] [--show-paths] [--json]", output); Assert.Contains("cdidx license", output); Assert.Contains("cdidx completions ", output); Assert.Contains("cdidx --completions ", output); diff --git a/tests/CodeIndex.Tests/DbCommandRunnerTests.cs b/tests/CodeIndex.Tests/DbCommandRunnerTests.cs index c9582392f..6de4d3cd0 100644 --- a/tests/CodeIndex.Tests/DbCommandRunnerTests.cs +++ b/tests/CodeIndex.Tests/DbCommandRunnerTests.cs @@ -120,6 +120,23 @@ public void ParseArgs_HelpFlagSetsShowHelp() Assert.True(options.ShowHelp); } + [Fact] + public void ParseArgs_ShowPathsEnablesDiagnosticPathDisclosure_Issue4856() + { + var options = DbCommandRunner.ParseArgs(["integrity", "--show-paths"]); + + Assert.True(options.ShowPaths); + Assert.Null(options.ParseError); + } + + [Fact] + public void ParseArgs_ShowPathsRejectsUnrelatedDbModes_Issue4856() + { + var options = DbCommandRunner.ParseArgs(["schema", "--show-paths"]); + + Assert.Contains("only valid with `cdidx db integrity`", options.ParseError); + } + [Fact] public void ParseArgs_UnknownOptionRecordsParseError() { @@ -264,7 +281,10 @@ public void Run_MissingDb_ReturnsNotFoundWithHint() var (exitCode, _, stderr) = RunAndCaptureStreams(["--integrity-check", "--db", missingDb]); Assert.Equal(CommandExitCodes.NotFound, exitCode); - Assert.Contains("database not found", stderr); + Assert.Contains("database file was not found", stderr); + Assert.Contains("database_missing", stderr); + Assert.Contains("", stderr); + Assert.DoesNotContain(missingDb, stderr, StringComparison.Ordinal); Assert.Contains("cdidx index ", stderr); } @@ -277,8 +297,9 @@ public void Run_IntegrityCheck_FileUriSemicolonPayloadDoesNotCreateDatabase_Issu { var (exitCode, _, stderr) = RunAndCaptureStreams(["--integrity-check", "--db", uri]); - Assert.Equal(CommandExitCodes.DatabaseError, exitCode); - Assert.Contains("failed to run integrity check", stderr); + Assert.Equal(CommandExitCodes.NotFound, exitCode); + Assert.Contains("database file was not found", stderr); + Assert.Contains("database_missing", stderr); Assert.False(File.Exists(missingDb)); } finally @@ -336,7 +357,12 @@ public void Run_MissingDb_JsonShapeIncludesHint() Assert.Equal(CommandExitCodes.NotFound, exitCode); Assert.Equal("error", json.GetProperty("status").GetString()); - Assert.Contains("database not found", json.GetProperty("message").GetString()); + Assert.Equal(CommandErrorCodes.DbNotFound, json.GetProperty("error_code").GetString()); + Assert.Equal("database_missing", json.GetProperty("category").GetString()); + Assert.Equal("1", json.GetProperty("database_error_classifier_version").GetString()); + Assert.Equal("", json.GetProperty("path").GetString()); + Assert.True(json.GetProperty("path_redacted").GetBoolean()); + Assert.Contains("database file was not found", json.GetProperty("message").GetString()); Assert.Contains("cdidx index ", json.GetProperty("hint").GetString()); } @@ -432,7 +458,7 @@ public void Run_IntegrityCheck_JsonCancellationReturnsInterrupted_Issue3811() public void Run_IntegrityCheck_JsonReportsStableErrorSeverity() { var dbPath = TestProjectHelper.CreateTempDbPath("cdidx_db_integrity_error_json"); - DbCommandRunner.IntegrityCheckRowsForTesting = () => ["simulated corruption"]; + DbCommandRunner.IntegrityCheckRowsForTesting = () => [$"simulated corruption at {dbPath}"]; try { InitializeEmptyDb(dbPath); @@ -440,10 +466,15 @@ public void Run_IntegrityCheck_JsonReportsStableErrorSeverity() var (exitCode, json) = RunAndCaptureJson(["--integrity-check", "--db", dbPath, "--json"]); Assert.Equal(CommandExitCodes.DatabaseError, exitCode); - Assert.False(json.GetProperty("ok").GetBoolean()); - Assert.Equal("error", json.GetProperty("severity").GetString()); - Assert.Equal("integrity_failed", json.GetProperty("diagnostic_code").GetString()); - Assert.Equal("simulated corruption", json.GetProperty("issues")[0].GetString()); + Assert.Equal("error", json.GetProperty("status").GetString()); + Assert.Equal(CommandErrorCodes.DbIntegrityFailed, json.GetProperty("error_code").GetString()); + Assert.Equal("database_corrupt", json.GetProperty("category").GetString()); + Assert.Equal("1", json.GetProperty("database_error_classifier_version").GetString()); + Assert.Contains("simulated corruption at ", json.GetProperty("details")[0].GetString()); + Assert.DoesNotContain(dbPath, json.GetProperty("details")[0].GetString(), StringComparison.Ordinal); + Assert.Equal("", json.GetProperty("path").GetString()); + Assert.True(json.GetProperty("path_redacted").GetBoolean()); + Assert.DoesNotContain("locked", json.GetProperty("hint").GetString(), StringComparison.OrdinalIgnoreCase); } finally { @@ -2025,7 +2056,11 @@ public void Run_CorruptedDb_ReturnsDatabaseError() // non-"ok" rows; both paths must produce DatabaseError, never Success. // PRAGMA が例外を投げるか non-"ok" 行を返すかのいずれでも DatabaseError を返すべき。 Assert.Equal(CommandExitCodes.DatabaseError, exitCode); - Assert.NotEmpty(stderr); + Assert.Contains($"Error [{CommandErrorCodes.DbNotDatabase}]", stderr); + Assert.Contains("database_not_a_database", stderr); + Assert.Contains("", stderr); + Assert.DoesNotContain(dbPath, stderr, StringComparison.Ordinal); + Assert.DoesNotContain("retry with backoff", stderr, StringComparison.OrdinalIgnoreCase); } finally { @@ -2049,14 +2084,12 @@ public void Run_IntegrityCheck_JsonCapsRowsAndText_Issue2881() var (exitCode, json) = RunAndCaptureJson(["--integrity-check", "--db", dbPath, "--json"]); Assert.Equal(CommandExitCodes.DatabaseError, exitCode); - Assert.True(json.GetProperty("truncated").GetBoolean()); - Assert.True(json.GetProperty("rows_truncated").GetBoolean()); - Assert.True(json.GetProperty("text_truncated").GetBoolean()); - Assert.Equal(DbCommandRunner.IntegrityCheckRowLimit, json.GetProperty("row_limit").GetInt32()); - Assert.Equal(DbCommandRunner.IntegrityCheckTextLimit, json.GetProperty("text_limit").GetInt32()); - var issues = json.GetProperty("issues"); - Assert.Equal(DbCommandRunner.IntegrityCheckRowLimit, issues.GetArrayLength()); - Assert.EndsWith(" [truncated]", issues[0].GetString()); + Assert.Equal(CommandErrorCodes.DbIntegrityFailed, json.GetProperty("error_code").GetString()); + Assert.Equal("database_corrupt", json.GetProperty("category").GetString()); + Assert.True(json.GetProperty("details_truncated").GetBoolean()); + var details = json.GetProperty("details"); + Assert.Equal(DbCommandRunner.IntegrityCheckRowLimit, details.GetArrayLength()); + Assert.EndsWith(" [truncated]", details[0].GetString()); } finally { diff --git a/tests/CodeIndex.Tests/IndexCommandRunnerTests.cs b/tests/CodeIndex.Tests/IndexCommandRunnerTests.cs index 504a37be6..a96dcb272 100644 --- a/tests/CodeIndex.Tests/IndexCommandRunnerTests.cs +++ b/tests/CodeIndex.Tests/IndexCommandRunnerTests.cs @@ -5242,8 +5242,10 @@ public void RunBackfillFold_MissingDb_PrintsActionableHint() var exitCode = IndexCommandRunner.RunBackfillFold(["--db", missingDb], _jsonOptions); Assert.Equal(CommandExitCodes.NotFound, exitCode); - Assert.Contains("database not found", stderr.ToString()); - Assert.Contains("Point `--db` at an existing `codeindex.db`", stderr.ToString()); + Assert.Contains("database file was not found", stderr.ToString()); + Assert.Contains("Create or refresh the index", stderr.ToString()); + Assert.Contains("", stderr.ToString()); + Assert.DoesNotContain(missingDb, stderr.ToString(), StringComparison.Ordinal); } finally { @@ -5270,8 +5272,13 @@ public void RunBackfillFold_MissingDb_JsonIncludesHint() Assert.Equal(CommandExitCodes.NotFound, exitCode); Assert.Equal("error", json.GetProperty("status").GetString()); - Assert.Contains("database not found", json.GetProperty("message").GetString()); - Assert.Contains("Point `--db` at an existing `codeindex.db`", json.GetProperty("hint").GetString()); + Assert.Equal(CommandErrorCodes.DbNotFound, json.GetProperty("error_code").GetString()); + Assert.Equal("database_missing", json.GetProperty("category").GetString()); + Assert.Equal("1", json.GetProperty("database_error_classifier_version").GetString()); + Assert.Equal("", json.GetProperty("path").GetString()); + Assert.True(json.GetProperty("path_redacted").GetBoolean()); + Assert.Contains("database file was not found", json.GetProperty("message").GetString()); + Assert.Contains("Create or refresh the index", json.GetProperty("hint").GetString()); } finally { @@ -5371,8 +5378,12 @@ public void RunBackfillFold_PublishedTrimmedBinary_SerializesSuccessAndErrorJson Assert.Equal(CommandExitCodes.NotFound, errorExitCode); Assert.Equal("error", errorJson.GetProperty("status").GetString()); - Assert.Contains("database not found", errorJson.GetProperty("message").GetString()); - Assert.Contains("Point `--db` at an existing `codeindex.db`", errorJson.GetProperty("hint").GetString()); + Assert.Equal(CommandErrorCodes.DbNotFound, errorJson.GetProperty("error_code").GetString()); + Assert.Equal("database_missing", errorJson.GetProperty("category").GetString()); + Assert.Equal("1", errorJson.GetProperty("database_error_classifier_version").GetString()); + Assert.Equal("", errorJson.GetProperty("path").GetString()); + Assert.True(errorJson.GetProperty("path_redacted").GetBoolean()); + Assert.Contains("Create or refresh the index", errorJson.GetProperty("hint").GetString()); } finally { @@ -5875,8 +5886,7 @@ public void RunOptimizeFts_LockHeld_ReportsDbLocked() using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath)) db.InitializeSchema(); - Directory.CreateDirectory(Path.GetDirectoryName(lockPath)!); - using (var holder = new FileStream(lockPath, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None)) + using (var holder = IndexLock.Acquire(lockPath, Path.GetDirectoryName(dbPath)!)) { int previewExitCode; JsonElement previewJson; @@ -5920,11 +5930,17 @@ public void RunOptimizeFts_LockHeld_ReportsDbLocked() } } - Assert.Equal(CommandExitCodes.DatabaseError, exitCode); + Assert.Equal(CommandExitCodes.TransientDatabaseError, exitCode); Assert.Equal("error", json.GetProperty("status").GetString()); Assert.Equal(CommandErrorCodes.DbLocked, json.GetProperty("error_code").GetString()); - Assert.Contains("another cdidx index is already running", json.GetProperty("message").GetString()); - Assert.Contains("retry `cdidx optimize`", json.GetProperty("hint").GetString()); + Assert.Equal("database_locked", json.GetProperty("category").GetString()); + Assert.Equal("database is locked or busy", json.GetProperty("message").GetString()); + Assert.Contains("retry with backoff", json.GetProperty("hint").GetString()); + Assert.Equal("", json.GetProperty("path").GetString()); + Assert.True(json.GetProperty("path_redacted").GetBoolean()); + var detail = Assert.Single(json.GetProperty("details").EnumerateArray()); + Assert.Contains($"PID {Environment.ProcessId}", detail.GetString()); + Assert.DoesNotContain(dbPath, detail.GetString(), StringComparison.Ordinal); } } finally @@ -5936,6 +5952,73 @@ public void RunOptimizeFts_LockHeld_ReportsDbLocked() } } + [Fact] + public void RunOptimizeFts_MissingRelativePath_PreservesCallerSpelling_Issue4856() + { + var dbPath = $"cdidx_optimize_relative_missing_{Guid.NewGuid():N}.db"; + try + { + int exitCode; + JsonElement json; + lock (TestConsoleLock.Gate) + { + var originalOut = Console.Out; + try + { + using var stdout = new StringWriter(); + Console.SetOut(stdout); + exitCode = IndexCommandRunner.RunOptimizeFts(["--db", dbPath, "--json"], _jsonOptions); + using var document = JsonDocument.Parse(stdout.ToString()); + json = document.RootElement.Clone(); + } + finally + { + Console.SetOut(originalOut); + } + } + + Assert.Equal(CommandExitCodes.NotFound, exitCode); + Assert.Equal(CommandErrorCodes.DbNotFound, json.GetProperty("error_code").GetString()); + Assert.Equal(dbPath, json.GetProperty("path").GetString()); + Assert.False(json.GetProperty("path_redacted").GetBoolean()); + } + finally + { + DeleteFile(dbPath); + } + } + + [Fact] + public void Run_IndexOptimizeMissingDatabase_RedactsHumanPreambleUnlessEnabled_Issue4856() + { + var projectRoot = CreateTempProject(); + var dbPath = Path.Combine(projectRoot, "private", "missing.db"); + try + { + var (exitCode, stdout, stderr) = RunAndCaptureStreams( + [projectRoot, "--optimize", "--db", dbPath]); + var output = stdout + stderr; + + Assert.Equal(CommandExitCodes.NotFound, exitCode); + Assert.Contains("", output); + Assert.Contains(CommandErrorCodes.DbNotFound, output); + Assert.DoesNotContain(projectRoot, output, StringComparison.Ordinal); + Assert.DoesNotContain(dbPath, output, StringComparison.Ordinal); + + var (diagnosticExitCode, diagnosticStdout, diagnosticStderr) = RunAndCaptureStreams( + [projectRoot, "--optimize", "--db", dbPath, "--show-paths"]); + var diagnosticOutput = diagnosticStdout + diagnosticStderr; + + Assert.Equal(CommandExitCodes.NotFound, diagnosticExitCode); + Assert.Contains(projectRoot, diagnosticOutput, StringComparison.Ordinal); + Assert.Contains(dbPath, diagnosticOutput, StringComparison.Ordinal); + } + finally + { + DeleteDirectory(projectRoot); + } + } + [Fact] public void RunOptimizeFts_ReadOnlyUri_ReturnsDbNotWritable() { @@ -5995,7 +6078,8 @@ public void RunOptimizeFts_ReadOnlyUri_ReturnsDbNotWritable() Assert.Equal(CommandExitCodes.DatabaseError, exitCode); Assert.Equal("error", json.GetProperty("status").GetString()); Assert.Equal(CommandErrorCodes.DbNotWritable, json.GetProperty("error_code").GetString()); - Assert.Contains("database must be writable for optimize", json.GetProperty("message").GetString()); + Assert.Equal("database_not_writable", json.GetProperty("category").GetString()); + Assert.Equal("database is not writable", json.GetProperty("message").GetString()); using var verifyDb = new DbContext(DbOpenIntent.WriteIndex, dbPath); Assert.Equal("1", verifyDb.GetMetaString(DbWriter.FtsIncrementalWritesSinceOptimizeMetaKey)); @@ -6668,7 +6752,9 @@ public void RunBackfillFold_BlankFile_ReturnsDatabaseError() Assert.Equal(CommandExitCodes.DatabaseError, exitCode); Assert.Equal("error", json.GetProperty("status").GetString()); - Assert.Contains("not an existing CodeIndex DB", json.GetProperty("message").GetString()); + Assert.Equal(CommandErrorCodes.DbNotDatabase, json.GetProperty("error_code").GetString()); + Assert.Equal("database_not_a_database", json.GetProperty("category").GetString()); + Assert.Contains("not a valid SQLite CodeIndex database", json.GetProperty("message").GetString()); } finally { @@ -6704,7 +6790,9 @@ public void RunBackfillFold_NonexistentFileUri_ReturnsNotFound() Assert.Equal(CommandExitCodes.NotFound, exitCode); Assert.Equal("error", json.GetProperty("status").GetString()); - Assert.Contains("database not found", json.GetProperty("message").GetString()); + Assert.Equal(CommandErrorCodes.DbNotFound, json.GetProperty("error_code").GetString()); + Assert.Equal("database_missing", json.GetProperty("category").GetString()); + Assert.Contains("database file was not found", json.GetProperty("message").GetString()); } [Fact] diff --git a/tests/CodeIndex.Tests/LegacySchemaMigrationTests.cs b/tests/CodeIndex.Tests/LegacySchemaMigrationTests.cs index d6b5ecfdb..2b070d9fd 100644 --- a/tests/CodeIndex.Tests/LegacySchemaMigrationTests.cs +++ b/tests/CodeIndex.Tests/LegacySchemaMigrationTests.cs @@ -1049,6 +1049,35 @@ public void TryValidateExistingCodeIndexDb_ClassifiesCantOpenCausesWithoutLeakin } } + [Fact] + public void TryValidateExistingCodeIndexDb_PreservesRetryExhaustionForMaintenanceClassifier_Issue4856() + { + var valid = DbContext.TryValidateExistingCodeIndexDb( + _dbPath, + _ => new SqliteConnection("Data Source=:memory:"), + _ => throw CreateTransientBusyException(), + sleep: _ => { }, + out _, + out var isNotFound, + out var validationException); + + Assert.False(valid); + Assert.False(isNotFound); + var structured = Assert.IsType(validationException); + Assert.Equal(CommandErrorCodes.DbLocked, structured.Code); + + var error = MaintenanceDatabaseErrorClassifier.FromValidation( + "vacuum", + _dbPath, + showPaths: false, + ExistingCodeIndexDbValidationFailure.Exception, + validationException); + Assert.Equal(CommandErrorCodes.DbLocked, error.ErrorCode); + Assert.Equal("database_locked", error.Category); + Assert.Equal(CommandExitCodes.TransientDatabaseError, error.ExitCode); + Assert.Equal(5, error.SqliteErrorCode); + } + private static SqliteException CreateTransientBusyException() => CreateSqliteException("busy", 5); diff --git a/tests/CodeIndex.Tests/QueryCommandRunnerTests.cs b/tests/CodeIndex.Tests/QueryCommandRunnerTests.cs index dfccf2cd7..11a7055be 100644 --- a/tests/CodeIndex.Tests/QueryCommandRunnerTests.cs +++ b/tests/CodeIndex.Tests/QueryCommandRunnerTests.cs @@ -729,7 +729,90 @@ public void RunVacuum_RejectsMissingDatabase_Issue1631() Assert.Equal(CommandExitCodes.NotFound, exitCode); Assert.Equal(string.Empty, stdout); Assert.Contains(CommandErrorCodes.DbNotFound, stderr); + Assert.Contains("database_missing", stderr); + Assert.Contains("", stderr); + Assert.DoesNotContain(project.Root, stderr, StringComparison.Ordinal); Assert.False(File.Exists(dbPath)); + + var (jsonExitCode, jsonStdout, jsonStderr) = CaptureConsole(() => QueryCommandRunner.RunVacuum( + ["--db", dbPath, "--json"], + _jsonOptions)); + + Assert.Equal(CommandExitCodes.NotFound, jsonExitCode); + Assert.Equal(string.Empty, jsonStderr); + using (var document = ParseJsonOutput(jsonStdout)) + { + var json = document.RootElement; + Assert.Equal("error", json.GetProperty("status").GetString()); + Assert.Equal(CommandErrorCodes.DbNotFound, json.GetProperty("error_code").GetString()); + Assert.Equal("database_missing", json.GetProperty("category").GetString()); + Assert.Equal("1", json.GetProperty("database_error_classifier_version").GetString()); + Assert.Equal("", json.GetProperty("path").GetString()); + Assert.True(json.GetProperty("path_redacted").GetBoolean()); + } + + var (diagnosticExitCode, diagnosticStdout, diagnosticStderr) = CaptureConsole(() => QueryCommandRunner.RunVacuum( + ["--db", dbPath, "--json", "--show-paths"], + _jsonOptions)); + + Assert.Equal(CommandExitCodes.NotFound, diagnosticExitCode); + Assert.Equal(string.Empty, diagnosticStderr); + using (var document = ParseJsonOutput(diagnosticStdout)) + { + Assert.Equal(dbPath, document.RootElement.GetProperty("path").GetString()); + Assert.False(document.RootElement.GetProperty("path_redacted").GetBoolean()); + } + } + + [Fact] + public void RunVacuum_NewerSchemaJson_PreservesStructuredClassification_Issue4856() + { + using var project = TestProjectHelper.CreateTempProjectScope("cdidx_vacuum_newer_schema"); + var dbPath = TestProjectHelper.CreateProjectDb(project.Root); + using (var connection = new SqliteConnection( + new SqliteConnectionStringBuilder { DataSource = dbPath }.ConnectionString)) + { + connection.Open(); + using var command = connection.CreateCommand(); + command.CommandText = $"PRAGMA user_version = {DbContext.CurrentSchemaVersion | (DbContext.CurrentSchemaVersion + 1)};"; + command.ExecuteNonQuery(); + } + + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunVacuum( + ["--db", dbPath, "--json"], + _jsonOptions)); + + Assert.Equal(CommandExitCodes.DatabaseError, exitCode); + Assert.Equal(string.Empty, stderr); + using var document = ParseJsonOutput(stdout); + var json = document.RootElement; + Assert.Equal(CommandErrorCodes.SchemaTooNew, json.GetProperty("error_code").GetString()); + Assert.Equal("database_schema_too_new", json.GetProperty("category").GetString()); + Assert.Equal("", json.GetProperty("path").GetString()); + Assert.DoesNotContain(project.Root, stdout, StringComparison.Ordinal); + } + + [Fact] + public void RunVacuum_DirectoryTargetJson_ReportsInaccessibleWithoutRebuildAdvice_Issue4856() + { + using var project = TestProjectHelper.CreateTempProjectScope("cdidx_vacuum_directory_target"); + var dbUri = new Uri(project.Root).AbsoluteUri; + + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunVacuum( + ["--db", dbUri, "--json"], + _jsonOptions)); + + Assert.Equal(CommandExitCodes.DatabaseError, exitCode); + Assert.Equal(string.Empty, stderr); + using var document = ParseJsonOutput(stdout); + var json = document.RootElement; + Assert.Equal(CommandErrorCodes.DbError, json.GetProperty("error_code").GetString()); + Assert.Equal("database_inaccessible", json.GetProperty("category").GetString()); + Assert.Equal("", json.GetProperty("path").GetString()); + Assert.True(json.GetProperty("path_redacted").GetBoolean()); + Assert.Contains("readable regular database file", json.GetProperty("hint").GetString()); + Assert.DoesNotContain("--rebuild", json.GetProperty("hint").GetString(), StringComparison.Ordinal); + Assert.DoesNotContain(project.Root, stdout, StringComparison.Ordinal); } [Fact] @@ -751,8 +834,30 @@ public void RunVacuum_RejectsNonCodeIndexDatabase_Issue1631() Assert.Equal(CommandExitCodes.DatabaseError, exitCode); Assert.Equal(string.Empty, stdout); - Assert.Contains(CommandErrorCodes.DbError, stderr); - Assert.Contains("not an existing CodeIndex DB", stderr); + Assert.Contains(CommandErrorCodes.DbNotDatabase, stderr); + Assert.Contains("not a valid SQLite CodeIndex database", stderr); + } + + [Fact] + public void RunVacuum_InvalidHeaderJson_ReportsNotDatabaseWithoutLeakingAbsolutePath_Issue4856() + { + using var project = TestProjectHelper.CreateTempProjectScope("cdidx_vacuum_invalid_header"); + var dbPath = Path.Combine(project.Root, "invalid-header.db"); + File.WriteAllText(dbPath, "this is not sqlite"); + + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunVacuum( + ["--db", dbPath, "--json"], + _jsonOptions)); + + Assert.Equal(CommandExitCodes.DatabaseError, exitCode); + Assert.Equal(string.Empty, stderr); + using var document = ParseJsonOutput(stdout); + var json = document.RootElement; + Assert.Equal(CommandErrorCodes.DbNotDatabase, json.GetProperty("error_code").GetString()); + Assert.Equal("database_not_a_database", json.GetProperty("category").GetString()); + Assert.Equal(26, json.GetProperty("sqlite_error_code").GetInt32()); + Assert.Equal("", json.GetProperty("path").GetString()); + Assert.DoesNotContain(project.Root, stdout, StringComparison.Ordinal); } [Fact] @@ -777,8 +882,8 @@ public void RunVacuum_RejectsLookalikeNonCodeIndexDatabase_Issue1631() Assert.Equal(CommandExitCodes.DatabaseError, exitCode); Assert.Equal(string.Empty, stdout); - Assert.Contains(CommandErrorCodes.DbError, stderr); - Assert.Contains("not an existing CodeIndex DB", stderr); + Assert.Contains(CommandErrorCodes.DbNotDatabase, stderr); + Assert.Contains("not a valid SQLite CodeIndex database", stderr); } [Fact] @@ -794,8 +899,8 @@ public void RunVacuum_RejectsReadOnlyUriWithNeutralWritableMessage_Issue1631() Assert.Equal(CommandExitCodes.DatabaseError, exitCode); Assert.Equal(string.Empty, stdout); - Assert.Contains(CommandErrorCodes.DbError, stderr); - Assert.Contains("database must be writable", stderr); + Assert.Contains(CommandErrorCodes.DbNotWritable, stderr); + Assert.Contains("database is not writable", stderr); Assert.DoesNotContain("backfill-fold", stderr); }