diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index adbbc6c5b..725209698 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -44,6 +44,17 @@ metadata, unknown-extension counts, filesystem case-sensitivity, MAC profile, and DB/WAL/status diagnostics. These stamps let readers distinguish a feature that is absent, stale, or newer than the running binary. +The canonical C# partial-declaration change in #4914 raises +`hotspot_family_version_csharp` from `2` to `14` and the reference-identity +contract from `6` to `8`. The minimum compatible implementation is therefore a +binary that understands hotspot-family contract `14` and reference-identity +contract `8`. Existing databases remain readable, but their C# family and +reference-identity data is reported as stale until a rebuild with +`cdidx index --rebuild` refreshes the persisted partial metadata +and reference candidates. +Older binaries treat those newer stamps as forward-version data and must retain +their normal query degradation and write-refusal behavior. + Reference-extraction cap hits use existing per-file `file_issues` rows rather than a new schema bit. Current readers aggregate those rows into `reference_extraction_cap_hits` and set `reference_graph_complete=false`; a @@ -121,6 +132,16 @@ metadata、unknown-extension count、filesystem case-sensitivity、MAC profile DB/WAL/status diagnostics が含まれます。reader はこれらの stamp により、feature が存在しないのか、stale なのか、実行中 binary より新しいのかを判別できます。 +#4914 の canonical C# partial declaration 対応では、 +`hotspot_family_version_csharp` を `2` から `14` へ、reference identity contract +を `6` から `8` へ更新します。したがって最低互換実装は hotspot-family contract +`14` と reference-identity contract `8` を理解する binary です。既存 database は +引き続き読み取り可能ですが、`cdidx index --rebuild` で永続 partial +metadata と reference candidate を更新するまでは、C# family / reference identity +data が stale として報告されます。古い binary はこれらの新しい stamp を +forward-version data として扱い、通常の query degradation と write refusal を +維持しなければなりません。 + reference-extraction cap hit は新しい schema bit ではなく、既存の file ごとの `file_issues` row を使います。current reader はそれを `reference_extraction_cap_hits` へ集約して `reference_graph_complete=false` にします。issue state を確認できない diff --git a/TESTING_GUIDE.md b/TESTING_GUIDE.md index d432821fc..6e809b867 100644 --- a/TESTING_GUIDE.md +++ b/TESTING_GUIDE.md @@ -562,6 +562,8 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding indexes one real C# fixture containing user-defined generic classes and delegates, block-comment generic trivia, repeated same-line generic names under indentation, unrelated same-name properties, ordinal-case type/property collisions, `Action` / `Stream` references, and uppercase property receivers split across partial-class and inherited-base files. Keep CLI `references`, `inspect`, and `deps` assertions together with the MCP `references`, `analyze_symbol`, and `deps` wrappers so unresolved diagnostics, generic-arity selection, case-sensitive receiver handling, false dependency-edge suppression, and preservation of real member dependencies cannot drift across surfaces. - `QueryCommandRunnerTests.ConstructorCommands_KeepCallableAndPartialTypeResultsSeparate_Issue4850` keeps explicit constructors split across partial declarations in one fixture. Preserve the separate `definition` / `goto --kind function` assertions and the grouped `--kind class --group-partials` assertion so callable constructor results cannot merge back into logical type-family results. +- `QueryCommandRunnerTests.PartialCanonicalRepresentative_UsesSemanticRulesAndExposesFamilyNavigation_Issue4914`, `PartialCanonicalRepresentative_GroupsSplitModifierAndRanksLeadingEvidence_Issue4914`, `PartialCanonicalRepresentative_PersistsSplitModifierAndLeadingSemanticEvidence_Issue4914`, `PartialCanonicalRepresentative_RespectsFileLocalAndLexedEvidence_Issue4914`, `PartialCanonicalRepresentative_HonorsCaseSensitiveProjectMarkerCasing_Issue4914Review`, `GroupPartials_DegradesSafelyWhenCSharpFamilyContractIsStale_Issue4914`, and `PartialCanonicalRepresentative_FallsBackToDesignerPathForOldDatabaseMetadata_Issue4914` + keep canonical partial-family ranking and navigation in one contract. Preserve partial classes, records (stored under the existing C# class kind) whose base lists repeat the declared name, file-local partial hosts that group only within one source file, partial types and containing generic types with distinct arity, case-sensitive project-marker casing when resolving nullable custom types, and declaration/implementation partial methods whose parameter attributes, names, quoted or commented defaults, declaration/type-position comments, comments between the identifier/generic parameters/parameter list, predefined and `dynamic` / `object` aliases, nullable value-type equivalents, predefined reference-type nullable annotations, explicitly global-rooted framework aliases with verbatim segments, shadowable unrooted `System` types, meaningful global-root qualification, qualified type leaves that collide with method generic-parameter names, verbatim method or type escapes, or user-type casing differ; lexer-confirmed identifier columns and semantic evidence when same-name attribute calls or documentation-like block-comment decoys precede the declaration; preceding-line `partial` modifiers across blank or C# comment trivia, modifiers after a balanced same-line attribute list, attributes bound only to their same-line declaration occurrence, repeated same-line declarations with distinct navigation columns, and blank-detached XML documentation that must not rank across extraction-worker transport, post-extraction hook cloning, and DB persistence; conservative physical-row fallback for stale family metadata whose arity-less keys would otherwise collide, while the Issue4850 LSP constructor fixture still distinguishes stale-index partial type targets; the non-C# function-column fallback; generated metadata and designer-name fallback; reversed insertion order and rebuild-stable family IDs; a semantic primary that remains primary after a path rename; normalized identifier-aligned member columns and normalized verbatim-identifier navigation ranges; a bounded family list that retains a late-sorting representative; grouped `definition` metadata; canonical `goto`; and physical `goto --all`. Keep the legacy-column fixture separate so schema initialization cannot silently restore the metadata being tested as absent. - `LspServerTests.cs` LSP JSON-RPC behavior and indexed editor semantics. Reference parity coverage must compare LSP locations with the matching CLI candidate bundle, including overload identity and both `includeDeclaration` states. Keep document/workspace identifier ranges and explicit-versus-inferred inlay hints in one source-semantics fixture so persisted-column anchoring, source confirmation, and hint suppression cannot drift independently. The issue-4870 kind-mapping matrix seeds every registered built-in internal kind plus semantic constructor and enum-member rows, then asserts decoded document-symbol, workspace-symbol, and completion responses. It also feeds real TypeScript, Kotlin, Solidity, Java, Swift, Dart, Visual Basic, Pascal, C#, and Shell extractor output through the shared mapper so keyword, compact, named, dedicated-name, and verbatim-identifier constructors; same-name, return-typed, object/static, and shell non-constructors; decorated, escaped, indirect, and comma-grouped enum entries; and nested enums cannot invert classification. It separately pins the conservative parameter and unknown/plugin fallback. Keep all three protocol surfaces in that one fixture so constructor/import handling and deliberate fallback mappings cannot drift. @@ -1518,6 +1520,8 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests" は user-defined generic class / delegate、generic 間の block comment trivia、indent 下で同一行に反復する同名 generic、無関係な同名 property、大文字小文字だけ異なる type / property collision、`Action` / `Stream` reference、partial class の別ファイルおよび継承 base file に分かれた大文字始まりの property receiver を含む実 C# fixture を1回 index します。CLI の `references`、`inspect`、`deps` と MCP wrapper の `references`、`analyze_symbol`、`deps` を同じ fixture で検証し、unresolved 診断、generic arity による選択、case-sensitive receiver 判定、偽 dependency edge の抑制、実在する member dependency の維持が surface 間でずれないよう維持してください。 - `QueryCommandRunnerTests.ConstructorCommands_KeepCallableAndPartialTypeResultsSeparate_Issue4850` は partial declaration の別ファイルに分かれた明示 constructor を 1 つの fixture にまとめます。callable constructor result が logical type-family result に再び混在しないよう、`definition` / `goto --kind function` の検証と、`--kind class --group-partials` による grouped type の検証を分離したまま維持してください。 +- `QueryCommandRunnerTests.PartialCanonicalRepresentative_UsesSemanticRulesAndExposesFamilyNavigation_Issue4914`、`PartialCanonicalRepresentative_GroupsSplitModifierAndRanksLeadingEvidence_Issue4914`、`PartialCanonicalRepresentative_PersistsSplitModifierAndLeadingSemanticEvidence_Issue4914`、`PartialCanonicalRepresentative_RespectsFileLocalAndLexedEvidence_Issue4914`、`GroupPartials_DegradesSafelyWhenCSharpFamilyContractIsStale_Issue4914`、`PartialCanonicalRepresentative_FallsBackToDesignerPathForOldDatabaseMetadata_Issue4914` + は canonical な partial-family ranking と navigation を1つの契約として維持します。partial class、base list で宣言名が再登場する record（既存の C# class kind として保存）、同一 source file 内だけで集約する file-local partial host、arity が異なる partial type 自身と外側 generic type、parameter attribute・名前・quote または comment を含む default・declaration / type 内の comment・identifier / generic parameter / parameter list 間の comment・predefined alias と `dynamic` / `object` alias・nullable value type の同値表記・predefined reference type の nullable annotation・verbatim segment を含む明示的に global-rooted な framework alias・shadow 可能な root のない `System` type・意味のある global-root 修飾・method generic parameter 名と衝突する qualified type leaf・method または type の verbatim escape・user type の大文字小文字が異なる宣言 / 実装の partial method、同名 attribute call や documentation 風 block-comment decoy が宣言より前にある場合も lexer で確認した identifier column と semantic evidence を使うこと、C# comment を含む前の行へ分割された `partial` 修飾子、balanced な同一行 attribute list 後の modifier、正しい同一行 declaration occurrence だけに属する attribute、別々の navigation column を持つ同一行の反復宣言、先行 XML documentation が extraction-worker 搬送、post-extraction hook の clone、DB 保存を越えて残ること、blank-detached XML documentation は rank しないこと、arity のない key が衝突する stale family metadata では物理 row へ保守的に fallback しつつ Issue4850 の LSP constructor fixture では stale-index の partial type target を区別すること、non-C# function の column fallback、generated metadata と designer filename fallback、挿入順を逆転した再構築でも安定する family ID、path rename 後も primary であり続ける semantic declaration、正規化後の identifier に揃った member column と正規化後の verbatim identifier に揃った navigation range、遅い path の representative を保持する上限付き family list、grouped `definition` metadata、canonical `goto`、物理位置を返す `goto --all` を維持してください。欠落 metadata が schema initialization によって暗黙に復元されないよう、legacy-column fixture は独立したままにしてください。 - `LspServerTests.cs` LSP の JSON-RPC 挙動と indexed editor semantics のテスト。reference parity coverage では、overload identity と `includeDeclaration` の両状態を含め、LSP location と対応する CLI candidate bundle を比較してください。document/workspace の identifier range と explicit/inferred inlay hint は 1 つの source-semantics fixture にまとめ、保存済み column の anchoring、source 上の確認、hint 抑制が別々に drift しないようにします。 issue-4870 の kind-mapping matrix は、登録済みの全 built-in internal kind に加えて semantic な constructor と enum-member の row を seed し、decode 済みの document-symbol、workspace-symbol、completion response を検証します。また、実際の TypeScript、Kotlin、Solidity、Java、Swift、Dart、Visual Basic、Pascal、C#、Shell extractor 出力を shared mapper に通し、keyword / compact / named / 専用名 / 逐語識別子 constructor、同名 / 戻り値型付き / object・static / shell の非 constructor、annotation 付き / escape 済み / indirect / comma-grouped enum entry、nested enum の分類が逆転しないようにします。保守的な parameter および未知/plugin fallback は別途固定します。constructor/import の処理と意図的な fallback mapping がずれないよう、3つの protocol surface を同じ fixture に維持してください。 diff --git a/USER_GUIDE.md b/USER_GUIDE.md index b69e9a0a5..a728c6562 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -1847,7 +1847,9 @@ cdidx definition QueryCommandRunner --exact-name --group-partials --count --json `definition` uses indexed symbol ranges plus chunk reconstruction to return the actual declaration text, and optional body content when the language extractor can infer a body range. -For C# partial types, add `--group-partials` to `definition`, `symbols`, or symbol-mode `inspect` to collapse actual `partial` declarations with the same persisted, qualified family identity into one logical family. The default remains one row per physical declaration, and unrelated non-partial or merely same-named types—including nested non-partial types inside a partial host—are never collapsed. Grouped rows use a stable path/line representative and report the family's physical declaration count in `definition_sites`; grouped count JSON reports both `logical_count` and `physical_count` (plus `physical_file_count`). Human summaries distinguish the logical rows shown after `--limit` from query-wide logical and physical totals. Audit-sorted `symbols` rows use the family's maximum rank metric while retaining that stable representative, so `--sort` remains monotonic before `--limit` is applied. `impact` uses the same family key automatically, reports `logical_definition_count`, and counts every matching physical site while materializing only the bounded logical representatives needed for output. File-mode `inspect`, whether selected by a positional path or `--path ... --line ...`, remains a physical lookup and rejects `--group-partials`. +For C# partial types and partial methods, add `--group-partials` to `definition`, `symbols`, or symbol-mode `inspect` to collapse actual `partial` declarations with the same persisted, qualified family identity into one logical family. The default remains one row per physical declaration, and unrelated non-partial or merely same-named types—including nested non-partial types inside a partial host—are never collapsed. A `file partial` type and its partial members remain scoped to their source file, so matching file-local declarations in different files form distinct families. Family identity preserves a partial type's own generic arity, each containing type's generic arity, user-type casing, and meaningful `global::` root qualification, while normalizing C# predefined aliases (including the `dynamic` / `object` runtime identity), explicitly global-rooted `System` predefined aliases (including verbatim `@System` / `@Int32` segments), nullable value-type equivalents such as `int?` / `global::System.Nullable`, predefined reference-type nullable annotations, verbatim identifier escapes, declaration/type-position comments, parameter attributes and names, default values, and comment trivia between a method identifier, its generic parameters, and its parameter list. Method type parameters normalize by ordinal only when used as unqualified type variables; qualified leaves such as `N.T` remain concrete types even when the method declares ``. Unrooted spellings such as `System.Int32` remain distinct from `int` because an enclosing namespace or `using` alias can shadow `System`. Extraction-owned declaration metadata survives post-extraction hook cloning and preserves a `partial` modifier split onto preceding modifier-only lines across blank or comment trivia, including modifiers that trail a balanced leading attribute list. Leading attributes are bound to the declaration occurrence they prefix when multiple declarations share one line, while only adjacent lexer-confirmed XML documentation outside the stored signature contributes semantic rank; documentation-like text inside block comments or strings, and XML documentation detached by a blank line, does not affect representative rank. Repeated same-name partial declarations on one line retain distinct identifier columns for family navigation. The family-key contract is versioned so older C# rows are not interpreted with current grouping rules: a missing or stale contract conservatively returns physical rows until a full reindex republishes current family metadata, while LSP position resolution may still reconstruct a partial-type identity locally to keep type and constructor targets separate without collapsing query output. The canonical representative is chosen deterministically from the matched family: an implementation-bearing partial method precedes a declaration-only method, non-generated source precedes generated/designer source, declarations whose extraction metadata records leading attributes or XML documentation—or whose lexed indexed signature retains attributes, base lists, or constraints—precede otherwise equivalent declarations, and comment-insensitive normalized declaration identity is considered before ordinal path and source position. Generated sites participate when `--include-generated` is set; legacy databases without generated-file metadata fall back to generated/designer filename conventions. + +Grouped structured rows report the physical declaration count in `definition_sites` and expose `partial_family_id`, `representative_reason`, and up to 50 stable `family_members`; the bounded list always retains the representative and uses normalized identifier-aligned columns (after a verbatim `@` escape), and `family_members_truncated` is true when more sites exist. `goto` uses the same canonical representative by default and includes that family metadata in its LSP-shaped JSON, while `goto --all` intentionally returns every matching physical location. Grouped count JSON reports both `logical_count` and `physical_count` (plus `physical_file_count`). Human summaries distinguish the logical rows shown after `--limit` from query-wide logical and physical totals. Audit-sorted `symbols` rows use the family's maximum rank metric while retaining the canonical representative, so `--sort` remains monotonic before `--limit` is applied. `impact` uses the same family key and representative ordering automatically, reports `logical_definition_count`, and counts every matching physical site while materializing only the bounded logical representatives needed for output. File-mode `inspect`, whether selected by a positional path or `--path ... --line ...`, remains a physical lookup and rejects `--group-partials`. ### Inspect one symbol in one round-trip @@ -5238,7 +5240,9 @@ cdidx definition QueryCommandRunner --exact-name --group-partials --count --json `definition` は、インデックス済みシンボル範囲とチャンク再構成を使って実際の宣言テキストを返します。言語抽出器が本体範囲を推論できる場合は、`--body` で本体内容も返します。 -C# の partial type では、`definition`、`symbols`、または symbol mode の `inspect` に `--group-partials` を付けると、persist 済みの qualified family identity が同じ実際の `partial` 宣言を1つの論理 family に集約できます。既定は従来どおり物理宣言ごとに1行で、無関係な non-partial type、単に同名の type、partial host 内のネストした non-partial type は集約しません。集約行は安定した path / line の代表を使い、family 内の物理宣言数を `definition_sites` で返します。grouped count JSON は `logical_count` と `physical_count`（および `physical_file_count`）を併記し、human summary は `--limit` 適用後に表示した論理行数と query 全体の論理・物理総数を区別します。audit sort を使う `symbols` は family 内の rank metric の最大値で並べつつ安定した代表を維持するため、`--limit` 適用前の `--sort` 順序も単調です。`impact` は同じ family key を自動的に使い、`logical_definition_count` を返し、出力に必要な上限付き論理代表だけを materialize しながら一致した全物理 site を数えます。positional path または `--path ... --line ...` で選ぶ file mode の `inspect` は物理位置の lookup のままで、`--group-partials` を拒否します。 +C# の partial type と partial method では、`definition`、`symbols`、または symbol mode の `inspect` に `--group-partials` を付けると、persist 済みの qualified family identity が同じ実際の `partial` 宣言を1つの論理 family に集約できます。既定は従来どおり物理宣言ごとに1行で、無関係な non-partial type、単に同名の type、partial host 内のネストした non-partial type は集約しません。`file partial` type とその partial member は source file 内に限定されるため、別ファイルにある同名の file-local 宣言は別 family になります。family identity は partial type 自身と各外側 generic type の arity、user type の大文字小文字、意味のある `global::` root 修飾を保持しつつ、`dynamic` / `object` の runtime identity を含む C# predefined alias、明示的に global root を持つ `System` predefined alias（verbatim な `@System` / `@Int32` segment を含む）、`int?` / `global::System.Nullable` のような nullable value type の同値表記、predefined reference type の nullable annotation、verbatim identifier escape、declaration / type 内の comment、parameter attribute・名前・default value、method identifier・generic parameter・parameter list の間にある comment trivia を正規化します。method type parameter は unqualified な type variable として使われた場合だけ ordinal で正規化し、method が `` を宣言していても `N.T` のような qualified leaf は実型として保持します。外側 namespace や `using` alias が `System` を shadow できるため、root のない `System.Int32` は `int` と区別します。抽出器が所有する declaration metadata は post-extraction hook の clone 後も維持され、空行や comment trivia をまたいで modifier-only 行へ分割された `partial` 修飾子を保持します。先行 attribute list の閉じ括弧に続く modifier も認識します。同一行に複数宣言がある場合、先行 attribute は直後の declaration occurrence だけに関連付け、保存済み signature の外側にある lexer 確認済み XML documentation は空行を挟まず隣接する場合だけ semantic rank に使います。block comment・string 内の documentation 風 text や、空行で宣言から切り離された XML documentation は representative rank に影響しません。同一行で反復する同名 partial 宣言は、family navigation 用に別々の identifier column を保持します。family-key 契約は version 管理され、旧 C# row を現行 grouping rule で解釈しません。契約が未登録または stale の場合は、full reindex が現行 family metadata を再公開するまで物理 row を保守的に返します。ただし LSP の位置解決は query 出力を集約せず、type と constructor の target を分離するためだけに partial-type identity を局所的に再構築できます。一致した family 内の canonical representative は決定的に選ばれます。本体を持つ partial method は宣言だけの method より先、非生成 source は generated / designer source より先、抽出 metadata が先行 attribute / XML documentation を記録した宣言、または lexer で解析した indexed signature に attribute・base list・constraint が保持された宣言は、それ以外が同等の宣言より先となり、その後に comment を無視して正規化した declaration identity、ordinal path、source position を使います。generated site は `--include-generated` 指定時に候補へ入り、generated-file metadata を持たない旧 database では generated / designer の filename 規約へ fallback します。 + +集約した structured row は、family 内の物理宣言数を `definition_sites` で返し、`partial_family_id`、`representative_reason`、安定順で最大50件の `family_members` を公開します。上限付き list には必ず representative が残り、column は正規化後の identifier（verbatim escape の `@` より後ろ）に揃います。site がさらにある場合は `family_members_truncated` が true になります。`goto` は既定で同じ canonical representative を使い、その family metadata を LSP 形式の JSON に含めます。全物理 location を意図的に取得する場合は `goto --all` を使います。grouped count JSON は `logical_count` と `physical_count`（および `physical_file_count`）を併記し、human summary は `--limit` 適用後に表示した論理行数と query 全体の論理・物理総数を区別します。audit sort を使う `symbols` は family 内の rank metric の最大値で並べつつ canonical representative を維持するため、`--limit` 適用前の `--sort` 順序も単調です。`impact` は同じ family key と代表順を自動的に使い、`logical_definition_count` を返し、出力に必要な上限付き論理代表だけを materialize しながら一致した全物理 site を数えます。positional path または `--path ... --line ...` で選ぶ file mode の `inspect` は物理位置の lookup のままで、`--group-partials` を拒否します。 ### 1往復でシンボルを精査する diff --git a/changelog.d/unreleased/4914.changed.md b/changelog.d/unreleased/4914.changed.md new file mode 100644 index 000000000..763f679f6 --- /dev/null +++ b/changelog.d/unreleased/4914.changed.md @@ -0,0 +1,139 @@ +--- +category: changed +issues: + - 4914 +affected: + - src/CodeIndex/Cli/IndexCommandRunner.FullScan.ExtractionWorkers.cs + - src/CodeIndex/Cli/IndexCommandRunner.FullScan.FilePersistence.cs + - src/CodeIndex/Cli/IndexCommandRunner.FullScan.Readiness.cs + - src/CodeIndex/Cli/IndexCommandRunner.FullScan.cs + - src/CodeIndex/Cli/IndexCommandRunner.Update.FilePersistence.cs + - src/CodeIndex/Cli/IndexCommandRunner.Update.Readiness.cs + - src/CodeIndex/Cli/IndexCommandRunner.Update.cs + - src/CodeIndex/Cli/IndexCommandRunner.WorkItems.cs + - src/CodeIndex/Cli/CliFlagSchema.cs + - src/CodeIndex/Cli/DiffCommandRunner.cs + - src/CodeIndex/Cli/IndexCommandRunner.DryRun.cs + - src/CodeIndex/Cli/ProjectionFieldRegistry.cs + - src/CodeIndex/Cli/QueryCommandRunner.Definition.cs + - src/CodeIndex/Cli/QueryCommandRunner.Locations.cs + - src/CodeIndex/Cli/ReportCommandRunner.cs + - src/CodeIndex/Database/DbContext.ConnectionFunctions.cs + - src/CodeIndex/Database/DbContext.ReadMigrations.cs + - src/CodeIndex/Database/DbContext.SchemaInitialization.cs + - src/CodeIndex/Database/DbContext.SchemaMetadata.cs + - src/CodeIndex/Database/DbContext.cs + - src/CodeIndex/Database/CSharpCallableTypeKindLookup.cs + - src/CodeIndex/Database/DbReader.cs + - src/CodeIndex/Database/DbReader.GraphQueries.cs + - src/CodeIndex/Database/DbReader.References.cs + - src/CodeIndex/Database/DbSymbolReader.Definitions.cs + - src/CodeIndex/Database/DbSymbolReader.Search.cs + - src/CodeIndex/Database/DbWriter.ChunkSymbolBatches.cs + - src/CodeIndex/Database/DbWriter.ContractMetadata.cs + - src/CodeIndex/Database/DbWriter.Files.cs + - src/CodeIndex/Database/DbWriter.FoldBackfill.cs + - src/CodeIndex/Database/DbWriter.References.cs + - src/CodeIndex/Database/DbWriter.cs + - src/CodeIndex/Database/LogicalPartialSymbolGrouper.cs + - src/CodeIndex/Indexer/Scanning/FileIndexer.GeneratedCode.cs + - src/CodeIndex/Indexer/Scanning/FileIndexer.ProjectMarkers.cs + - src/CodeIndex/Indexer/Hooks/PostExtractionHookMutationMaterializer.cs + - src/CodeIndex/Indexer/Hooks/PostExtractionHooks.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.ContainerAssignment.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpLexing.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpPartialMetadata.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractCore.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs + - src/CodeIndex/Indexer/CSharpTypeReferenceArity.cs + - src/CodeIndex/Lsp/LspServer.PositionResolution.cs + - src/CodeIndex/Mcp/McpToolHandlers.Indexing.Execution.cs + - src/CodeIndex/Models/QueryResults.cs + - src/CodeIndex/Models/SymbolRecord.cs + - tests/CodeIndex.HookIsolationFixture/HookIsolationFixture.cs + - tests/CodeIndex.Tests/DbReaderTests.cs + - tests/CodeIndex.Tests/PostExtractionHookTests.cs + - tests/CodeIndex.Tests/QueryCommandRunnerPartialGroupingIssue4566Tests.cs + - tests/CodeIndex.Tests/QueryCommandRunnerSymbolTests.cs + - tests/CodeIndex.Tests/QueryCommandRunnerTests.cs + - tests/CodeIndex.Tests/ReportCommandRunnerTests.cs + - tests/CodeIndex.Tests/SymbolExtractorTests.cs + - tests/CodeIndex.Tests/JsonEnvelopeWrapperIssue4585Tests.cs + - tests/CodeIndex.Tests/PostExtractionHookContractTests.cs + - tests/CodeIndex.Tests/IndexCommandRunnerUpdateTests.cs + - USER_GUIDE.md + - TESTING_GUIDE.md + - COMPATIBILITY.md +--- + +## English + +- **Partial declaration families now choose and explain one canonical representative (#4914)** — grouped C# partial classes, records, and methods—including implementations classified as `test.method` by an attached test attribute—prefer implementations, non-generated source, and semantically richer declarations before stable declaration identity and source location; nested partial types retain their own names and generic arities instead of inheriting one container family identity; file-local partial hosts and their nested descendants stay scoped to one source file across grouped queries and hotspots even when only one same-file host part carries the `file` modifier; plain-record arity is resolved from the lexer-confirmed declaration rather than an earlier same-name attribute argument, explicit `record class` / `record struct` introducers retain their `partial` modifier, and readiness treats same-leaf C# types of different arities as separate families without changing the VB, F#, or MSBuild family-contract versions, including support reports that treat indexes without a family-aware language as ready, fall back to a legacy global stamp for family-aware languages actually present in an index, and reject missing or incomplete marker fingerprints on per-language stamps; generic-constraint colons do not count as base lists for semantic representative rank; extraction metadata survives post-extraction hook cloning, preserves synthesized positional-record component containment across successful no-op hooks, selects same-name generic containers by source range after explicit hook mutations, and retains preceding-line `partial` modifiers across blank or comment trivia, modifiers after balanced leading attribute lists, attributes bound to the correct same-line declaration occurrence, adjacent lexer-confirmed C# XML documentation including blank lines inside delimited documentation and delimited documentation on the declaration line, and exact declaration-name columns for records, repeated same-line declarations, normalized verbatim identifiers, and impact results for test methods while excluding blank-detached documentation, `assembly:` / `module:` global attributes, block-comment, string, and same-line declaration-body decoys from modifiers, semantic rank, declaration identity, and generic constraints; only declaration-modifier positions count as `partial`, so parameter or constraint identifiers with that spelling remain physical non-partial declarations; callable families require comment-aware persisted partial evidence, and their identities accept comment trivia between the method identifier, generic parameters, and parameter list, comment trivia inside generic optional-default member access, plus Unicode combining marks in parameter identifiers, normalize explicitly global-rooted predefined aliases, their verbatim segments, `S?` / `global::System.Nullable` value-type spellings, nullable explicitly rooted `System.ValueTuple` spellings, predefined reference-type nullable annotations, nullable annotations on arity-aware nested and file-scoped indexed custom reference types, constraints distributed across declarations of the same generic partial container, and nullable annotations on unconstrained or reference-constrained method type parameters while indexed custom value types and value-constrained method type parameters retain their nullable distinction, preserving project scope when same-qualified custom types have different kinds in separate C# projects, verbatim `@global` aliases, shadowable unrooted `System` types, qualified type leaves that match method generic-parameter names, and unresolved import-scoped leaves that merely collide with unrelated indexed types; custom type facts are loaded only for the bounded callable candidates of partial grouping and refresh after external index commits; database diffs include the new partial/ranking/location metadata with legacy-column fallbacks; stale family contracts conservatively preserve authoritative physical rows until reindex while LSP type/constructor discrimination remains available; structured results expose a stable family ID, selection reason, and bounded members; and `goto` returns that representative by default while `goto --all` retains every physical location, including deterministic fallback behavior for legacy generated-file metadata. + +- **Large ordinary method sets no longer trigger full partial-callable scans (#4914)** — candidate and fallback scans now retain only callables confirmed as partial by persisted metadata or the legacy signature predicate before applying scan limits. + +- **Grouped type-only and stale-contract queries no longer preload C# callable type facts (#4914)** — nullable callable identity facts are refreshed only when the current C# family contract can use them and the requested symbol kind can include a partial callable. + +- **Nullable partial-callable lookup now follows project-marker casing (#4914)** — custom type facts use the indexed workspace's case-sensitivity policy, so uppercase lookalike project files cannot change family scope on case-sensitive filesystems. + +- **Qualified impact traversal now follows every partial-method declaration site (#4914)** — the canonical definition remains the single displayed representative, while graph traversal starts from both declaration and implementation symbol identities so callers resolved to either physical row are retained. + +- **Partial-family boundaries remain complete for uncommon paths and large mixed callable families (#4914)** — Unix file-local identities preserve literal backslashes instead of conflating them with directory separators, and truncated impact expansion includes both `function` and `test.method` physical rows from one logical partial-callable family. + +- **Equivalent function-pointer spellings now share one partial-callable family (#4914)** — an explicit default `managed` calling convention is normalized with the equivalent omitted form, so valid partial-method pairs remain grouped. + +- **Post-extraction C# hook mutations keep partial-family metadata coherent (#4914)** — renaming a declaration preserves preceding-line `partial` and `file` modifiers that are absent from its stored signature, and moving a containing type refreshes the accepted container path and family identity of unchanged descendants. + +- **Transaction and Unix path boundaries no longer leak C# partial type facts (#4914)** — type facts observed in a transaction are rebuilt after commit or rollback, hook kind/sub-kind reclassification retains preceding-line declaration modifiers, and literal Unix backslashes remain distinct from directory separators in project scopes even when the persisted database is queried on another operating system. + +- **C# declaration trivia no longer disrupts partial-family identity or representative ranking (#4914)** — partial methods whose modifier is on a preceding line still group when their stored signature contains the word `partial` only inside a comment; tab-separated `where` constraints receive semantic declaration rank; and adjacent lexer-confirmed `/** ... */` documentation keeps that rank even when the block exceeds the ordinary bounded evidence lookback. + +- **Support-report family readiness now validates physical rows (#4914)** — support manifests use the same partial-family completeness scan as normal query and status paths, so current metadata cannot mask partial declarations with missing family keys or mixed family populations. + +- **C# preprocessor trivia no longer hides declaration modifiers (#4914)** — leading `file` and `partial` modifiers remain attached across directive-only gaps and before matching conditional branches, while tokens and modifiers inside sibling or inactive branches are skipped rather than borrowed by another declaration. + +- **C# declaration keyword lookup now excludes attributes and escaped identifiers (#4914)** — declaration-keyword-like generic attributes no longer hide a plain record's `partial` modifier, and an escaped `@record` class name retains its declaration identifier column instead of resolving to a later constraint reference. + +- **C# partial families preserve escaped record-class names and Unix project scopes (#4914)** — `record class @class` navigation now points at the escaped declaration identifier rather than the contextual `class` suffix, and collision-safe scope encoding keeps partial methods grouped when their project path contains `|`, `%`, or the internal family separator. + +- **C# family-scope delimiters no longer alter legacy language contracts or file-local type resolution (#4914)** — collision-safe project-scope encoding is limited to the current C# family contract so incremental VB, F#, and MSBuild indexes retain their v2 key format, and a raw `|` inside a Unix file-local source path no longer hides the `file-local:` identity or leaks nested type facts into project-wide partial-method classification. + +- **C# partial families distinguish namespace and nested-type boundaries and navigate multiline identifiers (#4914)** — persisted family identities encode every type boundary, including root-level types, so mutually exclusive namespace and nested-type declarations with the same dotted source name remain separate in grouped queries and nullable callable classification; multiline partial declarations retain their full source range while navigation points to the actual identifier line and column. The C# hotspot-family contract is v14 and the reference-identity contract is v8. + +- **Partial upgrades no longer publish reference identity v8 over legacy C# family rows (#4914)** — full, scoped, MCP, and fold-backfill graph refreshes keep the reference-identity contract stale until authoritative C# partial-family metadata can be restamped, preventing unchanged pre-v8 family identities from contaminating newly resolved reference candidates; fresh writers that own the complete current C# population retain v8 after fold backfill, and fresh CLI/MCP finalization reuses authoritative C# presence instead of adding database probes. + +## 日本語 + +- **通常methodが大量にあってもpartial callableの全件走査へ退避しなくなりました (#4914)** — 候補走査とfallback走査は、走査上限を適用する前に、永続metadataまたは旧signature predicateでpartialと確認できたcallableだけを保持します。 + +- **型だけの grouped query と stale-contract query が C# callable type 情報を事前読み込みしなくなりました (#4914)** — nullable callable identity 情報は、current な C# family contract が利用でき、要求された symbol kind が partial callable を含み得る場合だけ更新します。 + +- **nullable partial-callable lookup が project-marker の大小区別規則に従うようになりました (#4914)** — custom type 情報が indexed workspace の case-sensitivity policy を使うため、case-sensitive filesystem で大文字拡張子の類似 project file が family scope を変えません。 + +- **qualified impact traversal が partial method の全宣言siteを辿るようになりました (#4914)** — definition の表示は canonical representative 1件のまま維持し、グラフ探索は宣言側と実装側の両 symbol identity から開始するため、どちらの物理rowへ解決された caller も保持します。 + +- **特殊な path と大きな mixed callable family でも partial-family 境界と member が欠落しなくなりました (#4914)** — Unix の file-local identity は literal な backslash を directory separator と混同せず保持し、truncated な impact 展開は同じ logical partial-callable family の `function` と `test.method` の両方の物理 row を含めます。 + +- **同値な function-pointer 表記が同じ partial-callable family になりました (#4914)** — 既定の `managed` calling convention を明示した表記と省略した同値表記を正規化し、valid な partial-method pair を一つに集約します。 + +- **post-extraction C# hook mutation 後も partial-family metadata の整合性を維持します (#4914)** — declaration の rename 時に保存済み signature に含まれない前行の `partial` / `file` modifier を保持し、containing type の移動時には未変更 descendant の受理済み container path と family identity を更新します。 + +- **transaction と Unix path の境界から C# partial type fact が漏れなくなりました (#4914)** — transaction 内で観測した type fact は commit / rollback 後に再構築し、hook の kind / sub-kind 再分類でも前行の declaration modifier を保持し、永続化した database を別の OS から query する場合も Unix project scope の literal な backslash を directory separator と区別します。 + +- **C# declaration の trivia が partial family identity と representative rank を妨げなくなりました (#4914)** — modifier が前の行にあり、保存済み signature の `partial` が comment 内だけに現れる partial method も集約します。タブ区切りの `where` constraint に semantic declaration rank を与え、lexer で確認した隣接 `/** ... */` documentation block が通常の evidence lookback 上限を超えてもその rank を保持します。 + +- **support report の family readiness が物理 row も検証するようになりました (#4914)** — support manifest は通常の query / status と同じ partial-family completeness scan を使うため、metadata が current でも family key のない partial declaration や family population の混在を ready と誤判定しません。 + +- **C# の preprocessor trivia が declaration modifier を隠さなくなりました (#4914)** — 先行する `file` と `partial` modifier を directive だけの間隔と対応する条件分岐の前から宣言に結び付ける一方、兄弟分岐や inactive な条件分岐内の token と modifier は別の宣言へ借用せず読み飛ばします。 + +- **C# の declaration keyword 検索が attribute と escaped identifier を除外するようになりました (#4914)** — 宣言 keyword と同名の generic attribute が plain record の `partial` modifier を隠さず、escaped な `@record` class 名は後続 constraint の参照ではなく宣言 identifier の正確な column を保持します。 + +- **C# partial family が escaped record-class 名と Unix project scope を保持するようになりました (#4914)** — `record class @class` の navigation は contextual な `class` suffix ではなく escaped declaration identifier を指し、衝突しない scope encoding により project path に `|`・`%`・内部 family separator が含まれても partial method を集約します。 + +- **C# family-scope の区切りが旧言語契約と file-local 型解決へ影響しなくなりました (#4914)** — 衝突しない project-scope encoding を current な C# family contract だけに限定するため、VB・F#・MSBuild の増分 index は v2 key 形式を維持します。また Unix の file-local source path 内に raw な `|` があっても `file-local:` identity を見失わず、nested type 情報を project-wide な partial-method 型分類へ漏らしません。 + +- **C# partial family が namespace 境界と nested-type 境界を区別し、複数行 identifier へ移動するようになりました (#4914)** — 永続 family identity は root-level type を含むすべての type 境界を符号化するため、同じ dotted source 名を持つ相互排他的な namespace 宣言と nested-type 宣言を grouped query と nullable callable 分類で別 family として保持します。複数行の partial 宣言は source range 全体を保ったまま、実際の identifier 行・列を navigation 先にします。C# hotspot-family contract は v14、reference-identity contract は v8 です。 + +- **部分 upgrade が旧 C# family row のまま reference identity v8 を公開しなくなりました (#4914)** — full・scoped・MCP・fold-backfill の graph refresh は authoritative な C# partial-family metadata を再 stamp できるまで reference-identity contract を stale のまま維持し、未変更の v8 より前の family identity が新しい reference candidate 解決へ混入することを防ぎます。current な C# population 全体を所有する fresh writer は fold backfill 後も v8 を維持し、fresh CLI / MCP の finalization は authoritative な C# presence を再利用して database probe を追加しません。 + +- **partial declaration family が canonical representative を選び、その理由を公開するようになりました (#4914)** — 集約した C# の partial class、record、method は、test attribute により実装側だけが `test.method` に分類される場合も含めて、実装、非生成 source、意味情報が豊富な宣言を優先し、その後に安定した declaration identity と source location を使います。nested partial type は container の family identity をそのまま継承せず、自身の名前と generic arity を保持します。file-local な partial host とその nested descendant は同一 source file 内だけで集約し、同一ファイルの host の一部だけに `file` 修飾子がある場合も grouped query と hotspots の両方で別ファイルの family と混同しません。plain record の arity は先行する attribute 内の同名 argument ではなく lexer で確認した宣言から解決し、明示的な `record class` / `record struct` でも `partial` 修飾子を保持します。readiness は leaf 名が同じでも arity が異なる C# type を別 family として扱い、VB・F#・MSBuild の family contract version は変更しません。サポートレポートも、family 対象言語を含まない index は ready と扱い、インデックスに実在する family 対象言語について旧 global stamp へフォールバックし、言語別 stamp の marker fingerprint が欠落または未完了の場合は ready と判定しません。generic constraint の `:` は semantic representative rank 用の base list として数えません。抽出 metadata は post-extraction hook の clone 後も維持され、成功した no-op hook の後も合成した positional record component の container 所属を保持し、明示的な hook mutation 後の同名 generic container は source range で選択します。また、空行や comment trivia をまたぐ前の行の `partial` 修飾子、balanced な先行 attribute list の後ろにある modifier、正しい同一行 declaration occurrence に属する attribute、delimited documentation 内の空行や宣言と同じ行の delimited documentation を含む隣接した lexer 確認済み C# XML documentation、record・同一行で反復する宣言・正規化後の verbatim identifier・test method の impact result の正確な宣言名列を保持し、空行で切り離された documentation、`assembly:` / `module:` の global attribute、block comment、string、同一行の declaration body 内の decoy を modifier、semantic rank、declaration identity、generic constraint から除外します。`partial` は declaration modifier の位置にある場合だけ認識するため、同名の parameter や constraint identifier を持つ non-partial 宣言は物理 row のままです。callable family には comment-aware な永続化済み partial evidence を必須とします。また callable identity は method identifier、generic parameter、parameter list の間、および generic optional-default member access 内の comment trivia と parameter identifier 内の Unicode 結合文字を許容し、明示的に global-rooted な predefined alias、その verbatim segment、value type の `S?` / `global::System.Nullable` という同値表記、nullable な明示的 global-rooted `System.ValueTuple` 表記、predefined reference type の nullable annotation、arity を考慮した nested / file-scoped indexed custom reference type の nullable annotation、同じ generic partial container の各宣言に分散した constraint、unconstrained または reference-constrained な method type parameter の nullable annotation を正規化する一方、別々の C# project に同じ qualified name で kind の異なる custom type がある場合も project scope を維持し、indexed custom value type と value-constrained method type parameter の nullable の区別、verbatim な `@global` alias、shadow 可能な root のない `System` type、method generic parameter 名に一致する qualified type leaf、無関係な indexed type と leaf 名が衝突しただけの import-scope 未解決型を保ちます。custom type 情報は partial grouping の bounded callable candidate に必要な分だけ読み込み、外部の index commit 後には更新します。database diff は新しい partial / ranking / location metadata を legacy column fallback 付きで比較します。stale な family 契約では reindex まで authoritative な物理 row を保守的に維持しながら、LSP の type / constructor 判別は利用できます。structured result は安定した family ID、selection reason、上限付き member を返し、`goto` は既定でその代表を返す一方、`goto --all` は全物理 location を維持し、旧 generated-file metadata に対しても決定的に fallback します。 diff --git a/src/CodeIndex/Cli/CliFlagSchema.cs b/src/CodeIndex/Cli/CliFlagSchema.cs index deb33bdcb..d050a7703 100644 --- a/src/CodeIndex/Cli/CliFlagSchema.cs +++ b/src/CodeIndex/Cli/CliFlagSchema.cs @@ -481,7 +481,7 @@ private static IReadOnlyList BuildAll() new() { Name = "--confidence", ValueDomain = Values(["medium", "low"]), Description = "Unused: alias for --min-confidence", PrimaryCommands = Set(UnusedFilterCommands) }, new() { Name = "--min-confidence", ValueDomain = Values(["medium", "low"]), Description = "Unused: return symbols at or above this confidence", PrimaryCommands = Set(UnusedFilterCommands) }, new() { Name = "--actionable", Description = "Unused: preset for private medium-confidence cleanup candidates", PrimaryCommands = Set(UnusedFilterCommands) }, - new() { Name = "--all", Description = "goto: return all matching LSP locations; find: search all indexed files instead of requiring --path; unused: include low-confidence contract-domain candidates suppressed by default", PrimaryCommands = Set(AllResultCommands) }, + new() { Name = "--all", Description = "goto: return all matching physical LSP locations; find: search all indexed files instead of requiring --path; unused: include low-confidence contract-domain candidates suppressed by default", PrimaryCommands = Set(AllResultCommands) }, new() { Name = "--line-scan-limit", ValuePlaceholder = "", Description = "Find: override the --all indexed-line scan cap", PrimaryCommands = Set("find") }, new() { Name = "--rank-by", ValueDomain = Values(["weighted", "count", "kind"]), Description = "Rank callers/callees by the selected primary recipe, then exact-name relevance, production/test/docs path category, and stable location/name tie-breakers", PrimaryCommands = Set(RankByCommands) }, new() { Name = "--sort", ValuePlaceholder = "", Description = "Symbols/outline: order audit output by a ranking signal; outline also accepts source, kind, references, size, complexity, path, and name", PrimaryCommands = Set(SymbolSortCommands) }, @@ -489,7 +489,7 @@ private static IReadOnlyList BuildAll() new() { Name = "--include-qualified-common-calls", Description = "Include unresolved receiver/type-qualified C# calls with common member names", PrimaryCommands = Set("references", "callers", "callees") }, new() { Name = "--include-member-reads", Description = "Include canonical member/value-read edges in callers, callees, and impact results", PrimaryCommands = Set("callers", "callees", "impact") }, new() { Name = "--count", Description = "Count only; result limits are ignored by count modes, but scan caps can still mark approximate counts as degraded", PrimaryCommands = Set(CountCommands) }, - new() { Name = "--group-partials", Description = "Definition/Symbols/Inspect: collapse C# partial-type declarations into logical families", PrimaryCommands = Set("definition", "symbols", "inspect") }, + new() { Name = "--group-partials", Description = "Definition/Symbols/Inspect: collapse C# partial type and method declarations into logical families", PrimaryCommands = Set("definition", "symbols", "inspect") }, new() { Name = "--strict-not-found", Description = "Return exit code 2 when a valid query has zero rows", PrimaryCommands = Set(StrictNotFoundCommands), Safety = CliOptionSafety.StrictFailure }, new() { Name = "--allow-partial", Description = "Return exit code 0 instead of 11 for accepted partial query output or an incomplete index generation", PrimaryCommands = Set(AllowPartialCommands) }, new() { Name = "--strict", Description = "Return exit code 4 when impact preconditions are unmet", PrimaryCommands = Set("impact") }, diff --git a/src/CodeIndex/Cli/DiffCommandRunner.cs b/src/CodeIndex/Cli/DiffCommandRunner.cs index fe7e2ecd4..7c921591c 100644 --- a/src/CodeIndex/Cli/DiffCommandRunner.cs +++ b/src/CodeIndex/Cli/DiffCommandRunner.cs @@ -236,6 +236,10 @@ private static DiffJsonResult CompareDatabasesCore( "family_key", "visibility", "return_type", + "is_partial_declaration", + "is_file_local_declaration", + "declaration_semantic_score", + "identifier_start_column", "is_metadata_target", "metadata_target_source", ]); @@ -1452,6 +1456,18 @@ private static string BuildSymbolRowsSql(SqliteConnection connection) var displayNameFoldedExpr = ColumnExists(connection, "symbols", "display_name_folded") ? "symbols.display_name_folded" : "NULL"; + var isPartialDeclarationExpr = ColumnExists(connection, "symbols", "is_partial_declaration") + ? "symbols.is_partial_declaration" + : "NULL"; + var isFileLocalDeclarationExpr = ColumnExists(connection, "symbols", "is_file_local_declaration") + ? "symbols.is_file_local_declaration" + : "NULL"; + var declarationSemanticScoreExpr = ColumnExists(connection, "symbols", "declaration_semantic_score") + ? "symbols.declaration_semantic_score" + : "NULL"; + var identifierStartColumnExpr = ColumnExists(connection, "symbols", "identifier_start_column") + ? "symbols.identifier_start_column" + : "NULL"; return $$""" SELECT @@ -1474,6 +1490,10 @@ private static string BuildSymbolRowsSql(SqliteConnection connection) symbols.family_key, symbols.visibility, symbols.return_type, + {{isPartialDeclarationExpr}}, + {{isFileLocalDeclarationExpr}}, + {{declarationSemanticScoreExpr}}, + {{identifierStartColumnExpr}}, {{metadataTargetExpr}}, {{metadataTargetSourceExpr}} FROM symbols @@ -1498,6 +1518,10 @@ ORDER BY symbols.family_key, symbols.visibility, symbols.return_type, + {{isPartialDeclarationExpr}}, + {{isFileLocalDeclarationExpr}}, + {{declarationSemanticScoreExpr}}, + {{identifierStartColumnExpr}}, {{metadataTargetExpr}}, {{metadataTargetSourceExpr}} """; diff --git a/src/CodeIndex/Cli/IndexCommandRunner.ChangeDetection.cs b/src/CodeIndex/Cli/IndexCommandRunner.ChangeDetection.cs index 93dccdfdf..f57da77b0 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.ChangeDetection.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.ChangeDetection.cs @@ -61,9 +61,9 @@ private static void RestampHotspotFamilyTrustForUpdate( IReadOnlyDictionary priorFingerprints, IReadOnlyDictionary currentFingerprints) { - var currentVersion = DbContext.HotspotFamilyVersion.ToString(System.Globalization.CultureInfo.InvariantCulture); foreach (var lang in FileIndexer.GetHotspotFamilyMarkerLanguages()) { + var currentVersion = DbContext.GetHotspotFamilyVersion(lang).ToString(System.Globalization.CultureInfo.InvariantCulture); if (!currentFingerprints.TryGetValue(lang, out var currentFingerprint)) continue; @@ -90,9 +90,9 @@ private static void RestampHotspotFamilyTrustForFullScan( IReadOnlyDictionary priorFingerprints, IReadOnlyDictionary currentFingerprints) { - var currentVersion = DbContext.HotspotFamilyVersion.ToString(System.Globalization.CultureInfo.InvariantCulture); foreach (var lang in FileIndexer.GetHotspotFamilyMarkerLanguages()) { + var currentVersion = DbContext.GetHotspotFamilyVersion(lang).ToString(System.Globalization.CultureInfo.InvariantCulture); if (!currentFingerprints.TryGetValue(lang, out var currentFingerprint)) continue; @@ -114,10 +114,10 @@ private static Dictionary GetHotspotFamilyTrustMatchesCurrent( IReadOnlyDictionary priorFingerprints, IReadOnlyDictionary currentFingerprints) { - var currentVersion = DbContext.HotspotFamilyVersion.ToString(System.Globalization.CultureInfo.InvariantCulture); var values = new Dictionary(StringComparer.Ordinal); foreach (var lang in FileIndexer.GetHotspotFamilyMarkerLanguages()) { + var currentVersion = DbContext.GetHotspotFamilyVersion(lang).ToString(System.Globalization.CultureInfo.InvariantCulture); currentFingerprints.TryGetValue(lang, out var currentFingerprint); priorVersions.TryGetValue(lang, out var priorVersion); priorFingerprints.TryGetValue(lang, out var priorFingerprint); diff --git a/src/CodeIndex/Cli/IndexCommandRunner.DryRun.cs b/src/CodeIndex/Cli/IndexCommandRunner.DryRun.cs index ea8bf9a8a..96ec61c70 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.DryRun.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.DryRun.cs @@ -1093,7 +1093,8 @@ private static DryRunParsedMutationEstimate BuildDryRunParsedMutationEstimate( SymbolExtractor.ApplyFamilyScope( symbols, - indexer.GetFamilyScopeKey(absolutePath, record.Lang)); + indexer.GetFamilyScopeKey(absolutePath, record.Lang), + record.Lang); var symbolsDroppedByKindFilter = options.SymbolKindFilter.Apply(symbols); if (symbols.Count > options.MaxSymbolsPerFile) { diff --git a/src/CodeIndex/Cli/IndexCommandRunner.FullScan.ExtractionWorkers.cs b/src/CodeIndex/Cli/IndexCommandRunner.FullScan.ExtractionWorkers.cs index ec04a4c1c..5ac74710a 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.FullScan.ExtractionWorkers.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.FullScan.ExtractionWorkers.cs @@ -177,7 +177,10 @@ private static Task[] StartFullScanExtractionWorkers( extractionCancellationToken); continue; } - SymbolExtractor.ApplyFamilyScope(symbols, indexer.GetFamilyScopeKey(filePath, record.Lang)); + SymbolExtractor.ApplyFamilyScope( + symbols, + indexer.GetFamilyScopeKey(filePath, record.Lang), + record.Lang); FileIssue? referenceRegexTimeoutIssue = null; ReferenceExtractionResult? referenceExtraction = null; if (options.SymbolsOnly) @@ -239,7 +242,12 @@ private static Task[] StartFullScanExtractionWorkers( references!, issues!, generatedSuppressionIssue, - generatedSuppressionChecked: true) + generatedSuppressionChecked: true, + content: postExtractionHooks.HasHooks ? content : null, + hasOversizeLine: postExtractionHooks.HasHooks ? hasOversizeLine : null, + conflictMarkerLine: postExtractionHooks.HasHooks + ? loaded.ConflictMarkerLine + : null) : FullScanFileWorkItem.Success( fileIndex, filePath, diff --git a/src/CodeIndex/Cli/IndexCommandRunner.FullScan.FilePersistence.cs b/src/CodeIndex/Cli/IndexCommandRunner.FullScan.FilePersistence.cs index 1c2ac61b2..557953949 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.FullScan.FilePersistence.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.FullScan.FilePersistence.cs @@ -167,16 +167,20 @@ private static FullScanFilePersistenceResult PersistFullScanFile( $" [SKIP] {record.Path} ({issue.Message})"); } + var familyScopeKey = context.Indexer.GetFamilyScopeKey(item.FilePath, record.Lang); if (item.Symbols == null) { SymbolExtractor.ApplyFamilyScope( symbols, - context.Indexer.GetFamilyScopeKey(item.FilePath, record.Lang)); + familyScopeKey, + record.Lang); } var mutableSymbols = symbols as IList ?? symbols.ToList(); context.PostExtractionHooks.OnSymbolsExtractedAfterSourceObservation( fileContext, - mutableSymbols); + mutableSymbols, + item.Content, + familyScopeKey); symbolsDroppedByKindFilter = options.SymbolKindFilter.Apply(mutableSymbols); symbols = (IReadOnlyList)mutableSymbols; if (symbols.Count > options.MaxSymbolsPerFile) diff --git a/src/CodeIndex/Cli/IndexCommandRunner.FullScan.Readiness.cs b/src/CodeIndex/Cli/IndexCommandRunner.FullScan.Readiness.cs index 375ec93d9..676b1ab2b 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.FullScan.Readiness.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.FullScan.Readiness.cs @@ -157,6 +157,13 @@ private static FullScanReadinessResult FinalizeFullScanReadiness( context.PriorHotspotFamilyVersions, context.PriorHotspotFamilyMarkerFingerprints, context.CurrentHotspotFamilyMarkerFingerprints); + if (!options.SymbolsOnly) + { + if (writer.CSharpFamilyTrustAllowsReferenceIdentityReady(context.HasCSharpFilesAfter)) + writer.MarkReferenceIdentityContractReady(); + else + writer.ClearReferenceIdentityContractReady(); + } writer.StampSymbolExtractorVersions(context.IndexedSymbolExtractorLanguages); writer.StampDynamicReferenceGraphContracts(context.IndexedSymbolExtractorLanguages); diff --git a/src/CodeIndex/Cli/IndexCommandRunner.FullScan.cs b/src/CodeIndex/Cli/IndexCommandRunner.FullScan.cs index b64bb2947..612c31594 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.FullScan.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.FullScan.cs @@ -1027,7 +1027,15 @@ void RecordFullScanTargetStatSkip(int fileIndex, IndexedFileStatReuseResult exis var referenceGraphHeartbeat = StartFullScanJsonPhaseHeartbeat(options, "finalizing reference graph"); try { - writer.RefreshMutualRecursionFlags(cancellationToken); + writer.RefreshMutualRecursionFlags( + cancellationToken, + stampReferenceIdentityContractReady: + writer.CSharpFamilyTrustAllowsReferenceIdentityReady( + (options.Rebuild || startedWithNoIndexedFiles) + && !scanHadErrors + && errors == 0 + ? languageCounts.ContainsKey("csharp") + : null)); } finally { diff --git a/src/CodeIndex/Cli/IndexCommandRunner.Update.FilePersistence.cs b/src/CodeIndex/Cli/IndexCommandRunner.Update.FilePersistence.cs index 65aa6bf4b..47d3f2df4 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.Update.FilePersistence.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.Update.FilePersistence.cs @@ -194,12 +194,13 @@ private static UpdateFilePersistenceResult PersistUpdateFile( $" [SKIP] {context.RelativePath} ({issue.Message})"); } - SymbolExtractor.ApplyFamilyScope( - symbols, - context.Indexer.GetFamilyScopeKey(context.AbsolutePath, record.Lang)); + var familyScopeKey = context.Indexer.GetFamilyScopeKey(context.AbsolutePath, record.Lang); + SymbolExtractor.ApplyFamilyScope(symbols, familyScopeKey, record.Lang); context.PostExtractionHooks.OnSymbolsExtractedAfterSourceObservation( fileContext, - symbols); + symbols, + loaded.Content, + familyScopeKey); symbolsDroppedByKindFilter = options.SymbolKindFilter.Apply(symbols); if (symbols.Count > options.MaxSymbolsPerFile) { diff --git a/src/CodeIndex/Cli/IndexCommandRunner.Update.Readiness.cs b/src/CodeIndex/Cli/IndexCommandRunner.Update.Readiness.cs index 52bf0d406..a47819fac 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.Update.Readiness.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.Update.Readiness.cs @@ -112,12 +112,6 @@ private static UpdateReadinessResult FinalizeUpdateReadiness(UpdateReadinessCont writer.MarkGraphReady(); graphTableAvailableAfter = true; } - if (!options.SymbolsOnly - && !context.MutualRecursionRefreshNeeded - && context.ReferenceIdentityContractMatchedBeforeMutation) - { - writer.MarkReferenceIdentityContractReady(); - } writer.StampSymbolExtractorVersions(context.FullyRefreshedDynamicGraphLanguages); writer.StampDynamicReferenceGraphContracts(context.FullyRefreshedDynamicGraphLanguages); if ((context.PriorReadiness & DbContext.IssuesReadyFlag) != 0) @@ -174,6 +168,13 @@ private static UpdateReadinessResult FinalizeUpdateReadiness(UpdateReadinessCont HotspotFamilyUpdateRestampReadyForCommitForTesting?.Invoke(); hotspotFamilyTxn.Commit(); } + if (!options.SymbolsOnly) + { + if (writer.CSharpFamilyTrustAllowsReferenceIdentityReady(hasCSharpFilesAfter)) + writer.MarkReferenceIdentityContractReady(); + else + writer.ClearReferenceIdentityContractReady(); + } if ((context.PriorReadiness & DbContext.FoldReadyFlag) != 0 && context.PriorFoldVersion == context.CurrentFoldVersion && context.PriorFoldFingerprint == context.CurrentFoldFingerprint diff --git a/src/CodeIndex/Cli/IndexCommandRunner.Update.cs b/src/CodeIndex/Cli/IndexCommandRunner.Update.cs index 8d39c2148..93d0812f8 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.Update.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.Update.cs @@ -724,7 +724,10 @@ bool TryValidateCSharpWorkspaceInputSnapshot( ThrowIfUpdateCancelled(); mutualRecursionRefreshNeeded |= !options.SymbolsOnly && (removed > 0 || purgedRefs > 0); if (mutualRecursionRefreshNeeded) - writer.RefreshMutualRecursionFlags(cancellationToken); + writer.RefreshMutualRecursionFlags( + cancellationToken, + stampReferenceIdentityContractReady: + writer.CSharpFamilyTrustAllowsReferenceIdentityReady()); if (options.MemoryTrace) memorySamples.Add(CaptureMemorySample("reference_graph", stopwatch)); ThrowIfUpdateCancelled(); diff --git a/src/CodeIndex/Cli/IndexCommandRunner.WorkItems.cs b/src/CodeIndex/Cli/IndexCommandRunner.WorkItems.cs index 35475ee12..52068f51c 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.WorkItems.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.WorkItems.cs @@ -130,16 +130,19 @@ public static FullScanFileWorkItem Precomputed( IReadOnlyList references, IReadOnlyList issues, FileIssue? generatedSuppressionIssue = null, - bool generatedSuppressionChecked = false) + bool generatedSuppressionChecked = false, + string? content = null, + bool? hasOversizeLine = null, + int? conflictMarkerLine = null) { return new FullScanFileWorkItem( fileIndex, filePath, relativePath, record, - null, - null, - null, + content, + hasOversizeLine, + conflictMarkerLine, warning, chunks, symbols, diff --git a/src/CodeIndex/Cli/ProjectionFieldRegistry.cs b/src/CodeIndex/Cli/ProjectionFieldRegistry.cs index 2c1e7bfac..bbf5aee80 100644 --- a/src/CodeIndex/Cli/ProjectionFieldRegistry.cs +++ b/src/CodeIndex/Cli/ProjectionFieldRegistry.cs @@ -171,7 +171,8 @@ private static ProjectionCommandFieldSchema CreateDefinitionSchema() "disambiguator", "api_version", "path", "symbol_id", "lang", "kind", "sub_kind", "name", "line", "start_line", "start_column", "end_line", "body_start_line", "body_end_line", "signature", "container_kind", "container_name", "visibility", - "return_type", "definition_sites", "exact_index_available", "degraded_reason", + "return_type", "definition_sites", "partial_family_id", "representative_reason", + "family_members", "family_members_truncated", "exact_index_available", "degraded_reason", "content_omitted", "content_omitted_reason", "body_content", "body_content_start_line", "body_content_end_line", "body_requested_start_line", "body_requested_end_line", "body_effective_start_line", "body_effective_end_line", "body_content_truncated", @@ -295,7 +296,8 @@ private static ProjectionCommandFieldSchema CreateImpactSchema() "api_version", "path", "symbol_id", "lang", "kind", "sub_kind", "name", "line", "start_line", "start_column", "end_line", "body_start_line", "body_end_line", "signature", "container_kind", "container_name", "visibility", "return_type", - "definition_sites", + "definition_sites", "partial_family_id", "representative_reason", "family_members", + "family_members_truncated", }; return Create( "impact", diff --git a/src/CodeIndex/Cli/QueryCommandRunner.Definition.cs b/src/CodeIndex/Cli/QueryCommandRunner.Definition.cs index d6c8f7fee..6e96c5544 100644 --- a/src/CodeIndex/Cli/QueryCommandRunner.Definition.cs +++ b/src/CodeIndex/Cli/QueryCommandRunner.Definition.cs @@ -339,7 +339,7 @@ public static int RunGoto(string[] cmdArgs, JsonSerializerOptions jsonOptions) var limit = all ? options.LimitExplicit ? options.Limit : int.MaxValue : Math.Max(options.Limit, 2); - var results = reader.GetDefinitions(options.Query, limit, options.Kind, options.Lang, includeBody: false, options.PathPatterns, options.ExcludePaths, options.ExcludeTests, options.Since, exact, visibilityFilters: options.VisibilityFilters, excludeVisibilityFilters: options.ExcludeVisibilityFilters); + var results = reader.GetDefinitions(options.Query, limit, options.Kind, options.Lang, includeBody: false, options.PathPatterns, options.ExcludePaths, options.ExcludeTests, options.Since, exact, visibilityFilters: options.VisibilityFilters, excludeVisibilityFilters: options.ExcludeVisibilityFilters, groupPartials: !all); if (results.Count == 0) { return CommandErrorWriter.WriteJsonOrHuman( @@ -360,12 +360,6 @@ public static int RunGoto(string[] cmdArgs, JsonSerializerOptions jsonOptions) if (results.Count > 1) { - if (TrySelectLogicalPartialDefinition(results, out var representative)) - { - Console.WriteLine(SerializeQueryJson(ToLspLocation(representative), CliJsonSerializerContextFactory.Create(jsonOptions).LspLocation, jsonOptions)); - return CommandExitCodes.Success; - } - CommandErrorWriter.WriteStderr($"Error: goto found {results.Count} matching definitions for '{options.Query}'."); CommandErrorWriter.WriteStderr("Hint: narrow the query with --kind, --lang, --path, or pass --all to return all LSP locations."); return CommandExitCodes.UsageError; @@ -375,18 +369,4 @@ public static int RunGoto(string[] cmdArgs, JsonSerializerOptions jsonOptions) return CommandExitCodes.Success; }); } - - private static bool TrySelectLogicalPartialDefinition(IReadOnlyList results, out SymbolResult representative) - { - representative = results[0]; - if (results.Count <= 1) - return false; - - var grouped = LogicalPartialSymbolGrouper.Group(results); - if (grouped.Count != 1 || grouped[0].DefinitionSites != results.Count) - return false; - - representative = grouped[0]; - return true; - } } diff --git a/src/CodeIndex/Cli/QueryCommandRunner.Locations.cs b/src/CodeIndex/Cli/QueryCommandRunner.Locations.cs index 2f632fe13..3a255831c 100644 --- a/src/CodeIndex/Cli/QueryCommandRunner.Locations.cs +++ b/src/CodeIndex/Cli/QueryCommandRunner.Locations.cs @@ -85,7 +85,29 @@ private static LspLocation BuildSymbolLspLocation(SymbolResult result) { var line = result.Line > 0 ? result.Line : Math.Max(1, result.StartLine); var column = result.StartColumn.HasValue ? result.StartColumn.Value + 1 : 1; - return BuildLspLocation(result.Path, line, column, line, column + Math.Max(1, result.Name.Length)); + var location = BuildLspLocation(result.Path, line, column, line, column + Math.Max(1, result.Name.Length)); + if (result.PartialFamilyId == null) + return location; + + location.SymbolId = result.SymbolId; + location.PartialFamilyId = result.PartialFamilyId; + location.RepresentativeReason = result.RepresentativeReason; + location.FamilyMembersTruncated = result.FamilyMembersTruncated; + location.Representative = true; + location.FamilyMembers = result.FamilyMembers? + .Select(member => BuildPartialFamilyMemberLspLocation(member, result.Name)) + .ToList(); + return location; + } + + private static LspLocation BuildPartialFamilyMemberLspLocation(PartialFamilyMember member, string symbolName) + { + var line = member.Line > 0 ? member.Line : Math.Max(1, member.StartLine); + var column = member.StartColumn.HasValue ? member.StartColumn.Value + 1 : 1; + var location = BuildLspLocation(member.Path, line, column, line, column + Math.Max(1, symbolName.Length)); + location.SymbolId = member.SymbolId; + location.Representative = member.Representative; + return location; } private static LspLocation ToLspLocation(CallerResult result) diff --git a/src/CodeIndex/Cli/ReportCommandRunner.cs b/src/CodeIndex/Cli/ReportCommandRunner.cs index a68a77eea..4623afecd 100644 --- a/src/CodeIndex/Cli/ReportCommandRunner.cs +++ b/src/CodeIndex/Cli/ReportCommandRunner.cs @@ -578,10 +578,7 @@ private static ReportReadinessSnapshot BuildReadinessSnapshot(string? dbPath, bo GetMeta(meta, DbContext.SqlGraphContractVersionMetaKey), DbContext.SqlGraphContractVersion.ToString(System.Globalization.CultureInfo.InvariantCulture), StringComparison.Ordinal); - var hotspotFamilyReady = string.Equals( - GetMeta(meta, DbContext.HotspotFamilyVersionMetaKey), - DbContext.HotspotFamilyVersion.ToString(System.Globalization.CultureInfo.InvariantCulture), - StringComparison.Ordinal); + var hotspotFamilyReady = IsHotspotFamilyReady(connection, symbolColumns, meta); var foldReady = (userVersion & DbContext.FoldReadyFlag) == DbContext.FoldReadyFlag && symbolColumns.Contains("name_folded"); @@ -619,6 +616,66 @@ private static ReportReadinessSnapshot BuildReadinessSnapshot(string? dbPath, bo } } + private static bool IsHotspotFamilyReady( + SqliteConnection connection, + IReadOnlySet symbolColumns, + Dictionary meta) + { + var legacyGlobalVersion = GetMeta(meta, DbContext.HotspotFamilyVersionMetaKey); + var indexedLanguages = new List(); + foreach (var language in FileIndexer.GetHotspotFamilyMarkerLanguages()) + { + if (CountFilesByLanguage(connection, language) == 0) + continue; + + var perLanguageVersion = GetMeta( + meta, + DbContext.GetHotspotFamilyVersionMetaKey(language)); + var storedVersion = perLanguageVersion ?? legacyGlobalVersion; + var expectedVersion = DbContext.GetHotspotFamilyVersion(language) + .ToString(System.Globalization.CultureInfo.InvariantCulture); + if (!string.Equals(storedVersion, expectedVersion, StringComparison.Ordinal)) + return false; + + if (perLanguageVersion != null) + { + var markerFingerprint = GetMeta( + meta, + DbContext.GetHotspotFamilyMarkerFingerprintMetaKey(language)); + if (string.IsNullOrWhiteSpace(markerFingerprint) + || DbContext.IsIncompleteHotspotFamilyMarkerFingerprint(markerFingerprint)) + { + return false; + } + } + + indexedLanguages.Add(language); + } + + if (indexedLanguages.Count > 0 + && (!symbolColumns.Contains("family_key") + || !symbolColumns.Contains("container_qualified_name"))) + { + return false; + } + + if (DbReader.LoadIncompleteHotspotFamilyLanguages( + connection, + symbolColumns, + indexedLanguages).Count > 0) + { + return false; + } + + // All indexed family-aware languages passed above. If none were indexed, the + // contract is irrelevant; this matches DbReader.GetHotspotFamilySignal and keeps + // legacy non-family-language indexes ready. + // indexed 済みの family-aware 言語はすべて上で検証済み。該当言語がなければ + // contract は非該当なので、DbReader の signal と揃え、family 非対象言語だけを + // 含む旧 index も ready のままにする。 + return true; + } + private static ReportDiagnosticSummary BuildDiagnosticSummary() { var extractorStatus = ExtractorPluginRegistry.GetStatusSnapshot(); diff --git a/src/CodeIndex/Database/CSharpCallableTypeKindLookup.cs b/src/CodeIndex/Database/CSharpCallableTypeKindLookup.cs new file mode 100644 index 000000000..58a519851 --- /dev/null +++ b/src/CodeIndex/Database/CSharpCallableTypeKindLookup.cs @@ -0,0 +1,1183 @@ +using CodeIndex.Indexer; +using Microsoft.Data.Sqlite; + +namespace CodeIndex.Database; + +/// +/// Connection-scoped C# source type facts used to interpret nullable annotations in +/// partial-callable identities. The lookup is intentionally conservative: ambiguous or +/// unresolved names retain their source-level ? marker. +/// partial callable identity の nullable annotation 解釈に使う connection-scoped な +/// C# source type 情報。曖昧または未解決の名前では source 上の ? を保持する。 +/// +internal sealed class CSharpCallableTypeKindLookup +{ + internal enum TypeKind + { + Unknown, + Reference, + Value, + } + + private const int ReferenceKindFlag = 1; + private const int ValueKindFlag = 2; + private const int CandidateTypeNameLimit = 512; + private const int CandidateCallableLimit = 4_096; + private static readonly AsyncLocal ScanObserver = new(); + private static readonly AsyncLocal?> CandidateScanObserver = new(); + private readonly object _gate = new(); + private Dictionary _identityKinds = new(StringComparer.Ordinal); + private Dictionary _scopedIdentityKinds = new(); + private Dictionary _fileIdentityKinds = new(); + private Dictionary _callableFileIds = new(); + private Dictionary _callableTypeParameterKinds = new(); + private long? _loadedTotalChanges; + private long? _loadedDataVersion; + private string? _loadedScopeKey; + + internal static Action? ScanForTesting + { + get => ScanObserver.Value; + set => ScanObserver.Value = value; + } + + internal static Action? CandidateScanForTesting + { + get => CandidateScanObserver.Value; + set => CandidateScanObserver.Value = value; + } + + internal void RefreshIfChanged( + SqliteConnection connection, + IReadOnlySet fileColumns, + IReadOnlySet symbolColumns, + IReadOnlyList? candidateQueries = null, + bool exact = false, + bool useFoldedNames = false) + { + if (!fileColumns.Contains("lang") + || !symbolColumns.Contains("name") + || !symbolColumns.Contains("kind") + || !symbolColumns.Contains("signature") + || !symbolColumns.Contains("container_qualified_name")) + { + return; + } + + lock (_gate) + { + var totalChanges = ReadTotalChanges(connection); + var dataVersion = ReadDataVersion(connection); + var scopeKey = BuildScopeKey(candidateQueries, exact, useFoldedNames); + var transactionActive = SQLitePCL.raw.sqlite3_get_autocommit(connection.Handle) == 0; + if (!transactionActive + && _loadedTotalChanges == totalChanges + && _loadedDataVersion == dataVersion + && string.Equals(_loadedScopeKey, scopeKey, StringComparison.Ordinal)) + { + return; + } + + ScanForTesting?.Invoke(); + var identityKinds = new Dictionary(StringComparer.Ordinal); + var scopedIdentityKinds = new Dictionary(); + var fileIdentityKinds = new Dictionary(); + var callableFileIds = new Dictionary(); + var callables = new List(); + var candidateTypeNames = LoadCandidateCallables( + connection, + symbolColumns, + candidateQueries, + exact, + useFoldedNames, + callableFileIds, + callables); + var useFullScan = candidateTypeNames == null + || candidateTypeNames.Count > CandidateTypeNameLimit; + if (useFullScan) + { + callableFileIds.Clear(); + callables.Clear(); + LoadAllCallableFacts(connection, symbolColumns, callableFileIds, callables); + } + var facts = LoadTypeFacts( + connection, + symbolColumns, + useFullScan ? null : candidateTypeNames, + LoadCSharpProjectMarkerCounts(connection)); + CandidateScanForTesting?.Invoke(new CandidateScanStats( + useFullScan, + callableFileIds.Count, + facts.Count)); + + var factsByIdentity = facts + .GroupBy(BuildUnqualifiedIdentity, StringComparer.Ordinal) + .ToDictionary(group => group.Key, group => group.ToArray(), StringComparer.Ordinal); + var containingFacts = facts.Distinct().ToDictionary( + fact => fact, + fact => ResolveContainingFact(fact, factsByIdentity)); + var resolvedIdentities = new Dictionary>(); + foreach (var fact in facts) + { + foreach (var identity in ResolveIdentities( + fact, + containingFacts, + resolvedIdentities, + new HashSet())) + { + Add(fileIdentityKinds, new FileTypeIdentity(fact.FileId, identity), fact.Kind); + if (!fact.IsFileLocal) + { + Add(identityKinds, identity, fact.Kind); + Add( + scopedIdentityKinds, + new ScopedTypeIdentity(fact.ProjectScope, identity), + fact.Kind); + } + } + } + + _identityKinds = identityKinds; + _scopedIdentityKinds = scopedIdentityKinds; + _fileIdentityKinds = fileIdentityKinds; + _callableFileIds = callableFileIds; + _callableTypeParameterKinds = BuildCallableTypeParameterKinds(callables, facts); + // SQLite total_changes() includes rolled-back writes and data_version does not + // identify transaction-local snapshots. Never cache facts observed inside a + // transaction, so the first query after commit or rollback rebuilds them. + // SQLite の total_changes() は rollback 済み write も含み、data_version では + // transaction-local snapshot を識別できない。transaction 中の fact は cache + // せず、commit / rollback 後の最初の query で必ず再構築する。 + _loadedTotalChanges = transactionActive ? null : totalChanges; + _loadedDataVersion = transactionActive ? null : dataVersion; + _loadedScopeKey = transactionActive ? null : scopeKey; + } + } + + internal TypeKind Resolve( + string sourceIdentity, + string? containerQualifiedName, + long? symbolId = null) + { + var normalizedSource = NormalizeIdentity(sourceIdentity); + if (normalizedSource.Length == 0) + return TypeKind.Unknown; + + lock (_gate) + { + var projectScope = ExtractProjectScope(containerQualifiedName); + var fileId = symbolId.HasValue + && _callableFileIds.TryGetValue(symbolId.Value, out var resolvedFileId) + ? resolvedFileId + : (long?)null; + if (symbolId.HasValue + && IsSimpleIdentifier(sourceIdentity) + && _callableTypeParameterKinds.TryGetValue( + new CallableTypeParameterIdentity(symbolId.Value, normalizedSource.TrimStart('@')), + out var typeParameterKind)) + { + return typeParameterKind; + } + if (sourceIdentity.Contains("::", StringComparison.Ordinal) + && !sourceIdentity.StartsWith("global::", StringComparison.Ordinal)) + { + // Using/extern aliases cannot be bound from the persisted type facts. + // In particular, escaped @global is an ordinary alias, not the root + // qualifier. Preserve nullable syntax instead of rebinding its leaf name. + // using/extern alias は永続 type fact だけでは bind できない。特に escaped + // @global は root qualifier ではなく通常 alias なので、leaf 名へ再 binding + // せず nullable syntax を保持する。 + return TypeKind.Unknown; + } + if (sourceIdentity.StartsWith("global::", StringComparison.Ordinal)) + return ResolveIdentity(normalizedSource, fileId, projectScope); + + var container = NormalizeFamilyIdentity(containerQualifiedName); + while (container.Length > 0) + { + var familyQualified = CombineFamilyContainerAndType(container, normalizedSource); + var resolved = ResolveIdentity(familyQualified, fileId, projectScope); + if (resolved != TypeKind.Unknown) + return resolved; + + var sourceQualified = $"{ConvertFamilyIdentityToSourceIdentity(container)}.{normalizedSource}"; + if (!string.Equals(sourceQualified, familyQualified, StringComparison.Ordinal)) + { + resolved = ResolveIdentity(sourceQualified, fileId, projectScope); + if (resolved != TypeKind.Unknown) + return resolved; + } + + var separator = container.LastIndexOfAny(['.', '+']); + container = separator < 0 ? string.Empty : container[..separator]; + } + + var direct = ResolveIdentity(normalizedSource, fileId, projectScope); + if (direct != TypeKind.Unknown) + return direct; + // A same-leaf declaration elsewhere in the index does not prove C# binding. + // Keep unqualified/import-dependent names unresolved unless a qualified container + // identity above established the target. + // index 内の同名 leaf だけでは C# binding の根拠にならない。上記の qualified + // container identity で対象を確定できない名前は unresolved のまま保持する。 + return TypeKind.Unknown; + } + } + + private static string BuildScopeKey( + IReadOnlyList? candidateQueries, + bool exact, + bool useFoldedNames) + => candidateQueries is not { Count: > 0 } + ? "*" + : $"{(useFoldedNames ? 'f' : 'n')}{(exact ? 'e' : 'l')}:{string.Join('\u001f', candidateQueries)}"; + + private static HashSet? LoadCandidateCallables( + SqliteConnection connection, + IReadOnlySet symbolColumns, + IReadOnlyList? candidateQueries, + bool exact, + bool useFoldedNames, + IDictionary callableFileIds, + ICollection callables) + { + if (candidateQueries is not { Count: > 0 }) + return null; + + using var command = connection.CreateCommand(); + var clauses = new List(candidateQueries.Count); + for (var index = 0; index < candidateQueries.Count; index++) + { + var parameterName = $"@candidate{index}"; + var query = candidateQueries[index]; + var leaf = SqlNameResolver.GetLeafName(query).TrimStart('@'); + if (leaf.Length == 0) + continue; + + var useFoldedColumn = useFoldedNames && symbolColumns.Contains("name_folded"); + var nameSql = useFoldedColumn ? "s.name_folded" : "s.name"; + var candidate = useFoldedColumn ? NameFold.Fold(leaf) ?? leaf : leaf; + var collation = useFoldedColumn ? "BINARY" : "NOCASE"; + + if (exact) + { + var explicitParameterName = $"@candidateExplicit{index}"; + clauses.Add($"({nameSql} = {parameterName} COLLATE {collation} OR {nameSql} LIKE {explicitParameterName} ESCAPE '\\')"); + SqliteCommandPolicy.Add(command, parameterName, candidate); + SqliteCommandPolicy.Add(command, explicitParameterName, $"%.{EscapeLike(candidate)}"); + } + else + { + clauses.Add($"{nameSql} LIKE {parameterName} ESCAPE '\\'"); + SqliteCommandPolicy.Add(command, parameterName, $"%{EscapeLike(candidate)}%"); + } + } + + if (clauses.Count == 0) + return []; + + var returnTypeSql = symbolColumns.Contains("return_type") ? "s.return_type" : "NULL"; + var startLineSql = symbolColumns.Contains("start_line") + ? "COALESCE(s.start_line, s.line)" + : "s.line"; + var endLineSql = symbolColumns.Contains("end_line") + ? "COALESCE(s.end_line, s.line)" + : "s.line"; + var partialDeclarationSql = BuildPartialDeclarationPredicateSql(symbolColumns); + command.CommandText = $""" + SELECT s.id, + s.file_id, + s.signature, + s.container_qualified_name, + {returnTypeSql}, + {startLineSql}, + {endLineSql} + FROM symbols AS s + JOIN files AS f ON f.id = s.file_id + WHERE f.lang = 'csharp' + AND s.kind IN ('function', 'test.method') + AND {partialDeclarationSql} + AND ({string.Join(" OR ", clauses)}) + LIMIT {CandidateCallableLimit + 1} + """; + var typeNames = new HashSet(StringComparer.Ordinal); + using var reader = command.ExecuteReader(); + var count = 0; + while (reader.Read()) + { + if (++count > CandidateCallableLimit) + return null; + var symbolId = reader.GetInt64(0); + var fileId = reader.GetInt64(1); + callableFileIds[symbolId] = fileId; + callables.Add(new CallableFact( + symbolId, + fileId, + reader.GetInt32(5), + reader.GetInt32(6), + NormalizeIdentity(reader.IsDBNull(3) ? null : reader.GetString(3)))); + AddIdentifiers(typeNames, reader.IsDBNull(2) ? null : reader.GetString(2)); + AddIdentifiers(typeNames, reader.IsDBNull(3) ? null : reader.GetString(3)); + AddIdentifiers(typeNames, reader.IsDBNull(4) ? null : reader.GetString(4)); + if (typeNames.Count > CandidateTypeNameLimit) + return null; + } + return typeNames; + } + + private static List LoadTypeFacts( + SqliteConnection connection, + IReadOnlySet symbolColumns, + IReadOnlySet? typeNames, + IReadOnlyDictionary projectMarkerCounts) + { + var facts = new List(); + if (typeNames is { Count: 0 }) + return facts; + + using var command = connection.CreateCommand(); + var startLineSql = symbolColumns.Contains("start_line") + ? "COALESCE(s.start_line, s.line)" + : "s.line"; + var endLineSql = symbolColumns.Contains("end_line") + ? "COALESCE(s.end_line, s.line)" + : "s.line"; + var familyKeySql = symbolColumns.Contains("family_key") ? "s.family_key" : "NULL"; + var isPartialSql = symbolColumns.Contains("is_partial_declaration") + ? "s.is_partial_declaration" + : "NULL"; + var isFileLocalSql = symbolColumns.Contains("is_file_local_declaration") + ? "s.is_file_local_declaration" + : "NULL"; + var typeNameFilter = string.Empty; + if (typeNames is { Count: > 0 }) + { + var orderedNames = typeNames.OrderBy(name => name, StringComparer.Ordinal).ToArray(); + var parameters = new string[orderedNames.Length]; + for (var index = 0; index < orderedNames.Length; index++) + { + parameters[index] = $"@typeName{index}"; + SqliteCommandPolicy.Add(command, parameters[index], orderedNames[index]); + } + typeNameFilter = $" AND s.name IN ({string.Join(", ", parameters)})"; + } + command.CommandText = $""" + SELECT s.name, + s.container_qualified_name, + s.signature, + s.kind, + {familyKeySql}, + s.file_id, + {startLineSql}, + {endLineSql}, + {isPartialSql}, + {isFileLocalSql}, + f.path + FROM symbols AS s + JOIN files AS f ON f.id = s.file_id + WHERE f.lang = 'csharp' + AND s.name IS NOT NULL + AND s.kind IN ('class', 'struct', 'interface', 'record', 'enum', 'delegate') + {typeNameFilter} + """; + using var reader = command.ExecuteReader(); + while (reader.Read()) + { + var name = reader.GetString(0); + var container = reader.IsDBNull(1) ? null : reader.GetString(1); + var signature = reader.IsDBNull(2) ? null : reader.GetString(2); + var kind = reader.GetString(3); + var familyKey = reader.IsDBNull(4) ? null : reader.GetString(4); + var fileId = reader.GetInt64(5); + var startLine = reader.GetInt32(6); + var endLine = reader.GetInt32(7); + var arity = CSharpTypeReferenceArity.GetDefinitionArity(signature, name, kind) ?? 0; + var ownsFamilyIdentity = OwnsFamilyIdentity(familyKey, name, arity); + var isPartial = reader.IsDBNull(8) + ? ownsFamilyIdentity + : reader.GetBoolean(8); + var isFileLocal = reader.IsDBNull(9) + ? ContainsDeclarationModifier(signature, "file") || IsFileLocalFamily(familyKey) + : reader.GetBoolean(9) || IsFileLocalFamily(familyKey); + var filePath = reader.GetString(10); + var projectScope = ExtractProjectScope(familyKey); + if (projectScope.Length == 0) + projectScope = ResolveProjectScope(filePath, projectMarkerCounts); + facts.Add(new TypeFact( + NormalizeIdentity(name), + NormalizeIdentity(container), + arity, + IsValueTypeDeclaration(signature, kind) ? TypeKind.Value : TypeKind.Reference, + isPartial && ownsFamilyIdentity ? NormalizeFamilyIdentity(familyKey) : string.Empty, + isFileLocal, + projectScope, + fileId, + startLine, + endLine, + signature)); + } + return facts; + } + + private static Dictionary LoadCSharpProjectMarkerCounts(SqliteConnection connection) + { + var pathCaseSensitive = false; + try + { + using var metadataCommand = connection.CreateCommand(); + metadataCommand.CommandText = + "SELECT value FROM codeindex_meta WHERE key = @workspacePathCaseSensitive LIMIT 1"; + SqliteCommandPolicy.Add( + metadataCommand, + "@workspacePathCaseSensitive", + DbContext.WorkspacePathCaseSensitiveMetaKey); + var raw = metadataCommand.ExecuteScalar() as string; + pathCaseSensitive = bool.TryParse(raw, out var parsed) && parsed; + } + catch (SqliteException) + { + // Legacy query-only databases may predate codeindex_meta. Their C# family + // contract is not current, so retain the historical insensitive fallback. + // codeindex_meta より古い query-only DB では current C# family contract を + // 利用できないため、従来の case-insensitive fallback を維持する。 + } + + using var command = connection.CreateCommand(); + // Match the filesystem casing policy that produced the persisted family scopes. + // SQLite LIKE is ASCII case-insensitive regardless of COLLATE, so use an explicit + // binary suffix comparison when the indexed workspace is case-sensitive. + // 永続 family scope を生成した filesystem と同じ大小区別規則で marker を数える。 + // SQLite LIKE は COLLATE にかかわらず ASCII の大小を無視するため、case-sensitive + // workspace では binary suffix 比較を明示する。 + command.CommandText = pathCaseSensitive + ? "SELECT path FROM files WHERE SUBSTR(path, -7) = '.csproj' COLLATE BINARY" + : "SELECT path FROM files WHERE LOWER(SUBSTR(path, -7)) = '.csproj'"; + var markerCounts = new Dictionary(StringComparer.Ordinal); + using var reader = command.ExecuteReader(); + while (reader.Read()) + { + var directory = GetPathDirectory(NormalizeIndexedPath(reader.GetString(0))); + markerCounts.TryGetValue(directory, out var count); + markerCounts[directory] = count + 1; + } + return markerCounts; + } + + private static string ResolveProjectScope( + string filePath, + IReadOnlyDictionary projectMarkerCounts) + { + var normalizedPath = NormalizeIndexedPath(filePath); + var directory = GetPathDirectory(normalizedPath); + while (true) + { + if (projectMarkerCounts.TryGetValue(directory, out var markerCount)) + { + if (markerCount == 1) + return SymbolExtractor.EncodeFamilyScopeKey(directory); + return SymbolExtractor.EncodeFamilyScopeKey( + DeriveAmbiguousProjectScope(normalizedPath, directory)); + } + + if (directory == ".") + break; + directory = GetPathDirectory(directory); + } + + return SymbolExtractor.EncodeFamilyScopeKey( + FileIndexer.DeriveFallbackFamilyScopeKey(normalizedPath)); + } + + private static string DeriveAmbiguousProjectScope(string filePath, string anchorScope) + { + var relativeFromAnchor = anchorScope == "." + ? filePath + : filePath[(anchorScope.Length + 1)..]; + var firstSeparator = relativeFromAnchor.IndexOf('/'); + var childScope = firstSeparator < 0 + ? $"__file__/{relativeFromAnchor}" + : relativeFromAnchor[..firstSeparator]; + return anchorScope == "." ? childScope : $"{anchorScope}/{childScope}"; + } + + private static string NormalizeIndexedPath(string path) + { + // files.path is already persisted with '/' separators by the writer. Any remaining + // backslash is a literal filename character from a Unix workspace and must retain that + // meaning when the database is later opened on another operating system. + // files.path は writer が '/' separator へ正規化して保存済みである。残っている + // backslash は Unix workspace の literal filename character なので、別 OS から + // database を開いた場合もその意味を保持する。 + var normalized = path.Trim('/'); + return normalized.Length == 0 ? "." : normalized; + } + + private static string GetPathDirectory(string path) + { + var separator = path.LastIndexOf('/'); + return separator < 0 ? "." : path[..separator]; + } + + private static void LoadAllCallableFacts( + SqliteConnection connection, + IReadOnlySet symbolColumns, + IDictionary callableFileIds, + ICollection callables) + { + using var command = connection.CreateCommand(); + var startLineSql = symbolColumns.Contains("start_line") + ? "COALESCE(s.start_line, s.line)" + : "s.line"; + var endLineSql = symbolColumns.Contains("end_line") + ? "COALESCE(s.end_line, s.line)" + : "s.line"; + var partialDeclarationSql = BuildPartialDeclarationPredicateSql(symbolColumns); + command.CommandText = $""" + SELECT s.id, + s.file_id, + {startLineSql}, + {endLineSql}, + s.container_qualified_name + FROM symbols AS s + JOIN files AS f ON f.id = s.file_id + WHERE f.lang = 'csharp' + AND s.kind IN ('function', 'test.method') + AND {partialDeclarationSql} + """; + using var reader = command.ExecuteReader(); + while (reader.Read()) + { + var symbolId = reader.GetInt64(0); + var fileId = reader.GetInt64(1); + callableFileIds[symbolId] = fileId; + callables.Add(new CallableFact( + symbolId, + fileId, + reader.GetInt32(2), + reader.GetInt32(3), + NormalizeIdentity(reader.IsDBNull(4) ? null : reader.GetString(4)))); + } + } + + private static string BuildPartialDeclarationPredicateSql( + IReadOnlySet symbolColumns) + { + const string fallbackSql = + "CASE WHEN csharp_is_partial_declaration(s.signature, s.kind, s.name) THEN 1 ELSE 0 END"; + return symbolColumns.Contains("is_partial_declaration") + ? $"COALESCE(s.is_partial_declaration, {fallbackSql}) <> 0" + : $"{fallbackSql} <> 0"; + } + + private static Dictionary BuildCallableTypeParameterKinds( + IReadOnlyCollection callables, + IReadOnlyCollection facts) + { + var result = new Dictionary(); + var factsByFile = facts.GroupBy(fact => fact.FileId) + .ToDictionary(group => group.Key, group => group.ToArray()); + var factsByPartialFamily = facts + .Where(fact => fact.FamilyIdentity.Length > 0) + .GroupBy(BuildPartialTypeFamilyIdentity) + .ToDictionary(group => group.Key, group => group.ToArray()); + foreach (var callable in callables) + { + if (!factsByFile.TryGetValue(callable.FileId, out var fileFacts)) + continue; + + // Process outer declarations before inner declarations so a nested type + // parameter correctly shadows a same-named parameter from its container. + // outer 宣言から inner 宣言の順に処理し、nested type parameter が包含型の + // 同名 parameter を正しく shadow するようにする。 + foreach (var fact in fileFacts + .Where(fact => fact.StartLine <= callable.StartLine + && fact.EndLine >= callable.EndLine) + .Where(fact => IsContainingTypeIdentity( + callable.Container, + BuildUnqualifiedIdentity(fact))) + .OrderByDescending(fact => fact.EndLine - fact.StartLine) + .ThenBy(fact => fact.StartLine)) + { + var parameters = ReadTypeParameterKinds(fact).ToArray(); + if (factsByPartialFamily.TryGetValue( + BuildPartialTypeFamilyIdentity(fact), + out var familyFacts)) + { + // C# permits a generic partial type constraint to appear on only one + // declaration. Merge constraints by ordinal across the logical family, + // while the key keeps project and file-local scopes isolated. + // generic partial type の constraint は一方の宣言だけにも記述できる。 + // project / file-local scope を保った logical family 内で ordinal ごとに + // constraint を統合する。 + foreach (var familyFact in familyFacts) + { + var familyParameters = ReadTypeParameterKinds(familyFact); + if (familyParameters.Count != parameters.Length) + continue; + + for (var index = 0; index < parameters.Length; index++) + { + if (familyParameters[index].Kind == TypeKind.Value) + parameters[index] = parameters[index] with { Kind = TypeKind.Value }; + } + } + } + + foreach (var parameter in parameters) + { + result[new CallableTypeParameterIdentity(callable.SymbolId, parameter.Name)] = parameter.Kind; + } + } + } + + return result; + } + + private static bool IsContainingTypeIdentity(string callableContainer, string typeIdentity) + => callableContainer.Equals(typeIdentity, StringComparison.Ordinal) + || callableContainer.StartsWith($"{typeIdentity}.", StringComparison.Ordinal); + + private static PartialTypeFamilyIdentity BuildPartialTypeFamilyIdentity(TypeFact fact) + => new( + fact.ProjectScope, + fact.FamilyIdentity, + fact.IsFileLocal ? fact.FileId : null); + + private static IReadOnlyList ReadTypeParameterKinds(TypeFact fact) + { + var result = new List(); + if (string.IsNullOrWhiteSpace(fact.Signature)) + return result; + + var declaration = SymbolExtractor.SanitizeCSharpDeclarationSignature(fact.Signature); + if (!TryFindTypeParameterList(declaration, fact.Name, out var open, out var close)) + return result; + + foreach (var parameter in SplitTypeParameterList(declaration[(open + 1)..close])) + { + var name = ReadLastIdentifier(parameter); + if (name.Length > 0) + result.Add(new TypeParameterKind(name, TypeKind.Reference)); + } + + var constraintTokens = ReadIdentifierTokens(declaration[(close + 1)..]); + for (var index = 0; index + 1 < constraintTokens.Count; index++) + { + if (!constraintTokens[index].Equals("where", StringComparison.Ordinal)) + continue; + + var parameterName = constraintTokens[index + 1].TrimStart('@'); + var parameterIndex = result.FindIndex( + parameter => parameter.Name.Equals(parameterName, StringComparison.Ordinal)); + if (parameterIndex < 0) + continue; + + // T? on an unconstrained or reference-constrained type parameter is a + // nullable annotation for callable identity. Only the two unescaped C# + // value constraints change it to Nullable; ordinary escaped identifiers + // such as @struct are type constraints and remain reference-compatible. + // unconstrained / reference constraint の type parameter に対する T? は + // callable identity 上の nullable annotation。Nullable へ変えるのは + // escape なしの2つの C# value constraint だけで、@struct のような通常の + // escaped identifier は type constraint として reference-compatible に保つ。 + var kind = TypeKind.Reference; + for (var constraintIndex = index + 2; + constraintIndex < constraintTokens.Count + && !constraintTokens[constraintIndex].Equals("where", StringComparison.Ordinal); + constraintIndex++) + { + var constraint = constraintTokens[constraintIndex]; + if (constraint is "struct" or "unmanaged") + { + kind = TypeKind.Value; + break; + } + if (constraint == "class") + kind = TypeKind.Reference; + } + result[parameterIndex] = result[parameterIndex] with { Kind = kind }; + } + + return result; + } + + private static bool TryFindTypeParameterList( + string declaration, + string expectedName, + out int open, + out int close) + { + open = -1; + close = -1; + var cursor = 0; + while (TryReadNextIdentifier(declaration, ref cursor, out var keyword, out _)) + { + var normalizedKeyword = keyword; + if (normalizedKeyword is not ("class" or "struct" or "interface" or "record")) + continue; + + if (!TryReadNextIdentifier(declaration, ref cursor, out var declaredName, out var nameEnd)) + return false; + if (normalizedKeyword == "record" && declaredName is "class" or "struct") + { + if (!TryReadNextIdentifier(declaration, ref cursor, out declaredName, out nameEnd)) + return false; + } + if (!declaredName.TrimStart('@').Equals(expectedName.TrimStart('@'), StringComparison.Ordinal)) + continue; + + open = nameEnd; + while (open < declaration.Length && char.IsWhiteSpace(declaration[open])) + open++; + if (open >= declaration.Length || declaration[open] != '<') + return false; + close = FindBalancedAngleEnd(declaration, open); + return close >= 0; + } + + return false; + } + + private static int FindBalancedAngleEnd(string value, int open) + { + var depth = 0; + for (var index = open; index < value.Length; index++) + { + if (value[index] == '<') + depth++; + else if (value[index] == '>' && --depth == 0) + return index; + } + return -1; + } + + private static IReadOnlyList SplitTypeParameterList(string parameters) + { + var result = new List(); + var start = 0; + var angleDepth = 0; + var bracketDepth = 0; + var parenthesisDepth = 0; + for (var index = 0; index < parameters.Length; index++) + { + switch (parameters[index]) + { + case '<': angleDepth++; break; + case '>' when angleDepth > 0: angleDepth--; break; + case '[': bracketDepth++; break; + case ']' when bracketDepth > 0: bracketDepth--; break; + case '(': parenthesisDepth++; break; + case ')' when parenthesisDepth > 0: parenthesisDepth--; break; + case ',' when angleDepth == 0 && bracketDepth == 0 && parenthesisDepth == 0: + result.Add(parameters[start..index]); + start = index + 1; + break; + } + } + result.Add(parameters[start..]); + return result; + } + + private static string ReadLastIdentifier(string value) + { + var tokens = ReadIdentifierTokens(value); + return tokens.Count == 0 ? string.Empty : tokens[^1].TrimStart('@'); + } + + private static List ReadIdentifierTokens(string value) + { + var result = new List(); + var cursor = 0; + while (TryReadNextIdentifier(value, ref cursor, out var identifier, out _)) + result.Add(identifier); + return result; + } + + private static bool TryReadNextIdentifier( + string value, + ref int cursor, + out string identifier, + out int end) + { + identifier = string.Empty; + end = cursor; + while (cursor < value.Length + && !IsIdentifierStart(value[cursor]) + && !(value[cursor] == '@' + && cursor + 1 < value.Length + && IsIdentifierStart(value[cursor + 1]))) + { + cursor++; + } + if (cursor >= value.Length) + return false; + + var start = cursor; + if (value[cursor] == '@') + cursor++; + cursor++; + while (cursor < value.Length && IsIdentifierPart(value[cursor])) + cursor++; + end = cursor; + identifier = value[start..cursor]; + return true; + } + + private static bool IsSimpleIdentifier(string value) + { + var remaining = value.AsSpan().Trim(); + if (!remaining.IsEmpty && remaining[0] == '@') + remaining = remaining[1..]; + if (remaining.IsEmpty || !IsIdentifierStart(remaining[0])) + return false; + for (var index = 1; index < remaining.Length; index++) + { + if (!IsIdentifierPart(remaining[index])) + return false; + } + return true; + } + + private static void AddIdentifiers(ISet names, string? text) + { + if (string.IsNullOrEmpty(text)) + return; + + for (var index = 0; index < text.Length;) + { + if (!IsIdentifierStart(text[index])) + { + index++; + continue; + } + + var start = index++; + while (index < text.Length && IsIdentifierPart(text[index])) + index++; + names.Add(text[start..index].TrimStart('@')); + } + } + + private static bool IsIdentifierStart(char value) + => value == '_' + || char.GetUnicodeCategory(value) is + System.Globalization.UnicodeCategory.UppercaseLetter or + System.Globalization.UnicodeCategory.LowercaseLetter or + System.Globalization.UnicodeCategory.TitlecaseLetter or + System.Globalization.UnicodeCategory.ModifierLetter or + System.Globalization.UnicodeCategory.OtherLetter or + System.Globalization.UnicodeCategory.LetterNumber; + + private static bool IsIdentifierPart(char value) + => IsIdentifierStart(value) + || char.GetUnicodeCategory(value) is + System.Globalization.UnicodeCategory.NonSpacingMark or + System.Globalization.UnicodeCategory.SpacingCombiningMark or + System.Globalization.UnicodeCategory.DecimalDigitNumber or + System.Globalization.UnicodeCategory.ConnectorPunctuation or + System.Globalization.UnicodeCategory.Format; + + private static string EscapeLike(string value) + => value.Replace("\\", "\\\\", StringComparison.Ordinal) + .Replace("%", "\\%", StringComparison.Ordinal) + .Replace("_", "\\_", StringComparison.Ordinal); + + private TypeKind ResolveIdentity(string identity, long? fileId, string projectScope) + { + if (fileId.HasValue) + { + var fileKind = GetUnambiguousKind( + _fileIdentityKinds, + new FileTypeIdentity(fileId.Value, identity)); + if (fileKind != TypeKind.Unknown) + return fileKind; + } + + if (projectScope.Length > 0) + { + var scopedKind = GetUnambiguousKind( + _scopedIdentityKinds, + new ScopedTypeIdentity(projectScope, identity)); + // A scoped miss or ambiguity must not borrow a declaration from another + // project through the repository-global map. + // scoped lookup の miss/ambiguity を別 project の宣言で補完しない。 + return scopedKind; + } + + return GetUnambiguousKind(_identityKinds, identity); + } + + private static bool IsValueTypeDeclaration(string? signature, string kind) + => kind is "struct" or "enum" + || CSharpTypeReferenceArity.IsValueTypeDeclaration(signature, kind); + + private static long ReadTotalChanges(SqliteConnection connection) + { + using var command = connection.CreateCommand(); + command.CommandText = "SELECT total_changes()"; + return Convert.ToInt64(command.ExecuteScalar(), System.Globalization.CultureInfo.InvariantCulture); + } + + private static long ReadDataVersion(SqliteConnection connection) + { + using var command = connection.CreateCommand(); + command.CommandText = "PRAGMA data_version"; + return Convert.ToInt64(command.ExecuteScalar(), System.Globalization.CultureInfo.InvariantCulture); + } + + private static IReadOnlyList ResolveIdentities( + TypeFact fact, + IReadOnlyDictionary containingFacts, + IDictionary> cache, + ISet visiting) + { + if (cache.TryGetValue(fact, out var cached)) + return cached; + + var arityName = AppendArity(fact.Name, fact.Arity); + if (fact.FamilyIdentity.Length > 0) + { + var familyIdentity = CSharpTypeReferenceArity.NormalizeTypeIdentityArity(fact.FamilyIdentity); + var precise = familyIdentity.Length == 0 + ? Array.Empty() + : new[] { familyIdentity, ConvertFamilyIdentityToSourceIdentity(familyIdentity) } + .Distinct(StringComparer.Ordinal) + .ToArray(); + cache[fact] = precise; + return precise; + } + + if (fact.Container.Length == 0 || !visiting.Add(fact)) + { + var root = BuildRootTypeIdentities(fact.Container, arityName); + cache[fact] = root; + return root; + } + + IReadOnlyList resolved; + if (!containingFacts.TryGetValue(fact, out var parent) || parent == null) + { + resolved = BuildRootTypeIdentities(fact.Container, arityName); + } + else + { + var parentIdentities = ResolveIdentities(parent, containingFacts, cache, visiting); + var canonicalIdentity = $"{parentIdentities[0]}+{arityName}"; + resolved = new[] + { + canonicalIdentity, + ConvertFamilyIdentityToSourceIdentity(canonicalIdentity), + } + .Distinct(StringComparer.Ordinal) + .ToArray(); + } + + visiting.Remove(fact); + cache[fact] = resolved; + return resolved; + } + + private static TypeFact? ResolveContainingFact( + TypeFact fact, + IReadOnlyDictionary factsByIdentity) + { + if (fact.Container.Length == 0 + || !factsByIdentity.TryGetValue(fact.Container, out var candidates)) + { + return null; + } + + return candidates + .Where(candidate => candidate.FileId == fact.FileId && candidate != fact) + .Where(candidate => candidate.StartLine <= fact.StartLine && candidate.EndLine >= fact.EndLine) + .OrderBy(candidate => candidate.EndLine - candidate.StartLine) + .ThenByDescending(candidate => candidate.StartLine) + .FirstOrDefault(); + } + + private static string BuildUnqualifiedIdentity(TypeFact fact) + => CombineIdentity(fact.Container, fact.Name); + + private static string CombineIdentity(string container, string name) + => container.Length == 0 ? name : $"{container}.{name}"; + + private static string AppendArity(string name, int arity) + => arity > 0 ? $"{name}`{arity}" : name; + + private static string NormalizeFamilyIdentity(string? familyKey) + { + var normalized = NormalizeIdentity(familyKey); + if (normalized.Length == 0) + return string.Empty; + + // The current C# family contract encodes pipes in the scope, while a file-local identity may still contain + // raw pipes in its source path. Only the first pipe is the scope boundary. + // current C# family contract では scope 内の pipe は encode 済みだが、file-local identity の source path + // には raw pipe が残り得る。scope 境界は最初の pipe だけである。 + var scopeSeparator = normalized.IndexOf('|'); + if (scopeSeparator >= 0) + normalized = normalized[(scopeSeparator + 1)..]; + var fileLocalSeparator = normalized.IndexOf('\u001f'); + if (fileLocalSeparator >= 0) + normalized = normalized[(fileLocalSeparator + 1)..]; + return normalized.StartsWith("file-local:", StringComparison.Ordinal) + ? string.Empty + : normalized; + } + + private static string CombineFamilyContainerAndType(string container, string typeIdentity) + => $"{container}+{typeIdentity}"; + + private static string[] BuildRootTypeIdentities(string container, string arityName) + { + var canonicalIdentity = container.Length == 0 + ? $"+{arityName}" + : $"{container}+{arityName}"; + return new[] + { + canonicalIdentity, + ConvertFamilyIdentityToSourceIdentity(canonicalIdentity), + } + .Distinct(StringComparer.Ordinal) + .ToArray(); + } + + private static string ConvertFamilyIdentityToSourceIdentity(string familyIdentity) + => familyIdentity.Replace('+', '.').TrimStart('.'); + + private static string ExtractProjectScope(string? familyKey) + { + if (string.IsNullOrWhiteSpace(familyKey)) + return string.Empty; + + // ApplyFamilyScope percent-encodes delimiter-bearing path characters before + // persistence, so the first raw pipe remains the unambiguous scope boundary. + // ApplyFamilyScope は delimiter を含む path 文字を永続化前に percent-encode + // するため、最初の raw pipe を曖昧さのない scope 境界として扱える。 + var normalized = familyKey.Trim(); + var scopeSeparator = normalized.IndexOf('|'); + return scopeSeparator > 0 ? normalized[..scopeSeparator] : string.Empty; + } + + private static bool IsFileLocalFamily(string? familyKey) + { + if (string.IsNullOrWhiteSpace(familyKey)) + return false; + + var normalized = familyKey.Trim(); + var scopeSeparator = normalized.IndexOf('|'); + if (scopeSeparator >= 0) + normalized = normalized[(scopeSeparator + 1)..]; + return normalized.StartsWith("file-local:", StringComparison.Ordinal); + } + + private static bool OwnsFamilyIdentity(string? familyKey, string name, int arity) + { + var familyIdentity = NormalizeFamilyIdentity(familyKey); + if (familyIdentity.Length == 0) + return false; + + var separator = familyIdentity.LastIndexOfAny(['.', '+']); + var leaf = separator < 0 ? familyIdentity : familyIdentity[(separator + 1)..]; + return string.Equals( + leaf, + AppendArity(NormalizeIdentity(name), arity), + StringComparison.Ordinal); + } + + private static bool ContainsDeclarationModifier(string? signature, string modifier) + { + if (string.IsNullOrWhiteSpace(signature)) + return false; + + return signature.Split( + [' ', '\t', '\r', '\n', '(', ')', '[', ']', '{', '}', ':'], + StringSplitOptions.RemoveEmptyEntries) + .Contains(modifier, StringComparer.Ordinal); + } + + private static void Add(Dictionary kinds, string identity, TypeKind kind) + { + if (identity.Length == 0) + return; + + var flag = kind == TypeKind.Value ? ValueKindFlag : ReferenceKindFlag; + kinds.TryGetValue(identity, out var existing); + kinds[identity] = existing | flag; + } + + private static TypeKind GetUnambiguousKind(IReadOnlyDictionary kinds, string identity) + { + if (!kinds.TryGetValue(identity, out var flags)) + return TypeKind.Unknown; + return flags switch + { + ReferenceKindFlag => TypeKind.Reference, + ValueKindFlag => TypeKind.Value, + _ => TypeKind.Unknown, + }; + } + + private static void Add(Dictionary kinds, TKey identity, TypeKind kind) + where TKey : notnull + { + var flag = kind == TypeKind.Value ? ValueKindFlag : ReferenceKindFlag; + kinds.TryGetValue(identity, out var existing); + kinds[identity] = existing | flag; + } + + private static TypeKind GetUnambiguousKind(IReadOnlyDictionary kinds, TKey identity) + where TKey : notnull + { + if (!kinds.TryGetValue(identity, out var flags)) + return TypeKind.Unknown; + return flags switch + { + ReferenceKindFlag => TypeKind.Reference, + ValueKindFlag => TypeKind.Value, + _ => TypeKind.Unknown, + }; + } + + private static string NormalizeIdentity(string? identity) + { + if (string.IsNullOrWhiteSpace(identity)) + return string.Empty; + + return CSharpTypeReferenceArity.NormalizeTypeIdentityArity(identity); + } + + private sealed record TypeFact( + string Name, + string Container, + int Arity, + TypeKind Kind, + string FamilyIdentity, + bool IsFileLocal, + string ProjectScope, + long FileId, + int StartLine, + int EndLine, + string? Signature); + + private sealed record CallableFact( + long SymbolId, + long FileId, + int StartLine, + int EndLine, + string Container); + + internal sealed record CandidateScanStats( + bool UsedFullScan, + int CallableCount, + int TypeFactCount); + + private readonly record struct FileTypeIdentity(long FileId, string Identity); + private readonly record struct ScopedTypeIdentity(string ProjectScope, string Identity); + private readonly record struct CallableTypeParameterIdentity(long SymbolId, string ParameterName); + private readonly record struct PartialTypeFamilyIdentity( + string ProjectScope, + string FamilyIdentity, + long? FileId); + private readonly record struct TypeParameterKind(string Name, TypeKind Kind); +} diff --git a/src/CodeIndex/Database/DbContext.ConnectionFunctions.cs b/src/CodeIndex/Database/DbContext.ConnectionFunctions.cs index 131abec64..b33e80a8c 100644 --- a/src/CodeIndex/Database/DbContext.ConnectionFunctions.cs +++ b/src/CodeIndex/Database/DbContext.ConnectionFunctions.cs @@ -4,6 +4,7 @@ using CodeIndex.Models; using Microsoft.Data.Sqlite; using System.Globalization; +using System.Runtime.CompilerServices; using System.Runtime.ExceptionServices; using System.Text.Json; @@ -11,6 +12,15 @@ namespace CodeIndex.Database; public partial class DbContext : IDisposable { + private static readonly ConditionalWeakTable + CSharpCallableTypeKindLookups = new(); + private static readonly ConditionalWeakTable + ConnectionFunctionRegistrations = new(); + private static readonly object ConnectionFunctionRegistrationLock = new(); + private static readonly ConditionalWeakTable + CSharpPartialDeclarationFunctionRegistrations = new(); + private static readonly object CSharpPartialDeclarationFunctionRegistrationLock = new(); + private static SqliteConnection OpenArtifactPreservingQueryOnly(string dbPath) { var connection = CreateArtifactPreservingQueryOnlyConnection( @@ -24,6 +34,18 @@ private static SqliteConnection OpenArtifactPreservingQueryOnly(string dbPath) } internal static void RegisterConnectionFunctions(SqliteConnection connection) + { + lock (ConnectionFunctionRegistrationLock) + { + if (ConnectionFunctionRegistrations.TryGetValue(connection, out _)) + return; + + RegisterConnectionFunctionsCore(connection); + ConnectionFunctionRegistrations.Add(connection, new object()); + } + } + + private static void RegisterConnectionFunctionsCore(SqliteConnection connection) { static int? ToNullableInt(long? value) => value is null || value < int.MinValue || value > int.MaxValue ? null : (int)value.Value; @@ -105,6 +127,53 @@ internal static void RegisterConnectionFunctions(SqliteConnection connection) "csharp_constructor_parameter_count", (string? signature, string? identifier, string? symbolKind) => CSharpTypeReferenceArity.GetConstructorParameterCount(signature, identifier, symbolKind)); + var csharpCallableTypeKinds = CSharpCallableTypeKindLookups.GetValue( + connection, + static _ => new CSharpCallableTypeKindLookup()); + connection.CreateFunction( + "csharp_partial_callable_identity", + (string? signature, string? identifier, string? returnType) => + LogicalPartialSymbolGrouper.BuildCallableIdentity( + signature, + identifier, + returnType, + containerQualifiedName: null, + typeKinds: null), + isDeterministic: true); + connection.CreateFunction( + "csharp_partial_callable_identity", + (string? signature, string? identifier, string? returnType, string? containerQualifiedName) => + LogicalPartialSymbolGrouper.BuildCallableIdentity( + signature, + identifier, + returnType, + containerQualifiedName, + csharpCallableTypeKinds)); + connection.CreateFunction( + "csharp_partial_callable_identity", + (string? signature, string? identifier, string? returnType, string? containerQualifiedName, long? symbolId) => + LogicalPartialSymbolGrouper.BuildCallableIdentity( + signature, + identifier, + returnType, + containerQualifiedName, + csharpCallableTypeKinds, + symbolId)); + connection.CreateFunction( + "csharp_partial_semantic_score", + (string? signature, string? symbolKind) => + LogicalPartialSymbolGrouper.GetSemanticScore(signature, symbolKind), + isDeterministic: true); + connection.CreateFunction( + "csharp_partial_declaration_identity", + (string? signature) => + LogicalPartialSymbolGrouper.BuildCanonicalDeclarationIdentity(signature), + isDeterministic: true); + RegisterCSharpPartialDeclarationFunction(connection); + connection.CreateFunction( + "codeindex_generated_file_name", + (string? path) => FileIndexer.HasGeneratedCodeFileName(path ?? string.Empty), + isDeterministic: true); connection.CreateFunction( "csharp_invocation_argument_count", (string? context, string? identifier, long? columnNumber) => @@ -210,6 +279,48 @@ internal static void RegisterConnectionFunctions(SqliteConnection connection) SqlNameResolver.AllowLeafFallbackAtColumn(symbolName, context, containerName, ToNullableInt(columnNumber)) ? 1 : 0); } + internal static void RegisterCSharpPartialDeclarationFunction(SqliteConnection connection) + { + lock (CSharpPartialDeclarationFunctionRegistrationLock) + { + // DbReader also accepts caller-owned raw connections, so it must ensure this + // function exists. DbContext connections already registered it, however, and + // SQLite rejects replacing a function while any statement is active. + // DbReader は caller-owned raw connection も受け付けるため、この function を + // 保証する。一方 DbContext connection では登録済みであり、active statement 中の + // 再登録を SQLite が拒否するため、connection 単位で一度だけ登録する。 + if (CSharpPartialDeclarationFunctionRegistrations.TryGetValue(connection, out _)) + return; + + connection.CreateFunction( + "csharp_is_partial_declaration", + (string? signature, string? kind, string? name) => + LogicalPartialSymbolGrouper.ContainsPartialModifier(signature, kind, name), + isDeterministic: true); + CSharpPartialDeclarationFunctionRegistrations.Add(connection, new object()); + } + } + + internal static void RefreshCSharpCallableTypeKinds( + SqliteConnection connection, + IReadOnlySet fileColumns, + IReadOnlySet symbolColumns, + IReadOnlyList? candidateQueries = null, + bool exact = false, + bool useFoldedNames = false) + { + var lookup = CSharpCallableTypeKindLookups.GetValue( + connection, + static _ => new CSharpCallableTypeKindLookup()); + lookup.RefreshIfChanged( + connection, + fileColumns, + symbolColumns, + candidateQueries, + exact, + useFoldedNames); + } + internal static int CountCSharpIdentifierOccurrences(string? text, string? identifier) { if (string.IsNullOrEmpty(text) || string.IsNullOrEmpty(identifier)) diff --git a/src/CodeIndex/Database/DbContext.ReadMigrations.cs b/src/CodeIndex/Database/DbContext.ReadMigrations.cs index 157b22ba7..f37d64a66 100644 --- a/src/CodeIndex/Database/DbContext.ReadMigrations.cs +++ b/src/CodeIndex/Database/DbContext.ReadMigrations.cs @@ -246,6 +246,10 @@ PRIMARY KEY(reference_id, symbol_id) yield return ("EnsureColumn symbols.family_key", () => EnsureColumn("symbols", "family_key", "TEXT")); yield return ("EnsureColumn symbols.visibility", () => EnsureColumn("symbols", "visibility", "TEXT")); yield return ("EnsureColumn symbols.return_type", () => EnsureColumn("symbols", "return_type", "TEXT")); + yield return ("EnsureColumn symbols.is_partial_declaration", () => EnsureColumn("symbols", "is_partial_declaration", "INTEGER")); + yield return ("EnsureColumn symbols.is_file_local_declaration", () => EnsureColumn("symbols", "is_file_local_declaration", "INTEGER")); + yield return ("EnsureColumn symbols.declaration_semantic_score", () => EnsureColumn("symbols", "declaration_semantic_score", "INTEGER")); + yield return ("EnsureColumn symbols.identifier_start_column", () => EnsureColumn("symbols", "identifier_start_column", "INTEGER")); yield return ("EnsureColumn symbols.is_metadata_target", () => EnsureColumn("symbols", "is_metadata_target", "INTEGER")); yield return ("EnsureColumn symbols.metadata_target_source", () => EnsureColumn("symbols", "metadata_target_source", "TEXT")); yield return ("CREATE INDEX idx_symbols_name_nocase", diff --git a/src/CodeIndex/Database/DbContext.SchemaInitialization.cs b/src/CodeIndex/Database/DbContext.SchemaInitialization.cs index b84020008..2ff523b9f 100644 --- a/src/CodeIndex/Database/DbContext.SchemaInitialization.cs +++ b/src/CodeIndex/Database/DbContext.SchemaInitialization.cs @@ -136,6 +136,10 @@ container_kind TEXT CHECK (container_kind IS NULL OR container_kind IN (" + sym family_key TEXT, visibility TEXT, return_type TEXT, + is_partial_declaration INTEGER, + is_file_local_declaration INTEGER, + declaration_semantic_score INTEGER, + identifier_start_column INTEGER, is_metadata_target INTEGER, metadata_target_source TEXT, name_folded TEXT, @@ -214,6 +218,10 @@ private void MigrateCoreTableColumns() EnsureColumn("symbols", "family_key", "TEXT"); EnsureColumn("symbols", "visibility", "TEXT"); EnsureColumn("symbols", "return_type", "TEXT"); + EnsureColumn("symbols", "is_partial_declaration", "INTEGER"); + EnsureColumn("symbols", "is_file_local_declaration", "INTEGER"); + EnsureColumn("symbols", "declaration_semantic_score", "INTEGER"); + EnsureColumn("symbols", "identifier_start_column", "INTEGER"); EnsureColumn("file_issues", "origin", "TEXT"); EnsureColumn("file_issues", "severity", "TEXT"); EnsureColumn("symbols", "is_metadata_target", "INTEGER"); @@ -438,13 +446,17 @@ container_kind TEXT CHECK (container_kind IS NULL OR container_kind IN ({symbol family_key TEXT, visibility TEXT, return_type TEXT, + is_partial_declaration INTEGER, + is_file_local_declaration INTEGER, + declaration_semantic_score INTEGER, + identifier_start_column INTEGER, is_metadata_target INTEGER, metadata_target_source TEXT, name_folded TEXT, display_name_folded TEXT ) """, - "id, file_id, kind, sub_kind, name, line, start_line, start_column, end_line, body_start_line, body_end_line, signature, container_kind, container_name, container_qualified_name, family_key, visibility, return_type, is_metadata_target, metadata_target_source, name_folded, display_name_folded"); + "id, file_id, kind, sub_kind, name, line, start_line, start_column, end_line, body_start_line, body_end_line, signature, container_kind, container_name, container_qualified_name, family_key, visibility, return_type, is_partial_declaration, is_file_local_declaration, declaration_semantic_score, identifier_start_column, is_metadata_target, metadata_target_source, name_folded, display_name_folded"); RebuildReferenceLineTablesWithRequiredFileId(); RebuildTableWithRequiredFileId( "file_issues", @@ -733,13 +745,17 @@ container_kind TEXT CHECK (container_kind IS NULL OR container_kind IN ({symbol family_key TEXT, visibility TEXT, return_type TEXT, + is_partial_declaration INTEGER, + is_file_local_declaration INTEGER, + declaration_semantic_score INTEGER, + identifier_start_column INTEGER, is_metadata_target INTEGER, metadata_target_source TEXT, name_folded TEXT, display_name_folded TEXT ) """; - const string symbolsColumns = "id, file_id, kind, sub_kind, name, line, start_line, start_column, end_line, body_start_line, body_end_line, signature, container_kind, container_name, container_qualified_name, family_key, visibility, return_type, is_metadata_target, metadata_target_source, name_folded, display_name_folded"; + const string symbolsColumns = "id, file_id, kind, sub_kind, name, line, start_line, start_column, end_line, body_start_line, body_end_line, signature, container_kind, container_name, container_qualified_name, family_key, visibility, return_type, is_partial_declaration, is_file_local_declaration, declaration_semantic_score, identifier_start_column, is_metadata_target, metadata_target_source, name_folded, display_name_folded"; var symbolReferencesCreateSql = $""" CREATE TABLE symbol_references ( diff --git a/src/CodeIndex/Database/DbContext.SchemaMetadata.cs b/src/CodeIndex/Database/DbContext.SchemaMetadata.cs index de1351fc3..0542e988f 100644 --- a/src/CodeIndex/Database/DbContext.SchemaMetadata.cs +++ b/src/CodeIndex/Database/DbContext.SchemaMetadata.cs @@ -44,10 +44,15 @@ public partial class DbContext : IDisposable // (`family_key` / `container_qualified_name` are authoritative for the whole DB), not // low-level table availability. // hotspots family grouping 用 readiness。table の有無ではなく query 意味論の trust を表す。 - public const int HotspotFamilyVersion = 2; + public const int LegacyHotspotFamilyVersion = 2; + public const int HotspotFamilyVersion = 14; public const string HotspotFamilyVersionMetaKey = "hotspot_family_version"; public const string HotspotFamilyMarkerFingerprintMetaKey = "hotspot_family_marker_fingerprint"; public const string HotspotFamilyIncompleteMarkerFingerprintPrefix = "incomplete:"; + public static int GetHotspotFamilyVersion(string lang) + => string.Equals(lang, "csharp", StringComparison.Ordinal) + ? HotspotFamilyVersion + : LegacyHotspotFamilyVersion; public static string GetHotspotFamilyVersionMetaKey(string lang) => $"hotspot_family_version_{lang}"; public static string GetHotspotFamilyMarkerFingerprintMetaKey(string lang) => $"hotspot_family_marker_fingerprint_{lang}"; public static bool IsIncompleteHotspotFamilyMarkerFingerprint(string? fingerprint) @@ -62,13 +67,18 @@ public static string BuildIncompleteHotspotFamilyMarkerFingerprint(string? finge public const string SqlGraphContractVersionMetaKey = "sql_graph_contract_version"; public const int HdlGraphContractVersion = 1; public const string HdlGraphContractVersionMetaKey = "hdl_graph_contract_version"; - // Version 6 (#4850) invalidates C# instantiate candidates written before constructor - // callables and logical partial-type families had separate identities. Version 5 - // (#4846) previously made Markdown fragment resolution document/path-scoped. - // バージョン 6 (#4850) では、constructor callable と logical partial type family の - // identity 分離前に書かれた C# instantiate candidate を無効化する。バージョン 5 - // (#4846) では Markdown fragment 解決を document/path 内に限定した。 - public const int ReferenceIdentityContractVersion = 6; + // Version 8 (#4914) invalidates C# candidates whose partial family identity did not + // distinguish namespace boundaries from nested-type boundaries. Version 7 (#4914) + // invalidated candidates written before file-local partial families carried source-file + // identity. Version 6 (#4850) previously separated constructor + // callables from logical partial-type families; version 5 (#4846) made Markdown fragment + // resolution document/path-scoped. + // バージョン 8 (#4914) では namespace 境界と nested-type 境界を区別しない partial family + // identity を持つ C# candidate を無効化する。バージョン 7 (#4914) では、file-local partial + // family が source-file identity を持つ前の C# candidate を無効化する。バージョン 6 (#4850) は constructor callable と logical + // partial type family を分離し、バージョン 5 (#4846) は Markdown fragment 解決を + // document/path 内に限定した。 + public const int ReferenceIdentityContractVersion = 8; public const string ReferenceIdentityContractVersionMetaKey = "reference_identity_contract_version"; public static string GetDynamicReferenceGraphContractVersionMetaKey(string lang) => $"dynamic_reference_graph_contract_version_{lang}"; diff --git a/src/CodeIndex/Database/DbContext.cs b/src/CodeIndex/Database/DbContext.cs index f65e6a38e..ea5ddbf87 100644 --- a/src/CodeIndex/Database/DbContext.cs +++ b/src/CodeIndex/Database/DbContext.cs @@ -194,6 +194,10 @@ private static readonly (string Table, string Column)[] ReadMigrationRequiredCol ("symbols", "family_key"), ("symbols", "visibility"), ("symbols", "return_type"), + ("symbols", "is_partial_declaration"), + ("symbols", "is_file_local_declaration"), + ("symbols", "declaration_semantic_score"), + ("symbols", "identifier_start_column"), ("symbols", "is_metadata_target"), ("symbols", "metadata_target_source"), ("symbols", "name_folded"), diff --git a/src/CodeIndex/Database/DbReader.GraphQueries.cs b/src/CodeIndex/Database/DbReader.GraphQueries.cs index b5905dd7b..bfdf3d824 100644 --- a/src/CodeIndex/Database/DbReader.GraphQueries.cs +++ b/src/CodeIndex/Database/DbReader.GraphQueries.cs @@ -1412,9 +1412,7 @@ private static string BuildImpactTraversalNodeKey(long? symbolId, string name) && rootDefinitions.All(definition => definition.Lang == "csharp") && rootDefinitions.All(definition => definition.SymbolId != null) && rootDefinitionResolution.LogicalCount == rootDefinitions.Count - ? rootDefinitions - .Select(definition => definition.SymbolId!.Value) - .ToHashSet() + ? rootDefinitionResolution.PhysicalSymbolIds.ToHashSet() : []; if (hasResolvedIdentityGraph && rootDefinitionPaths.Count > 1 @@ -1457,7 +1455,8 @@ private static string BuildImpactTraversalNodeKey(long? symbolId, string name) queue.Enqueue((resolvedName, null, rootTraversalNodeKey, 0)); } visited.Add(resolvedName); - var truncated = false; + var truncated = qualifiedCSharpRootSymbolIds.Count > 0 + && rootDefinitionResolution.PhysicalSymbolIdsTruncated; var maxDepthReached = false; var cycles = new List(); var cycleKeys = new HashSet(StringComparer.Ordinal); @@ -1467,7 +1466,7 @@ private static string BuildImpactTraversalNodeKey(long? symbolId, string name) // (the input graph is likely pathological). See Issue #1533. // truncatedReason は強い方の信号を保持する: safety_cap は --limit を緩和しても解消しない // ことを示すため、user_limit より優先する (#1533)。 - string? truncatedReason = null; + string? truncatedReason = truncated ? ImpactTruncatedReasons.SafetyCap : null; // Safety cap to prevent infinite loops on pathological graphs / 病的グラフでの無限ループ防止 const int maxFetchIterations = 1000; var graphStateEntryBudget = GetImpactGraphStateEntryBudget(resultWindowEnd); @@ -2440,7 +2439,9 @@ private sealed record ImpactDefinitionResolution( int PreciseDefinitionCount, int PreciseDefinitionFileCount, int NonCallableDefinitionCount, - SymbolResult? SinglePreciseDefinition); + SymbolResult? SinglePreciseDefinition, + HashSet PhysicalSymbolIds, + bool PhysicalSymbolIdsTruncated); private ImpactDefinitionResolution ResolveImpactDefinitions( string resolvedName, @@ -2455,17 +2456,41 @@ private ImpactDefinitionResolution ResolveImpactDefinitions( var leafName = SqlNameResolver.GetLeafName(resolvedName); var segmentCount = SqlNameResolver.GetSegmentCount(resolvedName); var allowLeafFallback = !SqlNameResolver.HasQualifier(resolvedName); + EnsureCSharpCallableTypeKinds(lang, [leafName], exact: true); using var cmd = _conn.CreateCommand(); var supportedLangFilter = BuildGraphSupportedLanguagePredicate(cmd, "f", "impactDefLang"); + var signatureSql = GetSymbolColumnSql("signature"); + var returnTypeSql = GetSymbolColumnSql("return_type"); + var bodyStartLineSql = GetSymbolColumnSql("body_start_line"); + var bodyEndLineSql = GetSymbolColumnSql("body_end_line"); + var startColumnSql = GetSymbolColumnSql("start_column"); + var identifierStartColumnSql = GetSymbolColumnSql("identifier_start_column"); var logicalPartialKeySql = LogicalPartialSymbolGrouper.BuildSqlKeyExpression( "f.lang", "s.kind", "s.name", "s.id", - GetSymbolColumnSql("signature"), + "f.path", + signatureSql, GetSymbolColumnSql("container_name"), GetSymbolColumnSql("container_qualified_name"), - GetSymbolColumnSql("family_key")); + GetSymbolColumnSql("family_key"), + returnTypeSql, + GetSymbolColumnSql("is_partial_declaration"), + _hotspotFamilyReadyLanguages.Contains("csharp")); + var generatedSql = _fileColumns.Contains("generated") + ? "CASE WHEN COALESCE(f.generated, 0) <> 0 OR codeindex_generated_file_name(f.path) THEN 1 ELSE 0 END" + : "CASE WHEN codeindex_generated_file_name(f.path) THEN 1 ELSE 0 END"; + var canonicalPrimaryRankSql = LogicalPartialSymbolGrouper.BuildSqlPrimaryRankExpression( + "s.kind", + bodyStartLineSql, + bodyEndLineSql); + var canonicalSemanticScoreSql = LogicalPartialSymbolGrouper.BuildSqlSemanticScoreExpression( + signatureSql, + "s.kind", + GetSymbolColumnSql("declaration_semantic_score")); + var fallbackCanonicalDeclarationIdentitySql = BuildCanonicalDeclarationIdentitySql(signatureSql); + var canonicalDeclarationIdentitySql = $"CASE WHEN s.kind IN ('function', 'test.method') THEN COALESCE(csharp_partial_callable_identity({signatureSql}, s.name, {returnTypeSql}), {fallbackCanonicalDeclarationIdentitySql}) ELSE {fallbackCanonicalDeclarationIdentitySql} END"; var nameCondition = _foldReady ? allowLeafFallback ? $"({BuildPersistedFoldedNameMatchSql("s.name_folded", "@resolvedNameFolded")} OR (f.lang = 'sql' AND ((sql_segment_count(s.name) = @resolvedNameSegmentCount AND sql_normalize_name_folded(s.name) = @resolvedNameNormalizedFolded) OR sql_leaf_name_folded(s.name) = @resolvedNameLeafFolded)))" @@ -2497,14 +2522,15 @@ ELSE 5 var matchingSql = $@" SELECT f.path, f.lang, s.kind, s.name, s.line, {GetSymbolColumnSql("start_line", "s.line")} AS start_line, + {startColumnSql} AS start_column, {GetSymbolColumnSql("end_line", "s.line")} AS end_line, - {GetSymbolColumnSql("body_start_line")} AS body_start_line, - {GetSymbolColumnSql("body_end_line")} AS body_end_line, - {GetSymbolColumnSql("signature")} AS signature, + {bodyStartLineSql} AS body_start_line, + {bodyEndLineSql} AS body_end_line, + {signatureSql} AS signature, {GetSymbolColumnSql("container_kind")} AS container_kind, {GetSymbolColumnSql("container_name")} AS container_name, {GetSymbolColumnSql("visibility")} AS visibility, - {GetSymbolColumnSql("return_type")} AS return_type, + {returnTypeSql} AS return_type, {GetSymbolColumnSql("container_qualified_name")} AS container_qualified_name, {logicalPartialKeySql} AS logical_partial_key, s.id AS symbol_id, @@ -2512,7 +2538,13 @@ ELSE 5 {PathBucketOrder} AS path_bucket, {VisibilityOrder} AS visibility_rank, CASE WHEN s.kind IN ('class', 'struct', 'interface') THEN 1 ELSE 0 END AS is_precise, - CASE WHEN s.kind IN ('namespace', 'import') THEN 1 ELSE 0 END AS is_non_callable + CASE WHEN s.kind IN ('namespace', 'import') THEN 1 ELSE 0 END AS is_non_callable, + {canonicalPrimaryRankSql} AS canonical_primary_rank, + {generatedSql} AS canonical_generated_rank, + {canonicalSemanticScoreSql} AS canonical_semantic_score, + {canonicalDeclarationIdentitySql} AS canonical_declaration_identity, + COALESCE({GetSymbolColumnSql("start_column")}, 2147483647) AS stable_start_column, + {identifierStartColumnSql} AS identifier_start_column FROM symbols s JOIN files f ON s.file_id = f.id WHERE {nameCondition} @@ -2527,7 +2559,8 @@ FROM symbols s var precisePathDistinctSql = ReferenceEquals(GetIndexedPathComparer(), StringComparer.Ordinal) ? "COUNT(DISTINCT CASE WHEN is_precise = 1 THEN path END)" : "COUNT(DISTINCT CASE WHEN is_precise = 1 THEN path END COLLATE NOCASE)"; - const string representativeOrder = "match_order, path_bucket, visibility_rank, name, path COLLATE BINARY, line, symbol_id"; + const string canonicalRepresentativeOrder = "canonical_primary_rank, canonical_generated_rank, canonical_semantic_score DESC, canonical_declaration_identity COLLATE BINARY, path COLLATE BINARY, start_line, stable_start_column, symbol_id"; + const string resultOrder = "match_order, path_bucket, visibility_rank, name, path COLLATE BINARY, line, symbol_id"; var sql = $@" WITH matching_definitions AS ( {matchingSql} @@ -2536,27 +2569,73 @@ ranked_definitions AS ( SELECT matching_definitions.*, ROW_NUMBER() OVER ( PARTITION BY logical_partial_key - ORDER BY {representativeOrder} + ORDER BY {canonicalRepresentativeOrder} ) AS logical_row_number, + ROW_NUMBER() OVER ( + PARTITION BY logical_partial_key + ORDER BY path COLLATE BINARY, start_line, stable_start_column, symbol_id + ) AS family_member_row_number, COUNT(*) OVER (PARTITION BY logical_partial_key) AS logical_definition_sites FROM matching_definitions ), + family_ranked_definitions AS ( + SELECT ranked_definitions.*, + MAX(CASE WHEN logical_row_number = 1 THEN family_member_row_number END) OVER ( + PARTITION BY logical_partial_key + ) AS representative_member_row_number + FROM ranked_definitions + ), + family_metadata_definitions AS ( + SELECT family_ranked_definitions.*, + MIN(canonical_primary_rank) OVER (PARTITION BY logical_partial_key) AS logical_primary_rank_min, + MAX(canonical_primary_rank) OVER (PARTITION BY logical_partial_key) AS logical_primary_rank_max, + MIN(canonical_generated_rank) OVER (PARTITION BY logical_partial_key) AS logical_generated_rank_min, + MAX(canonical_generated_rank) OVER (PARTITION BY logical_partial_key) AS logical_generated_rank_max, + MIN(canonical_semantic_score) OVER (PARTITION BY logical_partial_key) AS logical_semantic_score_min, + MAX(canonical_semantic_score) OVER (PARTITION BY logical_partial_key) AS logical_semantic_score_max, + MIN(canonical_declaration_identity) OVER (PARTITION BY logical_partial_key) AS logical_declaration_identity_min, + MAX(canonical_declaration_identity) OVER (PARTITION BY logical_partial_key) AS logical_declaration_identity_max, + json_group_array(json_object( + 'symbol_id', symbol_id, + 'path', path, + 'line', line, + 'start_line', start_line, + 'start_column', start_column, + 'end_line', end_line, + 'name', name, + 'signature', signature, + 'identifier_start_column', identifier_start_column, + 'generated', canonical_generated_rank + )) FILTER (WHERE + family_member_row_number <= CASE + WHEN representative_member_row_number <= {LogicalPartialSymbolGrouper.FamilyMemberLimit} + THEN {LogicalPartialSymbolGrouper.FamilyMemberLimit} + ELSE {LogicalPartialSymbolGrouper.FamilyMemberLimit - 1} + END + OR logical_row_number = 1 + ) OVER ( + PARTITION BY logical_partial_key + ORDER BY path COLLATE BINARY, start_line, stable_start_column, symbol_id + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING + ) AS logical_family_members_json + FROM family_ranked_definitions + ), logical_definitions AS ( SELECT * - FROM ranked_definitions + FROM family_metadata_definitions WHERE logical_row_number = 1 ), requested_definitions AS ( SELECT logical_partial_key, 1 AS requested_row FROM logical_definitions - ORDER BY {representativeOrder} + ORDER BY {resultOrder} LIMIT @definitionLimit OFFSET @definitionOffset ), single_precise_definition AS ( SELECT logical_partial_key, 0 AS requested_row FROM logical_definitions WHERE is_precise = 1 - ORDER BY {representativeOrder} + ORDER BY {resultOrder} LIMIT 1 ), selected_definition_keys AS ( @@ -2581,19 +2660,29 @@ SELECT COUNT(*) AS physical_count, FROM matching_definitions ) SELECT logical.path, logical.lang, logical.kind, logical.name, logical.line, - logical.start_line, logical.end_line, + logical.start_line, logical.start_column, logical.end_line, logical.body_start_line, logical.body_end_line, logical.signature, logical.container_kind, logical.container_name, logical.visibility, logical.return_type, logical.container_qualified_name, logical.logical_partial_key, logical.symbol_id, logical.logical_definition_sites, selected.requested_row, stats.physical_count, stats.physical_file_count, stats.logical_count, - stats.precise_count, stats.precise_file_count, stats.non_callable_count + stats.precise_count, stats.precise_file_count, stats.non_callable_count, + CASE + WHEN logical.logical_primary_rank_min <> logical.logical_primary_rank_max THEN '{LogicalPartialSymbolGrouper.ImplementationBodyReason}' + WHEN logical.logical_generated_rank_min <> logical.logical_generated_rank_max THEN '{LogicalPartialSymbolGrouper.NonGeneratedSourceReason}' + WHEN logical.logical_semantic_score_min <> logical.logical_semantic_score_max THEN '{LogicalPartialSymbolGrouper.SemanticDeclarationReason}' + WHEN logical.logical_declaration_identity_min <> logical.logical_declaration_identity_max THEN '{LogicalPartialSymbolGrouper.CanonicalDeclarationIdentityReason}' + ELSE '{LogicalPartialSymbolGrouper.StableLocationReason}' + END AS representative_reason, + logical.logical_family_members_json, + CASE WHEN logical.logical_definition_sites > {LogicalPartialSymbolGrouper.FamilyMemberLimit} THEN 1 ELSE 0 END AS family_members_truncated, + logical.identifier_start_column FROM selected_definition_keys selected JOIN logical_definitions logical ON logical.logical_partial_key = selected.logical_partial_key CROSS JOIN definition_stats stats - ORDER BY {representativeOrder}"; + ORDER BY {resultOrder}"; cmd.CommandText = sql; SqliteCommandPolicy.Add(cmd, "@resolvedName", resolvedName); @@ -2628,7 +2717,7 @@ CROSS JOIN definition_stats stats using var reader = cmd.ExecuteTrackedReader(); while (reader.TrackedRead()) { - var definitionSites = reader.GetInt32(17); + var definitionSites = reader.GetInt32(18); var result = new SymbolResult { Path = reader.GetString(0), @@ -2637,31 +2726,88 @@ CROSS JOIN definition_stats stats Name = reader.GetString(3), Line = reader.GetInt32(4), StartLine = !reader.IsDBNull(5) ? reader.GetInt32(5) : reader.GetInt32(4), - EndLine = !reader.IsDBNull(6) ? reader.GetInt32(6) : reader.GetInt32(4), - BodyStartLine = !reader.IsDBNull(7) ? reader.GetInt32(7) : null, - BodyEndLine = !reader.IsDBNull(8) ? reader.GetInt32(8) : null, - Signature = !reader.IsDBNull(9) ? reader.GetString(9) : null, - ContainerKind = !reader.IsDBNull(10) ? reader.GetString(10) : null, - ContainerName = !reader.IsDBNull(11) ? reader.GetString(11) : null, - ContainerQualifiedName = !reader.IsDBNull(14) ? reader.GetString(14) : null, - LogicalPartialKey = !reader.IsDBNull(15) ? reader.GetString(15) : null, - Visibility = !reader.IsDBNull(12) ? reader.GetString(12) : null, - ReturnType = !reader.IsDBNull(13) ? reader.GetString(13) : null, - SymbolId = reader.GetInt64(16), + StartColumn = !reader.IsDBNull(29) + ? reader.GetInt32(29) + : ResolveSymbolIdentifierStartColumn( + !reader.IsDBNull(6) ? reader.GetInt32(6) : null, + !reader.IsDBNull(10) ? reader.GetString(10) : null, + reader.GetString(3), + reader.GetString(2)), + EndLine = !reader.IsDBNull(7) ? reader.GetInt32(7) : reader.GetInt32(4), + BodyStartLine = !reader.IsDBNull(8) ? reader.GetInt32(8) : null, + BodyEndLine = !reader.IsDBNull(9) ? reader.GetInt32(9) : null, + Signature = !reader.IsDBNull(10) ? reader.GetString(10) : null, + ContainerKind = !reader.IsDBNull(11) ? reader.GetString(11) : null, + ContainerName = !reader.IsDBNull(12) ? reader.GetString(12) : null, + ContainerQualifiedName = !reader.IsDBNull(15) ? reader.GetString(15) : null, + LogicalPartialKey = !reader.IsDBNull(16) ? reader.GetString(16) : null, + Visibility = !reader.IsDBNull(13) ? reader.GetString(13) : null, + ReturnType = !reader.IsDBNull(14) ? reader.GetString(14) : null, + SymbolId = reader.GetInt64(17), DefinitionSites = definitionSites > 1 ? definitionSites : null, }; - physicalCount = reader.GetInt32(19); - physicalFileCount = reader.GetInt32(20); - logicalCount = reader.GetInt32(21); - preciseCount = reader.GetInt32(22); - preciseFileCount = reader.GetInt32(23); - nonCallableCount = reader.GetInt32(24); + if (definitionSites > 1) + { + result.PartialFamilyId = LogicalPartialSymbolGrouper.BuildPartialFamilyId(result.LogicalPartialKey!); + result.RepresentativeReason = reader.GetString(26); + result.FamilyMembers = ReadPartialFamilyMembers(reader.GetString(27), result); + result.FamilyMembersTruncated = reader.GetInt64(28) != 0; + } + physicalCount = reader.GetInt32(20); + physicalFileCount = reader.GetInt32(21); + logicalCount = reader.GetInt32(22); + preciseCount = reader.GetInt32(23); + preciseFileCount = reader.GetInt32(24); + nonCallableCount = reader.GetInt32(25); if (IsPreciseImpactFallbackKind(result.Kind)) preciseDefinition ??= result; - if (reader.GetInt32(18) == 1) + if (reader.GetInt32(19) == 1) results.Add(result); } + // Qualified C# graph traversal is identity-scoped. Keep the representative-only + // definition payload, but retain every selected physical family ID internally so a + // call resolved to a partial declaration reaches the same graph as its implementation. + // qualified C# のグラフ探索は identity 単位で行う。definition の出力は代表1件のまま + // としつつ、partial 宣言側へ解決された call も実装側と同じグラフへ到達できるよう、 + // 選択された family の全 physical ID を内部的に保持する。 + reader.Dispose(); + var physicalSymbolIds = new HashSet(); + var physicalSymbolIdsTruncated = false; + foreach (var definition in results) + { + AddPhysicalSymbolId(definition.SymbolId!.Value); + if (physicalSymbolIdsTruncated || definition.DefinitionSites is not > 1) + continue; + + if (!definition.FamilyMembersTruncated) + { + foreach (var member in definition.FamilyMembers ?? []) + { + if (member.SymbolId is long memberSymbolId) + AddPhysicalSymbolId(memberSymbolId); + if (physicalSymbolIdsTruncated) + break; + } + continue; + } + + var (familySymbolIds, familyIdsTruncated) = ResolveImpactPhysicalFamilySymbolIds( + definition, + logicalPartialKeySql, + lang, + pathPatterns, + excludePathPatterns, + excludeTests); + foreach (var familySymbolId in familySymbolIds) + { + AddPhysicalSymbolId(familySymbolId); + if (physicalSymbolIdsTruncated) + break; + } + physicalSymbolIdsTruncated |= familyIdsTruncated; + } + return new ImpactDefinitionResolution( results, physicalCount, @@ -2670,7 +2816,70 @@ CROSS JOIN definition_stats stats preciseCount, preciseFileCount, nonCallableCount, - preciseCount == 1 ? preciseDefinition : null); + preciseCount == 1 ? preciseDefinition : null, + physicalSymbolIds, + physicalSymbolIdsTruncated); + + void AddPhysicalSymbolId(long symbolId) + { + if (physicalSymbolIds.Contains(symbolId)) + return; + if (physicalSymbolIds.Count >= DefaultImpactGraphStateEntryBudget) + { + physicalSymbolIdsTruncated = true; + return; + } + physicalSymbolIds.Add(symbolId); + } + } + + private (List SymbolIds, bool Truncated) ResolveImpactPhysicalFamilySymbolIds( + SymbolResult definition, + string logicalPartialKeySql, + string? lang, + IReadOnlyList? pathPatterns, + IReadOnlyList? excludePathPatterns, + bool excludeTests) + { + using var cmd = _conn.CreateCommand(); + var supportedLangFilter = BuildGraphSupportedLanguagePredicate(cmd, "f", "impactFamilyLang"); + var familyKindPredicate = definition.Kind is "function" or "test.method" + ? "s.kind IN ('function', 'test.method')" + : "s.kind = @familyKind"; + var sql = $@" + SELECT s.id + FROM symbols s + JOIN files f ON s.file_id = f.id + WHERE f.lang = @familyLang + AND {familyKindPredicate} + AND s.name = @familyName COLLATE BINARY + AND ({logicalPartialKeySql}) = @logicalPartialKey + AND {supportedLangFilter}"; + if (lang != null) + sql += " AND f.lang = @lang"; + AppendPathFilters(ref sql, pathPatterns, excludePathPatterns, excludeTests); + sql += " ORDER BY s.id LIMIT @familyMemberLimit"; + + cmd.CommandText = sql; + SqliteCommandPolicy.Add(cmd, "@familyLang", definition.Lang!); + if (definition.Kind is not ("function" or "test.method")) + SqliteCommandPolicy.Add(cmd, "@familyKind", definition.Kind); + SqliteCommandPolicy.Add(cmd, "@familyName", definition.Name); + SqliteCommandPolicy.Add(cmd, "@logicalPartialKey", definition.LogicalPartialKey!); + if (lang != null) + SqliteCommandPolicy.Add(cmd, "@lang", lang); + SqliteCommandPolicy.Add(cmd, "@familyMemberLimit", DefaultImpactGraphStateEntryBudget + 1); + AddPathFilterParameters(cmd, pathPatterns, excludePathPatterns); + + var symbolIds = new List(); + using var reader = cmd.ExecuteTrackedReader(); + while (reader.TrackedRead()) + { + if (symbolIds.Count >= DefaultImpactGraphStateEntryBudget) + return (symbolIds, true); + symbolIds.Add(reader.GetInt64(0)); + } + return (symbolIds, false); } // C# convention: a class `FooAttribute` is used in source as `[Foo]`, so the reference diff --git a/src/CodeIndex/Database/DbReader.References.cs b/src/CodeIndex/Database/DbReader.References.cs index 493586cf3..1828d53f9 100644 --- a/src/CodeIndex/Database/DbReader.References.cs +++ b/src/CodeIndex/Database/DbReader.References.cs @@ -174,6 +174,7 @@ internal ReferencePositionResolution GetReferencePositionResolution( return new ReferencePositionResolution(false, false, []); } + EnsureCSharpCallableTypeKinds(candidateQueries: [symbolName], exact: true); using var txn = _conn.BeginTransaction(deferred: true); using var cmd = _conn.CreateCommand(); var startLineSql = GetSymbolColumnSql("start_line", "s.line"); @@ -184,10 +185,14 @@ internal ReferencePositionResolution GetReferencePositionResolution( "s.kind", "s.name", "s.id", + "target_file.path", signatureSql, GetSymbolColumnSql("container_name"), GetSymbolColumnSql("container_qualified_name"), - GetSymbolColumnSql("family_key")); + GetSymbolColumnSql("family_key"), + GetSymbolColumnSql("return_type"), + GetSymbolColumnSql("is_partial_declaration"), + _hotspotFamilyReadyLanguages.Contains("csharp")); cmd.CommandText = $@" SELECT target_file.path, target_file.lang, @@ -244,7 +249,11 @@ GROUP BY s.id Name = name, Line = reader.GetInt32(5), StartLine = GetInt32OrFallback(reader, 6, 5), - StartColumn = ResolveSymbolIdentifierStartColumn(GetNullableInt32(reader, 7), signature, name), + StartColumn = ResolveSymbolIdentifierStartColumn( + GetNullableInt32(reader, 7), + signature, + name, + reader.GetString(2)), EndLine = GetInt32OrFallback(reader, 8, 5), BodyStartLine = GetNullableInt32(reader, 9), BodyEndLine = GetNullableInt32(reader, 10), diff --git a/src/CodeIndex/Database/DbReader.cs b/src/CodeIndex/Database/DbReader.cs index 8807fa7d4..f20ebd4d8 100644 --- a/src/CodeIndex/Database/DbReader.cs +++ b/src/CodeIndex/Database/DbReader.cs @@ -484,7 +484,8 @@ public DbReader(DbContext context) context.DatabasePermissionPolicyName, context.DatabasePermissionDiagnostics, context.QueryOnlySnapshotRequiresRefresh - || (context.ImmutableReadOnly && !context.ImmutableReadOnlyWalRisk)) + || (context.ImmutableReadOnly && !context.ImmutableReadOnlyWalRisk), + connectionFunctionsAlreadyRegistered: true) { } @@ -516,7 +517,8 @@ public DbReader(DbContext context, CancellationToken cancellation) context.DatabasePermissionPolicyName, context.DatabasePermissionDiagnostics, context.QueryOnlySnapshotRequiresRefresh - || (context.ImmutableReadOnly && !context.ImmutableReadOnlyWalRisk)) + || (context.ImmutableReadOnly && !context.ImmutableReadOnlyWalRisk), + connectionFunctionsAlreadyRegistered: true) { } @@ -563,16 +565,25 @@ private DbReader( long? walCheckpointRemainingPageCount = null, string databasePermissionPolicy = DatabasePermissionPolicy.BestEffortName, IReadOnlyList? databasePermissionDiagnostics = null, - bool databaseFileSnapshotStable = false) + bool databaseFileSnapshotStable = false, + bool connectionFunctionsAlreadyRegistered = false) { _conn = connection; _commandCache = commandCache; RecoverInterruptedFtsBulkLoadForRead(_conn, isReadOnly); - // SQL user functions are registered once per connection by `DbContext` when the - // connection is opened. Re-registering on every `DbReader` construction wasted CPU - // on hot MCP/CLI paths that build a short-lived reader per request (#1564). - // SQL ユーザー関数は接続オープン時に `DbContext` が一度だけ登録するため、 - // ここでの再登録は不要 (#1564)。 + // DbContext registers every SQL user function when it opens a connection. Public + // raw-connection constructors do not have that guarantee, so register the same full + // set before any reader query can prepare a statement. The explicit flag preserves + // the no-reregistration hot path for DbContext-backed readers (#1564/#4914). + // DbContext は接続開始時に全 SQL user function を登録するが、public な raw connection + // constructor にはその保証がない。reader query が statement を prepare する前に同じ + // 完全な集合を登録し、DbContext-backed reader は明示 flag で再登録を避ける (#1564/#4914)。 + if (!connectionFunctionsAlreadyRegistered) + { + DbContext.RegisterConnectionFunctionsWithRetry( + connection, + cancellationToken: cancellation); + } _isReadOnly = isReadOnly; _readOnlyFallback = readOnlyFallback; _walCheckpointAttempted = walCheckpointAttempted; @@ -664,6 +675,11 @@ private DbReader( TryGetMetaString(_conn, DbContext.ReferenceIdentityContractVersionMetaKey), DbContext.ReferenceIdentityContractVersion.ToString(System.Globalization.CultureInfo.InvariantCulture), StringComparison.Ordinal); + // Public DbReader constructors accept caller-owned raw connections that have not + // necessarily passed through DbContext's full function registration path. + // public DbReader は DbContext の function 登録を通っていない caller-owned raw + // connection も受け付けるため、readiness が直接使う UDF はここでも保証する。 + DbContext.RegisterCSharpPartialDeclarationFunction(connection); (_hotspotFamilyReadyLanguages, _incompleteHotspotFamilyLanguages) = LoadHotspotFamilyReadiness(connection); // NOTE: row presence is intentionally NOT used as a fallback. A legacy DB or an @@ -683,6 +699,30 @@ private DbReader( (_indexNewerThanReader, _indexNewerThanReaderReason) = DetectNewerThanReaderContracts(_conn, userVersion); } + private void EnsureCSharpCallableTypeKinds( + string? lang = null, + IReadOnlyList? candidateQueries = null, + bool exact = false, + string? kind = null) + { + if (!_hotspotFamilyReadyLanguages.Contains("csharp") + || (lang != null && !string.Equals(lang, "csharp", StringComparison.OrdinalIgnoreCase)) + || (kind != null + && !string.Equals(kind, "function", StringComparison.OrdinalIgnoreCase) + && !string.Equals(kind, "test.method", StringComparison.OrdinalIgnoreCase))) + { + return; + } + + DbContext.RefreshCSharpCallableTypeKinds( + _conn, + _fileColumns, + _symbolColumns, + candidateQueries, + exact, + _foldReady); + } + private static void RecoverInterruptedFtsBulkLoadForRead(SqliteConnection conn, bool isReadOnly) { if (isReadOnly) @@ -782,7 +822,7 @@ internal static (bool Newer, string? Reason) DetectNewerThanReaderContracts(Sqli newerContracts); } foreach (var lang in FileIndexer.GetHotspotFamilyMarkerLanguages()) - AppendIfStoredGreater(conn, DbContext.GetHotspotFamilyVersionMetaKey(lang), DbContext.HotspotFamilyVersion, $"hotspot_family_version_{lang}", newerContracts); + AppendIfStoredGreater(conn, DbContext.GetHotspotFamilyVersionMetaKey(lang), DbContext.GetHotspotFamilyVersion(lang), $"hotspot_family_version_{lang}", newerContracts); // PRAGMA user_version is a bitmap of readiness flags. A bit outside the known // `CurrentSchemaVersion` mask means a newer cdidx introduced a readiness flag this @@ -864,7 +904,7 @@ private string ResolveHotspotFamilyDegradedReason(string lang) return DegradationReasonCodes.HotspotFamilySupportNotIndexed; if (!int.TryParse(raw, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var version) - || version != DbContext.HotspotFamilyVersion) + || version != DbContext.GetHotspotFamilyVersion(lang)) { return DegradationReasonCodes.HotspotFamilyMetadataStale; } @@ -1035,7 +1075,7 @@ FROM files var fingerprint = TryGetMetaString(conn, DbContext.GetHotspotFamilyMarkerFingerprintMetaKey(lang)); if (raw is string s && int.TryParse(s, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var version) - && version == DbContext.HotspotFamilyVersion + && version == DbContext.GetHotspotFamilyVersion(lang) && !string.IsNullOrWhiteSpace(fingerprint) && !DbContext.IsIncompleteHotspotFamilyMarkerFingerprint(fingerprint)) { @@ -1057,12 +1097,42 @@ FROM files HotspotFamilyReadinessBatchForTesting?.Invoke(candidateLangs); - // Detect every mixed NULL/non-NULL family in one grouped scan. The previous - // correlated EXISTS probe rescanned symbols for every stamped language (including - // languages absent from the workspace), making reader construction grow toward - // O(language-count * symbol-count^2) on large indexes. - // NULL/non-NULL が混在する family を全言語まとめて一度の group scan で検出する。 - // 旧 correlated EXISTS は未使用言語まで symbols を反復走査していた。 + // C# has an explicit declaration-level partial fact, so readiness can test the + // actual rows that require a family key. Reconstructing a family with only leaf, + // container, and arity incorrectly joins file-local types and nested types whose + // containing arities differ. Other languages retain the bounded mixed-population + // scan until they expose an equivalent declaration fact. + // C# には declaration 単位の partial 情報があるため、family key が必要な実際の行を + // 直接検査する。leaf・container・arity だけで family を再構築すると、file-local type + // や containing arity が異なる nested type を誤って結合する。ほかの言語は同等の + // declaration 情報を持つまで bounded な混在 population scan を維持する。 + incompleteLangs.UnionWith(LoadIncompleteHotspotFamilyLanguages( + conn, + _symbolColumns, + candidateLangs)); + + foreach (var lang in candidateLangs) + { + if (!incompleteLangs.Contains(lang)) + readyLangs.Add(lang); + } + + return (readyLangs, incompleteLangs); + } + + internal static HashSet LoadIncompleteHotspotFamilyLanguages( + SqliteConnection conn, + IReadOnlySet symbolColumns, + IReadOnlyList candidateLangs) + { + var incompleteLangs = new HashSet(StringComparer.Ordinal); + if (candidateLangs.Count == 0) + return incompleteLangs; + + DbContext.RegisterCSharpPartialDeclarationFunction(conn); + var csharpPartialSql = symbolColumns.Contains("is_partial_declaration") + ? "(s.is_partial_declaration = 1 OR (s.is_partial_declaration IS NULL AND s.kind IN ('class', 'struct', 'interface', 'record', 'function', 'test.method') AND csharp_is_partial_declaration(s.signature, s.kind, s.name)))" + : "(s.kind IN ('class', 'struct', 'interface', 'record', 'function', 'test.method') AND csharp_is_partial_declaration(s.signature, s.kind, s.name))"; using (var cmd = conn.CreateCommand()) { var parameterNames = new string[candidateLangs.Count]; @@ -1073,12 +1143,23 @@ FROM files SqliteCommandPolicy.Add(cmd, parameterName, candidateLangs[index]); } cmd.CommandText = $""" - SELECT DISTINCT grouped.lang + SELECT DISTINCT incomplete.lang FROM ( + SELECT f.lang + FROM symbols s + JOIN files f ON f.id = s.file_id + WHERE f.lang = 'csharp' + AND f.lang IN ({string.Join(", ", parameterNames)}) + AND {csharpPartialSql} + AND NULLIF(TRIM(s.family_key), '') IS NULL + + UNION ALL + SELECT f.lang FROM symbols s JOIN files f ON f.id = s.file_id WHERE f.lang IN ({string.Join(", ", parameterNames)}) + AND f.lang <> 'csharp' AND s.name IS NOT NULL GROUP BY f.lang, @@ -1087,20 +1168,14 @@ GROUP BY COALESCE(s.container_qualified_name, '') HAVING COUNT(s.family_key) > 0 AND COUNT(s.family_key) < COUNT(*) - ) grouped + ) incomplete """; using var reader = cmd.ExecuteTrackedReader(); while (reader.TrackedRead()) incompleteLangs.Add(reader.GetString(0)); } - foreach (var lang in candidateLangs) - { - if (!incompleteLangs.Contains(lang)) - readyLangs.Add(lang); - } - - return (readyLangs, incompleteLangs); + return incompleteLangs; } /// diff --git a/src/CodeIndex/Database/DbSymbolReader.Definitions.cs b/src/CodeIndex/Database/DbSymbolReader.Definitions.cs index 4fb5a9f9f..e04a9e975 100644 --- a/src/CodeIndex/Database/DbSymbolReader.Definitions.cs +++ b/src/CodeIndex/Database/DbSymbolReader.Definitions.cs @@ -151,6 +151,11 @@ public List GetDefinitions(string query, int limit = 20, strin Visibility = symbol.Visibility, ReturnType = symbol.ReturnType, DefinitionSites = symbol.DefinitionSites, + PartialFamilyId = symbol.PartialFamilyId, + RepresentativeReason = symbol.RepresentativeReason, + FamilyMembers = symbol.FamilyMembers, + FamilyMembersTruncated = symbol.FamilyMembersTruncated, + IsGeneratedCode = symbol.IsGeneratedCode, Disambiguator = BuildDefinitionDisambiguator(symbol), Content = definitionExcerpt.Content, BodyContent = bodyContent, @@ -300,6 +305,12 @@ private static string ExtractParameterType(string parameter) public QueryCountResult CountDefinitionsTotal(string query, string? kind = null, string? lang = null, IReadOnlyList? pathPatterns = null, IReadOnlyList? excludePathPatterns = null, bool excludeTests = false, DateTime? since = null, bool exact = false, IReadOnlyList? visibilityFilters = null, IReadOnlyList? excludeVisibilityFilters = null, bool groupPartials = false) { var normalizedQuery = NormalizeSymbolSearchQueryForSymbolSearch(query, lang, exact); + if (groupPartials) + EnsureCSharpCallableTypeKinds( + DbReader.NormalizeQueryLanguage(lang), + normalizedQuery == null ? null : [normalizedQuery], + exact, + kind); using var cmd = _conn.CreateCommand(); var logicalPartialKeySql = LogicalPartialSymbolGrouper.BuildSqlKeyExpression( @@ -307,10 +318,14 @@ public QueryCountResult CountDefinitionsTotal(string query, string? kind = null, "s.kind", "s.name", "s.id", + "f.path", GetSymbolColumnSql("signature"), GetSymbolColumnSql("container_name"), GetSymbolColumnSql("container_qualified_name"), - GetSymbolColumnSql("family_key")); + GetSymbolColumnSql("family_key"), + GetSymbolColumnSql("return_type"), + GetSymbolColumnSql("is_partial_declaration"), + _hotspotFamilyReadyLanguages.Contains("csharp")); var countSql = groupPartials ? $"COUNT(DISTINCT ({logicalPartialKeySql}))" : "COUNT(*)"; diff --git a/src/CodeIndex/Database/DbSymbolReader.Search.cs b/src/CodeIndex/Database/DbSymbolReader.Search.cs index 091c5e933..37d812f66 100644 --- a/src/CodeIndex/Database/DbSymbolReader.Search.cs +++ b/src/CodeIndex/Database/DbSymbolReader.Search.cs @@ -1,4 +1,5 @@ using System.Globalization; +using System.Text.Json; using System.Text.RegularExpressions; using CodeIndex.Indexer; using Microsoft.Data.Sqlite; @@ -401,6 +402,9 @@ public QueryCountResult CountSearchSymbolsTotal(string? query = null, string? ki public QueryCountResult CountSearchSymbolsTotal(IReadOnlyList? queries, string? kind = null, string? lang = null, IReadOnlyList? pathPatterns = null, IReadOnlyList? excludePathPatterns = null, bool excludeTests = false, DateTime? since = null, bool exact = false, IReadOnlyList? visibilityFilters = null, IReadOnlyList? excludeVisibilityFilters = null, bool groupPartials = false) { lang = DbReader.NormalizeQueryLanguage(lang); + var effectiveQueries = NormalizeSymbolSearchQueries(queries, lang, exact); + if (groupPartials) + EnsureCSharpCallableTypeKinds(lang, effectiveQueries, exact, kind); using var cmd = _conn.CreateCommand(); var logicalPartialKeySql = LogicalPartialSymbolGrouper.BuildSqlKeyExpression( @@ -408,10 +412,14 @@ public QueryCountResult CountSearchSymbolsTotal(IReadOnlyList? queries, "s.kind", "s.name", "s.id", + "f.path", GetSymbolColumnSql("signature"), GetSymbolColumnSql("container_name"), GetSymbolColumnSql("container_qualified_name"), - GetSymbolColumnSql("family_key")); + GetSymbolColumnSql("family_key"), + GetSymbolColumnSql("return_type"), + GetSymbolColumnSql("is_partial_declaration"), + _hotspotFamilyReadyLanguages.Contains("csharp")); var countSql = groupPartials ? $"COUNT(DISTINCT ({logicalPartialKeySql}))" : "COUNT(*)"; @@ -421,7 +429,6 @@ FROM symbols s JOIN files f ON s.file_id = f.id WHERE 1=1"; - var effectiveQueries = NormalizeSymbolSearchQueries(queries, lang, exact); if (effectiveQueries != null && effectiveQueries.Count > 0) { var orClauses = exact @@ -598,6 +605,8 @@ public List SearchSymbols(IReadOnlyList? queries, int limi return merged.Skip(Math.Max(0, offset)).Take(limit).ToList(); } + if (groupPartials) + EnsureCSharpCallableTypeKinds(lang, validQueries, exact, kind); using var cmd = _conn.CreateCommand(); var startLineSql = GetSymbolColumnSql("start_line", "s.line"); @@ -686,10 +695,27 @@ ELSE 0.0 "s.kind", "s.name", "s.id", + "f.path", signatureSql, containerNameSql, containerQualifiedNameSql, - familyKeySql); + familyKeySql, + returnTypeSql, + GetSymbolColumnSql("is_partial_declaration"), + _hotspotFamilyReadyLanguages.Contains("csharp")); + var generatedSql = _fileColumns.Contains("generated") + ? "CASE WHEN COALESCE(f.generated, 0) <> 0 OR codeindex_generated_file_name(f.path) THEN 1 ELSE 0 END" + : "CASE WHEN codeindex_generated_file_name(f.path) THEN 1 ELSE 0 END"; + var canonicalPrimaryRankSql = LogicalPartialSymbolGrouper.BuildSqlPrimaryRankExpression( + "s.kind", + bodyStartLineSql, + bodyEndLineSql); + var canonicalSemanticScoreSql = LogicalPartialSymbolGrouper.BuildSqlSemanticScoreExpression( + signatureSql, + "s.kind", + GetSymbolColumnSql("declaration_semantic_score")); + var fallbackCanonicalDeclarationIdentitySql = BuildCanonicalDeclarationIdentitySql(signatureSql); + var canonicalDeclarationIdentitySql = $"CASE WHEN s.kind IN ('function', 'test.method') THEN COALESCE(csharp_partial_callable_identity({signatureSql}, s.name, {returnTypeSql}), {fallbackCanonicalDeclarationIdentitySql}) ELSE {fallbackCanonicalDeclarationIdentitySql} END"; var exactNameOrderSql = "CASE " + "WHEN @preferLiteralExactMatch = 1 AND s.name = @rawQuery THEN 0 " + "WHEN @preferLiteralNormalizedSqlMatch = 1 AND f.lang = 'sql' AND sql_segment_count(s.name) = @rawQuerySegmentCount AND sql_normalize_name(s.name) = @rawQueryNormalized THEN 1 " + @@ -725,7 +751,12 @@ ELSE 0.0 {exactNameOrderSql} AS exact_name_order, {PathBucketOrder} AS path_bucket, {VisibilityOrder} AS visibility_rank, - {startColumnSql} AS stable_start_column + {startColumnSql} AS stable_start_column, + {canonicalPrimaryRankSql} AS canonical_primary_rank, + {generatedSql} AS canonical_generated_rank, + {canonicalSemanticScoreSql} AS canonical_semantic_score, + {canonicalDeclarationIdentitySql} AS canonical_declaration_identity, + {GetSymbolColumnSql("identifier_start_column")} AS identifier_start_column FROM symbols s JOIN files f ON s.file_id = f.id {symbolRankJoin} @@ -895,12 +926,13 @@ FROM symbols s var includeRankingMetadata = sortMode != SymbolSortMode.Name; var sortModeName = sortMode.ToString().ToLowerInvariant(); + var identifierStartColumnIndex = groupPartials ? 31 : 36; var results = new List(); using var reader = cmd.ExecuteTrackedReader(); while (reader.TrackedRead()) { var definitionSites = Convert.ToInt32(reader.GetInt64(22)); - results.Add(new SymbolResult + var result = new SymbolResult { Path = reader.GetString(0), Lang = GetNullableString(reader, 1), @@ -909,10 +941,12 @@ FROM symbols s Name = reader.GetString(4), Line = reader.GetInt32(5), StartLine = GetInt32OrFallback(reader, 6, 5), - StartColumn = ResolveSymbolIdentifierStartColumn( - GetNullableInt32(reader, 7), - GetNullableString(reader, 11), - reader.GetString(4)), + StartColumn = GetNullableInt32(reader, identifierStartColumnIndex) + ?? ResolveSymbolIdentifierStartColumn( + GetNullableInt32(reader, 7), + GetNullableString(reader, 11), + reader.GetString(4), + reader.GetString(2)), EndLine = GetInt32OrFallback(reader, 8, 5), BodyStartLine = GetNullableInt32(reader, 9), BodyEndLine = GetNullableInt32(reader, 10), @@ -934,25 +968,88 @@ FROM symbols s SizeLines = includeRankingMetadata ? Convert.ToInt32(reader.GetInt64(23)) : null, ComplexityScore = includeRankingMetadata ? Math.Round(reader.GetDouble(24), 3) : null, SymbolId = reader.GetInt64(27), - }); + }; + if (groupPartials && definitionSites > 1) + { + result.PartialFamilyId = LogicalPartialSymbolGrouper.BuildPartialFamilyId(result.LogicalPartialKey!); + result.RepresentativeReason = reader.GetString(28); + result.FamilyMembers = ReadPartialFamilyMembers(reader.GetString(29), result); + result.FamilyMembersTruncated = reader.GetInt64(30) != 0; + } + results.Add(result); } return results; } - private static int? ResolveSymbolIdentifierStartColumn(int? declarationStartColumn, string? signature, string name) + private static List ReadPartialFamilyMembers(string json, SymbolResult representative) + { + using var document = JsonDocument.Parse(json); + var members = new List(); + foreach (var element in document.RootElement.EnumerateArray()) + { + var symbolId = element.GetProperty("symbol_id").GetInt64(); + var path = element.GetProperty("path").GetString() ?? string.Empty; + var startLine = element.GetProperty("start_line").GetInt32(); + var rawStartColumn = element.GetProperty("start_column").ValueKind == JsonValueKind.Null + ? (int?)null + : element.GetProperty("start_column").GetInt32(); + var memberName = element.GetProperty("name").GetString() ?? representative.Name; + var memberSignature = element.GetProperty("signature").ValueKind == JsonValueKind.Null + ? null + : element.GetProperty("signature").GetString(); + var identifierStartColumn = element.TryGetProperty("identifier_start_column", out var identifierColumnElement) + && identifierColumnElement.ValueKind != JsonValueKind.Null + ? identifierColumnElement.GetInt32() + : (int?)null; + members.Add(new PartialFamilyMember + { + SymbolId = symbolId, + Path = path, + Line = element.GetProperty("line").GetInt32(), + StartLine = startLine, + StartColumn = identifierStartColumn + ?? ResolveSymbolIdentifierStartColumn( + rawStartColumn, + memberSignature, + memberName, + representative.Kind), + EndLine = element.GetProperty("end_line").GetInt32(), + Generated = element.GetProperty("generated").GetInt32() != 0, + Representative = representative.SymbolId == symbolId + || (representative.SymbolId == null + && string.Equals(representative.Path, path, StringComparison.Ordinal) + && representative.StartLine == startLine), + }); + } + return members; + } + + private static int? ResolveSymbolIdentifierStartColumn( + int? declarationStartColumn, + string? signature, + string name, + string kind) { if (!declarationStartColumn.HasValue || string.IsNullOrWhiteSpace(signature) || string.IsNullOrEmpty(name)) return declarationStartColumn; var firstLineEnd = signature.IndexOfAny(['\r', '\n']); var firstLine = firstLineEnd >= 0 ? signature[..firstLineEnd] : signature; - var relativeColumn = firstLine.IndexOf(name, StringComparison.Ordinal); + var callable = kind is "function" or "test.method"; + var relativeColumn = callable + ? LogicalPartialSymbolGrouper.FindCallableNameOffset(firstLine, name) + : firstLine.IndexOf(name, StringComparison.Ordinal); + if (relativeColumn < 0 && callable) + relativeColumn = firstLine.IndexOf(name, StringComparison.Ordinal); return relativeColumn >= 0 ? declarationStartColumn.Value + relativeColumn : declarationStartColumn; } private static string GetGenericSymbolRankNamePenaltySql(string nameSql) => $"CASE WHEN lower({nameSql}) IN {GenericSymbolRankNamesSql} THEN {GenericSymbolRankNamePenaltySqlLiteral} ELSE 1.0 END"; + private static string BuildCanonicalDeclarationIdentitySql(string signatureSql) + => $"csharp_partial_declaration_identity({signatureSql})"; + private static string BuildLogicalPartialSymbolQuery(string matchingSymbolsSql, SymbolSortMode sortMode) { var orderBy = BuildLogicalPartialSortOrderBy(sortMode); @@ -960,13 +1057,36 @@ private static string BuildLogicalPartialSymbolQuery(string matchingSymbolsSql, WITH matching_symbols AS ( {matchingSymbolsSql} ), - logical_symbols AS ( + ranked_symbols AS ( SELECT matching_symbols.*, ROW_NUMBER() OVER ( PARTITION BY logical_partial_key - ORDER BY path COLLATE BINARY, start_line, stable_start_column, symbol_id + ORDER BY canonical_primary_rank, + canonical_generated_rank, + canonical_semantic_score DESC, + canonical_declaration_identity COLLATE BINARY, + logical_partial_key COLLATE BINARY, + path COLLATE BINARY, + start_line, + stable_start_column, + symbol_id ) AS logical_row_number, - COUNT(*) OVER (PARTITION BY logical_partial_key) AS logical_definition_sites, + ROW_NUMBER() OVER ( + PARTITION BY logical_partial_key + ORDER BY path COLLATE BINARY, start_line, stable_start_column, symbol_id + ) AS family_member_row_number, + COUNT(*) OVER (PARTITION BY logical_partial_key) AS logical_definition_sites + FROM matching_symbols + ), + family_ranked_symbols AS ( + SELECT ranked_symbols.*, + MAX(CASE WHEN logical_row_number = 1 THEN family_member_row_number END) OVER ( + PARTITION BY logical_partial_key + ) AS representative_member_row_number + FROM ranked_symbols + ), + logical_symbols AS ( + SELECT family_ranked_symbols.*, MAX(reference_count) OVER (PARTITION BY logical_partial_key) AS logical_reference_count, MAX(hotspot_score) OVER (PARTITION BY logical_partial_key) AS logical_hotspot_score, MAX(ranking_reference_score) OVER (PARTITION BY logical_partial_key) AS logical_ranking_reference_score, @@ -977,8 +1097,39 @@ PARTITION BY logical_partial_key MAX(complexity_score) OVER (PARTITION BY logical_partial_key) AS logical_complexity_score, MIN(exact_name_order) OVER (PARTITION BY logical_partial_key) AS logical_exact_name_order, MIN(path_bucket) OVER (PARTITION BY logical_partial_key) AS logical_path_bucket, - MIN(visibility_rank) OVER (PARTITION BY logical_partial_key) AS logical_visibility_rank - FROM matching_symbols + MIN(visibility_rank) OVER (PARTITION BY logical_partial_key) AS logical_visibility_rank, + MIN(canonical_primary_rank) OVER (PARTITION BY logical_partial_key) AS logical_primary_rank_min, + MAX(canonical_primary_rank) OVER (PARTITION BY logical_partial_key) AS logical_primary_rank_max, + MIN(canonical_generated_rank) OVER (PARTITION BY logical_partial_key) AS logical_generated_rank_min, + MAX(canonical_generated_rank) OVER (PARTITION BY logical_partial_key) AS logical_generated_rank_max, + MIN(canonical_semantic_score) OVER (PARTITION BY logical_partial_key) AS logical_semantic_score_min, + MAX(canonical_semantic_score) OVER (PARTITION BY logical_partial_key) AS logical_semantic_score_max, + MIN(canonical_declaration_identity) OVER (PARTITION BY logical_partial_key) AS logical_declaration_identity_min, + MAX(canonical_declaration_identity) OVER (PARTITION BY logical_partial_key) AS logical_declaration_identity_max, + json_group_array(json_object( + 'symbol_id', symbol_id, + 'path', path, + 'line', line, + 'start_line', start_line, + 'start_column', start_column, + 'end_line', end_line, + 'name', name, + 'signature', signature, + 'identifier_start_column', identifier_start_column, + 'generated', canonical_generated_rank + )) FILTER (WHERE + family_member_row_number <= CASE + WHEN representative_member_row_number <= {LogicalPartialSymbolGrouper.FamilyMemberLimit} + THEN {LogicalPartialSymbolGrouper.FamilyMemberLimit} + ELSE {LogicalPartialSymbolGrouper.FamilyMemberLimit - 1} + END + OR logical_row_number = 1 + ) OVER ( + PARTITION BY logical_partial_key + ORDER BY path COLLATE BINARY, start_line, stable_start_column, symbol_id + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING + ) AS logical_family_members_json + FROM family_ranked_symbols ) SELECT path, lang, kind, sub_kind, name, line, start_line, start_column, end_line, @@ -988,7 +1139,17 @@ FROM matching_symbols logical_ranking_reference_score, logical_ranking_hotspot_score, logical_generic_name_penalty, logical_structural_rank_penalty, logical_definition_sites, logical_size_lines, logical_complexity_score, - container_qualified_name, logical_partial_key, symbol_id + container_qualified_name, logical_partial_key, symbol_id, + CASE + WHEN logical_primary_rank_min <> logical_primary_rank_max THEN '{LogicalPartialSymbolGrouper.ImplementationBodyReason}' + WHEN logical_generated_rank_min <> logical_generated_rank_max THEN '{LogicalPartialSymbolGrouper.NonGeneratedSourceReason}' + WHEN logical_semantic_score_min <> logical_semantic_score_max THEN '{LogicalPartialSymbolGrouper.SemanticDeclarationReason}' + WHEN logical_declaration_identity_min <> logical_declaration_identity_max THEN '{LogicalPartialSymbolGrouper.CanonicalDeclarationIdentityReason}' + ELSE '{LogicalPartialSymbolGrouper.StableLocationReason}' + END AS representative_reason, + logical_family_members_json, + CASE WHEN logical_definition_sites > {LogicalPartialSymbolGrouper.FamilyMemberLimit} THEN 1 ELSE 0 END AS family_members_truncated, + identifier_start_column FROM logical_symbols WHERE logical_row_number = 1 {orderBy}"; diff --git a/src/CodeIndex/Database/DbWriter.ChunkSymbolBatches.cs b/src/CodeIndex/Database/DbWriter.ChunkSymbolBatches.cs index 472ae68e8..f2c453334 100644 --- a/src/CodeIndex/Database/DbWriter.ChunkSymbolBatches.cs +++ b/src/CodeIndex/Database/DbWriter.ChunkSymbolBatches.cs @@ -54,11 +54,12 @@ public void InsertSymbols(IReadOnlyList symbols, CancellationToken { if (symbols.Count == 0) return; cancellationToken.ThrowIfCancellationRequested(); + TrackCurrentWriterCSharpFamilyRows(symbols); _typeScriptAugmentationDirtyNameScope?.TrackInsertedSymbols(symbols, cancellationToken); TrackReferenceGraphInsertedSymbols(symbols); InvalidateReferenceIdentityContractForMutation(); - int rowsPerStatement = GetRowsPerInsertStatement(columnCount: 21); + int rowsPerStatement = GetRowsPerInsertStatement(columnCount: 25); var foldedNameCache = CreateFoldedNameCache( Math.Min(symbols.Count, rowsPerStatement), namesPerRow: 1); @@ -82,6 +83,25 @@ public void InsertSymbols(IReadOnlyList symbols, CancellationToken CheckBatchCancellationAndReportProgress("insert_symbols", symbols.Count, symbols.Count, cancellationToken); } + private void TrackCurrentWriterCSharpFamilyRows(IReadOnlyList symbols) + { + if (_currentWriterOwnsAllCSharpFamilyRows != true) + return; + + foreach (var symbol in symbols) + { + if (!_currentWriterCSharpFileIds.Contains(symbol.FileId)) + continue; + if (symbol.Kind is not ("function" or "test.method" or "class" or "struct" or "interface" or "record" or "enum" or "delegate")) + continue; + if (symbol.IsPartialDeclaration.HasValue) + continue; + + _currentWriterOwnsAllCSharpFamilyRows = false; + return; + } + } + private void InsertChunksWithRowSkip(IReadOnlyList chunks, int start, int end, SqliteException batchException, CancellationToken cancellationToken) { using var transaction = !IsInTransaction() ? BeginTransaction(cancellationToken, "insert chunks row skip") : null; @@ -240,6 +260,14 @@ private void InsertSymbolBatch(IReadOnlyList symbols, int start, i cmd.Parameters[parameterIndex++].Value = (object?)symbol.FamilyKey ?? DBNull.Value; cmd.Parameters[parameterIndex++].Value = (object?)symbol.Visibility ?? DBNull.Value; cmd.Parameters[parameterIndex++].Value = (object?)symbol.ReturnType ?? DBNull.Value; + cmd.Parameters[parameterIndex++].Value = symbol.IsPartialDeclaration.HasValue + ? (symbol.IsPartialDeclaration.Value ? 1 : 0) + : (object)DBNull.Value; + cmd.Parameters[parameterIndex++].Value = symbol.IsFileLocalDeclaration + ? 1 + : 0; + cmd.Parameters[parameterIndex++].Value = (object?)symbol.DeclarationSemanticScore ?? DBNull.Value; + cmd.Parameters[parameterIndex++].Value = (object?)symbol.IdentifierStartColumn ?? DBNull.Value; cmd.Parameters[parameterIndex++].Value = symbol.IsMetadataTarget.HasValue ? (symbol.IsMetadataTarget.Value ? 1 : 0) : (object)DBNull.Value; @@ -296,6 +324,8 @@ INSERT INTO symbols ( body_start_line, body_end_line, signature, container_kind, container_name, container_qualified_name, family_key, visibility, return_type, + is_partial_declaration, is_file_local_declaration, + declaration_semantic_score, identifier_start_column, is_metadata_target, metadata_target_source, name_folded, display_name_folded ) @@ -305,7 +335,7 @@ INSERT INTO symbols ( { if (row > 0) sql.Append(", "); - AppendBatchParameterTuple(sql, ref parameterIndex, columnCount: 21); + AppendBatchParameterTuple(sql, ref parameterIndex, columnCount: 25); } return sql.ToString(); } @@ -333,6 +363,10 @@ private static void AddSymbolInsertParameters(SqliteCommand cmd, int rowCount) AddBatchParameter(cmd, ref parameterIndex, SqliteType.Text); AddBatchParameter(cmd, ref parameterIndex, SqliteType.Text); AddBatchParameter(cmd, ref parameterIndex, SqliteType.Integer); + AddBatchParameter(cmd, ref parameterIndex, SqliteType.Integer); + AddBatchParameter(cmd, ref parameterIndex, SqliteType.Integer); + AddBatchParameter(cmd, ref parameterIndex, SqliteType.Integer); + AddBatchParameter(cmd, ref parameterIndex, SqliteType.Integer); AddBatchParameter(cmd, ref parameterIndex, SqliteType.Text); AddBatchParameter(cmd, ref parameterIndex, SqliteType.Text); AddBatchParameter(cmd, ref parameterIndex, SqliteType.Text); diff --git a/src/CodeIndex/Database/DbWriter.ContractMetadata.cs b/src/CodeIndex/Database/DbWriter.ContractMetadata.cs index 6dda80ccb..65b0c6bfc 100644 --- a/src/CodeIndex/Database/DbWriter.ContractMetadata.cs +++ b/src/CodeIndex/Database/DbWriter.ContractMetadata.cs @@ -113,6 +113,64 @@ public bool ReferenceIdentityContractMatchesCurrent() DbContext.ReferenceIdentityContractVersion.ToString(System.Globalization.CultureInfo.InvariantCulture), StringComparison.Ordinal); + /// + /// Return whether the current reference identity contract may trust every persisted C# family row. + /// A partial upgrade cannot publish the current contract while untouched legacy C# symbols remain. + /// Callers may supply an authoritative C# presence result to avoid another database probe; + /// a writer that started without other C# rows may also trust the current rows it inserted. + /// Standalone full graph refreshes may opt into validating unstamped persisted rows directly. + /// current reference identity contract が全 C# family row を信頼できるかを返す。 + /// 未更新の旧 C# symbol が残る部分 upgrade では current contract を公開しない。 + /// caller は既知の C# presence を渡して再照会を避けられ、既存 C# row がない状態から + /// current row を挿入した writer はその session 内の row も信頼できる。standalone の + /// full graph refresh は未 stamp の保存済み row を直接検証する方式も選べる。 + /// + internal bool CSharpFamilyTrustAllowsReferenceIdentityReady( + bool? hasCSharpFiles = null, + bool validatePersistedRows = false) + { + if (!(hasCSharpFiles ?? HasAnyFilesWithLanguage("csharp"))) + return true; + + var version = GetMetaString(DbContext.GetHotspotFamilyVersionMetaKey("csharp")); + var fingerprint = GetMetaString(DbContext.GetHotspotFamilyMarkerFingerprintMetaKey("csharp")); + var stampedFamilyTrustIsCurrent = string.Equals( + version, + DbContext.GetHotspotFamilyVersion("csharp").ToString( + System.Globalization.CultureInfo.InvariantCulture), + StringComparison.Ordinal) + && !string.IsNullOrWhiteSpace(fingerprint) + && !DbContext.IsIncompleteHotspotFamilyMarkerFingerprint(fingerprint); + if (stampedFamilyTrustIsCurrent || _currentWriterOwnsAllCSharpFamilyRows == true) + return true; + + return validatePersistedRows && PersistedCSharpFamilyRowsAreComplete(); + } + + private bool PersistedCSharpFamilyRowsAreComplete() + { + DbContext.RegisterCSharpPartialDeclarationFunction(_conn); + using var cmd = _conn.CreateCommand(); + cmd.Transaction = _activeTransaction; + cmd.CommandText = """ + SELECT NOT EXISTS ( + SELECT 1 + FROM symbols s + JOIN files f ON f.id = s.file_id + WHERE f.lang = 'csharp' + AND s.kind IN ('function', 'test.method', 'class', 'struct', 'interface', 'record') + AND ( + (s.is_partial_declaration IS NULL + AND csharp_is_partial_declaration(s.signature, s.kind, s.name)) + OR (s.is_partial_declaration = 1 + AND NULLIF(TRIM(s.family_key), '') IS NULL) + ) + LIMIT 1 + ) + """; + return Convert.ToInt64(cmd.ExecuteScalar()) != 0; + } + public void MarkReferenceIdentityContractReady() { SetMeta( @@ -152,7 +210,7 @@ public void MarkHotspotFamilyReady(string lang, string? markerFingerprint = null // Clear the superseded global keys so mixed-version DBs don't leave confusing stale metadata behind. // 廃止した global key を掃除し、混在 DB に紛らわしい古い metadata を残さない。 SetMetaValues( - (DbContext.GetHotspotFamilyVersionMetaKey(lang), DbContext.HotspotFamilyVersion.ToString(System.Globalization.CultureInfo.InvariantCulture)), + (DbContext.GetHotspotFamilyVersionMetaKey(lang), DbContext.GetHotspotFamilyVersion(lang).ToString(System.Globalization.CultureInfo.InvariantCulture)), (DbContext.GetHotspotFamilyMarkerFingerprintMetaKey(lang), markerFingerprint), (DbContext.HotspotFamilyVersionMetaKey, null), (DbContext.HotspotFamilyMarkerFingerprintMetaKey, null)); @@ -161,7 +219,7 @@ public void MarkHotspotFamilyReady(string lang, string? markerFingerprint = null public void MarkHotspotFamilyMarkerFingerprintIncomplete(string lang, string? markerFingerprint) { SetMetaValues( - (DbContext.GetHotspotFamilyVersionMetaKey(lang), DbContext.HotspotFamilyVersion.ToString(System.Globalization.CultureInfo.InvariantCulture)), + (DbContext.GetHotspotFamilyVersionMetaKey(lang), DbContext.GetHotspotFamilyVersion(lang).ToString(System.Globalization.CultureInfo.InvariantCulture)), (DbContext.GetHotspotFamilyMarkerFingerprintMetaKey(lang), DbContext.BuildIncompleteHotspotFamilyMarkerFingerprint(markerFingerprint)), (DbContext.HotspotFamilyVersionMetaKey, null), (DbContext.HotspotFamilyMarkerFingerprintMetaKey, null)); diff --git a/src/CodeIndex/Database/DbWriter.Files.cs b/src/CodeIndex/Database/DbWriter.Files.cs index 3ca46b44a..93285a0cf 100644 --- a/src/CodeIndex/Database/DbWriter.Files.cs +++ b/src/CodeIndex/Database/DbWriter.Files.cs @@ -131,6 +131,7 @@ public long UpsertFile( out bool referenceIdentityChanged, bool cleanExistingData = true) { + TrackCSharpFamilyFileBeforeWrite(file); var typeScriptDirtyNameScope = _typeScriptAugmentationDirtyNameScope; var wasExistingTypeScript = cleanExistingData && typeScriptDirtyNameScope?.TrackExistingFile(file.Path) == true; @@ -188,6 +189,7 @@ ON CONFLICT(path) DO UPDATE SET // RETURNING reader と prepared command を解放してから cleanup command を借りる。 // 既存行は同じIDを保ち、新規行のDELETEはno-op。fresh bulk loadはInsertNewFileを使う。 typeScriptDirtyNameScope?.TrackCurrentFile(fileId, file.Lang, wasExistingTypeScript); + TrackCurrentWriterCSharpFile(fileId, file.Lang); referenceIdentityChanged = cleanExistingData && DeleteFileDataCore(fileId, trackTypeScriptInterfaceNames: false); return fileId; @@ -203,6 +205,7 @@ ON CONFLICT(path) DO UPDATE SET /// public long InsertNewFile(FileRecord file) { + TrackCSharpFamilyFileBeforeWrite(file); var cmd = RentCommand( @" INSERT INTO files (path, lang, size, lines, checksum, modified, generated, indexed_at) @@ -238,10 +241,39 @@ INSERT INTO files (path, lang, size, lines, checksum, modified, generated, index ReleaseCommand(cmd); } _typeScriptAugmentationDirtyNameScope?.TrackCurrentFile(fileId, file.Lang); + TrackCurrentWriterCSharpFile(fileId, file.Lang); TrackReferenceGraphFileIds([fileId]); return fileId; } + private void TrackCSharpFamilyFileBeforeWrite(FileRecord file) + { + if (!string.Equals(file.Lang, "csharp", StringComparison.Ordinal) + || _currentWriterOwnsAllCSharpFamilyRows.HasValue) + { + return; + } + + using var cmd = _conn.CreateCommand(); + cmd.Transaction = _activeTransaction; + cmd.CommandText = """ + SELECT 1 + FROM symbols s + JOIN files f ON f.id = s.file_id + WHERE f.lang = 'csharp' + LIMIT 1 + """; + _currentWriterOwnsAllCSharpFamilyRows = cmd.ExecuteScalar() == null; + } + + private void TrackCurrentWriterCSharpFile(long fileId, string? lang) + { + if (string.Equals(lang, "csharp", StringComparison.Ordinal)) + _currentWriterCSharpFileIds.Add(fileId); + else + _currentWriterCSharpFileIds.Remove(fileId); + } + /// /// Delete old chunks and symbols for a file before re-indexing. /// 再インデックス前にファイルの古いチャンクとシンボルを削除する。 diff --git a/src/CodeIndex/Database/DbWriter.FoldBackfill.cs b/src/CodeIndex/Database/DbWriter.FoldBackfill.cs index 86ee1aada..f25156e0f 100644 --- a/src/CodeIndex/Database/DbWriter.FoldBackfill.cs +++ b/src/CodeIndex/Database/DbWriter.FoldBackfill.cs @@ -373,7 +373,10 @@ private bool ExtractorContractsMatchCurrentForReuse(string? lang) // Refresh them before advertising the rewritten rows as current. // candidate と解決済み identity は永続化 folded key に依存するため、 // 書換え後の key を current と公開する前に graph を再解決する。 - RefreshMutualRecursionFlags(cancellationToken); + RefreshMutualRecursionFlags( + cancellationToken, + stampReferenceIdentityContractReady: + CSharpFamilyTrustAllowsReferenceIdentityReady()); SetMeta(FoldBackfillGraphRefreshPendingMetaKey, null); } if (rewriteAll) diff --git a/src/CodeIndex/Database/DbWriter.References.cs b/src/CodeIndex/Database/DbWriter.References.cs index 14ad08c90..730d4c8ec 100644 --- a/src/CodeIndex/Database/DbWriter.References.cs +++ b/src/CodeIndex/Database/DbWriter.References.cs @@ -129,6 +129,18 @@ PRIMARY KEY(derived_qualified_name, base_qualified_name) private static string BuildCSharpProjectPrefixSql(string symbolAlias) => $""" CASE + WHEN INSTR(COALESCE({symbolAlias}.family_key, ''), CHAR(31)) > 0 + AND ( + SUBSTR(COALESCE({symbolAlias}.family_key, ''), 1, 11) = 'file-local:' + OR SUBSTR( + COALESCE({symbolAlias}.family_key, ''), + INSTR(COALESCE({symbolAlias}.family_key, ''), '|') + 1, + 11) = 'file-local:' + ) + THEN SUBSTR( + {symbolAlias}.family_key, + 1, + INSTR({symbolAlias}.family_key, CHAR(31))) WHEN INSTR(COALESCE({symbolAlias}.family_key, ''), '|') > 0 THEN SUBSTR( {symbolAlias}.family_key, @@ -1967,11 +1979,15 @@ private bool HasPersistedReferenceResolutionState(CancellationToken cancellation } } - internal void RefreshMutualRecursionFlags(CancellationToken cancellationToken = default) + internal void RefreshMutualRecursionFlags( + CancellationToken cancellationToken = default, + bool? stampReferenceIdentityContractReady = null) { cancellationToken.ThrowIfCancellationRequested(); MutualRecursionRefreshForTesting?.Invoke(); cancellationToken.ThrowIfCancellationRequested(); + var stampReferenceIdentityContract = stampReferenceIdentityContractReady + ?? CSharpFamilyTrustAllowsReferenceIdentityReady(validatePersistedRows: true); var graphScope = _referenceGraphRefreshScope; using var transaction = BeginTransaction(cancellationToken, "refresh reference identities"); if (graphScope != null) @@ -2035,10 +2051,15 @@ internal void RefreshMutualRecursionFlags(CancellationToken cancellationToken = ? RefreshMutualRecursionFlagsSql : RefreshScopedMutualRecursionFlagsSql, static _ => { }); - // Stamp inside the same transaction, but before the graph refresh so the - // public SQLite changes() result continues to describe recursion updates. - // 同一トランザクション内で先に marker を設定し、公開 changes() は再帰更新件数を維持する。 - MarkReferenceIdentityContractReady(); + // Reconcile the marker inside the same transaction, but before the graph refresh + // so the public SQLite changes() result continues to describe recursion updates. + // High-level indexing defers v7 while untouched legacy C# family rows remain. + // 同一 transaction 内で先に marker を調整して公開 changes() を維持する。 + // high-level index は未更新の旧 C# family row が残る間 v7 を保留する。 + if (stampReferenceIdentityContract) + MarkReferenceIdentityContractReady(); + else + ClearReferenceIdentityContractReady(); cancellationToken.ThrowIfCancellationRequested(); refreshIdentityCommand.ExecuteNonQuery(); cancellationToken.ThrowIfCancellationRequested(); diff --git a/src/CodeIndex/Database/DbWriter.cs b/src/CodeIndex/Database/DbWriter.cs index ea5122fa5..ab9f624e3 100644 --- a/src/CodeIndex/Database/DbWriter.cs +++ b/src/CodeIndex/Database/DbWriter.cs @@ -172,6 +172,8 @@ internal static Action? BatchStatementExecutingForTestin private string? _transactionOwnerOperation; private DateTimeOffset _transactionOwnerAcquiredAtUtc; private bool? _hasIssueMetadataColumns; + private bool? _currentWriterOwnsAllCSharpFamilyRows; + private readonly HashSet _currentWriterCSharpFileIds = []; internal static TimeSpan? TransactionStateContentionTimeoutForTesting { get; set; } // Outermost SqliteTransaction currently held open by this writer (null when no // transaction is active OR after the outermost transaction has been committed / diff --git a/src/CodeIndex/Database/LogicalPartialSymbolGrouper.cs b/src/CodeIndex/Database/LogicalPartialSymbolGrouper.cs index fbdc241eb..899dc3e2b 100644 --- a/src/CodeIndex/Database/LogicalPartialSymbolGrouper.cs +++ b/src/CodeIndex/Database/LogicalPartialSymbolGrouper.cs @@ -1,35 +1,130 @@ +using System.Security.Cryptography; +using System.Text; +using System.Globalization; +using CodeIndex.Indexer; + namespace CodeIndex.Database; internal static class LogicalPartialSymbolGrouper { private const char KeySeparator = '\u001f'; + private static readonly IReadOnlyDictionary CSharpPredefinedTypeIdentities = + new Dictionary(StringComparer.Ordinal) + { + ["bool"] = "System.Boolean", + ["byte"] = "System.Byte", + ["sbyte"] = "System.SByte", + ["char"] = "System.Char", + ["decimal"] = "System.Decimal", + ["double"] = "System.Double", + ["float"] = "System.Single", + ["int"] = "System.Int32", + ["uint"] = "System.UInt32", + ["nint"] = "System.IntPtr", + ["nuint"] = "System.UIntPtr", + ["long"] = "System.Int64", + ["ulong"] = "System.UInt64", + ["short"] = "System.Int16", + ["ushort"] = "System.UInt16", + ["object"] = "System.Object", + ["dynamic"] = "System.Object", + ["string"] = "System.String", + ["void"] = "System.Void", + }; + private static readonly IReadOnlyDictionary CSharpFrameworkTypeIdentities = + CSharpPredefinedTypeIdentities.Values + .Distinct(StringComparer.Ordinal) + .ToDictionary( + value => value["System.".Length..], + value => value, + StringComparer.Ordinal); + private static readonly HashSet CSharpFrameworkReferenceTypeIdentities = + [ + "global::System.Object", + "global::System.String", + ]; + internal const int FamilyMemberLimit = 50; + internal const string ImplementationBodyReason = "implementation_body"; + internal const string NonGeneratedSourceReason = "non_generated_source"; + internal const string SemanticDeclarationReason = "semantic_declaration"; + internal const string CanonicalDeclarationIdentityReason = "canonical_declaration_identity"; + internal const string StableLocationReason = "stable_path_and_position"; internal static string BuildSqlKeyExpression( string languageSql, string kindSql, string nameSql, string symbolIdSql, + string fileIdentitySql, string signatureSql, string containerNameSql, string containerQualifiedNameSql, - string familyKeySql) + string familyKeySql, + string? returnTypeSql = null, + string? isPartialDeclarationSql = null, + bool csharpFamilyContractReady = true) { + if (!csharpFamilyContractReady) + return $"'symbol:' || {symbolIdSql}"; + var persistedFamilySql = $"NULLIF(TRIM({familyKeySql}), '')"; + var scopedPersistedFamilySql = persistedFamilySql; var fallbackContainerSql = $"COALESCE(NULLIF(TRIM({containerQualifiedNameSql}), ''), NULLIF(TRIM({containerNameSql}), ''), '')"; var normalizedSignatureSql = $"REPLACE(REPLACE(REPLACE(LOWER(COALESCE({signatureSql}, '')), CHAR(9), ' '), CHAR(10), ' '), CHAR(13), ' ')"; - var partialDeclarationSql = $"INSTR(' ' || {normalizedSignatureSql} || ' ', ' partial ') > 0"; + var signaturePartialDeclarationSql = $"INSTR(' ' || {normalizedSignatureSql} || ' ', ' partial ') > 0"; + var partialDeclarationSql = isPartialDeclarationSql == null + ? signaturePartialDeclarationSql + : $"COALESCE({isPartialDeclarationSql}, CASE WHEN {signaturePartialDeclarationSql} THEN 1 ELSE 0 END) <> 0"; var projectPrefixSql = $"CASE WHEN INSTR(COALESCE({persistedFamilySql}, ''), '|') > 0 THEN SUBSTR({persistedFamilySql}, 1, INSTR({persistedFamilySql}, '|')) ELSE '' END"; - var selfFamilySql = $"{projectPrefixSql} || CASE WHEN {fallbackContainerSql} = '' THEN {nameSql} ELSE {fallbackContainerSql} || '.' || {nameSql} END"; + var typeAritySql = $"COALESCE(csharp_definition_type_arity({signatureSql}, {nameSql}, {kindSql}), 0)"; + var typeIdentitySql = $"{nameSql} || CASE WHEN {typeAritySql} > 0 THEN '`' || {typeAritySql} ELSE '' END"; + var reconstructedSelfFamilySql = $"{projectPrefixSql} || CASE WHEN {fallbackContainerSql} = '' THEN {typeIdentitySql} ELSE {fallbackContainerSql} || '.' || {typeIdentitySql} END"; + var selfFamilySql = $"COALESCE({scopedPersistedFamilySql}, {reconstructedSelfFamilySql})"; + // Persisted partial evidence is lexer-aware. Always supply an explicit modifier + // to the callable parser for those rows; a raw `partial` substring may be comment + // trivia, while a duplicated real modifier is harmless to identity extraction. + // 永続化した partial evidence は lexer-aware である。その row では callable + // parser に明示的な modifier を常に渡す。raw な `partial` substring は comment + // trivia の可能性があり、実 modifier の重複は identity 抽出に影響しない。 + var callableSignatureSql = $"CASE WHEN {partialDeclarationSql} THEN 'partial ' || COALESCE({signatureSql}, '') ELSE {signatureSql} END"; + var callableContainerSql = $"COALESCE({scopedPersistedFamilySql}, NULLIF({fallbackContainerSql}, ''))"; + var callableIdentitySql = returnTypeSql == null + ? "NULL" + : $"csharp_partial_callable_identity({callableSignatureSql}, {nameSql}, {returnTypeSql}, {callableContainerSql}, {symbolIdSql})"; return $@"CASE WHEN {languageSql} = 'csharp' AND {kindSql} IN ('class', 'struct', 'interface', 'record') AND {partialDeclarationSql} THEN 'family:' || {languageSql} || CHAR(31) || {kindSql} || CHAR(31) || {selfFamilySql} + WHEN {languageSql} = 'csharp' + AND {kindSql} IN ('function', 'test.method') + AND {partialDeclarationSql} + AND {callableContainerSql} IS NOT NULL + AND {callableIdentitySql} IS NOT NULL + THEN 'family:' || {languageSql} || CHAR(31) || 'function' || CHAR(31) || + {callableContainerSql} || CHAR(31) || {callableIdentitySql} ELSE 'symbol:' || {symbolIdSql} END"; } + internal static string BuildSqlPrimaryRankExpression( + string kindSql, + string bodyStartLineSql, + string bodyEndLineSql) + => $"CASE WHEN {kindSql} IN ('function', 'test.method') AND ({bodyStartLineSql} IS NULL OR {bodyEndLineSql} IS NULL) THEN 1 ELSE 0 END"; + + internal static string BuildSqlSemanticScoreExpression( + string signatureSql, + string kindSql, + string? declarationSemanticScoreSql = null) + { + var signatureScoreSql = $"csharp_partial_semantic_score({signatureSql}, {kindSql})"; + return declarationSemanticScoreSql == null + ? signatureScoreSql + : $"COALESCE({declarationSemanticScoreSql}, {signatureScoreSql})"; + } + public static List Group(IReadOnlyList symbols) where T : SymbolResult { @@ -58,10 +153,16 @@ public static List Group(IReadOnlyList symbols) continue; var representative = group - .OrderBy(result => result.Path, StringComparer.Ordinal) + .OrderBy(GetPrimaryRank) + .ThenBy(IsGeneratedCode) + .ThenByDescending(GetSemanticScore) + .ThenBy(GetCanonicalDeclarationIdentity, StringComparer.Ordinal) + .ThenBy(result => result.Path, StringComparer.Ordinal) .ThenBy(result => result.StartLine) + .ThenBy(result => result.StartColumn ?? int.MaxValue) + .ThenBy(result => result.SymbolId ?? long.MaxValue) .First(); - representative.DefinitionSites = group.Count; + PopulateFamilyMetadata(representative, group, key); results.Add(representative); } @@ -70,40 +171,1998 @@ public static List Group(IReadOnlyList symbols) public static bool TryBuildKey(SymbolResult symbol, out string key) { - if (!string.Equals(symbol.Lang, "csharp", StringComparison.OrdinalIgnoreCase) - || !IsLogicalPartialKind(symbol.Kind) - || string.IsNullOrWhiteSpace(symbol.Signature) - || !ContainsPartialModifier(symbol.Signature)) + if (!string.Equals(symbol.Lang, "csharp", StringComparison.OrdinalIgnoreCase)) { key = string.Empty; return false; } - if (!string.IsNullOrWhiteSpace(symbol.LogicalPartialKey) - && symbol.LogicalPartialKey.StartsWith("family:", StringComparison.Ordinal)) + if (!string.IsNullOrWhiteSpace(symbol.LogicalPartialKey)) { - key = symbol.LogicalPartialKey; + if (symbol.LogicalPartialKey.StartsWith("family:", StringComparison.Ordinal)) + { + key = symbol.LogicalPartialKey; + return true; + } + + // A persisted physical key is authoritative when the index contract is stale. + // Reconstructing a family from the signature here would contradict the SQL + // readiness gate. stale index の physical key は SQL readiness gate の結果なので、 + // signature から family を再構築してはならない。 + key = string.Empty; + return false; + } + + return TryBuildDeclarationKey(symbol, out key); + } + + internal static bool TryBuildTypeFamilyKeyForReferenceResolution( + SymbolResult symbol, + out string key) + { + if (TryBuildKey(symbol, out key)) return true; + + // A stale family contract deliberately exposes physical `symbol:*` rows to + // ordinary grouping, but LSP position resolution still needs to distinguish + // partial type declarations from same-name constructors. Reconstruct only a + // partial *type* identity for that local disambiguation step; never use this + // degraded fallback for result collapsing. + // stale family contract は通常 query では意図的に physical `symbol:*` row を + // 維持する。一方 LSP の位置解決では partial type 宣言と同名 constructor を + // 区別する必要があるため、この局所的な判定に限って partial *type* identity + // を再構築し、result grouping には流用しない。 + if (!IsLogicalPartialTypeKind(symbol.Kind)) + { + key = string.Empty; + return false; + } + + return TryBuildDeclarationKey(symbol, out key); + } + + private static bool TryBuildDeclarationKey(SymbolResult symbol, out string key) + { + + if (string.IsNullOrWhiteSpace(symbol.Signature) + || !ContainsPartialModifier(symbol.Signature, symbol.Kind, symbol.Name)) + { + key = string.Empty; + return false; } var containerIdentity = symbol.ContainerQualifiedName ?? symbol.ContainerName ?? string.Empty; + if (IsLogicalPartialCallableKind(symbol.Kind)) + { + var callableIdentity = BuildCallableIdentity(symbol.Signature, symbol.Name, symbol.ReturnType); + if (callableIdentity == null || string.IsNullOrWhiteSpace(containerIdentity)) + { + key = string.Empty; + return false; + } + + key = string.Join( + KeySeparator, + symbol.Lang?.ToLowerInvariant() ?? string.Empty, + "function", + containerIdentity, + callableIdentity); + return true; + } + + if (!IsLogicalPartialTypeKind(symbol.Kind)) + { + key = string.Empty; + return false; + } + + var genericArity = CSharpTypeReferenceArity.GetDefinitionArity( + symbol.Signature, + symbol.Name, + symbol.Kind); + var typeIdentity = genericArity > 0 + ? $"{symbol.Name}`{genericArity.Value.ToString(System.Globalization.CultureInfo.InvariantCulture)}" + : symbol.Name; key = string.Join( KeySeparator, symbol.Lang?.ToLowerInvariant() ?? string.Empty, symbol.Kind.ToLowerInvariant(), - symbol.Name, + typeIdentity, containerIdentity); return true; } - private static bool ContainsPartialModifier(string signature) + internal static string? BuildCallableIdentity( + string? signature, + string? name, + string? returnType, + string? containerQualifiedName = null, + CSharpCallableTypeKindLookup? typeKinds = null, + long? symbolId = null) { - var tokens = signature.Split( - [' ', '\t', '\r', '\n', '(', ')', '[', ']', '{', '}', ':'], - StringSplitOptions.RemoveEmptyEntries); - return tokens.Contains("partial", StringComparer.Ordinal); + if (string.IsNullOrWhiteSpace(signature) + || string.IsNullOrWhiteSpace(name) + || string.IsNullOrWhiteSpace(returnType) + || !ContainsPartialModifier(signature, "function", name)) + { + return null; + } + + var normalizedName = name.TrimStart('@'); + var nameOffset = FindCallableNameOffset(signature, normalizedName); + if (nameOffset < 0) + return null; + + var cursor = SkipCSharpDeclarationTrivia(signature, nameOffset + name.Length); + + var genericArity = 0; + var genericParameterNames = new List(); + if (cursor < signature.Length && signature[cursor] == '<') + { + var genericEnd = FindBalancedEnd(signature, cursor, '<', '>'); + if (genericEnd < 0) + return null; + genericParameterNames = SplitTopLevel(signature[(cursor + 1)..genericEnd]) + .Select(ExtractGenericParameterName) + .ToList(); + genericArity = genericParameterNames.Count; + cursor = SkipCSharpDeclarationTrivia(signature, genericEnd + 1); + } + + if (cursor >= signature.Length || signature[cursor] != '(') + return null; + + var closeParenthesis = FindBalancedEnd(signature, cursor, '(', ')'); + if (closeParenthesis < 0) + return null; + + var valueConstrainedGenericParameters = ReadValueConstrainedGenericParameters( + signature, + closeParenthesis, + genericParameterNames); + var parameterIdentity = BuildCallableParameterIdentity( + signature[(cursor + 1)..closeParenthesis], + genericParameterNames, + valueConstrainedGenericParameters, + containerQualifiedName, + typeKinds, + symbolId); + return string.Join( + KeySeparator, + normalizedName, + NormalizeCallableTypeIdentity( + RemoveCallableReturnDeclarationTrivia(returnType), + genericParameterNames, + valueConstrainedGenericParameters, + containerQualifiedName, + typeKinds, + symbolId), + genericArity.ToString(System.Globalization.CultureInfo.InvariantCulture), + parameterIdentity); } - private static bool IsLogicalPartialKind(string kind) - => kind is "class" or "struct" or "interface" or "record"; + private static string RemoveCallableReturnDeclarationTrivia(string returnType) + { + var remaining = RemoveLeadingParameterAttributes(RemoveCSharpComments(returnType)); + while (TryReadLeadingIdentifier(remaining, out var modifier, out var modifierLength) + && modifier.ToLowerInvariant() is + "public" or "private" or "protected" or "internal" or + "new" or "static" or "virtual" or "sealed" or "abstract" or + "override" or "extern" or "unsafe" or "async" or "partial") + { + remaining = RemoveLeadingParameterAttributes(remaining[modifierLength..]); + } + + // Keep ref/readonly as part of the return identity; only declaration attributes + // and method modifiers are discarded. This also repairs legacy rows where an + // inline attribute caused those tokens to leak into return_type. + // ref/readonly は return identity の一部として保持し、宣言 attribute と method + // modifier だけを除去する。行内 attribute によりそれらが return_type へ漏れた + // legacy row も同じ処理で補正する。 + return remaining.Trim(); + } + + internal static int GetSemanticScore(string? signature, string? kind) + { + if (string.IsNullOrWhiteSpace(signature)) + return 0; + + var declaration = ExtractCSharpDeclarationHeader( + SymbolExtractor.SanitizeCSharpDeclarationSignature(signature)); + var score = 0; + if (declaration.Contains('[')) + score += 2; + if (IsLogicalPartialTypeKind(kind ?? string.Empty) + && SymbolExtractor.ContainsCSharpTypeBaseList(declaration)) + { + score += 4; + } + if (SymbolExtractor.ContainsCSharpWhereConstraint(declaration)) + score += 1; + return score; + } + + internal static string BuildCanonicalDeclarationIdentity(string? signature) + => NormalizeIdentityToken( + string.IsNullOrWhiteSpace(signature) + ? signature + : ExtractCSharpDeclarationHeader(RemoveCSharpComments(signature))); + + private static string ExtractCSharpDeclarationHeader(string declaration) + { + var parenthesisDepth = 0; + var bracketDepth = 0; + for (var index = 0; index < declaration.Length; index++) + { + switch (declaration[index]) + { + case '(': + parenthesisDepth++; + break; + case ')' when parenthesisDepth > 0: + parenthesisDepth--; + break; + case '[': + bracketDepth++; + break; + case ']' when bracketDepth > 0: + bracketDepth--; + break; + case '{' when parenthesisDepth == 0 && bracketDepth == 0: + return declaration[..index]; + case '=' when parenthesisDepth == 0 + && bracketDepth == 0 + && index + 1 < declaration.Length + && declaration[index + 1] == '>': + return declaration[..index]; + } + } + + return declaration; + } + + internal static string BuildPartialFamilyId(string key) + { + var digest = SHA256.HashData(Encoding.UTF8.GetBytes(key)); + return $"partial:{Convert.ToHexString(digest.AsSpan(0, 12)).ToLowerInvariant()}"; + } + + internal static string ResolveRepresentativeReason(T representative, IReadOnlyList group) + where T : SymbolResult + { + if (group.Any(candidate => GetPrimaryRank(candidate) != GetPrimaryRank(representative))) + return ImplementationBodyReason; + if (group.Any(candidate => IsGeneratedCode(candidate) != IsGeneratedCode(representative))) + return NonGeneratedSourceReason; + if (group.Any(candidate => GetSemanticScore(candidate) != GetSemanticScore(representative))) + return SemanticDeclarationReason; + if (group.Any(candidate => !string.Equals( + GetCanonicalDeclarationIdentity(candidate), + GetCanonicalDeclarationIdentity(representative), + StringComparison.Ordinal))) + { + return CanonicalDeclarationIdentityReason; + } + return StableLocationReason; + } + + private static void PopulateFamilyMetadata(T representative, IReadOnlyList group, string key) + where T : SymbolResult + { + representative.DefinitionSites = group.Count; + representative.PartialFamilyId = BuildPartialFamilyId(key); + representative.RepresentativeReason = ResolveRepresentativeReason(representative, group); + representative.FamilyMembersTruncated = group.Count > FamilyMemberLimit; + var orderedMembers = group + .OrderBy(result => result.Path, StringComparer.Ordinal) + .ThenBy(result => result.StartLine) + .ThenBy(result => result.StartColumn ?? int.MaxValue) + .ThenBy(result => result.SymbolId ?? long.MaxValue) + .Take(FamilyMemberLimit) + .ToList(); + if (!orderedMembers.Any(member => ReferenceEquals(member, representative))) + { + orderedMembers[^1] = representative; + orderedMembers = orderedMembers + .OrderBy(result => result.Path, StringComparer.Ordinal) + .ThenBy(result => result.StartLine) + .ThenBy(result => result.StartColumn ?? int.MaxValue) + .ThenBy(result => result.SymbolId ?? long.MaxValue) + .ToList(); + } + representative.FamilyMembers = orderedMembers + .Select(result => new PartialFamilyMember + { + SymbolId = result.SymbolId, + Path = result.Path, + Line = result.Line, + StartLine = result.StartLine, + StartColumn = result.StartColumn, + EndLine = result.EndLine, + Generated = IsGeneratedCode(result), + Representative = ReferenceEquals(result, representative), + }) + .ToList(); + } + + private static string BuildCallableParameterIdentity( + string parameters, + IReadOnlyList genericParameterNames, + IReadOnlySet valueConstrainedGenericParameters, + string? containerQualifiedName, + CSharpCallableTypeKindLookup? typeKinds, + long? symbolId) + { + if (string.IsNullOrWhiteSpace(parameters)) + return string.Empty; + + return string.Join( + ",", + SplitCallableParameters(parameters) + .Select(RemoveLeadingParameterAttributes) + .Select(RemoveCSharpComments) + .Select(RemoveLeadingParameterAttributes) + .Select(RemoveTrailingParameterName) + .Select(parameter => BuildParameterTypeAndRefIdentity( + parameter, + genericParameterNames, + valueConstrainedGenericParameters, + containerQualifiedName, + typeKinds, + symbolId))); + } + + private static string BuildParameterTypeAndRefIdentity( + string parameter, + IReadOnlyList genericParameterNames, + IReadOnlySet valueConstrainedGenericParameters, + string? containerQualifiedName, + CSharpCallableTypeKindLookup? typeKinds, + long? symbolId) + { + var remaining = parameter.TrimStart(); + var refKind = string.Empty; + while (TryReadLeadingIdentifier(remaining, out var modifier, out var modifierLength)) + { + var normalizedModifier = modifier.ToLowerInvariant(); + if (normalizedModifier is "this" or "params" or "scoped") + { + remaining = remaining[modifierLength..].TrimStart(); + continue; + } + if (normalizedModifier is "ref" or "out" or "in") + { + refKind = normalizedModifier; + remaining = remaining[modifierLength..].TrimStart(); + if (normalizedModifier == "ref" + && TryReadLeadingIdentifier(remaining, out var readonlyModifier, out var readonlyLength) + && string.Equals(readonlyModifier, "readonly", StringComparison.OrdinalIgnoreCase)) + { + refKind = "ref_readonly"; + remaining = remaining[readonlyLength..].TrimStart(); + } + } + break; + } + return $"{refKind}:{NormalizeCallableTypeIdentity( + remaining, + genericParameterNames, + valueConstrainedGenericParameters, + containerQualifiedName, + typeKinds, + symbolId)}"; + } + + private static bool TryReadLeadingIdentifier( + string value, + out string identifier, + out int identifierLength) + { + identifier = string.Empty; + identifierLength = 0; + if (value.Length == 0 || !IsIdentifierStartCharacter(value[0])) + return false; + + while (identifierLength < value.Length && IsIdentifierCharacter(value[identifierLength])) + identifierLength++; + identifier = value[..identifierLength]; + return true; + } + + private static string ExtractGenericParameterName(string parameter) + { + var remaining = RemoveLeadingParameterAttributes( + RemoveCSharpComments(RemoveLeadingParameterAttributes(parameter))).Trim(); + var end = remaining.Length - 1; + while (end >= 0 && !IsIdentifierCharacter(remaining[end])) + end--; + if (end < 0) + return remaining; + + var start = end; + while (start >= 0 && IsIdentifierCharacter(remaining[start])) + start--; + return remaining[(start + 1)..(end + 1)]; + } + + private static string NormalizeCallableTypeIdentity( + string? value, + IReadOnlyList genericParameterNames, + IReadOnlySet? valueConstrainedGenericParameters = null, + string? containerQualifiedName = null, + CSharpCallableTypeKindLookup? typeKinds = null, + long? symbolId = null) + { + if (string.IsNullOrWhiteSpace(value)) + return string.Empty; + + var tokens = TokenizeCallableType(value); + var builder = new StringBuilder(value.Length); + for (var offset = 0; offset < tokens.Count;) + { + if (TryReadTupleTypeIdentity( + tokens, + offset, + genericParameterNames, + valueConstrainedGenericParameters, + containerQualifiedName, + typeKinds, + symbolId, + out var tupleIdentity, + out var tupleConsumedTokens)) + { + builder.Append(tupleIdentity); + offset += tupleConsumedTokens; + continue; + } + + if (TryReadFrameworkNullableTypeIdentity(tokens, offset, out var nullableConsumedTokens)) + { + builder.Append("global::System.Nullable"); + offset += nullableConsumedTokens; + continue; + } + + if (TryReadFrameworkTypeIdentity(tokens, offset, out var frameworkIdentity, out var consumedTokens)) + { + var nullableSuffixOffset = offset + consumedTokens; + if (nullableSuffixOffset < tokens.Count && tokens[nullableSuffixOffset] == "?") + { + if (CSharpFrameworkReferenceTypeIdentities.Contains(frameworkIdentity)) + { + builder.Append(frameworkIdentity); + } + else if (frameworkIdentity != "global::System.Void") + { + builder.Append("global::System.Nullable<"); + builder.Append(frameworkIdentity); + builder.Append('>'); + } + else + { + builder.Append(frameworkIdentity); + builder.Append('?'); + } + offset = nullableSuffixOffset + 1; + continue; + } + + builder.Append(frameworkIdentity); + offset = nullableSuffixOffset; + continue; + } + + var token = tokens[offset]; + if (token == "managed" + && offset >= 2 + && tokens[offset - 2] == "delegate" + && tokens[offset - 1] == "*" + && offset + 1 < tokens.Count + && tokens[offset + 1] == "<") + { + // `managed` is the default function-pointer calling convention, so + // `delegate* managed` and `delegate*` have the same C# identity. + // `managed` は function pointer の既定 calling convention なので、 + // `delegate* managed` と `delegate*` は同じ C# identity になる。 + offset++; + continue; + } + if (token == "?" && offset > 0 && tokens[offset - 1] == "]") + { + // Arrays are reference types; their nullable annotation does not + // participate in CLR callable identity. + // array は reference type なので nullable annotation を CLR callable + // identity に含めない。 + offset++; + continue; + } + if (token == "?" && offset > 0 && tokens[offset - 1] == ")") + { + // A nullable tuple shorthand is a known value type even without indexed + // source facts. Normalize it to the same CLR identity as System.Nullable. + // nullable tuple shorthand は indexed source 情報がなくても value type と + // 確定できるため、System.Nullable と同じ CLR identity に正規化する。 + var sourceIdentity = ReadNullableTupleSourceIdentity(tokens, offset); + var normalizedSourceIdentity = NormalizeCallableTypeIdentity( + sourceIdentity, + genericParameterNames, + valueConstrainedGenericParameters, + containerQualifiedName, + typeKinds, + symbolId); + var currentIdentity = builder.ToString(); + if (normalizedSourceIdentity.Length > 0 + && currentIdentity.EndsWith(normalizedSourceIdentity, StringComparison.Ordinal)) + { + builder.Length -= normalizedSourceIdentity.Length; + builder.Append("global::System.Nullable<"); + builder.Append(normalizedSourceIdentity); + builder.Append('>'); + offset++; + continue; + } + } + if (token == "?") + { + var sourceIdentity = ReadCustomNullableSourceIdentity(tokens, offset); + var resolvedKind = IsExplicitFrameworkValueTupleIdentity(sourceIdentity) + ? CSharpCallableTypeKindLookup.TypeKind.Value + : typeKinds?.Resolve( + sourceIdentity, + containerQualifiedName, + symbolId); + if (resolvedKind == CSharpCallableTypeKindLookup.TypeKind.Reference) + { + // Nullable reference annotations do not participate in a C# callable + // signature. Remove one only when indexed source facts resolve the + // annotated custom type as a reference type. + // nullable reference annotation は C# callable signature の一部ではない。 + // indexed source 情報で reference type と確定した場合だけ除去する。 + offset++; + continue; + } + if (resolvedKind == CSharpCallableTypeKindLookup.TypeKind.Value) + { + var normalizedSourceIdentity = NormalizeCallableTypeIdentity( + sourceIdentity, + genericParameterNames, + valueConstrainedGenericParameters, + containerQualifiedName, + typeKinds, + symbolId); + var currentIdentity = builder.ToString(); + if (normalizedSourceIdentity.Length > 0 + && currentIdentity.EndsWith(normalizedSourceIdentity, StringComparison.Ordinal)) + { + builder.Length -= normalizedSourceIdentity.Length; + builder.Append("global::System.Nullable<"); + builder.Append(normalizedSourceIdentity); + builder.Append('>'); + offset++; + continue; + } + } + } + if (!IsIdentifierCharacter(token[0])) + { + builder.Append(token); + offset++; + continue; + } + + var genericParameterIndex = -1; + for (var index = 0; index < genericParameterNames.Count; index++) + { + if (string.Equals( + token.TrimStart('@'), + genericParameterNames[index].TrimStart('@'), + StringComparison.Ordinal)) + { + genericParameterIndex = index; + break; + } + } + + // A method type parameter is substituted only when it appears as an + // unqualified type reference. In N.T / Alias::T, the leaf T names a + // real qualified type even when the method also declares . + // method type parameter は unqualified な type reference の場合だけ置換する。 + // N.T / Alias::T の末尾 T は、method が を宣言していても実型である。 + var isQualifiedTypeSegment = + (offset > 0 && tokens[offset - 1] is "." or ":") + || (offset + 1 < tokens.Count && tokens[offset + 1] is "." or ":"); + if (genericParameterIndex >= 0 && !isQualifiedTypeSegment) + { + var genericIdentity = $"`{genericParameterIndex.ToString(System.Globalization.CultureInfo.InvariantCulture)}"; + var hasNullableSuffix = offset + 1 < tokens.Count && tokens[offset + 1] == "?"; + if (hasNullableSuffix + && valueConstrainedGenericParameters?.Contains(genericParameterIndex) == true) + { + builder.Append("global::System.Nullable<"); + builder.Append(genericIdentity); + builder.Append('>'); + offset += 2; + continue; + } + + builder.Append(genericIdentity); + offset++; + if (offset < tokens.Count + && tokens[offset] == "?" + && valueConstrainedGenericParameters?.Contains(genericParameterIndex) != true) + { + // For unconstrained and reference-constrained method parameters, T? + // is a nullable annotation and not part of the CLR signature. Under a + // struct/unmanaged constraint it represents Nullable and must remain. + // unconstrained / reference constraint の method type parameter では T? + // は CLR signature 外の annotation。struct / unmanaged constraint では + // Nullable を表すため保持する。 + offset++; + } + continue; + } + else + { + // `@global::` can name an ordinary using alias, while only unescaped + // `global::` is the reserved root qualifier. Keep those identities distinct. + // `@global::` は通常の using alias を指し得る。予約済み root qualifier は + // escape なしの `global::` だけなので、両 identity を区別する。 + var isVerbatimGlobalAliasQualifier = + token == "@global" + && offset + 2 < tokens.Count + && tokens[offset + 1] == ":" + && tokens[offset + 2] == ":"; + builder.Append(isVerbatimGlobalAliasQualifier ? token : token.TrimStart('@')); + } + offset++; + } + return builder.ToString(); + } + + private static bool TryReadTupleTypeIdentity( + IReadOnlyList tokens, + int offset, + IReadOnlyList genericParameterNames, + IReadOnlySet? valueConstrainedGenericParameters, + string? containerQualifiedName, + CSharpCallableTypeKindLookup? typeKinds, + long? symbolId, + out string identity, + out int consumedTokens) + { + identity = string.Empty; + consumedTokens = 0; + if (offset >= tokens.Count || tokens[offset] != "(") + return false; + + var elementRanges = new List<(int Start, int End)>(); + var elementStart = offset + 1; + var parenthesisDepth = 0; + var angleDepth = 0; + var bracketDepth = 0; + var closeOffset = -1; + for (var current = elementStart; current < tokens.Count; current++) + { + switch (tokens[current]) + { + case "(": + parenthesisDepth++; + break; + case ")" when parenthesisDepth > 0: + parenthesisDepth--; + break; + case ")" when angleDepth == 0 && bracketDepth == 0: + closeOffset = current; + current = tokens.Count; + break; + case "<": + angleDepth++; + break; + case ">" when angleDepth > 0: + angleDepth--; + break; + case "[": + bracketDepth++; + break; + case "]" when bracketDepth > 0: + bracketDepth--; + break; + case "," when parenthesisDepth == 0 && angleDepth == 0 && bracketDepth == 0: + elementRanges.Add((elementStart, current)); + elementStart = current + 1; + break; + } + } + + if (closeOffset < 0 || elementRanges.Count == 0 || elementStart >= closeOffset) + return false; + + elementRanges.Add((elementStart, closeOffset)); + var elementIdentities = new List(elementRanges.Count); + var elementNames = new List(elementRanges.Count); + foreach (var range in elementRanges) + { + if (range.Start >= range.End) + return false; + + var nameOffset = GetTupleElementNameOffset(tokens, range.Start, range.End); + var typeEnd = nameOffset >= 0 ? nameOffset : range.End; + if (typeEnd <= range.Start) + return false; + + var elementSource = string.Concat(tokens.Skip(range.Start).Take(typeEnd - range.Start)); + elementIdentities.Add(NormalizeCallableTypeIdentity( + elementSource, + genericParameterNames, + valueConstrainedGenericParameters, + containerQualifiedName, + typeKinds, + symbolId)); + elementNames.Add(nameOffset >= 0 ? tokens[nameOffset].TrimStart('@') : null); + } + + var tupleIdentity = BuildValueTupleIdentity(elementIdentities); + if (elementNames.Any(name => name != null)) + { + // Tuple element names participate in the partial declaration contract but are + // absent from the CLR ValueTuple type. Append a source-impossible marker so + // matching named tuples remain stable without collapsing into an explicitly + // unnamed ValueTuple spelling. Normalize only each element's type above; names + // that happen to equal method type parameters must remain literal names. + // tuple element name は partial 宣言 contract に含まれる一方、CLR ValueTuple + // type には含まれない。source に現れない marker で保持し、method type parameter + // と同名の element name を型 parameter として置換しない。 + tupleIdentity += $"#tuple_names({string.Join(',', elementNames.Select(name => name ?? string.Empty))})"; + } + consumedTokens = closeOffset - offset + 1; + if (closeOffset + 1 < tokens.Count && tokens[closeOffset + 1] == "?") + { + identity = $"global::System.Nullable<{tupleIdentity}>"; + consumedTokens++; + } + else + { + identity = tupleIdentity; + } + + return true; + } + + private static int GetTupleElementNameOffset( + IReadOnlyList tokens, + int start, + int end) + { + var last = end - 1; + if (last <= start || !IsIdentifierCharacter(tokens[last][0])) + return -1; + + // A final identifier preceded by member/alias qualification is the leaf type name. + // Every other final identifier following a complete tokenized type is a tuple element + // name (for example `int value`, `T[] values`, or `(int, int) pair`). + // member / alias qualifier 直後の末尾 identifier は型名。それ以外で完全な型の + // 後ろに続く末尾 identifier は tuple element name とみなす。 + return tokens[last - 1] is not "." and not ":" ? last : -1; + } + + private static string BuildValueTupleIdentity(IReadOnlyList elementIdentities) + { + const int directElementLimit = 7; + if (elementIdentities.Count <= directElementLimit) + return $"global::System.ValueTuple<{string.Join(',', elementIdentities)}>"; + + var directElements = elementIdentities.Take(directElementLimit).ToList(); + directElements.Add(BuildValueTupleIdentity(elementIdentities.Skip(directElementLimit).ToList())); + return $"global::System.ValueTuple<{string.Join(',', directElements)}>"; + } + + private static IReadOnlySet ReadValueConstrainedGenericParameters( + string signature, + int closeParenthesis, + IReadOnlyList genericParameterNames) + { + var constrained = new HashSet(); + if (genericParameterNames.Count == 0 || closeParenthesis + 1 >= signature.Length) + return constrained; + + var declarationHeaderEnd = FindCallableDeclarationHeaderEnd(signature, closeParenthesis + 1); + var tokens = TokenizeCallableType(signature[(closeParenthesis + 1)..declarationHeaderEnd]); + for (var offset = 0; offset + 1 < tokens.Count; offset++) + { + if (!string.Equals(tokens[offset], "where", StringComparison.Ordinal)) + continue; + + var parameterIndex = FindGenericParameterIndex(tokens[offset + 1], genericParameterNames); + if (parameterIndex < 0) + continue; + + for (var constraintOffset = offset + 2; constraintOffset < tokens.Count; constraintOffset++) + { + var token = tokens[constraintOffset]; + if (constraintOffset > offset + 2 + && (token == "where" || token is "{" or ";")) + { + offset = constraintOffset - 1; + break; + } + + if (token is "struct" or "unmanaged") + constrained.Add(parameterIndex); + } + } + + return constrained; + } + + private static int FindCallableDeclarationHeaderEnd(string signature, int searchStart) + { + var sanitizedSignature = SymbolExtractor.SanitizeCSharpDeclarationSignature(signature); + for (var offset = Math.Clamp(searchStart, 0, sanitizedSignature.Length); + offset < sanitizedSignature.Length; + offset++) + { + if (sanitizedSignature[offset] is '{' or ';' + || (sanitizedSignature[offset] == '=' + && offset + 1 < sanitizedSignature.Length + && sanitizedSignature[offset + 1] == '>')) + { + return offset; + } + } + + return signature.Length; + } + + private static int FindGenericParameterIndex( + string token, + IReadOnlyList genericParameterNames) + { + for (var index = 0; index < genericParameterNames.Count; index++) + { + if (string.Equals( + token.TrimStart('@'), + genericParameterNames[index].TrimStart('@'), + StringComparison.Ordinal)) + { + return index; + } + } + + return -1; + } + + private static string ReadCustomNullableSourceIdentity( + IReadOnlyList tokens, + int nullableOffset) + { + var identityEnd = nullableOffset - 1; + if (identityEnd < 0) + return string.Empty; + + var nameStart = FindCustomTypeSegmentStart(tokens, identityEnd); + if (nameStart < 0) + return string.Empty; + + while (nameStart >= 2 && tokens[nameStart - 1] == ".") + { + var previousStart = FindCustomTypeSegmentStart(tokens, nameStart - 2); + if (previousStart < 0) + break; + nameStart = previousStart; + } + + if (nameStart >= 3 + && tokens[nameStart - 1] == ":" + && tokens[nameStart - 2] == ":" + && IsIdentifierCharacter(tokens[nameStart - 3][0])) + { + // Retain every alias qualifier, including escaped @global. Only literal + // global:: has reserved root semantics; other aliases must remain visible + // so the source-fact lookup can conservatively leave them unresolved. + // escaped @global を含むすべての alias qualifier を保持する。予約済み root + // semantics を持つのは literal global:: だけであり、ほかの alias は source + // fact lookup が保守的に unresolved と判断できるよう残す。 + nameStart -= 3; + } + + return string.Concat(tokens.Skip(nameStart).Take(identityEnd - nameStart + 1)); + } + + private static bool IsExplicitFrameworkValueTupleIdentity(string sourceIdentity) + { + var tokens = TokenizeCallableType(sourceIdentity); + return tokens.Count >= 6 + && tokens[0] == "global" + && tokens[1] == ":" + && tokens[2] == ":" + && tokens[3].TrimStart('@') == "System" + && tokens[4] == "." + && tokens[5].TrimStart('@') == "ValueTuple" + && (tokens.Count == 6 || tokens[6] == "<"); + } + + private static string ReadNullableTupleSourceIdentity( + IReadOnlyList tokens, + int nullableOffset) + { + var depth = 0; + for (var offset = nullableOffset - 1; offset >= 0; offset--) + { + if (tokens[offset] == ")") + { + depth++; + continue; + } + if (tokens[offset] != "(") + continue; + + depth--; + if (depth == 0) + return string.Concat(tokens.Skip(offset).Take(nullableOffset - offset)); + } + + return string.Empty; + } + + private static int FindCustomTypeSegmentStart( + IReadOnlyList tokens, + int segmentEnd) + { + if (segmentEnd < 0) + return -1; + if (tokens[segmentEnd] != ">") + return IsIdentifierCharacter(tokens[segmentEnd][0]) ? segmentEnd : -1; + + var depth = 0; + for (var offset = segmentEnd; offset >= 0; offset--) + { + if (tokens[offset] == ">") + { + depth++; + } + else if (tokens[offset] == "<" && --depth == 0) + { + var nameOffset = offset - 1; + return nameOffset >= 0 && IsIdentifierCharacter(tokens[nameOffset][0]) + ? nameOffset + : -1; + } + } + + return -1; + } + + private static bool TryReadFrameworkNullableTypeIdentity( + IReadOnlyList tokens, + int offset, + out int consumedTokens) + { + consumedTokens = 0; + if (offset + 5 >= tokens.Count + || tokens[offset] != "global" + || tokens[offset + 1] != ":" + || tokens[offset + 2] != ":" + || tokens[offset + 3].TrimStart('@') != "System" + || tokens[offset + 4] != "." + || tokens[offset + 5].TrimStart('@') != "Nullable") + { + return false; + } + + consumedTokens = 6; + return true; + } + + private static List TokenizeCallableType(string value) + { + var tokens = new List(); + for (var offset = 0; offset < value.Length;) + { + if (value[offset] == '/' && offset + 1 < value.Length) + { + if (value[offset + 1] == '/') + break; + if (value[offset + 1] == '*') + { + var commentEnd = value.IndexOf("*/", offset + 2, StringComparison.Ordinal); + offset = commentEnd < 0 ? value.Length : commentEnd + 2; + continue; + } + } + if (char.IsWhiteSpace(value[offset])) + { + offset++; + continue; + } + if (!IsIdentifierStartCharacter(value[offset])) + { + tokens.Add(value[offset].ToString()); + offset++; + continue; + } + + var end = offset + 1; + while (end < value.Length && IsIdentifierCharacter(value[end])) + end++; + tokens.Add(value[offset..end]); + offset = end; + } + return tokens; + } + + private static bool TryReadFrameworkTypeIdentity( + IReadOnlyList tokens, + int offset, + out string identity, + out int consumedTokens) + { + identity = string.Empty; + consumedTokens = 0; + var token = tokens[offset]; + if (CSharpPredefinedTypeIdentities.TryGetValue(token, out var predefinedIdentity)) + { + identity = $"global::{predefinedIdentity}"; + consumedTokens = 1; + return true; + } + + // Only an explicit global alias proves that System refers to the framework + // namespace. An unrooted System.Int32 can bind to an enclosing namespace or + // using alias and must retain its source identity. Verbatim escapes on the + // namespace/type segments do not change the explicitly rooted identity. + // explicit global alias だけが System を framework namespace と確定できる。 + // unrooted System.Int32 は外側 namespace / using alias に bind し得るため + // source identity を保持し、rooted segment の verbatim escape だけを外す。 + if (offset + 5 >= tokens.Count + || token != "global" + || tokens[offset + 1] != ":" + || tokens[offset + 2] != ":") + { + return false; + } + + var systemOffset = offset + 3; + if (systemOffset + 2 >= tokens.Count + || tokens[systemOffset].TrimStart('@') != "System" + || tokens[systemOffset + 1] != "." + || !CSharpFrameworkTypeIdentities.TryGetValue( + tokens[systemOffset + 2].TrimStart('@'), + out var frameworkIdentity)) + { + return false; + } + + identity = $"global::{frameworkIdentity}"; + consumedTokens = systemOffset - offset + 3; + return true; + } + + private static List SplitTopLevel(string value) + { + var items = new List(); + var start = 0; + var parenthesisDepth = 0; + var angleDepth = 0; + var bracketDepth = 0; + var braceDepth = 0; + var quote = '\0'; + var rawQuoteLength = 0; + var escaped = false; + var verbatim = false; + var lineComment = false; + var blockComment = false; + for (var i = 0; i < value.Length; i++) + { + var ch = value[i]; + if (lineComment) + { + if (ch is '\r' or '\n') + lineComment = false; + continue; + } + if (blockComment) + { + if (ch == '*' && i + 1 < value.Length && value[i + 1] == '/') + { + blockComment = false; + i++; + } + continue; + } + if (rawQuoteLength > 0) + { + if (ch == '"' && CountRepeatedCharacter(value, i, '"') >= rawQuoteLength) + { + i += rawQuoteLength - 1; + rawQuoteLength = 0; + } + continue; + } + if (quote != '\0') + { + if (verbatim && ch == '"' && i + 1 < value.Length && value[i + 1] == '"') + { + i++; + continue; + } + if (escaped) + { + escaped = false; + continue; + } + if (!verbatim && ch == '\\') + { + escaped = true; + continue; + } + if (ch == quote) + { + quote = '\0'; + verbatim = false; + } + continue; + } + + if (ch == '"') + { + var quoteLength = CountRepeatedCharacter(value, i, '"'); + if (quoteLength >= 3) + { + rawQuoteLength = quoteLength; + i += quoteLength - 1; + } + else + { + quote = ch; + verbatim = IsVerbatimStringStart(value, i); + } + continue; + } + if (ch == '\'') + { + quote = ch; + continue; + } + if (ch == '/' && i + 1 < value.Length) + { + if (value[i + 1] == '/') + { + lineComment = true; + i++; + continue; + } + if (value[i + 1] == '*') + { + blockComment = true; + i++; + continue; + } + } + + switch (ch) + { + case '(': + parenthesisDepth++; + break; + case ')': + parenthesisDepth--; + break; + case '<': + angleDepth++; + break; + case '>': + angleDepth--; + break; + case '[': + bracketDepth++; + break; + case ']': + bracketDepth--; + break; + case '{': + braceDepth++; + break; + case '}': + braceDepth--; + break; + case ',' when parenthesisDepth == 0 && angleDepth == 0 && bracketDepth == 0 && braceDepth == 0: + items.Add(value[start..i]); + start = i + 1; + break; + } + } + items.Add(value[start..]); + return items; + } + + private static List SplitCallableParameters(string value) + { + var items = new List(); + var start = 0; + var defaultValueStart = -1; + var parenthesisDepth = 0; + var angleDepth = 0; + var bracketDepth = 0; + var braceDepth = 0; + var quote = '\0'; + var rawQuoteLength = 0; + var escaped = false; + var verbatim = false; + var lineComment = false; + var blockComment = false; + for (var i = 0; i < value.Length; i++) + { + var ch = value[i]; + if (lineComment) + { + if (ch is '\r' or '\n') + lineComment = false; + continue; + } + if (blockComment) + { + if (ch == '*' && i + 1 < value.Length && value[i + 1] == '/') + { + blockComment = false; + i++; + } + continue; + } + if (rawQuoteLength > 0) + { + if (ch == '"' && CountRepeatedCharacter(value, i, '"') >= rawQuoteLength) + { + i += rawQuoteLength - 1; + rawQuoteLength = 0; + } + continue; + } + if (quote != '\0') + { + if (verbatim && ch == '"' && i + 1 < value.Length && value[i + 1] == '"') + { + i++; + continue; + } + if (escaped) + { + escaped = false; + continue; + } + if (!verbatim && ch == '\\') + { + escaped = true; + continue; + } + if (ch == quote) + { + quote = '\0'; + verbatim = false; + } + continue; + } + if (ch == '"') + { + var quoteLength = CountRepeatedCharacter(value, i, '"'); + if (quoteLength >= 3) + { + rawQuoteLength = quoteLength; + i += quoteLength - 1; + } + else + { + quote = ch; + verbatim = IsVerbatimStringStart(value, i); + } + continue; + } + if (ch == '\'') + { + quote = ch; + continue; + } + if (ch == '/' && i + 1 < value.Length) + { + if (value[i + 1] == '/') + { + lineComment = true; + i++; + continue; + } + if (value[i + 1] == '*') + { + blockComment = true; + i++; + continue; + } + } + + switch (ch) + { + case '(': + parenthesisDepth++; + break; + case ')': + parenthesisDepth--; + break; + case '<' when bracketDepth == 0 + && (defaultValueStart < 0 + || angleDepth > 0 + || IsDefaultExpressionGenericMemberAccess(value, i)): + angleDepth++; + break; + case '>' when bracketDepth == 0 && angleDepth > 0: + angleDepth--; + break; + case '[': + bracketDepth++; + break; + case ']': + bracketDepth--; + break; + case '{': + braceDepth++; + break; + case '}': + braceDepth--; + break; + case '=' when defaultValueStart < 0 + && parenthesisDepth == 0 + && angleDepth == 0 + && bracketDepth == 0 + && braceDepth == 0: + defaultValueStart = i; + break; + case ',' when parenthesisDepth == 0 + && bracketDepth == 0 + && braceDepth == 0 + && angleDepth == 0: + items.Add(value[start..(defaultValueStart >= 0 ? defaultValueStart : i)]); + start = i + 1; + defaultValueStart = -1; + angleDepth = 0; + break; + } + } + items.Add(value[start..(defaultValueStart >= 0 ? defaultValueStart : value.Length)]); + return items; + } + + private static bool IsDefaultExpressionGenericMemberAccess(string value, int openAngleOffset) + { + // Optional defaults can reference a generic const as `G.Value`. + // Track only angle lists that close into member access: treating every `<` after + // `=` as generic would break valid relational constants such as `1 < 2`. + // optional default の `G.Value` は generic comma を含む。一方、`=` + // 以降の全 `<` を generic とみなすと `1 < 2` を壊すため、member access へ閉じる + // angle list だけを追跡する。 + var depth = 0; + var quote = '\0'; + var rawQuoteLength = 0; + var escaped = false; + var verbatim = false; + var lineComment = false; + var blockComment = false; + for (var cursor = openAngleOffset; cursor < value.Length; cursor++) + { + var ch = value[cursor]; + if (lineComment) + { + if (ch is '\r' or '\n') + lineComment = false; + continue; + } + if (blockComment) + { + if (ch == '*' && cursor + 1 < value.Length && value[cursor + 1] == '/') + { + blockComment = false; + cursor++; + } + continue; + } + if (rawQuoteLength > 0) + { + if (ch == '"' && CountRepeatedCharacter(value, cursor, '"') >= rawQuoteLength) + { + cursor += rawQuoteLength - 1; + rawQuoteLength = 0; + } + continue; + } + if (quote != '\0') + { + if (verbatim && ch == '"' && cursor + 1 < value.Length && value[cursor + 1] == '"') + { + cursor++; + continue; + } + if (escaped) + { + escaped = false; + continue; + } + if (!verbatim && ch == '\\') + { + escaped = true; + continue; + } + if (ch == quote) + { + quote = '\0'; + verbatim = false; + } + continue; + } + if (ch == '"') + { + var quoteLength = CountRepeatedCharacter(value, cursor, '"'); + if (quoteLength >= 3) + { + rawQuoteLength = quoteLength; + cursor += quoteLength - 1; + } + else + { + quote = ch; + verbatim = IsVerbatimStringStart(value, cursor); + } + continue; + } + if (ch == '\'') + { + quote = ch; + continue; + } + if (ch == '/' && cursor + 1 < value.Length) + { + if (value[cursor + 1] == '/') + { + lineComment = true; + cursor++; + continue; + } + if (value[cursor + 1] == '*') + { + blockComment = true; + cursor++; + continue; + } + } + + if (ch == '<') + { + depth++; + continue; + } + if (ch != '>' || --depth != 0) + continue; + + cursor = SkipCSharpDeclarationTrivia(value, cursor + 1); + return cursor < value.Length && value[cursor] == '.'; + } + + return false; + } + + private static string RemoveLeadingParameterAttributes(string parameter) + { + var remaining = parameter.TrimStart(); + while (remaining.StartsWith("[", StringComparison.Ordinal)) + { + var attributeEnd = FindBalancedEnd(remaining, 0, '[', ']'); + if (attributeEnd < 0) + break; + remaining = remaining[(attributeEnd + 1)..].TrimStart(); + } + return remaining; + } + + private static string RemoveCSharpComments(string value) + { + var builder = new StringBuilder(value.Length); + var quote = '\0'; + var rawQuoteLength = 0; + var escaped = false; + var verbatim = false; + var lineComment = false; + var blockComment = false; + for (var offset = 0; offset < value.Length;) + { + var ch = value[offset]; + if (lineComment) + { + if (ch is '\r' or '\n') + { + lineComment = false; + builder.Append(ch); + } + offset++; + continue; + } + if (blockComment) + { + if (ch == '*' && offset + 1 < value.Length && value[offset + 1] == '/') + { + blockComment = false; + offset += 2; + continue; + } + if (ch is '\r' or '\n') + builder.Append(ch); + offset++; + continue; + } + if (rawQuoteLength > 0) + { + var repeatedQuotes = ch == '"' ? CountRepeatedCharacter(value, offset, '"') : 0; + if (repeatedQuotes >= rawQuoteLength) + { + builder.Append(value, offset, rawQuoteLength); + offset += rawQuoteLength; + rawQuoteLength = 0; + continue; + } + + builder.Append(ch); + offset++; + continue; + } + if (quote != '\0') + { + builder.Append(ch); + if (verbatim && ch == '"' && offset + 1 < value.Length && value[offset + 1] == '"') + { + builder.Append('"'); + offset += 2; + continue; + } + if (escaped) + escaped = false; + else if (!verbatim && ch == '\\') + escaped = true; + else if (ch == quote) + { + quote = '\0'; + verbatim = false; + } + offset++; + continue; + } + + if (ch == '/' && offset + 1 < value.Length) + { + if (value[offset + 1] == '/') + { + if (builder.Length > 0 && !char.IsWhiteSpace(builder[^1])) + builder.Append(' '); + lineComment = true; + offset += 2; + continue; + } + if (value[offset + 1] == '*') + { + if (builder.Length > 0 && !char.IsWhiteSpace(builder[^1])) + builder.Append(' '); + blockComment = true; + offset += 2; + continue; + } + } + if (ch == '"') + { + var repeatedQuotes = CountRepeatedCharacter(value, offset, '"'); + if (repeatedQuotes >= 3) + { + builder.Append(value, offset, repeatedQuotes); + rawQuoteLength = repeatedQuotes; + offset += repeatedQuotes; + continue; + } + quote = ch; + verbatim = IsVerbatimStringStart(value, offset); + } + else if (ch == '\'') + { + quote = ch; + } + builder.Append(ch); + offset++; + } + return builder.ToString(); + } + + private static string RemoveTrailingParameterName(string parameter) + { + var end = parameter.Length - 1; + while (end >= 0 && char.IsWhiteSpace(parameter[end])) + end--; + if (end < 0 || !IsIdentifierCharacter(parameter[end])) + return parameter; + + var start = end; + while (start >= 0 && IsIdentifierCharacter(parameter[start])) + start--; + var typeAndModifiers = parameter[..(start + 1)].TrimEnd(); + return typeAndModifiers.Length == 0 ? parameter : typeAndModifiers; + } + + private static int GetPrimaryRank(SymbolResult symbol) + => IsLogicalPartialCallableKind(symbol.Kind) && (!symbol.BodyStartLine.HasValue || !symbol.BodyEndLine.HasValue) + ? 1 + : 0; + + private static bool IsGeneratedCode(SymbolResult symbol) + => symbol.IsGeneratedCode + ?? FileIndexer.HasGeneratedCodeFileName(symbol.Path); + + private static int GetSemanticScore(SymbolResult symbol) + => GetSemanticScore(symbol.Signature, symbol.Kind); + + private static string GetCanonicalDeclarationIdentity(SymbolResult symbol) + => IsLogicalPartialCallableKind(symbol.Kind) + ? BuildCallableIdentity(symbol.Signature, symbol.Name, symbol.ReturnType) ?? string.Empty + : BuildCanonicalDeclarationIdentity(symbol.Signature); + + internal static int FindCallableNameOffset(string signature, string name) + { + var offset = 0; + while ((offset = signature.IndexOf(name, offset, StringComparison.Ordinal)) >= 0) + { + var verbatimPrefix = offset > 0 + && signature[offset - 1] == '@' + && (offset == 1 || !IsIdentifierCharacter(signature[offset - 2])); + var beforeIsIdentifier = offset > 0 + && IsIdentifierCharacter(signature[offset - 1]) + && !verbatimPrefix; + var after = offset + name.Length; + var afterIsIdentifier = after < signature.Length && IsIdentifierCharacter(signature[after]); + if (!beforeIsIdentifier + && !afterIsIdentifier + && IsTopLevelCSharpDeclarationOffset(signature, offset)) + { + var cursor = SkipCSharpDeclarationTrivia(signature, after); + if (cursor < signature.Length && signature[cursor] == '<') + { + var genericEnd = FindBalancedEnd(signature, cursor, '<', '>'); + if (genericEnd >= 0) + cursor = SkipCSharpDeclarationTrivia(signature, genericEnd + 1); + } + if (cursor < signature.Length && signature[cursor] == '(') + return offset; + } + offset += name.Length; + } + return -1; + } + + private static int SkipCSharpDeclarationTrivia(string text, int start) + { + var cursor = Math.Clamp(start, 0, text.Length); + while (cursor < text.Length) + { + while (cursor < text.Length && char.IsWhiteSpace(text[cursor])) + cursor++; + + if (cursor + 1 >= text.Length || text[cursor] != '/') + break; + + if (text[cursor + 1] == '/') + { + cursor += 2; + while (cursor < text.Length && text[cursor] is not ('\r' or '\n')) + cursor++; + continue; + } + + if (text[cursor + 1] != '*') + break; + + var commentEnd = text.IndexOf("*/", cursor + 2, StringComparison.Ordinal); + if (commentEnd < 0) + return text.Length; + cursor = commentEnd + 2; + } + + return cursor; + } + + private static bool IsTopLevelCSharpDeclarationOffset(string text, int targetOffset) + { + var parenthesisDepth = 0; + var bracketDepth = 0; + var braceDepth = 0; + var quote = '\0'; + var rawQuoteLength = 0; + var escaped = false; + var verbatim = false; + var lineComment = false; + var blockComment = false; + for (var i = 0; i <= targetOffset && i < text.Length; i++) + { + var ch = text[i]; + if (i == targetOffset) + { + return !lineComment + && !blockComment + && rawQuoteLength == 0 + && quote == '\0' + && parenthesisDepth == 0 + && bracketDepth == 0 + && braceDepth == 0; + } + if (lineComment) + { + if (ch is '\r' or '\n') + lineComment = false; + continue; + } + if (blockComment) + { + if (ch == '*' && i + 1 < text.Length && text[i + 1] == '/') + { + blockComment = false; + i++; + } + continue; + } + if (rawQuoteLength > 0) + { + if (ch == '"' && CountRepeatedCharacter(text, i, '"') >= rawQuoteLength) + { + i += rawQuoteLength - 1; + rawQuoteLength = 0; + } + continue; + } + if (quote != '\0') + { + if (verbatim && ch == '"' && i + 1 < text.Length && text[i + 1] == '"') + { + i++; + continue; + } + if (escaped) + { + escaped = false; + continue; + } + if (!verbatim && ch == '\\') + { + escaped = true; + continue; + } + if (ch == quote) + { + quote = '\0'; + verbatim = false; + } + continue; + } + if (ch == '/' && i + 1 < text.Length) + { + if (text[i + 1] == '/') + { + lineComment = true; + i++; + continue; + } + if (text[i + 1] == '*') + { + blockComment = true; + i++; + continue; + } + } + if (ch == '"') + { + var quoteLength = CountRepeatedCharacter(text, i, '"'); + if (quoteLength >= 3) + { + rawQuoteLength = quoteLength; + i += quoteLength - 1; + } + else + { + quote = ch; + verbatim = IsVerbatimStringStart(text, i); + } + continue; + } + if (ch == '\'') + { + quote = ch; + continue; + } + switch (ch) + { + case '(': + parenthesisDepth++; + break; + case ')' when parenthesisDepth > 0: + parenthesisDepth--; + break; + case '[': + bracketDepth++; + break; + case ']' when bracketDepth > 0: + bracketDepth--; + break; + case '{': + braceDepth++; + break; + case '}' when braceDepth > 0: + braceDepth--; + break; + } + } + return false; + } + + private static int FindBalancedEnd(string text, int start, char open, char close) + { + var depth = 0; + var quote = '\0'; + var rawQuoteLength = 0; + var escaped = false; + var verbatim = false; + var lineComment = false; + var blockComment = false; + for (var i = start; i < text.Length; i++) + { + var ch = text[i]; + if (lineComment) + { + if (ch is '\r' or '\n') + lineComment = false; + continue; + } + if (blockComment) + { + if (ch == '*' && i + 1 < text.Length && text[i + 1] == '/') + { + blockComment = false; + i++; + } + continue; + } + if (rawQuoteLength > 0) + { + if (ch == '"') + { + var quoteLength = CountRepeatedCharacter(text, i, '"'); + if (quoteLength >= rawQuoteLength) + { + i += rawQuoteLength - 1; + rawQuoteLength = 0; + } + } + continue; + } + if (quote != '\0') + { + if (verbatim && ch == '"' && i + 1 < text.Length && text[i + 1] == '"') + { + i++; + continue; + } + if (escaped) + { + escaped = false; + continue; + } + if (!verbatim && ch == '\\') + { + escaped = true; + continue; + } + if (ch == quote) + { + quote = '\0'; + verbatim = false; + } + continue; + } + if (ch == '/' && i + 1 < text.Length) + { + if (text[i + 1] == '/') + { + lineComment = true; + i++; + continue; + } + if (text[i + 1] == '*') + { + blockComment = true; + i++; + continue; + } + } + if (ch == '"') + { + var quoteLength = CountRepeatedCharacter(text, i, '"'); + if (quoteLength >= 3) + { + rawQuoteLength = quoteLength; + i += quoteLength - 1; + } + else + { + quote = ch; + verbatim = IsVerbatimStringStart(text, i); + } + continue; + } + if (ch == '\'') + { + quote = ch; + continue; + } + if (ch == open) + depth++; + else if (ch == close && --depth == 0) + return i; + } + return -1; + } + + private static int CountRepeatedCharacter(string text, int start, char value) + { + var length = 0; + while (start + length < text.Length && text[start + length] == value) + length++; + return length; + } + + private static bool IsVerbatimStringStart(string text, int quoteOffset) + { + var cursor = quoteOffset - 1; + while (cursor >= 0 && text[cursor] == '$') + cursor--; + return cursor >= 0 && text[cursor] == '@'; + } + + private static string NormalizeIdentityToken(string? value) + { + if (string.IsNullOrWhiteSpace(value)) + return string.Empty; + + var builder = new StringBuilder(value.Length); + foreach (var ch in value) + { + if (!char.IsWhiteSpace(ch)) + builder.Append(char.ToLowerInvariant(ch)); + } + return builder.ToString(); + } + + private static bool IsIdentifierCharacter(char value) + { + if (IsIdentifierStartCharacter(value)) + return true; + + return char.GetUnicodeCategory(value) is + UnicodeCategory.DecimalDigitNumber or + UnicodeCategory.ConnectorPunctuation or + UnicodeCategory.NonSpacingMark or + UnicodeCategory.SpacingCombiningMark or + UnicodeCategory.Format; + } + + private static bool IsIdentifierStartCharacter(char value) + => value is '_' or '@' + || char.IsLetter(value) + || char.GetUnicodeCategory(value) == UnicodeCategory.LetterNumber; + + internal static bool ContainsPartialModifier( + string? signature, + string? kind, + string? name) + => SymbolExtractor.ContainsCSharpPartialDeclarationModifier(signature, kind, name); + + private static bool IsLogicalPartialTypeKind(string kind) + => kind is "class" or "struct" or "interface" or "record"; + + private static bool IsLogicalPartialCallableKind(string? kind) + => kind is "function" or "test.method"; } diff --git a/src/CodeIndex/Indexer/CSharpTypeReferenceArity.cs b/src/CodeIndex/Indexer/CSharpTypeReferenceArity.cs index f0a5ce54c..5222f7bfa 100644 --- a/src/CodeIndex/Indexer/CSharpTypeReferenceArity.cs +++ b/src/CodeIndex/Indexer/CSharpTypeReferenceArity.cs @@ -1,3 +1,5 @@ +using System.Text; + namespace CodeIndex.Indexer; internal static class CSharpTypeReferenceArity @@ -70,8 +72,13 @@ internal static bool IsMemberReceiver(string? context, string? symbolName, long? if (string.IsNullOrWhiteSpace(signature)) return 0; - var searchStart = FindDeclarationKeywordEnd(signature, symbolKind); - var occurrence = FindDefinitionIdentifierOccurrence(signature, symbolName, searchStart, symbolKind); + var declarationSignature = SymbolExtractor.SanitizeCSharpDeclarationSignature(signature); + var searchStart = FindDeclarationKeywordEnd(declarationSignature, symbolKind, symbolName); + var occurrence = FindDefinitionIdentifierOccurrence( + declarationSignature, + symbolName, + searchStart, + symbolKind); return occurrence < 0 ? null : ReadArityAfterIdentifier(signature, occurrence, symbolName.Length); } @@ -88,15 +95,20 @@ internal static bool IsMemberReceiver(string? context, string? symbolName, long? if (!typeDeclaration && !constructorFunction) return null; - var searchStart = typeDeclaration ? FindDeclarationKeywordEnd(signature, symbolKind) : 0; - for (var searchAt = Math.Clamp(searchStart, 0, signature.Length); - searchAt <= signature.Length - symbolName.Length;) + var searchableSignature = typeDeclaration + ? SymbolExtractor.SanitizeCSharpDeclarationSignature(signature) + : signature; + var searchStart = typeDeclaration + ? FindDeclarationKeywordEnd(searchableSignature, symbolKind, symbolName) + : 0; + for (var searchAt = Math.Clamp(searchStart, 0, searchableSignature.Length); + searchAt <= searchableSignature.Length - symbolName.Length;) { - var occurrence = signature.IndexOf(symbolName, searchAt, StringComparison.Ordinal); + var occurrence = searchableSignature.IndexOf(symbolName, searchAt, StringComparison.Ordinal); if (occurrence < 0) return null; searchAt = occurrence + Math.Max(1, symbolName.Length); - if (!IsIdentifierOccurrence(signature, occurrence, symbolName.Length)) + if (!IsIdentifierOccurrence(searchableSignature, occurrence, symbolName.Length)) continue; if (constructorFunction) @@ -144,6 +156,59 @@ internal static bool IsValueTypeDeclaration(string? signature, string? symbolKin && !string.IsNullOrWhiteSpace(signature) && ContainsIdentifier(signature, "struct", signature.Length)); + internal static string NormalizeTypeIdentityArity(string? identity) + { + if (string.IsNullOrWhiteSpace(identity)) + return string.Empty; + + var value = identity.Trim(); + if (value.StartsWith("global::", StringComparison.Ordinal)) + value = value["global::".Length..]; + + var normalized = new StringBuilder(value.Length); + for (var index = 0; index < value.Length;) + { + var current = value[index]; + if (char.IsWhiteSpace(current) || current == '@') + { + index++; + continue; + } + + if (!IsIdentifierPart(current) || char.IsDigit(current)) + { + if (current != '?') + normalized.Append(current); + index++; + continue; + } + + var identifierStart = index; + while (index < value.Length && IsIdentifierPart(value[index])) + index++; + normalized.Append(value, identifierStart, index - identifierStart); + + var genericStart = index; + SkipWhitespace(value, ref genericStart); + if (genericStart >= value.Length || value[genericStart] != '<') + continue; + if (!TryCountTopLevelTypeArguments( + value, + genericStart, + out var arity, + out var closeAngleIndex)) + { + return string.Empty; + } + + normalized.Append('`'); + normalized.Append(arity.ToString(System.Globalization.CultureInfo.InvariantCulture)); + index = closeAngleIndex + 1; + } + + return normalized.ToString(); + } + private static int FindClosestIdentifierOccurrence(string text, string identifier, long? columnNumber) { var expectedIndex = columnNumber is > 0 and <= int.MaxValue @@ -248,35 +313,91 @@ private static int FindDefinitionIdentifierOccurrence( return -1; } - private static int FindDeclarationKeywordEnd(string signature, string? symbolKind) + private static int FindDeclarationKeywordEnd( + string signature, + string? symbolKind, + string? symbolName = null) { if (string.IsNullOrWhiteSpace(symbolKind)) return 0; - var keyword = symbolKind switch + string[]? keywords = symbolKind switch { - "class" => "class", - "struct" => "struct", - "record" => "record", - "interface" => "interface", - "enum" => "enum", - "delegate" => "delegate", + // Plain records are emitted through the existing class kind, while record + // structs use the struct kind. Accept the source declaration keyword for + // both representations so earlier same-name attribute arguments cannot be + // mistaken for the declaration identifier. + // plain record は既存の class kind、record struct は struct kind で出力される。 + // source 上の record keyword も候補にし、先行 attribute 内の同名参照を + // declaration identifier と誤認しない。 + "class" => ["class", "record"], + "struct" => ["struct", "record"], + "record" => ["record"], + "interface" => ["interface"], + "enum" => ["enum"], + "delegate" => ["delegate"], _ => null, }; - if (keyword == null) + if (keywords == null) return 0; - for (var searchAt = 0; searchAt <= signature.Length - keyword.Length;) + var bestDeclarationStart = int.MaxValue; + foreach (var keyword in keywords) { - var occurrence = signature.IndexOf(keyword, searchAt, StringComparison.Ordinal); - if (occurrence < 0) - return 0; - if (IsIdentifierOccurrence(signature, occurrence, keyword.Length)) - return occurrence + keyword.Length; - searchAt = occurrence + keyword.Length; + for (var searchAt = 0; searchAt <= signature.Length - keyword.Length;) + { + var occurrence = signature.IndexOf(keyword, searchAt, StringComparison.Ordinal); + if (occurrence < 0) + break; + searchAt = occurrence + keyword.Length; + if (!IsIdentifierOccurrence(signature, occurrence, keyword.Length)) + continue; + + var declarationStart = searchAt; + if (!SkipCSharpTrivia(signature, ref declarationStart)) + continue; + if (keyword == "record") + SkipOptionalRecordTypeKeyword(signature, ref declarationStart); + if (!string.IsNullOrWhiteSpace(symbolName) + && !IsDeclarationIdentifierAt(signature, declarationStart, symbolName)) + { + continue; + } + + bestDeclarationStart = Math.Min(bestDeclarationStart, declarationStart); + } + } + + return bestDeclarationStart == int.MaxValue ? 0 : bestDeclarationStart; + } + + private static void SkipOptionalRecordTypeKeyword(string signature, ref int cursor) + { + foreach (var keyword in new[] { "class", "struct" }) + { + if (cursor + keyword.Length > signature.Length + || !signature.AsSpan(cursor, keyword.Length).SequenceEqual(keyword) + || !IsIdentifierOccurrence(signature, cursor, keyword.Length)) + { + continue; + } + + cursor += keyword.Length; + SkipCSharpTrivia(signature, ref cursor); + return; } + } - return 0; + private static bool IsDeclarationIdentifierAt( + string signature, + int cursor, + string symbolName) + { + if (cursor < signature.Length && signature[cursor] == '@') + cursor++; + return cursor + symbolName.Length <= signature.Length + && signature.AsSpan(cursor, symbolName.Length).SequenceEqual(symbolName) + && IsIdentifierOccurrence(signature, cursor, symbolName.Length); } private static int? ReadArityAfterIdentifier(string text, int occurrence, int identifierLength) diff --git a/src/CodeIndex/Indexer/Hooks/PostExtractionHookMutationMaterializer.cs b/src/CodeIndex/Indexer/Hooks/PostExtractionHookMutationMaterializer.cs index 6c36fdeb5..a1b33a84e 100644 --- a/src/CodeIndex/Indexer/Hooks/PostExtractionHookMutationMaterializer.cs +++ b/src/CodeIndex/Indexer/Hooks/PostExtractionHookMutationMaterializer.cs @@ -96,6 +96,76 @@ internal static void RefreshLanguageIdentity(string? language, IEnumerable sourceSymbols, + IReadOnlyList mutatedSymbols) + { + if (!string.Equals(language, "csharp", StringComparison.Ordinal)) + return; + + var sourceFacts = new Dictionary>(); + var sourceFactsByLocation = new Dictionary>(); + foreach (var symbol in sourceSymbols) + { + var sourceFact = new SourceCSharpDeclarationFacts(CSharpDeclarationFacts.From(symbol)); + var state = HookSymbolDeclarationState.From(symbol); + if (!sourceFacts.TryGetValue(state, out var facts)) + { + facts = new Queue(); + sourceFacts[state] = facts; + } + facts.Enqueue(sourceFact); + + var location = HookSymbolDeclarationLocation.From(symbol); + if (!sourceFactsByLocation.TryGetValue(location, out var locationFacts)) + { + locationFacts = new Queue(); + sourceFactsByLocation[location] = locationFacts; + } + locationFacts.Enqueue(sourceFact); + } + + foreach (var symbol in mutatedSymbols) + { + var state = HookSymbolDeclarationState.From(symbol); + var exactSource = sourceFacts.TryGetValue(state, out var facts) + ? DequeueUnmatched(facts) + : null; + if (exactSource != null) + { + exactSource.Matched = true; + exactSource.Facts.Apply(symbol); + continue; + } + + SymbolExtractor.RefreshCSharpPartialDeclarationMetadataFromHookSignature(symbol); + var location = HookSymbolDeclarationLocation.From(symbol); + var positionalSource = sourceFactsByLocation.TryGetValue(location, out var locationFacts) + ? DequeueUnmatched(locationFacts) + : null; + if (positionalSource != null) + { + positionalSource.Matched = true; + positionalSource.Facts.PreserveLeadingSourceModifiers(symbol); + } + symbol.DeclarationStructureMutatedByHook = true; + } + } + + private static SourceCSharpDeclarationFacts? DequeueUnmatched( + Queue candidates) + { + while (candidates.Count > 0) + { + var candidate = candidates.Dequeue(); + if (!candidate.Matched) + return candidate; + } + + return null; + } + internal static void RefreshLanguageIdentity(string? language, IEnumerable references) { if (!string.Equals(language, "nim", StringComparison.Ordinal)) @@ -131,6 +201,12 @@ private static SymbolRecord CloneSymbol(SymbolRecord symbol) FamilyKey = symbol.FamilyKey, Visibility = symbol.Visibility, ReturnType = symbol.ReturnType, + IsPartialDeclaration = symbol.IsPartialDeclaration, + IsFileLocalDeclaration = symbol.IsFileLocalDeclaration, + IsExplicitFileLocalDeclaration = symbol.IsExplicitFileLocalDeclaration, + DeclarationStructureMutatedByHook = symbol.DeclarationStructureMutatedByHook, + DeclarationSemanticScore = symbol.DeclarationSemanticScore, + IdentifierStartColumn = symbol.IdentifierStartColumn, IsMetadataTarget = symbol.IsMetadataTarget, MetadataTargetSource = symbol.MetadataTargetSource, SameLineSignatureOccurrenceIndex = symbol.SameLineSignatureOccurrenceIndex, @@ -156,4 +232,121 @@ private static ReferenceRecord CloneReference(ReferenceRecord reference) IsSelfReference = reference.IsSelfReference, IsMutualRecursion = reference.IsMutualRecursion, }; + + private readonly record struct HookSymbolDeclarationState( + long Id, + long FileId, + string Kind, + string? SubKind, + string Name, + int Line, + int StartLine, + int? StartColumn, + int EndLine, + int? BodyStartLine, + int? BodyEndLine, + string? Signature, + string? ContainerKind, + string? ContainerName, + string? ContainerQualifiedName, + int? SameLineSignatureOccurrenceIndex) + { + internal static HookSymbolDeclarationState From(SymbolRecord symbol) + => new( + symbol.Id, + symbol.FileId, + symbol.Kind, + symbol.SubKind, + symbol.Name, + symbol.Line, + symbol.StartLine, + symbol.StartColumn, + symbol.EndLine, + symbol.BodyStartLine, + symbol.BodyEndLine, + symbol.Signature, + symbol.ContainerKind, + symbol.ContainerName, + symbol.ContainerQualifiedName, + symbol.SameLineSignatureOccurrenceIndex); + } + + private readonly record struct HookSymbolDeclarationLocation( + long Id, + long FileId, + int Line, + int StartLine, + int? StartColumn, + int EndLine, + int? BodyStartLine, + int? BodyEndLine, + int? SameLineSignatureOccurrenceIndex) + { + internal static HookSymbolDeclarationLocation From(SymbolRecord symbol) + => new( + symbol.Id, + symbol.FileId, + symbol.Line, + symbol.StartLine, + symbol.StartColumn, + symbol.EndLine, + symbol.BodyStartLine, + symbol.BodyEndLine, + symbol.SameLineSignatureOccurrenceIndex); + } + + private sealed class SourceCSharpDeclarationFacts(CSharpDeclarationFacts facts) + { + internal CSharpDeclarationFacts Facts { get; } = facts; + internal bool Matched { get; set; } + } + + private readonly record struct CSharpDeclarationFacts( + bool? IsPartialDeclaration, + bool IsFileLocalDeclaration, + bool? IsExplicitFileLocalDeclaration, + int? DeclarationSemanticScore, + int? IdentifierStartColumn, + bool SignatureDeclaresPartial, + bool SignatureDeclaresFileLocal) + { + internal static CSharpDeclarationFacts From(SymbolRecord symbol) + { + var signatureFacts = new SymbolRecord + { + Kind = symbol.Kind, + Name = symbol.Name, + Signature = symbol.Signature, + }; + SymbolExtractor.RefreshCSharpPartialDeclarationMetadataFromHookSignature(signatureFacts); + return new( + symbol.IsPartialDeclaration, + symbol.IsFileLocalDeclaration, + symbol.IsExplicitFileLocalDeclaration, + symbol.DeclarationSemanticScore, + symbol.IdentifierStartColumn, + signatureFacts.IsPartialDeclaration == true, + signatureFacts.IsExplicitFileLocalDeclaration == true); + } + + internal void Apply(SymbolRecord symbol) + { + symbol.IsPartialDeclaration = IsPartialDeclaration; + symbol.IsFileLocalDeclaration = IsFileLocalDeclaration; + symbol.IsExplicitFileLocalDeclaration = IsExplicitFileLocalDeclaration; + symbol.DeclarationSemanticScore = DeclarationSemanticScore; + symbol.IdentifierStartColumn = IdentifierStartColumn; + } + + internal void PreserveLeadingSourceModifiers(SymbolRecord symbol) + { + if (IsPartialDeclaration == true && !SignatureDeclaresPartial) + symbol.IsPartialDeclaration = true; + if (IsExplicitFileLocalDeclaration == true && !SignatureDeclaresFileLocal) + { + symbol.IsExplicitFileLocalDeclaration = true; + symbol.IsFileLocalDeclaration = true; + } + } + } } diff --git a/src/CodeIndex/Indexer/Hooks/PostExtractionHooks.cs b/src/CodeIndex/Indexer/Hooks/PostExtractionHooks.cs index 6a0def1ee..f163f1aec 100644 --- a/src/CodeIndex/Indexer/Hooks/PostExtractionHooks.cs +++ b/src/CodeIndex/Indexer/Hooks/PostExtractionHooks.cs @@ -312,6 +312,8 @@ private IReadOnlyList DiscoverAssemblyManifest( public IReadOnlyList Hooks => hooks.Count == 0 ? [] : hooks.Select(hook => hook.Info).ToList(); + internal bool HasHooks => hooks.Count > 0; + internal int ParentLoadContextCountForTests => 0; internal bool SawCSharpStaticInterfaceSourceContract @@ -344,22 +346,30 @@ public void OnSymbolsExtracted(FileContext context, IList symbols, context, symbols, sourceSymbolsAlreadyObserved: false, + content: null, + familyScopeKey: null, cancellationToken); internal void OnSymbolsExtractedAfterSourceObservation( FileContext context, IList symbols, + string? content = null, + string? familyScopeKey = null, CancellationToken cancellationToken = default) => OnSymbolsExtractedCore( context, symbols, sourceSymbolsAlreadyObserved: true, + content, + familyScopeKey, cancellationToken); private void OnSymbolsExtractedCore( FileContext context, IList symbols, bool sourceSymbolsAlreadyObserved, + string? content, + string? familyScopeKey, CancellationToken cancellationToken) { ObjectDisposedException.ThrowIf(disposed, this); @@ -398,11 +408,27 @@ private void OnSymbolsExtractedCore( null, cancellationToken)) { + PostExtractionHookMutationMaterializer.RefreshCSharpDeclarationMetadataAfterHookMutation( + context.Language, + symbols as IReadOnlyList ?? symbols.ToList(), + workingSymbols); PostExtractionHookMutationMaterializer.ReplaceList(symbols, workingSymbols); acceptedHookMutation = true; } } + if (acceptedHookMutation + && string.Equals(context.Language, "csharp", StringComparison.Ordinal) + && !string.IsNullOrWhiteSpace(familyScopeKey)) + { + SymbolExtractor.RefreshCSharpContainerAndFamilyScopeAfterHookMutation( + symbols, + content, + context.FullPath, + context.ProjectRoot, + familyScopeKey); + } + // Hooks can rename or add records but cannot set the internal persisted identity key. // Re-derive it from the accepted public name after all mutations. // hook は record の rename/add はできるが内部の永続化 identity key は設定できないため、 diff --git a/src/CodeIndex/Indexer/Scanning/FileIndexer.GeneratedCode.cs b/src/CodeIndex/Indexer/Scanning/FileIndexer.GeneratedCode.cs index 73e7a1552..5ae544259 100644 --- a/src/CodeIndex/Indexer/Scanning/FileIndexer.GeneratedCode.cs +++ b/src/CodeIndex/Indexer/Scanning/FileIndexer.GeneratedCode.cs @@ -28,7 +28,7 @@ internal bool IsGeneratedCodeExtractionSuppressed(string relativePath) } : null; - private static bool HasGeneratedCodeFileName(string relativePath) + internal static bool HasGeneratedCodeFileName(string relativePath) { var fileName = Path.GetFileName(relativePath.AsSpan()); return fileName.EndsWith(".Designer.cs".AsSpan(), StringComparison.OrdinalIgnoreCase) diff --git a/src/CodeIndex/Indexer/Scanning/FileIndexer.ProjectMarkers.cs b/src/CodeIndex/Indexer/Scanning/FileIndexer.ProjectMarkers.cs index 12ee8ad92..a5ce94ca0 100644 --- a/src/CodeIndex/Indexer/Scanning/FileIndexer.ProjectMarkers.cs +++ b/src/CodeIndex/Indexer/Scanning/FileIndexer.ProjectMarkers.cs @@ -264,11 +264,12 @@ public static string DeriveFallbackFamilyScopeKey(string relativePath) private static string NormalizeScopeKey(string relativePath) { + relativePath = NormalizePathSeparators(relativePath); var start = 0; var end = relativePath.Length; - while (start < end && IsScopeKeySeparator(relativePath[start])) + while (start < end && relativePath[start] == '/') start++; - while (end > start && IsScopeKeySeparator(relativePath[end - 1])) + while (end > start && relativePath[end - 1] == '/') end--; if (start == end) @@ -278,16 +279,11 @@ private static string NormalizeScopeKey(string relativePath) if (span.Length == 1 && span[0] == '.') return "."; - if (span.IndexOf('\\') >= 0) - return span.ToString().Replace('\\', '/'); - return start == 0 && end == relativePath.Length ? relativePath : relativePath[start..end]; } - private static bool IsScopeKeySeparator(char value) => value is '/' or '\\'; - private string DeriveAmbiguousProjectScopeKey(string absolutePath, string anchorDir) { var anchorScope = NormalizeScopeKey(ToRelativePath(anchorDir)); diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpLexing.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpLexing.cs index 9b5a712af..f2db41d41 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpLexing.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpLexing.cs @@ -7,6 +7,35 @@ namespace CodeIndex.Indexer; public static partial class SymbolExtractor { + internal static string SanitizeCSharpDeclarationSignature(string signature) + { + if (!signature.Contains('\n', StringComparison.Ordinal)) + return LexCSharpLine(signature, new CSharpLexState()).SanitizedLine; + + // Preserve offsets while resetting line comments at each newline and carrying + // block-comment/raw-string state across lines. + // offset を保ったまま、改行ごとに line comment を終了し、block comment / raw string + // の state は次行へ引き継ぐ。 + var sanitized = new StringBuilder(signature.Length); + var state = new CSharpLexState(); + var start = 0; + while (start < signature.Length) + { + var newlineOffset = signature.AsSpan(start).IndexOf('\n'); + var lineEnd = newlineOffset < 0 ? signature.Length : start + newlineOffset; + var lexed = LexCSharpLine(signature[start..lineEnd], state); + sanitized.Append(lexed.SanitizedLine); + state = lexed.EndState; + if (lineEnd >= signature.Length) + break; + + sanitized.Append('\n'); + start = lineEnd + 1; + } + + return sanitized.ToString(); + } + private static CSharpLexedLine LexCSharpLine(string line, CSharpLexState state) { if (state.Mode == CSharpLexMode.Code diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpPartialMetadata.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpPartialMetadata.cs new file mode 100644 index 000000000..ca2fdba11 --- /dev/null +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpPartialMetadata.cs @@ -0,0 +1,1547 @@ +using CodeIndex.Models; + +namespace CodeIndex.Indexer; + +public static partial class SymbolExtractor +{ + private const int CSharpLeadingDeclarationLookbackLines = 64; + private static readonly HashSet CSharpStandaloneDeclarationModifiers = new(StringComparer.Ordinal) + { + "public", "protected", "internal", "private", "file", "new", "static", "abstract", + "sealed", "virtual", "override", "readonly", "unsafe", "extern", "partial", "async", + "ref", "required", + }; + private static readonly string[] CSharpTypeDeclarationKeywords = + ["class", "struct", "interface", "record", "enum", "delegate"]; + + private static void PopulateCSharpPartialDeclarationMetadata( + IReadOnlyList lines, + IReadOnlyList symbols, + Func? getCSharpLineStartStates) + { + var lineStartStates = getCSharpLineStartStates?.Invoke(); + var firstDeclarationColumns = GetFirstCSharpDeclarationColumns( + lines, + symbols, + lineStartStates); + foreach (var symbol in symbols) + { + if (symbol.Kind is not ("function" or "test.method" or "class" or "struct" or "interface" or "record" or "enum" or "delegate")) + continue; + + var signature = symbol.Signature ?? string.Empty; + var sanitizedSignature = SanitizeCSharpDeclarationEvidence(signature); + var declarationHeader = ExtractCSharpDeclarationHeader(sanitizedSignature); + var declarationModifierPrefix = ExtractCSharpDeclarationModifierPrefix( + declarationHeader, + symbol); + var leading = ReadCSharpLeadingDeclarationEvidence( + lines, + symbol, + lineStartStates, + IsFirstCSharpDeclarationOnLine( + lines, + symbol, + lineStartStates, + firstDeclarationColumns)); + var supportsPartialDeclaration = symbol.Kind is + "function" or "test.method" or "class" or "struct" or "interface" or "record"; + symbol.IsPartialDeclaration = supportsPartialDeclaration + && (ContainsCSharpLeadingModifier( + declarationModifierPrefix, + "partial", + requireTrailingDeclarationType: symbol.Kind is "function" or "test.method") + || leading.HasPartialModifier); + symbol.IsExplicitFileLocalDeclaration = + symbol.Kind is "class" or "struct" or "interface" or "record" or "enum" or "delegate" + && (ContainsCSharpLeadingModifier(declarationModifierPrefix, "file") + || leading.HasFileModifier); + symbol.IsFileLocalDeclaration = symbol.IsExplicitFileLocalDeclaration == true; + if (symbol.IsPartialDeclaration == true) + { + var identifierPosition = FindCSharpDeclarationIdentifierPosition( + lines, + symbol, + lineStartStates); + if (identifierPosition != null) + { + symbol.Line = identifierPosition.Value.Line; + symbol.IdentifierStartColumn = identifierPosition.Value.Column; + } + } + + var semanticScore = 0; + if (ContainsCSharpAttributeEvidence(declarationHeader) || leading.HasAttribute) + semanticScore += 2; + if (leading.HasDocumentation) + semanticScore += 1; + if (symbol.Kind is "class" or "struct" or "interface" or "record" + && ContainsCSharpTypeBaseList(declarationHeader)) + { + semanticScore += 4; + } + if (ContainsCSharpWhereConstraint(declarationHeader)) + semanticScore += 1; + symbol.DeclarationSemanticScore = semanticScore; + } + } + + internal static void RefreshCSharpPartialDeclarationMetadataFromHookSignature( + SymbolRecord symbol) + { + symbol.IsPartialDeclaration = null; + symbol.IsFileLocalDeclaration = false; + symbol.IsExplicitFileLocalDeclaration = null; + symbol.DeclarationSemanticScore = null; + symbol.IdentifierStartColumn = null; + + if (symbol.Kind is not ("function" or "test.method" or "class" or "struct" or "interface" or "record" or "enum" or "delegate")) + return; + + var sanitizedSignature = SanitizeCSharpDeclarationEvidence(symbol.Signature ?? string.Empty); + var declarationHeader = ExtractCSharpDeclarationHeader(sanitizedSignature); + var declarationModifierPrefix = ExtractCSharpDeclarationModifierPrefix( + declarationHeader, + symbol); + var supportsPartialDeclaration = symbol.Kind is + "function" or "test.method" or "class" or "struct" or "interface" or "record"; + symbol.IsPartialDeclaration = supportsPartialDeclaration + && ContainsCSharpLeadingModifier( + declarationModifierPrefix, + "partial", + requireTrailingDeclarationType: symbol.Kind is "function" or "test.method"); + symbol.IsExplicitFileLocalDeclaration = + symbol.Kind is "class" or "struct" or "interface" or "record" or "enum" or "delegate" + && ContainsCSharpLeadingModifier(declarationModifierPrefix, "file"); + symbol.IsFileLocalDeclaration = symbol.IsExplicitFileLocalDeclaration == true; + + var semanticScore = 0; + if (ContainsCSharpAttributeEvidence(declarationHeader)) + semanticScore += 2; + if (symbol.Kind is "class" or "struct" or "interface" or "record" + && ContainsCSharpTypeBaseList(declarationHeader)) + { + semanticScore += 4; + } + if (ContainsCSharpWhereConstraint(declarationHeader)) + semanticScore += 1; + symbol.DeclarationSemanticScore = semanticScore; + } + + private static Dictionary GetFirstCSharpDeclarationColumns( + IReadOnlyList lines, + IReadOnlyList symbols, + IReadOnlyList? lineStartStates) + { + var firstColumns = new Dictionary(); + foreach (var symbol in symbols) + { + var declarationLine = symbol.Line > 0 ? symbol.Line : symbol.StartLine; + if (declarationLine <= 0 || declarationLine > lines.Count) + continue; + + var lineIndex = declarationLine - 1; + var lineStartState = lineStartStates != null && lineIndex < lineStartStates.Count + ? lineStartStates[lineIndex] + : new CSharpLexState(); + var declarationColumn = FindCSharpDeclarationOccurrenceStartColumn( + lines[lineIndex], + symbol, + lineStartState); + if (declarationColumn < 0) + declarationColumn = symbol.StartColumn ?? int.MaxValue; + + if (!firstColumns.TryGetValue(declarationLine, out var firstColumn) + || declarationColumn < firstColumn) + { + firstColumns[declarationLine] = declarationColumn; + } + } + + return firstColumns; + } + + private static bool IsFirstCSharpDeclarationOnLine( + IReadOnlyList lines, + SymbolRecord symbol, + IReadOnlyList? lineStartStates, + IReadOnlyDictionary firstDeclarationColumns) + { + var declarationLine = symbol.Line > 0 ? symbol.Line : symbol.StartLine; + if (declarationLine <= 0 + || declarationLine > lines.Count + || !firstDeclarationColumns.TryGetValue(declarationLine, out var firstColumn)) + { + return true; + } + + var lineIndex = declarationLine - 1; + var lineStartState = lineStartStates != null && lineIndex < lineStartStates.Count + ? lineStartStates[lineIndex] + : new CSharpLexState(); + var declarationColumn = FindCSharpDeclarationOccurrenceStartColumn( + lines[lineIndex], + symbol, + lineStartState); + if (declarationColumn < 0) + declarationColumn = symbol.StartColumn ?? int.MaxValue; + return declarationColumn <= firstColumn; + } + + private static string SanitizeCSharpDeclarationEvidence(string signature) + { + if (string.IsNullOrEmpty(signature)) + return string.Empty; + + var sanitized = new System.Text.StringBuilder(signature.Length); + var state = new CSharpLexState(); + var lineStart = 0; + while (lineStart <= signature.Length) + { + var lineEnd = signature.IndexOf('\n', lineStart); + if (lineEnd < 0) + lineEnd = signature.Length; + + var lexed = LexCSharpLine(signature[lineStart..lineEnd], state); + if (sanitized.Length > 0) + sanitized.Append('\n'); + sanitized.Append(lexed.SanitizedLine); + state = lexed.EndState; + + if (lineEnd == signature.Length) + break; + lineStart = lineEnd + 1; + } + + return sanitized.ToString(); + } + + private static string ExtractCSharpDeclarationHeader(string declaration) + { + var parenthesisDepth = 0; + var bracketDepth = 0; + for (var index = 0; index < declaration.Length; index++) + { + switch (declaration[index]) + { + case '(': + parenthesisDepth++; + break; + case ')' when parenthesisDepth > 0: + parenthesisDepth--; + break; + case '[': + bracketDepth++; + break; + case ']' when bracketDepth > 0: + bracketDepth--; + break; + case '{' when parenthesisDepth == 0 && bracketDepth == 0: + return declaration[..index]; + case '=' when parenthesisDepth == 0 + && bracketDepth == 0 + && index + 1 < declaration.Length + && declaration[index + 1] == '>': + return declaration[..index]; + } + } + + return declaration; + } + + private static string ExtractCSharpTypeDeclarationModifierPrefix( + string declarationHeader, + string symbolName) + { + var header = declarationHeader.AsSpan(); + var name = symbolName.AsSpan().TrimStart('@'); + var bestKeywordIndex = -1; + var bestKeywordDistance = int.MaxValue; + foreach (var keyword in CSharpTypeDeclarationKeywords) + { + var searchStart = 0; + while (searchStart < header.Length) + { + var keywordIndex = FindCSharpDeclarationKeywordToken( + header, + keyword, + searchStart); + if (keywordIndex < 0) + break; + + if (keyword is "class" or "struct" + && IsCSharpExplicitRecordSuffix(header, keywordIndex)) + { + searchStart = keywordIndex + keyword.Length; + continue; + } + + var nameIndex = FindCSharpIdentifierToken( + header, + name, + keywordIndex + keyword.Length); + if (nameIndex >= 0) + { + var distance = nameIndex - keywordIndex - keyword.Length; + if (distance < bestKeywordDistance) + { + bestKeywordIndex = keywordIndex; + bestKeywordDistance = distance; + } + } + + searchStart = keywordIndex + keyword.Length; + } + } + + if (bestKeywordIndex < 0) + return declarationHeader; + + var prefix = declarationHeader[..bestKeywordIndex]; + var attributeDepth = 0; + var lastAttributeEnd = -1; + for (var index = 0; index < prefix.Length; index++) + { + if (prefix[index] == '[') + { + attributeDepth++; + } + else if (prefix[index] == ']' && attributeDepth > 0) + { + attributeDepth--; + if (attributeDepth == 0) + lastAttributeEnd = index; + } + } + + return lastAttributeEnd >= 0 + ? prefix[(lastAttributeEnd + 1)..] + : prefix; + } + + internal static bool ContainsCSharpTypeBaseList(string declarationHeader) + { + var bracketDepth = 0; + var parenthesisDepth = 0; + var angleDepth = 0; + var declarationKeywordSeen = false; + var declarationNameSeen = false; + var recordMayHaveExplicitKind = false; + + for (var index = 0; index < declarationHeader.Length;) + { + var character = declarationHeader[index]; + switch (character) + { + case '[': + bracketDepth++; + index++; + continue; + case ']' when bracketDepth > 0: + bracketDepth--; + index++; + continue; + case '(' when bracketDepth == 0: + parenthesisDepth++; + index++; + continue; + case ')' when bracketDepth == 0 && parenthesisDepth > 0: + parenthesisDepth--; + index++; + continue; + case '<' when bracketDepth == 0 && parenthesisDepth == 0: + angleDepth++; + index++; + continue; + case '>' when bracketDepth == 0 && parenthesisDepth == 0 && angleDepth > 0: + angleDepth--; + index++; + continue; + case ':' when declarationNameSeen + && bracketDepth == 0 + && parenthesisDepth == 0 + && angleDepth == 0: + return true; + } + + if (bracketDepth != 0 + || parenthesisDepth != 0 + || angleDepth != 0 + || !(character is '@' or '_' || char.IsLetter(character))) + { + index++; + continue; + } + + var tokenStart = index; + var tokenIsVerbatim = character == '@'; + index++; + while (index < declarationHeader.Length && IsCSharpIdentifierPart(declarationHeader[index])) + index++; + var token = declarationHeader.AsSpan(tokenStart, index - tokenStart).TrimStart('@'); + + if (!declarationKeywordSeen) + { + if (!tokenIsVerbatim + && (token.SequenceEqual("class") + || token.SequenceEqual("struct") + || token.SequenceEqual("interface"))) + { + declarationKeywordSeen = true; + } + else if (!tokenIsVerbatim && token.SequenceEqual("record")) + { + declarationKeywordSeen = true; + recordMayHaveExplicitKind = true; + } + continue; + } + + if (!declarationNameSeen) + { + if (recordMayHaveExplicitKind + && !tokenIsVerbatim + && (token.SequenceEqual("class") || token.SequenceEqual("struct"))) + { + recordMayHaveExplicitKind = false; + continue; + } + + declarationNameSeen = true; + continue; + } + + if (!tokenIsVerbatim && token.SequenceEqual("where")) + return false; + } + + return false; + } + + private static bool IsCSharpExplicitRecordSuffix( + ReadOnlySpan declarationHeader, + int keywordIndex) + { + var cursor = keywordIndex - 1; + while (cursor >= 0 && char.IsWhiteSpace(declarationHeader[cursor])) + cursor--; + + const string RecordKeyword = "record"; + var recordStart = cursor - RecordKeyword.Length + 1; + return recordStart >= 0 + && declarationHeader.Slice(recordStart, RecordKeyword.Length).SequenceEqual(RecordKeyword) + && (recordStart == 0 || !IsCSharpIdentifierPart(declarationHeader[recordStart - 1])); + } + + private static string ExtractCSharpDeclarationModifierPrefix( + string declarationHeader, + SymbolRecord symbol) + => symbol.Kind is "class" or "struct" or "interface" or "record" or "enum" or "delegate" + ? ExtractCSharpTypeDeclarationModifierPrefix(declarationHeader, symbol.Name) + : ExtractCSharpCallableDeclarationModifierPrefix(declarationHeader, symbol.Name); + + private static string ExtractCSharpCallableDeclarationModifierPrefix( + string declarationHeader, + string symbolName) + { + var header = declarationHeader.AsSpan(); + var name = symbolName.AsSpan().TrimStart('@'); + var searchStart = 0; + while (searchStart < header.Length) + { + var nameColumn = FindCSharpIdentifierToken(header, name, searchStart); + if (nameColumn < 0) + break; + + if (IsOutsideCSharpAttributeList(header, nameColumn) + && IsCSharpCallableNameOccurrence(header, nameColumn, name.Length)) + { + return declarationHeader[..nameColumn]; + } + + searchStart = nameColumn + Math.Max(1, name.Length); + } + + return declarationHeader; + } + + private static bool ContainsCSharpLeadingModifier( + string declarationPrefix, + string modifier, + bool requireTrailingDeclarationType = false) + { + var remaining = declarationPrefix.AsSpan().Trim(); + while (!remaining.IsEmpty && remaining[0] == '[') + { + var depth = 0; + var attributeEnd = -1; + for (var index = 0; index < remaining.Length; index++) + { + if (remaining[index] == '[') + depth++; + else if (remaining[index] == ']' && --depth == 0) + { + attributeEnd = index; + break; + } + } + + if (attributeEnd < 0) + return false; + remaining = remaining[(attributeEnd + 1)..].TrimStart(); + } + + var hasModifier = false; + while (!remaining.IsEmpty) + { + if (remaining[0] == '@') + return requireTrailingDeclarationType && hasModifier; + + var tokenLength = 0; + while (tokenLength < remaining.Length + && (remaining[tokenLength] == '_' + || char.IsLetterOrDigit(remaining[tokenLength]))) + { + tokenLength++; + } + + if (tokenLength == 0) + return requireTrailingDeclarationType && hasModifier; + var token = remaining[..tokenLength]; + if (!CSharpStandaloneDeclarationModifiers.Contains(token.ToString())) + return requireTrailingDeclarationType && hasModifier; + + var trailing = remaining[tokenLength..].TrimStart(); + if (requireTrailingDeclarationType + && (trailing.IsEmpty || IsCSharpTypeContinuation(trailing[0]))) + { + // A contextual keyword at the end of a callable prefix, or followed by + // type punctuation, is the return type rather than a declaration modifier. + // callable prefix 末尾、または型 punctuation の直前にある contextual + // keyword は declaration modifier ではなく return type とみなす。 + return hasModifier; + } + + hasModifier |= token.SequenceEqual(modifier); + remaining = trailing; + } + + return !requireTrailingDeclarationType && hasModifier; + } + + private static bool IsCSharpTypeContinuation(char value) + => value is '.' or ':' or '<' or '[' or '?' or '*'; + + internal static bool ContainsCSharpPartialDeclarationModifier( + string? signature, + string? kind, + string? symbolName) + { + if (string.IsNullOrWhiteSpace(signature) + || string.IsNullOrWhiteSpace(kind) + || string.IsNullOrWhiteSpace(symbolName) + || kind is not ("function" or "test.method" or "class" or "struct" or "interface" or "record")) + { + return false; + } + + var declarationHeader = ExtractCSharpDeclarationHeader( + SanitizeCSharpDeclarationEvidence(signature)); + var modifierPrefix = kind is "class" or "struct" or "interface" or "record" + ? ExtractCSharpTypeDeclarationModifierPrefix(declarationHeader, symbolName) + : ExtractCSharpCallableDeclarationModifierPrefix(declarationHeader, symbolName); + return ContainsCSharpLeadingModifier( + modifierPrefix, + "partial", + requireTrailingDeclarationType: kind is "function" or "test.method"); + } + + private static bool ContainsCSharpAttributeEvidence(string declarationHeader) + { + for (var index = 0; index < declarationHeader.Length; index++) + { + if (declarationHeader[index] != '[') + continue; + + var previous = index - 1; + while (previous >= 0 && char.IsWhiteSpace(declarationHeader[previous])) + previous--; + if ((previous < 0 || declarationHeader[previous] is '(' or ',' or '<') + && !IsCSharpGlobalAttributeTarget(declarationHeader.AsSpan(index + 1))) + return true; + } + + return false; + } + + private static bool IsCSharpGlobalAttributeTarget(ReadOnlySpan text) + { + var remaining = text.TrimStart(); + if (!remaining.IsEmpty && remaining[0] == '[') + remaining = remaining[1..].TrimStart(); + + foreach (var target in new[] { "assembly", "module" }) + { + if (!remaining.StartsWith(target, StringComparison.Ordinal)) + continue; + + var cursor = target.Length; + if (cursor < remaining.Length && IsCSharpIdentifierPart(remaining[cursor])) + continue; + while (cursor < remaining.Length && char.IsWhiteSpace(remaining[cursor])) + cursor++; + if (cursor < remaining.Length && remaining[cursor] == ':') + return true; + } + + return false; + } + + private static bool ContainsCSharpModifier(string declaration, string modifier) + { + var searchStart = 0; + while (searchStart <= declaration.Length - modifier.Length) + { + var relative = declaration.AsSpan(searchStart).IndexOf(modifier, StringComparison.Ordinal); + if (relative < 0) + return false; + + var index = searchStart + relative; + var beforeIsIdentifier = index > 0 && IsCSharpIdentifierPart(declaration[index - 1]); + var afterIndex = index + modifier.Length; + var afterIsIdentifier = afterIndex < declaration.Length && IsCSharpIdentifierPart(declaration[afterIndex]); + if (!beforeIsIdentifier && !afterIsIdentifier) + return true; + + searchStart = index + Math.Max(1, modifier.Length); + } + + return false; + } + + internal static bool ContainsCSharpWhereConstraint(string declarationHeader) + => ContainsCSharpModifier(declarationHeader, "where"); + + private static CSharpLeadingDeclarationEvidence ReadCSharpLeadingDeclarationEvidence( + IReadOnlyList lines, + SymbolRecord symbol, + IReadOnlyList? lineStartStates, + bool consumePreviousLineEvidence) + { + var declarationStartLine = symbol.StartLine; + var lineIndex = Math.Min(lines.Count, Math.Max(1, declarationStartLine)) - 2; + var minimumLineIndex = Math.Max(0, lineIndex - CSharpLeadingDeclarationLookbackLines + 1); + if (lineStartStates != null) + { + // Keep the ordinary evidence scan bounded, but if its boundary lands inside + // one delimited comment, extend only to that comment's lexer-confirmed opener. + // This preserves adjacent long `/** ... */` documentation without allowing + // unrelated modifiers or attributes arbitrarily far above the declaration. + // 通常の evidence scan は上限を維持する。ただし境界が delimited comment + // 内なら lexer が確認した opener までだけ延長し、離れた modifier / attribute + // を拾わずに長い `/** ... */` documentation の隣接性を保持する。 + while (minimumLineIndex > 0 + && minimumLineIndex < lineStartStates.Count + && lineStartStates[minimumLineIndex].Mode == CSharpLexMode.BlockComment) + { + minimumLineIndex--; + } + } + var hasPartialModifier = false; + var hasFileModifier = false; + var hasAttribute = HasCSharpDeclarationLineLeadingAttribute( + lines, + symbol, + lineStartStates); + var hasDocumentation = HasCSharpDeclarationLineLeadingDocumentation( + lines, + symbol, + lineStartStates); + var documentationEvidenceAdjacent = true; + var attributeDepth = 0; + var pendingAttributeEvidence = false; + var pendingAttributeIsGlobal = false; + var closedConditionalDirectiveDepth = 0; + var skippingConditionalSiblingBranch = false; + + // Standalone modifiers, attributes, and documentation on preceding lines bind to + // the first declaration occurrence on the next line. Later same-line declarations + // must derive evidence only from their own declaration text. + // preceding line の standalone modifier・attribute・documentation は次行の最初の + // declaration occurrence にだけ属する。同一行の後続宣言は自身の宣言 text だけを使う。 + if (!consumePreviousLineEvidence) + { + return new CSharpLeadingDeclarationEvidence( + HasPartialModifier: false, + HasFileModifier: false, + HasAttribute: hasAttribute, + HasDocumentation: hasDocumentation); + } + + for (; lineIndex >= minimumLineIndex; lineIndex--) + { + var raw = lines[lineIndex].AsSpan().Trim(); + var lineStartState = lineStartStates != null && lineIndex < lineStartStates.Count + ? lineStartStates[lineIndex] + : new CSharpLexState(); + if (raw.IsEmpty) + { + // Whitespace is valid declaration trivia between standalone modifiers + // and the declaration. Outside an active delimited comment it detaches + // XML documentation from the declaration for representative ranking; + // inside `/** ... */` it remains part of the same documentation comment. + // standalone modifier と宣言の間の空行は有効な declaration trivia だが、 + // active な `/** ... */` の外側なら XML documentation の representative + // rank 上の隣接性を切り、内側なら同じ documentation comment として維持する。 + if (lineStartState.Mode != CSharpLexMode.BlockComment) + documentationEvidenceAdjacent = false; + continue; + } + + var startsInDeclarationCode = lineStartState.Mode == CSharpLexMode.Code + && lineStartState.InterpolationBraceDepth == 0; + var sanitizedLine = LexCSharpLine(lines[lineIndex], lineStartState).SanitizedLine; + var trimmed = sanitizedLine.AsSpan().Trim(); + if (startsInDeclarationCode + && TryReadCSharpDirectiveKeyword(trimmed, out var directiveKeyword)) + { + if (directiveKeyword == "endif" + && closedConditionalDirectiveDepth == 0 + && TryReadClosedCSharpConditionalDeclarationEvidence( + lines, + lineStartStates, + minimumLineIndex, + lineIndex, + out var conditionalEvidence)) + { + if (conditionalEvidence.HasCodeBoundary) + break; + + // A completed conditional contributes `partial` and declaration-attribute + // evidence only when every possible branch contributes it. Conversely, any + // branch-local `file` is retained because grouping that declaration across + // files would be unsafe in that compilation. Without an explicit `#else`, + // include an implicit empty branch in all decisions. + // 完了した conditional の `partial` と declaration attribute は全分岐が + // 供給する場合だけ採用する。一方、branch-local な `file` は、その + // compilation で別ファイルと grouping すると危険なため一分岐だけでも + // 保持する。明示的な `#else` がなければ全判定に暗黙の空分岐を含める。 + hasPartialModifier |= conditionalEvidence.HasPartialModifier; + hasFileModifier |= conditionalEvidence.HasFileModifier; + hasAttribute |= conditionalEvidence.HasAttribute; + lineIndex = conditionalEvidence.OpeningDirectiveLineIndex; + continue; + } + + switch (directiveKeyword) + { + case "endif": + closedConditionalDirectiveDepth++; + break; + case "if" when closedConditionalDirectiveDepth > 0: + closedConditionalDirectiveDepth--; + if (closedConditionalDirectiveDepth == 0) + skippingConditionalSiblingBranch = false; + break; + case "else" or "elif" when closedConditionalDirectiveDepth == 0: + // Skip the sibling branch, then resume before its matching `#if`. + // A standalone modifier before that conditional still belongs to + // every declaration alternative inside it. + // 兄弟分岐を読み飛ばし、対応する `#if` より前から走査を再開する。 + // conditional より前の standalone modifier は各宣言候補に属する。 + closedConditionalDirectiveDepth = 1; + skippingConditionalSiblingBranch = true; + break; + } + continue; + } + + if (closedConditionalDirectiveDepth > 0) + { + // A sibling branch must be ignored through its matching `#if`, but code in + // a completed conditional before the declaration is still a declaration + // boundary. This keeps an outer modifier bound to the declaration inside + // that completed block instead of lending it to the following declaration. + // 兄弟分岐は対応する `#if` まで無視する一方、宣言前に完了した conditional + // 内の code は declaration 境界とする。外側 modifier を block 内の宣言から + // 後続宣言へ貸し出さない。 + if (skippingConditionalSiblingBranch || trimmed.IsEmpty) + continue; + break; + } + + if (startsInDeclarationCode && raw.StartsWith("///", StringComparison.Ordinal)) + { + hasDocumentation |= documentationEvidenceAdjacent; + continue; + } + + if (startsInDeclarationCode && raw.StartsWith("/**", StringComparison.Ordinal)) + { + hasDocumentation |= documentationEvidenceAdjacent; + continue; + } + + if (trimmed.IsEmpty) + continue; + + var lastAttributeClose = trimmed.LastIndexOf(']'); + var trailingModifiers = lastAttributeClose >= 0 + ? trimmed[(lastAttributeClose + 1)..].Trim() + : ReadOnlySpan.Empty; + var trailingHasPartial = false; + var trailingHasFile = false; + var hasTrailingModifiers = !trailingModifiers.IsEmpty + && TryReadStandaloneCSharpModifiers( + trailingModifiers, + out trailingHasPartial, + out trailingHasFile); + var isAttributeLine = attributeDepth > 0 + || trimmed[0] == '[' + || trimmed[^1] == ']' + || hasTrailingModifiers; + if (isAttributeLine) + { + if (!trailingModifiers.IsEmpty) + { + if (!hasTrailingModifiers) + break; + + hasPartialModifier |= trailingHasPartial; + hasFileModifier |= trailingHasFile; + } + pendingAttributeEvidence = true; + pendingAttributeIsGlobal |= IsCSharpGlobalAttributeTarget(trimmed); + attributeDepth += CountCharacter(trimmed, ']') - CountCharacter(trimmed, '['); + attributeDepth = Math.Max(0, attributeDepth); + if (attributeDepth == 0) + { + hasAttribute |= pendingAttributeEvidence && !pendingAttributeIsGlobal; + pendingAttributeEvidence = false; + pendingAttributeIsGlobal = false; + } + continue; + } + + if (!TryReadStandaloneCSharpModifiers(trimmed, out var hasPartial, out var hasFile)) + break; + + hasPartialModifier |= hasPartial; + hasFileModifier |= hasFile; + } + + return new CSharpLeadingDeclarationEvidence( + hasPartialModifier, + hasFileModifier, + hasAttribute, + hasDocumentation); + } + + private static bool TryReadClosedCSharpConditionalDeclarationEvidence( + IReadOnlyList lines, + IReadOnlyList? lineStartStates, + int minimumLineIndex, + int closingDirectiveLineIndex, + out CSharpClosedConditionalDeclarationEvidence evidence) + { + evidence = default; + var depth = 0; + for (var lineIndex = closingDirectiveLineIndex; + lineIndex >= minimumLineIndex; + lineIndex--) + { + if (!TryReadCSharpDirectiveKeywordAtLine( + lines, + lineStartStates, + lineIndex, + out var directiveKeyword)) + { + continue; + } + + if (directiveKeyword == "endif") + { + depth++; + continue; + } + + if (directiveKeyword != "if") + continue; + + depth--; + if (depth != 0) + continue; + + var branchEvidence = ReadCSharpConditionalBranchEvidence( + lines, + lineStartStates, + lineIndex, + closingDirectiveLineIndex); + evidence = new CSharpClosedConditionalDeclarationEvidence( + lineIndex, + branchEvidence.HasCodeBoundary, + branchEvidence.HasPartialModifier, + branchEvidence.HasFileModifier, + branchEvidence.HasAttribute); + return true; + } + + return false; + } + + private static CSharpConditionalBranchEvidence ReadCSharpConditionalBranchEvidence( + IReadOnlyList lines, + IReadOnlyList? lineStartStates, + int openingDirectiveLineIndex, + int closingDirectiveLineIndex) + { + var branches = new List(); + var current = new CSharpConditionalBranchEvidence(); + var hasExplicitElse = false; + + for (var lineIndex = openingDirectiveLineIndex + 1; + lineIndex < closingDirectiveLineIndex; + lineIndex++) + { + if (TryReadCSharpDirectiveKeywordAtLine( + lines, + lineStartStates, + lineIndex, + out var directiveKeyword)) + { + if (directiveKeyword == "if") + { + if (!TryFindMatchingCSharpEndifDirective( + lines, + lineStartStates, + lineIndex, + closingDirectiveLineIndex, + out var nestedClosingDirectiveLineIndex)) + { + return current with { HasCodeBoundary = true }; + } + + var nested = ReadCSharpConditionalBranchEvidence( + lines, + lineStartStates, + lineIndex, + nestedClosingDirectiveLineIndex); + current = current with + { + HasCodeBoundary = current.HasCodeBoundary || nested.HasCodeBoundary, + HasPartialModifier = current.HasPartialModifier || nested.HasPartialModifier, + HasFileModifier = current.HasFileModifier || nested.HasFileModifier, + HasAttribute = current.HasAttribute || nested.HasAttribute, + }; + lineIndex = nestedClosingDirectiveLineIndex; + continue; + } + + if (directiveKeyword is "else" or "elif") + { + branches.Add(CompleteCSharpConditionalBranchEvidence(current)); + current = new CSharpConditionalBranchEvidence(); + hasExplicitElse |= directiveKeyword == "else"; + } + + // Non-conditional directives are declaration trivia. Nested `#endif` + // directives are consumed together with their matching `#if` above. + // conditional 以外の directive は declaration trivia とする。nested + // `#endif` は対応する `#if` と一緒に上で消費済みである。 + continue; + } + + current = AccumulateCSharpConditionalBranchLineEvidence( + lines, + lineStartStates, + lineIndex, + current); + } + + branches.Add(CompleteCSharpConditionalBranchEvidence(current)); + if (!hasExplicitElse) + branches.Add(new CSharpConditionalBranchEvidence()); + + return new CSharpConditionalBranchEvidence( + HasCodeBoundary: branches.Any(branch => branch.HasCodeBoundary), + HasPartialModifier: branches.All(branch => branch.HasPartialModifier), + HasFileModifier: branches.Any(branch => branch.HasFileModifier), + HasAttribute: branches.All(branch => branch.HasAttribute)); + } + + private static CSharpConditionalBranchEvidence CompleteCSharpConditionalBranchEvidence( + CSharpConditionalBranchEvidence evidence) + => evidence.AttributeDepth == 0 + ? evidence + : evidence with { HasCodeBoundary = true }; + + private static CSharpConditionalBranchEvidence AccumulateCSharpConditionalBranchLineEvidence( + IReadOnlyList lines, + IReadOnlyList? lineStartStates, + int lineIndex, + CSharpConditionalBranchEvidence evidence) + { + var lineStartState = lineStartStates != null && lineIndex < lineStartStates.Count + ? lineStartStates[lineIndex] + : new CSharpLexState(); + var trimmed = LexCSharpLine(lines[lineIndex], lineStartState).SanitizedLine.AsSpan().Trim(); + if (trimmed.IsEmpty) + return evidence; + + if (evidence.AttributeDepth > 0 || trimmed[0] == '[') + { + var pendingAttributeIsGlobal = evidence.PendingAttributeIsGlobal + || IsCSharpGlobalAttributeTarget(trimmed); + var attributeDepth = Math.Max( + 0, + evidence.AttributeDepth + + CountCharacter(trimmed, '[') + - CountCharacter(trimmed, ']')); + var updated = evidence with + { + AttributeDepth = attributeDepth, + PendingAttributeIsGlobal = pendingAttributeIsGlobal, + }; + if (attributeDepth > 0) + return updated; + + updated = updated with + { + HasAttribute = updated.HasAttribute || !pendingAttributeIsGlobal, + PendingAttributeIsGlobal = false, + }; + + var lastAttributeClose = trimmed.LastIndexOf(']'); + var trailing = lastAttributeClose >= 0 + ? trimmed[(lastAttributeClose + 1)..].Trim() + : ReadOnlySpan.Empty; + if (trailing.IsEmpty) + return updated; + if (!TryReadStandaloneCSharpModifiers( + trailing, + out var trailingHasPartial, + out var trailingHasFile)) + { + return updated with { HasCodeBoundary = true }; + } + + return updated with + { + HasPartialModifier = updated.HasPartialModifier || trailingHasPartial, + HasFileModifier = updated.HasFileModifier || trailingHasFile, + }; + } + + if (!TryReadStandaloneCSharpModifiers( + trimmed, + out var hasPartial, + out var hasFile)) + { + return evidence with { HasCodeBoundary = true }; + } + + return evidence with + { + HasPartialModifier = evidence.HasPartialModifier || hasPartial, + HasFileModifier = evidence.HasFileModifier || hasFile, + }; + } + + private static bool TryFindMatchingCSharpEndifDirective( + IReadOnlyList lines, + IReadOnlyList? lineStartStates, + int openingDirectiveLineIndex, + int exclusiveMaximumLineIndex, + out int closingDirectiveLineIndex) + { + var depth = 0; + for (var lineIndex = openingDirectiveLineIndex; + lineIndex < exclusiveMaximumLineIndex; + lineIndex++) + { + if (!TryReadCSharpDirectiveKeywordAtLine( + lines, + lineStartStates, + lineIndex, + out var directiveKeyword)) + { + continue; + } + + if (directiveKeyword == "if") + { + depth++; + continue; + } + + if (directiveKeyword != "endif") + continue; + + depth--; + if (depth == 0) + { + closingDirectiveLineIndex = lineIndex; + return true; + } + } + + closingDirectiveLineIndex = -1; + return false; + } + + private static bool TryReadCSharpDirectiveKeywordAtLine( + IReadOnlyList lines, + IReadOnlyList? lineStartStates, + int lineIndex, + out string directiveKeyword) + { + directiveKeyword = string.Empty; + var lineStartState = lineStartStates != null && lineIndex < lineStartStates.Count + ? lineStartStates[lineIndex] + : new CSharpLexState(); + if (lineStartState.Mode != CSharpLexMode.Code + || lineStartState.InterpolationBraceDepth != 0) + { + return false; + } + + var trimmed = LexCSharpLine(lines[lineIndex], lineStartState).SanitizedLine.AsSpan().Trim(); + return TryReadCSharpDirectiveKeyword(trimmed, out directiveKeyword); + } + + private static bool TryReadCSharpDirectiveKeyword( + ReadOnlySpan line, + out string keyword) + { + keyword = string.Empty; + if (line.IsEmpty || line[0] != '#') + return false; + + var directive = line[1..].TrimStart(); + foreach (var candidate in new[] { "endif", "elif", "else", "if" }) + { + if (!directive.StartsWith(candidate, StringComparison.Ordinal) + || (directive.Length > candidate.Length + && IsCSharpIdentifierPart(directive[candidate.Length]))) + { + continue; + } + + keyword = candidate; + return true; + } + + // Other directives (`#nullable`, `#pragma`, `#line`, and so on) are also + // declaration trivia, but do not change the conditional-branch depth. + // `#nullable`、`#pragma`、`#line` などのほかの directive も declaration + // trivia だが、条件分岐の深さは変更しない。 + return true; + } + + private static bool HasCSharpDeclarationLineLeadingAttribute( + IReadOnlyList lines, + SymbolRecord symbol, + IReadOnlyList? lineStartStates) + { + var declarationLine = symbol.Line > 0 ? symbol.Line : symbol.StartLine; + if (declarationLine <= 0 || declarationLine > lines.Count) + return false; + + var lineIndex = declarationLine - 1; + var lineStartState = lineStartStates != null && lineIndex < lineStartStates.Count + ? lineStartStates[lineIndex] + : new CSharpLexState(); + if (lineStartState.Mode != CSharpLexMode.Code || lineStartState.InterpolationBraceDepth != 0) + return false; + + var declarationStartColumn = FindCSharpDeclarationOccurrenceStartColumn( + lines[lineIndex], + symbol, + lineStartState); + if (declarationStartColumn <= 0) + return false; + + var sanitizedLine = LexCSharpLine(lines[lineIndex], lineStartState).SanitizedLine.AsSpan(); + var cursor = Math.Min(declarationStartColumn, sanitizedLine.Length) - 1; + while (cursor >= 0 && char.IsWhiteSpace(sanitizedLine[cursor])) + cursor--; + + // An attribute belongs to this declaration only when its closing bracket is the + // last code token before this declaration occurrence. Assembly/module targets are + // compilation-unit metadata, not semantic evidence for the following declaration. + // attribute の閉じ括弧がこの宣言 occurrence 直前の最後の code token である場合だけ、 + // この宣言の attribute とみなす。assembly/module target は compilation-unit の + // metadata であり、後続宣言の semantic evidence ではない。 + if (cursor < 0 || sanitizedLine[cursor] != ']') + return false; + + var attributeStart = FindCSharpAttributeStart(sanitizedLine, cursor); + return attributeStart >= 0 + && !IsCSharpGlobalAttributeTarget(sanitizedLine[attributeStart..(cursor + 1)]); + } + + private static int FindCSharpAttributeStart(ReadOnlySpan line, int closingBracket) + { + var depth = 0; + for (var cursor = closingBracket; cursor >= 0; cursor--) + { + if (line[cursor] == ']') + { + depth++; + continue; + } + + if (line[cursor] != '[') + continue; + + depth--; + if (depth == 0) + return cursor; + } + + return -1; + } + + private static bool HasCSharpDeclarationLineLeadingDocumentation( + IReadOnlyList lines, + SymbolRecord symbol, + IReadOnlyList? lineStartStates) + { + var declarationLine = symbol.Line > 0 ? symbol.Line : symbol.StartLine; + if (declarationLine <= 0 || declarationLine > lines.Count) + return false; + + var lineIndex = declarationLine - 1; + var lineStartState = lineStartStates != null && lineIndex < lineStartStates.Count + ? lineStartStates[lineIndex] + : new CSharpLexState(); + if (lineStartState.Mode != CSharpLexMode.Code || lineStartState.InterpolationBraceDepth != 0) + return false; + + var rawLine = lines[lineIndex]; + var declarationStartColumn = FindCSharpDeclarationOccurrenceStartColumn( + rawLine, + symbol, + lineStartState); + if (declarationStartColumn <= 0) + return false; + + var cursor = Math.Min(declarationStartColumn, rawLine.Length) - 1; + while (cursor >= 0 && char.IsWhiteSpace(rawLine[cursor])) + cursor--; + if (cursor < 1 || rawLine[cursor - 1] != '*' || rawLine[cursor] != '/') + return false; + + var expectedCommentEnd = cursor - 1; + var commentStart = rawLine.LastIndexOf("/**", expectedCommentEnd, StringComparison.Ordinal); + while (commentStart >= 0) + { + // Re-lex the prefix so a `/**` sequence inside a normal block comment, + // string, character literal, or interpolation hole cannot become documentation. + // prefix を再 lex し、通常 block comment・string・character literal・ + // interpolation hole 内の `/**` を documentation と誤認しない。 + var stateAtCommentStart = LexCSharpLine(rawLine[..commentStart], lineStartState).EndState; + if (stateAtCommentStart.Mode == CSharpLexMode.Code + && stateAtCommentStart.InterpolationReturnMode == CSharpLexMode.Code + && stateAtCommentStart.InterpolationBraceDepth == 0 + && rawLine.IndexOf("*/", commentStart + 2, StringComparison.Ordinal) == expectedCommentEnd) + { + return true; + } + + commentStart = commentStart == 0 + ? -1 + : rawLine.LastIndexOf("/**", commentStart - 1, StringComparison.Ordinal); + } + + return false; + } + + private static int FindCSharpDeclarationOccurrenceStartColumn( + string rawLine, + SymbolRecord symbol, + CSharpLexState lineStartState) + { + if (!string.IsNullOrEmpty(symbol.Signature)) + { + var signatureColumn = FindSignatureOccurrenceStartColumn( + rawLine, + symbol.Signature, + symbol.SameLineSignatureOccurrenceIndex ?? 0, + lineStartState); + if (signatureColumn >= 0) + return signatureColumn; + } + + return symbol.StartColumn ?? -1; + } + + private static bool TryReadStandaloneCSharpModifiers( + ReadOnlySpan line, + out bool hasPartial, + out bool hasFile) + { + hasPartial = false; + hasFile = false; + var remaining = line; + var found = false; + while (!remaining.IsEmpty) + { + var separator = remaining.IndexOfAny(' ', '\t'); + var token = separator < 0 ? remaining : remaining[..separator]; + if (!CSharpStandaloneDeclarationModifiers.Contains(token.ToString())) + return false; + + found = true; + hasPartial |= token.SequenceEqual("partial"); + hasFile |= token.SequenceEqual("file"); + if (separator < 0) + break; + remaining = remaining[(separator + 1)..].TrimStart(); + } + return found; + } + + private static (int Line, int Column)? FindCSharpDeclarationIdentifierPosition( + IReadOnlyList lines, + SymbolRecord symbol, + IReadOnlyList? lineStartStates) + { + if (symbol.Line <= 0 || symbol.Line > lines.Count || string.IsNullOrWhiteSpace(symbol.Name)) + return null; + + var name = symbol.Name.AsSpan().TrimStart('@'); + if (name.IsEmpty) + return null; + + var firstLine = symbol.Line; + var lastLine = Math.Clamp( + symbol.BodyStartLine ?? symbol.EndLine, + firstLine, + lines.Count); + (int Line, int Column)? fallback = null; + for (var lineNumber = firstLine; lineNumber <= lastLine; lineNumber++) + { + var lineIndex = lineNumber - 1; + var lineStartState = lineStartStates != null && lineIndex < lineStartStates.Count + ? lineStartStates[lineIndex] + : new CSharpLexState(); + var line = LexCSharpLine(lines[lineIndex], lineStartState).SanitizedLine.AsSpan(); + var declarationSearchStart = lineNumber == firstLine + ? Math.Max( + 0, + FindCSharpDeclarationOccurrenceStartColumn( + lines[lineIndex], + symbol, + lineStartState)) + : 0; + + if (symbol.Kind == "class") + { + // Plain records use the existing class kind. Resolve their declaration + // keyword before the class-kind lookup can fall through to a later + // same-name occurrence in a base list. Attribute names and escaped + // identifiers such as `@record` are not declaration introducers. + // plain record は既存の class kind を使うため、base list 内の同名参照へ + // fallback する前に record declaration keyword から宣言名を解決する。 + // attribute 名や `@record` のような escaped identifier は introducer ではない。 + var recordKeywordColumn = FindCSharpDeclarationKeywordToken( + line, + "record".AsSpan(), + declarationSearchStart); + if (recordKeywordColumn >= 0) + { + var recordNameSearchStart = recordKeywordColumn + "record".Length; + var recordClassSuffixColumn = FindCSharpDeclarationKeywordToken( + line, + "class".AsSpan(), + recordNameSearchStart); + if (recordClassSuffixColumn >= 0 + && line[recordNameSearchStart..recordClassSuffixColumn].Trim().IsEmpty) + { + // `record class` has an optional contextual-keyword suffix before the + // declaration identifier. In particular, `record class @class` must + // resolve the escaped identifier rather than the suffix itself. + // `record class` では宣言 identifier の前に contextual keyword suffix が + // ある。特に `record class @class` は suffix ではなく escaped identifier + // を宣言位置として解決する。 + recordNameSearchStart = recordClassSuffixColumn + "class".Length; + } + var recordNameColumn = FindCSharpIdentifierToken( + line, + name, + recordNameSearchStart); + if (recordNameColumn >= 0) + return (lineNumber, recordNameColumn); + } + } + + if (symbol.Kind is "class" or "struct" or "interface" or "record") + { + var keywordColumn = FindCSharpDeclarationKeywordToken( + line, + symbol.Kind.AsSpan(), + declarationSearchStart); + if (keywordColumn >= 0) + { + var nameColumn = FindCSharpIdentifierToken( + line, + name, + keywordColumn + symbol.Kind.Length); + if (nameColumn >= 0) + return (lineNumber, nameColumn); + } + } + + var searchStart = declarationSearchStart; + while (searchStart < line.Length) + { + var nameColumn = FindCSharpIdentifierToken(line, name, searchStart); + if (nameColumn >= 0) + { + fallback ??= (lineNumber, nameColumn); + if (symbol.Kind is "function" or "test.method" + && IsOutsideCSharpAttributeList(line, nameColumn) + && IsCSharpCallableNameOccurrence(line, nameColumn, name.Length)) + { + return (lineNumber, nameColumn); + } + searchStart = nameColumn + Math.Max(1, name.Length); + continue; + } + + break; + } + } + + return fallback; + } + + private static bool IsOutsideCSharpAttributeList(ReadOnlySpan line, int column) + { + var depth = 0; + for (var i = 0; i < column; i++) + { + if (line[i] == '[') + depth++; + else if (line[i] == ']' && depth > 0) + depth--; + } + + return depth == 0; + } + + private static int FindCSharpDeclarationKeywordToken( + ReadOnlySpan line, + ReadOnlySpan keyword, + int startIndex) + { + var searchIndex = Math.Clamp(startIndex, 0, line.Length); + while (searchIndex <= line.Length - keyword.Length) + { + var keywordIndex = FindCSharpIdentifierToken(line, keyword, searchIndex); + if (keywordIndex < 0) + return -1; + + if ((keywordIndex == 0 || line[keywordIndex - 1] != '@') + && IsOutsideCSharpAttributeList(line, keywordIndex)) + { + return keywordIndex; + } + + searchIndex = keywordIndex + Math.Max(1, keyword.Length); + } + + return -1; + } + + private static int FindCSharpIdentifierToken( + ReadOnlySpan line, + ReadOnlySpan token, + int startIndex) + { + var searchIndex = Math.Clamp(startIndex, 0, line.Length); + while (searchIndex <= line.Length - token.Length) + { + var relativeIndex = line[searchIndex..].IndexOf(token, StringComparison.Ordinal); + if (relativeIndex < 0) + return -1; + + var index = searchIndex + relativeIndex; + var tokenStart = index > 0 && line[index - 1] == '@' ? index - 1 : index; + var beforeIsIdentifier = tokenStart > 0 && IsCSharpIdentifierPart(line[tokenStart - 1]); + var afterIndex = index + token.Length; + var afterIsIdentifier = afterIndex < line.Length && IsCSharpIdentifierPart(line[afterIndex]); + if (!beforeIsIdentifier && !afterIsIdentifier) + return index; + + searchIndex = index + Math.Max(1, token.Length); + } + + return -1; + } + + private static bool IsCSharpCallableNameOccurrence( + ReadOnlySpan line, + int nameColumn, + int nameLength) + { + var cursor = nameColumn; + if (cursor < line.Length && line[cursor] == '@') + cursor++; + cursor += nameLength; + while (cursor < line.Length && char.IsWhiteSpace(line[cursor])) + cursor++; + + if (cursor < line.Length && line[cursor] == '<') + { + var depth = 0; + do + { + if (line[cursor] == '<') + depth++; + else if (line[cursor] == '>') + depth--; + cursor++; + } + while (cursor < line.Length && depth > 0); + + while (cursor < line.Length && char.IsWhiteSpace(line[cursor])) + cursor++; + } + + return cursor < line.Length && line[cursor] == '('; + } + + private static int CountCharacter(ReadOnlySpan text, char value) + { + var count = 0; + foreach (var character in text) + { + if (character == value) + count++; + } + return count; + } + + private readonly record struct CSharpLeadingDeclarationEvidence( + bool HasPartialModifier, + bool HasFileModifier, + bool HasAttribute, + bool HasDocumentation); + + private readonly record struct CSharpClosedConditionalDeclarationEvidence( + int OpeningDirectiveLineIndex, + bool HasCodeBoundary, + bool HasPartialModifier, + bool HasFileModifier, + bool HasAttribute); + + private readonly record struct CSharpConditionalBranchEvidence( + bool HasCodeBoundary = false, + bool HasPartialModifier = false, + bool HasFileModifier = false, + bool HasAttribute = false, + bool PendingAttributeIsGlobal = false, + int AttributeDepth = 0); +} diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ContainerAssignment.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ContainerAssignment.cs index fa1cf7796..ae8731f91 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ContainerAssignment.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ContainerAssignment.cs @@ -5,6 +5,7 @@ namespace CodeIndex.Indexer; public static partial class SymbolExtractor { + private const string CSharpFileLocalFamilyPrefix = "file-local:"; private readonly record struct DeclaredContainerIdentity(long FileId, string Kind, string Name); private static void PopulateDeclaredContainerQualifiedNames(List symbols) @@ -78,7 +79,11 @@ private static void PopulateDeclaredContainerQualifiedNames(List s private static void AssignContainers( List symbols, string[]? rawLines = null, - Func? getCSharpLineStartStates = null) + Func? getCSharpLineStartStates = null, + string? filePath = null, + string? projectRoot = null, + Func? preserveExistingContainerAssignment = null, + bool finalizeCSharpFileLocalFamilies = true) { if (symbols.Count == 0) return; @@ -86,6 +91,8 @@ private static void AssignContainers( if (symbols.Count == 1) { AssignTopLevelFamilyKey(symbols[0]); + if (finalizeCSharpFileLocalFamilies) + FinalizeCSharpFileLocalFamilyKeys(symbols, filePath, projectRoot); return; } @@ -94,6 +101,8 @@ private static void AssignContainers( { foreach (var symbol in symbols) AssignTopLevelFamilyKey(symbol); + if (finalizeCSharpFileLocalFamilies) + FinalizeCSharpFileLocalFamilyKeys(symbols, filePath, projectRoot); return; } @@ -114,7 +123,8 @@ private static void AssignContainers( rawLines, getCSharpLineStartStates); - if (containerPath.Count > 0) + if (containerPath.Count > 0 + && preserveExistingContainerAssignment?.Invoke(symbol) != true) { var effectiveContainer = containerPath[^1]; if (symbol.ContainerKind != null && symbol.ContainerName != null) @@ -162,17 +172,542 @@ private static void AssignContainers( symbol.ContainerName ??= effectiveContainer.Name; var qualifiedContainerName = BuildQualifiedContainerName(containerPath); symbol.ContainerQualifiedName = qualifiedContainerName; - symbol.FamilyKey = BuildInheritedFamilyKey(effectiveContainer, qualifiedContainerName); + symbol.FamilyKey = BuildInheritedFamilyKey(effectiveContainer, containerPath); } } - symbol.FamilyKey ??= BuildSelfFamilyKey(symbol, containerPath); + // Type declarations own their family identity. A nested partial type must + // not retain the inherited key of its nearest partial container, because + // sibling names and generic arities would then collapse into that parent. + // type declaration は自身の family identity を持つ。nested partial type が + // 親 partial container の key を保持すると sibling / arity を誤集約する。 + symbol.FamilyKey = BuildSelfFamilyKey(symbol, containerPath) ?? symbol.FamilyKey; + if (symbol.FamilyKey == null + && symbol.Kind is "function" or "test.method" + && ContainsFileLocalType(containerPath)) + { + symbol.FamilyKey = BuildFileLocalContainerFamilyKey(containerPath); + } if (CanContainSymbols(symbol, includeCallableContainers)) stack.Push(symbol); } + + if (finalizeCSharpFileLocalFamilies) + FinalizeCSharpFileLocalFamilyKeys(symbols, filePath, projectRoot); + } + + internal static void RefreshCSharpContainerAndFamilyScopeAfterHookMutation( + IList symbols, + string? content, + string? filePath, + string? projectRoot, + string familyScopeKey) + { + if (symbols.Count == 0) + return; + + var materialized = symbols as List ?? symbols.ToList(); + var derivedRecordComponentContainers = CaptureDerivedCSharpRecordComponentContainers(materialized); + var hookMutatedTypeContainers = materialized + .Where(symbol => symbol.DeclarationStructureMutatedByHook + && IsCSharpTypeFamilyKind(symbol.Kind)) + .Select(symbol => new HookMutatedCSharpTypeContainer( + symbol, + symbol.ContainerKind, + symbol.ContainerName, + symbol.ContainerQualifiedName)) + .ToList(); + var hookMutatedTypeSymbols = hookMutatedTypeContainers + .Select(assignment => assignment.Symbol) + .ToHashSet(); + foreach (var symbol in materialized) + { + symbol.FamilyKey = null; + symbol.IsFileLocalDeclaration = + symbol.IsExplicitFileLocalDeclaration ?? symbol.IsFileLocalDeclaration; + if (symbol.DeclarationStructureMutatedByHook + && !hookMutatedTypeSymbols.Contains(symbol)) + { + continue; + } + + // Container fields emitted by extraction are derived state. Clear them on + // unchanged records so a hook rename/arity change on an enclosing declaration + // propagates to descendants. Mutated type declarations are also cleared for + // this positional pass; their accepted public values are restored below. + // extraction が設定した container field は derived state なので、unchanged record + // では消去し、hook による enclosing declaration の rename/arity 変更を descendant + // へ伝播する。変更された type declaration もこの位置ベースの pass では消去し、 + // 受理済みの public value は後段で復元する。 + symbol.ContainerKind = null; + symbol.ContainerName = null; + symbol.ContainerQualifiedName = null; + } + + var lines = content == null ? null : SplitContentLines(content); + CSharpLexState[]? lineStartStates = null; + Func? getLineStartStates = lines == null + ? null + : () => lineStartStates ??= BuildCSharpLineStartStates(lines); + AssignContainers( + materialized, + lines, + getLineStartStates, + filePath, + projectRoot, + symbol => symbol.DeclarationStructureMutatedByHook + && !hookMutatedTypeSymbols.Contains(symbol), + finalizeCSharpFileLocalFamilies: false); + + foreach (var assignment in hookMutatedTypeContainers) + { + assignment.PositionalDeclaredQualifiedName = BuildDeclaredQualifiedName(assignment.Symbol); + assignment.RestoreAcceptedContainer(); + } + + // AssignContainers uses positional containment. A hook may intentionally move a + // declaration by editing its public container fields, so rebuild those records from + // the accepted container identity after unchanged descendants have been refreshed. + // AssignContainers は位置包含を使うが、hook は public container field の変更で宣言を + // 明示的に移動できる。unchanged descendant 更新後、accepted container identity から + // 変更 record の family を再構築する。 + foreach (var symbol in materialized.Where( + symbol => symbol.DeclarationStructureMutatedByHook + && IsCSharpTypeFamilyKind(symbol.Kind))) + { + RefreshHookMutatedCSharpFamilyKey(symbol, materialized); + } + RefreshCSharpDescendantsAfterHookContainerMoves( + hookMutatedTypeContainers, + materialized); + foreach (var symbol in materialized.Where( + symbol => symbol.DeclarationStructureMutatedByHook + && !IsCSharpTypeFamilyKind(symbol.Kind))) + { + RefreshHookMutatedCSharpFamilyKey(symbol, materialized); + } + RestoreDerivedCSharpRecordComponentContainers(derivedRecordComponentContainers); + + FinalizeCSharpFileLocalFamilyKeys(materialized, filePath, projectRoot); + ApplyFamilyScope(materialized, familyScopeKey, "csharp"); + foreach (var symbol in materialized) + symbol.DeclarationStructureMutatedByHook = false; + } + + private sealed class HookMutatedCSharpTypeContainer( + SymbolRecord symbol, + string? acceptedContainerKind, + string? acceptedContainerName, + string? acceptedContainerQualifiedName) + { + internal SymbolRecord Symbol { get; } = symbol; + internal string? PositionalDeclaredQualifiedName { get; set; } + + internal void RestoreAcceptedContainer() + { + Symbol.ContainerKind = acceptedContainerKind; + Symbol.ContainerName = acceptedContainerName; + Symbol.ContainerQualifiedName = acceptedContainerQualifiedName; + } + } + + private readonly record struct HookMutatedCSharpContainerMove( + SymbolRecord Symbol, + string PositionalDeclaredQualifiedName, + string AcceptedDeclaredQualifiedName); + + private static void RefreshCSharpDescendantsAfterHookContainerMoves( + IReadOnlyList assignments, + IReadOnlyList symbols) + { + var moves = assignments + .Select(assignment => new HookMutatedCSharpContainerMove( + assignment.Symbol, + assignment.PositionalDeclaredQualifiedName ?? assignment.Symbol.Name, + BuildDeclaredQualifiedName(assignment.Symbol))) + .Where(move => !string.Equals( + move.PositionalDeclaredQualifiedName, + move.AcceptedDeclaredQualifiedName, + StringComparison.Ordinal)) + .OrderByDescending(move => move.PositionalDeclaredQualifiedName.Length) + .ToList(); + if (moves.Count == 0) + return; + + var refreshedDescendants = new List(); + foreach (var symbol in symbols) + { + if (symbol.DeclarationStructureMutatedByHook) + continue; + + var positionalContainer = symbol.ContainerQualifiedName; + if (string.IsNullOrWhiteSpace(positionalContainer)) + continue; + + foreach (var move in moves) + { + if (!IsPositionallyWithinHookContainer(move.Symbol, symbol) + || !TryReplaceCSharpContainerPrefix( + positionalContainer, + move.PositionalDeclaredQualifiedName, + move.AcceptedDeclaredQualifiedName, + out var acceptedContainer)) + { + continue; + } + + if (string.Equals( + positionalContainer, + move.PositionalDeclaredQualifiedName, + StringComparison.Ordinal)) + { + symbol.ContainerKind = move.Symbol.Kind; + symbol.ContainerName = move.Symbol.Name; + } + symbol.ContainerQualifiedName = acceptedContainer; + symbol.FamilyKey = null; + refreshedDescendants.Add(symbol); + break; + } + } + + foreach (var symbol in refreshedDescendants.Where( + symbol => IsCSharpTypeFamilyKind(symbol.Kind))) + { + RefreshHookMutatedCSharpFamilyKey(symbol, symbols); + } + foreach (var symbol in refreshedDescendants.Where( + symbol => !IsCSharpTypeFamilyKind(symbol.Kind))) + { + RefreshHookMutatedCSharpFamilyKey(symbol, symbols); + } + } + + private static bool IsPositionallyWithinHookContainer( + SymbolRecord container, + SymbolRecord symbol) + => !ReferenceEquals(container, symbol) + && container.FileId == symbol.FileId + && container.StartLine <= symbol.StartLine + && container.EndLine >= symbol.EndLine; + + private static bool TryReplaceCSharpContainerPrefix( + string value, + string oldPrefix, + string newPrefix, + out string replaced) + { + if (string.Equals(value, oldPrefix, StringComparison.Ordinal)) + { + replaced = newPrefix; + return true; + } + + if (value.StartsWith(oldPrefix + ".", StringComparison.Ordinal)) + { + replaced = newPrefix + value[oldPrefix.Length..]; + return true; + } + + replaced = value; + return false; + } + + private readonly record struct DerivedCSharpRecordComponentContainer( + SymbolRecord Component, + SymbolRecord Container); + + private static List CaptureDerivedCSharpRecordComponentContainers( + IReadOnlyList symbols) + { + var captured = new List(); + var containersByIdentity = new Dictionary>(); + var bodylessRecordContainers = new List(); + foreach (var candidate in symbols) + { + if (candidate.BodyStartLine != null + || candidate.BodyEndLine != null + || !IsBodylessCSharpRecordDeclaration(candidate)) + { + continue; + } + + bodylessRecordContainers.Add(candidate); + var identity = new DeclaredContainerIdentity( + candidate.FileId, + candidate.Kind, + candidate.Name); + if (!containersByIdentity.TryGetValue(identity, out var candidates)) + { + candidates = []; + containersByIdentity.Add(identity, candidates); + } + candidates.Add(candidate); + } + + if (bodylessRecordContainers.Count == 0) + return captured; + + foreach (var component in symbols) + { + if (component.DeclarationStructureMutatedByHook + || component.Kind != "property" + || component.ContainerKind is not ("class" or "struct") + || string.IsNullOrWhiteSpace(component.ContainerName)) + { + continue; + } + + containersByIdentity.TryGetValue( + new DeclaredContainerIdentity( + component.FileId, + component.ContainerKind, + component.ContainerName), + out var exactCandidates); + var container = exactCandidates == null + ? null + : FindDeclaredContainerSymbol(exactCandidates, component); + container ??= FindDeclaredContainerSymbol( + bodylessRecordContainers.Where(candidate => + candidate.FileId == component.FileId + && candidate.Kind == component.ContainerKind + && component.Signature != null + && candidate.Signature?.Contains(component.Signature, StringComparison.Ordinal) == true).ToList(), + component); + if (container != null) + captured.Add(new DerivedCSharpRecordComponentContainer(component, container)); + } + + return captured; } + private static bool IsBodylessCSharpRecordDeclaration(SymbolRecord symbol) + { + if (symbol.Signature == null || symbol.Kind is not ("class" or "struct")) + return false; + + var declarationHeader = ExtractCSharpDeclarationHeader( + SanitizeCSharpDeclarationEvidence(symbol.Signature)); + var recordKeywordColumn = FindCSharpIdentifierToken( + declarationHeader.AsSpan(), + "record".AsSpan(), + 0); + if (recordKeywordColumn < 0) + return false; + + return FindCSharpIdentifierToken( + declarationHeader.AsSpan(), + symbol.Name.AsSpan().TrimStart('@'), + recordKeywordColumn + "record".Length) >= 0; + } + + private static void RestoreDerivedCSharpRecordComponentContainers( + IReadOnlyList captured) + { + foreach (var assignment in captured) + { + assignment.Component.ContainerKind = assignment.Container.Kind; + assignment.Component.ContainerName = assignment.Container.Name; + assignment.Component.ContainerQualifiedName = BuildDeclaredQualifiedName(assignment.Container); + assignment.Component.FamilyKey = assignment.Container.FamilyKey; + } + } + + private static void RefreshHookMutatedCSharpFamilyKey( + SymbolRecord symbol, + IReadOnlyList symbols) + { + if (IsCSharpTypeFamilyKind(symbol.Kind) && symbol.IsPartialDeclaration == true) + { + var builder = new StringBuilder(); + var containerIdentity = BuildHookCSharpContainerFamilyIdentity( + symbol.ContainerQualifiedName, + symbol, + symbols); + if (!string.IsNullOrWhiteSpace(containerIdentity)) + builder.Append(containerIdentity); + AppendFamilySegment(builder, symbol); + symbol.FamilyKey = symbol.IsFileLocalDeclaration + ? CSharpFileLocalFamilyPrefix + builder.ToString() + : builder.ToString(); + return; + } + + var container = FindHookCSharpContainerSymbol( + symbol.ContainerQualifiedName, + symbol, + symbols); + symbol.FamilyKey = container?.FamilyKey; + if (symbol.FamilyKey == null + && symbol.IsPartialDeclaration == true + && !string.IsNullOrWhiteSpace(symbol.ContainerQualifiedName)) + { + symbol.FamilyKey = BuildHookCSharpContainerFamilyIdentity( + symbol.ContainerQualifiedName, + symbol, + symbols); + } + } + + private static SymbolRecord? FindHookCSharpContainerSymbol( + string? containerQualifiedName, + SymbolRecord symbol, + IReadOnlyList symbols) + { + if (string.IsNullOrWhiteSpace(containerQualifiedName)) + return null; + + SymbolRecord? fallback = null; + foreach (var candidate in symbols) + { + if (!IsCSharpTypeFamilyKind(candidate.Kind) + || !string.Equals( + BuildDeclaredQualifiedName(candidate), + containerQualifiedName, + StringComparison.Ordinal)) + { + continue; + } + + fallback ??= candidate; + if (candidate.StartLine <= symbol.StartLine + && candidate.EndLine >= symbol.EndLine) + { + return candidate; + } + } + + return fallback; + } + + private static string BuildHookCSharpContainerFamilyIdentity( + string? containerQualifiedName, + SymbolRecord symbol, + IReadOnlyList symbols) + { + if (string.IsNullOrWhiteSpace(containerQualifiedName)) + return string.Empty; + + var segments = containerQualifiedName.Split('.', StringSplitOptions.RemoveEmptyEntries); + var sourcePrefix = new StringBuilder(); + var familyIdentity = new StringBuilder(); + foreach (var segment in segments) + { + if (sourcePrefix.Length > 0) + sourcePrefix.Append('.'); + sourcePrefix.Append(segment); + + var type = FindHookCSharpContainerSymbol( + sourcePrefix.ToString(), + symbol, + symbols); + if (type != null) + familyIdentity.Append('+'); + else if (familyIdentity.Length > 0) + familyIdentity.Append('.'); + familyIdentity.Append(segment); + var arity = type == null + ? null + : CSharpTypeReferenceArity.GetDefinitionArity( + type.Signature, + type.Name, + type.Kind); + if (arity > 0) + { + familyIdentity.Append('`'); + familyIdentity.Append( + arity.Value.ToString(System.Globalization.CultureInfo.InvariantCulture)); + } + } + + return familyIdentity.ToString(); + } + + private static string BuildDeclaredQualifiedName(SymbolRecord symbol) + => string.IsNullOrWhiteSpace(symbol.ContainerQualifiedName) + ? symbol.Name + : $"{symbol.ContainerQualifiedName}.{symbol.Name}"; + + private static void FinalizeCSharpFileLocalFamilyKeys( + IReadOnlyList symbols, + string? filePath, + string? projectRoot) + { + var fileLocalFamilyBodies = symbols + .Select(symbol => symbol.FamilyKey) + .Where(familyKey => familyKey?.StartsWith(CSharpFileLocalFamilyPrefix, StringComparison.Ordinal) == true) + .Select(familyKey => familyKey![CSharpFileLocalFamilyPrefix.Length..]) + .ToHashSet(StringComparer.Ordinal); + if (fileLocalFamilyBodies.Count == 0) + return; + + // C# permits the `file` modifier on only one part of a same-file partial type. + // Propagate that scope to every matching declaration and inherited member before + // persistence, then make the persisted key file-specific so all consumers, + // including hotspots, observe the same boundary without reconstructing it. + // C# では同一ファイル内の partial type の一部だけに `file` を付けられる。 + // 永続化前に同じ family の全宣言と配下 member へ scope を伝播し、さらに + // 永続 key をファイル固有にして hotspots を含む全 consumer の境界を揃える。 + var fileIdentity = BuildCSharpFileLocalIdentity(filePath, projectRoot, symbols); + foreach (var symbol in symbols) + { + if (string.IsNullOrWhiteSpace(symbol.FamilyKey)) + continue; + + var alreadyFileLocal = symbol.FamilyKey.StartsWith( + CSharpFileLocalFamilyPrefix, + StringComparison.Ordinal); + var familyBody = alreadyFileLocal + ? symbol.FamilyKey[CSharpFileLocalFamilyPrefix.Length..] + : symbol.FamilyKey; + if (!IsWithinCSharpFileLocalFamily(fileLocalFamilyBodies, familyBody)) + continue; + + symbol.FamilyKey = $"{CSharpFileLocalFamilyPrefix}{fileIdentity}\u001f{familyBody}"; + if (IsCSharpTypeFamilyKind(symbol.Kind) && symbol.IsPartialDeclaration == true) + symbol.IsFileLocalDeclaration = true; + } + } + + private static bool IsWithinCSharpFileLocalFamily( + IReadOnlySet fileLocalFamilyBodies, + string familyBody) + { + foreach (var fileLocalFamilyBody in fileLocalFamilyBodies) + { + if (string.Equals(familyBody, fileLocalFamilyBody, StringComparison.Ordinal) + || familyBody.StartsWith(fileLocalFamilyBody + "+", StringComparison.Ordinal)) + { + return true; + } + } + + return false; + } + + private static string BuildCSharpFileLocalIdentity( + string? filePath, + string? projectRoot, + IReadOnlyList symbols) + { + if (!string.IsNullOrWhiteSpace(filePath)) + { + var identity = filePath; + if (Path.IsPathRooted(identity) && !string.IsNullOrWhiteSpace(projectRoot)) + identity = Path.GetRelativePath(projectRoot, identity); + return Path.DirectorySeparatorChar == '\\' + ? identity.Replace('\\', '/') + : identity; + } + + var fileId = symbols.Count > 0 ? symbols[0].FileId : 0; + return $"file-id:{fileId.ToString(System.Globalization.CultureInfo.InvariantCulture)}"; + } + + private static bool IsCSharpTypeFamilyKind(string kind) => + kind is "class" or "struct" or "interface" or "record"; + private static void AssignTopLevelFamilyKey(SymbolRecord symbol) => symbol.FamilyKey ??= BuildSelfFamilyKey(symbol, Array.Empty()); @@ -298,47 +833,90 @@ private static List GetEffectiveContainerPath( return builder?.ToString(); } - private static string? BuildInheritedFamilyKey(SymbolRecord container, string? qualifiedContainerName) => - SupportsCrossFileFamily(container) - ? qualifiedContainerName - : null; + private static string? BuildInheritedFamilyKey( + SymbolRecord container, + IReadOnlyList containers) + { + if (!SupportsCrossFileFamily(container)) + return null; + + var familyName = BuildQualifiedFamilyName(containers); + return familyName == null || !ContainsFileLocalType(containers) + ? familyName + : CSharpFileLocalFamilyPrefix + familyName; + } private static string? BuildSelfFamilyKey(SymbolRecord symbol, IReadOnlyList containers) { if (!SupportsCrossFileFamily(symbol)) return null; - var symbolName = symbol.Name; - if (containers.Count == 0) - return symbolName; - - StringBuilder? builder = null; - for (var i = 0; i < containers.Count; i++) - { - var name = containers[i].Name; - if (string.IsNullOrWhiteSpace(name)) - continue; + var builder = new StringBuilder(); + AppendQualifiedFamilySegments(builder, containers); + AppendFamilySegment(builder, symbol); + return symbol.IsFileLocalDeclaration || ContainsFileLocalType(containers) + ? CSharpFileLocalFamilyPrefix + builder.ToString() + : builder.ToString(); + } - builder ??= new StringBuilder(name.Length + symbolName.Length + 1); - if (builder.Length > 0) - builder.Append('.'); + private static string? BuildFileLocalContainerFamilyKey(IReadOnlyList containers) + { + var familyName = BuildQualifiedFamilyName(containers); + return familyName == null ? null : CSharpFileLocalFamilyPrefix + familyName; + } - builder.Append(name); + private static bool ContainsFileLocalType(IReadOnlyList symbols) + { + foreach (var symbol in symbols) + { + if (symbol.IsFileLocalDeclaration) + return true; } - builder ??= new StringBuilder(symbolName.Length); - if (builder.Length > 0) - builder.Append('.'); + return false; + } - builder.Append(symbolName); + private static string? BuildQualifiedFamilyName(IReadOnlyList symbols) + { + var builder = new StringBuilder(); + AppendQualifiedFamilySegments(builder, symbols); + return builder.Length == 0 ? null : builder.ToString(); + } - return builder?.ToString(); + private static void AppendQualifiedFamilySegments( + StringBuilder builder, + IReadOnlyList symbols) + { + foreach (var symbol in symbols) + AppendFamilySegment(builder, symbol); + } + + private static void AppendFamilySegment(StringBuilder builder, SymbolRecord symbol) + { + if (string.IsNullOrWhiteSpace(symbol.Name)) + return; + if (IsCSharpTypeFamilyKind(symbol.Kind)) + builder.Append('+'); + else if (builder.Length > 0) + builder.Append('.'); + builder.Append(symbol.Name); + var genericArity = CSharpTypeReferenceArity.GetDefinitionArity( + symbol.Signature, + symbol.Name, + symbol.Kind); + if (genericArity > 0) + { + builder.Append('`'); + builder.Append(genericArity.Value.ToString(System.Globalization.CultureInfo.InvariantCulture)); + } } private static bool SupportsCrossFileFamily(SymbolRecord symbol) => - symbol.Kind is "class" or "interface" or "struct" - && !string.IsNullOrWhiteSpace(symbol.Signature) - && PartialModifierRegex.IsMatch(symbol.Signature); + symbol.Kind is "class" or "interface" or "struct" or "record" + && (symbol.IsPartialDeclaration == true + || (symbol.IsPartialDeclaration == null + && !string.IsNullOrWhiteSpace(symbol.Signature) + && PartialModifierRegex.IsMatch(symbol.Signature))); private static bool TryGetObjCCategoryDisplayName(string objcDeclaration, string baseName, out string displayName) { diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractCore.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractCore.cs index 39dd03b38..2eccedae6 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractCore.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractCore.cs @@ -1264,10 +1264,18 @@ private static List ExtractCore( pythonModulePrefix, prologMultilineHeads); } + if (lang == "csharp") + { + PopulateCSharpPartialDeclarationMetadata( + lines, + symbols, + getCSharpLineStartStates); + } FinalizePatternSymbols( fileId, lang, filePath, + projectRoot, lines, symbols, extractionState, diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractionPhases.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractionPhases.cs index c763bcc77..3fc7554ed 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractionPhases.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractionPhases.cs @@ -729,13 +729,19 @@ private static void FinalizePatternSymbols( long fileId, string lang, string? filePath, + string? projectRoot, string[] lines, SymbolExtractionList symbols, SymbolExtractionState extractionState, Func? getCSharpLineStartStates, List? pendingRecordPrimaryComponents) { - AssignContainers(symbols, lines, getCSharpLineStartStates); + AssignContainers( + symbols, + lines, + getCSharpLineStartStates, + filePath, + projectRoot); if (lang is "shell" or "powershell") AddScriptScopeSymbol(fileId, lines, symbols); if (lang == "csharp") diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs index a2e6902e9..fc0ed3852 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs @@ -684,16 +684,37 @@ private static int FindFirstNonWhitespaceColumn(string text) // Java 識別子の先頭: Unicode の letter / letter-number / underscore / dollar。 // 継続文字は数字・connector punctuation・結合文字も許可し、`RÉSUMÉ` のような enum member を切らない。 public static void ApplyFamilyScope(IEnumerable symbols, string scopeKey) + => ApplyFamilyScope(symbols, scopeKey, lang: null); + + public static void ApplyFamilyScope( + IEnumerable symbols, + string scopeKey, + string? lang) { + // The current C# family contract owns encoded scopes. Other languages retain their v2 raw + // family keys so incremental updates cannot mix two key formats under one ready stamp. + // encoded scope は current C# family contract でのみ使用する。他言語は v2 の raw family key を + // 維持し、増分更新で異なる形式が ready 状態に混在することを防ぐ。 + var persistedScopeKey = string.Equals(lang, "csharp", StringComparison.Ordinal) + ? EncodeFamilyScopeKey(scopeKey) + : scopeKey; foreach (var symbol in symbols) { if (string.IsNullOrWhiteSpace(symbol.FamilyKey)) continue; - symbol.FamilyKey = $"{scopeKey}|{symbol.FamilyKey}"; + symbol.FamilyKey = $"{persistedScopeKey}|{symbol.FamilyKey}"; } } + internal static string EncodeFamilyScopeKey(string scopeKey) + // `%` is escaped first so a literal `%7C` path cannot collide with an encoded pipe. + // `%` を先に escape し、literal な `%7C` path と encoded pipe の衝突を防ぐ。 + => scopeKey + .Replace("%", "%25", StringComparison.Ordinal) + .Replace("|", "%7C", StringComparison.Ordinal) + .Replace("\u001f", "%1F", StringComparison.Ordinal); + private static bool TryAddRPacmanPackageLoaderSymbols( long fileId, string line, diff --git a/src/CodeIndex/Lsp/LspServer.PositionResolution.cs b/src/CodeIndex/Lsp/LspServer.PositionResolution.cs index bf5055ce1..cfe716415 100644 --- a/src/CodeIndex/Lsp/LspServer.PositionResolution.cs +++ b/src/CodeIndex/Lsp/LspServer.PositionResolution.cs @@ -122,7 +122,9 @@ private List ResolveReferenceTargetsAtPosition(PositionTokenCo var typeFamilyKeys = resolution.Candidates .Select(candidate => - LogicalPartialSymbolGrouper.TryBuildKey(candidate.Definition, out var key) + LogicalPartialSymbolGrouper.TryBuildTypeFamilyKeyForReferenceResolution( + candidate.Definition, + out var key) ? key : null) .Where(static key => key != null) @@ -131,7 +133,9 @@ private List ResolveReferenceTargetsAtPosition(PositionTokenCo .ToList(); if (typeFamilyKeys.Count != 1 || resolution.Candidates.Any(candidate => - !LogicalPartialSymbolGrouper.TryBuildKey(candidate.Definition, out _))) + !LogicalPartialSymbolGrouper.TryBuildTypeFamilyKeyForReferenceResolution( + candidate.Definition, + out _))) { return []; } diff --git a/src/CodeIndex/Mcp/McpToolHandlers.Graph.cs b/src/CodeIndex/Mcp/McpToolHandlers.Graph.cs index 4752199f5..6f76bb042 100644 --- a/src/CodeIndex/Mcp/McpToolHandlers.Graph.cs +++ b/src/CodeIndex/Mcp/McpToolHandlers.Graph.cs @@ -919,9 +919,9 @@ private static void RestampHotspotFamilyTrust( IReadOnlyDictionary priorFingerprints, IReadOnlyDictionary currentFingerprints) { - var currentVersion = DbContext.HotspotFamilyVersion.ToString(System.Globalization.CultureInfo.InvariantCulture); foreach (var lang in FileIndexer.GetHotspotFamilyMarkerLanguages()) { + var currentVersion = DbContext.GetHotspotFamilyVersion(lang).ToString(System.Globalization.CultureInfo.InvariantCulture); if (!currentFingerprints.TryGetValue(lang, out var currentFingerprint)) continue; @@ -943,10 +943,10 @@ private static Dictionary GetHotspotFamilyTrustMatchesCurrent( IReadOnlyDictionary priorFingerprints, IReadOnlyDictionary currentFingerprints) { - var currentVersion = DbContext.HotspotFamilyVersion.ToString(System.Globalization.CultureInfo.InvariantCulture); var values = new Dictionary(StringComparer.Ordinal); foreach (var lang in FileIndexer.GetHotspotFamilyMarkerLanguages()) { + var currentVersion = DbContext.GetHotspotFamilyVersion(lang).ToString(System.Globalization.CultureInfo.InvariantCulture); currentFingerprints.TryGetValue(lang, out var currentFingerprint); priorVersions.TryGetValue(lang, out var priorVersion); priorFingerprints.TryGetValue(lang, out var priorFingerprint); diff --git a/src/CodeIndex/Mcp/McpToolHandlers.Indexing.Execution.cs b/src/CodeIndex/Mcp/McpToolHandlers.Indexing.Execution.cs index 07d299d1d..34e24bbfd 100644 --- a/src/CodeIndex/Mcp/McpToolHandlers.Indexing.Execution.cs +++ b/src/CodeIndex/Mcp/McpToolHandlers.Indexing.Execution.cs @@ -1513,9 +1513,15 @@ await EmitProgressNotificationAsync( patternConfigsAlreadyLoaded: true); symbolRegexTimeoutIssue = IndexCommandRunner.BuildRegexTimeoutIssue(record.Path, regexTimeouts); } - SymbolExtractor.ApplyFamilyScope(symbols, indexer.GetFamilyScopeKey(filePath, record.Lang)); + var familyScopeKey = indexer.GetFamilyScopeKey(filePath, record.Lang); + SymbolExtractor.ApplyFamilyScope(symbols, familyScopeKey, record.Lang); var fileContext = new FileContext(projectPath, record.Path, filePath, record.Lang); - postExtractionHooks.Value.OnSymbolsExtracted(fileContext, symbols); + postExtractionHooks.Value.ObserveCSharpStaticInterfaceSourceSymbols(fileContext, symbols); + postExtractionHooks.Value.OnSymbolsExtractedAfterSourceObservation( + fileContext, + symbols, + content, + familyScopeKey); symbolsDroppedByKindFilter += symbolKindFilter.Apply(symbols); var committedChunkCount = 0; var committedSymbolCount = 0; @@ -1753,7 +1759,15 @@ await EmitProgressNotificationAsync( { requestToken.ThrowIfCancellationRequested(); await EmitProgressNotificationAsync(progressToken, processed, files.Count, "Finalizing reference graph.").ConfigureAwait(false); - writer.RefreshMutualRecursionFlags(requestToken); + writer.RefreshMutualRecursionFlags( + requestToken, + stampReferenceIdentityContractReady: + writer.CSharpFamilyTrustAllowsReferenceIdentityReady( + startedWithNoIndexedFiles + && !scanHadErrors + && errors == 0 + ? csharpPrepassTargets.Count > 0 + : null)); } if (ftsBulkLoad != null) @@ -1908,8 +1922,6 @@ await EmitProgressNotificationAsync( writer.MarkHdlGraphContractReady(); if (csharpSourceEvidenceComplete && !preservePriorPositiveCSharpSourceNoOp) writer.SetCSharpStaticInterfaceSourceEvidence(csharpSourceEvidenceForStamp); - if (!mutualRecursionRefreshNeeded && referenceIdentityContractMatchedBeforeMutation) - writer.MarkReferenceIdentityContractReady(); csharpSymbolNameReadyAfter = true; if (hasCSharpFilesAfter) { @@ -1952,6 +1964,10 @@ await EmitProgressNotificationAsync( indexSnapshot.HotspotFamilyVersions, indexSnapshot.HotspotFamilyMarkerFingerprints, currentHotspotFamilyMarkerFingerprints); + if (writer.CSharpFamilyTrustAllowsReferenceIdentityReady(hasCSharpFilesAfter)) + writer.MarkReferenceIdentityContractReady(); + else + writer.ClearReferenceIdentityContractReady(); // A successful refresh can stamp the languages it regenerated even when the // independent fold-key contract remains stale. // 成功した refresh で再生成した言語は、独立した fold-key 契約が stale の diff --git a/src/CodeIndex/Models/QueryResults.cs b/src/CodeIndex/Models/QueryResults.cs index b557824c3..1a7ee8004 100644 --- a/src/CodeIndex/Models/QueryResults.cs +++ b/src/CodeIndex/Models/QueryResults.cs @@ -389,11 +389,37 @@ public class SymbolResult [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] public int? DefinitionSites { get; set; } [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] + public string? PartialFamilyId { get; set; } + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] + public string? RepresentativeReason { get; set; } + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] + public List? FamilyMembers { get; set; } + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)] + public bool FamilyMembersTruncated { get; set; } + [JsonIgnore] + internal bool? IsGeneratedCode { get; set; } + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] public int? SizeLines { get; set; } [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] public double? ComplexityScore { get; set; } } +public sealed class PartialFamilyMember +{ + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] + public long? SymbolId { get; set; } + public string Path { get; set; } = string.Empty; + public int Line { get; set; } + public int StartLine { get; set; } + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] + public int? StartColumn { get; set; } + public int EndLine { get; set; } + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)] + public bool Generated { get; set; } + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)] + public bool Representative { get; set; } +} + public class UnusedSymbolResult : SymbolResult { public string UnusedBucket { get; set; } = string.Empty; @@ -690,6 +716,18 @@ public sealed class LspLocation public string? Severity { get; set; } [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] public string? Source { get; set; } + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] + public long? SymbolId { get; set; } + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] + public string? PartialFamilyId { get; set; } + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] + public string? RepresentativeReason { get; set; } + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] + public List? FamilyMembers { get; set; } + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)] + public bool FamilyMembersTruncated { get; set; } + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)] + public bool Representative { get; set; } } public class ExactZeroHintResult diff --git a/src/CodeIndex/Models/SymbolRecord.cs b/src/CodeIndex/Models/SymbolRecord.cs index 40987e249..4fe0534da 100644 --- a/src/CodeIndex/Models/SymbolRecord.cs +++ b/src/CodeIndex/Models/SymbolRecord.cs @@ -70,6 +70,63 @@ public class SymbolRecord /// Return type when known / 戻り値型 public string? ReturnType { get; set; } + /// + /// Extraction-owned C# declaration facts used by logical partial-family grouping. + /// These values cross the extraction-worker boundary and are persisted, but are + /// intentionally omitted from public symbol-query JSON. + /// logical partial-family grouping が使う extraction-owned な C# 宣言情報。 + /// extraction worker 境界を越えて DB には保存するが、public な symbol query + /// JSON には公開しない。 + /// + [JsonInclude] + internal bool? IsPartialDeclaration { get; set; } + + /// + /// Extraction-owned marker for a C# top-level type declared with the file + /// modifier or belonging to the same-file partial family of such a declaration. + /// It crosses worker and hook boundaries, is persisted for callable type-kind lookup, + /// and keeps inherited partial-family keys and type facts scoped to one source file. + /// file 修飾子を持つ C# top-level type、またはその同一ファイル partial family + /// に属する type の extraction-owned marker。worker と hook の境界を越えて DB に保存し、 + /// callable type-kind lookup と配下の partial-family key を同一 source file 内に制限する。 + /// + [JsonInclude] + internal bool IsFileLocalDeclaration { get; set; } + + /// + /// Signature/source-evidence marker before same-file partial-family propagation. + /// Hook-time family rebuilding uses it to distinguish the declaration carrying the + /// file modifier from sibling parts that merely inherit file-local scope. + /// same-file partial-family 伝播前の signature/source-evidence marker。 + /// hook 後の family 再構築時に、file modifier 自体を持つ宣言と scope を継承した + /// sibling part を区別する。 + /// + [JsonInclude] + internal bool? IsExplicitFileLocalDeclaration { get; set; } + + /// + /// Internal hook-pipeline marker used to preserve explicit container mutations while + /// derived container metadata is rebuilt for unchanged descendants. + /// unchanged descendant の derived container metadata を再構築しつつ、hook が明示的に + /// 変更した container 情報を保持するための内部 marker。 + /// + [JsonInclude] + internal bool DeclarationStructureMutatedByHook { get; set; } + + [JsonInclude] + internal int? DeclarationSemanticScore { get; set; } + + /// + /// Zero-based column of the declared identifier on . For a multiline + /// declaration, extraction moves from the modifier anchor to the + /// identifier line while retains the declaration-range start. + /// 上の宣言識別子列（0始まり）。複数行宣言では extraction が + /// を modifier anchor から識別子行へ移し、 は + /// 宣言 range の開始行を保持する。 + /// + [JsonInclude] + internal int? IdentifierStartColumn { get; set; } + /// /// Authoritative metadata-target flag (e.g. C# attribute class derived from System.Attribute). /// Persisted in `symbols.is_metadata_target` after a per-language resolver pass and gated by diff --git a/tests/CodeIndex.HookIsolationFixture/HookIsolationFixture.cs b/tests/CodeIndex.HookIsolationFixture/HookIsolationFixture.cs index 6f5985a6a..31f669eb4 100644 --- a/tests/CodeIndex.HookIsolationFixture/HookIsolationFixture.cs +++ b/tests/CodeIndex.HookIsolationFixture/HookIsolationFixture.cs @@ -9,6 +9,7 @@ public static class HookIsolationFixtureEnvironment { public const string ModuleInitializerPidPath = "CDIDX_TEST_HOOK_MODULE_INITIALIZER_PID_PATH"; public const string SelectiveSlowHookAssembly = "CDIDX_TEST_SELECTIVE_SLOW_HOOK_ASSEMBLY"; + public const string MutateCSharpPartialFamily = "CDIDX_TEST_MUTATE_CSHARP_PARTIAL_FAMILY"; public const string RemoveCSharpStaticInterfaceMemberMarkerFileName = ".cdidx-test-remove-csharp-static-interface-member"; @@ -21,6 +22,53 @@ public static void Initialize() } } +public sealed class CSharpPartialFamilyMutationPostExtractionHook : IPostExtractionHook +{ + public void OnSymbolsExtracted(FileContext context, IList symbols) + { + if (Environment.GetEnvironmentVariable( + HookIsolationFixtureEnvironment.MutateCSharpPartialFamily) != "1" + || !string.Equals(context.Language, "csharp", StringComparison.Ordinal)) + { + return; + } + + var container = symbols.FirstOrDefault(symbol => symbol.Name == "HookContainer"); + if (container != null) + { + container.Name = "HookContainerRenamed"; + container.Signature = "file partial class HookContainerRenamed"; + } + + var existing = symbols.FirstOrDefault(symbol => symbol.Name == "HookPartial"); + if (existing != null) + { + existing.Name = "HookOrdinary"; + existing.Signature = "void HookOrdinary();"; + existing.ContainerName = "HookContainerRenamed"; + existing.ContainerQualifiedName = "HookContainerRenamed"; + } + + symbols.Add(new SymbolRecord + { + FileId = existing?.FileId ?? 0, + Kind = "function", + Name = "HookAddedPartial", + Signature = "[Obsolete] partial void HookAddedPartial();", + ContainerKind = "class", + ContainerName = "HookContainerRenamed", + ContainerQualifiedName = "HookContainerRenamed", + Line = 3, + StartLine = 3, + EndLine = 3, + }); + } + + public void OnReferencesExtracted(FileContext context, IList references) + { + } +} + public sealed class PathSelectivePostExtractionHook : IPostExtractionHook { public void OnSymbolsExtracted(FileContext context, IList symbols) diff --git a/tests/CodeIndex.Tests/DatabaseTests.cs b/tests/CodeIndex.Tests/DatabaseTests.cs index 88afde0df..477a2b7ea 100644 --- a/tests/CodeIndex.Tests/DatabaseTests.cs +++ b/tests/CodeIndex.Tests/DatabaseTests.cs @@ -1273,6 +1273,64 @@ FROM symbols Assert.Empty(reader.GetCallees("run_graph", exact: true)); } + [Fact] + public void ReferenceIdentityRefresh_UnstampedFreshCSharpRowsRemainReady_Issue4914Review() + { + const string path = "src/FreshPartial.cs"; + const string content = "public partial class FreshPartial { public partial void Run(); }"; + var fileId = UpsertTestFileWithLanguage(path, "csharp", "fresh-partial"); + var symbols = SymbolExtractor.Extract(fileId, "csharp", content, filePath: path); + SymbolExtractor.ApplyFamilyScope( + symbols, + FileIndexer.DeriveFallbackFamilyScopeKey(path)); + _writer.InsertSymbols(symbols); + _writer.ClearReferenceIdentityContractReady(); + + _writer.BackfillFoldedColumns(rewriteAll: true); + + Assert.Equal( + DbContext.ReferenceIdentityContractVersion.ToString(CultureInfo.InvariantCulture), + _db.GetMetaString(DbContext.ReferenceIdentityContractVersionMetaKey)); + + var standaloneRefreshWriter = new DbWriter(_db.Connection); + standaloneRefreshWriter.ClearReferenceIdentityContractReady(); + + standaloneRefreshWriter.RefreshMutualRecursionFlags(); + + Assert.True(standaloneRefreshWriter.ReferenceIdentityContractMatchesCurrent()); + } + + [Fact] + public void ReferenceIdentityRefresh_UnstampedLegacyCSharpRowsRemainStale_Issue4914Review() + { + var fileId = UpsertTestFileWithLanguage( + "src/LegacyPartial.cs", + "csharp", + "legacy-partial"); + _writer.InsertSymbols([ + new SymbolRecord + { + FileId = fileId, + Kind = "class", + Name = "LegacyPartial", + Signature = "public partial class LegacyPartial", + Line = 1, + StartLine = 1, + EndLine = 1, + }, + ]); + _writer.MarkReferenceIdentityContractReady(); + + _writer.RefreshMutualRecursionFlags(); + + Assert.Null(_db.GetMetaString(DbContext.ReferenceIdentityContractVersionMetaKey)); + + _writer.MarkReferenceIdentityContractReady(); + _writer.BackfillFoldedColumns(rewriteAll: true); + + Assert.Null(_db.GetMetaString(DbContext.ReferenceIdentityContractVersionMetaKey)); + } + [Fact] public void ReferenceGraph_NimBackfillRewritesLegacyKeysAndRefreshesCandidates_Issue4738() { @@ -6992,10 +7050,25 @@ public void InsertSymbols_ChunksLargeInputUnderSqlVariableLimit() }) .ToList(); - _writer.InsertSymbols(symbols); + var checkpoints = new List(); + var previousProgressHook = DbWriter.BatchProgressCheckpointForTesting; + try + { + DbWriter.BatchProgressCheckpointForTesting = progress => + { + if (progress.Operation == "insert_symbols") + checkpoints.Add(progress.RowsProcessed); + }; + _writer.InsertSymbols(symbols); + } + finally + { + DbWriter.BatchProgressCheckpointForTesting = previousProgressHook; + } var (_, _, symbolCount, _) = _writer.GetCounts(); Assert.Equal(120, symbolCount); + Assert.Equal([0, 39, 78, 117, 120], checkpoints); } [Fact] diff --git a/tests/CodeIndex.Tests/DbReaderTests.cs b/tests/CodeIndex.Tests/DbReaderTests.cs index be38b71c5..ccf33d13b 100644 --- a/tests/CodeIndex.Tests/DbReaderTests.cs +++ b/tests/CodeIndex.Tests/DbReaderTests.cs @@ -2944,6 +2944,73 @@ WHERE file_id IN ( Assert.Contains("partial_family_key_population=csharp", signal.DegradedReason); } + [Fact] + public void GetHotspotFamilySignal_SeparatesFileLocalAndContainingArityFamilies_Issue4914() + { + InsertIndexedFile("src/A.Host.cs", "csharp", + """ + namespace Demo; + file partial class Host { } + file partial class Host { } + partial class Outer { public partial class Child { } } + """); + InsertIndexedFile("src/B.Host.cs", "csharp", + """ + namespace Demo; + class Host { } + partial class Outer { public partial class Child { } } + """); + InsertIndexedFile("src/C.Host.cs", "csharp", + """ + namespace Demo; + class Outer { public class Child { } } + """); + InsertIndexedFile("src/D.PartialReturnType.cs", "csharp", + """ + namespace Demo; + class partial { } + class PartialReturnTypeHost + { + partial M() => new partial(); + } + """); + using (var command = _db.Connection.CreateCommand()) + { + command.CommandText = """ + UPDATE symbols + SET is_partial_declaration = NULL + WHERE name = 'M' + AND file_id IN ( + SELECT id FROM files WHERE path = 'src/D.PartialReturnType.cs' + ) + """; + Assert.Equal(1, command.ExecuteNonQuery()); + } + + var reader = new DbReader(_db.Connection); + var signal = reader.GetHotspotFamilySignal("csharp"); + + Assert.True(signal.Relevant); + Assert.True(signal.Ready, signal.DegradedReason); + Assert.DoesNotContain("partial_family_key_population=csharp", signal.DegradedReason); + } + + [Fact] + public void HotspotFamilyVersion_ChangesOnlyForCSharp_Issue4914() + { + Assert.Equal(14, DbContext.GetHotspotFamilyVersion("csharp")); + Assert.Equal(2, DbContext.GetHotspotFamilyVersion("vb")); + Assert.Equal(2, DbContext.GetHotspotFamilyVersion("fsharp")); + Assert.Equal(2, DbContext.GetHotspotFamilyVersion("msbuild")); + + foreach (var lang in FileIndexer.GetHotspotFamilyMarkerLanguages()) + { + Assert.Equal( + DbContext.GetHotspotFamilyVersion(lang).ToString(System.Globalization.CultureInfo.InvariantCulture), + _db.GetMetaString(DbContext.GetHotspotFamilyVersionMetaKey(lang))); + } + } + [Fact] public void GetHotspotFamilySignal_GroupedReadinessDetectsPartialRowsAcrossLanguages() { @@ -2964,7 +3031,15 @@ public void GetHotspotFamilySignal_GroupedReadinessDetectsPartialRowsAcrossLangu { command.CommandText = """ UPDATE symbols - SET family_key = NULL + SET family_key = NULL, + is_partial_declaration = CASE + WHEN kind = 'class' + AND file_id IN ( + SELECT id FROM files WHERE path = 'src/csharp/Api.Part2.cs' + ) + THEN NULL + ELSE is_partial_declaration + END WHERE file_id IN ( SELECT id FROM files diff --git a/tests/CodeIndex.Tests/DiffCommandRunnerTests.cs b/tests/CodeIndex.Tests/DiffCommandRunnerTests.cs index a36cbfcf6..d3d0bad66 100644 --- a/tests/CodeIndex.Tests/DiffCommandRunnerTests.cs +++ b/tests/CodeIndex.Tests/DiffCommandRunnerTests.cs @@ -1302,6 +1302,77 @@ public void Run_DetailedJsonHandlesLegacySymbolRowsWithoutMetadataTargetSource_I } } + [Fact] + public void Run_DetailedJsonIncludesPartialRepresentativeMetadata_Issue4914() + { + var leftRoot = TestProjectHelper.CreateTempProject("cdidx_diff_partial_metadata_left_issue4914"); + var rightRoot = TestProjectHelper.CreateTempProject("cdidx_diff_partial_metadata_right_issue4914"); + try + { + var leftDb = TestProjectHelper.CreateProjectDb(leftRoot); + var rightDb = TestProjectHelper.CreateProjectDb(rightRoot); + TestProjectHelper.InsertIndexedFile(leftDb, "src/Same.cs", "csharp", "public partial class Same { }"); + TestProjectHelper.InsertIndexedFile(rightDb, "src/Same.cs", "csharp", "public partial class Same { }"); + ExecuteNonQuery( + rightDb, + """ + UPDATE symbols + SET is_partial_declaration = 1, + is_file_local_declaration = 1, + declaration_semantic_score = 7, + identifier_start_column = 19 + WHERE id = (SELECT MIN(id) FROM symbols) + """); + + var (exitCode, output) = RunWithCapturedOut( + [leftDb, rightDb, "--json", "--detailed", "--data-only", "--include-content", "--limit", "10"]); + + Assert.Equal(1, exitCode); + using var document = JsonDocument.Parse(output); + var root = document.RootElement; + Assert.Equal("different", root.GetProperty("status").GetString()); + var rightSymbol = Assert.Single(GetRecords(root, "symbol", "right")); + Assert.Equal("1", GetField(rightSymbol, "is_partial_declaration").GetProperty("value").GetString()); + Assert.Equal("1", GetField(rightSymbol, "is_file_local_declaration").GetProperty("value").GetString()); + Assert.Equal("7", GetField(rightSymbol, "declaration_semantic_score").GetProperty("value").GetString()); + Assert.Equal("19", GetField(rightSymbol, "identifier_start_column").GetProperty("value").GetString()); + } + finally + { + TestProjectHelper.DeleteDirectory(leftRoot); + TestProjectHelper.DeleteDirectory(rightRoot); + } + } + + [Fact] + public void Run_DetailedJsonHandlesLegacyRowsWithoutPartialRepresentativeMetadata_Issue4914() + { + var leftRoot = TestProjectHelper.CreateTempProject("cdidx_diff_partial_legacy_left_issue4914"); + var rightRoot = TestProjectHelper.CreateTempProject("cdidx_diff_partial_legacy_right_issue4914"); + try + { + var leftDb = TestProjectHelper.CreateProjectDb(leftRoot); + var rightDb = TestProjectHelper.CreateProjectDb(rightRoot); + TestProjectHelper.InsertIndexedFile(leftDb, "src/Same.cs", "csharp", "public partial class Same { }"); + TestProjectHelper.InsertIndexedFile(rightDb, "src/Same.cs", "csharp", "public partial class Same { }"); + DropPartialRepresentativeMetadataColumns(leftDb); + DropPartialRepresentativeMetadataColumns(rightDb); + + var (exitCode, output) = RunWithCapturedOut( + [leftDb, rightDb, "--json", "--detailed", "--data-only", "--limit", "10"]); + + Assert.Equal(0, exitCode); + using var document = JsonDocument.Parse(output); + Assert.Equal("identical", document.RootElement.GetProperty("status").GetString()); + Assert.True(document.RootElement.GetProperty("identical").GetBoolean()); + } + finally + { + TestProjectHelper.DeleteDirectory(leftRoot); + TestProjectHelper.DeleteDirectory(rightRoot); + } + } + [Fact] public void Run_ReturnsSuccessForSeparatelyBuiltIdenticalDatabases_Issue1724() { @@ -1830,25 +1901,47 @@ file_id INTEGER NOT NULL REFERENCES files(id) ON DELETE CASCADE, family_key TEXT, visibility TEXT, return_type TEXT, + is_partial_declaration INTEGER, + is_file_local_declaration INTEGER, + declaration_semantic_score INTEGER, + identifier_start_column INTEGER, is_metadata_target INTEGER ); INSERT INTO symbols ( id, file_id, kind, sub_kind, name, name_folded, line, start_line, start_column, end_line, body_start_line, body_end_line, signature, container_kind, container_name, - container_qualified_name, family_key, visibility, return_type, is_metadata_target + container_qualified_name, family_key, visibility, return_type, + is_partial_declaration, is_file_local_declaration, + declaration_semantic_score, identifier_start_column, + is_metadata_target ) SELECT id, file_id, kind, sub_kind, name, name_folded, line, start_line, start_column, end_line, body_start_line, body_end_line, signature, container_kind, container_name, - container_qualified_name, family_key, visibility, return_type, is_metadata_target + container_qualified_name, family_key, visibility, return_type, + is_partial_declaration, is_file_local_declaration, + declaration_semantic_score, identifier_start_column, + is_metadata_target FROM symbols_old; DROP TABLE symbols_old; PRAGMA foreign_keys = ON; """); } + private static void DropPartialRepresentativeMetadataColumns(string dbPath) + { + ExecuteNonQuery( + dbPath, + """ + ALTER TABLE symbols DROP COLUMN is_partial_declaration; + ALTER TABLE symbols DROP COLUMN is_file_local_declaration; + ALTER TABLE symbols DROP COLUMN declaration_semantic_score; + ALTER TABLE symbols DROP COLUMN identifier_start_column; + """); + } + private static void ExecuteNonQuery(string dbPath, string sql, Action? configure = null) { using var connection = new SqliteConnection(new SqliteConnectionStringBuilder diff --git a/tests/CodeIndex.Tests/IndexCommandRunnerUpdateTests.cs b/tests/CodeIndex.Tests/IndexCommandRunnerUpdateTests.cs index 164d45109..1596f6bc9 100644 --- a/tests/CodeIndex.Tests/IndexCommandRunnerUpdateTests.cs +++ b/tests/CodeIndex.Tests/IndexCommandRunnerUpdateTests.cs @@ -286,7 +286,7 @@ public void Run_UpdateMode_NoOpRepairsVersion4MarkdownCandidates_Issue4846() var (initialExitCode, _) = RunAndCaptureJson([projectRoot, "--json"]); Assert.Equal(CommandExitCodes.Success, initialExitCode); - Assert.Equal(6, DbContext.ReferenceIdentityContractVersion); + Assert.Equal(8, DbContext.ReferenceIdentityContractVersion); var dbPath = Path.Combine(projectRoot, ".cdidx", "codeindex.db"); using (var connection = new SqliteConnection($"Data Source={dbPath}")) @@ -351,7 +351,7 @@ UPDATE codeindex_meta using var markerCommand = verification.CreateCommand(); markerCommand.CommandText = "SELECT value FROM codeindex_meta WHERE key = @key"; markerCommand.Parameters.AddWithValue("@key", DbContext.ReferenceIdentityContractVersionMetaKey); - Assert.Equal("6", Convert.ToString(markerCommand.ExecuteScalar(), CultureInfo.InvariantCulture)); + Assert.Equal("8", Convert.ToString(markerCommand.ExecuteScalar(), CultureInfo.InvariantCulture)); using var resolutionCommand = verification.CreateCommand(); resolutionCommand.CommandText = """ @@ -6056,7 +6056,7 @@ public void Run_UpdateMode_DoesNotRestampHotspotFamilyReadyWhenMarkerFingerprint } [ProductionRuntimeFact] - public void Run_Update_WhenHotspotFamilyMetadataCannotBeRestamped_ReportsDegradedReadiness() + public void Run_Update_WhenHotspotFamilyMetadataCannotBeRestamped_KeepsReferenceIdentityStale_Issue4914() { var projectRoot = CreateTempProject(); try @@ -6078,6 +6078,10 @@ public void Run_Update_WhenHotspotFamilyMetadataCannotBeRestamped_ReportsDegrade var writer = new DbWriter(db.Connection); writer.SetMeta(DbContext.GetHotspotFamilyVersionMetaKey("csharp"), null); writer.SetMeta(DbContext.GetHotspotFamilyMarkerFingerprintMetaKey("csharp"), null); + writer.SetMeta( + DbContext.ReferenceIdentityContractVersionMetaKey, + (DbContext.ReferenceIdentityContractVersion - 1).ToString( + CultureInfo.InvariantCulture)); } File.WriteAllText(callerPath, "public class Caller { public void Call(Api api) { api.Run(); api.Run(1); api.Run(); } }"); @@ -6088,6 +6092,13 @@ public void Run_Update_WhenHotspotFamilyMetadataCannotBeRestamped_ReportsDegrade Assert.False(updateJson.GetProperty("hotspot_family_ready").GetBoolean()); Assert.Contains("hotspot_family_support_not_indexed=csharp", updateJson.GetProperty("hotspot_family_degraded_reason").GetString()); + using (var verifyDb = new DbContext(DbOpenIntent.WriteIndex, dbPath)) + { + Assert.NotEqual( + DbContext.ReferenceIdentityContractVersion.ToString(CultureInfo.InvariantCulture), + verifyDb.GetMetaString(DbContext.ReferenceIdentityContractVersionMetaKey)); + } + File.WriteAllText(callerPath, "public class Caller { public void Call(Api api) { api.Run(); api.Run(1); api.Run(); api.Run(1); } }"); File.SetLastWriteTimeUtc(callerPath, DateTime.UtcNow.AddSeconds(4)); diff --git a/tests/CodeIndex.Tests/JsonEnvelopeWrapperIssue4585Tests.cs b/tests/CodeIndex.Tests/JsonEnvelopeWrapperIssue4585Tests.cs index a47c2babc..c6832cd52 100644 --- a/tests/CodeIndex.Tests/JsonEnvelopeWrapperIssue4585Tests.cs +++ b/tests/CodeIndex.Tests/JsonEnvelopeWrapperIssue4585Tests.cs @@ -825,5 +825,6 @@ private static void MarkGraphAndFoldReady(string dbPath) writer.MarkGraphReady(); writer.MarkFoldReady(); writer.MarkCSharpSymbolNameContractReady(); + writer.MarkHotspotFamilyReady("csharp", "test-fixture-csharp-family"); } } diff --git a/tests/CodeIndex.Tests/PostExtractionHookContractTests.cs b/tests/CodeIndex.Tests/PostExtractionHookContractTests.cs index 9a49489fc..e9b85d471 100644 --- a/tests/CodeIndex.Tests/PostExtractionHookContractTests.cs +++ b/tests/CodeIndex.Tests/PostExtractionHookContractTests.cs @@ -99,6 +99,10 @@ public void MutationMaterializer_ClonesAndTrimsRecordsWithinContracts_Issue4185( Kind = "class", Name = "Original", IdentityNameFolded = "original-key", + IsPartialDeclaration = true, + IsFileLocalDeclaration = true, + DeclarationSemanticScore = 7, + IdentifierStartColumn = 13, Line = 1, StartLine = 1, EndLine = 1, @@ -111,6 +115,10 @@ public void MutationMaterializer_ClonesAndTrimsRecordsWithinContracts_Issue4185( Assert.True(inputTruncated); var clonedSymbol = Assert.Single(cloned); Assert.Equal("original-key", clonedSymbol.IdentityNameFolded); + Assert.True(clonedSymbol.IsPartialDeclaration); + Assert.True(clonedSymbol.IsFileLocalDeclaration); + Assert.Equal(7, clonedSymbol.DeclarationSemanticScore); + Assert.Equal(13, clonedSymbol.IdentifierStartColumn); clonedSymbol.Name = "ChangedByHook"; Assert.Equal("Original", symbols[0].Name); diff --git a/tests/CodeIndex.Tests/PostExtractionHookTests.cs b/tests/CodeIndex.Tests/PostExtractionHookTests.cs index 59ebd2322..fe18461f4 100644 --- a/tests/CodeIndex.Tests/PostExtractionHookTests.cs +++ b/tests/CodeIndex.Tests/PostExtractionHookTests.cs @@ -3,6 +3,7 @@ using System.Runtime.Loader; using System.Text.Json; using CodeIndex.HookIsolationFixture; +using CodeIndex.Indexer; using CodeIndex.Indexer.Hooks; using CodeIndex.Models; @@ -19,6 +20,7 @@ public class PostExtractionHookTests internal const string StatefulHookEnvironmentVariable = "CDIDX_TEST_STATEFUL_POST_EXTRACTION_HOOK"; internal const string ThrowingConstructorHookEnvironmentVariable = "CDIDX_TEST_THROWING_CTOR_POST_EXTRACTION_HOOK"; internal const string ExpandingHookEnvironmentVariable = "CDIDX_TEST_EXPANDING_POST_EXTRACTION_HOOK"; + internal const string CSharpDeclarationMutationEnvironmentVariable = "CDIDX_TEST_CSHARP_DECLARATION_MUTATION_HOOK"; internal const string ModuleInitializerDelayEnvironmentVariable = "CDIDX_TEST_HOOK_MODULE_INITIALIZER_DELAY_MS"; internal const string PersistentDiscoveryWorkerPidPathEnvironmentVariable = "CDIDX_TEST_HOOK_DISCOVERY_PERSISTENT_PID_PATH"; internal const string PersistentDiscoveryDescendantPidPathEnvironmentVariable = "CDIDX_TEST_HOOK_DISCOVERY_DESCENDANT_PID_PATH"; @@ -100,6 +102,358 @@ public void Discover_LoadsHooksAndAllowsSymbolAndReferenceMutation() } } + [ProductionRuntimeFact] + public void Discover_RecomputesCSharpDeclarationMetadataAfterHookMutation_Issue4914() + { + var projectRoot = TestProjectHelper.CreateExecutableExtensionTestProject("post-extraction-hooks-csharp-metadata-4914"); + lock (TestConsoleLock.Gate) + { + using var mutation = EnvironmentVariableScope.Capture(CSharpDeclarationMutationEnvironmentVariable); + try + { + var hooksDir = Path.Combine(projectRoot, "hooks"); + Directory.CreateDirectory(hooksDir); + File.Copy(Assembly.GetExecutingAssembly().Location, Path.Combine(hooksDir, "CodeIndex.Tests.dll")); + mutation.Set(CSharpDeclarationMutationEnvironmentVariable, "1"); + + using var runner = PostExtractionHookRunner.Discover(hooksDir); + var context = new FileContext(projectRoot, "src/App.cs", Path.Combine(projectRoot, "src", "App.cs"), "csharp"); + const string content = """ + partial class HookContainer + { + [Obsolete] partial void HookPartial(); + } + """; + var symbols = new List + { + new() + { + FileId = 10, + Kind = "class", + Name = "HookContainer", + Signature = "partial class HookContainer", + IsPartialDeclaration = true, + IsExplicitFileLocalDeclaration = false, + FamilyKey = "stale-project|HookContainer", + Line = 1, + StartLine = 1, + EndLine = 4, + BodyStartLine = 2, + BodyEndLine = 4, + }, + new() + { + FileId = 10, + Kind = "function", + Name = "HookPartial", + Signature = "[Obsolete] partial void HookPartial();", + IsPartialDeclaration = true, + DeclarationSemanticScore = 2, + IdentifierStartColumn = 24, + ContainerKind = "class", + ContainerName = "HookContainer", + ContainerQualifiedName = "HookContainer", + FamilyKey = "stale-project|HookContainer", + Line = 3, + StartLine = 3, + EndLine = 3, + }, + }; + + runner.ObserveCSharpStaticInterfaceSourceSymbols(context, symbols); + runner.OnSymbolsExtractedAfterSourceObservation( + context, + symbols, + content, + "hook-project"); + + var container = Assert.Single(symbols, symbol => symbol.Name == "HookContainerRenamed"); + var expectedContainerFamily = + "hook-project|file-local:src/App.cs\u001f+HookContainerRenamed`1"; + Assert.Equal(expectedContainerFamily, container.FamilyKey); + Assert.True(container.IsFileLocalDeclaration); + var mutated = Assert.Single(symbols, symbol => symbol.Name == "HookOrdinary"); + Assert.False(mutated.IsPartialDeclaration); + Assert.False(mutated.IsFileLocalDeclaration); + Assert.Equal(0, mutated.DeclarationSemanticScore); + Assert.Null(mutated.IdentifierStartColumn); + Assert.Equal("HookContainerRenamed", mutated.ContainerQualifiedName); + Assert.Equal(expectedContainerFamily, mutated.FamilyKey); + var addedPartial = Assert.Single(symbols, symbol => symbol.Name == "HookAddedPartial"); + Assert.True(addedPartial.IsPartialDeclaration); + Assert.Equal(2, addedPartial.DeclarationSemanticScore); + Assert.Null(addedPartial.IdentifierStartColumn); + Assert.Equal("HookContainerRenamed", addedPartial.ContainerQualifiedName); + Assert.Equal(expectedContainerFamily, addedPartial.FamilyKey); + var addedFileType = Assert.Single(symbols, symbol => symbol.Name == "HookFileType"); + Assert.True(addedFileType.IsFileLocalDeclaration); + Assert.Equal( + "hook-project|file-local:src/App.cs\u001f+HookFileType", + addedFileType.FamilyKey); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + } + + [ProductionRuntimeFact] + public void Discover_PreservesSplitModifiersAndRefreshesMovedContainerDescendants_Issue4914() + { + var projectRoot = TestProjectHelper.CreateExecutableExtensionTestProject("post-extraction-hooks-csharp-moves-4914"); + lock (TestConsoleLock.Gate) + { + using var mutation = EnvironmentVariableScope.Capture(CSharpDeclarationMutationEnvironmentVariable); + try + { + var hooksDir = Path.Combine(projectRoot, "hooks"); + Directory.CreateDirectory(hooksDir); + File.Copy(Assembly.GetExecutingAssembly().Location, Path.Combine(hooksDir, "CodeIndex.Tests.dll")); + mutation.Set(CSharpDeclarationMutationEnvironmentVariable, "split-and-move"); + + using var runner = PostExtractionHookRunner.Discover(hooksDir); + var context = new FileContext(projectRoot, "src/App.cs", Path.Combine(projectRoot, "src", "App.cs"), "csharp"); + const string content = """ + partial class RenameContainer + { + partial + void M(); + } + class Old + { + class Inner + { + partial void Stay(); + } + } + class New { } + file + partial + class SplitType { } + """; + var symbols = new List + { + new() + { + FileId = 10, Kind = "class", Name = "RenameContainer", + Signature = "partial class RenameContainer", IsPartialDeclaration = true, + IsExplicitFileLocalDeclaration = false, + Line = 1, StartLine = 1, EndLine = 5, BodyStartLine = 2, BodyEndLine = 5, + }, + new() + { + FileId = 10, Kind = "function", Name = "M", Signature = "void M();", + IsPartialDeclaration = true, ContainerKind = "class", + ContainerName = "RenameContainer", ContainerQualifiedName = "RenameContainer", + Line = 4, StartLine = 3, EndLine = 4, + }, + new() + { + FileId = 10, Kind = "class", Name = "Old", Signature = "class Old", + IsPartialDeclaration = false, IsExplicitFileLocalDeclaration = false, + Line = 6, StartLine = 6, EndLine = 12, BodyStartLine = 7, BodyEndLine = 12, + }, + new() + { + FileId = 10, Kind = "class", Name = "Inner", Signature = "class Inner", + IsPartialDeclaration = false, IsExplicitFileLocalDeclaration = false, + ContainerKind = "class", ContainerName = "Old", ContainerQualifiedName = "Old", + Line = 8, StartLine = 8, EndLine = 11, BodyStartLine = 9, BodyEndLine = 11, + }, + new() + { + FileId = 10, Kind = "function", Name = "Stay", Signature = "partial void Stay();", + IsPartialDeclaration = true, ContainerKind = "class", + ContainerName = "Inner", ContainerQualifiedName = "Old.Inner", + Line = 10, StartLine = 10, EndLine = 10, + }, + new() + { + FileId = 10, Kind = "class", Name = "New", Signature = "class New { }", + IsPartialDeclaration = false, IsExplicitFileLocalDeclaration = false, + Line = 13, StartLine = 13, EndLine = 13, + }, + new() + { + FileId = 10, Kind = "class", Name = "SplitType", Signature = "class SplitType { }", + IsPartialDeclaration = true, IsFileLocalDeclaration = true, + IsExplicitFileLocalDeclaration = true, + Line = 16, StartLine = 14, EndLine = 16, + }, + }; + + runner.ObserveCSharpStaticInterfaceSourceSymbols(context, symbols); + runner.OnSymbolsExtractedAfterSourceObservation( + context, + symbols, + content, + "hook-project"); + + var renamedMethod = Assert.Single(symbols, symbol => symbol.Name == "N"); + Assert.Equal("test.method", renamedMethod.Kind); + Assert.Equal("hook-reclassified", renamedMethod.SubKind); + Assert.True(renamedMethod.IsPartialDeclaration); + Assert.Equal("hook-project|+RenameContainer", renamedMethod.FamilyKey); + + var movedContainer = Assert.Single(symbols, symbol => symbol.Name == "Inner"); + Assert.Equal("New", movedContainer.ContainerQualifiedName); + var unchangedDescendant = Assert.Single(symbols, symbol => symbol.Name == "Stay"); + Assert.Equal("Inner", unchangedDescendant.ContainerName); + Assert.Equal("New.Inner", unchangedDescendant.ContainerQualifiedName); + Assert.Equal("hook-project|+New+Inner", unchangedDescendant.FamilyKey); + + var renamedFileType = Assert.Single(symbols, symbol => symbol.Name == "SplitTypeRenamed"); + Assert.True(renamedFileType.IsPartialDeclaration); + Assert.True(renamedFileType.IsFileLocalDeclaration); + Assert.Equal( + "hook-project|file-local:src/App.cs\u001f+SplitTypeRenamed", + renamedFileType.FamilyKey); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + } + + [Fact] + public void HookMutation_RemovingFileModifierClearsInheritedFamilyScope_Issue4914() + { + var symbols = new List + { + new() + { + FileId = 10, + Kind = "class", + Name = "Part", + Signature = "partial class Part { }", + IsPartialDeclaration = true, + IsFileLocalDeclaration = false, + IsExplicitFileLocalDeclaration = false, + DeclarationStructureMutatedByHook = true, + FamilyKey = "stale|file-local:src/App.cs\u001fPart", + Line = 1, + StartLine = 1, + EndLine = 1, + }, + new() + { + FileId = 10, + Kind = "class", + Name = "Part", + Signature = "partial class Part { }", + IsPartialDeclaration = true, + IsFileLocalDeclaration = true, + IsExplicitFileLocalDeclaration = false, + FamilyKey = "stale|file-local:src/App.cs\u001fPart", + Line = 2, + StartLine = 2, + EndLine = 2, + }, + }; + + SymbolExtractor.RefreshCSharpContainerAndFamilyScopeAfterHookMutation( + symbols, + "partial class Part { }\npartial class Part { }", + "/project/src/App.cs", + "/project", + "project"); + + Assert.All(symbols, symbol => + { + Assert.False(symbol.IsFileLocalDeclaration); + Assert.Equal("project|+Part", symbol.FamilyKey); + }); + } + + [Fact] + public void HookMutation_NoOpPreservesPositionalRecordComponentContainer_Issue4914() + { + const string content = "namespace Demo;\npublic record Person(string Name);"; + var symbols = SymbolExtractor.ExtractNormalized( + 10, + "csharp", + content, + hasOversizeLine: false, + filePath: "/project/src/App.cs"); + var component = Assert.Single(symbols, symbol => symbol is { Kind: "property", Name: "Name" }); + + Assert.Equal("class", component.ContainerKind); + Assert.Equal("Person", component.ContainerName); + Assert.Equal("Demo.Person", component.ContainerQualifiedName); + + SymbolExtractor.RefreshCSharpContainerAndFamilyScopeAfterHookMutation( + symbols, + content, + "/project/src/App.cs", + "/project", + "hook-project"); + + Assert.Equal("class", component.ContainerKind); + Assert.Equal("Person", component.ContainerName); + Assert.Equal("Demo.Person", component.ContainerQualifiedName); + } + + [Fact] + public void HookMutation_NestedTypeUsesContainingGenericArity_Issue4914() + { + const string content = """ + class Outer + { + partial class Inner { } + } + class Outer + { + partial class Inner { } + } + """; + var symbols = new List + { + new() + { + Kind = "class", Name = "Outer", Signature = "class Outer", + Line = 1, StartLine = 1, EndLine = 4, BodyStartLine = 2, BodyEndLine = 4, + }, + new() + { + Kind = "class", Name = "Inner", Signature = "partial class Inner { }", + IsPartialDeclaration = true, + ContainerKind = "class", ContainerName = "Outer", ContainerQualifiedName = "Outer", + Line = 3, StartLine = 3, EndLine = 3, + }, + new() + { + Kind = "class", Name = "Outer", Signature = "class Outer", + Line = 5, StartLine = 5, EndLine = 8, BodyStartLine = 6, BodyEndLine = 8, + }, + new() + { + Kind = "class", Name = "Inner", Signature = "partial class Inner { }", + IsPartialDeclaration = true, + ContainerKind = "class", ContainerName = "Outer", ContainerQualifiedName = "Outer", + DeclarationStructureMutatedByHook = true, + Line = 7, StartLine = 7, EndLine = 7, + }, + }; + + SymbolExtractor.RefreshCSharpContainerAndFamilyScopeAfterHookMutation( + symbols, + content, + "/project/src/App.cs", + "/project", + "project"); + + var innerFamilies = symbols + .Where(symbol => symbol.Name == "Inner") + .Select(symbol => symbol.FamilyKey) + .OrderBy(family => family, StringComparer.Ordinal) + .ToList(); + Assert.Equal( + ["project|+Outer`1+Inner", "project|+Outer`2+Inner"], + innerFamilies); + } + [ProductionRuntimeFact] public void Discover_UsesWorkerWithoutParentLoadContext_Issue4600() { @@ -1126,6 +1480,78 @@ public sealed class SamplePostExtractionHook : IPostExtractionHook { public void OnSymbolsExtracted(FileContext context, IList symbols) { + var csharpMutation = Environment.GetEnvironmentVariable( + PostExtractionHookTests.CSharpDeclarationMutationEnvironmentVariable); + if (csharpMutation == "split-and-move") + { + var method = symbols.FirstOrDefault(symbol => symbol.Name == "M"); + if (method != null) + { + method.Name = "N"; + method.Kind = "test.method"; + method.SubKind = "hook-reclassified"; + method.Signature = "void N();"; + } + + var movedContainer = symbols.FirstOrDefault(symbol => symbol.Name == "Inner"); + if (movedContainer != null) + { + movedContainer.ContainerKind = "class"; + movedContainer.ContainerName = "New"; + movedContainer.ContainerQualifiedName = "New"; + } + + var fileType = symbols.FirstOrDefault(symbol => symbol.Name == "SplitType"); + if (fileType != null) + { + fileType.Name = "SplitTypeRenamed"; + fileType.Signature = "class SplitTypeRenamed { }"; + } + return; + } + + if (csharpMutation == "1") + { + var container = symbols.FirstOrDefault(symbol => symbol.Name == "HookContainer"); + if (container != null) + { + container.Name = "HookContainerRenamed"; + container.Signature = "file partial class HookContainerRenamed"; + } + var existing = symbols.FirstOrDefault(symbol => symbol.Name == "HookPartial"); + if (existing != null) + { + existing.Name = "HookOrdinary"; + existing.Signature = "void HookOrdinary();"; + existing.ContainerName = "HookContainerRenamed"; + existing.ContainerQualifiedName = "HookContainerRenamed"; + } + symbols.Add(new SymbolRecord + { + FileId = existing?.FileId ?? 0, + Kind = "function", + Name = "HookAddedPartial", + Signature = "[Obsolete] partial void HookAddedPartial();", + ContainerKind = "class", + ContainerName = "HookContainerRenamed", + ContainerQualifiedName = "HookContainerRenamed", + Line = 3, + StartLine = 3, + EndLine = 3, + }); + symbols.Add(new SymbolRecord + { + FileId = existing?.FileId ?? 0, + Kind = "class", + Name = "HookFileType", + Signature = "file partial class HookFileType { }", + Line = 4, + StartLine = 4, + EndLine = 4, + }); + return; + } + symbols.Add(new SymbolRecord { FileId = symbols.FirstOrDefault()?.FileId ?? 0, diff --git a/tests/CodeIndex.Tests/QueryCommandRunnerPartialGroupingIssue4566Tests.cs b/tests/CodeIndex.Tests/QueryCommandRunnerPartialGroupingIssue4566Tests.cs index 77537895f..e30ea1f98 100644 --- a/tests/CodeIndex.Tests/QueryCommandRunnerPartialGroupingIssue4566Tests.cs +++ b/tests/CodeIndex.Tests/QueryCommandRunnerPartialGroupingIssue4566Tests.cs @@ -1,4 +1,10 @@ using CodeIndex.Cli; +using CodeIndex.Database; +using CodeIndex.Indexer; +using CodeIndex.Indexer.Hooks; +using CodeIndex.Models; +using Microsoft.Data.Sqlite; +using System.Text.Json; namespace CodeIndex.Tests; @@ -287,12 +293,19 @@ public void Five() { } dbPath, $"src/Wide.{i:D2}.cs", "csharp", - """ - namespace Demo.Wide; - public partial class Wide - { - } - """); + i == 50 + ? """ + namespace Demo.Wide; + public partial class Wide : BaseWide + { + } + """ + : """ + namespace Demo.Wide; + public partial class Wide + { + } + """); } MarkGraphAndFoldReady(dbPath); @@ -376,12 +389,4485 @@ public partial class Wide Assert.Equal(51, impact.GetProperty("definition_count").GetInt32()); Assert.Equal(1, impact.GetProperty("logical_definition_count").GetInt32()); Assert.Equal(1, impact.GetProperty("definition_output_count").GetInt32()); - Assert.Equal(51, impact.GetProperty("definitions")[0].GetProperty("definition_sites").GetInt32()); + var impactDefinition = impact.GetProperty("definitions")[0]; + Assert.Equal(51, impactDefinition.GetProperty("definition_sites").GetInt32()); + Assert.StartsWith("partial:", impactDefinition.GetProperty("partial_family_id").GetString()); + Assert.Equal("semantic_declaration", impactDefinition.GetProperty("representative_reason").GetString()); + Assert.Equal(50, impactDefinition.GetProperty("family_members").GetArrayLength()); + Assert.True(impactDefinition.GetProperty("family_members_truncated").GetBoolean()); + Assert.Contains( + impactDefinition.GetProperty("family_members").EnumerateArray(), + member => member.GetProperty("path").GetString() == "src/Wide.50.cs" + && member.GetProperty("representative").GetBoolean()); Assert.False(impact.TryGetProperty("definitions_truncated", out _)); + + var (wideSymbolsExitCode, wideSymbolsStdout, wideSymbolsStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["Wide", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--group-partials", "--limit", "1"], + _jsonOptions)); + using var wideSymbolsDocument = ParseJsonOutput(wideSymbolsStdout); + var wideSymbol = Assert.Single(wideSymbolsDocument.RootElement.EnumerateArray().ToList()); + + Assert.Equal(CommandExitCodes.Success, wideSymbolsExitCode); + Assert.Equal(string.Empty, wideSymbolsStderr); + Assert.Equal(50, wideSymbol.GetProperty("family_members").GetArrayLength()); + Assert.True(wideSymbol.GetProperty("family_members_truncated").GetBoolean()); + Assert.Equal("src/Wide.50.cs", wideSymbol.GetProperty("path").GetString()); + Assert.Contains( + wideSymbol.GetProperty("family_members").EnumerateArray(), + member => member.GetProperty("path").GetString() == "src/Wide.50.cs" + && member.GetProperty("representative").GetBoolean()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_UsesSemanticRulesAndExposesFamilyNavigation_Issue4914() + { + Assert.Equal(14, DbContext.HotspotFamilyVersion); + var constraintOnlyType = Assert.Single( + SymbolExtractor.Extract( + 1, + "csharp", + "namespace Demo;\npublic partial class ConstraintRank where T : class { }") + .Where(symbol => symbol.Kind == "class" && symbol.Name == "ConstraintRank")); + var baseListType = Assert.Single( + SymbolExtractor.Extract( + 2, + "csharp", + "namespace Demo;\npublic partial class ConstraintRank : ConstraintBase { }") + .Where(symbol => symbol.Kind == "class" && symbol.Name == "ConstraintRank")); + Assert.Equal(1, constraintOnlyType.DeclarationSemanticScore); + Assert.Equal(4, baseListType.DeclarationSemanticScore); + + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_canonical_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Widget.Split.cs", + "csharp", + """ + namespace Demo; + + [System.Obsolete] + public partial class Widget : BaseWidget + { + } + """, + isGenerated: true); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.Widget.cs", + "csharp", + """ + namespace Demo; + + public partial class Widget + { + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.ConstraintOnly.cs", + "csharp", + """ + namespace Demo; + public partial class ConstraintRank where T : class { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.BaseList.cs", + "csharp", + """ + namespace Demo; + public partial class ConstraintRank : ConstraintBase { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Customer.g.cs", + "csharp", + """ + namespace Demo; + public partial record Customer : System.IComparable + { + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.Customer.cs", + "csharp", + """ + namespace Demo; + public partial record Customer : System.IComparable + { + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Controller.cs", + "csharp", + """ + using @global = Demo; + namespace Demo; + public class Item { } + public class item { } + public class Result { } + + public partial class Controller + { + partial void OnReady([P] int declarationValue = 0); + partial void Alias(int declarationValue); + partial void Defaults(bool flag = 1 < 2, int count = 0); + partial void Quoted(string text = ")", int count = 0); + partial void Escaped(@Item declarationValue); + partial void @event(int declarationValue); + partial void Dynamic(dynamic declarationValue); + partial /* CommentName( */ void CommentName(int declarationValue); + partial void CommentGap /* identity trivia */ (int value); + partial void GenericGap /* identity trivia */ (T value); + partial void AttrString([Marker("/*")] int declarationValue); + partial void Run(Item declarationValue); + partial void Run(item declarationValue); + partial void Rooted(Item declarationValue); + partial void Rooted(global::Item declarationValue); + partial void Shadowed(System.Int32 declarationValue); + partial void Shadowed(int declarationValue); + partial void ReferenceNullable(string? declarationValue); + partial void ValueNullable(int? declarationValue); + partial void QualifiedGeneric(N.T declarationValue); + partial void QualifiedGeneric(N.U declarationValue); + partial void VerbatimGlobal(@global::System.Int32 declarationValue); + partial void VerbatimGlobal(global::System.Int32 declarationValue); + private partial Result Result(); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Root.Item.cs", + "csharp", + "public class Item { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/QualifiedGenericTypes.cs", + "csharp", + "namespace N { public class T { } public class U { } }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.Controller.cs", + "csharp", + """ + using @global = Demo; + namespace Demo; + public partial class Controller + { + partial void OnReady(int implementationValue) + { + } + + partial void Alias(global::@System.@Int32 implementationValue) { } + partial void Defaults(bool flag, int count) { } + partial void Quoted(string text, int count) { } + partial void Escaped(Item implementationValue) { } + partial void @event(int implementationValue) { } + partial void Dynamic(object implementationValue) { } + partial void CommentName(int implementationValue) { } + partial void CommentGap(int value) { } + partial void GenericGap(T value) { } + partial void AttrString(int implementationValue) { } + partial void Run(Item implementationValue) { } + partial void Run(item implementationValue) { } + partial void Rooted(Item implementationValue) { } + partial void Rooted(global::Item implementationValue) { } + partial void Shadowed(System.Int32 implementationValue) { } + partial void Shadowed(int implementationValue) { } + partial void ReferenceNullable(string implementationValue) { } + partial void ValueNullable(global::System.Nullable implementationValue) { } + partial void QualifiedGeneric(N.T implementationValue) { } + partial void QualifiedGeneric(N.U implementationValue) { } + partial void VerbatimGlobal(@global::System.Int32 implementationValue) { } + partial void VerbatimGlobal(global::System.Int32 implementationValue) { } + private partial Result Result() => new(); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Demo.System.Int32.cs", + "csharp", + """ + namespace Demo.System; + public class Int32 { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "scripts/functions.sh", + "shell", + """ + function hello { + echo hi + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.GenericContainers.cs", + "csharp", + """ + namespace Demo; + public + partial // one-arity host modifier + class GenericHost + { + partial // declaration modifier + void ContainerMethod(); + } + + public + partial /* two-arity host modifier */ + class GenericHost + { + partial /* declaration modifier */ + void ContainerMethod(); + } + + public partial class Outer + { + public partial class Nested { } + } + + public partial class Outer + { + public partial class Nested { } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.GenericContainers.cs", + "csharp", + """ + namespace Demo; + public + partial // one-arity host modifier + class GenericHost + { + partial // implementation modifier + void ContainerMethod() { } + } + + public + partial /* two-arity host modifier */ + class GenericHost + { + partial /* implementation modifier */ + void ContainerMethod() { } + } + + public partial class Outer + { + public partial class Nested { } + } + + public partial class Outer + { + public partial class Nested { } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/B.Equal.cs", + "csharp", + """ + namespace Demo; + public partial class Equal { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Equal.cs", + "csharp", + """ + namespace Demo; + public partial class Equal { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.Profile.Split.cs", + "csharp", + """ + namespace Demo; + public partial class Profile { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Profile.Primary.cs", + "csharp", + """ + namespace Demo; + [System.Serializable] + public partial class Profile : BaseProfile { } + """); + MarkGraphAndFoldReady(dbPath); + + using (var connection = new SqliteConnection($"Data Source={dbPath}")) + { + connection.Open(); + using var rename = connection.CreateCommand(); + rename.CommandText = "UPDATE files SET path = 'src/ZZZ.Profile.Primary.cs' WHERE path = 'src/A.Profile.Primary.cs'"; + Assert.Equal(1, rename.ExecuteNonQuery()); + } + + var widget = RunGroupedSymbol(dbPath, "Widget", "class"); + Assert.Equal("src/Z.Widget.cs", widget.GetProperty("path").GetString()); + Assert.Equal("non_generated_source", widget.GetProperty("representative_reason").GetString()); + Assert.StartsWith("partial:", widget.GetProperty("partial_family_id").GetString()); + Assert.Equal(2, widget.GetProperty("definition_sites").GetInt32()); + var widgetMembers = widget.GetProperty("family_members").EnumerateArray().ToList(); + Assert.Equal(2, widgetMembers.Count); + Assert.All(widgetMembers, member => Assert.Equal(21, member.GetProperty("start_column").GetInt32())); + Assert.Contains(widgetMembers, member => member.GetProperty("path").GetString() == "src/A.Widget.Split.cs" && member.GetProperty("generated").GetBoolean()); + Assert.Single(widgetMembers, member => member.TryGetProperty("representative", out var representative) && representative.GetBoolean()); + + var constraintRank = RunGroupedSymbol(dbPath, "ConstraintRank", "class"); + Assert.Equal("src/Z.BaseList.cs", constraintRank.GetProperty("path").GetString()); + Assert.Equal("semantic_declaration", constraintRank.GetProperty("representative_reason").GetString()); + Assert.Equal( + 1, + LogicalPartialSymbolGrouper.GetSemanticScore( + "public partial class ConstraintRank where T : class { }", + "class")); + Assert.Equal( + 4, + LogicalPartialSymbolGrouper.GetSemanticScore( + "public partial class ConstraintRank : ConstraintBase { }", + "class")); + Assert.Equal( + 1, + LogicalPartialSymbolGrouper.GetSemanticScore( + "public partial record @class where T : class { }", + "record")); + + var customer = RunGroupedSymbol(dbPath, "Customer", "class"); + Assert.Equal("src/Z.Customer.cs", customer.GetProperty("path").GetString()); + Assert.Equal("non_generated_source", customer.GetProperty("representative_reason").GetString()); + Assert.Equal(22, customer.GetProperty("start_column").GetInt32()); + Assert.All( + customer.GetProperty("family_members").EnumerateArray(), + member => Assert.Equal(22, member.GetProperty("start_column").GetInt32())); + + var onReady = RunGroupedSymbol(dbPath, "OnReady", "function"); + Assert.Equal("src/Z.Controller.cs", onReady.GetProperty("path").GetString()); + Assert.Equal("implementation_body", onReady.GetProperty("representative_reason").GetString()); + Assert.Equal(2, onReady.GetProperty("definition_sites").GetInt32()); + Assert.All( + onReady.GetProperty("family_members").EnumerateArray(), + member => Assert.Equal(17, member.GetProperty("start_column").GetInt32())); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void OnReady([P] int declarationValue = 0);", + "OnReady", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void OnReady(int implementationValue) { }", + "OnReady", + "void")); + Assert.NotEqual( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void OnReady(int value);", + "OnReady", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void OnReady(string value);", + "OnReady", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial T Transform(T declarationValue);", + "Transform", + "T"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial TResult Transform(TResult implementationValue) { }", + "Transform", + "TResult")); + Assert.NotEqual( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void QualifiedGeneric(N.T declarationValue);", + "QualifiedGeneric", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void QualifiedGeneric(N.U implementationValue) { }", + "QualifiedGeneric", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Collect(params int[] declarationValues);", + "Collect", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Collect(int[] implementationValues) { }", + "Collect", + "void")); + Assert.NotEqual( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Mutate(int value);", + "Mutate", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Mutate(ref int value);", + "Mutate", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Alias(int value);", + "Alias", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Alias(global::@System.@Int32 value) { }", + "Alias", + "global::@System.@Void")); + Assert.NotEqual( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Alias(int value);", + "Alias", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Alias(System.Int32 value) { }", + "Alias", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void ReferenceNullable(string? value);", + "ReferenceNullable", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void ReferenceNullable(string value) { }", + "ReferenceNullable", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void ValueNullable(int? value);", + "ValueNullable", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void ValueNullable(global::System.Nullable value) { }", + "ValueNullable", + "void")); + Assert.NotEqual( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void ValueNullable(int value);", + "ValueNullable", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void ValueNullable(int? value) { }", + "ValueNullable", + "void")); + Assert.NotEqual( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Global(global::System.Uri declarationValue);", + "Global", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Global(System.Uri implementationValue) { }", + "Global", + "void")); + Assert.NotEqual( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void VerbatimGlobal(@global::System.Int32 declarationValue);", + "VerbatimGlobal", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void VerbatimGlobal(global::System.Int32 implementationValue) { }", + "VerbatimGlobal", + "void")); + Assert.NotEqual( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Rooted(Item declarationValue);", + "Rooted", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Rooted(global::Item implementationValue) { }", + "Rooted", + "void")); + var (shadowedExitCode, shadowedStdout, shadowedStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["Shadowed", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var shadowedDocument = ParseJsonOutput(shadowedStdout); + var shadowedFamilies = shadowedDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, shadowedExitCode); + Assert.Equal(string.Empty, shadowedStderr); + Assert.Equal(2, shadowedFamilies.Count); + Assert.All(shadowedFamilies, family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + Assert.Equal( + 2, + shadowedFamilies + .Select(family => family.GetProperty("partial_family_id").GetString()) + .Distinct(StringComparer.Ordinal) + .Count()); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Defaults(bool flag = 1 < 2, int count = 0);", + "Defaults", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Defaults(bool flag, int count) { }", + "Defaults", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Decorated([P(1 < 2)] int declarationValue, string secondValue);", + "Decorated", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Decorated(int implementationValue, string secondValue) { }", + "Decorated", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + """partial void Quoted(string text = ")", int count = 0);""", + "Quoted", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Quoted(string text, int count) { }", + "Quoted", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Commented(int value = (/* ) */ 0), string text = \"x\");", + "Commented", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Commented(int value, string text) { }", + "Commented", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Escaped(@Item declarationValue);", + "Escaped", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Escaped(Item implementationValue) { }", + "Escaped", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void @event(int declarationValue);", + "event", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void @event(int implementationValue) { }", + "event", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Dynamic(dynamic declarationValue);", + "Dynamic", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Dynamic(object implementationValue) { }", + "Dynamic", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void CommentTrivia(int /* declaration */ value);", + "CommentTrivia", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void CommentTrivia(int implementationValue /* implementation */) { }", + "CommentTrivia", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial /* CommentName( */ void CommentName(int declarationValue);", + "CommentName", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void CommentName(int implementationValue) { }", + "CommentName", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void CommentGap /* identity trivia */ (int value);", + "CommentGap", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void CommentGap(int value) { }", + "CommentGap", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void GenericGap /* identity trivia */ (T value);", + "GenericGap", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void GenericGap(T value) { }", + "GenericGap", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void AttrString([Marker(\"/*\")] int declarationValue);", + "AttrString", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void AttrString(int implementationValue) { }", + "AttrString", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void RawDefault(string declarationValue = \"\"\"\"a,b\"\"\"\", int count = 0);", + "RawDefault", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void RawDefault(string implementationValue, int count) { }", + "RawDefault", + "void")); + Assert.NotEqual( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Run(Item value);", + "Run", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Run(item value) { }", + "Run", + "void")); + Assert.NotNull( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "private partial Result Result();", + "Result", + "Result")); + + var alias = RunGroupedSymbol(dbPath, "Alias", "function"); + Assert.Equal(2, alias.GetProperty("definition_sites").GetInt32()); + + var defaults = RunGroupedSymbol(dbPath, "Defaults", "function"); + Assert.Equal(2, defaults.GetProperty("definition_sites").GetInt32()); + + var quoted = RunGroupedSymbol(dbPath, "Quoted", "function"); + Assert.Equal(2, quoted.GetProperty("definition_sites").GetInt32()); + + var escaped = RunGroupedSymbol(dbPath, "Escaped", "function"); + Assert.Equal(2, escaped.GetProperty("definition_sites").GetInt32()); + + var verbatim = RunGroupedSymbol(dbPath, "event", "function"); + Assert.Equal(2, verbatim.GetProperty("definition_sites").GetInt32()); + + var (verbatimGotoExitCode, verbatimGotoStdout, verbatimGotoStderr) = CaptureConsole(() => QueryCommandRunner.RunGoto( + ["event", "--db", dbPath, "--exact-name", "--lang", "csharp", "--kind", "function", "--include-generated"], + _jsonOptions)); + using var verbatimGotoDocument = ParseJsonOutput(verbatimGotoStdout); + var verbatimGoto = verbatimGotoDocument.RootElement; + + Assert.Equal(CommandExitCodes.Success, verbatimGotoExitCode); + Assert.Equal(string.Empty, verbatimGotoStderr); + Assert.Equal(18, verbatimGoto.GetProperty("range").GetProperty("start").GetProperty("character").GetInt32()); + Assert.Equal(23, verbatimGoto.GetProperty("range").GetProperty("end").GetProperty("character").GetInt32()); + Assert.All( + verbatimGoto.GetProperty("family_members").EnumerateArray(), + member => + { + Assert.Equal(18, member.GetProperty("range").GetProperty("start").GetProperty("character").GetInt32()); + Assert.Equal(23, member.GetProperty("range").GetProperty("end").GetProperty("character").GetInt32()); + }); + + var dynamicAlias = RunGroupedSymbol(dbPath, "Dynamic", "function"); + Assert.Equal(2, dynamicAlias.GetProperty("definition_sites").GetInt32()); + + var commentName = RunGroupedSymbol(dbPath, "CommentName", "function"); + Assert.Equal(2, commentName.GetProperty("definition_sites").GetInt32()); + + var commentGap = RunGroupedSymbol(dbPath, "CommentGap", "function"); + Assert.Equal(2, commentGap.GetProperty("definition_sites").GetInt32()); + + var genericGap = RunGroupedSymbol(dbPath, "GenericGap", "function"); + Assert.Equal(2, genericGap.GetProperty("definition_sites").GetInt32()); + + var attrString = RunGroupedSymbol(dbPath, "AttrString", "function"); + Assert.Equal(2, attrString.GetProperty("definition_sites").GetInt32()); + + var referenceNullable = RunGroupedSymbol(dbPath, "ReferenceNullable", "function"); + Assert.Equal(2, referenceNullable.GetProperty("definition_sites").GetInt32()); + + var valueNullable = RunGroupedSymbol(dbPath, "ValueNullable", "function"); + Assert.Equal(2, valueNullable.GetProperty("definition_sites").GetInt32()); + + var (qualifiedGenericExitCode, qualifiedGenericStdout, qualifiedGenericStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["QualifiedGeneric", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var qualifiedGenericDocument = ParseJsonOutput(qualifiedGenericStdout); + var qualifiedGenericFamilies = qualifiedGenericDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, qualifiedGenericExitCode); + Assert.Equal(string.Empty, qualifiedGenericStderr); + Assert.Equal(2, qualifiedGenericFamilies.Count); + Assert.All(qualifiedGenericFamilies, family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + Assert.Contains(qualifiedGenericFamilies, family => family.GetProperty("signature").GetString()!.Contains("N.T", StringComparison.Ordinal)); + Assert.Contains(qualifiedGenericFamilies, family => family.GetProperty("signature").GetString()!.Contains("N.U", StringComparison.Ordinal)); + + var (rootedExitCode, rootedStdout, rootedStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["Rooted", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var rootedDocument = ParseJsonOutput(rootedStdout); + var rootedFamilies = rootedDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, rootedExitCode); + Assert.Equal(string.Empty, rootedStderr); + Assert.Equal(2, rootedFamilies.Count); + Assert.All(rootedFamilies, family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + + var (verbatimGlobalExitCode, verbatimGlobalStdout, verbatimGlobalStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["VerbatimGlobal", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var verbatimGlobalDocument = ParseJsonOutput(verbatimGlobalStdout); + var verbatimGlobalFamilies = verbatimGlobalDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, verbatimGlobalExitCode); + Assert.Equal(string.Empty, verbatimGlobalStderr); + Assert.Equal(2, verbatimGlobalFamilies.Count); + Assert.All(verbatimGlobalFamilies, family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + Assert.Equal( + 2, + verbatimGlobalFamilies + .Select(family => family.GetProperty("partial_family_id").GetString()) + .Distinct(StringComparer.Ordinal) + .Count()); + + var (shellExitCode, shellStdout, shellStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["hello", "--db", dbPath, "--json=array", "--exact-name", "--lang", "shell", "--kind", "function", "--limit", "1"], + _jsonOptions)); + using var shellDocument = ParseJsonOutput(shellStdout); + var shellFunction = Assert.Single(shellDocument.RootElement.EnumerateArray().ToList()); + + Assert.Equal(CommandExitCodes.Success, shellExitCode); + Assert.Equal(string.Empty, shellStderr); + Assert.Equal(9, shellFunction.GetProperty("start_column").GetInt32()); + + var (containerExitCode, containerStdout, containerStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["ContainerMethod", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var containerDocument = ParseJsonOutput(containerStdout); + var containerFamilies = containerDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, containerExitCode); + Assert.Equal(string.Empty, containerStderr); + Assert.Equal(2, containerFamilies.Count); + Assert.All(containerFamilies, family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + + var (genericTypeExitCode, genericTypeStdout, genericTypeStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["GenericHost", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var genericTypeDocument = ParseJsonOutput(genericTypeStdout); + var genericTypeFamilies = genericTypeDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, genericTypeExitCode); + Assert.Equal(string.Empty, genericTypeStderr); + Assert.Equal(2, genericTypeFamilies.Count); + Assert.All(genericTypeFamilies, family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + Assert.Equal( + 2, + genericTypeFamilies + .Select(family => family.GetProperty("partial_family_id").GetString()) + .Distinct(StringComparer.Ordinal) + .Count()); + + var (nestedTypeExitCode, nestedTypeStdout, nestedTypeStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["Nested", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var nestedTypeDocument = ParseJsonOutput(nestedTypeStdout); + var nestedTypeFamilies = nestedTypeDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, nestedTypeExitCode); + Assert.Equal(string.Empty, nestedTypeStderr); + Assert.Equal(2, nestedTypeFamilies.Count); + Assert.All(nestedTypeFamilies, family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + + var resultFactory = RunGroupedSymbol(dbPath, "Result", "function"); + Assert.Equal(2, resultFactory.GetProperty("definition_sites").GetInt32()); + var resultFactoryMembers = resultFactory.GetProperty("family_members").EnumerateArray().ToList(); + Assert.Contains( + resultFactoryMembers, + member => member.GetProperty("path").GetString() == "src/A.Controller.cs" + && member.GetProperty("start_column").GetInt32() == 32); + Assert.Contains( + resultFactoryMembers, + member => member.GetProperty("path").GetString() == "src/Z.Controller.cs" + && member.GetProperty("start_column").GetInt32() == 51); + + var (caseExitCode, caseStdout, caseStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["Run", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var caseDocument = ParseJsonOutput(caseStdout); + var caseFamilies = caseDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, caseExitCode); + Assert.Equal(string.Empty, caseStderr); + Assert.Equal(2, caseFamilies.Count); + Assert.All(caseFamilies, family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + Assert.Contains(caseFamilies, family => family.GetProperty("signature").GetString()!.Contains("Item implementationValue", StringComparison.Ordinal)); + Assert.Contains(caseFamilies, family => family.GetProperty("signature").GetString()!.Contains("item implementationValue", StringComparison.Ordinal)); + + var equal = RunGroupedSymbol(dbPath, "Equal", "class"); + Assert.Equal("src/A.Equal.cs", equal.GetProperty("path").GetString()); + Assert.Equal("stable_path_and_position", equal.GetProperty("representative_reason").GetString()); + + var rebuildRoot = TestProjectHelper.CreateDirectory(projectRoot, "rebuild-order"); + var rebuildDbPath = TestProjectHelper.CreateProjectDb(rebuildRoot); + TestProjectHelper.InsertIndexedFile( + rebuildDbPath, + "src/A.Equal.cs", + "csharp", + """ + namespace Demo; + public partial class Equal { } + """); + TestProjectHelper.InsertIndexedFile( + rebuildDbPath, + "src/B.Equal.cs", + "csharp", + """ + namespace Demo; + public partial class Equal { } + """); + MarkGraphAndFoldReady(rebuildDbPath); + var rebuiltEqual = RunGroupedSymbol(rebuildDbPath, "Equal", "class"); + Assert.Equal("src/A.Equal.cs", rebuiltEqual.GetProperty("path").GetString()); + Assert.Equal(equal.GetProperty("partial_family_id").GetString(), rebuiltEqual.GetProperty("partial_family_id").GetString()); + + var profile = RunGroupedSymbol(dbPath, "Profile", "class"); + Assert.Equal("src/ZZZ.Profile.Primary.cs", profile.GetProperty("path").GetString()); + Assert.Equal("semantic_declaration", profile.GetProperty("representative_reason").GetString()); + + var (definitionExitCode, definitionStdout, definitionStderr) = CaptureConsole(() => QueryCommandRunner.RunDefinition( + ["Widget", "--db", dbPath, "--json", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--include-generated", "--limit", "1"], + _jsonOptions)); + using var definitionDocument = ParseJsonOutput(definitionStdout); + + Assert.Equal(CommandExitCodes.Success, definitionExitCode); + Assert.Equal(string.Empty, definitionStderr); + Assert.Equal(widget.GetProperty("partial_family_id").GetString(), definitionDocument.RootElement.GetProperty("partial_family_id").GetString()); + Assert.Equal(2, definitionDocument.RootElement.GetProperty("family_members").GetArrayLength()); + + var (projectedExitCode, projectedStdout, projectedStderr) = CaptureConsole(() => ProgramRunner.Run( + ["definition", "Widget", "--db", dbPath, "--json", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--include-generated", "--fields", "family_members"], + _jsonOptions, + "1.0.0-test")); + using var projectedDocument = ParseJsonOutput(projectedStdout); + var projectedDefinition = Assert.Single(projectedDocument.RootElement.GetProperty("results").EnumerateArray()); + + Assert.Equal(CommandExitCodes.Success, projectedExitCode); + Assert.Equal(string.Empty, projectedStderr); + Assert.Single(projectedDefinition.EnumerateObject()); + Assert.Equal(2, projectedDefinition.GetProperty("family_members").GetArrayLength()); + + var (nestedFieldExitCode, nestedFieldStdout, nestedFieldStderr) = CaptureConsole(() => ProgramRunner.Run( + ["definition", "Widget", "--db", dbPath, "--json", "--fields", "family_members.path"], + _jsonOptions, + "1.0.0-test")); + using var nestedFieldDocument = ParseJsonOutput(nestedFieldStdout); + + Assert.Equal(CommandExitCodes.UsageError, nestedFieldExitCode); + Assert.Equal(string.Empty, nestedFieldStderr); + Assert.Contains("Unknown --fields value 'family_members.path'", nestedFieldDocument.RootElement.GetProperty("message").GetString(), StringComparison.Ordinal); + + var (gotoExitCode, gotoStdout, gotoStderr) = CaptureConsole(() => QueryCommandRunner.RunGoto( + ["Widget", "--db", dbPath, "--exact-name", "--lang", "csharp", "--kind", "class", "--include-generated"], + _jsonOptions)); + using var gotoDocument = ParseJsonOutput(gotoStdout); + var gotoLocation = gotoDocument.RootElement; + + Assert.Equal(CommandExitCodes.Success, gotoExitCode); + Assert.Equal(string.Empty, gotoStderr); + Assert.EndsWith("/src/Z.Widget.cs", new Uri(gotoLocation.GetProperty("uri").GetString()!).AbsolutePath, StringComparison.Ordinal); + Assert.Equal("non_generated_source", gotoLocation.GetProperty("representative_reason").GetString()); + Assert.Equal(2, gotoLocation.GetProperty("family_members").GetArrayLength()); + Assert.All( + gotoLocation.GetProperty("family_members").EnumerateArray(), + member => Assert.Equal(21, member.GetProperty("range").GetProperty("start").GetProperty("character").GetInt32())); + + var (gotoAllExitCode, gotoAllStdout, gotoAllStderr) = CaptureConsole(() => QueryCommandRunner.RunGoto( + ["Widget", "--db", dbPath, "--exact-name", "--lang", "csharp", "--kind", "class", "--include-generated", "--all"], + _jsonOptions)); + using var gotoAllDocument = ParseJsonOutput(gotoAllStdout); + var allLocations = gotoAllDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, gotoAllExitCode); + Assert.Equal(string.Empty, gotoAllStderr); + Assert.Equal(2, allLocations.Count); + Assert.Contains(allLocations, location => new Uri(location.GetProperty("uri").GetString()!).AbsolutePath.EndsWith("/src/A.Widget.Split.cs", StringComparison.Ordinal)); + Assert.Contains(allLocations, location => new Uri(location.GetProperty("uri").GetString()!).AbsolutePath.EndsWith("/src/Z.Widget.cs", StringComparison.Ordinal)); } finally { TestProjectHelper.DeleteDirectory(projectRoot); } } + + [Fact] + public void PartialCanonicalRepresentative_PersistsSplitModifierAndLeadingSemanticEvidence_Issue4914() + { + const string source = + """ + namespace Demo; + public partial class Container + { + /// Primary declaration. + [System.Obsolete] + partial // declaration modifier + void OnReady( + int value) + { + } + } + file class WorkerLocal { } + """; + var symbols = SymbolExtractor.Extract( + 1, + "csharp", + source); + + var method = Assert.Single(symbols.Where(symbol => symbol.Kind == "function" && symbol.Name == "OnReady")); + Assert.True(method.IsPartialDeclaration); + Assert.Equal(3, method.DeclarationSemanticScore); + Assert.Equal(9, method.IdentifierStartColumn); + Assert.DoesNotContain("partial", method.Signature, StringComparison.Ordinal); + var fileLocalType = Assert.Single( + symbols.Where(symbol => symbol.Kind == "class" && symbol.Name == "WorkerLocal")); + Assert.True(fileLocalType.IsFileLocalDeclaration); + + var persistedFamily = new SymbolResult + { + Lang = "csharp", + Kind = "function", + Name = "OnReady", + Signature = method.Signature, + LogicalPartialKey = "family:csharp\u001ffunction\u001fDemo.Container\u001fOnReady/0(System.Int32):System.Void", + }; + Assert.True(LogicalPartialSymbolGrouper.TryBuildKey(persistedFamily, out var persistedFamilyKey)); + Assert.Equal(persistedFamily.LogicalPartialKey, persistedFamilyKey); + + var persistedPhysical = new SymbolResult + { + Lang = "csharp", + Kind = "function", + Name = "OnReady", + Signature = "partial void OnReady(int value);", + ReturnType = "void", + ContainerName = "Container", + LogicalPartialKey = "symbol:42", + }; + Assert.False(LogicalPartialSymbolGrouper.TryBuildKey(persistedPhysical, out _)); + + var persistedPhysicalType = new SymbolResult + { + Lang = "csharp", + Kind = "class", + Name = "Container", + Signature = "public partial class Container", + ContainerName = "Demo", + LogicalPartialKey = "symbol:43", + }; + Assert.False(LogicalPartialSymbolGrouper.TryBuildKey(persistedPhysicalType, out _)); + Assert.True( + LogicalPartialSymbolGrouper.TryBuildTypeFamilyKeyForReferenceResolution( + persistedPhysicalType, + out var degradedTypeFamilyKey)); + Assert.Contains("Container", degradedTypeFamilyKey, StringComparison.Ordinal); + + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_worker_metadata_issue4914"); + try + { + var request = new SymbolExtractionWorker.WorkerRequest( + 1, + "csharp", + source, + Path.Combine(projectRoot, "Container.cs"), + projectRoot); + using var input = new StringReader( + JsonSerializer.Serialize(request, SymbolExtractionWorker.JsonOptions) + "\n"); + using var output = new StringWriter(); + using var error = new StringWriter(); + + var handled = SymbolExtractionWorker.TryRunCommand( + [SymbolExtractionWorker.CommandName], + input, + output, + error, + out var exitCode); + var response = JsonSerializer.Deserialize( + output.ToString(), + SymbolExtractionWorker.JsonOptions); + var transportedMethod = Assert.Single( + response!.Symbols!.Where(symbol => symbol.Kind == "function" && symbol.Name == "OnReady")); + var transportedFileLocalType = Assert.Single( + response.Symbols!.Where(symbol => symbol.Kind == "class" && symbol.Name == "WorkerLocal")); + + Assert.True(handled); + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, error.ToString()); + Assert.True(transportedMethod.IsPartialDeclaration); + Assert.Equal(3, transportedMethod.DeclarationSemanticScore); + Assert.Equal(9, transportedMethod.IdentifierStartColumn); + Assert.True(transportedFileLocalType.IsFileLocalDeclaration); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_IgnoresSameLineBodiesAndCommentedOrdinaryMethods_Issue4914() + { + const string extractionSource = + """ + namespace Demo; + public partial class FileBodyHost { private int file; } + public partial class BracketBodyHost { private int[] values = []; } + public partial class PrimaryConstructorHost(int file) { } + public class ParameterNamedPartial(int partial) { } + public class partial { } + public class PartialReturnTypeHost + { + partial M() => new partial(); + } + """; + var extractedTypes = SymbolExtractor.Extract(1, "csharp", extractionSource); + var fileBodyHost = Assert.Single( + extractedTypes.Where(symbol => symbol.Kind == "class" && symbol.Name == "FileBodyHost")); + var bracketBodyHost = Assert.Single( + extractedTypes.Where(symbol => symbol.Kind == "class" && symbol.Name == "BracketBodyHost")); + var primaryConstructorHost = Assert.Single( + extractedTypes.Where(symbol => symbol.Kind == "class" && symbol.Name == "PrimaryConstructorHost")); + var parameterNamedPartial = Assert.Single( + extractedTypes.Where(symbol => symbol.Kind == "class" && symbol.Name == "ParameterNamedPartial")); + var partialReturnTypeMethod = Assert.Single( + extractedTypes.Where(symbol => symbol.Kind == "function" && symbol.Name == "M")); + + Assert.True(fileBodyHost.IsPartialDeclaration); + Assert.False(fileBodyHost.IsFileLocalDeclaration); + Assert.Equal(0, fileBodyHost.DeclarationSemanticScore); + Assert.True(bracketBodyHost.IsPartialDeclaration); + Assert.False(bracketBodyHost.IsFileLocalDeclaration); + Assert.Equal(0, bracketBodyHost.DeclarationSemanticScore); + Assert.True(primaryConstructorHost.IsPartialDeclaration); + Assert.False(primaryConstructorHost.IsFileLocalDeclaration); + Assert.False(parameterNamedPartial.IsPartialDeclaration); + Assert.False(partialReturnTypeMethod.IsPartialDeclaration); + Assert.False(LogicalPartialSymbolGrouper.ContainsPartialModifier( + partialReturnTypeMethod.Signature, + partialReturnTypeMethod.Kind, + partialReturnTypeMethod.Name)); + Assert.True(LogicalPartialSymbolGrouper.ContainsPartialModifier( + "partial (int, int) Pair();", + "function", + "Pair")); + + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_header_evidence_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Host.cs", + "csharp", + """ + namespace Demo; + public partial class Host { private int file; } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.Host.cs", + "csharp", + """ + namespace Demo; + public partial class Host { private int[] values = []; } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/ConditionalMethods.cs", + "csharp", + """ + namespace Demo; + public partial class MethodHost + { + #if A + void M(/* partial */ int value) { } + void N(int partial) { } + #else + void M(/* partial */ int value) { } + void N(int partial) { } + #endif + } + #if A + public class ConstructorHost(int partial) { } + #else + public class ConstructorHost(int partial) { } + #endif + """); + MarkGraphAndFoldReady(dbPath); + + var groupedHost = RunGroupedSymbol(dbPath, "Host", "class"); + Assert.Equal(2, groupedHost.GetProperty("definition_sites").GetInt32()); + Assert.Equal("src/A.Host.cs", groupedHost.GetProperty("path").GetString()); + Assert.Equal("stable_path_and_position", groupedHost.GetProperty("representative_reason").GetString()); + + var (methodsExitCode, methodsStdout, methodsStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["M", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var methodsDocument = ParseJsonOutput(methodsStdout); + var methods = methodsDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, methodsExitCode); + Assert.Equal(string.Empty, methodsStderr); + Assert.Equal(2, methods.Count); + Assert.All(methods, method => Assert.False(method.TryGetProperty("definition_sites", out _))); + Assert.All(methods, method => Assert.False(method.TryGetProperty("partial_family_id", out _))); + + var (parameterMethodsExitCode, parameterMethodsStdout, parameterMethodsStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["N", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var parameterMethodsDocument = ParseJsonOutput(parameterMethodsStdout); + var parameterMethods = parameterMethodsDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, parameterMethodsExitCode); + Assert.Equal(string.Empty, parameterMethodsStderr); + Assert.Equal(2, parameterMethods.Count); + Assert.All(parameterMethods, method => Assert.False(method.TryGetProperty("definition_sites", out _))); + Assert.All(parameterMethods, method => Assert.False(method.TryGetProperty("partial_family_id", out _))); + + var (constructorsExitCode, constructorsStdout, constructorsStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["ConstructorHost", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var constructorsDocument = ParseJsonOutput(constructorsStdout); + var constructors = constructorsDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, constructorsExitCode); + Assert.Equal(string.Empty, constructorsStderr); + Assert.Equal(2, constructors.Count); + Assert.All(constructors, constructor => Assert.False(constructor.TryGetProperty("definition_sites", out _))); + Assert.All(constructors, constructor => Assert.False(constructor.TryGetProperty("partial_family_id", out _))); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_GroupsTestClassifiedImplementation_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_test_method_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.AttributedPartial.cs", + "csharp", + """ + namespace Demo; + public partial class AttributedPartial + { + public partial void Execute(); + public partial void InlineExecute(); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.AttributedPartial.cs", + "csharp", + """ + namespace Demo; + public partial class AttributedPartial + { + [Fact] + public partial void Execute() { } + [Fact] public partial void InlineExecute() { } + } + """); + MarkGraphAndFoldReady(dbPath); + + var (symbolsExitCode, symbolsStdout, symbolsStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["Execute", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var symbolsDocument = ParseJsonOutput(symbolsStdout); + var grouped = Assert.Single(symbolsDocument.RootElement.EnumerateArray().ToList()); + + Assert.Equal(CommandExitCodes.Success, symbolsExitCode); + Assert.Equal(string.Empty, symbolsStderr); + Assert.Equal("test.method", grouped.GetProperty("kind").GetString()); + Assert.Equal("src/Z.AttributedPartial.cs", grouped.GetProperty("path").GetString()); + Assert.Equal(2, grouped.GetProperty("definition_sites").GetInt32()); + Assert.Equal("implementation_body", grouped.GetProperty("representative_reason").GetString()); + + var (gotoExitCode, gotoStdout, gotoStderr) = CaptureConsole(() => QueryCommandRunner.RunGoto( + ["Execute", "--db", dbPath, "--exact-name", "--lang", "csharp"], + _jsonOptions)); + using var gotoDocument = ParseJsonOutput(gotoStdout); + var location = gotoDocument.RootElement; + + Assert.Equal(CommandExitCodes.Success, gotoExitCode); + Assert.Equal(string.Empty, gotoStderr); + Assert.Contains("Z.AttributedPartial.cs", location.GetProperty("uri").GetString(), StringComparison.Ordinal); + Assert.Equal(2, location.GetProperty("family_members").GetArrayLength()); + + var (inlineExitCode, inlineStdout, inlineStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["InlineExecute", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var inlineDocument = ParseJsonOutput(inlineStdout); + var inlineExecute = Assert.Single(inlineDocument.RootElement.EnumerateArray().ToList()); + Assert.Equal(CommandExitCodes.Success, inlineExitCode); + Assert.Equal(string.Empty, inlineStderr); + Assert.True(inlineExecute.TryGetProperty("definition_sites", out var inlineDefinitionSites)); + Assert.Equal(2, inlineDefinitionSites.GetInt32()); + Assert.Equal( + ["src/A.AttributedPartial.cs", "src/Z.AttributedPartial.cs"], + inlineExecute.GetProperty("family_members") + .EnumerateArray() + .Select(member => member.GetProperty("path").GetString()) + .Order(StringComparer.Ordinal) + .ToArray()); + + var (inlineGotoExitCode, inlineGotoStdout, inlineGotoStderr) = CaptureConsole(() => + QueryCommandRunner.RunGoto( + ["InlineExecute", "--db", dbPath, "--exact-name", "--lang", "csharp"], + _jsonOptions)); + using var inlineGotoDocument = ParseJsonOutput(inlineGotoStdout); + Assert.Equal(CommandExitCodes.Success, inlineGotoExitCode); + Assert.Equal(string.Empty, inlineGotoStderr); + Assert.Equal(2, inlineGotoDocument.RootElement.GetProperty("family_members").GetArrayLength()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_GroupsSplitModifierAndRanksLeadingEvidence_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_leading_evidence_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Split.cs", + "csharp", + """ + namespace Demo; + public partial class SplitHost + { + partial + void OnSplit(int first, string second); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.Split.cs", + "csharp", + """ + namespace Demo; + public partial class SplitHost + { + partial + void OnSplit(int value, string text) { } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.CommentedMethod.cs", + "csharp", + """ + namespace Demo; + public partial class CommentedMethodHost + { + partial + void /* partial */ CommentedMethod(); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.CommentedMethod.cs", + "csharp", + """ + namespace Demo; + public partial class CommentedMethodHost + { + partial + void /* partial */ CommentedMethod() { } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.TabConstraint.cs", + "csharp", + "namespace Demo;\npublic partial class TabConstraint { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.TabConstraint.cs", + "csharp", + "namespace Demo;\npublic partial class TabConstraint\twhere\tT : class { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.LongDocumented.cs", + "csharp", + "namespace Demo;\npublic partial class LongDocumented { }"); + var longDocumentedSource = "namespace Demo;\n/**\n" + + string.Concat(Enumerable.Repeat(" * documentation\n", 70)) + + " */\npublic partial class LongDocumented { }"; + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.LongDocumented.cs", + "csharp", + longDocumentedSource); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Documented.cs", + "csharp", + """ + namespace Demo; + public partial class Documented { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.Documented.cs", + "csharp", + """ + namespace Demo; + /** + * Primary declaration. + + */ + public partial class Documented { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.CommentDecoy.cs", + "csharp", + """ + namespace Demo; + public partial class CommentDecoy { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.CommentDecoy.cs", + "csharp", + """ + namespace Demo; + /* + /// Not documentation. + */ + public partial class CommentDecoy { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Attributed.cs", + "csharp", + """ + namespace Demo; + public partial class Attributed { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.Attributed.cs", + "csharp", + """ + namespace Demo; + [System.Obsolete] + public partial class Attributed { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.AttributeModifier.cs", + "csharp", + """ + namespace Demo; + public partial class AttributeModifier { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.AttributeModifier.cs", + "csharp", + """ + namespace Demo; + [System.Obsolete] public partial + class AttributeModifier { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.BlankModifier.cs", + "csharp", + """ + namespace Demo; + public partial class BlankModifier { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.BlankModifier.cs", + "csharp", + """ + namespace Demo; + public + partial + + class BlankModifier { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.InlineAttributed.cs", + "csharp", + """ + namespace Demo; + public partial class InlineAttributed { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.InlineAttributed.cs", + "csharp", + """ + namespace Demo; + [System.Obsolete] public partial class InlineAttributed { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.BlankDocumentation.cs", + "csharp", + """ + namespace Demo; + public partial class BlankDocumentation { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.InlineDocumented.cs", + "csharp", + "public partial class InlineDocumented { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.InlineDocumented.cs", + "csharp", + "/** Primary declaration. */ public partial class InlineDocumented { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.InlineDocumentationDecoy.cs", + "csharp", + "public partial class InlineDocumentationDecoy { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.InlineDocumentationDecoy.cs", + "csharp", + "/* /** Not documentation. */ public partial class InlineDocumentationDecoy { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.AssemblyTarget.cs", + "csharp", + "public partial class AssemblyTarget { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.AssemblyTarget.cs", + "csharp", + """ + [assembly: System.CLSCompliant(true)] + public partial class AssemblyTarget { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.ModuleTarget.cs", + "csharp", + "public partial class ModuleTarget { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.ModuleTarget.cs", + "csharp", + """ + [module: System.CLSCompliant(true)] + public partial class ModuleTarget { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.InlineAssemblyTarget.cs", + "csharp", + "public partial class InlineAssemblyTarget { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.InlineAssemblyTarget.cs", + "csharp", + "[assembly: System.CLSCompliant(true)] public partial class InlineAssemblyTarget { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.InlineModuleTarget.cs", + "csharp", + "public partial class InlineModuleTarget { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.InlineModuleTarget.cs", + "csharp", + "[module: System.CLSCompliant(true)] public partial class InlineModuleTarget { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.BlankDocumentation.cs", + "csharp", + """ + namespace Demo; + /// Detached documentation. + + public partial class BlankDocumentation { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.UnattributedAfterSibling.cs", + "csharp", + "public partial class UnattributedAfterSibling { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.UnattributedAfterSibling.cs", + "csharp", + "[System.Obsolete] public class AttributeOwner { } public partial class UnattributedAfterSibling { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/SameLineDuplicate.cs", + "csharp", + "public partial class SameLineDuplicate { } public partial class SameLineDuplicate { }"); + MarkGraphAndFoldReady(dbPath); + + var split = RunGroupedSymbol(dbPath, "OnSplit", "function"); + Assert.Equal(2, split.GetProperty("definition_sites").GetInt32()); + Assert.Contains( + split.GetProperty("family_members").EnumerateArray(), + member => member.GetProperty("path").GetString() == "src/A.Split.cs" + && member.GetProperty("start_column").GetInt32() == 9); + Assert.Contains( + split.GetProperty("family_members").EnumerateArray(), + member => member.GetProperty("path").GetString() == "src/Z.Split.cs" + && member.GetProperty("start_column").GetInt32() == 9); + Assert.Equal(9, split.GetProperty("start_column").GetInt32()); + + var commentedMethod = RunGroupedSymbol(dbPath, "CommentedMethod", "function"); + Assert.Equal(2, commentedMethod.GetProperty("definition_sites").GetInt32()); + Assert.Equal("src/Z.CommentedMethod.cs", commentedMethod.GetProperty("path").GetString()); + Assert.Equal("implementation_body", commentedMethod.GetProperty("representative_reason").GetString()); + + var tabConstraint = RunGroupedSymbol(dbPath, "TabConstraint", "class"); + Assert.Equal("src/Z.TabConstraint.cs", tabConstraint.GetProperty("path").GetString()); + Assert.Equal("semantic_declaration", tabConstraint.GetProperty("representative_reason").GetString()); + + var longDocumented = RunGroupedSymbol(dbPath, "LongDocumented", "class"); + Assert.Equal("src/Z.LongDocumented.cs", longDocumented.GetProperty("path").GetString()); + Assert.Equal("semantic_declaration", longDocumented.GetProperty("representative_reason").GetString()); + + var documented = RunGroupedSymbol(dbPath, "Documented", "class"); + Assert.Equal("src/Z.Documented.cs", documented.GetProperty("path").GetString()); + Assert.Equal("semantic_declaration", documented.GetProperty("representative_reason").GetString()); + + var commentDecoy = RunGroupedSymbol(dbPath, "CommentDecoy", "class"); + Assert.Equal("src/A.CommentDecoy.cs", commentDecoy.GetProperty("path").GetString()); + Assert.Equal("stable_path_and_position", commentDecoy.GetProperty("representative_reason").GetString()); + + var attributed = RunGroupedSymbol(dbPath, "Attributed", "class"); + Assert.Equal("src/Z.Attributed.cs", attributed.GetProperty("path").GetString()); + Assert.Equal("semantic_declaration", attributed.GetProperty("representative_reason").GetString()); + + var attributeModifier = RunGroupedSymbol(dbPath, "AttributeModifier", "class"); + Assert.Equal(2, attributeModifier.GetProperty("definition_sites").GetInt32()); + Assert.Equal("src/Z.AttributeModifier.cs", attributeModifier.GetProperty("path").GetString()); + Assert.Equal("semantic_declaration", attributeModifier.GetProperty("representative_reason").GetString()); + + var blankModifier = RunGroupedSymbol(dbPath, "BlankModifier", "class"); + Assert.Equal(2, blankModifier.GetProperty("definition_sites").GetInt32()); + + var inlineAttributed = RunGroupedSymbol(dbPath, "InlineAttributed", "class"); + Assert.Equal("src/Z.InlineAttributed.cs", inlineAttributed.GetProperty("path").GetString()); + Assert.Equal("semantic_declaration", inlineAttributed.GetProperty("representative_reason").GetString()); + + var blankDocumentation = RunGroupedSymbol(dbPath, "BlankDocumentation", "class"); + Assert.Equal("src/A.BlankDocumentation.cs", blankDocumentation.GetProperty("path").GetString()); + Assert.Equal("stable_path_and_position", blankDocumentation.GetProperty("representative_reason").GetString()); + + var inlineDocumented = RunGroupedSymbol(dbPath, "InlineDocumented", "class"); + Assert.Equal("src/Z.InlineDocumented.cs", inlineDocumented.GetProperty("path").GetString()); + Assert.Equal("semantic_declaration", inlineDocumented.GetProperty("representative_reason").GetString()); + + var inlineDocumentationDecoy = RunGroupedSymbol(dbPath, "InlineDocumentationDecoy", "class"); + Assert.Equal("src/A.InlineDocumentationDecoy.cs", inlineDocumentationDecoy.GetProperty("path").GetString()); + Assert.Equal("stable_path_and_position", inlineDocumentationDecoy.GetProperty("representative_reason").GetString()); + + var assemblyTarget = RunGroupedSymbol(dbPath, "AssemblyTarget", "class"); + Assert.Equal("src/A.AssemblyTarget.cs", assemblyTarget.GetProperty("path").GetString()); + Assert.Equal("stable_path_and_position", assemblyTarget.GetProperty("representative_reason").GetString()); + + var moduleTarget = RunGroupedSymbol(dbPath, "ModuleTarget", "class"); + Assert.Equal("src/A.ModuleTarget.cs", moduleTarget.GetProperty("path").GetString()); + Assert.Equal("stable_path_and_position", moduleTarget.GetProperty("representative_reason").GetString()); + + foreach (var inlineGlobalTarget in new[] { "InlineAssemblyTarget", "InlineModuleTarget" }) + { + var grouped = RunGroupedSymbol(dbPath, inlineGlobalTarget, "class"); + Assert.Equal($"src/A.{inlineGlobalTarget}.cs", grouped.GetProperty("path").GetString()); + Assert.Equal("stable_path_and_position", grouped.GetProperty("representative_reason").GetString()); + } + + var unattributedAfterSibling = RunGroupedSymbol(dbPath, "UnattributedAfterSibling", "class"); + Assert.Equal("src/A.UnattributedAfterSibling.cs", unattributedAfterSibling.GetProperty("path").GetString()); + Assert.Equal("stable_path_and_position", unattributedAfterSibling.GetProperty("representative_reason").GetString()); + + var sameLineDuplicate = RunGroupedSymbol(dbPath, "SameLineDuplicate", "class"); + Assert.Equal(2, sameLineDuplicate.GetProperty("definition_sites").GetInt32()); + Assert.Equal( + [21, 64], + sameLineDuplicate.GetProperty("family_members") + .EnumerateArray() + .Select(member => member.GetProperty("start_column").GetInt32()) + .Order() + .ToArray()); + + var (gotoAllExitCode, gotoAllStdout, gotoAllStderr) = CaptureConsole(() => QueryCommandRunner.RunGoto( + ["SameLineDuplicate", "--db", dbPath, "--exact-name", "--lang", "csharp", "--kind", "class", "--all"], + _jsonOptions)); + using var gotoAllDocument = ParseJsonOutput(gotoAllStdout); + var allLocations = gotoAllDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, gotoAllExitCode); + Assert.Equal(string.Empty, gotoAllStderr); + Assert.Equal( + [21, 64], + allLocations + .Select(location => location.GetProperty("range").GetProperty("start").GetProperty("character").GetInt32()) + .Order() + .ToArray()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_SkipsPreprocessorTriviaWithoutCrossingBranches_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_directive_trivia_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + foreach (var path in new[] { "src/A.DirectiveLocal.cs", "src/B.DirectiveLocal.cs" }) + { + TestProjectHelper.InsertIndexedFile( + dbPath, + path, + "csharp", + """ + file + #if true + #endif + partial class DirectiveLocal { } + """); + } + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.DirectivePartial.cs", + "csharp", + """ + partial + #nullable enable + #if true + #endif + class DirectivePartial { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/B.DirectivePartial.cs", + "csharp", + "partial class DirectivePartial { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/DirectiveBranchDecoy.cs", + "csharp", + """ + #if false + partial + #endif + class DirectiveBranchDecoy { } + partial class DirectiveBranchDecoy { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.DirectiveBranchPartial.cs", + "csharp", + """ + partial + #if FIRST + class DirectiveBranchPartial { } + #else + class DirectiveBranchPartial { } + #endif + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/B.DirectiveBranchPartial.cs", + "csharp", + "partial class DirectiveBranchPartial { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/DirectiveClosedBoundary.cs", + "csharp", + """ + partial + #if FIRST + class DirectiveClosedBoundary { } + #endif + class DirectiveClosedBoundary { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.DirectiveJoinedPartial.cs", + "csharp", + """ + #if FIRST + partial + #else + partial + #endif + class DirectiveJoinedPartial { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/B.DirectiveJoinedPartial.cs", + "csharp", + "partial class DirectiveJoinedPartial { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.DirectiveConditionalAttribute.cs", + "csharp", + "partial class DirectiveConditionalAttribute { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.DirectiveConditionalAttribute.cs", + "csharp", + """ + #if FIRST + [System.Obsolete] + #else + [System.Serializable] + #endif + partial class DirectiveConditionalAttribute { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/DirectiveJoinedDecoy.cs", + "csharp", + """ + #if FIRST + partial + #else + public + #endif + class DirectiveJoinedDecoy { } + partial class DirectiveJoinedDecoy { } + """); + foreach (var path in new[] { "src/A.DirectiveConditionalFile.cs", "src/B.DirectiveConditionalFile.cs" }) + { + TestProjectHelper.InsertIndexedFile( + dbPath, + path, + "csharp", + """ + #if FIRST + file + #else + public + #endif + partial class DirectiveConditionalFile { } + """); + } + MarkGraphAndFoldReady(dbPath); + + var (localExitCode, localStdout, localStderr) = CaptureConsole(() => + QueryCommandRunner.RunSymbols( + ["DirectiveLocal", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var localDocument = ParseJsonOutput(localStdout); + var localFamilies = localDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, localExitCode); + Assert.Equal(string.Empty, localStderr); + Assert.Equal(2, localFamilies.Count); + Assert.All(localFamilies, family => Assert.False(family.TryGetProperty("definition_sites", out _))); + + var directivePartial = RunGroupedSymbol(dbPath, "DirectivePartial", "class"); + Assert.Equal(2, directivePartial.GetProperty("definition_sites").GetInt32()); + + var directiveBranchPartial = RunGroupedSymbol( + dbPath, + "DirectiveBranchPartial", + "class"); + Assert.Equal(3, directiveBranchPartial.GetProperty("definition_sites").GetInt32()); + + var (closedExitCode, closedStdout, closedStderr) = CaptureConsole(() => + QueryCommandRunner.RunSymbols( + ["DirectiveClosedBoundary", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var closedDocument = ParseJsonOutput(closedStdout); + var closedRows = closedDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, closedExitCode); + Assert.Equal(string.Empty, closedStderr); + Assert.Equal(2, closedRows.Count); + Assert.All(closedRows, row => Assert.False(row.TryGetProperty("definition_sites", out _))); + + var directiveJoinedPartial = RunGroupedSymbol( + dbPath, + "DirectiveJoinedPartial", + "class"); + Assert.Equal(2, directiveJoinedPartial.GetProperty("definition_sites").GetInt32()); + + var directiveConditionalAttribute = RunGroupedSymbol( + dbPath, + "DirectiveConditionalAttribute", + "class"); + Assert.Equal( + "src/Z.DirectiveConditionalAttribute.cs", + directiveConditionalAttribute.GetProperty("path").GetString()); + Assert.Equal( + "semantic_declaration", + directiveConditionalAttribute.GetProperty("representative_reason").GetString()); + + var (joinedDecoyExitCode, joinedDecoyStdout, joinedDecoyStderr) = CaptureConsole(() => + QueryCommandRunner.RunSymbols( + ["DirectiveJoinedDecoy", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var joinedDecoyDocument = ParseJsonOutput(joinedDecoyStdout); + var joinedDecoyRows = joinedDecoyDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, joinedDecoyExitCode); + Assert.Equal(string.Empty, joinedDecoyStderr); + Assert.Equal(2, joinedDecoyRows.Count); + Assert.All(joinedDecoyRows, row => Assert.False(row.TryGetProperty("definition_sites", out _))); + + var (conditionalFileExitCode, conditionalFileStdout, conditionalFileStderr) = CaptureConsole(() => + QueryCommandRunner.RunSymbols( + ["DirectiveConditionalFile", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var conditionalFileDocument = ParseJsonOutput(conditionalFileStdout); + var conditionalFileRows = conditionalFileDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, conditionalFileExitCode); + Assert.Equal(string.Empty, conditionalFileStderr); + Assert.Equal(2, conditionalFileRows.Count); + Assert.All(conditionalFileRows, row => Assert.False(row.TryGetProperty("definition_sites", out _))); + + var (decoyExitCode, decoyStdout, decoyStderr) = CaptureConsole(() => + QueryCommandRunner.RunSymbols( + ["DirectiveBranchDecoy", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var decoyDocument = ParseJsonOutput(decoyStdout); + var decoyRows = decoyDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, decoyExitCode); + Assert.Equal(string.Empty, decoyStderr); + Assert.Equal(2, decoyRows.Count); + Assert.All(decoyRows, row => Assert.False(row.TryGetProperty("definition_sites", out _))); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_RespectsFileLocalAndLexedEvidence_Issue4914() + { + const string definingLine = " [M()] partial /* M( */ void M();"; + var publicDefiningLine = definingLine.Replace("partial", "public partial", StringComparison.Ordinal); + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_file_local_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + foreach (var path in new[] { "src/A.Local.cs", "src/B.Local.cs" }) + { + TestProjectHelper.InsertIndexedFile( + dbPath, + path, + "csharp", + $$""" + using System; + file sealed class MAttribute : Attribute { } + file partial class Host + { + {{publicDefiningLine}} + } + partial class Host + { + public partial void M() { } + } + file sealed class Use { public void Call() { new Host().M(); } } + """); + } + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Widget.cs", + "csharp", + "public partial class Widget { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.Widget.cs", + "csharp", + "public partial /* [ : where */ class Widget { }"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Shared.cs", + "csharp", + """ + namespace Demo; + public partial class Shared { } + public partial class Ranked { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.LeadingEvidence.cs", + "csharp", + """ + namespace Demo; + file partial + class LocalOnly { } public partial class Shared { } + [System.Obsolete] + class Decorated { } public partial class Ranked { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.SplitLocalNode.cs", + "csharp", + """ + namespace Demo; + file + class Node { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/B.ProjectNode.cs", + "csharp", + """ + namespace Demo; + public struct Node { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/C.NullableNode.cs", + "csharp", + """ + namespace Demo; + public partial class NullableContainer + { + partial void NullableNode(Node? value); + partial void NullableNode(global::System.Nullable value) { } + } + """); + MarkGraphAndFoldReady(dbPath); + + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["M", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var document = ParseJsonOutput(stdout); + var families = document.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + Assert.Equal(2, families.Count); + Assert.All(families, family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + Assert.Equal( + 2, + families + .Select(family => family.GetProperty("partial_family_id").GetString()) + .Distinct(StringComparer.Ordinal) + .Count()); + Assert.All( + families, + family => + { + var path = family.GetProperty("path").GetString(); + var members = family.GetProperty("family_members").EnumerateArray().ToList(); + Assert.All(members, member => Assert.Equal(path, member.GetProperty("path").GetString())); + Assert.Contains( + members, + member => member.GetProperty("line").GetInt32() == 5 + && member.GetProperty("start_column").GetInt32() + == publicDefiningLine.LastIndexOf("M", StringComparison.Ordinal)); + }); + + var (hostExitCode, hostStdout, hostStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["Host", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var hostDocument = ParseJsonOutput(hostStdout); + var hostFamilies = hostDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, hostExitCode); + Assert.Equal(string.Empty, hostStderr); + Assert.Equal(2, hostFamilies.Count); + Assert.All(hostFamilies, family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + Assert.All( + hostFamilies, + family => Assert.Single( + family.GetProperty("family_members") + .EnumerateArray() + .Select(member => member.GetProperty("path").GetString()) + .Distinct(StringComparer.Ordinal))); + + var (hotspotsExitCode, hotspotsStdout, hotspotsStderr) = CaptureConsole(() => QueryCommandRunner.RunHotspots( + ["--db", dbPath, "--json", "--lang", "csharp", "--kind", "function", "--limit", "10"], + _jsonOptions)); + using var hotspotsDocument = ParseJsonOutput(hotspotsStdout); + var methodHotspots = hotspotsDocument.RootElement.GetProperty("hotspots") + .EnumerateArray() + .Where(hotspot => hotspot.GetProperty("name").GetString() == "M") + .ToList(); + + Assert.Equal(CommandExitCodes.Success, hotspotsExitCode); + Assert.Equal(string.Empty, hotspotsStderr); + Assert.Equal(2, methodHotspots.Count); + Assert.All(methodHotspots, hotspot => Assert.Equal(2, hotspot.GetProperty("reference_count").GetInt32())); + Assert.Equal( + 2, + methodHotspots + .Select(hotspot => hotspot.GetProperty("path").GetString()) + .Distinct(StringComparer.Ordinal) + .Count()); + + using (var connection = new SqliteConnection($"Data Source={dbPath}")) + { + connection.Open(); + using var candidatePaths = connection.CreateCommand(); + candidatePaths.CommandText = + """ + SELECT source_file.path, target_file.path + FROM symbol_references AS reference + JOIN files AS source_file ON source_file.id = reference.file_id + JOIN symbol_reference_candidates AS candidate ON candidate.reference_id = reference.id + JOIN symbols AS target ON target.id = candidate.symbol_id + JOIN files AS target_file ON target_file.id = target.file_id + WHERE reference.symbol_name IN ('Host', 'M') + AND reference.reference_kind IN ('instantiate', 'call') + ORDER BY source_file.path, target_file.path + """; + using var reader = candidatePaths.ExecuteReader(); + var resolvedPaths = new List<(string Source, string Target)>(); + while (reader.Read()) + resolvedPaths.Add((reader.GetString(0), reader.GetString(1))); + + Assert.NotEmpty(resolvedPaths); + Assert.All(resolvedPaths, paths => Assert.Equal(paths.Source, paths.Target)); + Assert.Equal( + ["src/A.Local.cs", "src/B.Local.cs"], + resolvedPaths.Select(paths => paths.Source).Distinct(StringComparer.Ordinal).ToArray()); + } + + var widget = RunGroupedSymbol(dbPath, "Widget", "class"); + Assert.Equal("src/A.Widget.cs", widget.GetProperty("path").GetString()); + Assert.Equal("stable_path_and_position", widget.GetProperty("representative_reason").GetString()); + + var shared = RunGroupedSymbol(dbPath, "Shared", "class"); + Assert.Equal(2, shared.GetProperty("definition_sites").GetInt32()); + Assert.Equal("src/A.Shared.cs", shared.GetProperty("path").GetString()); + + var ranked = RunGroupedSymbol(dbPath, "Ranked", "class"); + Assert.Equal(2, ranked.GetProperty("definition_sites").GetInt32()); + Assert.Equal("src/A.Shared.cs", ranked.GetProperty("path").GetString()); + Assert.Equal("stable_path_and_position", ranked.GetProperty("representative_reason").GetString()); + + var nullableNode = RunGroupedSymbol(dbPath, "NullableNode", "function"); + Assert.Equal(2, nullableNode.GetProperty("definition_sites").GetInt32()); + Assert.Equal("implementation_body", nullableNode.GetProperty("representative_reason").GetString()); + + using var metadataConnection = new SqliteConnection($"Data Source={dbPath}"); + metadataConnection.Open(); + using var fileLocalMetadata = metadataConnection.CreateCommand(); + fileLocalMetadata.CommandText = + """ + SELECT f.path, s.is_file_local_declaration + FROM symbols AS s + JOIN files AS f ON f.id = s.file_id + WHERE s.name = 'Node' + ORDER BY f.path + """; + using var fileLocalReader = fileLocalMetadata.ExecuteReader(); + Assert.True(fileLocalReader.Read()); + Assert.Equal("src/A.SplitLocalNode.cs", fileLocalReader.GetString(0)); + Assert.True(fileLocalReader.GetBoolean(1)); + Assert.True(fileLocalReader.Read()); + Assert.Equal("src/B.ProjectNode.cs", fileLocalReader.GetString(0)); + Assert.False(fileLocalReader.GetBoolean(1)); + Assert.False(fileLocalReader.Read()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void GroupPartials_DegradesSafelyWhenCSharpFamilyContractIsStale_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_stale_family_contract_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + foreach (var path in new[] { "src/A.Hosts.cs", "src/Z.Hosts.cs" }) + { + TestProjectHelper.InsertIndexedFile( + dbPath, + path, + "csharp", + """ + namespace Demo; + public partial class Host + { + partial void OnReady(); + } + public partial class Host + { + partial void OnReady(); + } + """); + } + MarkGraphAndFoldReady(dbPath); + + using (var connection = new SqliteConnection($"Data Source={dbPath}")) + { + connection.Open(); + using (var staleFamilyKeys = connection.CreateCommand()) + { + staleFamilyKeys.CommandText = + """ + UPDATE symbols + SET family_key = REPLACE(REPLACE(family_key, 'Host`1', 'Host'), 'Host`2', 'Host') + WHERE name = 'OnReady' + """; + Assert.Equal(4, staleFamilyKeys.ExecuteNonQuery()); + } + + using var staleContract = connection.CreateCommand(); + staleContract.CommandText = + """ + INSERT INTO codeindex_meta(key, value) + VALUES ($key, $value) + ON CONFLICT(key) DO UPDATE SET value = excluded.value + """; + staleContract.Parameters.AddWithValue( + "$key", + DbContext.GetHotspotFamilyVersionMetaKey("csharp")); + staleContract.Parameters.AddWithValue( + "$value", + (DbContext.HotspotFamilyVersion - 1).ToString(System.Globalization.CultureInfo.InvariantCulture)); + Assert.Equal(1, staleContract.ExecuteNonQuery()); + } + + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["OnReady", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var document = ParseJsonOutput(stdout); + var rows = document.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + Assert.Equal(4, rows.Count); + Assert.All(rows, row => Assert.False(row.TryGetProperty("definition_sites", out _))); + Assert.All(rows, row => Assert.False(row.TryGetProperty("partial_family_id", out _))); + + var (impactExitCode, impactStdout, impactStderr) = CaptureConsole(() => QueryCommandRunner.RunImpact( + ["Host", "--db", dbPath, "--json", "--lang", "csharp", "--max-hops", "0", "--limit", "10"], + _jsonOptions)); + using var impactDocument = ParseJsonOutput(impactStdout); + var impact = impactDocument.RootElement; + + Assert.Equal(CommandExitCodes.Success, impactExitCode); + Assert.Equal(string.Empty, impactStderr); + Assert.Equal(4, impact.GetProperty("definition_count").GetInt32()); + Assert.Equal(4, impact.GetProperty("logical_definition_count").GetInt32()); + Assert.Equal(4, impact.GetProperty("definition_output_count").GetInt32()); + Assert.False(impact.GetProperty("definitions_collapsed").GetBoolean()); + Assert.All( + impact.GetProperty("definitions").EnumerateArray(), + definition => Assert.False(definition.TryGetProperty("partial_family_id", out _))); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_FallsBackToDesignerPathForOldDatabaseMetadata_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_canonical_old_db_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.LegacyWidget.Designer.cs", + "csharp", + """ + namespace Demo; + [System.Obsolete] + public partial class LegacyWidget : LegacyBase { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.LegacyWidget.cs", + "csharp", + """ + namespace Demo; + public partial class LegacyWidget { } + """); + MarkGraphAndFoldReady(dbPath); + + using (var connection = new SqliteConnection($"Data Source={dbPath}")) + { + connection.Open(); + using (var dropGeneratedIndex = connection.CreateCommand()) + { + dropGeneratedIndex.CommandText = "DROP INDEX IF EXISTS idx_files_generated"; + dropGeneratedIndex.ExecuteNonQuery(); + } + using var dropGeneratedColumn = connection.CreateCommand(); + dropGeneratedColumn.CommandText = "ALTER TABLE files DROP COLUMN generated"; + dropGeneratedColumn.ExecuteNonQuery(); + } + + var legacyWidget = RunGroupedSymbol(dbPath, "LegacyWidget", "class"); + Assert.Equal("src/Z.LegacyWidget.cs", legacyWidget.GetProperty("path").GetString()); + Assert.Equal("non_generated_source", legacyWidget.GetProperty("representative_reason").GetString()); + Assert.Contains( + legacyWidget.GetProperty("family_members").EnumerateArray(), + member => member.GetProperty("path").GetString() == "src/A.LegacyWidget.Designer.cs" + && member.GetProperty("generated").GetBoolean()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_UsesIndexedTypeFactsForCustomNullability_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_custom_nullability_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Declarations.cs", + "csharp", + """ + #nullable enable + namespace Demo; + public class Node { } + public struct Token { } + public class Box { } + public struct Box { } + public class Outer { public class Nested { } } + public class Outer { public struct Nested { } } + public partial class Container + { + partial void Reference(Node? value); + partial void Generic(Box? value); + partial void NestedGeneric(Outer.Nested? value); + partial void Value(Token value); + partial void Value(Token? value); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Z.Implementations.cs", + "csharp", + """ + #nullable enable + namespace Demo; + public partial class Container + { + partial void Reference(Node value) { } + partial void Generic(Box value) { } + partial void NestedGeneric(Outer.Nested value) { } + partial void Value(Token value) { } + partial void Value(Token? value) { } + } + """); + MarkGraphAndFoldReady(dbPath); + + var reference = RunGroupedSymbol(dbPath, "Reference", "function"); + Assert.Equal(2, reference.GetProperty("definition_sites").GetInt32()); + Assert.Equal("src/Z.Implementations.cs", reference.GetProperty("path").GetString()); + Assert.Equal("implementation_body", reference.GetProperty("representative_reason").GetString()); + + var generic = RunGroupedSymbol(dbPath, "Generic", "function"); + Assert.True(generic.TryGetProperty("definition_sites", out var genericDefinitionSites), generic.GetRawText()); + Assert.Equal(2, genericDefinitionSites.GetInt32()); + var nestedGeneric = RunGroupedSymbol(dbPath, "NestedGeneric", "function"); + Assert.Equal(2, nestedGeneric.GetProperty("definition_sites").GetInt32()); + + var (valueExitCode, valueStdout, valueStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["Value", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var valueDocument = ParseJsonOutput(valueStdout); + var valueFamilies = valueDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, valueExitCode); + Assert.Equal(string.Empty, valueStderr); + Assert.Equal(2, valueFamilies.Count); + Assert.All(valueFamilies, family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + Assert.Contains(valueFamilies, family => family.GetProperty("signature").GetString()!.Contains("Token value", StringComparison.Ordinal)); + Assert.Contains(valueFamilies, family => family.GetProperty("signature").GetString()!.Contains("Token? value", StringComparison.Ordinal)); + + var (gotoExitCode, gotoStdout, gotoStderr) = CaptureConsole(() => QueryCommandRunner.RunGoto( + ["Reference", "--db", dbPath, "--exact-name", "--lang", "csharp", "--kind", "function"], + _jsonOptions)); + + Assert.Equal(CommandExitCodes.Success, gotoExitCode); + Assert.Equal(string.Empty, gotoStderr); + using var gotoDocument = JsonDocument.Parse(gotoStdout); + Assert.EndsWith( + "/src/Z.Implementations.cs", + new Uri(gotoDocument.RootElement.GetProperty("uri").GetString()!).AbsolutePath, + StringComparison.Ordinal); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_PreservesProjectScopeForNullableTypeFacts_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_project_nullable_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "A/A.csproj", + "msbuild", + ""); + TestProjectHelper.InsertIndexedFile( + dbPath, + "A/Types.cs", + "csharp", + """ + namespace Demo; + public class Node { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "A/Partials.cs", + "csharp", + """ + #nullable enable + namespace Demo; + public partial class Container + { + partial void Scoped(Node? value); + partial void Scoped(Node value) { } + partial void ExternalScoped(External.Foo? value); + partial void ExternalScoped(External.Foo? value) { } + partial void ExternalScoped(External.Foo value); + partial void ExternalScoped(External.Foo value) { } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "B/B.csproj", + "msbuild", + ""); + TestProjectHelper.InsertIndexedFile( + dbPath, + "B/Types.cs", + "csharp", + """ + namespace Demo; + public struct Node { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "B/ExternalTypes.cs", + "csharp", + "namespace External; public class Foo { }"); + using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath)) + using (var command = db.Connection.CreateCommand()) + { + command.CommandText = + """ + UPDATE symbols + SET family_key = 'A|' || family_key + WHERE family_key IS NOT NULL + AND file_id IN (SELECT id FROM files WHERE path LIKE 'A/%'); + UPDATE symbols + SET family_key = 'B|' || family_key + WHERE family_key IS NOT NULL + AND file_id IN (SELECT id FROM files WHERE path LIKE 'B/%'); + """; + command.ExecuteNonQuery(); + } + MarkGraphAndFoldReady(dbPath); + + var grouped = RunGroupedSymbol(dbPath, "Scoped", "function"); + + Assert.Equal(2, grouped.GetProperty("definition_sites").GetInt32()); + Assert.Contains("Node value", grouped.GetProperty("signature").GetString(), StringComparison.Ordinal); + + var (externalExitCode, externalStdout, externalStderr) = CaptureConsole(() => + QueryCommandRunner.RunSymbols( + ["ExternalScoped", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var externalDocument = ParseJsonOutput(externalStdout); + var externalFamilies = externalDocument.RootElement.EnumerateArray().ToList(); + Assert.Equal(CommandExitCodes.Success, externalExitCode); + Assert.Equal(string.Empty, externalStderr); + Assert.Equal(2, externalFamilies.Count); + Assert.All( + externalFamilies, + family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_HonorsCaseSensitiveProjectMarkerCasing_Issue4914Review() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_project_marker_case_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "APP.CSPROJ", + "msbuild", + ""); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Types.cs", + "csharp", + """ + namespace Demo; + public class Node { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Partials.cs", + "csharp", + """ + #nullable enable + namespace Demo; + public partial class Container + { + partial void Scoped(Node? value); + partial void Scoped(Node value) { } + } + """); + using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath)) + { + var writer = new DbWriter(db.Connection); + writer.SetMeta(DbContext.WorkspacePathCaseSensitiveMetaKey, "true"); + using var command = db.Connection.CreateCommand(); + command.CommandText = """ + UPDATE symbols + SET family_key = 'src|' || family_key + WHERE family_key IS NOT NULL + AND file_id IN (SELECT id FROM files WHERE path LIKE 'src/%'); + """; + command.ExecuteNonQuery(); + } + MarkGraphAndFoldReady(dbPath); + + var grouped = RunGroupedSymbol(dbPath, "Scoped", "function"); + + Assert.True( + grouped.TryGetProperty("definition_sites", out var definitionSites), + grouped.GetRawText()); + Assert.Equal(2, definitionSites.GetInt32()); + Assert.Equal("implementation_body", grouped.GetProperty("representative_reason").GetString()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_CanonicalizesExplicitNullableValueType_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_explicit_nullable_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Container.cs", + "csharp", + """ + #nullable enable + public struct Token { } + public partial class Container + { + partial void NullableValue(Token? value); + partial void NullableValue(global::System.Nullable value) { } + partial void NullableGeneric(T? value) where T : struct; + partial void NullableGeneric(global::System.Nullable value) where T : struct { } + } + """); + MarkGraphAndFoldReady(dbPath); + + var grouped = RunGroupedSymbol(dbPath, "NullableValue", "function"); + var groupedGeneric = RunGroupedSymbol(dbPath, "NullableGeneric", "function"); + + Assert.Equal(2, grouped.GetProperty("definition_sites").GetInt32()); + Assert.Contains( + "global::System.Nullable", + grouped.GetProperty("signature").GetString(), + StringComparison.Ordinal); + Assert.Equal(2, groupedGeneric.GetProperty("definition_sites").GetInt32()); + Assert.Contains( + "global::System.Nullable", + groupedGeneric.GetProperty("signature").GetString(), + StringComparison.Ordinal); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_ResolvesScopedAndGenericNullableIdentities_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_scoped_nullable_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Scoped.cs", + "csharp", + """ + #nullable enable + using External; + using @global = Values; + using @struct = External.ConstraintClass; + namespace Demo; + public struct Node { } + public class Target { } + public partial class Outer + { + public class Node { } + partial void Nested(Node? value); + partial void Nested(Node value) { } + } + public partial class ReferenceContainer where T : class + { + partial void ContainingReference(T? value); + } + public partial class ValueContainer where T : struct + { + partial void ContainingValue(T? value); + } + public partial class SplitConstraintContainer + { + partial void SplitContainingValue(T? value); + } + public partial class EscapedConstraintContainer where T : @struct + { + partial void ContainingEscapedConstraint(T? value); + } + public partial class ShadowOuter where T : class + { + public partial class ShadowInner where T : struct + { + partial void Shadowed(T? value); + } + } + public class QualifiedOuter + { + public class QualifiedNode { } + } + public partial class Container + { + partial void Qualified(QualifiedOuter.QualifiedNode? value); + partial void Qualified(QualifiedOuter.QualifiedNode value) { } + partial void Generic(T? value) where T : class; + partial void Generic(T value) where T : class { } + partial void EscapedMethodConstraint(T? value) where T : @struct; + partial void EscapedMethodConstraint(T value) where T : @struct { } + partial void Aliased(@global::Target value); + partial void Aliased(@global::Target value) { } + partial void Aliased(@global::Target? value); + partial void Aliased(@global::Target? value) { } + partial void Combining(int á); + partial void Combining(int b́) { } + partial void Imported(ImportedNode value); + partial void Imported(ImportedNode? value); + partial void Imported(ImportedNode value) { } + partial void Imported(ImportedNode? value) { } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/ScopedImplementations.cs", + "csharp", + """ + #nullable enable + using @struct = External.ConstraintClass; + namespace Demo; + public partial class ReferenceContainer where T : class + { + partial void ContainingReference(T value) { } + } + public partial class ValueContainer where T : struct + { + partial void ContainingValue(global::System.Nullable value) { } + } + public partial class SplitConstraintContainer where T : struct + { + partial void SplitContainingValue(global::System.Nullable value) { } + } + public partial class EscapedConstraintContainer where T : @struct + { + partial void ContainingEscapedConstraint(T value) { } + } + public partial class ShadowOuter where T : class + { + public partial class ShadowInner where T : struct + { + partial void Shadowed(global::System.Nullable value) { } + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/UnrelatedLeaf.cs", + "csharp", + """ + namespace Other; + public class ImportedNode { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/AliasTypes.cs", + "csharp", + """ + namespace Values + { + public struct Target { } + } + namespace External + { + public class ConstraintClass { } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.FileLocal.cs", + "csharp", + """ + #nullable enable + namespace Demo; + file class LocalNode { } + file partial class LocalContainer + { + partial void Local(LocalNode? value); + partial void Local(LocalNode value) { } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/B.FileLocal.cs", + "csharp", + """ + namespace Demo; + file struct LocalNode { } + """); + MarkGraphAndFoldReady(dbPath); + + foreach (var name in new[] + { + "Nested", + "ContainingReference", + "ContainingValue", + "SplitContainingValue", + "ContainingEscapedConstraint", + "Shadowed", + "Qualified", + "Generic", + "EscapedMethodConstraint", + "Combining", + "Local", + }) + { + var grouped = RunGroupedSymbol(dbPath, name, "function"); + Assert.True(grouped.TryGetProperty("definition_sites", out var definitionSites), grouped.GetRawText()); + Assert.Equal(2, definitionSites.GetInt32()); + } + + var (importedExitCode, importedStdout, importedStderr) = CaptureConsole(() => + QueryCommandRunner.RunSymbols( + ["Imported", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var importedDocument = ParseJsonOutput(importedStdout); + var importedFamilies = importedDocument.RootElement.EnumerateArray().ToList(); + Assert.Equal(CommandExitCodes.Success, importedExitCode); + Assert.Equal(string.Empty, importedStderr); + Assert.Equal(2, importedFamilies.Count); + Assert.All( + importedFamilies, + family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + + var (aliasedExitCode, aliasedStdout, aliasedStderr) = CaptureConsole(() => + QueryCommandRunner.RunSymbols( + ["Aliased", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var aliasedDocument = ParseJsonOutput(aliasedStdout); + var aliasedFamilies = aliasedDocument.RootElement.EnumerateArray().ToList(); + Assert.Equal(CommandExitCodes.Success, aliasedExitCode); + Assert.Equal(string.Empty, aliasedStderr); + Assert.Equal(2, aliasedFamilies.Count); + Assert.All( + aliasedFamilies, + family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial T? Generic(T? value) where T : class;", + "Generic", + "T?"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial T Generic(TResult value) where TResult : class { }", + "Generic", + "TResult")); + Assert.NotEqual( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Value(T? value) where T : struct;", + "Value", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void Value(T value) where T : struct { }", + "Value", + "void")); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_LazilyRefreshesExternalTypeFacts_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_external_type_facts_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Initial.cs", + "csharp", + """ + #nullable enable + namespace Demo; + public class InitialNode { } + public partial class Container + { + partial void Initial(InitialNode? value); + partial void Initial(InitialNode value) { } + } + """); + MarkGraphAndFoldReady(dbPath); + + var scans = 0; + var candidateScans = new List(); + CSharpCallableTypeKindLookup.ScanForTesting = () => scans++; + CSharpCallableTypeKindLookup.CandidateScanForTesting = candidateScans.Add; + try + { + using var db = new DbContext(DbOpenIntent.WriteIndex, dbPath); + using var reader = new DbReader(db); + Assert.Equal(0, scans); + + var physical = reader.SearchSymbols( + ["Initial"], + limit: 10, + kind: "function", + lang: "csharp", + exact: true, + groupPartials: false); + Assert.Equal(2, physical.Count); + Assert.Equal(0, scans); + + var initial = Assert.Single(reader.SearchSymbols( + ["Initial"], + limit: 10, + kind: "function", + lang: "csharp", + exact: true, + groupPartials: true)); + Assert.Equal(2, initial.DefinitionSites); + Assert.Equal(1, scans); + var initialScan = Assert.Single(candidateScans); + Assert.False(initialScan.UsedFullScan); + Assert.Equal(2, initialScan.CallableCount); + Assert.InRange(initialScan.TypeFactCount, 1, 4); + + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Late.cs", + "csharp", + """ + #nullable enable + namespace Demo; + public class LateNode { } + public partial class Container + { + partial void Late(LateNode? value); + partial void Late(LateNode value) { } + } + """); + + var late = Assert.Single(reader.SearchSymbols( + ["Late"], + limit: 10, + kind: "function", + lang: "csharp", + exact: true, + groupPartials: true)); + Assert.Equal(2, late.DefinitionSites); + Assert.Equal(2, scans); + Assert.Equal(2, candidateScans.Count); + var lateScan = candidateScans[^1]; + Assert.False(lateScan.UsedFullScan); + Assert.Equal(2, lateScan.CallableCount); + Assert.InRange(lateScan.TypeFactCount, 1, 5); + } + finally + { + CSharpCallableTypeKindLookup.ScanForTesting = null; + CSharpCallableTypeKindLookup.CandidateScanForTesting = null; + } + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_DiscardsTransactionLocalTypeFactsAfterRollback_Issue4914Review() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_rollback_type_facts_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Container.cs", + "csharp", + """ + #nullable enable + namespace Demo; + public partial class Container + { + partial void M(Ghost? value); + partial void M(Ghost value) { } + } + """); + MarkGraphAndFoldReady(dbPath); + + var scans = 0; + CSharpCallableTypeKindLookup.ScanForTesting = () => scans++; + try + { + using var connection = new SqliteConnection($"Data Source={dbPath}"); + connection.Open(); + using var reader = new DbReader(connection); + + Assert.Equal(2, reader.SearchSymbols( + ["M"], 10, "function", "csharp", exact: true, groupPartials: true).Count); + + using (var transaction = connection.BeginTransaction()) + { + using var command = connection.CreateCommand(); + command.Transaction = transaction; + command.CommandText = """ + INSERT INTO symbols( + file_id, kind, name, line, start_line, end_line, signature, + is_partial_declaration, is_file_local_declaration) + SELECT id, 'class', 'Ghost', 20, 20, 20, 'class Ghost', 0, 0 + FROM files + WHERE lang = 'csharp' + LIMIT 1 + """; + Assert.Equal(1, command.ExecuteNonQuery()); + + var grouped = Assert.Single(reader.SearchSymbols( + ["M"], 10, "function", "csharp", exact: true, groupPartials: true)); + Assert.Equal(2, grouped.DefinitionSites); + transaction.Rollback(); + } + + Assert.Equal(2, reader.SearchSymbols( + ["M"], 10, "function", "csharp", exact: true, groupPartials: true).Count); + Assert.Equal(3, scans); + } + finally + { + CSharpCallableTypeKindLookup.ScanForTesting = null; + } + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_SkipsCallableScansWhenGroupingCannotUseThem_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_callable_scan_gate_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Container.cs", + "csharp", + """ + namespace Demo; + public partial class Container + { + partial void Run(int value); + partial void Run(int value) { } + } + """); + MarkGraphAndFoldReady(dbPath); + + var scans = 0; + CSharpCallableTypeKindLookup.ScanForTesting = () => scans++; + try + { + using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath)) + using (var reader = new DbReader(db)) + { + Assert.Single(reader.SearchSymbols( + queries: null, + limit: 10, + kind: "class", + lang: "csharp", + groupPartials: true)); + Assert.Equal( + 1, + reader.CountSearchSymbolsTotal( + queries: (IReadOnlyList?)null, + kind: "class", + lang: "csharp", + groupPartials: true).Count); + Assert.Equal( + 1, + reader.CountDefinitionsTotal( + "Container", + kind: "class", + lang: "csharp", + exact: true, + groupPartials: true).Count); + Assert.Equal(0, scans); + + var callable = Assert.Single(reader.SearchSymbols( + ["Run"], + limit: 10, + kind: "function", + lang: "csharp", + exact: true, + groupPartials: true)); + Assert.Equal(2, callable.DefinitionSites); + Assert.Equal(1, scans); + } + + using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath)) + { + var writer = new DbWriter(db.Connection); + writer.ClearHotspotFamilyReady(); + } + + scans = 0; + using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath)) + using (var reader = new DbReader(db)) + { + Assert.Equal( + 2, + reader.SearchSymbols( + ["Run"], + limit: 10, + kind: "function", + lang: "csharp", + exact: true, + groupPartials: true).Count); + Assert.Equal( + 2, + reader.CountSearchSymbolsTotal( + queries: (IReadOnlyList?)null, + kind: "function", + lang: "csharp", + groupPartials: true).Count); + Assert.Equal( + 2, + reader.CountDefinitionsTotal( + "Run", + kind: "function", + lang: "csharp", + exact: true, + groupPartials: true).Count); + Assert.Equal(0, scans); + } + } + finally + { + CSharpCallableTypeKindLookup.ScanForTesting = null; + } + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_CandidateCapIgnoresOrdinarySameNameMethods_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_candidate_cap_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + var source = new System.Text.StringBuilder("namespace Demo;\n"); + for (var index = 0; index <= 4_096; index++) + { + source.Append("public class Ordinary") + .Append(index) + .Append(" { public void Run() { } }\n"); + } + source.Append( + """ + public partial class Container + { + partial void Run(int value); + partial void Run(int value) { } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/CandidateCap.cs", + "csharp", + source.ToString()); + using (var connection = new SqliteConnection($"Data Source={dbPath}")) + { + connection.Open(); + using var command = connection.CreateCommand(); + command.CommandText = """ + UPDATE symbols + SET is_partial_declaration = NULL + WHERE signature LIKE 'partial %' + """; + Assert.Equal(2, command.ExecuteNonQuery()); + } + MarkGraphAndFoldReady(dbPath); + + var candidateScans = new List(); + CSharpCallableTypeKindLookup.CandidateScanForTesting = candidateScans.Add; + try + { + using var db = new DbContext(DbOpenIntent.WriteIndex, dbPath); + using var reader = new DbReader(db); + var results = reader.SearchSymbols( + ["Run"], + limit: 5_000, + kind: "function", + lang: "csharp", + exact: true, + groupPartials: true); + + Assert.Equal(4_098, results.Count); + var grouped = Assert.Single(results.Where(result => result.DefinitionSites == 2)); + Assert.Equal(2, grouped.DefinitionSites); + var scan = Assert.Single(candidateScans); + Assert.False(scan.UsedFullScan); + Assert.Equal(2, scan.CallableCount); + } + finally + { + CSharpCallableTypeKindLookup.CandidateScanForTesting = null; + } + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_RawConnectionReaderRegistersGroupingFunctions_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_raw_reader_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Raw.cs", + "csharp", + """ + namespace Demo; + public partial class RawContainer + { + partial void Execute(int value); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/B.Raw.cs", + "csharp", + """ + namespace Demo; + public partial class RawContainer + { + partial void Execute(int value) { } + } + """); + MarkGraphAndFoldReady(dbPath); + + using var connection = new SqliteConnection($"Data Source={dbPath}"); + connection.Open(); + for (var readerIndex = 0; readerIndex < 2; readerIndex++) + { + using var reader = new DbReader(connection, isReadOnly: true); + var grouped = Assert.Single(reader.SearchSymbols( + ["Execute"], + limit: 10, + kind: "function", + lang: "csharp", + exact: true, + groupPartials: true)); + Assert.Equal(2, grouped.DefinitionSites); + Assert.Equal("implementation_body", grouped.RepresentativeReason); + } + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [ProductionRuntimeFact] + public void PartialCanonicalRepresentative_IndexPersistsHookRebuiltFamilyScope_Issue4914() + { + var projectRoot = TestProjectHelper.CreateExecutableExtensionTestProject( + "cdidx_partial_hook_family_scope_issue4914"); + lock (TestConsoleLock.Gate) + { + using var env = EnvironmentVariableScope.Capture( + PostExtractionHookRunner.HooksDirectoryEnvironmentVariable, + CodeIndex.HookIsolationFixture.HookIsolationFixtureEnvironment + .MutateCSharpPartialFamily); + try + { + var hooksDir = Path.Combine(projectRoot, "hooks"); + Directory.CreateDirectory(hooksDir); + File.Copy( + typeof(CodeIndex.HookIsolationFixture.CSharpPartialFamilyMutationPostExtractionHook) + .Assembly.Location, + Path.Combine(hooksDir, "CodeIndex.HookIsolationFixture.dll")); + env.Set(PostExtractionHookRunner.HooksDirectoryEnvironmentVariable, hooksDir); + env.Set( + CodeIndex.HookIsolationFixture.HookIsolationFixtureEnvironment + .MutateCSharpPartialFamily, + "1"); + File.WriteAllText( + Path.Combine( + projectRoot, + CodeIndex.HookIsolationFixture.HookIsolationFixtureEnvironment + .RemoveCSharpStaticInterfaceMemberMarkerFileName), + string.Empty); + var sourcePath = Path.Combine(projectRoot, "src", "App.cs"); + Directory.CreateDirectory(Path.GetDirectoryName(sourcePath)!); + File.WriteAllText( + sourcePath, + """ + partial class HookContainer + { + [Obsolete] partial void HookPartial(); + } + """); + + Assert.Equal( + CommandExitCodes.Success, + IndexCommandRunner.Run([projectRoot, "--json", "--quiet"], _jsonOptions)); + AssertPersistedFamily(); + + File.AppendAllText(sourcePath, "\n// update\n"); + File.SetLastWriteTimeUtc(sourcePath, DateTime.UtcNow.AddSeconds(2)); + Assert.Equal( + CommandExitCodes.Success, + IndexCommandRunner.Run( + [projectRoot, "--files", "src/App.cs", "--json", "--quiet"], + _jsonOptions)); + AssertPersistedFamily(); + + void AssertPersistedFamily() + { + var dbPath = Path.Combine(projectRoot, ".cdidx", "codeindex.db"); + using var connection = new SqliteConnection($"Data Source={dbPath}"); + connection.Open(); + using var command = connection.CreateCommand(); + command.CommandText = """ + SELECT name, family_key + FROM symbols + WHERE name IN ( + 'HookContainerRenamed', + 'HookOrdinary', + 'HookAddedPartial') + ORDER BY name + """; + var families = new Dictionary(StringComparer.Ordinal); + using var rows = command.ExecuteReader(); + while (rows.Read()) + families.Add(rows.GetString(0), rows.GetString(1)); + + Assert.Equal(3, families.Count); + var family = families["HookContainerRenamed"]; + Assert.EndsWith( + "|file-local:src/App.cs\u001f+HookContainerRenamed`1", + family, + StringComparison.Ordinal); + Assert.Equal(family, families["HookOrdinary"]); + Assert.Equal(family, families["HookAddedPartial"]); + } + } + finally + { + SqliteConnection.ClearAllPools(); + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + } + + [Fact] + public void Impact_UsesCallableIdentifierColumnForTestMethod_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_test_method_column_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/ResultTests.cs", + "csharp", + """ + public class Result { } + public class ResultTests + { + [Fact] + public Result Result() => new(); + } + """); + MarkGraphAndFoldReady(dbPath); + + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunImpact( + ["Result", "--db", dbPath, "--json", "--lang", "csharp", "--max-hops", "0", "--limit", "10"], + _jsonOptions)); + using var document = ParseJsonOutput(stdout); + var testMethod = Assert.Single( + document.RootElement + .GetProperty("definitions") + .EnumerateArray() + .Where(definition => definition.GetProperty("kind").GetString() == "test.method")); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + Assert.Equal(18, testMethod.GetProperty("start_column").GetInt32()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void Impact_QualifiedPartialMethodTraversesDeclarationAndImplementationIds_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_qualified_impact_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Container.cs", + "csharp", + """ + namespace Demo; + public partial class Container + { + partial void M(); + public void Invoke() { M(); } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/B.Container.cs", + "csharp", + """ + namespace Demo; + public partial class Container + { + partial void M() { } + } + """); + MarkGraphAndFoldReady(dbPath); + using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath)) + { + var writer = new DbWriter(db.Connection); + writer.MarkReferenceIdentityContractReady(); + } + + long declarationSymbolId; + using (var connection = new SqliteConnection($"Data Source={dbPath}")) + { + connection.Open(); + using var command = connection.CreateCommand(); + command.CommandText = """ + SELECT target.id, reference.target_symbol_id + FROM symbol_references reference + JOIN files source_file ON source_file.id = reference.file_id + JOIN symbols target ON target.name = reference.symbol_name + JOIN files target_file ON target_file.id = target.file_id + WHERE source_file.path = 'src/A.Container.cs' + AND target_file.path = 'src/A.Container.cs' + AND reference.symbol_name = 'M' + AND reference.reference_kind = 'call' + LIMIT 1 + """; + using var row = command.ExecuteReader(); + Assert.True(row.Read()); + declarationSymbolId = row.GetInt64(0); + Assert.Equal(declarationSymbolId, row.GetInt64(1)); + } + + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunImpact( + ["Container.M", "--db", dbPath, "--json", "--lang", "csharp", "--max-hops", "1", "--limit", "10"], + _jsonOptions)); + using var document = ParseJsonOutput(stdout); + var impact = document.RootElement; + var caller = Assert.Single(impact.GetProperty("callers").EnumerateArray()); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + Assert.Equal("src/B.Container.cs", impact.GetProperty("definitions")[0].GetProperty("path").GetString()); + Assert.Equal("Invoke", caller.GetProperty("caller_name").GetString()); + Assert.Equal(declarationSymbolId, caller.GetProperty("callee_symbol_id").GetInt64()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void Impact_TruncatedMixedCallableFamilyTraversesTestMethodMember_Issue4914Review() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_mixed_kind_impact_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + for (var index = 0; index < 49; index++) + { + TestProjectHelper.InsertIndexedFile( + dbPath, + $"src/{index:D2}.Container.cs", + "csharp", + """ + namespace Demo; + public partial class Container + { + partial void M(); + } + """); + } + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/49.Container.cs", + "csharp", + """ + namespace Demo; + public partial class Container + { + [Fact] + partial void M(); + public void InvokeTarget() { M(); } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/50.Container.cs", + "csharp", + """ + namespace Demo; + public partial class Container + { + partial void M() { } + } + """); + MarkGraphAndFoldReady(dbPath); + + long targetSymbolId; + using (var connection = new SqliteConnection($"Data Source={dbPath}")) + { + connection.Open(); + using var command = connection.CreateCommand(); + command.CommandText = """ + SELECT symbol.id + FROM symbols symbol + JOIN files file ON file.id = symbol.file_id + WHERE file.path = 'src/49.Container.cs' + AND symbol.name = 'M' + AND symbol.kind = 'test.method'; + """; + targetSymbolId = Assert.IsType(command.ExecuteScalar()); + + command.CommandText = """ + UPDATE symbol_references + SET target_symbol_id = @targetSymbolId, + resolution_state = 'resolved', + resolution_candidate_count = 1 + WHERE file_id = (SELECT id FROM files WHERE path = 'src/49.Container.cs') + AND symbol_name = 'M' + AND reference_kind = 'call'; + """; + command.Parameters.AddWithValue("@targetSymbolId", targetSymbolId); + Assert.Equal(1, command.ExecuteNonQuery()); + } + using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath)) + { + var writer = new DbWriter(db.Connection); + writer.MarkReferenceIdentityContractReady(); + } + + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunImpact( + ["Container.M", "--db", dbPath, "--json", "--lang", "csharp", "--max-hops", "1", "--limit", "10"], + _jsonOptions)); + using var document = ParseJsonOutput(stdout); + var impact = document.RootElement; + var definition = Assert.Single(impact.GetProperty("definitions").EnumerateArray()); + var caller = Assert.Single(impact.GetProperty("callers").EnumerateArray()); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + Assert.Equal(51, definition.GetProperty("definition_sites").GetInt32()); + Assert.True(definition.GetProperty("family_members_truncated").GetBoolean()); + Assert.Equal("InvokeTarget", caller.GetProperty("caller_name").GetString()); + Assert.Equal(targetSymbolId, caller.GetProperty("callee_symbol_id").GetInt64()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_DistinguishesNestedTypeNamesAndArities_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_nested_identity_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + foreach (var path in new[] { "src/A.Host.cs", "src/B.Host.cs" }) + { + TestProjectHelper.InsertIndexedFile( + dbPath, + path, + "csharp", + """ + namespace Demo; + public partial class Host + { + public partial class Child { } + public partial class Sibling { } + public partial class Nested { } + public partial class Nested { } + } + """); + } + MarkGraphAndFoldReady(dbPath); + + var child = RunGroupedSymbol(dbPath, "Child", "class"); + var sibling = RunGroupedSymbol(dbPath, "Sibling", "class"); + Assert.Equal(2, child.GetProperty("definition_sites").GetInt32()); + Assert.Equal(2, sibling.GetProperty("definition_sites").GetInt32()); + Assert.NotEqual( + child.GetProperty("partial_family_id").GetString(), + sibling.GetProperty("partial_family_id").GetString()); + + var (symbolsExitCode, symbolsStdout, symbolsStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["Nested", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var symbolsDocument = ParseJsonOutput(symbolsStdout); + var nestedFamilies = symbolsDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, symbolsExitCode); + Assert.Equal(string.Empty, symbolsStderr); + Assert.Equal(2, nestedFamilies.Count); + Assert.All(nestedFamilies, family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + Assert.Equal( + 2, + nestedFamilies + .Select(family => family.GetProperty("partial_family_id").GetString()) + .Distinct(StringComparer.Ordinal) + .Count()); + + var (gotoExitCode, gotoStdout, gotoStderr) = CaptureConsole(() => QueryCommandRunner.RunGoto( + ["Nested", "--db", dbPath, "--exact-name", "--lang", "csharp", "--kind", "class"], + _jsonOptions)); + + Assert.Equal(CommandExitCodes.UsageError, gotoExitCode); + Assert.Equal(string.Empty, gotoStdout); + Assert.Contains("goto found 2 matching definitions", gotoStderr, StringComparison.Ordinal); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_UsesPlainRecordDeclarationArityAfterAttributes_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_record_attribute_arity_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Item.cs", + "csharp", + """ + using System; + namespace Demo; + public sealed class MarkerAttribute : Attribute + { + public MarkerAttribute(Type type) { } + } + public class Item { } + [Marker(typeof(Item))] public partial record Item; + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/B.Item.cs", + "csharp", + """ + namespace Demo; + public partial record class Item; + """); + MarkGraphAndFoldReady(dbPath); + + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["Item", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var document = ParseJsonOutput(stdout); + var symbols = document.RootElement.EnumerateArray().ToList(); + var recordFamily = Assert.Single( + symbols.Where(symbol => symbol.TryGetProperty("definition_sites", out _))); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + Assert.Equal(2, symbols.Count); + Assert.Equal(2, recordFamily.GetProperty("definition_sites").GetInt32()); + Assert.Equal(2, recordFamily.GetProperty("family_members").GetArrayLength()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_IgnoresRecordLikeAttributesAndEscapedTypeNames_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_record_token_boundaries_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Records.cs", + "csharp", + """ + using System; + namespace Demo; + public class recordAttribute : Attribute { } + [record] public partial record Collision; + public partial class @record where T : @record { } + public partial record class @class; + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/B.Records.cs", + "csharp", + """ + namespace Demo; + public partial record Collision; + public partial class @record where T : @record { } + public partial record class @class; + """); + MarkGraphAndFoldReady(dbPath); + + var collision = RunGroupedSymbol(dbPath, "Collision", "class"); + Assert.Equal(2, collision.GetProperty("definition_sites").GetInt32()); + Assert.Equal("src/A.Records.cs", collision.GetProperty("path").GetString()); + Assert.Equal(42, collision.GetProperty("start_column").GetInt32()); + Assert.Collection( + collision.GetProperty("family_members").EnumerateArray().OrderBy( + member => member.GetProperty("path").GetString(), + StringComparer.Ordinal), + member => Assert.Equal(42, member.GetProperty("start_column").GetInt32()), + member => Assert.Equal(22, member.GetProperty("start_column").GetInt32())); + + var escapedRecord = RunGroupedSymbol(dbPath, "record", "class"); + Assert.Equal(2, escapedRecord.GetProperty("definition_sites").GetInt32()); + Assert.Equal(22, escapedRecord.GetProperty("start_column").GetInt32()); + Assert.All( + escapedRecord.GetProperty("family_members").EnumerateArray(), + member => Assert.Equal(22, member.GetProperty("start_column").GetInt32())); + + var escapedRecordClass = RunGroupedSymbol(dbPath, "class", "class"); + Assert.Equal(2, escapedRecordClass.GetProperty("definition_sites").GetInt32()); + Assert.Equal(29, escapedRecordClass.GetProperty("start_column").GetInt32()); + Assert.All( + escapedRecordClass.GetProperty("family_members").EnumerateArray(), + member => Assert.Equal(29, member.GetProperty("start_column").GetInt32())); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCallableGrouping_PreservesPipeInUnixProjectScope_Issue4914() + { + if (OperatingSystem.IsWindows()) + return; + + Assert.Equal( + "Proj%257C%7COne%1F", + SymbolExtractor.EncodeFamilyScopeKey("Proj%7C|One\u001f")); + + var csharpScopeProbe = new SymbolRecord { FamilyKey = "N.Host" }; + SymbolExtractor.ApplyFamilyScope([csharpScopeProbe], "Proj|One", "csharp"); + Assert.Equal("Proj%7COne|N.Host", csharpScopeProbe.FamilyKey); + + var legacyScopeProbe = new SymbolRecord { FamilyKey = "N.Host" }; + SymbolExtractor.ApplyFamilyScope([legacyScopeProbe], "Proj|One", "visualbasic"); + Assert.Equal("Proj|One|N.Host", legacyScopeProbe.FamilyKey); + + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_pipe_scope_issue4914"); + try + { + TestProjectHelper.WriteTextFile( + projectRoot, + "Proj|One/Test.csproj", + "net8.0"); + TestProjectHelper.WriteTextFile( + projectRoot, + "Proj|One/Test.cs", + """ + #nullable enable + namespace N; + partial class Host { partial void M(S value); } + partial class Host { partial void M(S? value) { } } + """); + TestProjectHelper.WriteTextFile( + projectRoot, + "Proj|One/S.cs", + """ + namespace N; + class S { } + """); + + var (indexExitCode, _, indexStderr) = CaptureConsole(() => IndexCommandRunner.Run( + [projectRoot, "--json", "--quiet"], + _jsonOptions)); + Assert.Equal(CommandExitCodes.Success, indexExitCode); + Assert.Equal(string.Empty, indexStderr); + + var dbPath = Path.Combine(projectRoot, ".cdidx", "codeindex.db"); + var grouped = RunGroupedSymbol(dbPath, "M", "function"); + Assert.Equal(2, grouped.GetProperty("definition_sites").GetInt32()); + Assert.Equal("implementation_body", grouped.GetProperty("representative_reason").GetString()); + + using var connection = new SqliteConnection($"Data Source={dbPath}"); + connection.Open(); + using var command = connection.CreateCommand(); + command.CommandText = "SELECT DISTINCT family_key FROM symbols WHERE name = 'M'"; + Assert.Equal("Proj%7COne|N+Host", Assert.IsType(command.ExecuteScalar())); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_PreservesLiteralBackslashInUnixFileLocalIdentity_Issue4914Review() + { + if (OperatingSystem.IsWindows()) + return; + + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_file_local_backslash_issue4914"); + try + { + TestProjectHelper.WriteTextFile( + projectRoot, + "Test.csproj", + "net8.0"); + const string content = """ + namespace N; + file partial class Hidden { } + """; + TestProjectHelper.WriteTextFile(projectRoot, "A/B.cs", content); + TestProjectHelper.WriteTextFile(projectRoot, "A\\B.cs", content); + + var (indexExitCode, _, indexStderr) = CaptureConsole(() => IndexCommandRunner.Run( + [projectRoot, "--json", "--quiet"], + _jsonOptions)); + Assert.Equal(CommandExitCodes.Success, indexExitCode); + Assert.Equal(string.Empty, indexStderr); + + var dbPath = Path.Combine(projectRoot, ".cdidx", "codeindex.db"); + var (symbolsExitCode, symbolsStdout, symbolsStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["Hidden", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var symbolsDocument = ParseJsonOutput(symbolsStdout); + var rows = symbolsDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, symbolsExitCode); + Assert.Equal(string.Empty, symbolsStderr); + Assert.Equal(2, rows.Count); + Assert.All(rows, row => Assert.False(row.TryGetProperty("definition_sites", out _))); + + using var connection = new SqliteConnection($"Data Source={dbPath}"); + connection.Open(); + using var command = connection.CreateCommand(); + command.CommandText = "SELECT COUNT(DISTINCT family_key) FROM symbols WHERE name = 'Hidden'"; + Assert.Equal(2L, Assert.IsType(command.ExecuteScalar())); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_PreservesLiteralBackslashInUnixProjectScope_Issue4914Review() + { + if (OperatingSystem.IsWindows()) + return; + + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_project_backslash_issue4914"); + try + { + foreach (var projectDirectory in new[] { "A\\B", "A/B" }) + { + TestProjectHelper.WriteTextFile( + projectRoot, + $"{projectDirectory}/Test.csproj", + "net8.0enable"); + TestProjectHelper.WriteTextFile( + projectRoot, + $"{projectDirectory}/One.cs", + """ + namespace Demo; + public partial class Host + { + partial void M(Node? value); + } + """); + TestProjectHelper.WriteTextFile( + projectRoot, + $"{projectDirectory}/Two.cs", + """ + namespace Demo; + public partial class Host + { + partial void M(Node value) { } + } + """); + } + TestProjectHelper.WriteTextFile( + projectRoot, + "A\\B/Types.cs", + """ + namespace Demo; + public class Node { } + """); + TestProjectHelper.WriteTextFile( + projectRoot, + "A/B/Types.cs", + """ + namespace Demo; + public struct Node { } + """); + + var (indexExitCode, _, indexStderr) = CaptureConsole(() => IndexCommandRunner.Run( + [projectRoot, "--json", "--quiet"], + _jsonOptions)); + Assert.Equal(CommandExitCodes.Success, indexExitCode); + Assert.Equal(string.Empty, indexStderr); + + var dbPath = Path.Combine(projectRoot, ".cdidx", "codeindex.db"); + var (symbolsExitCode, symbolsStdout, symbolsStderr) = CaptureConsole(() => + QueryCommandRunner.RunSymbols( + ["M", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var symbolsDocument = ParseJsonOutput(symbolsStdout); + var rows = symbolsDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, symbolsExitCode); + Assert.Equal(string.Empty, symbolsStderr); + Assert.Equal(3, rows.Count); + var grouped = Assert.Single(rows, row => row.TryGetProperty("definition_sites", out _)); + Assert.Equal(2, grouped.GetProperty("definition_sites").GetInt32()); + var groupedPaths = grouped.GetProperty("family_members") + .EnumerateArray() + .Select(member => member.GetProperty("path").GetString()) + .ToList(); + Assert.All(groupedPaths, path => Assert.StartsWith("A\\B/", path, StringComparison.Ordinal)); + Assert.Contains("A\\B/One.cs", groupedPaths); + Assert.Contains("A\\B/Two.cs", groupedPaths); + Assert.All( + rows.Where(row => !row.TryGetProperty("definition_sites", out _)), + row => Assert.StartsWith("A/B/", row.GetProperty("path").GetString(), StringComparison.Ordinal)); + + using var connection = new SqliteConnection($"Data Source={dbPath}"); + connection.Open(); + using var command = connection.CreateCommand(); + command.CommandText = "SELECT COUNT(DISTINCT family_key) FROM symbols WHERE name = 'M'"; + Assert.Equal(2L, Assert.IsType(command.ExecuteScalar())); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_PreservesPersistedUnixBackslashProjectScopeAcrossHosts_Issue4914Review() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_persisted_backslash_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + foreach (var projectDirectory in new[] { "A\\B", "A/B" }) + { + TestProjectHelper.InsertIndexedFile( + dbPath, + $"{projectDirectory}/Test.csproj", + "xml", + ""); + TestProjectHelper.InsertIndexedFile( + dbPath, + $"{projectDirectory}/One.cs", + "csharp", + """ + #nullable enable + namespace Demo; + public partial class Host + { + partial void M(Node? value); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + $"{projectDirectory}/Two.cs", + "csharp", + """ + #nullable enable + namespace Demo; + public partial class Host + { + partial void M(Node value) { } + } + """); + } + TestProjectHelper.InsertIndexedFile( + dbPath, + "A\\B/Types.cs", + "csharp", + """ + namespace Demo; + public class Node { } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "A/B/Types.cs", + "csharp", + """ + namespace Demo; + public struct Node { } + """); + + using (var connection = new SqliteConnection($"Data Source={dbPath}")) + { + connection.Open(); + foreach (var projectDirectory in new[] { "A\\B", "A/B" }) + { + using var command = connection.CreateCommand(); + command.CommandText = """ + UPDATE symbols + SET family_key = @scope || '|' || family_key + WHERE family_key IS NOT NULL + AND file_id IN ( + SELECT id + FROM files + WHERE path = @one OR path = @two OR path = @types) + """; + SqliteCommandPolicy.Add( + command, + "@scope", + SymbolExtractor.EncodeFamilyScopeKey(projectDirectory)); + SqliteCommandPolicy.Add(command, "@one", $"{projectDirectory}/One.cs"); + SqliteCommandPolicy.Add(command, "@two", $"{projectDirectory}/Two.cs"); + SqliteCommandPolicy.Add(command, "@types", $"{projectDirectory}/Types.cs"); + Assert.True(command.ExecuteNonQuery() > 0); + } + } + MarkGraphAndFoldReady(dbPath); + + var (symbolsExitCode, symbolsStdout, symbolsStderr) = CaptureConsole(() => + QueryCommandRunner.RunSymbols( + ["M", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--limit", "10"], + _jsonOptions)); + using var symbolsDocument = ParseJsonOutput(symbolsStdout); + var rows = symbolsDocument.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, symbolsExitCode); + Assert.Equal(string.Empty, symbolsStderr); + Assert.Equal(3, rows.Count); + var grouped = Assert.Single(rows, row => row.TryGetProperty("definition_sites", out _)); + Assert.Equal(2, grouped.GetProperty("definition_sites").GetInt32()); + Assert.All( + grouped.GetProperty("family_members").EnumerateArray(), + member => Assert.StartsWith( + "A\\B/", + member.GetProperty("path").GetString(), + StringComparison.Ordinal)); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCallableGrouping_NormalizesOptionalManagedFunctionPointerConvention_Issue4914Review() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_managed_function_pointer_issue4914"); + try + { + TestProjectHelper.WriteTextFile( + projectRoot, + "Test.csproj", + "net8.0true"); + TestProjectHelper.WriteTextFile( + projectRoot, + "Test.cs", + """ + unsafe partial class Container + { + partial void M(delegate* managed callback); + partial void M(delegate* callback) { } + } + """); + + var (indexExitCode, _, indexStderr) = CaptureConsole(() => IndexCommandRunner.Run( + [projectRoot, "--json", "--quiet"], + _jsonOptions)); + Assert.Equal(CommandExitCodes.Success, indexExitCode); + Assert.Equal(string.Empty, indexStderr); + + var dbPath = Path.Combine(projectRoot, ".cdidx", "codeindex.db"); + var grouped = RunGroupedSymbol(dbPath, "M", "function"); + Assert.Equal(2, grouped.GetProperty("definition_sites").GetInt32()); + Assert.Equal("implementation_body", grouped.GetProperty("representative_reason").GetString()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCallableGrouping_PipeInFileLocalPathDoesNotLeakTypeKind_Issue4914Review() + { + if (OperatingSystem.IsWindows()) + return; + + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_file_local_pipe_issue4914"); + try + { + TestProjectHelper.WriteTextFile( + projectRoot, + "Test.csproj", + "net8.0enable"); + TestProjectHelper.WriteTextFile( + projectRoot, + "src/A|B.cs", + """ + namespace N; + file partial class Outer { public struct Hidden { } } + """); + TestProjectHelper.WriteTextFile( + projectRoot, + "src/Global.cs", + """ + namespace N; + class Outer { public class Hidden { } } + """); + TestProjectHelper.WriteTextFile( + projectRoot, + "src/Host.cs", + """ + #nullable enable + namespace N; + partial class Host { partial void M(Outer.Hidden value); } + partial class Host { partial void M(Outer.Hidden? value) { } } + """); + + var (indexExitCode, _, indexStderr) = CaptureConsole(() => IndexCommandRunner.Run( + [projectRoot, "--json", "--quiet"], + _jsonOptions)); + Assert.Equal(CommandExitCodes.Success, indexExitCode); + Assert.Equal(string.Empty, indexStderr); + + var dbPath = Path.Combine(projectRoot, ".cdidx", "codeindex.db"); + var grouped = RunGroupedSymbol(dbPath, "M", "function"); + Assert.Equal(2, grouped.GetProperty("definition_sites").GetInt32()); + Assert.Equal("implementation_body", grouped.GetProperty("representative_reason").GetString()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_PropagatesFileLocalScopeToNestedFamilies_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_nested_file_local_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/A.Host.cs", + "csharp", + """ + namespace Demo; + file partial class Host { } + partial class Host { public partial class Child { } } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/B.Host.cs", + "csharp", + """ + namespace Demo; + partial class Host { public partial class Child { } } + """); + MarkGraphAndFoldReady(dbPath); + + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["Child", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var document = ParseJsonOutput(stdout); + var families = document.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + Assert.Equal(2, families.Count); + Assert.All(families, family => Assert.False(family.TryGetProperty("definition_sites", out _))); + Assert.Equal( + ["src/A.Host.cs", "src/B.Host.cs"], + families.Select(family => family.GetProperty("path").GetString()).Order().ToArray()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_IgnoresConstraintTextInsideImplementationBody_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_constraint_body_text_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Container.cs", + "csharp", + """ + #nullable enable + namespace Demo; + public partial class Container + { + partial void M(T? value); + partial void M(T? value) { var text = "where T : struct"; } + } + """); + MarkGraphAndFoldReady(dbPath); + + var grouped = RunGroupedSymbol(dbPath, "M", "function"); + + Assert.Equal(2, grouped.GetProperty("definition_sites").GetInt32()); + Assert.Equal("implementation_body", grouped.GetProperty("representative_reason").GetString()); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void M(T? value);", + "M", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void M(T? value) { var text = \"where T : struct\"; }", + "M", + "void")); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCallableGrouping_UsesFoldedCandidateNames_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_folded_candidate_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Container.cs", + "csharp", + """ + #nullable enable + namespace Demo; + public class Node { } + public partial class Container + { + partial void MÉTHODE(Node? value); + partial void MÉTHODE(Node value) { } + } + """); + MarkGraphAndFoldReady(dbPath); + + var grouped = RunGroupedSymbol(dbPath, "méthode", "function"); + + Assert.Equal(2, grouped.GetProperty("definition_sites").GetInt32()); + Assert.Equal("implementation_body", grouped.GetProperty("representative_reason").GetString()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCallableGrouping_NormalizesTupleAndOptionalDefaultIdentities_Issue4914() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_nullable_tuple_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Container.cs", + "csharp", + """ + #nullable enable + namespace Demo; + public class GenericDefaults + { + public const int Value = 1; + } + public partial class Container + { + partial void M((int, int)? value); + partial void M(global::System.Nullable<(int, int)> value) { } + partial void TupleEquivalent((int, string) value); + partial void TupleEquivalent(global::System.ValueTuple value) { } + partial void NestedTupleEquivalent((int, (string, bool)) value); + partial void NestedTupleEquivalent(global::System.ValueTuple> value) { } + partial void NullableTupleEquivalent((int, (string, bool))? value); + partial void NullableTupleEquivalent(global::System.Nullable>> value) { } + partial void NullableFrameworkTuple((int, int)? value); + partial void NullableFrameworkTuple(global::System.ValueTuple? value) { } + partial void LongTupleEquivalent((int, int, int, int, int, int, int, int) value); + partial void LongTupleEquivalent(global::System.ValueTuple> value) { } + partial void NamedTuple((int T, string S) value); + partial void NamedTuple((int T, string S) value) { } + partial void GenericDefault(int value = GenericDefaults.Value, int other = 0); + partial void GenericDefault(int value, int other) { } + partial void CommentedGenericDefault(int value = GenericDefaults */ int, string> /* > */ .Value, int other = 0); + partial void CommentedGenericDefault(int value, int other) { } + partial void RelationalDefault(bool value = 1 < 2, int other = 0); + partial void RelationalDefault(bool value, int other) { } + } + """); + MarkGraphAndFoldReady(dbPath); + + foreach (var name in new[] + { + "M", + "TupleEquivalent", + "NestedTupleEquivalent", + "NullableTupleEquivalent", + "NullableFrameworkTuple", + "LongTupleEquivalent", + "NamedTuple", + "GenericDefault", + "CommentedGenericDefault", + "RelationalDefault", + }) + { + var grouped = RunGroupedSymbol(dbPath, name, "function"); + Assert.Equal(2, grouped.GetProperty("definition_sites").GetInt32()); + Assert.Equal("implementation_body", grouped.GetProperty("representative_reason").GetString()); + } + + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void M((int, int)? value);", + "M", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void M(global::System.Nullable<(int, int)> value) { }", + "M", + "void")); + Assert.NotEqual( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void NamedTuple((int x, string y) value);", + "NamedTuple", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void NamedTuple(global::System.ValueTuple value) { }", + "NamedTuple", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void GenericDefault(int value = GenericDefaults.Value, int other = 0);", + "GenericDefault", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void GenericDefault(int value, int other) { }", + "GenericDefault", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void CommentedGenericDefault(int value = GenericDefaults */ int, string> /* > */ .Value, int other = 0);", + "CommentedGenericDefault", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void CommentedGenericDefault(int value, int other) { }", + "CommentedGenericDefault", + "void")); + Assert.Equal( + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void RelationalDefault(bool value = 1 < 2, int other = 0);", + "RelationalDefault", + "void"), + LogicalPartialSymbolGrouper.BuildCallableIdentity( + "partial void RelationalDefault(bool value, int other) { }", + "RelationalDefault", + "void")); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_DistinguishesNamespaceAndNestedTypeBoundaries_Issue4914Review() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_family_boundary_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Conditional.cs", + "csharp", + """ + #nullable enable + #if FLAT + namespace A.B { public partial class C { } } + #else + namespace A { public partial class B { public partial class C { } } } + #endif + #if ROOT_NAMESPACE + namespace Root { public partial class Leaf { } } + #else + public partial class Root { public partial class Leaf { } } + #endif + #if FLAT_NULLABLE + namespace Kind.A + { + public struct Node { } + public partial class Host + { + partial void M(Node? value); + partial void M(global::System.Nullable value) { } + } + } + #else + namespace Kind + { + public class A + { + public class Node { } + public partial class Host + { + partial void M(Node? value); + partial void M(Node value) { } + } + } + } + #endif + """); + MarkGraphAndFoldReady(dbPath); + + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["C", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var document = ParseJsonOutput(stdout); + var families = document.RootElement.EnumerateArray().ToList(); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + Assert.Equal(2, families.Count); + Assert.All(families, family => Assert.False(family.TryGetProperty("definition_sites", out _))); + + using var connection = new SqliteConnection($"Data Source={dbPath}"); + connection.Open(); + using var command = connection.CreateCommand(); + command.CommandText = "SELECT family_key FROM symbols WHERE name = 'C' ORDER BY line"; + using var reader = command.ExecuteReader(); + var familyKeys = new List(); + while (reader.Read()) + familyKeys.Add(reader.GetString(0)); + Assert.Collection( + familyKeys, + family => Assert.Equal("A.B+C", family), + family => Assert.Equal("A+B+C", family)); + reader.Close(); + + var (rootExitCode, rootStdout, rootStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["Leaf", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "class", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var rootDocument = ParseJsonOutput(rootStdout); + var rootFamilies = rootDocument.RootElement.EnumerateArray().ToList(); + Assert.Equal(CommandExitCodes.Success, rootExitCode); + Assert.Equal(string.Empty, rootStderr); + Assert.Equal(2, rootFamilies.Count); + Assert.All(rootFamilies, family => Assert.False(family.TryGetProperty("definition_sites", out _))); + + using var rootCommand = connection.CreateCommand(); + rootCommand.CommandText = "SELECT family_key FROM symbols WHERE name = 'Leaf' ORDER BY line"; + using var rootReader = rootCommand.ExecuteReader(); + var rootFamilyKeys = new List(); + while (rootReader.Read()) + rootFamilyKeys.Add(rootReader.GetString(0)); + Assert.Collection( + rootFamilyKeys, + family => Assert.Equal("Root+Leaf", family), + family => Assert.Equal("+Root+Leaf", family)); + rootReader.Close(); + + var (callableExitCode, callableStdout, callableStderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["M", "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", "function", "--group-partials", "--include-generated", "--limit", "10"], + _jsonOptions)); + using var callableDocument = ParseJsonOutput(callableStdout); + var callableFamilies = callableDocument.RootElement.EnumerateArray().ToList(); + Assert.Equal(CommandExitCodes.Success, callableExitCode); + Assert.Equal(string.Empty, callableStderr); + Assert.True(callableFamilies.Count == 2, callableStdout); + Assert.All( + callableFamilies, + family => Assert.Equal(2, family.GetProperty("definition_sites").GetInt32())); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void PartialCanonicalRepresentative_UsesIdentifierLineForMultilineCallable_Issue4914Review() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_partial_multiline_identifier_issue4914"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Container.cs", + "csharp", + """ + namespace Demo; + public partial class Container + { + partial void + M(int value); + partial void M(int value) { } + } + """); + MarkGraphAndFoldReady(dbPath); + + var grouped = RunGroupedSymbol(dbPath, "M", "function"); + Assert.Equal(2, grouped.GetProperty("definition_sites").GetInt32()); + Assert.Contains( + grouped.GetProperty("family_members").EnumerateArray(), + member => member.GetProperty("line").GetInt32() == 5 + && member.GetProperty("start_column").GetInt32() == 4); + + using var connection = new SqliteConnection($"Data Source={dbPath}"); + connection.Open(); + using var command = connection.CreateCommand(); + command.CommandText = + "SELECT line, start_line, identifier_start_column FROM symbols WHERE name = 'M' ORDER BY line"; + using var reader = command.ExecuteReader(); + Assert.True(reader.Read()); + Assert.Equal(5, reader.GetInt32(0)); + Assert.Equal(4, reader.GetInt32(1)); + Assert.Equal(4, reader.GetInt32(2)); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + private JsonElement RunGroupedSymbol(string dbPath, string name, string kind) + { + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + [name, "--db", dbPath, "--json=array", "--exact-name", "--lang", "csharp", "--kind", kind, "--group-partials", "--include-generated", "--limit", "1"], + _jsonOptions)); + Assert.False(string.IsNullOrWhiteSpace(stdout), stderr); + using var document = ParseJsonOutput(stdout); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + return Assert.Single(document.RootElement.EnumerateArray().ToList()).Clone(); + } } diff --git a/tests/CodeIndex.Tests/QueryCommandRunnerSymbolTests.cs b/tests/CodeIndex.Tests/QueryCommandRunnerSymbolTests.cs index 875b3c87f..c5fa46ab6 100644 --- a/tests/CodeIndex.Tests/QueryCommandRunnerSymbolTests.cs +++ b/tests/CodeIndex.Tests/QueryCommandRunnerSymbolTests.cs @@ -234,6 +234,7 @@ public partial class McpServer { } """); + MarkGraphAndFoldReady(dbPath); var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunGoto( ["McpServer", "--db", dbPath, "--kind", "class", "--lang", "csharp"], @@ -283,6 +284,7 @@ public partial class Widget public Widget(int value) { } } """); + MarkGraphAndFoldReady(dbPath); var (definitionExit, definitionStdout, definitionStderr) = CaptureConsole(() => QueryCommandRunner.RunDefinition( diff --git a/tests/CodeIndex.Tests/QueryCommandRunnerTests.cs b/tests/CodeIndex.Tests/QueryCommandRunnerTests.cs index a1febd9ad..9a94e3c60 100644 --- a/tests/CodeIndex.Tests/QueryCommandRunnerTests.cs +++ b/tests/CodeIndex.Tests/QueryCommandRunnerTests.cs @@ -9003,6 +9003,7 @@ private static void MarkGraphAndFoldReady(string dbPath) writer.MarkGraphReady(); writer.MarkFoldReady(); writer.MarkCSharpSymbolNameContractReady(); + writer.MarkHotspotFamilyReady("csharp", "test-fixture-csharp-family"); writer.MarkIssuesReady(); } diff --git a/tests/CodeIndex.Tests/ReportCommandRunnerTests.cs b/tests/CodeIndex.Tests/ReportCommandRunnerTests.cs index 2e11f4a2c..c4d36c8a7 100644 --- a/tests/CodeIndex.Tests/ReportCommandRunnerTests.cs +++ b/tests/CodeIndex.Tests/ReportCommandRunnerTests.cs @@ -2039,6 +2039,146 @@ is_metadata_target INTEGER } } + [Theory] + [InlineData("vb", true)] + [InlineData("csharp", false)] + [InlineData("python", true)] + public void Run_WithLegacyGlobalHotspotVersion_UsesIndexedLanguageContract_Issue4914( + string language, + bool expectedReady) + { + var workDir = CreateWorkDir(); + var dbPath = Path.Combine(workDir, "legacy-hotspot-family.db"); + try + { + using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath)) + db.InitializeSchema(); + using (var connection = new SqliteConnection(new SqliteConnectionStringBuilder { DataSource = dbPath }.ConnectionString)) + { + connection.Open(); + using var cmd = connection.CreateCommand(); + cmd.CommandText = $""" + INSERT INTO files (path, lang, size, lines, checksum, modified) + VALUES ('src/Legacy', '{language}', 1, 1, 'legacy', '2026-01-01T00:00:00Z'); + DELETE FROM codeindex_meta + WHERE key = '{DbContext.GetHotspotFamilyVersionMetaKey(language)}'; + INSERT OR REPLACE INTO codeindex_meta (key, value) + VALUES ('{DbContext.HotspotFamilyVersionMetaKey}', '{DbContext.LegacyHotspotFamilyVersion}'); + """; + cmd.ExecuteNonQuery(); + } + SqliteConnection.ClearAllPools(); + + var output = Path.Combine(workDir, "bundle.tgz"); + var (exitCode, _, _) = RunAndCaptureStreams([ + "--output", output, + "--db", dbPath, + "--no-log", + ]); + + Assert.Equal(CommandExitCodes.Success, exitCode); + var entries = ReadTarGzEntries(output); + using var manifest = ReadJsonEntry(entries, "support-manifest.json"); + var readiness = manifest.RootElement.GetProperty("readiness"); + Assert.Equal(expectedReady, readiness.GetProperty("hotspot_family_ready").GetBoolean()); + Assert.Equal( + !expectedReady, + JsonArrayContains(readiness.GetProperty("degraded_fields"), "hotspot_family_ready")); + } + finally + { + SqliteConnection.ClearAllPools(); + TestProjectHelper.DeleteDirectory(workDir); + } + } + + [Theory] + [InlineData(null, false, false)] + [InlineData("incomplete:marker_scan_cap", false, false)] + [InlineData("complete-fingerprint", true, false)] + [InlineData("complete-fingerprint", false, true)] + public void Run_WithPerLanguageHotspotVersion_RequiresCompleteMarkerFingerprint_Issue4914( + string? markerFingerprint, + bool expectedReady, + bool insertIncompletePartialFamily) + { + var workDir = CreateWorkDir(); + var dbPath = Path.Combine(workDir, "per-language-hotspot-family.db"); + try + { + using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath)) + db.InitializeSchema(); + using (var connection = new SqliteConnection(new SqliteConnectionStringBuilder { DataSource = dbPath }.ConnectionString)) + { + connection.Open(); + using var cmd = connection.CreateCommand(); + cmd.CommandText = $""" + INSERT INTO files (path, lang, size, lines, checksum, modified) + VALUES ('src/Current.cs', 'csharp', 1, 1, 'current', '2026-01-01T00:00:00Z'); + INSERT OR REPLACE INTO codeindex_meta (key, value) + VALUES ( + '{DbContext.GetHotspotFamilyVersionMetaKey("csharp")}', + '{DbContext.GetHotspotFamilyVersion("csharp")}'); + INSERT OR REPLACE INTO codeindex_meta (key, value) + VALUES ( + '{DbContext.GetHotspotFamilyMarkerFingerprintMetaKey("csharp")}', + @markerFingerprint); + INSERT INTO symbols ( + file_id, + kind, + name, + line, + start_line, + end_line, + signature, + is_partial_declaration, + family_key) + SELECT + id, + 'class', + 'IncompletePartial', + 1, + 1, + 1, + 'public partial class IncompletePartial', + 1, + NULL + FROM files + WHERE @insertIncompletePartialFamily = 1; + """; + cmd.Parameters.AddWithValue( + "@markerFingerprint", + (object?)markerFingerprint ?? DBNull.Value); + cmd.Parameters.AddWithValue( + "@insertIncompletePartialFamily", + insertIncompletePartialFamily ? 1 : 0); + cmd.ExecuteNonQuery(); + } + SqliteConnection.ClearAllPools(); + + var output = Path.Combine(workDir, "bundle.tgz"); + var (exitCode, _, _) = RunAndCaptureStreams([ + "--output", output, + "--db", dbPath, + "--no-log", + ]); + + Assert.Equal(CommandExitCodes.Success, exitCode); + var entries = ReadTarGzEntries(output); + using var manifest = ReadJsonEntry(entries, "support-manifest.json"); + var readiness = manifest.RootElement.GetProperty("readiness"); + Assert.Equal(expectedReady, readiness.GetProperty("hotspot_family_ready").GetBoolean()); + Assert.Equal( + !expectedReady, + JsonArrayContains(readiness.GetProperty("degraded_fields"), "hotspot_family_ready")); + } + finally + { + SqliteConnection.ClearAllPools(); + TestProjectHelper.DeleteDirectory(workDir); + } + } + private (int ExitCode, string StdOut, string StdErr) RunAndCaptureStreams(string[] args) { lock (TestConsoleLock.Gate) diff --git a/tests/CodeIndex.Tests/SymbolExtractorTests.cs b/tests/CodeIndex.Tests/SymbolExtractorTests.cs index 5014144e5..6004eb52b 100644 --- a/tests/CodeIndex.Tests/SymbolExtractorTests.cs +++ b/tests/CodeIndex.Tests/SymbolExtractorTests.cs @@ -10153,14 +10153,14 @@ public void Extract_CSharp_WrappedNestedPartialTypesKeepRootToLeafFamilyKeys() { // The #535 container-path fix changes `BuildSelfFamilyKey` to receive a root-to-leaf // effective container path. If the old `Reverse()` is left in place, nested partial - // types flip their family key order (`Host.ReviewFixtures.Wrapped`) and no longer - // match the container-qualified-name contract used by hotspot-family grouping. - // Pin a wrapped nested partial-type fixture so both the container path and family key - // stay in canonical root-to-leaf order. Closes #541. + // types flip their arity-aware family key order (for example, starting with `Host`) + // and no longer match the namespace/type-boundary-aware identity used by hotspot-family + // grouping. Pin a wrapped nested partial-type fixture so both the container path and + // family key stay in canonical root-to-leaf order. Closes #541. // #535 の container-path fix 以降、`BuildSelfFamilyKey` は root-to-leaf 順の // effective container path を受け取る。ここで旧 `Reverse()` が残ると、nested partial - // type の family key が `Host.ReviewFixtures.Wrapped` のように逆順化し、 - // hotspot-family grouping が依存する container-qualified-name 契約と食い違う。 + // type の arity-aware family key が `Host` から始まるように逆順化し、 + // hotspot-family grouping が使う namespace/type 境界を区別する identity と食い違う。 // wrapped な nested partial-type の fixture を固定し、container path と family key の // 両方が canonical な root-to-leaf 順を保つことを検証する。Closes #541. var content = string.Join( @@ -10181,11 +10181,11 @@ public void Extract_CSharp_WrappedNestedPartialTypesKeepRootToLeafFamilyKeys() var wrapped = Assert.Single(symbols.Where(s => s.Kind == "class" && s.Name == "Wrapped")); Assert.Equal("ReviewFixtures.Host", wrapped.ContainerQualifiedName); - Assert.Equal("ReviewFixtures.Host.Wrapped", wrapped.FamilyKey); + Assert.Equal("ReviewFixtures+Host+Wrapped`1", wrapped.FamilyKey); var child = Assert.Single(symbols.Where(s => s.Kind == "class" && s.Name == "Child")); Assert.Equal("ReviewFixtures.Host.Wrapped", child.ContainerQualifiedName); - Assert.Equal("ReviewFixtures.Host.Wrapped", child.FamilyKey); + Assert.Equal("ReviewFixtures+Host+Wrapped`1+Child", child.FamilyKey); } [Fact] diff --git a/tests/CodeIndex.Tests/golden/impact.json b/tests/CodeIndex.Tests/golden/impact.json index 836c5dd7e..e7928c197 100644 --- a/tests/CodeIndex.Tests/golden/impact.json +++ b/tests/CodeIndex.Tests/golden/impact.json @@ -43,6 +43,7 @@ "name": "FolderDiffService", "line": 1, "start_line": 1, + "start_column": 13, "end_line": 4, "body_start_line": 2, "body_end_line": 4,