Protobuf symbol extraction: `enum` mis-kinded as `class`, `package` / `oneof` / `extend` never captured (5,815 enum + 6,932 package + 2,572 oneof misses on googleapis)

[Widthdom]

Summary

The Protobuf pattern set in SymbolExtractor.cs has four coverage problems on a real .proto corpus:

1. enum X is indexed as class kind, not enum kind. Protobuf has first-class enum declarations (e.g. enum Severity { ... }) but the extractor maps them to class. On googleapis (the canonical Protobuf corpus, 6,800 .proto files), symbols --kind enum --lang protobuf returns 0 rows despite 5,815 enum declarations in source.
2. package X.Y.Z is not captured at all. The package declaration is the canonical namespace concept in Protobuf (it determines the generated language's namespace/module path) and there is no regex for it. 6,932 package X.Y.Z lines on googleapis → 0 namespace symbols.
3. oneof X { ... } declarations inside messages are never captured. oneof is the union-type primitive in Protobuf (used heavily in google.protobuf.Value, polymorphic API designs, etc.). 2,572 chunk-hits on googleapis → 0 symbols.
4. extend X { ... } declarations are never captured. Used for proto2 message extensions and for FileOptions / FieldOptions / etc. in many Google APIs. 52 chunk-hits on googleapis → 0 symbols.

Same family as #155 / #165 / #166 / #167 / #168 / #169 / #171 — per-line regex list missing common language constructs.

Repro

curl -fsSL https://raw.githubusercontent.com/Widthdom/CodeIndex/main/install.sh | bash
CDIDX=/root/.local/bin/cdidx

mkdir -p /tmp/srcs && cd /tmp/srcs
curl -fsSL -o googleapis.tar.gz https://codeload.github.com/googleapis/googleapis/tar.gz/refs/heads/master
tar xzf googleapis.tar.gz

"$CDIDX" /tmp/srcs/googleapis-master/google --db /tmp/proto.db
# → 6,800 .proto files, 90,382 symbols
#   Kinds:
#     class    54541
#     import   23138
#     function 12703
#     (no enum, no namespace)

1. enum mis-kinded as class

"$CDIDX" symbols --db /tmp/proto.db --lang protobuf --kind enum --limit 3
# → No symbols found.
#   Hint: no 'enum' symbols in the index. Indexed kinds: class, function, import

"$CDIDX" symbols "Severity" --db /tmp/proto.db --lang protobuf --exact --limit 3
# class      Severity    api/expr/conformance/v1alpha1/conformance_service.proto:160-173
# class      Severity    cloud/apihub/v1/common_fields.proto:64-79
# class      Severity    cloud/bigquery/migration/v2/translation_suggestion.proto:32-45

Source at conformance_service.proto:160:

message IssueDetails {
  // Severities of issues.
  enum Severity {
    SEVERITY_UNSPECIFIED = 0;
    DEPRECATION = 1;
    ...
  }
}

Severity is a Protobuf enum — the indexer's vocabulary already has an enum kind (used for C / C++ / C# / Java / Rust / Swift / Kotlin / etc.) but the protobuf pattern hard-codes class.

2. package not captured

"$CDIDX" symbols --db /tmp/proto.db --lang protobuf --kind namespace --limit 3
# → No symbols found.
#   Hint: no 'namespace' symbols in the index.

Despite essentially every .proto file starting with a package line:

grep -c '^package ' /tmp/srcs/googleapis-master/google/api/annotations.proto
# 1

Source like package google.api; — the canonical namespace concept in Protobuf — never enters the symbol index.

3. oneof not captured

grep -RcE '^\s+oneof\s+\w+\s*\{' /tmp/srcs/googleapis-master/google | head -3
# 2,572 chunk-hits across the corpus

"$CDIDX" symbols "kind" --db /tmp/proto.db --lang protobuf --exact --limit 3
# (returns no `oneof kind` rows; only message/enum/service rows that share the name)

Spot-check on google/protobuf/struct.proto:

message Value {
  oneof kind {
    NullValue null_value = 1;
    double number_value = 2;
    string string_value = 3;
    bool bool_value = 4;
    Struct struct_value = 5;
    ListValue list_value = 6;
  }
}

oneof kind is a structural element of Value. AI consumers asking "what are the variants of Value?" get only the field references in the chunk text, not a structured oneof kind symbol.

4. extend not captured

grep -RcE '^extend\s+\w' /tmp/srcs/googleapis-master/google | head -3
# 52 chunk-hits

# Examples (proto2 / API-extension-pattern files):
# extend google.protobuf.MessageOptions { ... }
# extend google.protobuf.FieldOptions   { ... }

Especially common in API metadata files (google/api/annotations.proto, google/api/client.proto, etc.) where the extension pattern attaches custom options to FieldOptions / MethodOptions.

Root cause

src/CodeIndex/Indexer/SymbolExtractor.cs:338-345:

["protobuf"] =
[
    new("class",    new Regex(@"^\s*message\s+(?<name>\w+)", RegexOptions.Compiled), BodyStyle.Brace),
    new("class",    new Regex(@"^\s*enum\s+(?<name>\w+)", RegexOptions.Compiled), BodyStyle.Brace),    // ← should be "enum"
    new("class",    new Regex(@"^\s*service\s+(?<name>\w+)", RegexOptions.Compiled), BodyStyle.Brace),
    new("function", new Regex(@"^\s*rpc\s+(?<name>\w+)", RegexOptions.Compiled), BodyStyle.None),
    new("import",   new Regex(@"^\s*import\s+""(?<name>[^""]+)"";", RegexOptions.Compiled), BodyStyle.None),
],

• Line 341 hard-codes class for enum. Should be enum.
• No package regex.
• No oneof regex.
• No extend regex.
• Optional polish: service X could be interface instead of class — services in Protobuf are RPC interface contracts, semantically closer to interface than class. Lower priority than the four primary fixes.

Why it matters

• symbols --kind enum --lang protobuf is empty across every .proto corpus. AI consumers using kind-based exploration get an incorrect picture: it looks like Protobuf has no enums at all.
• symbols --kind namespace --lang protobuf is empty. No package declarations means no way to query "show me the namespace structure of this proto API surface." map and inspect lose top-level structural signal.
• oneof invisibility breaks polymorphic API understanding. google.protobuf.Value, google.protobuf.Any, google.api.HttpRule and many gRPC streaming types are designed around oneof. Missing them means an AI consumer can't see "what are the alternatives" for any union-typed message.
• extend invisibility breaks API option / annotation exploration. Anyone exploring google.api.http, google.api.method_signature, google.api.client.default_host won't see the extend blocks that declare them.
• Mis-kinded enum pollutes --kind class. 5,815 fake "classes" on googleapis dominate the class list, making it hard for AI consumers to find real message-kinded "classes."
• Silent. No warning, no degraded-confidence signal.

Suggested direction

Four additive / reclassification fixes; all one-line:

1. Re-kind enum from class to enum:

   new("enum", new Regex(@"^\s*enum\s+(?<name>\w+)", RegexOptions.Compiled), BodyStyle.Brace),

2. Add package pattern as namespace:

   new("namespace", new Regex(@"^\s*package\s+(?<name>[\w.]+)\s*;", RegexOptions.Compiled), BodyStyle.None),

3. Add oneof pattern. Kind choice: class (matches Protobuf's "structural unit" feel) or enum (it's a tagged union / variant, semantically closer to enum). I lean class to mirror message:

   new("class", new Regex(@"^\s*oneof\s+(?<name>\w+)", RegexOptions.Compiled), BodyStyle.Brace),

4. Add extend pattern:

   new("class", new Regex(@"^\s*extend\s+(?<name>[\w.]+)", RegexOptions.Compiled), BodyStyle.Brace),

5. (Optional polish) Re-kind service from class to interface:

   new("interface", new Regex(@"^\s*service\s+(?<name>\w+)", RegexOptions.Compiled), BodyStyle.Brace),

   Mirrors how Java/C# extractors treat interface declarations.

Expected impact after fix

On /tmp/srcs/googleapis-master/google:

• symbols --kind enum --lang protobuf --count rises from 0 to 5,815.
• symbols --kind namespace --lang protobuf --count rises from 0 to 6,932.
• symbols --kind class --lang protobuf --count drops by ~5,815 (the re-kinded enums) and gains some from oneof / extend (smaller).
• inspect on a oneof's containing message starts showing the oneof's structure.

Scope

• src/CodeIndex/Indexer/SymbolExtractor.cs — re-kind enum, add package / oneof / extend patterns, optionally re-kind service.
• src/CodeIndex/Indexer/ReferenceExtractor.cs — verify whether the protobuf reference pass picks up cross-package / cross-file references; if it relies on the package symbol's existence, this fix should improve references / deps for protobuf.
• tests/CodeIndex.Tests/SymbolExtractorTests.cs — fixtures for: (a) enum Severity { ... } captured as enum kind, (b) package google.api; captured as namespace, (c) nested oneof kind { ... } inside a message captured, (d) extend google.protobuf.FieldOptions { ... } captured, (e) service Greeter { rpc SayHello(...) returns (...); } continues to capture both the service and the rpc.

Related

• JavaScript / TypeScript symbol extractor tags if / for / while / switch blocks as function definitions #154 — JS/TS keyword false positives.
• Go import extraction is broken: multi-line import ( ... ) blocks leak ( as a symbol name, individual imports never captured, and single-line imports swallow trailing comments #155 — Go import block coverage collapse.
• PHP import extraction: use function X\Y\Z captures "function", require $variable captures the whole expression, group-use and aliases are lost #162 — PHP import extraction gaps.
• Dart symbol extractor captures else if, case const Class() etc. as function if / function Class — negative lookahead missing else / case / more #163 — Dart else-if / case const Class() false positives.
• F# symbol extraction: top-level namespace dropped, module X.Y.Z miscategorized as class, and member this.Foo = ... methods never captured #165 — F# namespace/module/member missing.
• VB.NET symbol extraction: top-level Namespace X never captured, and all class/sub/function regexes require explicit visibility so unmodified declarations silently drop #166 — VB.NET namespace + visibility coverage.
• Makefile symbol extraction captures := / ::= variable assignments as targets (and ignores %-pattern rules entirely) #167 — Makefile := variables and % patterns.
• SCSS symbol extraction: @function declarations never captured, compound/nested selectors miss everything after the first class, @forward not recognized as import #168 — SCSS @function / @forward / compound selectors.
• Rust impl Trait for Struct captures the trait name as a class (hundreds of fake class Future / class From / class Default rows on tokio), and unsafe impl ... for ... is dropped entirely #169 — Rust impl Trait for Struct mis-captures and unsafe impl dropped.
• .kts (Kotlin Script / Gradle Kotlin DSL) files are silently skipped — LangMap has .kt but no .kts, so every build.gradle.kts / settings.gradle.kts is invisible #170 — .kts (Gradle Kotlin DSL) silently skipped.
• GraphQL symbol extraction: fragment X on Y never captured (426 misses on graphql-platform), directive @X never captured, extend only handles extend type #171 — GraphQL fragment / directive / extend coverage.

Same family: per-line regex list misses common language constructs and is silent about it. Protobuf's package / enum kind / oneof are unambiguous one-line fixes.

Environment

• cdidx: v1.10.0 (installed via install.sh).
• Corpus: googleapis/googleapis@master — 6,800 .proto files, 5,815 enum declarations, 6,932 package declarations, 2,572 oneof chunk-hits, 52 extend chunk-hits.
• Platform: linux-x64 container.
• Filed from a cloud Claude Code session per CLOUD_BOOTSTRAP_PROMPT.md.