Skip to content

Add JSON and YAML structured-data extraction budgets and accuracy checks #3808

Description

@Widthdom

Found while dogfooding the locally built cdidx on branch codex/search-audit-issues-20260619.

Duplicate check: no open issue found for structured data JSON YAML extraction budgets duplicate keys tab indentation signature length.

Evidence:

  • src/CodeIndex/Indexer/Symbols/SymbolExtractor.StructuredData.cs:28 parses JSON without an explicit MaxDepth.
  • src/CodeIndex/Indexer/Symbols/SymbolExtractor.StructuredData.cs:69 recursively walks JSON objects.
  • src/CodeIndex/Indexer/Symbols/SymbolExtractor.StructuredData.cs:179 maps JSON properties to source lines by searching encoded property names.
  • src/CodeIndex/Indexer/Symbols/SymbolExtractor.StructuredData.cs:113 counts only spaces for YAML indentation while the regex accepts tabs.
  • src/CodeIndex/Indexer/Symbols/SymbolExtractor.StructuredData.cs:172 stores trimmed source lines as signatures without an explicit structured-data signature cap.

Problem:
Deep or broad JSON/YAML files can stress extraction, and duplicate JSON keys or tab-indented YAML can produce inaccurate symbols.

Acceptance criteria:

  • Add explicit JSON/YAML depth, property/symbol count, and path-length budgets.
  • Bound structured-data signatures consistently with other extractors.
  • Improve duplicate-key line mapping and YAML tab/space handling.
  • Add tests for deep JSON, broad JSON/YAML, duplicate keys, tab indentation, and long lines.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions