Skip to content

Implement SQL documentation extraction via sqlparser #2

@RPG-Alex

Description

@RPG-Alex

Summary

Integrate sqlparser and implement a pipeline that parses SQL files, walks the original source text, and associates preceding -- comments with tables and columns for documentation generation.

Processing flow

For each .sql file:

  1. Parse SQL statements with sqlparser.

  2. For each CREATE TABLE statement:

    • Identify the source text span of the table declaration and its columns.
    • Walk upwards from each table/column line in the original source to collect contiguous preceding -- comments.
    • Stop when a non-comment line is reached.
  3. Build structured documentation from those collected comment blocks.

Requirements

  • Use sqlparser as the SQL frontend (AST + basic validation).

  • Always operate on the original source text to locate comments (don’t rely purely on AST).

  • Clearly distinguish:

    • Documentation comments (immediately preceding table/column).
    • Inline or interstitial comments that should not be treated as docs.

Tasks

  • Add sqlparser as a dependency.
  • Implement a function that:
    • Takes raw SQL text and parsed statements.
    • For each CREATE TABLE:
      • Extracts table-level doc comments from lines directly above the CREATE TABLE line.
      • Extracts column-level doc comments from lines directly above each column definition line.
  • Decide and document the exact rules for:
    • When a comment block is considered “attached” to a table/column.
    • When a comment should be ignored (e.g. separated by blank lines or other code).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions