Flink FULL_DATA_TYPE parser

[James Robinson (jlrobins)]

Summary of Changes

• Implement a traditional LL(1) recursive descent parser for Flink FULL_DATA_TYPE strings:

  • Parse all Flink type variants: scalars, parameterized, arrays, multisets
  • Support ROW types with backtick-quoted field names and optional comments
  • Support MAP types with exactly 2 members (key, value)
  • Minimal regex use - character-by-character parsing for clarity
  • Proper handling of nullability markers (NOT NULL, NULL)
  • Aimed to avoid most all O(n^2) string operations when building up intermediate strings.

• The parser is split into two layers, one general probably reusable recursive descent (peek(), consume(), ...) generic layer, and then a higher layer FlinkTypeParser which uses the lower layer to parse its particular grammar. The entirety of the parser is hidden away by a simple exported function parseFlinkType(full_data_type_string) which will return a single or a tree of FlinkType.

• Implements a type-safe discriminated union: FlinkType with two sub-types: ScalarFlinkType (no members) and CompoundFlinkType (required members), discriminated by either isCompoundFlinkType() type guard or just by checking if .kind === FlinkTypeKind.SCALAR as the output nodes from the parser. A toplevel node with or without interior members will be returned.

This is first of a two part effort. This creates the recursive descent parser core and a Flink FULL_DATA_TYPE-centric parser using it that can then parse Flink FULL_DATA_TYPE strings into object/object trees.

A followup branch will use those object/trees to make arbitrarily deep nested treeview items for the Flink table/view columns.

No UI / features changes present in this branch.

But while here, touched up Gulpfile.js to allow for regexes like "gulp test -t flinkTypeParser|ParserState' to be passed to gulp test and testRun.

Be not dismayed, there's only ~500 LOC of new models + parser code. And then so many tests and corner case tests. But since the strings we'll be parsing should all be well formed since coming from the Flink INFORMATION_SCHEMA, the defensiveness of the parsing and tests is ... Very Thorough Coding.

Pull request checklist

Please check if your PR fulfills the following (if applicable):

Tests

• Added new
• Updated existing
• Deleted existing

Release notes

• Does anything in this PR need to be mentioned in the user-facing CHANGELOG?

[Copilot]

Pull request overview

This pull request implements a recursive descent parser for Flink SQL data types, creating the foundation for displaying arbitrarily nested type structures in VS Code tree views. The parser handles all Flink type variants including scalars, parameterized types, arrays, multisets, ROW types with field names and comments, and MAP types.

Changes:

• Adds FlinkType model interfaces and enum for representing Flink SQL types in a simplified, UI-friendly structure
• Implements ParserState utility class providing character-level parsing primitives (peek/consume operations) with configurable delimiters
• Creates FlinkTypeParser using recursive descent parsing to handle all Flink type variants with proper nullability and comment support
• Enhances test infrastructure to support both exact string matching (FGREP) and regex patterns (GREP) for running specific tests

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/models/flinkTypes.ts	Defines FlinkType and CompoundFlinkType interfaces with type guard validation for representing Flink SQL types
src/parsers/parserState.ts	Implements reusable ParserState class with character-level parsing operations, delimiter matching, and identifier/keyword parsing
src/parsers/parserState.test.ts	Comprehensive unit tests for ParserState covering edge cases, error handling, and custom pattern support
src/parsers/flinkTypeParser.ts	Implements FlinkTypeParser using recursive descent to parse all Flink type variants with nullability and comment handling
src/parsers/flinkTypeParser.test.ts	Extensive test suite covering scalar types, nested structures, real-world examples, error cases, and degenerate inputs
src/testing.ts	Adds GREP support for regex-based test filtering alongside existing FGREP for exact string matching
Gulpfile.js	Implements regex pattern detection and conversion for test filtering with pipe-separated patterns

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

[Dave Shoup (shouples)]

Good stuff! I only had a couple of minor questions, but shouldn't block merging - looking forward to seeing this in action in the Flink Database view 👏

[sonarqube-confluent]

Quality Gate passed

Issues
1 New issue
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
98.3% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube