Skip to content

Conversation

@jlrobins
Copy link
Contributor

@jlrobins jlrobins commented Oct 22, 2025

Summary of Changes

  • New system catalog query performing the UNION ALL between INFORMATION_SCHEMA.TABLES (sic) and ....COLUMNS to learn about all of the relations (Kafka-backed tables, views, system tables, external tables) in a Flink database, wrapped by new CCloudResourceLoader method getFlinkRelations().
  • New models for Flink relations, columns, src/models/flinkRelation.ts.
  • New view mode over Flink relations, now the default mode, showing relations + their direct columns.

Optional: Click-testing instructions

  1. Log into CCloud.
  2. Tell the Flink Database view to set itself to a Flinkable cluster like RealworldData.
  3. Explore the tables and view, admire their columns + type definitions.

Optional: Any additional details or context that should be provided?

Pull request checklist

Please check if your PR fulfills the following (if applicable):

Tests

  • Added new
  • Updated existing
  • Deleted existing

Release notes

  • Does anything in this PR need to be mentioned in the user-facing CHANGELOG?

@jlrobins jlrobins changed the title Jlr/2841 tables views view mode New Flink Relations view mode showing toplevel relations + their direct columns Oct 22, 2025
@sonarqube-confluent

This comment has been minimized.

@jlrobins jlrobins marked this pull request as ready for review October 22, 2025 17:08
@jlrobins jlrobins requested a review from a team as a code owner October 22, 2025 17:08
Copilot AI review requested due to automatic review settings October 22, 2025 17:08
@jlrobins jlrobins marked this pull request as draft October 22, 2025 17:08
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new Flink Relations view mode for browsing database tables, views, and their columns. The relations view becomes the new default mode, replacing UDFs as the initial view. The implementation includes new models for Flink relations and columns, a system catalog query that unions table and column metadata, and comprehensive test coverage.

Key changes:

  • New default view mode showing Flink relations (tables/views) with their columns
  • Consolidated system catalog models in flinkSystemCatalog.ts (moved from flinkUDF.ts)
  • New system catalog query combining INFORMATION_SCHEMA.TABLES and COLUMNS data

Reviewed Changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tests/unit/testResources/makeRelationRow.ts New test helper functions for creating mock relation and column data
tests/unit/testResources/flinkUDF.ts Updated import path for consolidated system catalog models
tests/unit/testResources/flinkRelation.ts New test fixtures for relations and columns
src/viewProviders/multiViewDelegates/flinkUDFsDelegate.ts Updated imports for renamed model file
src/viewProviders/multiViewDelegates/flinkUDFsDelegate.test.ts Added extension context setup and updated imports
src/viewProviders/multiViewDelegates/flinkRelationsDelegate.ts New delegate implementing relations view logic
src/viewProviders/multiViewDelegates/flinkRelationsDelegate.test.ts Comprehensive tests for relations delegate
src/viewProviders/multiViewDelegates/constants.ts Added Relations view mode enum value
src/viewProviders/flinkDatabase.ts Integrated relations delegate and set as default view
src/viewProviders/flinkDatabase.test.ts Updated tests with async setup and UDFs mode initialization
src/viewProviders/baseModels/multiViewBase.ts Enhanced getChildren signature to support parent elements
src/storage/resourceManager.ts Updated import for consolidated models
src/storage/resourceManager.test.ts Updated import for consolidated models
src/models/flinkUDF.ts File deleted, moved to flinkSystemCatalog.ts
src/models/flinkUDF.test.ts File deleted, tests moved to flinkSystemCatalog.test.ts
src/models/flinkSystemCatalog.ts New consolidated file with UDF, relation, and column models
src/models/flinkSystemCatalog.test.ts Comprehensive tests for all system catalog models
src/loaders/utils/udfSystemCatalogQuery.ts Updated imports for consolidated models
src/loaders/utils/udfSystemCatalogQuery.test.ts Added explicit type annotations for parameters
src/loaders/utils/relationsAndColumnsSystemCatalogQuery.ts New query builder and parser for relations/columns data
src/loaders/utils/relationsAndColumnsSystemCatalogQuery.test.ts Tests for query generation and response parsing
src/loaders/ccloudResourceLoader.ts Added getFlinkRelations method
src/loaders/ccloudResourceLoader.test.ts Tests for new getFlinkRelations method
src/extension.ts Registered new view commands and updated default mode
src/commands/flinkUDFs.ts Updated imports for consolidated models
src/commands/flinkDatabaseView.ts New command for switching to relations view
src/commands/flinkDatabaseView.test.ts Tests for relations view command
package.json Added relations view mode command and menu entries

@jlrobins jlrobins marked this pull request as ready for review October 22, 2025 17:12
} from "../../models/flinkSystemCatalog";
import type { CCloudFlinkDbKafkaCluster } from "../../models/kafkaCluster";

const logger = new Logger("relationsAndColumnsSystemCatalogQuery");
Copy link
Contributor Author

@jlrobins jlrobins Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This module doing the system catalog query + parsing into model objs is 100% parallel to how we did it for UDFs + UDF parameters over in src/loaders/utils/udfSystemCatalogQuery.ts

See:

@shouples shouples self-assigned this Oct 22, 2025
@sonarqube-confluent

This comment has been minimized.

shouples
shouples previously approved these changes Oct 22, 2025
Copy link
Contributor

@shouples shouples left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good stuff, works great! Minor test suggestion, and I think moving all of the UDF models into models/flinkSystemCatalog might cause a little dev friction but I get the reasoning

@jlrobins jlrobins marked this pull request as draft October 22, 2025 17:46
@jlrobins jlrobins marked this pull request as ready for review October 22, 2025 18:54
@jlrobins jlrobins requested a review from shouples October 22, 2025 18:54

describe("flinkUDF.ts", () => {
describe("FlinkUdfParameter", () => {
describe("formatSqlType", () => {
Copy link
Contributor Author

@jlrobins jlrobins Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This external function / its test now moved over into src/utils/flinkTypes[.test].ts. The function is also used by src/models/flinkSystemCatalog.ts

@sonarqube-confluent
Copy link

Passed

Analysis Details

6 Issues

  • Bug 0 Bugs
  • Vulnerability 0 Vulnerabilities
  • Code Smell 6 Code Smells

Coverage and Duplications

  • Coverage 94.20% Coverage (73.80% Estimated after merge)
  • Duplications No duplication information (0.50% Estimated after merge)

Project ID: vscode

View in SonarQube

Copy link
Contributor

@shouples shouples left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more follow-up item: #2937

@jlrobins jlrobins merged commit 8077821 into main Oct 22, 2025
14 checks passed
@jlrobins jlrobins deleted the jlr/2841-tables-views-view-mode branch October 22, 2025 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

System catalog query + modeling for relation columns Make the new Flink Relations sub-mode of the existing Flink Database view

3 participants