[Feature] SSE Materialized View UI + Data Sources hub#18537
Merged
Conversation
Adds a controller-side Materialized View management UI and the supporting REST endpoints. Read-only details + drop only — full create/edit-from-UI is deferred to a follow-up.
**Backend**
- `PinotMaterializedViewRestletResource`:
- `GET /materializedViews` — listing with name, base tables, watermark, VALID / STALE / total partition counts, last refresh time, staleness SLO.
- `GET /materializedViews/{name}` — full definition (defined SQL, base tables, split spec, partition expression maps, staleness SLO) plus runtime (watermarkMs + per-bucket state map sorted by bucketStartMs).
- `DELETE /materializedViews/{name}` — delegates to `PinotHelixResourceManager.deleteTable` so the same dependent-MV safety guards apply.
- New `MATERIALIZED_VIEW_TAG` for swagger grouping.
**Frontend**
- New left-nav entry "Materialized Views" linking to `/materialized-views`.
- New listing page at `/materialized-views`: paginated table of all MVs with key health columns.
- New details page at `/materialized-views/{name}`:
- Summary (name, base tables, watermark, staleness SLO, partition counts by state).
- Defined SQL (CodeMirror, read-only).
- Full definition JSON (CodeMirror, read-only).
- Per-partition table (bucketStartMs as ISO timestamp, state, segmentCount, crc, lastRefreshTime).
- "Refresh" + "Drop MV" actions; drop goes through a confirmation dialog.
- New `requests/index.ts` helpers: `getMaterializedViewList`, `getMaterializedView`, `deleteMaterializedView`.
Existing controller wiring (PR 1) handles all the underlying ZK reads/writes and the dependent-MV guard on `deleteTable`; this PR is the operator-facing surface on top of it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Hongkun Xu <xuhongkun666@163.com>
hongkunxu
force-pushed
the
feat/sse_mv_view_ui
branch
from
68292c2 to
7a68658
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #18537 +/- ##
============================================
- Coverage 64.31% 64.22% -0.10%
Complexity 1126 1126
============================================
Files 3309 3310 +1
Lines 203571 203709 +138
Branches 31686 31704 +18
============================================
- Hits 130928 130831 -97
- Misses 62142 62391 +249
+ Partials 10501 10487 -14
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Apply @sqltools/formatter (already used by Query.tsx) to the persisted single-line definedSQL before rendering it in the read-only CodeMirror viewer, so the SELECT/GROUP BY structure is visually scannable. The formatter call is memoized on the SQL text and falls back to the raw string on parser errors so the page never blanks on an unrecognized dialect. Bumped the SQL box height from 220 to 280 to fit the now multi-line layout. Co-Authored-By: Hongkun Xu <xuhongkun666@163.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduce a new /data-sources page accessible from the left sidebar that surfaces both queryable surfaces (Tables, Materialized Views) as cards with live counts, making it the discoverable entry point for the data-source types Pinot exposes. The standalone "Materialized Views" sidebar entry is removed — MV is now reached via Data Sources — while the existing "Tables" card on the Cluster Manager homepage is left in place for backward compatibility. Co-Authored-By: Hongkun Xu <xuhongkun666@163.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
xiangfu0
changed the title
xiangfu0
approved these changes
May 20, 2026
Contributor
|
Docs PR opened: pinot-contrib/pinot-docs#811 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a read-only Controller UI and REST surface for SSE Materialized Views, building on the MV creation/ingestion wiring landed in #18528 (PR 1/2).
Operators can list all MVs, inspect definition + runtime health, and drop an MV — create/edit from the UI is intentionally deferred.
Context
deleteTableMV cleanup / dependent-MV guards.Overview
Changes
Backend (
PinotMaterializedViewRestletResource)GET/materializedViews{ materializedViewTableName, error }without failing the whole list.GET/materializedViews/{name}bucketStartMs). Definition and runtime are built from a single ZK snapshot so summary counts match the partition table.DELETE/materializedViews/{name}PinotHelixResourceManager.deleteTable— same segment cleanup and dependent-MV rejection (409) as table delete.MATERIALIZED_VIEW_TAG.GET_TABLE; per-MV GET/DELETE use table-scoped permissions on the path param (mirrors/tables/{name}).Frontend (Controller React app)
/materialized-views.error./materialized-views/{name}):getMaterializedViewList,getMaterializedView,deleteMaterializedView.Utils.formatEpochMillisfor optional MV timestamps (null/0/negative →—).Out of scope (follow-up)
Test plan
/materialized-views; verify columns match cluster state; confirm a corrupt/missing runtime znode shows an error row without breaking the page.GET /materializedViewsGET /materializedViews/{name}_OFFLINEDELETE /materializedViews/{name}_OFFLINE(test cluster only)