Skip to content

[26.2] Iceberg monitoring - #1822

Merged
kbatuigas merged 11 commits into
betafrom
DOC-2265-document-feature-iceberg-health-status
Jul 28, 2026
Merged

[26.2] Iceberg monitoring#1822
kbatuigas merged 11 commits into
betafrom
DOC-2265-document-feature-iceberg-health-status

Conversation

@kbatuigas

@kbatuigas kbatuigas commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Description

This pull request adds documentation for a new Iceberg health monitoring feature in Redpanda version 26.2 and updates related references. The most important changes are grouped below:

New Iceberg Health Monitoring Documentation:

  • Added a new section in about-iceberg-topics.adoc describing how to monitor Iceberg topic health using the new GetIcebergStatus Admin API endpoint, including catalog reachability and per-partition commit lag.
  • Announced the Iceberg health monitoring feature in the release notes (redpanda.adoc), including a brief description and a link to the new documentation section.

Admin API and Cross-References Updates:

  • Updated the Admin API usage guide (use-admin-api.adoc) to include Iceberg health monitoring as a feature available via ConnectRPC endpoints, with a link to the new documentation.

Related Admin API PR: redpanda-data/api-docs#76

Resolves https://redpandadata.atlassian.net/browse/DOC-2265
Review deadline: 27 July

Page previews

https://deploy-preview-1822--redpanda-docs-preview.netlify.app/streaming/26.2/manage/iceberg/about-iceberg-topics/#monitor-iceberg-health
https://deploy-preview-1822--redpanda-docs-preview.netlify.app/streaming/26.2/manage/iceberg/iceberg-troubleshooting/#troubleshooting-metrics
https://deploy-preview-1822--redpanda-docs-preview.netlify.app/streaming/26.2/get-started/release-notes/redpanda/

Checks

  • New feature
  • Content gap
  • Support Follow-up
  • Small fix (typos, links, copyedits, etc)

@kbatuigas
kbatuigas requested a review from a team as a code owner July 23, 2026 01:31
@netlify

netlify Bot commented Jul 23, 2026

Copy link
Copy Markdown

Deploy Preview for redpanda-docs-preview ready!

Name Link
🔨 Latest commit 734d6f7
🔍 Latest deploy log https://app.netlify.com/projects/redpanda-docs-preview/deploys/6a6829423271ce00081a64aa
😎 Deploy Preview https://deploy-preview-1822--redpanda-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ebf63917-b471-49dd-8bd1-fe986b2789b7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch DOC-2265-document-feature-iceberg-health-status

Comment @coderabbitai help to get the list of available commands.

@kbatuigas
kbatuigas marked this pull request as draft July 23, 2026 02:42
@kbatuigas
kbatuigas force-pushed the DOC-2265-document-feature-iceberg-health-status branch from a3666e1 to 042f7e4 Compare July 27, 2026 19:02
@kbatuigas
kbatuigas marked this pull request as ready for review July 27, 2026 20:11
@kbatuigas
kbatuigas requested a review from mattschumpert July 27, 2026 20:27
Comment thread modules/get-started/pages/release-notes/redpanda.adoc Outdated
Comment thread modules/manage/pages/iceberg/about-iceberg-topics.adoc Outdated
Comment thread modules/manage/pages/iceberg/iceberg-troubleshooting.adoc Outdated
kbatuigas and others added 9 commits July 27, 2026 18:14
Per PM review on PR #1822: GetIcebergStatus does more than a reachability
ping — it opens the external REST catalog connection (logs in with
credentials). Reword the three references from "reachability" to
"connectivity"/"connection" to reflect what the endpoint actually tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Group the three Iceberg entries (decoded keys/headers, Schema Registry
context, health monitoring) under a single "== Iceberg" section as a
bullet list instead of scattered top-level sections. Keep configuration
properties as its own section.

Also fixes two issues in the consolidated region:
- Broken xref: health-monitoring pointed to the deleted
  monitor-iceberg-health.adoc; now points to
  about-iceberg-topics.adoc#monitor-iceberg-health
- Missing blank line before the first section heading

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
"Iceberg" -> "Iceberg translation and health monitoring" to name both
themes covered by the section and match the feature-phrase style of the
other What's New headings.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kbatuigas
kbatuigas force-pushed the DOC-2265-document-feature-iceberg-health-status branch from 74d762e to cabcf15 Compare July 28, 2026 01:42
@micheleRP

Copy link
Copy Markdown
Contributor

Docs standards review

Findings only — 1 and 2 are the substantive ones, 3–9 are discretionary.

Critical issues (should fix before merge)

1. about-iceberg-topics.adoc:320 — "read-only" implies no superuser is needed, but the endpoint requires superuser.

The proto sets authorization explicitly:

rpc GetIcebergStatus(GetIcebergStatusRequest) returns (GetIcebergStatusResponse) {
    option (pbgen.rpc) = { authz: SUPERUSER };
}

(redpanda-data/redpanda, proto/redpanda/core/admin/v2/iceberg.proto)

This isn't just an omission. use-admin-api.adoc:14 (on beta) tells readers: "Superuser privileges, if authentication is enabled… (Some endpoints are read-only and do not require superuser access.)" So describing GetIcebergStatus as "the read-only … endpoint" signals to a reader who has read the Admin API guide that superuser is not required — and they'll hit an authz failure.

Fix: keep "read-only" (accurate — the proto calls the service a "public, read-only surface") but add the requirement, for example:

... use the read-only `GetIcebergStatus` Admin API endpoint. This endpoint
requires superuser privileges if authentication is enabled on your cluster.

2. Both new deep links 404 until api-docs #76 merges.

redpanda-data/api-docs#76 ("Admin API v2: add IcebergService and FeaturesService for 26.2") is still open against main, unmerged. Until it lands, this URL has no target:

link:/api/doc/admin/v2/operation/operation-redpanda-core-admin-v2-icebergservice-geticebergstatus

It's referenced from about-iceberg-topics.adoc:320 and, indirectly, from the release note. The PR description already flags #76 as related, so this is a merge-ordering item rather than a content defect — just worth confirming #76 merges first, since the beta docs are published.

Suggestions

3. The section is thin for an H2, and has no example request/response. It's currently one paragraph that defers entirely to the API reference. Every comparable Admin API v2 doc on beta shows a POST example — use-admin-api.adoc:101 (ShadowLinkService/FailOver), gbac-dp.adoc:251, 264, 359, 371. A short curl plus a trimmed response would also resolve finding 4 implicitly by showing the field names. Given that the standalone draft page was collapsed into this section, an example seems like the main thing lost in that consolidation.

4. Units of "commit lag" are ambiguous. "how far behind catalog commit is" doesn't say in what. The proto is specific: commit_lag is the "Number of records written to Parquet but not yet committed to the catalog" — not offsets, not time. Suggest saying "the number of records" so it isn't read as a duration.

5. Worth stating what the endpoint does not return, especially on the troubleshooting page. From the proto: "Translation lag (live-log high watermark minus the translated offset) is a client-side join with the Kafka API and is not returned here." Since iceberg-troubleshooting.adoc is where someone lands when translation appears stalled, a reader may reasonably expect translation lag from a "health" endpoint. One clause noting that commit lag ≠ translation lag would prevent a wrong diagnosis.

6. Release notes: bullets are inconsistent with the rest of the page. After this change, "Iceberg translation and health monitoring" is the only H2 on redpanda.adoc using a bulleted list; "Cluster health metrics" and "Kafka 4.x client compatibility" both use prose. Grouping the three Iceberg items under one heading is a reasonable call, but consider bolding the lead-ins (*Decoded message keys and headers*:) to match the **Iceberg:** treatment in the "New configuration properties" section below.

7. Minor — the release note slightly overstates the API. "identify the partitions with the highest commit lag" reads as though the endpoint ranks results; it returns all partitions unsorted, and the reader sorts. Fine as an outcome statement, but "to find the partitions with the highest commit lag" would be more literal.

8. Optional — (Redpanda version 26.2 and later) is redundant here. The page lives in the 26.2 component behind the version selector, and the text is inside ifndef::env-cloud[], so it can only render in self-managed 26.2+ docs. Harmless and arguably defensive; flagging only for consistency with how other 26.2 features are documented on this branch.

9. Judgment call — is the "About" page the right home? about-iceberg-topics.adoc is a concept page; "Monitor Iceberg health" is task-oriented. The standalone page was tried and reverted, so I assume this was deliberate — but if an example is added per finding 3, the section will outgrow the page's topic type, and iceberg-troubleshooting.adoc or a dedicated page may fit better.

@micheleRP micheleRP left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approving to not block for release, but please see review comments before merging

@kbatuigas
kbatuigas merged commit f7dbcff into beta Jul 28, 2026
7 checks passed
@kbatuigas
kbatuigas deleted the DOC-2265-document-feature-iceberg-health-status branch July 28, 2026 04:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants