[CF-4208] Add `--page-size` to on-prem Flink list commands by paras-negi-flink · Pull Request #3424 · confluentinc/cli

Paras Negi (paras-negi-flink) · 2026-07-31T15:26:15Z

What

The on-prem Flink (CPF/CMF) list commands paginate internally at a hardcoded size=100 and always fetch every page. Listing a large environment therefore costs many serial round trips — e.g. ~51 requests / ~35s for 5000 applications, where the UI does the equivalent in one request (see CF-4202).

This PR adds a --page-size flag to every paginated on-prem Flink list command, letting callers fetch the full list in fewer, larger requests (e.g. --page-size 5000 → ~1 round trip). It defaults to 100, so existing behavior and output are unchanged.

Changes

--page-size flag on all paginated on-prem list commands (application list, application event/instance list, catalog list, catalog database list, compute-pool list, detached-savepoint list, environment list, savepoint list, secret list, secret-mapping list, statement list). Shared addPageSizeFlag/getPageSize helpers in command.go (validates 0..MaxInt32; 0 = default 100).
CMF client refactor: the 12 duplicated pagination loops collapse into one generic listAllPages(pageSize, fetchPage) helper. The size is passed straight through to the CMF page/size API; the loop still terminates on the first empty page.
statement exception list is untouched — it hits a non-paginated endpoint, so --page-size doesn't apply.

Scope / follow-up

Server-side filtering for application list (--name/--status, or a --filter passthrough) is intentionally not in this PR. It will land as a separate PR stacked on this one, keeping pagination and filtering reviewable independently.
Together, page-size (fetch all faster) + filtering (fetch only what's relevant) are what actually close out CF-4202; --limit (cap results) was dropped as low-value.

Compatibility

New optional flag, default --page-size 0 → 100 per page → identical output to today. Non-breaking.

Testing

Unit test for listAllPages (default size, custom size, large size = fewer round trips, size > total, empty, error propagation).
The applications test-server mock now honors page/size, so application list --page-size 2 genuinely fetches the 3-app list across multiple round trips and asserts the full, unchanged output.
Integration cases for application list/statement list/detached-savepoint list --page-size, plus invalid --page-size -1. Help/error goldens regenerated.
make lint passes; full flink suite passes except the pre-existing TestFlinkShell/TestFlinkShellOnPrem TUI tests (fail identically on a clean main checkout).

🤖 Generated with Claude Code

confluent-cla-assistant · 2026-07-31T15:26:25Z

❌ Error getting contributor login(s).
Please ensure the email address associated with this commit is added to your Github account.

Copilot

Pull request overview

This PR closes CF-4208 by adding user-facing pagination control (--limit) across on-prem Flink “list” commands and introducing flink application list filtering (--name, --status). It also refactors the CMF REST client to centralize paging behavior (including limit short-circuiting) and updates test server behavior plus integration fixtures accordingly.

Changes:

Add a shared --limit flag (default 0 = unlimited) to on-prem Flink list commands, applying it server-side where endpoints are paginated and client-side for non-paginated exception listing.
Add --name / --status filtering to flink application list by composing CMF’s generic filter query parameter.
Refactor duplicated CMF paging loops into a generic listAllPages helper and add unit + integration coverage (with updated goldens).

Reviewed changes

Copilot reviewed 46 out of 47 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
test/test-server/flink_onprem_handler.go	Adds CMF applications `filter` query emulation in the test server.
test/flink_onprem_test.go	Adds integration coverage for `--limit` and application list filtering flags.
test/fixtures/output/flink/statement/list-limit-json.golden	New golden output for statement list with `--limit`.
test/fixtures/output/flink/statement/list-help-onprem.golden	Updates help golden to include `--limit`.
test/fixtures/output/flink/statement/list-env-missing-failure.golden	Updates failure/help golden to include `--limit`.
test/fixtures/output/flink/statement/exception/list-help-onprem.golden	Updates help golden to include `--limit` for exception list.
test/fixtures/output/flink/secret/list-help-onprem.golden	Updates help golden to include `--limit` for secret list.
test/fixtures/output/flink/secret-mapping/list-help-onprem.golden	Updates help golden to include `--limit` for secret-mapping list.
test/fixtures/output/flink/savepoint/list-help-onprem.golden	Updates help golden to include `--limit` for savepoint list.
test/fixtures/output/flink/savepoint/list-fail-both.golden	Updates failure/help golden to include `--limit`.
test/fixtures/output/flink/environment/list-help-onprem.golden	Updates help golden to include `--limit` for environment list.
test/fixtures/output/flink/detached-savepoint/list-limit-json.golden	New golden output for detached-savepoint list with `--limit`.
test/fixtures/output/flink/detached-savepoint/list-help-onprem.golden	Updates help golden to include `--limit`.
test/fixtures/output/flink/compute-pool/list-missing-env-flag-failure.golden	Updates failure/help golden to include `--limit`.
test/fixtures/output/flink/compute-pool/list-help-onprem.golden	Updates help golden to include `--limit`.
test/fixtures/output/flink/catalog/list-help-onprem.golden	Updates help golden to include `--limit`.
test/fixtures/output/flink/catalog/database/list-help-onprem.golden	Updates help golden to include `--limit`.
test/fixtures/output/flink/application/list-status-no-match-json.golden	New golden output for application list status filter no-match case.
test/fixtures/output/flink/application/list-status-filter-json.golden	New golden output for application list status filter match case.
test/fixtures/output/flink/application/list-name-filter-json.golden	New golden output for application list name filter case.
test/fixtures/output/flink/application/list-limit-json.golden	New golden output for application list with `--limit`.
test/fixtures/output/flink/application/list-limit-invalid.golden	New golden output for invalid `--limit` value.
test/fixtures/output/flink/application/list-help-onprem.golden	Updates help golden to include `--name`, `--status`, and `--limit`.
test/fixtures/output/flink/application/list-env-missing.golden	Updates env-missing output to include new flags in help text.
test/fixtures/output/flink/application/instance/list-help-onprem.golden	Updates help golden to include `--limit` for instance list.
test/fixtures/output/flink/application/instance-list-env-missing.golden	Updates failure/help golden to include `--limit`.
test/fixtures/output/flink/application/instance-list-app-missing.golden	Updates failure/help golden to include `--limit`.
test/fixtures/output/flink/application/event/list-help-onprem.golden	Updates help golden to include `--limit` for event list.
test/fixtures/output/flink/application/event-list-missing-flags.golden	Updates failure/help golden to include `--limit`.
test/fixtures/output/flink/application/event-list-app-missing.golden	Updates failure/help golden to include `--limit`.
pkg/flink/test/mock/cmf_client_mock.go	Updates generated CMF client mock for the new `ListStatements(..., limit)` signature.
pkg/flink/cmf_rest_client.go	Refactors paging into `listAllPages` and threads `limit` through CMF list calls.
pkg/flink/cmf_rest_client_test.go	Adds unit tests for `listAllPages` behavior across limits and paging scenarios.
internal/flink/command.go	Adds shared `--limit` flag registration and parsing helper (`getLimit`).
internal/flink/command_statement_list_onprem.go	Wires `--limit` into on-prem statement list via CMF client.
internal/flink/command_statement_exception_list_onprem.go	Adds `--limit` and truncates exceptions client-side for non-paginated endpoint.
internal/flink/command_secret_mapping_list.go	Wires `--limit` into secret-mapping list.
internal/flink/command_secret_list.go	Wires `--limit` into secret list.
internal/flink/command_savepoint_list.go	Wires `--limit` into savepoint list.
internal/flink/command_environment_list.go	Wires `--limit` into environment list.
internal/flink/command_detached_savepoint_list.go	Wires `--limit` into detached-savepoint list.
internal/flink/command_compute_pool_list_onprem.go	Wires `--limit` into compute-pool list.
internal/flink/command_catalog_list.go	Wires `--limit` into catalog list.
internal/flink/command_catalog_database_list.go	Wires `--limit` into catalog database list.
internal/flink/command_application_list.go	Adds `--name`/`--status` filtering and wires `--limit` into application list.
internal/flink/command_application_instance_list.go	Wires `--limit` into application instance list.
internal/flink/command_application_event_list.go	Wires `--limit` into application event list.

Files not reviewed (1)

pkg/flink/test/mock/cmf_client_mock.go: Generated file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+func getLimit(cmd *cobra.Command) (int32, error) {
+	limit, err := cmd.Flags().GetInt("limit")
+	if err != nil {
+		return 0, err
+	}
+	if limit < 0 {
+		return 0, fmt.Errorf("`--limit` must be a non-negative integer")
+	}
+	return int32(limit), nil
+}


The on-prem Flink (CPF/CMF) list commands loop over a hardcoded 100-item page size with no way to control it, so listing a large environment costs many serial round trips (~51 requests for 5000 applications). Add a --page-size flag to every paginated on-prem list command so callers can fetch the full list in fewer, larger requests. It defaults to 100, which preserves today's behavior and output. Collapse the 12 duplicated pagination loops in the CMF client into a single generic listAllPages helper that takes the page size. The requested size is passed through to the CMF page/size API; the loop still terminates on the first empty page. Server-side filtering for `application list` is deliberately left out and will follow in a separate PR stacked on this one. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI review requested due to automatic review settings July 31, 2026 15:26

Copilot started reviewing on behalf of Paras Negi (paras-negi-flink) July 31, 2026 15:26 View session

Copilot AI reviewed Jul 31, 2026

View reviewed changes

airlock-confluentinc Bot force-pushed the cli-cf-4208-flink-list-limit-filter branch from bba9ac2 to 1ecbfcb Compare August 2, 2026 08:38

Paras Negi (paras-negi-flink) changed the title ~~[CF-4208] Add --limit pagination and application list filtering to on-prem Flink~~ [CF-4208] Add --page-size to on-prem Flink list commands Aug 2, 2026

Paras Negi (paras-negi-flink) mentioned this pull request Aug 2, 2026

[CF-4208] Add --name/--status filtering to on-prem application list #3428

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[CF-4208] Add `--page-size` to on-prem Flink list commands - #3424

[CF-4208] Add `--page-size` to on-prem Flink list commands#3424
Paras Negi (paras-negi-flink) wants to merge 1 commit into
mainfrom
cli-cf-4208-flink-list-limit-filter

Paras Negi (paras-negi-flink) commented Jul 31, 2026 •

edited

Loading

Uh oh!

confluent-cla-assistant Bot commented Jul 31, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Paras Negi (paras-negi-flink) commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Changes

Scope / follow-up

Compatibility

Testing

Uh oh!

confluent-cla-assistant Bot commented Jul 31, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Paras Negi (paras-negi-flink) commented Jul 31, 2026 •

edited

Loading