Skip to content

Pipelines: R2 Data Catalog sinks enforce minimum write interval of 60 seconds#13951

Merged
edmundhung merged 2 commits into
cloudflare:mainfrom
sejoker:feat/PIPE-648-set-a-minimum-pipeline-frequency-to-1-minute
May 19, 2026
Merged

Pipelines: R2 Data Catalog sinks enforce minimum write interval of 60 seconds#13951
edmundhung merged 2 commits into
cloudflare:mainfrom
sejoker:feat/PIPE-648-set-a-minimum-pipeline-frequency-to-1-minute

Conversation

@sejoker
Copy link
Copy Markdown
Contributor

@sejoker sejoker commented May 18, 2026

  • R2 Data Catalog sinks: Minimum interval is 60 seconds
  • Regular R2 sinks: Minimum interval remains 10 seconds
  • The error message clearly explains: "Pipeline frequency must be at least 60 seconds for R2 Data Catalog sinks to prevent compaction issues"

  • Tests
    • Tests included/updated
    • Automated tests not possible - manual testing has been completed as follows:
Screenshot 2026-05-19 at 12 12 27 Screenshot 2026-05-19 at 12 13 57
Open in Devin Review

@sejoker sejoker requested a review from workers-devprod as a code owner May 18, 2026 12:04
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 18, 2026

🦋 Changeset detected

Latest commit: 8ca205e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
wrangler Patch
@cloudflare/vite-plugin Patch
@cloudflare/vitest-pool-workers Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-project-automation github-project-automation Bot moved this to Untriaged in workers-sdk May 18, 2026
@workers-devprod workers-devprod requested review from a team and edmundhung and removed request for a team May 18, 2026 12:05
@workers-devprod
Copy link
Copy Markdown
Contributor

workers-devprod commented May 18, 2026

Codeowners approval required for this PR:

  • ✅ @cloudflare/wrangler
Show detailed file reviewers

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

View 3 additional findings in Devin Review.

Open in Devin Review

Comment thread packages/wrangler/src/pipelines/defaults.ts Outdated
@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new Bot commented May 18, 2026

create-cloudflare

npm i https://pkg.pr.new/create-cloudflare@13951

@cloudflare/kv-asset-handler

npm i https://pkg.pr.new/@cloudflare/kv-asset-handler@13951

miniflare

npm i https://pkg.pr.new/miniflare@13951

@cloudflare/pages-shared

npm i https://pkg.pr.new/@cloudflare/pages-shared@13951

@cloudflare/unenv-preset

npm i https://pkg.pr.new/@cloudflare/unenv-preset@13951

@cloudflare/vite-plugin

npm i https://pkg.pr.new/@cloudflare/vite-plugin@13951

@cloudflare/vitest-pool-workers

npm i https://pkg.pr.new/@cloudflare/vitest-pool-workers@13951

@cloudflare/workers-editor-shared

npm i https://pkg.pr.new/@cloudflare/workers-editor-shared@13951

@cloudflare/workers-utils

npm i https://pkg.pr.new/@cloudflare/workers-utils@13951

wrangler

npm i https://pkg.pr.new/wrangler@13951

commit: 8ca205e

The validation that throws UserError for R2 Data Catalog sinks with
interval_seconds < 60 was incorrectly placed in applyDefaultsToSink,
which is used for display defaults. This broke the 'sinks get' command
for pre-existing sinks created before this validation was added.

The creation-time validation is already correctly handled in:
- create.ts validateArgs (CLI creation)
- setup.ts setupDataCatalogSink (interactive setup)
Copy link
Copy Markdown
Contributor

@workers-devprod workers-devprod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codeowners reviews satisfied

@github-project-automation github-project-automation Bot moved this from Untriaged to Approved in workers-sdk May 19, 2026
@edmundhung edmundhung merged commit e349fe0 into cloudflare:main May 19, 2026
65 of 68 checks passed
@github-project-automation github-project-automation Bot moved this from Approved to Done in workers-sdk May 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants