Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rpk/transform/deploy: support deploying to multiple output topics #16950

Merged

Conversation

rockwotj
Copy link
Contributor

@rockwotj rockwotj commented Mar 7, 2024

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v23.3.x
  • v23.2.x

Release Notes

Features

  • rpk transform deploy now supports multiple output topics

r-vasquez
r-vasquez previously approved these changes Mar 7, 2024
Copy link
Contributor

@r-vasquez r-vasquez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Mar 7, 2024

@rockwotj
Copy link
Contributor Author

rockwotj commented Mar 8, 2024

Force push: revert changing the flag name. @r-vasquez do you have a recommendation for this? The latest push you can specify --output-topic multiple times, but you can't do --output-topic=foo,bar, because I think --output-topics would be a better flag name for that behavior. Options:

  1. Force config file usage for multiple output topics (no flag changes).
  2. Have both flags --output-topic and --output-topics and they just append to the same slice
    In this model --output-topic could only be specified once, and --output-topics would be
    StringSliceVarP
  3. Breaking change to --output-topics (and make it StringSliceVarP)
  4. This PR - just allow --output-topic multiple times (StringArrayVarP)

@rockwotj rockwotj requested a review from r-vasquez March 8, 2024 04:06
@redpanda-data redpanda-data deleted a comment from vbotbuildovich Mar 8, 2024
@r-vasquez
Copy link
Contributor

r-vasquez commented Mar 8, 2024

@rockwotj Would StringSliceVar work here (keeping the name output-topic in singular)? it let you repeat and provide comma-separated values.

https://pkg.go.dev/github.com/spf13/pflag#StringSliceVar

We do that already in rpk, for example rpk acl delete:

      --topic strings                  Topic to remove ACLs for (repeatable)

We make sure to put (repeatable) in the help text though

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
@rockwotj rockwotj force-pushed the rpk-transform-deploy-multiple-outputs branch from 55cf542 to 3578c89 Compare March 9, 2024 15:12
@rockwotj
Copy link
Contributor Author

rockwotj commented Mar 9, 2024

Ah nice, yes that works for me thanks! (and done)

@rockwotj rockwotj merged commit 8dd6fbd into redpanda-data:dev Mar 11, 2024
22 checks passed
@rockwotj rockwotj deleted the rpk-transform-deploy-multiple-outputs branch March 11, 2024 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants