transform-sdk/rust: multiple output topics #17007
Conversation
|
@redpanda-data/documentation requesting a review for updates to data transforms SDK. The relevant changes are in |
|
Force push: Rebase against dev |
| impl<'a> WriteOptions<'a> { | ||
| /// Create a new options struct with the record destination to an optional `topic`. | ||
| /// | ||
| /// If `topic` is `None` then the record will be written to the first |
There was a problem hiding this comment.
Is None a rust syntax? Can the value be null?
There was a problem hiding this comment.
Rust doesn't have null - None for an Optional is semantically equivalent.
| @@ -116,13 +116,36 @@ impl<'a> From<&'a WrittenRecord<'a>> for BorrowedRecord<'a> { | |||
| } | |||
| } | |||
|
|
|||
| /// WriteOptions allows for customizing a RecordWriter's write. | |||
There was a problem hiding this comment.
I'm guessing that WriteOptions and RecordWriter are both functions that the user can use? If yes, should we encompass them in code brackets?
There was a problem hiding this comment.
Yes good idea, let me do this.
|
@rockwotj do you have an example output of those docs? Is this something similar to JavaDoc for example? |
rockwotj
force-pushed
the
rust-mot
branch
3 times, most recently
from
9cd01fe
to
8ed4c85
Compare
|
Force push: docs feedback, then rebase against dev @Deflaimun yes docs can be generated via |
rockwotj
force-pushed
the
rust-mot
branch
2 times, most recently
from
8184ce4
to
9ac5608
Compare
| @@ -116,13 +116,36 @@ impl<'a> From<&'a WrittenRecord<'a>> for BorrowedRecord<'a> { | |||
| } | |||
| } | |||
|
|
|||
| /// [`WriteOptions`] allows for customizing a [`RecordWriter`]'s write. | |||
| /// | |||
| /// An example is to customize the output topic which a write can go to. | |||
There was a problem hiding this comment.
| /// An example is to customize the output topic which a write can go to. | |
| /// For example, use [`WriteOptions`] to customize the output topic to write to. |
Are there any behaviors or cases (other than the output topic) that WriteOptions can be used to customize?
There was a problem hiding this comment.
Not now, but in the future maybe
| } | ||
|
|
||
| impl<'a> WriteOptions<'a> { | ||
| /// Create a new options struct with the record destination to an optional `topic`. |
There was a problem hiding this comment.
Is "record" here referring to Record specifically? And would "the record's destination" still be accurate?
| Self { topic } | ||
| } | ||
|
|
||
| /// Create a new options struct with the record destination to `topic`. |
There was a problem hiding this comment.
Same question as above regarding record/Record
There was a problem hiding this comment.
Yes, updated it, PTAL
|
Force push respond to documentation feedback |
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
|
Force push: fix conflicts with dev |
src/transform-sdk/go/transform/internal/testdata/tee/transform.go
Outdated
Show resolved
Hide resolved
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
This is a false positive, as it doesn't understand our test assertions. Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Currently, some leadership flapping happens when creating this many topics at once. Instead, we should fix the offset when we deploy, there is a followup ticket for this and once that is implemented we can revert this commit. Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
|
Force push: remember what year it is |
Add support for the new method to write options in v2 of the broker's
ABI. This includes specifying write options in the RecordWriter
interface that can then be used to call the new
write_with_optionsmethod with the specified topic name.
Backports Required
Release Notes
Features
This feature can only be used in Redpanda v24.1.x or newer.