Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

transform-sdk/go: introduce RecordWriter #15922

Merged
merged 1 commit into from
Jan 3, 2024

Conversation

rockwotj
Copy link
Contributor

@rockwotj rockwotj commented Jan 3, 2024

We've recieved good feedback that a more idiomatic interface for
transforms is using a pattern similar to http.HandlerFunc. This is much
cleaner IMO and allows for emitting records without buffering, as slices
are sometimes akward to work with.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v23.3.x
  • v23.2.x
  • v23.1.x

Release Notes

Improvements

  • Data Transforms written in Golang now use a non-buffered write mechanism. Transforms that used to be written as

    func myTransform(e transform.WriteEvent) ([]transform.Record, error) {
      return transform.Record[]{e.Record()}, nil
    }
    

    Can now be written as

    func myTransform(e transform.WriteEvent, w transform.RecordWriter) error {
      return w.Write(e.Record())
    }
    

We've recieved good feedback that a more ideomatic interface for
transforms is using a pattern similar to http.HandlerFunc. This is much
cleaner IMO and allows for emitting records without buffering, as slices
are sometimes akward to work with.

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
@cohix
Copy link

cohix commented Jan 3, 2024

Love this 👍🏼

@rockwotj rockwotj self-assigned this Jan 3, 2024
Copy link
Member

@oleiman oleiman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@rockwotj rockwotj merged commit 8cbb2b3 into redpanda-data:dev Jan 3, 2024
26 checks passed
@rockwotj rockwotj deleted the record-writer-go branch January 3, 2024 20:47
@vbotbuildovich
Copy link
Collaborator

/backport v23.3.x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/rpk area/wasm WASM Data Transforms
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants