transform-sdk/rust: introduce RecordWriter

[rockwotj]

In a similar change to our Golang SDK (#15922), also have a non-buffering record writer to output transformed records to.

Backports Required

• none - not a bug fix
• none - this is a backport
• none - issue does not exist in previous branches
• none - papercut/not impactful enough to backport
• v23.3.x
• v23.2.x
• v23.1.x

Release Notes

Improvements

• Data Transforms written in Rust now use a non-buffered write mechanism. Transforms that used to be written as

  fn my_transform(event: WriteEvent) -> Result<Vec<Record>> {
    Ok(vec![
      Record::new(
        event.record.key().map(|k| k.to_owned()),
        event.record.value().map(|v| v.to_owned()),
      )
    ])
  }
  

  Can now be written as

  fn my_transform(event: WriteEvent, writer: &mut dyn RecordWriter) -> Result<()> {
    writer.write(Record::new(
      event.record.key().map(|k| k.to_owned()),
      event.record.value().map(|v| v.to_owned()),
    ))?;
    Ok(())
  }
  

[rockwotj]

Force pushes: Update the public WriteError type to a better error type idiom.

[oleiman]

Rust really is quite nice 👌

[vbotbuildovich]

/backport v23.3.x