Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implements writing delimited and length-prefixed sequence types #701

Merged
merged 13 commits into from
Feb 8, 2024
Merged
20 changes: 4 additions & 16 deletions src/lazy/encoder/binary/v1_0/writer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ use crate::lazy::encoder::value_writer::SequenceWriter;
use crate::lazy::encoder::write_as_ion::WriteAsIon;
use crate::lazy::encoder::LazyRawWriter;
use crate::lazy::encoding::Encoding;
use crate::unsafe_helpers::{mut_ref_to_ptr, ptr_to_mut_ref};
use crate::{IonResult, WriteConfig};
use bumpalo::collections::Vec as BumpVec;
use bumpalo::Bump as BumpAllocator;
Expand Down Expand Up @@ -46,19 +47,6 @@ impl<W: Write> LazyRawBinaryWriter_1_0<W> {
})
}

/// Helper function that turns a raw pointer into a mutable reference of the specified type.
unsafe fn ptr_to_mut_ref<'a, T>(ptr: *mut ()) -> &'a mut T {
let typed_ptr: *mut T = ptr.cast();
&mut *typed_ptr
}

/// Helper function that turns a mutable reference into a raw pointer.
fn mut_ref_to_ptr<T>(reference: &mut T) -> *mut () {
let ptr: *mut T = reference;
let untyped_ptr: *mut () = ptr.cast();
untyped_ptr
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗺️ These methods were redundantly defined in two different modules, so I created a private unsafe_helpers module to house them.

/// Writes the given Rust value to the output stream as a top-level value.
pub fn write<V: WriteAsIon>(&mut self, value: V) -> IonResult<&mut Self> {
value.write_as_ion(self.value_writer())?;
Expand All @@ -79,7 +67,7 @@ impl<W: Write> LazyRawBinaryWriter_1_0<W> {

let encoding_buffer = match encoding_buffer_ptr {
// If `encoding_buffer_ptr` is set, get the slice of bytes to which it refers.
Some(ptr) => unsafe { Self::ptr_to_mut_ref::<'_, BumpVec<'_, u8>>(*ptr).as_slice() },
Some(ptr) => unsafe { ptr_to_mut_ref::<'_, BumpVec<'_, u8>>(*ptr).as_slice() },
// Otherwise, there's nothing in the buffer. Use an empty slice.
None => &[],
};
Expand All @@ -97,13 +85,13 @@ impl<W: Write> LazyRawBinaryWriter_1_0<W> {
// If the `encoding_buffer_ptr` is set, we already allocated an encoding buffer on
// a previous call to `value_writer()`. Dereference the pointer and continue encoding
// to that buffer.
Some(ptr) => unsafe { Self::ptr_to_mut_ref::<'_, BumpVec<'_, u8>>(ptr) },
Some(ptr) => unsafe { ptr_to_mut_ref::<'_, BumpVec<'_, u8>>(ptr) },
// Otherwise, allocate a new encoding buffer and set the pointer to refer to it.
None => {
let buffer = self
.allocator
.alloc_with(|| BumpVec::new_in(&self.allocator));
self.encoding_buffer_ptr = Some(Self::mut_ref_to_ptr(buffer));
self.encoding_buffer_ptr = Some(mut_ref_to_ptr(buffer));
buffer
}
};
Expand Down
220 changes: 59 additions & 161 deletions src/lazy/encoder/binary/v1_1/container_writers.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,187 +11,111 @@ use crate::IonResult;

/// A helper type that holds fields and logic that is common to [`BinaryListWriter_1_1`],
/// [`BinarySExpWriter_1_1`], and [`BinaryStructWriter_1_1`].
pub struct BinaryContainerWriter_1_1<'value, 'top> {
// A byte containing the high nibble of the encoded container's type descriptor.
type_code: u8,
///
/// The [`BinaryContainerWriter_1_1`] does not know which type of container it is writing or whether that container
/// is length-prefixed or delimited. It only encodes provided values to its buffer. The owner of the
/// [`BinaryContainerWriter_1_1`] is responsible for:
/// * writing the correct opcode.
/// * copying the encoded data to the parent buffer (in the case of a length-prefixed container).
/// * writing field names to the [`BinaryContainerWriter_1_1`]'s buffer before each value (in the case of a struct).
pub(crate) struct BinaryContainerWriter_1_1<'value, 'top> {
// An allocator reference that can be shared with nested container writers
allocator: &'top BumpAllocator,
// The buffer containing the parent's encoded body. When this list writer is finished encoding
// its own data, a header will be written to the parent and then the list body will be copied
// over.
parent_buffer: &'value mut BumpVec<'top, u8>,
// The buffer to which child values will be encoded. In the case of:
// 1. a length-prefixed container, this will be a new buffer bump-allocated specifically for this
// container.
// 2. a delimited container, this will be the parent's own encoding buffer, to which the delimited
// container start opcode has already been written.
buffer: &'value mut BumpVec<'top, u8>,
}

impl<'value, 'top> BinaryContainerWriter_1_1<'value, 'top> {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗺️ I did some refactoring of lifetimes that allowed me to remove some methods/types.

Previously, each container writer had two lifetime parameters and could instantiate an affiliated type with a single lifetime parameter to be used as a closure argument.

  • BinaryContainerWriter_1_1<'value, 'top> -> BinaryContainerValuesWriter_1_1<'value>
  • BinaryListWriter_1_1<'value, 'top> -> BinaryListValuesWriter_1_1<'value>
  • BinarySExpWriter_1_1<'value, 'top> -> BinarySExpValuesWriter_1_1<'value>
  • BinarStructWriter_1_1<'value, 'top> -> BinaryStructFieldsWriter_1_1<'value>

Now, the two-lifetime types can be used as the closure argument themselves which simplifies things quite a bit.

pub fn new(
type_code: u8,
allocator: &'top BumpAllocator,
parent_buffer: &'value mut BumpVec<'top, u8>,
) -> Self {
Self {
type_code,
allocator,
parent_buffer,
}
}

pub fn write_values<'a, F>(self, _write_fn: F) -> IonResult<()>
where
'top: 'a,
F: FnOnce(BinaryContainerValuesWriter_1_1<'a>) -> IonResult<BumpVec<'a, u8>>,
{
todo!()
}

fn write_header_and_encoded_body(&mut self, _body: &[u8]) -> IonResult<()> {
todo!()
}
}

pub struct BinaryContainerValuesWriter_1_1<'value> {
allocator: &'value BumpAllocator,
buffer: BumpVec<'value, u8>,
}

impl<'value> BinaryContainerValuesWriter_1_1<'value> {
pub fn new(allocator: &'value BumpAllocator) -> Self {
let buffer = BumpVec::new_in(allocator);
pub fn new(allocator: &'top BumpAllocator, buffer: &'value mut BumpVec<'top, u8>) -> Self {
Self { allocator, buffer }
}

pub fn write<V: WriteAsIon>(&mut self, value: V) -> IonResult<&mut Self> {
let annotated_value_writer =
BinaryAnnotatableValueWriter_1_1::new(self.allocator, &mut self.buffer);
value.write_as_ion(annotated_value_writer)?;
Ok(self)
/// Constructs a new [`BinaryAnnotatableValueWriter_1_1`] using this [`BinaryContainerWriter_1_1`]'s
/// allocator and targeting its buffer.
fn value_writer<'a>(&'a mut self) -> BinaryAnnotatableValueWriter_1_1<'a, 'top> {
BinaryAnnotatableValueWriter_1_1::new(self.allocator, self.buffer())
}
}

pub struct BinaryListValuesWriter_1_1<'value> {
values_writer: BinaryContainerValuesWriter_1_1<'value>,
}

impl<'value> BinaryListValuesWriter_1_1<'value> {
pub fn new(values_writer: BinaryContainerValuesWriter_1_1<'value>) -> Self {
Self { values_writer }
}
/// Encodes the provided `value` to the [`BinaryContainerWriter_1_1`]'s buffer.
pub fn write<V: WriteAsIon>(&mut self, value: V) -> IonResult<&mut Self> {
self.values_writer.write(value)?;
let annotated_value_writer = self.value_writer();
value.write_as_ion(annotated_value_writer)?;
Ok(self)
}
}

impl<'value> MakeValueWriter for BinaryListValuesWriter_1_1<'value> {
type ValueWriter<'a> = BinaryAnnotatableValueWriter_1_1<'a, 'value> where Self: 'a;

fn value_writer(&mut self) -> Self::ValueWriter<'_> {
BinaryAnnotatableValueWriter_1_1::new(
self.values_writer.allocator,
&mut self.values_writer.buffer,
)
pub fn allocator(&self) -> &'top BumpAllocator {
self.allocator
}
}

impl<'value> SequenceWriter for BinaryListValuesWriter_1_1<'value> {
delegate! {
to self {
fn write<V: WriteAsIon>(&mut self, value: V) -> IonResult<&mut Self>;
}
pub fn buffer(&mut self) -> &'_ mut BumpVec<'top, u8> {
self.buffer
}
}

pub struct BinaryListWriter_1_1<'value, 'top> {
container_writer: BinaryContainerWriter_1_1<'value, 'top>,
pub(crate) container_writer: BinaryContainerWriter_1_1<'value, 'top>,
}

impl<'value, 'top> BinaryListWriter_1_1<'value, 'top> {
pub fn new(container_writer: BinaryContainerWriter_1_1<'value, 'top>) -> Self {
pub(crate) fn new(container_writer: BinaryContainerWriter_1_1<'value, 'top>) -> Self {
Self { container_writer }
}

pub fn write_values<'a, F>(self, write_fn: F) -> IonResult<()>
where
'top: 'a,
F: FnOnce(&mut BinaryListValuesWriter_1_1<'a>) -> IonResult<()>,
{
self.container_writer
.write_values(|container_values_writer| {
let mut list_values_writer =
BinaryListValuesWriter_1_1::new(container_values_writer);
write_fn(&mut list_values_writer)?;
Ok(list_values_writer.values_writer.buffer)
})
}
}

pub struct BinarySExpValuesWriter_1_1<'value> {
values_writer: BinaryContainerValuesWriter_1_1<'value>,
}

impl<'value> BinarySExpValuesWriter_1_1<'value> {
pub fn new(values_writer: BinaryContainerValuesWriter_1_1<'value>) -> Self {
Self { values_writer }
}
pub fn write<V: WriteAsIon>(&mut self, value: V) -> IonResult<&mut Self> {
self.values_writer.write(value)?;
Ok(self)
}
}

impl<'value> MakeValueWriter for BinarySExpValuesWriter_1_1<'value> {
type ValueWriter<'a> = BinaryAnnotatableValueWriter_1_1<'a, 'value> where Self: 'a;
impl<'value, 'top> MakeValueWriter for BinaryListWriter_1_1<'value, 'top> {
type ValueWriter<'a> = BinaryAnnotatableValueWriter_1_1<'a, 'top> where Self: 'a;

fn value_writer(&mut self) -> Self::ValueWriter<'_> {
BinaryAnnotatableValueWriter_1_1::new(
self.values_writer.allocator,
&mut self.values_writer.buffer,
)
self.container_writer.value_writer()
}
}

impl<'value> SequenceWriter for BinarySExpValuesWriter_1_1<'value> {
delegate! {
to self {
fn write<V: WriteAsIon>(&mut self, value: V) -> IonResult<&mut Self>;
}
impl<'value, 'top> SequenceWriter for BinaryListWriter_1_1<'value, 'top> {
fn write<V: WriteAsIon>(&mut self, value: V) -> IonResult<&mut Self> {
self.container_writer.write(value)?;
Ok(self)
}
}

pub struct BinarySExpWriter_1_1<'value, 'top> {
container_writer: BinaryContainerWriter_1_1<'value, 'top>,
pub(crate) container_writer: BinaryContainerWriter_1_1<'value, 'top>,
}

impl<'value, 'top> BinarySExpWriter_1_1<'value, 'top> {
pub fn new(sequence_writer: BinaryContainerWriter_1_1<'value, 'top>) -> Self {
pub(crate) fn new(sequence_writer: BinaryContainerWriter_1_1<'value, 'top>) -> Self {
Self {
container_writer: sequence_writer,
}
}
}

impl<'value, 'top> MakeValueWriter for BinarySExpWriter_1_1<'value, 'top> {
type ValueWriter<'a> = BinaryAnnotatableValueWriter_1_1<'a, 'top> where Self: 'a;

pub fn write_values<'a, F>(self, write_fn: F) -> IonResult<()>
where
'top: 'a,
F: FnOnce(&mut BinarySExpValuesWriter_1_1<'a>) -> IonResult<()>,
{
self.container_writer
.write_values(|container_values_writer| {
let mut sexp_values_writer =
BinarySExpValuesWriter_1_1::new(container_values_writer);
write_fn(&mut sexp_values_writer)?;
Ok(sexp_values_writer.values_writer.buffer)
})
fn value_writer(&mut self) -> Self::ValueWriter<'_> {
BinaryAnnotatableValueWriter_1_1::new(
self.container_writer.allocator(),
self.container_writer.buffer(),
)
}
}

pub struct BinaryStructFieldsWriter_1_1<'value> {
container_values_writer: BinaryContainerValuesWriter_1_1<'value>,
impl<'value, 'top> SequenceWriter for BinarySExpWriter_1_1<'value, 'top> {
fn write<V: WriteAsIon>(&mut self, value: V) -> IonResult<&mut Self> {
self.container_writer.write(value)?;
Ok(self)
}
}

impl<'value> BinaryStructFieldsWriter_1_1<'value> {
pub fn new(container_values_writer: BinaryContainerValuesWriter_1_1<'value>) -> Self {
Self {
container_values_writer,
}
pub struct BinaryStructWriter_1_1<'value, 'top> {
container_writer: BinaryContainerWriter_1_1<'value, 'top>,
}

impl<'value, 'top> BinaryStructWriter_1_1<'value, 'top> {
pub(crate) fn new(container_writer: BinaryContainerWriter_1_1<'value, 'top>) -> Self {
Self { container_writer }
}

pub fn write<A: AsRawSymbolTokenRef, V: WriteAsIon>(
Expand All @@ -203,7 +127,7 @@ impl<'value> BinaryStructFieldsWriter_1_1<'value> {
}
}

impl<'value> StructWriter for BinaryStructFieldsWriter_1_1<'value> {
impl<'value, 'top> StructWriter for BinaryStructWriter_1_1<'value, 'top> {
delegate! {
to self {
fn write<A: AsRawSymbolTokenRef, V: WriteAsIon>(
Expand All @@ -214,29 +138,3 @@ impl<'value> StructWriter for BinaryStructFieldsWriter_1_1<'value> {
}
}
}

pub struct BinaryStructWriter_1_1<'value, 'top> {
container_writer: BinaryContainerWriter_1_1<'value, 'top>,
}

impl<'value, 'top> BinaryStructWriter_1_1<'value, 'top> {
pub fn new(container: BinaryContainerWriter_1_1<'value, 'top>) -> Self {
Self {
container_writer: container,
}
}

pub fn write_fields<'a, F>(self, write_fn: F) -> IonResult<()>
where
'top: 'a,
F: FnOnce(&mut BinaryStructFieldsWriter_1_1<'a>) -> IonResult<()>,
{
self.container_writer
.write_values(|container_values_writer| {
let mut struct_fields_writer =
BinaryStructFieldsWriter_1_1::new(container_values_writer);
write_fn(&mut struct_fields_writer)?;
Ok(struct_fields_writer.container_values_writer.buffer)
})
}
}
Loading
Loading