Skip to content

Support for ND Arrays in ILP #90

Open
@amunra

Description

@amunra

Following #89, implement support for ND arrays in ILP.

Guidelines

Rust API

  • We should introduce a new trait for what constitutes an ND array.
  • Via feature flags, we should introduce optional support the ndarray crate.

Something like this:

use ndarray::array;
let arr = array![
    [1, 2, 3],
    [4, 5, 6]
];
buf.column_arr(arr)?;
pub trait NdArrayView<T>
where
    T: ArrayElement,
{
    /// Return the rank of the array
    fn ndim(&self) -> usize;

    /// Return the size of the `index` dimension
    fn dim(&self, index: usize) -> Option<usize>;
    
    /// Write the array data in row-major order to the provided buffer.
    /// The provided `buffer` will be pre-sized to the exact size as computed
    /// from the shape and element size.
    /// You may _not_ assume that the buffer's start is aligned to `T`.
    fn write_row_major_buf(&self, buff: &mut [u8]);
}

pub trait ArrayElement: Copy + 'static {}
impl ArrayElement for i32 {}
impl ArrayElement for i64 {}
impl ArrayElement for f32 {}
impl ArrayElement for f64 {}

pub fn column_arr<T, A>(arr: &A) -> Result<...>
where
    T: ArrayElement,
    A: NdArrayView<T>,
{
    ...
}

Then the impl:

#[cfg(feature = "ndarray")]
impl<T> NdArrayView<T> for ndarray::ArrayView<'_, T, '_>
where
    T: ArrayElement,
{
    fn ndim(&self) -> usize {
        self.ndim()
    }

    fn dim(&self, index: usize) -> Option<usize> {
        self.shape().get(index)
    }

    fn write_row_major_buf(&self, buf: &mut [u8]) {
        // figure out if already in standard layout row major:
        //   -> do a byte copy
        // if strided:
        //   -> iterate in row major order, copy value by value
    }
}

C and C++ APIs:

The other API will need an exploded set of APIs for different types.
Each type (integer double etc) will take a buffer for the strides and a buffer for the data.
E.g.:

LINESENDER_API
bool line_sender_buffer_column_f64_arr(
    line_sender_buffer* buffer,
    line_sender_column_name name,
    size_t rank,
    const size_t* shape,
    size_t data_buffer_len,
    const uint8_t* data_buffer,
    line_sender_error** err_out);

In C++ we can use some template specialisation to go back to an API more similar to Rust's.
We should also use std::byte instead of uint8_t to refer to the buffer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions