Help to fix issues in JlrsReflect generated code

[Michael-Jing]

Hi, I have some types defined in Julia, then I used JlrsReflect generated rust wrappers, there's some names spaces changes in the most recent version (dev branch) of of jlrs, which I have fixed, but there's still some other issues, cau you help to fix? Thank you in advance!
Julia Code

struct JbTensor
    name::String
    dtype::UInt32
    dims::Vector{Int64}
    memory_ptr::Union{Array{Real, 1}, Array{Real, 2}, Array{Real, 3}, Array{Real, 4}}
    memory_type_id::Int64
    memory_type::UInt32
    byte_size::UInt64
end


struct JbError
    message::String
end
struct InferResponse
    output_tensors::Vector{JbTensor}
    error::JbError
end

struct InferRequest 
    request_id::String
    correlation_id::Int32
    model_name::String
    model_version::Int32
    flags::Int32
    inputs::Vector{JbTensor}
    requested_output_names::Vector{String}
end

generated rust code with Fixing

use jlrs::{
    convert::unbox::Unbox,
    data::{layout::valid_layout::{ValidLayout, ValidField}, types::typecheck::Typecheck},
    prelude::*,
};

#[repr(C)]
#[derive(Clone, Debug, Unbox, ValidLayout, ValidField, Typecheck)]
#[jlrs(julia_type = "Main.InferRequest")]
pub struct InferRequest<'frame, 'data> {
    pub request_id: ::std::option::Option<::jlrs::data::managed::string::StringRef<'frame>>,
    pub correlation_id: i32,
    pub model_name: ::std::option::Option<::jlrs::data::managed::string::StringRef<'frame>>,
    pub model_version: i32,
    pub flags: i32,
    pub inputs: ::std::option::Option<::jlrs::data::managed::array::ArrayRef<'frame, 'data>>,
    pub requested_output_names:
        ::std::option::Option<::jlrs::data::managed::array::ArrayRef<'frame, 'data>>,
}

#[repr(C)]
#[derive(Clone, Debug, Unbox, ValidLayout, ValidField, Typecheck)]
#[jlrs(julia_type = "Main.InferResponse")]
pub struct InferResponse<'frame, 'data> {
    pub output_tensors:
        ::std::option::Option<::jlrs::data::managed::array::ArrayRef<'frame, 'data>>,
    pub error: JbError<'frame>,
}

#[repr(C)]
#[derive(Clone, Debug, Unbox, ValidLayout, ValidField, Typecheck)]
#[jlrs(julia_type = "Main.JbError")]
pub struct JbError<'frame> {
    pub message: ::std::option::Option<::jlrs::data::managed::string::StringRef<'frame>>,
}

#[repr(C)]
#[derive(Clone, Debug, Unbox, ValidLayout, ValidField, Typecheck)]
#[jlrs(julia_type = "Main.JbTensor")]
pub struct JbTensor<'frame, 'data> {
    pub name: ::std::option::Option<::jlrs::data::managed::string::StringRef<'frame>>,
    pub dtype: u32,
    pub dims: ::std::option::Option<::jlrs::data::managed::array::ArrayRef<'frame, 'data>>,
    pub memory_ptr: ::std::option::Option<::jlrs::data::managed::value::ValueRef<'frame, 'data>>,
    pub memory_type_id: i64,
    pub memory_type: u32,
    pub byte_size: u64,
}

[Taaitaaiger]

If you're using the dev branch, you need to use the Reflect module from Jlrs.jl. Op di 14 mrt. 2023 01:59 schreef Yanqing Jing ***@***.***>:

…

Hi, I have some types defined in Julia, then I used JlrsReflect generated rust wrappers, there's some names spaces changes in the most recent version (dev branch) of of jlrs, which I have fixed, but there's still some other issues, cau you help to fix? Thank you in advance! Julia Code struct JbTensor name::String dtype::UInt32 dims::Vector{Int64} memory_ptr::Union{Array{Real, 1}, Array{Real, 2}, Array{Real, 3}, Array{Real, 4}} memory_type_id::Int64 memory_type::UInt32 byte_size::UInt64 end struct JbError message::String end struct InferResponse output_tensors::Vector{JbTensor} error::JbError end struct InferRequest request_id::String correlation_id::Int32 model_name::String model_version::Int32 flags::Int32 inputs::Vector{JbTensor} requested_output_names::Vector{String} end generated rust code with Fixing use jlrs::{ convert::unbox::Unbox, data::{layout::valid_layout::{ValidLayout, ValidField}, types::typecheck::Typecheck}, prelude::*, }; #[repr(C)] #[derive(Clone, Debug, Unbox, ValidLayout, ValidField, Typecheck)] #[jlrs(julia_type = "Main.InferRequest")] pub struct InferRequest<'frame, 'data> { pub request_id: ::std::option::Option<::jlrs::data::managed::string::StringRef<'frame>>, pub correlation_id: i32, pub model_name: ::std::option::Option<::jlrs::data::managed::string::StringRef<'frame>>, pub model_version: i32, pub flags: i32, pub inputs: ::std::option::Option<::jlrs::data::managed::array::ArrayRef<'frame, 'data>>, pub requested_output_names: ::std::option::Option<::jlrs::data::managed::array::ArrayRef<'frame, 'data>>, } #[repr(C)] #[derive(Clone, Debug, Unbox, ValidLayout, ValidField, Typecheck)] #[jlrs(julia_type = "Main.InferResponse")] pub struct InferResponse<'frame, 'data> { pub output_tensors: ::std::option::Option<::jlrs::data::managed::array::ArrayRef<'frame, 'data>>, pub error: JbError<'frame>, } #[repr(C)] #[derive(Clone, Debug, Unbox, ValidLayout, ValidField, Typecheck)] #[jlrs(julia_type = "Main.JbError")] pub struct JbError<'frame> { pub message: ::std::option::Option<::jlrs::data::managed::string::StringRef<'frame>>, } #[repr(C)] #[derive(Clone, Debug, Unbox, ValidLayout, ValidField, Typecheck)] #[jlrs(julia_type = "Main.JbTensor")] pub struct JbTensor<'frame, 'data> { pub name: ::std::option::Option<::jlrs::data::managed::string::StringRef<'frame>>, pub dtype: u32, pub dims: ::std::option::Option<::jlrs::data::managed::array::ArrayRef<'frame, 'data>>, pub memory_ptr: ::std::option::Option<::jlrs::data::managed::value::ValueRef<'frame, 'data>>, pub memory_type_id: i64, pub memory_type: u32, pub byte_size: u64, } — Reply to this email directly, view it on GitHub <#76>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOQBHH4VOG62PQKFQFF4MZDW367FRANCNFSM6AAAAAAVZZCTNQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Michael-Jing]

Hi, it seems that I still can not get this code working, The current issue is that when I try to create an array of JuliaString or some other structs like the JbTensor here using the Array::from_slice method, The compiler complains that ForeignType is not satisfied.

[Taaitaaiger]

That method requires that the element type implements IntoJulia, which can only be implemented by isbits types. If you want to create an array of Julia strings, use Array::new_for with the DataType returned by DataType::string_type as the element type. You can set its entries by calling Array::managed_data_mut (with T = StringRef), which returns a PtrArrayAccessor that provides a set method.

This area definitely deserves some more attention, I admit it's hard to use with non-isbits data.

[Michael-Jing]

Thank you very much for the comment, I'm sure your time is spent on more import issues and the project is being actively developed. When I tried the following code

let julia_test_string = JuliaString::new(&mut frame, "test"); 
let requested_output_names = Array::new_for(frame.as_extended_target(), (1, 2), DataType::string_type(&julia_test_string).as_value());

I got complains that trait bound 'Target' is not satisfied.

[Taaitaaiger]

DataType::string_type returns the DataType String, this type is used by Array::new_for to create the appropriate type for the array and create an instance of that array type. Functions like DataType::string_type take a reference to a target (e.g. a reference to a GcFrame) to ensure this data can't be accessed before Julia has been initialized.

Array::new_for takes this type object as a Value because the element type doesn't have to be a DataType, you can have arrays where the element type is a Union of several types for example.

[Michael-Jing]

Hi, the above comment helped me to pass a array of JuliaString from Rust to Julia.
Now I want to do the same for a custom struct, ie, I want to pass an array of custom struct from Rust to Julia, the questions are:

1. What should be the type passed to Array::new_for? DataType::any_type?
2. What should be the type parameter T for Array::managed_data_mut
3. When I try to call as_value() on a custom struct object, I got the complains that trait bounds managed::private::ManagedPriv<'_, '_> is not satisfied, What are some of the directions I can try to resolve this issue?

Thanks in advance!

[Taaitaaiger]

There are two cases:

1. The data can be represented in Julia. In this case I recommend defining the type in Julia and usimg JlrsCore.Reflect to generate the Rust implementation. If it's a bits type, it will implement IntoJulia as long as it doesn't have type parameters, otherwise you will have to use Array::new_for. The type can be constructed using the ConstructType trait in that case.
   2.You want to use data defined in Rust. In this case you can either box the data in Rust and leak a pointer to it, or implement OpaqueType use the julia_module to expose it as a Julia type. Because the layout is opaque to Julia, the only way to interact with such an object ia by by calling back into Julia via ccall.

The second approach only makes sense imo when writing a library that should be exposed to Julia (see RustFFT.jl for example).