Constructing fully immutable objects iteratively

[kmsquire]

I've been bothered by the fact that, for iterative construction of objects, JSON3 requires a mutable struct definition (when deserializing a type).

For fully immutable types (i.e., those which do not contain references outside to other objects, don't allow nothing or missing values, etc), at least, I'm wondering if there is some workaround.

For example, if mutable structs and structs were guaranteed to have the same memory layout, could one construct a mutable struct, and then reinterpret it as an immutable one? (I'm not sure.)

One workaround (for fully immutable types) that I believe would work (see VideoIO.jl/src/util.jl, which modifies fields of structs allocated by ffmpeg libraries):

1. Use introspection to determine the layout of (immutable) struct T using fieldoffset, fieldname, and fieldtype.
2. Allocate a Vector{T}(undef, 1)
3. Reinterpret as a Vector{UInt8}
4. Fill in field values using pointers + unsafe_wrap

Pseudocode might look something like this

obj = Vector{T}(undef, 1)  # should guarantee that the object is in memory; may want to zero the memory
ptr = convert(Ptr{UInt8}, pointer(obj))

# later

key = parsekey()  # parse key from JSON
offset, type = get_offset_type(T, key)

field_ptr = convert(Ptr{type}, ptr + offset)
field_arr = unsafe_wrap(Array, field_ptr, 1)
field_arr[1] = parsetype(type) # parse value from JSON

Obviously, the part which sets the value should be wrapped in a function, and there would probably need to be some way to make it type stable, etc. And it could be brittle. But it might work... :-)

(As a broader goal, I think it would be useful to have an extendable serialization/deserialization infrastructure similar to serde.rs, which allowed easy serialization and deserialization of immutable structs.)

[rapus95]

For example, if mutable structs and structs were guaranteed to have the same memory layout, could one construct a mutable struct, and then reinterpret it as an immutable one? (I'm not sure.)

IIRC that's the idea behind freeze & melt/thaw: JuliaLang/julia#31630 (comment)

[quinnj]

While an approach like this might work for structs w/ all isbitstype fields, it very quickly gets messy with Union or non-isbitstype fields (which are currently allowed in struct). After discussing with a few core devs, they also have doubts about even allowing this kind of immutable memory fiddling with all isbitstypes.

I think a better path forward is something like #70, or also allowing a "KeywordArgs" struct type that would pass key-value pairs as keyword arguments to struct constructors, like Base.@kwdef builds. I'm experimenting with things and we'll figure something out.

[quinnj]

Ok, as of #113 we've lifted the strict "order" requirement for StructTypes.Struct. If anyone is still interested in the KeywordArgs idea, feel free to open an issue at StructTypes.jl and we can figure out a solution.