use case: struct with embedded resizable last field

[andrewrk]

In C you can do something like this:

struct Buffer {
    size_t capacity;
    size_t length;
    unsigned char data[];
};

Here, instead of having data be a pointer in the struct, the struct itself is resized, along with data. The data is part of the struct; the data field points to the beginning.

Zig does not currently have a convenient way to represent this concept. How you would do this in status quo land is:

struct Buffer {
    capacity: usize,
    length: usize,
    data: u8,
}

fn getBuffer(wanted_data_len: usize) -> &Buffer {
    const ptr = (%%mem.allocate(u8, @sizeOf(Buffer) - 1 + wanted_data_len)).ptr;
    return (&Buffer)(ptr);
}

fn getSubString(buf: &Buffer, start: usize, end: usize) -> []u8 {
    assert(start <= buf.length);
    assert(end <= buf.length);
    (&buf.data)[start...end]
}

I guess that's not so bad. We could leave status quo as is.

[felipecrv]

Don't overlook the case where you have two (n) arbitrarily sized buffers

struct Model {
   size_t len;
   Vec3 *vertices;
   Vec2 *uv;
}

It would still be nice to allocate it all at once.

This could be solved by passing the sizes in the type declaration just like we do for C arrays (int arr[10]; // sizeof(arr) = 40):

struct Model<static_len> {
  size_t len;
  Vec3 vertices[static_len];
  Vec2 uv[static_len];
}

// You would store the data at the end of the struct and the two pointers to the data

sizeof(Model<10>)  // sizeof(size_t) + 10 * sizeof(Vec3) + 10 * sizeof(Vec2) + sizeof(*Vec3) + sizeof(*Vec2)

They would work like C arrays

struct Model *model = malloc(sizeof(Model<10>));    // analogous to malloc(sizeof(int[10]))
// TODO: figure out how to set the pointers automatically
model->len = 10;

sizeof(*model);  // = sizeof(size_t) + sizeof(*Vec3) + sizeof(*Vec2)
                           // It's not the size of the entire thing just like the size of *int_ptr_to_array
                           // is the size of a single int

// You could have them on the stack
Model<10> model;

And since you store the actual pointers in the struct, you can pass them to other functions

foo(&model);

void foo(Model *model) {
   for (int i = 0; i < model->len; i++) {
      // vertices and uv are pointers inside the struct
      model->vertices[i]; 
      model->uv[i]; 
   }
}

You could afford to store only one pointer here as you at least know where the first buffer starts. In general you would need n - 1 buffers to store n parameterized buffers in the struct.

[kyle-github]

As Philix says, it is nice to allocate it all at once. However, I find that the extra gymnastics I do to figure out how big to allocate a "variable length" struct in one memory block are worth it for the other end of the lifespan: freeing the memory. I only have one free(). I do not need to know much of anything about the memory, or the type if it is all one block.

Now, I can wrap allocation and deallocation in functions per struct type and call those specifically. That works too (and is probably not a bad thing).

You can still do the variable length structure as one block even if you do not have the empty array at the end. Instead, have the last part be a pointer and point it just past the rest of the object as if it was a separate allocation. But the case with the end element being a zero length array is handy...

[daurnimator]

How you would do this in status quo land is:

struct Buffer {
    capacity: usize,
    length: usize,
    data: u8,
}

I don't think this is correct: zig gives no assurances on struct layout, and may reorder fields.

[andrewrk]

You are correct. I believe when I originally made that statement struct layout had defined order, but as you note that is no longer the case.

[emekoi]

this seems to be quite old, did zig ever give guarantees on struct layout?

[tgschultz]

simply adding packed to the struct definitions should make the original example still apply... well, that and all the other changes necessary to update the code.

[daurnimator]

I'm currently using this:

const Block = struct {
    nbytes: usize,
    bytep: *u8,
    fn bytes(self: *Block) [*]u8 {
        return @ptrCast([*]u8, self) + @sizeOf(self); // assumes @alignOf(u8) < @alignOf(self)
    }
};

Allocating one is a bit weird...

const blk = @ptrCast(*Block, try allocator.alignedAlloc(u8, @alignOf(Block), @sizeOf(Block) + len));
blk.* = Block{
    .nbytes = nbytes,
    .bytep = bytep,
};

(and it gets even weirder when I use the VLA as a linkedlist node type)

The @sizeOf call in this code should really be the actual structure size in bytes (not the size in an array); or you end up forcing the variable size element to the alignment of the struct itself.

[andrewrk]

There is no plan to add an explicit language feature for this.

[daurnimator]

There is no plan to add an explicit language feature for this.

Is there at least a plan for how to conventionally represent them? note that translate-c will need to handle them.

[dsimunic]

One way for translate-c would be to represent a struct as a comptime function that forces one to set the explicit size of the array.

A c struct like this:

typedef struct FunctionCallInfoBaseData
{
 short nargs;
 NullableDatum args[];
} FunctionCallInfoBaseData;

Would translate to:

fn struct_FunctionCallInfoBaseData(comptime args_len: c_ushort) type {
    return extern struct {
        nargs: c_ushort,
        args: [args_len]NullableDatum,
    };
}

All references would get translated as calls with the array length parameter set to 0, making the build fail for any access and forcing the user to review all uses (or alternatively would default to maxInt to allow build without review):

pub export fn add_one(fcinfo: *struct_FunctionCallInfoBaseData(0)) void {
    // This fails the build with 'accessing a zero length array is not allowed',
    // forcing the programmer to review the code
    _ = fcinfo.args[0].value;
}

This works well for cases when one accesses the array explicitly like in the above example.

For cases when one has to rely on the length passed at runtime (nargs value), one can set args_len to maxInt(usize) to pass the compile check.