Proposal: partial structs

[hvenev]

The idea of this proposal is to provide a convenient way to represent structs with identical prefixes. For example, struct sockaddr, or mimicking C++-style inheritance.

A partial struct is effectively a sequence of fields laid out in an unspecified manner. Any struct or partial struct B can extend a partial struct A. In that case all fields of A are visible as fields of B and have the same offsets.

In terms of layout, the first new field of B may begin after the last field of A - padding A to its alignment is not needed. If A and B are extern, the layout of B is the same as if all fields were in a single extern struct. An extern struct or partial struct can only extend a partial struct.

A *Type pointer to a struct or partial struct can be coerced to a base type pointer. Partial structs have a @sizeOf 0 and cannot be used as types of expressions, so dereferencing a partial struct pointer using ptr.* is also not allowed. [*]Type pointers to partial structs are not allowed.

We need to decide what to do with empty partial structs. I think for now they should not be allowed.

Example:

const Interface = partial struct {
    vtable: *InterfaceMethods,
};

const Impl1 = struct(Interface) {
    ....
};

const PartialImpl = partial struct(Interface) {
    some_field: SomeType,

    fn some_helper_method(self: *@This()) void { ... }
};

const Impl2 = struct(PartialImpl) {
    ...
};

We can also use it with unions:

const SockAddrBase = extern partial struct {
    family: SaFamily,
};

const SockAddrUn = extern Struct(SockAddrBase) {
    path: [108]u8,
};

const SockAddrInBase = extern partial struct(SockAddrBase) {
    be_port: u16,

    pub fn port(self: *@This()) u16 {
        return fromBE16(self.be_port);
    }
};

const SockAddrIn = extern struct(SockAddrInBase) {
    pub usingnamespace SockAddrInBase;
    addr: InAddr,

    pub fn set(self: *Self(), addr: InAddr, port: u16) void {
        self.family = .AF_INET;
        self.be_port = toBE16(port);
        self.addr = addr;
    }
};

const SockAddrIn6 = extern struct(SockAddrInBase) {
    pub usingnamespace SockAddrInBase;
    flowinfo: u32,
    addr: In6Addr,
    scope_id: u32,

    pub fn set(self: *Self(), addr: InAddr6, port: u16) void {
        self.family = .AF_INET6;
        self.be_port = toBE16(port);
        self.flowinfo = 0;
        self.addr = addr;
        self.scope_id = 0;
    }
};

const SockAddrIn46 = extern union {
    base: SockAddrInBase,
    ipv4: SockAddrIn,
    ipv6: SockAddrIn6,

    fn isIPv4(self: *@This()) bool {
        return self.base.family == .AF_INET;
    }

    fn isIPv6(self: *@This()) bool {
        return self.base.family == .AF_INET6;
    }

    pub fn port(self: *@This()) u16 {
        return self.base.port();
    }
};

[ifreund]

This seems quite similar to #1214 which was rejected.

[hvenev]

Something similar can be achieved with #7311, but this proposal has the following features:

1. The base struct is always at offset 0. This means that extern unions of structs (even if they are not extern) with common bases work, for example like in the sockaddr case above.

In the vtable example, it may be possible to change the vtable and a large part of the layout without having to reallocate memory and/or adjust pointers to our object. My usecase was to implement promises/futures, where we either store the intermediate state of the computation or its result. It will also be possible to switch between (chain/tail call) different computations at runtime, provided we have pre-reserved the maximum storage size and alignment. The base of the future stays at the same address and is never invalidated, so users can always manage callbacks/cancellation.

2. More efficient layouts are possible. For example:

const Base = extern partial struct {
    id: u64,
    flags: u16,
};

const Derived = extern struct(Base) {
    more_stuff: u16,
    even_more_stuff: u32,
};

In this case, @sizeOf(Dervied) == 16. If we did it with fields, the size would be 24.

More interesting alignment optimizations are also possible for non-extern structs. Here is an example of a permitted layout:

const A = partial struct {
    a: u64, // offset 0
    b: u16, // offset 8
};
const B = partial struct(A) {
    c: u32, // offset 12
};
const C = struct(B) {
    d: u16, // offset 10
}

This is also very difficult to achieve using fields.

3. By forcing a strict separation between complete (final) and partial (extensible/abstract) structs, we force the users to consider which types represent whole objects and which are just parts and avoid potential footguns with object slicing which may appear.

[marler8997]

I'd like to see a set of use cases we can use to compare the various proposals to solve this class of issues. Like the io_uring example for one.

One question I have about this is why introduce a partial struct at all? Why shouldn't you be able to "extend" normal structs as well?

[hvenev]

I'd like to see a set of use cases we can use to compare the various proposals to solve this class of issues. Like the io_uring example for one.

const BaseSqe = partial extern struct {
    opcode: u8,
    flags: u8,
    ioprio: u16 = 0,
    fd: i32,
};

const ReadvSqe = extern struct(BaseSqe) {
    off: u64,
    addr: u64, // [*]u8
    len: u32,
    rw_flags: u32,
    user_data: u64,
    zero: [3]u64 = .{0,0,0},
};

const ConnectSqe = extern struct(BaseSqe) {
    addr_len: u64,
    addr: u64, // *const SockAddr
    zero1: u32 = 0,
    zero2: u32 = 0,
    user_data: u64,
    zero3: [3]u64 = .{0,0,0},
};

Sadly with this proposal it seems like it's impossible to put user_data in Sqe.

One question I have about this is why introduce a partial struct at all? Why shouldn't you be able to "extend" normal structs as well?

I think the difference between normal structs and ones that should be extended is significant enough that they should be separate at the language level. One advantage is that we make object slicing difficult.

Extending normal structs will not allow the layouts shown in #7969 (comment). People assume that they can memcpy structs around, and if there are fields in the padding, things will go wrong.

[marler8997]

Oh geeze, yeah "Object Slicing" looks like a really bad footgun. With that I see why you wouldn't want to extend a normal struct without some sort of marker saying you should never copy a value of the struct you're extending.

It looks like because this feature is limited to only appending fields, the io_uring example can't be solved with this feature. I wonder if this proposal could be modified to be able to solve it? Otherwise, I'd be much less inclined to choose this solution over one that can solve all the use cases.

[raulgrell]

In this case, @sizeof(Dervied) == 16. If we did it with fields, the size would be 24.

@hvenev Wouldn't the fields of the derived struct have to begin at an address aligned with the largest one, making the size 20?

[hvenev]

In this case, @sizeof(Dervied) == 16. If we did it with fields, the size would be 24.

@hvenev Wouldn't the fields of the derived struct have to begin at an address aligned with the largest one, making the size 20?

No. A partial struct is not a struct, it's just a set of fields. Consider this example:

const A = partial struct {
    f1: T1,
    f2: T2,
};

const B = struct(A) {
    f3b: T3b,
    f4b: T4b,
};

const C = struct(A) {
    f3c: T3c,
    f4c: T4c,
};

It is equivalent to

const B = struct {
    f1: T1,
    f2: T2,
    f3b: T3b,
    f4b: T4b,
};

const C = struct {
    f1: T1,
    f2: T2,
    f3c: T3c,
    f4c: T4c,
};

except that we also require that @bitOffsetOf(B, "f1") == @bitOffsetOf(C, "f1") and @bitOffsetOf(B, "f2") == @bitOffsetOf(C, "f2").