builtin function @reify to create a type from a TypeInfo instance

[andrewrk]

• for arrays, pointers, error unions, nullables: T.child_type field.
• for functions, T.return_type field
• for functions, T.is_var_args and
• T.arg_types which is of type [N]type
• @fieldsOf(T) where T is a struct or enum. Returns anonymous struct { T: type, name: []const u8 }
• accessing a field by comptime string name
• implement @scurest proposal below
• • @typeInfo
• • @reify

cc @raulgrell

[ranma42]

For functions, would it make sense to also expose the number of arguments and make them individually accessible? (cc @AndreaOrru )

[AndreaOrru]

Yes, that would be super useful to implement the IPC in my kernel, for example.

[andrewrk]

Added a third item for this above

[andrewrk]

cc @hasenj

<hasenj> for comptime is there a way to perform property access by string or something similar?
<hasenj> for example, say, @field(object, field_name)

[hasenj]

@fieldsOf(T) where T is a struct or enum. Returns anonymous struct { T: type, name: []const u8 }

Is the type even needed? I think a list of names as strings should suffice, given that there are other builtin functions that can be used to reflect on the type. For example:

@field(some_struct, name); // expands to some_struct.<value of name>
// e.g.
@field(a, "foo"); // expands to `a.foo`

@typeOf(@field(a, "foo")); // gives type of a.foo

[tiehuis]

It seems like with just these features we could implement a structural based interface mechanism, similar to go.

Here is a motivating example to implement a print trait of sorts, allowing any struct which implements a print field with the appropriate type to have it called. I don't think anything is too glaringly out of place here, but feel free to correct.

const std = @import("std");
const builtin = @import("builtin");
const TypeId = builtin.TypeId;

fn getField(comptime T: type, x: var, comptime name: []const u8) ->
    (fn (self: T) -> %void)
{
    for (@fieldsOf(x)) |field_name| {
        if (field_name == name) {
            const field = @field(x, name);
            const field_type = @typeOf(field);

            if (@typeId(field_type) != TypeId.Fn) {
                @panic("field is not a function");
            }

            if (field_type.is_var_args) {
                @panic("cannot handle varargs function");
            }

            // Would need to be a bit more in-depth to handle a &T self arg
            const expected_args = []type { T };
            if (!std.mem.eql(type, field_type.arg_types, expected_arg_types)) {
                @panic("prototype does not match");
            }

            if (@typeId(field_type.return_type) != TypeId.Error
                and field_type.return_type.child_type != void)
            {
                @panic("return type does not match");
            }

            return field;
        }
    }

    null
}

// Expects a struct with a field method of type `print(self) -> %void`.
pub fn printTrait(x: var) -> %void {
    const T = @typeOf(x);
    const Id = @typeId(T);

    if (Id != TypeId.Struct) {
        @panic("expected a struct");
    }

    if (getField(T, x, "print")) |func| {
        func(x);
    } else {
        @panic("no print field found!");
    }
}

[andrewrk]

With this example, if you were going to do printTrait(x) couldn't you instead do x.print() ?

[tiehuis]

You're right. This example would really only provide slightly more targeted error messages.

A better example would be a printDebug function which could recursively print fields of structs and enums, similar to println!("{:?}", x) in Rust. Another example as well that could be very useful would be for generic serialization code by inspection of field names.

[scurest]

Since zig can store types as regular data, I was wondering if it would be better to, rather than have magic fields that make types seem like structs, expose the reflected data in actual structs. Example:

const NullableType = struct {
    child: type,
};

@reflect(?u32) ==> NullableType { .child = u32 }

const ArrayType = struct {
    child: type,
    len: u64,
};

@reflect([4]u32) ==> ArrayType { .child = u32, .len = 4 }

// Possible example for a struct

const StructType = struct {
    field_names: [][]const u8,
    field_types: []type,
    field_offsets: []u64,
}

const S = struct { x: i32, y: u8 };

@reflect(S) ==> StructType {
    .field_names = [][]const u8 { "x", "y" },
    .field_types = []type { i32, u8 },
    .field_offsets = []u64 { 0, 4 },
}

etc. You get the idea. You could also add a souped-up version of @typeId that returns an enum with variants like Nullable: NullableType, etc.

Some pros: possibly easier documentation (you can now lookup what fields you can access just like for regular structs), fewer special cases in the field-lookup code.

There could also be a @deReflect (deflect? unreflect?) that turns one of these reflected structs into a type, so you could generate eg. a struct programmatically at compile time.

Of course, @reflect could also be implemented as a regular function in userland (I hit #586 when trying it) as long as some reflection mechanism exists.

[andrewrk]

I like this idea. I ran into an issue which is related which I will type up and link here.

[andrewrk]

I think #588 has to be solved before the idea @scurest outlined here can be implemented.

[andrewrk]

We could potentially even remove these functions:

• @sizeOf
• @alignOf
• @memberCount
• @minValue
• @maxValue
• @offsetOf
• @typeId
• @typeName
• @IntType

Instead these could all be fields or member functions of the struct returned from @reflect, or replaced with @MakeType (un-reflect, deflect, whatever it's called).

[Ilariel]

I like the idea of @reflect since it exists in another namespace as a builtin and can exist as part of the language instead of being a standard library hack.

The @MakeType name could be @reify given the meaning of the word: "to regard something abstract as if it were a concrete material thing"
However whether it is fitting to use it given the CS meaning related meaning is another question
https://en.wikipedia.org/wiki/Reification_(computer_science)

With the "maketype" functionality we could write type safe compile time type generators that can give proper compiler errors and flexibly generate types. I think we need to be able to name the types too if we want to export them to C as a visible structs. On Zig side we can just use aliases to address the generated types if we want to.

[tgschultz]

I had this same idea while looking over some older reflection code I'd written and threw together an example of what I imagine a TypeInfo struct (returned by @reify) might look like. This is with no knowledge of compiler internals, it's just spitballing.

This example uses pointer reform syntax as described by #770

pub const TypeId = enum {
    Void,
    Type,
    NoReturn,
    Pointer,
    Bool,
    Integer,
    Float,
    Array,
    Slice,
    Struct,
    Union,
    Enum,
    ErrorSet,
    Promise,
    Function,
    Literal,
    Namespace,
    Block,
};

pub const LiteralTypeId = enum {
    Null,
    Undefined,
    Integer,
    Float,
    String,
    CString,
};

pub const PointerTypeId = enum {
    Single,
    Block,
    NullTerminatedBlock,
}

pub const TypeInfo = struct {
    isNullable: bool
    isErrorable: bool
    Id: TypeId,
    Type: type,
    Name: []const u8,
    Size: usize,
    Alignment: u29,
    Detail: TypeInfoDetail,
};

pub const TypeInfoDetail = union(TypeId) {
    Void: void,
    Type: void,
    NoReturn: void,
    Pointer: PointerTypeInfoDetail,
    Bool: void,
    Integer: IntegerTypeInfoDetail,
    Float: FloatTypeInfoDetail,
    Array: ArrayTypeInfoDetail,
    Slice: ArrayTypeInfoDetail,
    Struct: StructTypeInfoDetail,
    Union: UnionTypeInfoDetail,
    Enum: EnumTypeInfoDetail,
    ErrorSet: EnumTypeInfoDetail,
    Promise: PromiseTypeInfoDetail,
    Function: FunctionTypeInfoDetail,
    Literal: LiteralTypeId,
    Namespace: void,
    Block: void
    Opaque: void,
};

pub const PointerTypeInfoDetail = struct {
    Id: PointerTypeId,
    Child: *TypeInfo,
};

pub const IntegerTypeInfoDetail = struct {
    isSigned: bool,
    bits: u8,
    maxValue: usize,
    minValue: usize,
};

pub const ErrorTypeInfoDetail = struct {
    ParentSet: *TypeInfo,
    value: usize;
};

pub const FloatTypeInfoDetail = struct {
    bits: u8,
    maxValue: f64,
    minValue: f64,
    epsilon: f64,
    //potentially maxExp, hasSubnorm, etc?
};

pub const ArrayTypeInfoDetail = struct {
    isNullTerminated: bool,
    Child: *TypeInfo,
    length: usize,
};

pub const StructTypeInfoDetail = struct {
    isPacked: bool,
    memberNames: [][]const u8,
    memberOffsets: []const usize,
    Members: []const *TypeInfo,
};

pub const UnionTypeInfoDetail = struct {
    Tag: *TypeInfo,
    memberNames: [][]const u8,
    Members: []const *TypeInfo,
};

pub const EnumTypeInfoDetail = struct {
    isErrorSet: bool,
    Tag: *TypeInfo,
    memberNames: [][]const u8,
    MemberValues: []const usize,
};

pub const PromiseTypeInfoDetail = struct {
    //???
};

pub const FunctionTypeInfoDetail = struct {
    Return: *TypeInfo,
    Args: []const *TypeInfo,
};

[shawnl]

For functions, would it make sense to also expose the number of arguments and make them individually accessible? (cc @AndreaOrru )

I think the solution to all of these type of things would be to expose the LLVM bindings, and allow them to be used at comptime on Zig code.

[eira-fransham]

I don't really like having syntax to emit fields from an inline for. I'd prefer to just be able to manipulate the typeinfo, either by having the returned TypeInfo be mutable or by having a @reify builtin. Here's what the former would look like.

var outType = struct {
  // ... everything that doesn't need to be generated dynamically ...
};

const inFields = &@typeInfo(inputType).Struct.fields;
const originalFields = &@typeInfo(outType).Struct.fields;
var outFields: [originalFields.len + inFields.len]StructField = undefined;

for (originalFields) |field, i| {
  outFields[i] = field;
}

for (inFields) |field, i| {
  outFields[i + originalFields.len] = do_something_with(field);
}

@typeInfo(mytype).Struct.fields = &outFields;

[Sahnvour]

I'm not sure types should be mutable, in your example (I assume the mytype at the end is in fact outType) what would happen if outType is used before and after its fields get reassigned ?

Generating fields, definitions and such inline has the advantage that they happen when the type is created and you can't end up with a mismatch that would necessitate more language rules. @reify is also immune to this problem.

Also, I don't really see what it adds that wouldn't be otherwise possible; yet it's introducing a new way of defining types on top of the already existing one. (same goes for @reify)
I think it's tempting to use the fact that we already have compile-time introspection and code working with @typeInfo(foo).Struct.fields and the like, because it really is handy for consuming this data. However for producing types it is just a more verbose way of using the natural language constructs. Consider a function that has many attributes for example, I'd rather read its definition expressed purely in Zig than multiple lines modifying a comptime object, because it's clearer and closer to the "normal" language.

[marler8997]

Not knowing about this proposal, I just created a duplicate proposal for the @reify function here: #2906

Only difference being that I called it @infoType instead of @reify. The thought being that it's the inverse of @typeInfo.

I created it because I found a use case for it which I describe here if you want to take a look: #2782 (comment)

[andrewrk]

I created it because I found a use case for it which I describe here if you want to take a look: #2782 (comment)

I think it's a good use case, and I think it asks the question, "Even if we don't go full bananas on @reify, what if it works for simple types such as integers, slices, and pointers, so we can get rid of @IntType, and so that @marler8997's use case is solved better?"

I'm going to make a separate issue for that.

[andrewrk]

This issue is now blocking on #2907.

[Sahnvour]

Another random thought on what @reify could achieve: help exchange zig types (think optionals, slices, unions...) accross ABI boundaries. We could create helpers to convert from/to C types representing them automatically and their instances accordingly, so that in the end we can make dynamic zig libraries easier to use. I understand and agree that static linking is to be preferred, but the usecase exists. Makes sense?

[cshenton]

Related to AoS to SoA transforms is creating an archetype based ECS. For example, given a user defined archetype with some component fields, I'd like to be able to generate an appropriate concrete "storage" type (here storage is shown by simple slices).

const MyArchetype = struct {
    location: Vec3,
    velocity: Vec3,
};

// I then write some function Storage() that uses the @reify feature
// const MyArchetypeStorage = Storage(MyArchetype);

// I want to generate something like this:
const MyArchetypeStorage  = struct {
    location: []Vec3,
    velocity: []Vec3,
}

Modern game engines tend to jump through lots of hoops (and runtime costs) to pay for this abstraction, because it enables things like batched compile-time dispatch (if I have a bunch of storages I can run some function on each one that contains a particular set of batched field).

Would be great to be able to generate things like this.

[Sobeston]

I recently came across a problem which @reify would be very useful for. There's a very long C struct I'm working with where large blocks of fields are selectively present for different platforms using macros.

There's no neat way of currently solving this in zig without resorting to code generation, or writing massive amounts of code.

With @reify I could split these blocks into their own structs, and neatly "join" them based on comptime values to form the struct I'm looking for.

This would greatly simplify and reduce the footprint of the code, which would be great for my sanity and for spotting bugs.

[az5112]

I would recommend having a look at Circle (https://www.circle-lang.org/). It's a compile-time machinery on top of C++. It's sometimes complex (not saying it in a bad way - C++ itself is very complex) but it does pass the printf test. (It's possible to create printf without compiler hooks.) And it does much more than that.

[bb010g]

I'm running into a desire for full-strength @reify from writing argument parsing machinery; it'd be nice to generate structs & tagged unions which represent the byte slices parsed. https://github.com/MasterQ32/zig-args manages to avoid this currently by restricting its API to a struct with duck-typed, specially-named metadata fields, but this restricts access to other aspects of Zig's rich type system in the API. You can't use a struct to store a few different fields about an option, or a tagged union to clearly show how choices are structured. Trying to avoid unintuitiveness with @reify leads to complexity getting shoved into (comptime) dynamically typed trees that are weakly typed by default.

[ikskuh]

After some discussion with @alexnask in IRC we came to the conclusion that allowing @Type(.Struct) would be more sane than all the hacks and workarounds based on @TypeOf(.{ … }) and similar techniques.

People are already working around the restrictions that you cannot create struct at comptime creating APIs that are convenient to use, but really hard to understand. Allowing @Type(.Struct) would make the code more straightforward, readable and maintainable.

The question that was not clear is whether to allow creating structs with .decls set to a non-empty set or not. Allowing it would probably open up possibilities for really convenient interface APIs similar to interface.zig, but making code harder to understand, but still easier than the current implementations

[alexnask]

For reference, the following is achievable today:

const std = @import("std");

// Create a tuple with one runtime-typed field
fn UniTuple(comptime T: type) type {
    const Hack = struct {
        var forced_runtime: T = undefined;
    };
    return @TypeOf(.{Hack.forced_runtime});
}

/// Types should be an iterable of types
fn Tuple(comptime types: anytype) type {
    const H = struct {
        value: anytype,
    };
    var empty_tuple = .{};

    var container = H{
        .value = empty_tuple,
    };

    for (types) |T| {
        container.value = container.value ++ UniTuple(T){ .@"0" = undefined };
    }
    return @TypeOf(container.value);
}

pub fn StructType2(comptime names: []const []const u8, comptime types: anytype) type {
    std.debug.assert(names.len == types.len);
    const Storage = Tuple(types);

    return struct {
        const field_names = names;
        const field_types = types;
        const Self = @This();

        storage: Storage,

        pub fn create(literal: anytype) Self {
            comptime std.debug.assert(std.meta.fields(@TypeOf(literal)).len == field_names.len);
            comptime std.debug.assert(std.meta.trait.hasFields(@TypeOf(literal), field_names));

            var self: Self = undefined;
            inline for (field_names) |name, idx| {
                self.storage[idx] = @field(literal, name);
            }
            return self;
        }

        fn fieldIndex(comptime name: []const u8) ?comptime_int {
            var i = 0;
            while (i < field_names.len) : (i += 1) {
                if (std.mem.eql(u8, name, field_names[i])) return i;
            }
            return null;
        }

        fn FieldType(comptime name: []const u8) type {
            const idx = fieldIndex(name) orelse @compileError("Field '" ++ name ++ "' not in struct '" ++ @typeName(Self) ++ "'.");
            return field_types[idx];
        }

        pub fn field(self: anytype, comptime name: []const u8) if (@TypeOf(self) == *Self) *FieldType(name) else *const FieldType(name) {
            const idx = fieldIndex(name).?;
            return &self.storage[idx];
        }

        pub fn format(
            self: Self,
            comptime fmt: []const u8,
            options: std.fmt.FormatOptions,
            writer: anytype,
        ) !void {
            try writer.writeAll(@typeName(Self));
            if (std.fmt.default_max_depth == 0) {
                return writer.writeAll("{ ... }");
            }
            try writer.writeAll("{");
            inline for (field_names) |f, i| {
                if (i == 0) {
                    try writer.writeAll(" .");
                } else {
                    try writer.writeAll(", .");
                }
                try writer.writeAll(f);
                try writer.writeAll(" = ");
                try std.fmt.formatType(self.storage[i], fmt, options, writer, std.fmt.default_max_depth - 1);
            }
            try writer.writeAll(" }");
        }
    };
}

pub fn StructType(comptime fields: anytype) type {
    var field_names: [fields.len][]const u8 = undefined;
    var field_types: [fields.len]type = undefined;

    for (fields) |f, idx| {
        field_names[idx] = f[0];
        field_types[idx] = f[1];
    }

    return StructType2(&field_names, &field_types);
}

pub fn main() void {
    const FooStruct = StructType(.{
        .{ "a", usize },
        .{ "b", bool },
    });

    var foo = FooStruct.create(.{ .a = 0, .b = false });
    foo.field("a").* = 1;
    std.debug.print("{}\n", .{foo});
}

This is a simple implementation of a @Type(.Struct)-type construction (which could be improved with default field values and other things).
In my opinion it is clear that @Type(.Struct) would be better solution to this (including .decls although I could see how this is somewhat controversial)

[andrewrk]

After some discussion with @alexnask in IRC we came to the conclusion that allowing @Type(.Struct) would be more sane than all the hacks and workarounds based on @TypeOf(.{ … }) and similar techniques.

I think this is a strong argument. I'd like to "accept" @Type supporting struct, enum, etc, but keep the decls as an open proposal, with no conclusion yet. Hopefully that feels like progress.

[ghost]

Note on decls: perhaps we could sidestep the issue by requiring method definitions, and in fact all decl values, be written by value? That is, (in the case of methods) use a function which has already been defined, rather than constructing it from scratch. #1717 will make this more symmetric for methods vs variables/constants. (No examples as I'm on mobile -- hopefully I'm being clear enough.)

[tadeokondrak]

This is now implemented for every type, and can generate everything that syntax can except

• Generic function types (these are weird anyway)
• Declarations in enums, structs, unions, and opaque types.

I think this issue should be renamed to be about allowing declarations, or closed and replaced with a new one for that.

[andrewrk]

I think this issue should be renamed to be about allowing declarations, or closed and replaced with a new one for that.

sounds good 👍