Packed and bitfield annotations

[fanpei91]

Currently solod does not have directives about using packed and bitfield on struct. I hope that there will be such two directive annotations, which can be used like this: .

//so:packed
type sdshdr16 struct {
    len uint16
    alloc uint16
    flags uint8
    buf [0]byte
}

type node struct {
    size uint64
    count uint32 //so:bitfield(2)
    encoding uint32 //so:bitfield(2)
    recompress uint32 //so:bitfield(1)
    extra uint32 //so:bitfield(27)
}

Thank your job! Solod is the system programming language I am looking for years.

[nalgeon]

Thank you! I can't promise these, but I'll look into it while working on v0.2. Do you have a specific use case for these structures (sdshdr16 and node), or are they just examples?

[fanpei91]

I am rewriting resis 8/valkey 9 using solod. These are all real data structures:

struct __attribute__((__packed__)) sdshdr5 {
    unsigned char flags; /* 3 lsb of type, and 5 msb of string length */
    char buf[];
};
struct __attribute__((__packed__)) sdshdr8 {
    uint8_t len;         /* used */
    uint8_t alloc;       /* excluding the header and null terminator */
    unsigned char flags; /* 3 lsb of type, 5 unused bits */
    char buf[];
};
struct __attribute__((__packed__)) sdshdr16 {
    uint16_t len;        /* used */
    uint16_t alloc;      /* excluding the header and null terminator */
    unsigned char flags; /* 3 lsb of type, 5 unused bits */
    char buf[];
};
struct __attribute__((__packed__)) sdshdr32 {
    uint32_t len;        /* used */
    uint32_t alloc;      /* excluding the header and null terminator */
    unsigned char flags; /* 3 lsb of type, 5 unused bits */
    char buf[];
};
struct __attribute__((__packed__)) sdshdr64 {
    uint64_t len;        /* used */
    uint64_t alloc;      /* excluding the header and null terminator */
    unsigned char flags; /* 3 lsb of type, 5 unused bits */
    char buf[];
};

typedef struct quicklistNode {
    struct quicklistNode *prev;
    struct quicklistNode *next;
    unsigned char *entry;
    size_t sz;             /* entry size in bytes */
    unsigned int count : 16;     /* count of items in listpack */
    unsigned int encoding : 2;   /* RAW==1 or LZF==2 */
    unsigned int container : 2;  /* PLAIN==1 or PACKED==2 */
    unsigned int recompress : 1; /* was this node previous compressed? */
    unsigned int attempted_compress : 1; /* node can't compress; too small */
    unsigned int dont_compress : 1; /* prevent compression of entry that will be used later */
    unsigned int extra : 9; /* more bits to steal for future usage */
} quicklistNode;

Due to the lack of packed, I use []byte to achieve this. For the absence of bitfield, I use a single uint32 field to partition it as 16 + 2 + 2 + 1 + 1 + 1 + 9 bits. The code readability is not good as the C language version.

[nalgeon]

I see, thanks for explaining!

[fanpei91]

Regarding the bitfield feature, if future support is planned, an approach like the following may be a choice:

type node struct {
    size uint64
    count uint32 `so:"bits:1"`
    encoding uint32 `so:"bits:2"`
    ...
}

[nalgeon]

Starting with 600e881, you can use the so:attr directive to add GCC attributes:

//so:attr packed
type sdshdr16 struct {
	len   uint16
	alloc uint16
	flags uint8
	buf   [0]byte
}

To install the latest version:

go install solod.dev/cmd/so@600e881fe72c
go get solod.dev@600e881fe72c

But honestly, I doubt that porting something as big as Redis to So makes much sense. It's probably better to just write some bindings and call it, instead of porting the whole thing.

[fanpei91]

Nice! Smart design!

I am implementing Redis using Solod in order to deeply understand it, and the best way to deeply understand it is to re-implement it in another language. I have previously implemented the standalone version of Redis 5 in Zig, but the experience was not enjoyable—it was too verbose, function parameters can not be reassigned, and goto was removed from the language. A few days ago, I looked into Odin; it is largely good, but lacks goto. I have always believed that an excellent systems programming language should have simple syntax and come with a rich standard library. As a Go developer, I appreciate Go’s design philosophy. Solod is exactly that kind of language—a pragmatic language I have been looking for for years.

Very thank you for your hard working!