POSIX/ZFS recommended dnodesize?

[crabique]

Hello!

With a ZFS backend that has a fast special VDEV for metadata storage, it makes sense to use xattr=sa for improved performance by inlining the xattrs into metadata (it's actually the default in modern ZFS versions).

Reading through wiki and testing the metadata sidecar locally, I see that the total size of meta/ hovers around 200 bytes.

This means that together with regular dnode metadata, the total size may exceed the 512 bytes available to the dnode struct with dnodesize=legacy (which is also the default).

In this case, ZFS will allocate an additional metadata buffer to store the extra data, which (if this happens often enough to matter) wastes more valuable special VDEV space and is technically both a write and read amplification.

Thus my questions is: how likely it is that dnodes exceed the 512B size, and would you recommend using dnodesize=auto to ensure the entire metadata for every object fits into its ZFS dnode?

I am aware of the potential issues with zfs send due to this, but I'm not even planning to ever snapshot this dataset, so this is not a concern.

[benmcclelland]

There is some metadata that is only bounded by the filesystem supported max xattr size. For example: tags, user metadata, bucket policy, etc can all grow to max xattr size. The default case is likely to only include a couple small attrs.

default bucket xattrs:

user.acl: {"Owner":"test","Grantees":[{"Permission":"FULL_CONTROL","Access":"test","Type":"CanonicalUser"}]}
user.ownership: BucketOwnerEnforced

default object xattrs:

user.checksums: {"Algorithm":"CRC64NVME","Type":"FULL_OBJECT","CRC32":null,"CRC32C":null,"SHA1":null,"SHA256":null,"CRC64NVME":"MlDF2FCcUf0="}
user.content-type: text/plain
user.etag: "cff9ad8b293a7ae7f17e187c47f368dc"

The total metadata size really depends on what the client is sending though

[crabique]

Thanks for the reply!

Great, I do not plan to use anything fancy on the client side, so in this case 512B should be plenty.