feat: specify the storage_dtype in ExtDType

[a10y]

Adds scalars_dtype to ExtDType.

This PR adds scalars_dtype (alternative name options: canonical_dtype, storage_dtype) to ExtDType.

This is desirable for a few reasons

• Makes it possible to canonicalize an empty chunked ExtensionArray
• Makes it possible to determine the storage DType for a ConstantArray without examining its value
• Makes it possible for Vortex to reason about externally authored extension types. This is still not fully complete, as an ideal experience would allow extension authors to override IntoCanonical, IntoArrow, Display, etc.

To avoid duplicating the nullability, we remove top-level nullability from the DType::Extension variant, instead nullability is accessed through the inner ExtDType.

This has the unfortunate effect of bringing size_of::<DType>() from 40 -> 48 bytes, which obviously makes every array's metadata 8 bytes larger.

[a10y]

this works now

[a10y]

this works now too

[a10y]

I couldn't figure out a better way to kill the clippy errors on the generated code.

I tried both #[allow(clippy)] and #[allow(clippy::all)] on the module declarations and that didn't do it 🤷

[robert3005]

Clippy::all doesn’t mean “all” it means the things that are not pedantic or nursery or something like that

[a10y]

this was changed in tokio-rs/prost#1115 so we now need to allow(clippy::nursery) in the module level

[lwwmanning]

👀

[a10y]

lol i guess dan wrote the same method at the same time as me we just wrote the args in a different order

[a10y]

i just sort of yolo resolved the conflict and this was the outcome

[lwwmanning]

the nullability split thing here might read weird (it will say NonNullable above... and then maybe-Nullable here?)