feat: add new __list__ parameter for list-based array types

[agoose77]

This PR adds a new __list__ nominal type parameter to associate with behaviors. It should only be added to lists. In addition to setting an array-class for ListOffsetArray (et al.) via layout.parameter("__list__"), we can also lookup the list in a purelist sense, allowing list-of-type to have an array class too.

Setting __list__ and __record__ an error for non-intended types, but that should be another PR (there are fixes that need to be made to both validity_error and broadcasting).

[codecov]

Codecov Report

Merging #2549 (dcd86b3) into main (1f6041e) will increase coverage by 0.38%.
The diff coverage is 89.06%.

Additional details and impacted files

Impacted Files	Coverage Δ
src/awkward/_parameters.py	82.95% <57.14%> (+0.19%)	⬆️
src/awkward/_connect/numpy.py	91.86% <80.00%> (+0.06%)	⬆️
src/awkward/record.py	83.53% <80.00%> (-0.42%)	⬇️
src/awkward/forms/emptyform.py	84.26% <83.33%> (+0.17%)	⬆️
src/awkward/forms/numpyform.py	95.83% <83.33%> (-0.81%)	⬇️
src/awkward/forms/bitmaskedform.py	85.41% <85.71%> (+0.31%)	⬆️
src/awkward/forms/bytemaskedform.py	84.44% <85.71%> (+0.35%)	⬆️
src/awkward/forms/unionform.py	87.50% <85.71%> (+0.26%)	⬆️
src/awkward/forms/unmaskedform.py	84.61% <85.71%> (+0.40%)	⬆️
src/awkward/contents/content.py	76.16% <87.50%> (+0.13%)	⬆️
... and 9 more

... and 12 files with indirect coverage changes

[agoose77]

The addition of the new __list__ nominal type parameter provides the ability to set __typestr__ for a list type. However, unlike records which can sensibly be considered atoms, lists are usually part of a deeper structure that we care about seeing the type of. Having
N * my-list instead of N * my-list * int64 is worse. I wonder if we should think about this before merging this PR.

[jpivarski]

Very nice! I like the new documentation examples, too.

Since this PR is more complicated and you might think of something you need to add at the last minute, I'll let you merge it. I think it's done.

[jpivarski]

✔️

[jpivarski]

The addition of the new __list__ nominal type parameter provides the ability to set __typestr__ for a list type. However, unlike records which can sensibly be considered atoms, lists are usually part of a deeper structure that we care about seeing the type of. Having N * my-list instead of N * my-list * int64 is worse. I wonder if we should think about this before merging this PR.

Sorry that I missed this question!

A __typestr__ for __list__ would be useful because a common reason to override a list would be to define an opaque binary blob. The developer of that library would want the type of the hidden data to show up as "my-list", without any hint about the contents. So, for instance,

1000 * var * PNG-image

for an array of lists of overloaded bytestrings that are interpreted as PNG images.

The __typestr__ was introduced to make it possible to label strings as string and adhere to Datashape, as part of the goal of making string entirely user-definable. I think that was already gone at the time when Awkward 1 was converted to Awkward 2.

Since then, the __typestr__ was useful for overloaded records, in which the library developer wanted to hide the details of the record structure. (I think this was in some version of Coffea.)

The bytestring use-case (I can't think of any others, involving __list__) would also want to hide the internal structure of the list, for the same reason. In fact, that was also true when __typestr__ was set to "string".

So yes, I think it would be a good addition, but it can be applied in the same way that it is for __record__, which is simpler than trying to let the inner contents through (or parameterize/templatize __typestr__ in some way).