Proposal for range selectors

[willscott]

This proposes a mechanism for selecting only a subset of a byte or string node that is an ADL over a larger number of nodes after reification.

[rvagg]

yeah, seems reasonable to me (I like the key renames too)

[warpfork]

I like the general outline of this. The data seems still placeholder though? Does something run these fixtures yet? I'm guessing not, because the data looks like an outline and doesn't look like it would actually fly.

I think that if this was fully real and wired to a working test, I would probably expect to see the fixtures grow to...

• have some real (if small) content -- maybe just "abc" and "def" in the next chunk, etc
• the hashes would have to become real
• the size numbers, I guess, should become real as well
• the subset indices in the selector would presumably become a bit smaller if we're using short fixture data
• the visit probably looks totally different -- I'd expect this to yield one bytes node, wouldn't I?
• which blocks are loaded is a different matter; maybe we want to have a fixture hunk for that as well (but we'd want both this and the expected visit, because they're very dissimilar, and that is highly worth making visible).

[warpfork]

Putting subset optional Slice as a field in Matcher -- rather than as its own whole exploration clause (which could contain a matcher inside it) -- gave me some pause for thought... but I guess it seems fine, and reasonable considering that it's meant for use on strings and bytes, which are considered nonrecursives.

[willscott]

the fixtures are waiting for ipld/go-ipld-prime#376 to complete - we don't currently have a way to convert a dag from the unix-fs dag-pb standard format to the json format that fixtures expect.

That said, this spec level change won't really be able to be evaluated for a bit still because we don't have a spec level definition of what 'unixfs' as an ADL does.

[willscott]

agree that the visit will have 1 node (and that the current setup is less relevant for the traversal behavior of interest here.)

[willscott]

Updated with a generated case of partial-selection over a unixfs file.
Note that this specification remains not fully complete at this level since it relies on what is currently a golang-specific implementation of unixfs.

[warpfork]

That caveat is applicable indeed... but also with real data in it, this kinda presses some buttons in my brain labelled "gorgeous" :D