Span<byte>.IndexOf(ReadOnlySpan<byte?> value) - Wildcard support

[Symbai]

Searching a byte or char array for a pattern is often being done and as of now requires the developer to fallback getting the first index match and then manually iterate through the array to see if it matches. Since the IndexOf method already accepts an array and C# has nullable support, it would be nice to have an extension method that allows wildcard support by passing a Span array of nullable byte/char.

APIs:

Span<byte>.IndexOf(this Span<byte> span, ReadOnlySpan<byte?> value)
Span<char>.IndexOf(this Span<char> span, ReadOnlySpan<char?> value

Example usage byte:

var byteSpan = new byte[] { 0x1, 0x2, 0x3, 0x4, 0x5 }.AsSpan();
var differentbyteSpan = new byte[] { 0x1, 0x2, 0xFF, 0x4, 0x5 }.AsSpan();

var pattern = new byte?[] {0x2, null, 0x4}.AsSpan();

var index = byteSpan.IndexOf(pattern);
var index2 = differentbyteSpan.IndexOf(pattern);

Debug.Assert(index == index2);

Example usage char:

var charSpan = "hello world!".AsSpan();
var differentcharSpan = "hello w0rld!".AsSpan();

var pattern = new char?[] { 'w', null, 'r', 'l', 'd' }.AsSpan();

var index = charSpan.IndexOf(pattern);
var index2 = differentcharSpan.IndexOf(pattern);

Debug.Assert(index == index2);

[GrabYourPitchforks]

@ahsonkhan speaking:

@JeremyKuhne - the usage of this type of API shows up more commonly in file paths and IO/Regex. Should we consider providing this capability there rather than on the low-level span primitives?

It will be hard to optimize/vectorize such an implementation. We would want to flesh out the API proposal more including the exact API shape in the proposal + details of the scenario:
https://github.com/dotnet/runtime/blob/master/docs/project/api-review-process.md

[stephentoub]

Should we consider providing this capability there rather than on the low-level span primitives?

Yes. This is a very slippery slope towards full-blown regular expressions. We should not add such support to span directly or even to MemoryExtensions. We should instead figure out how to make it work with Regex or some derivation thereof.

[JeremyKuhne]

We need richer matching on Span, particularly for file system enumeration (globbing). I think the fundamental thing we're missing is no/low-allocation regex functionality. (And having UTF-8 matching would be good too...)

cc: @eerhardt, @carlossanlop

[stephentoub]

We need richer matching on Span

When you say "on Span", are you suggesting literally as members on the System.Span type? Or do you mean "for Span"?

I'm strongly against the former. The latter makes sense, and is what #23602 is tracking.

[JeremyKuhne]

Or do you mean "for Span"?

For Span, not on.

[stephentoub]

In that case, anything we'd do here would be covered by #23602, so I'll close this. Thanks.