[Feature] null-delimited iteration

[wti]

Is your feature request related to a problem? Please describe.
Currently AsyncBufferSequence.LineSequence.AsyncIterator.next() only splits at newlines.

Since the newline character is permitted within many strings and splitting on newlines thus results in incomplete records, most OS and programming tools support outputting null-delimited records. Using null-delimited output is considered best practice in scripting, and required for programmatically consuming data from filesystem listing, git log records, etc.

Describe the solution you'd like
Support configuration to use the null character (0) as the separator, as well as some way to detect if an iterator is configured to split at the newline or null character.

Describe alternatives you've considered
Rejected alternatives:

• String or regex: Splitting input using some arbitrary String would be most flexible, but also entails much more complicated scanning.

• Naming: Currently the API is named using the default case of "lines". While this might seems inaccurate in the case of null-delimited output, I believe it is still clear, and being able to query the delimiter as newline or null is sufficient (and would avoid changing the API). However, if it's preferred to be more accurate, I would propose "segment" as the base concept supporting both newline- and null-delimited output.

• Duplicated iterator for null-delimited output: one could create a new iterator type with corresponding different names, to preserve the existing lines API and support null-terminated segments. I assume the iteration itself could be refactored so there would be only one delegate scanning function.

Additional context

As an example, the Shwift scripting library was updated to support arbitrary strings, but the driving use-case was null-terminated records:

GeorgeLyon/Shwift#27

[wti]

In terms of API, LineSequence could take and publish a Bool flag, and AsyncBufferSequence would have a new factory method like lines():

The existing lines() implementation would specify false, e.g.,:

public func lines() -> LineSequence<UTF8> {
    return LineSequence(
        underlying: self,
        encoding: UTF8.self,
        bufferingPolicy: .maxLineLength(128 * 1024),
        useNullDelimiter: false  // <------------------- added
    )
}

A new factory would specify true, e.g.,

  public func nullDelimitedSegments<Encoding: _UnicodeEncoding>(
      encoding: Encoding.Type,
      bufferingPolicy: LineSequence<Encoding>.BufferingPolicy = .maxLineLength(128 * 1024)
  ) -> LineSequence<Encoding> {
    return LineSequence(
      underlying: self,
      encoding: encoding,
      bufferingPolicy: bufferingPolicy,
      useNullDelimiter: true // <--------------------------------------
    )
  }

In LineSequence.next(), the null-delimited flow avoids the line-consuming logic:

buffer.append(first)

// add this section
if self.useNullDelimiter {
  if first == nullUnit {
    return yield()
  }
  continue // avoid consuming composite newlines below
} // end of added section

switch first {
   ... newline-based logic
}