Move general purpose data structures to separate product or package

[patrickfreed]

There are a number of general purpose data structures included in SwiftNIO that would be useful in projects that don't require the rest of SwiftNIO itself (e.g. ByteBuffer). To avoid having to pull in all of NIO as a dependency just to use these structures, it would be nice if they were contained in their own smaller product or package, similar to NIOConcurrencyHelpers.

Personally, this would help reduce the compilation times / binary sizes of swift-bson, which only uses ByteBuffer out of its NIO dependency. PriorityQueue is another structure that might be used similarly, as seen in this post on the Swift forums.

[Lukasa]

I am disinclined to want to do this for PriorityQueue. PriorityQueue is not API in NIO today, and I don’t really want to draw an API boundary around it.

For CircularBuffer and ByteBuffer there is something to be said for this idea. I’d be open to pulling each of those out to their own library target and product, though I don’t want to put them in another package. @weissi how do you feel about that idea?

[Lukasa]

Ping @weissi.

[weissi]

I'd think that swift-collections should be the package that vends stuff like CircularBuffer (Collections has it as Deque) and PriorityQueue (currently unavailable in Collections AFAIK). Wrt to ByteBuffer I guess I'd be open to make a NIOByteBuffer target that we @_exported import NIOByteBuffer from NIO.

[ajotaos]

I'm interested since I'm bringing over the entirety of NIOCore as a dependency so there's the size and because I'm also using the SystemPackage.FileDescriptor there's a name conflict.

I ignore if there's a reason why it's not feasible to migrate this to a standalone package or at least expose it as its own library product.

[Lukasa]

What are you using from NIOCore?

[ajotaos]

I'm using both ByteBuffer and Endianness.

[Lukasa]

ByteBuffer is fairly easy to extract by itself: extract all files with the name ByteBuffer in them and then trace the compiler errors to find the missing files. I think you only need to pull out about 6 files.

[ajotaos]

I'm understanding that you mean to copy over the files directly to the project. Definitely, it's a fairly easy solution for it to work now if it's an emergency kind of situation but going on the note from @patrickfreed's original comment, it would lack the possibility to use a mature generic structure, like ByteBuffer, which fits in many different project domains and remained "subscribed" to any improvements and new versions and patches through the SPM.

Would it be possible to accommodate the standalone import of ByteBuffer in the roadmap? I'm asking only about technical feasibility and early feedback of course, any proposals would go through the Swift Forums to check the response of the community.

[Lukasa]

At this time we remain negative on the idea: we'd like to see the Swift community work to solve this problem a bit more broadly. There's nothing technically infeasible about the idea though.