omnistreams is a specification defining a data streaming system. The project has the following core goals:
- Language-agnostic interfaces and semantics for barebones readable (producer) and writable (consumer) streams, including piping, cancellation, and operation (ie map, filter, etc) support.
- Dead-simple wire protocol with multiplexing. This is for sending streams between networked machines, processes, threads, coroutines, etc. The only requirement is an underlying reliable, in-order message passing system (ie WebSockets, WebRTC, ZeroMQ, reliable UDP, Go channels, etc)
- Pull-based (ie consumer-controlled) backpressure.
- Extremely simple to understand and implement. If a reasonable amount of performance loss means a greatly simplified design, choose the simple option. A first-year undergrad CS student should be able to make a usefully complete implementation in an arbitrary programming language.
You can read the specification here
omnistreams are heavily influenced by prior work, especially the excellent Reactive Streams, and ReactiveX. The primary differences are that omnistreams have less functionality (no semantics for multiple subscribers, no built-in operators, etc) and have a heavy focus on streaming data between different programming languages across a network. The most important thing we stole from reactive streams is the pull-based backpressure semantics (see also pull streams).
The WHATWG Streams API was not as much of a direct influence as the previous, but omnistreams shares a lot of similarity in API. They were both designed with a focus on I/O streaming.