Skip to content
This repository was archived by the owner on Oct 2, 2022. It is now read-only.

Conversation

@tmpvar
Copy link
Owner

@tmpvar tmpvar commented Oct 8, 2020

If the act of programming can be boiled down to moving data from one place to another, then it would make sense to have an abstraction that lets people reuse common data operations. This is a fundamental Lego brick in the rawkit toolset.

This is based on the Node.js pull-stream system, started by Dominic Tarr, but implemented in a C99 (C++11 compatible) way.

Note: this is a first round implementation and the contained streams will maybe become a bit more complex as monitoring/logging/similar bits are added, but the interfaces should be fairly solid.

At the core of this implementation is a ps_t (aka a "pull stream") to which a single function is bound on the fn property. For source streams, that is the only requirement. For sink and through streams another property must be populated before calling fn, the source property. source is a pointer to another ps_t and when hooked up it acts like two links in a chain or perhaps more c-esque, a linked list.

ps_t.fn has the signature ps_val_t * fn(ps_t *s, ps_stream_status status); which effectively breaks down to:

pull a value from a stream

ps_t *val = stream->fn(stream, PS_OK)
// if stream->status is not PS_OK then we're done - we can pass PS_ERR or PS_DONE to _signal_ the stream
// val can be NULL if the stream did not have any data for us
// if val is not NULL, the caller now owns `val` and `val->data` which are
// both required to be `ps_destroy`able.

TODO

  • add readme
  • link exposed tests in rawkit-test (will require a rebase of this branch)

@tmpvar tmpvar force-pushed the pull/stream branch 3 times, most recently from 70f0ddb to 01c2af5 Compare October 8, 2020 20:07
tmpvar added 15 commits October 8, 2020 16:10
To ps_status and ps_pull respectively. These changes will aid in consistency
as we start stacking things on top of eachother.
The main ones being duplex and stream - everything else errors
This also fixes an inconsistency with value data ownership. The rule
here is that the value->data field is owned by the value and unless
a special destroy_fn is configured `free` will be called.

This semantic change broke counter/multiplier because the requirement
of ownership was not clear. These streams now make a copy of the
incoming data.

This does not detract from the zero-copy goal (which is more applicable
in the IO case) as values can still flow data without copy. However,
this is determined on a stream by stream basis.
The purpose of this stream is to split a stream of bytes by a pattern
This stream allows te caller to populate the next value so it can
precisely control the contents of each packet.
@tmpvar tmpvar merged commit 1d9e53d into master Oct 8, 2020
@tmpvar tmpvar deleted the pull/stream branch October 8, 2020 23:11
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants