This repository was archived by the owner on Oct 2, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 3
[pull/stream] add a set of primitives used for moving data around #3
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
70f0ddb to
01c2af5
Compare
…he configured number of packets
… at the end of files
To ps_status and ps_pull respectively. These changes will aid in consistency as we start stacking things on top of eachother.
The main ones being duplex and stream - everything else errors
This also fixes an inconsistency with value data ownership. The rule here is that the value->data field is owned by the value and unless a special destroy_fn is configured `free` will be called. This semantic change broke counter/multiplier because the requirement of ownership was not clear. These streams now make a copy of the incoming data. This does not detract from the zero-copy goal (which is more applicable in the IO case) as values can still flow data without copy. However, this is determined on a stream by stream basis.
…uv stream functionality
The purpose of this stream is to split a stream of bytes by a pattern
This stream allows te caller to populate the next value so it can precisely control the contents of each packet.
…lding pipelines..
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If the act of programming can be boiled down to moving data from one place to another, then it would make sense to have an abstraction that lets people reuse common data operations. This is a fundamental Lego brick in the rawkit toolset.
This is based on the Node.js pull-stream system, started by Dominic Tarr, but implemented in a C99 (C++11 compatible) way.
Note: this is a first round implementation and the contained streams will maybe become a bit more complex as monitoring/logging/similar bits are added, but the interfaces should be fairly solid.
At the core of this implementation is a
ps_t(aka a "pull stream") to which a single function is bound on thefnproperty. Forsourcestreams, that is the only requirement. Forsinkandthroughstreams another property must be populated before callingfn, thesourceproperty.sourceis a pointer to anotherps_tand when hooked up it acts like two links in a chain or perhaps more c-esque, a linked list.ps_t.fnhas the signatureps_val_t * fn(ps_t *s, ps_stream_status status);which effectively breaks down to:pull a value from a stream
TODO
rawkit-test(will require a rebase of this branch)