Efficient parsing using generalized unfolds #942
Conversation
|
Allocations and bytesCopied are 0. The type can be further simplified, we can remove the |
harendra-kumar
force-pushed
the
parse-unfold
branch
from
0a1fab5
to
aad6ae8
Compare
| -- | ||
| -- /Internal/ | ||
| {-# INLINE_NORMAL lmap #-} | ||
| lmap :: (a -> a) -> Producer m a b -> Producer m a b |
There was a problem hiding this comment.
We should probably remove this or rename this. I don't think this justifies lmap.
There was a problem hiding this comment.
A search for f :: a -> a -> g a -> g a on hoogle gave only fmap/liftA/liftM. Its a map even though it is restricted to an endomorphism. Any alternative names? If we remove it can we achieve the same functionality in some other way?
| import Streamly.Internal.Data.Producer.Type | ||
| import Prelude hiding (concat) | ||
|
|
||
| -- XXX We should write unfolds as producers where possible and define |
There was a problem hiding this comment.
I don't think we need to define unfolds using simplify. The users can do that themselves if ever need (They should ideally use producers where they can)
| extract (ReadUState (ForeignPtr end contents) (Ptr p)) = | ||
| return $ Array (ForeignPtr p contents) (Ptr end) (Ptr end) |
There was a problem hiding this comment.
The type of extact :: s -> m a puts a constraint on a few things. I might want to return the entire array along with the index (ptr) till where we generated. No use-case but...
I'm sure we'll encounter a use case that forces us to change the type of extract to s -> m c.
Instead of using a closed loop we can stop an unfold and then resume it later. This allows us to break the loop and restart it at some other point. For example, we can parse a block from an input stream from a file handle or a socket and then return the socket/handle plus nay buffered data (due to backtracking) so that we can resume reading from it later on after doing some processing.
Plus: * Fix returning buffer in the "parse" function * Some more minor formatting/doc fixups.
harendra-kumar
force-pushed
the
parse-unfold
branch
from
d01c5fc
to
c8b9fac
Compare
Instead of using a closed loop we can stop an unfold and then resume it
later by using an explicit loop state. This allows us to break the loop and restart it at some other
point. For example, we can parse a block from an input stream from a
file handle or a socket and then return the socket/handle plus nay
buffered data (due to backtracking) so that we can resume reading from
it later on after doing some processing.