Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
A port to Scala of the Conduit library by Michael Snoyman
Scala Shell
Tree: ba0dc26619

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
benchmark/src/main/scala/conduits/benchmark
bytestring/src
conduits/src
examples
network
parse
project
resourcet/src
text
.gitignore
.sbtrc
README.md

README.md

This is a direct port to Scala of Micheal Snoyman's Conduit library. See Conduits are an approach to the streaming data problem. It is meant as an alternative to enumerators/iterators.

The most up-to-date documentation is available as an appendix of the Yesod book, at: http://www.yesodweb.com/book/conduits. Below follows a brief description of the core concepts.

Summary

Streaming data allows us to process large datasets without pulling all values into memory. Conduits allow us to process large streams of data while retaining deterministic resource handling They provide a unified interface for data streams, whether they come from files, sockets, or memory.

A Monad Transformer, ResourceT, can be used to safely allocate resources, knowing that they will always be reclaimed- even in the presence of exceptions.

Core classes

  • Source. A producer of data. The data could be in a file, coming from a socket, or in memory as a list. To access this data, we pull from the source.

  • Sink. A consumer of data. Basic examples would be a sum function (adding up a stream of numbers fed in), a file sink (which writes all incoming bytes to a file), or a socket. We push data into a sink. When the sink finishes processing it returns some value.

  • Conduit. A transformer of data. The simplest example is a map function. Like a sink, we push data into a conduit, but instead of returning a single value at the end, a conduit can return multiple outputs every time it is pushed to.

Something went wrong with that request. Please try again.