Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
ebonnal committed Aug 3, 2023
1 parent d01c6b6 commit 6eec67f
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ with open("/path/to/file.text", "r") as text_file:

## Features
- define:
- `Pipe`'s constructor takes an `Iterator[T]` or `Iterable[T]` object as data source (The `Pipe` class extends itself `Iterator[T]`, so that on which you can call on a pipe object any function working with iterators: `set(pipe)`, `functools.reduce(func, pipe, inital)`, `itertools.islice(pipe, n_samples)`, etc...)
- The `.__init__` of the `Pipe` class takes as argument an instance of `Iterator[T]` or `Iterable[T]` used as the source of elements.
- `.map` over pipe's elements and yield the results as they arrive, optionally using multiple threads.
- `.flatten` a pipe, whose elements are assumed to be iterators, creating a new pipe with individual elements.
- `.filter` a pipe.
Expand All @@ -89,4 +89,10 @@ with open("/path/to/file.text", "r") as text_file:
- consume:
- `.collect` a pipe into a list having an optional max size.
- `.superintend` a pipe: iterate over it entirely while catching exceptions + logging the iteration process + collecting and raising error samples.

Note that the `Pipe` class itself extends `Iterator[T]`, hence you can pass a pipe to any function supporting iterators:
- `set(pipe)`
- `functools.reduce(func, pipe, inital)`
- `itertools.islice(pipe, n_samples)`
- ...

0 comments on commit 6eec67f

Please sign in to comment.