Skip to content

Feature: Add Streaming API #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
nrednav merged 22 commits into from
Aug 9, 2025
Merged

Feature: Add Streaming API #2

nrednav merged 22 commits into from
Aug 9, 2025

Conversation

@nrednav
Copy link
Owner

@nrednav nrednav commented Aug 9, 2025 •
edited
Loading

What's changed

Added

  • Added a new Streaming API that processes data in chunks, reducing peak memory
    usage when handling large datasets or network streams
    • Introduced Msgpack.encode_stream/2 to lazily encode a stream of Elixir
      terms one by one
    • Introduced Msgpack.decode_stream/2 to lazily decode a stream of
      MessagePack objects, capable of handling data that arrives in multiple
      chunks

nrednav added 21 commits August 9, 2025 20:06
@nrednav
create stream_decoder module
926770c
@nrednav
refactor decode to use default_options/0 instead of module attributes
1bf6855
@nrednav
move internal decoding logic into separate module
42c1443
@nrednav
implement StreamDecoder
c3d8d02
@nrednav
implement StreamEncoder
465afb2
@nrednav
add StreamEncoder and StreamDecoder to library api
6135348
@nrednav
add happy path test for stream decoding complete objects
d7fb1be
@nrednav
add boundary-case test for stream decoder
98fb0ac
@nrednav
refactor stream decoder to use Stream.unfold/2 for incomplete data test
246b0b6
@nrednav
add test suite for stream encoder
6f33f7a
@nrednav
overhaul StreamDecoder for robustness
3224531
@nrednav
add docs for StreamDecoder
14ad7ba
@nrednav
rename test helper
b75bff5
@nrednav
add docs for StreamEncoder
6e0c041
@nrednav
document streaming functions in library interface
f9cfa34
@nrednav
update readme
54ee7cd
@nrednav
define types and specs
133e691
@nrednav
add default_opts/0 to encoder module
3b419dc
@nrednav
add tests for public interface
1fafe37
@nrednav
update changelog
d64e7d1
@nrednav
use map_join instead of map |> join
4564c6d
@nrednav nrednav force-pushed the feature/streaming-api branch from 2b278b8 to 4564c6d Compare August 9, 2025 19:08
@nrednav nrednav merged commit 5e4b787 into dev Aug 9, 2025
2 checks passed
@nrednav nrednav deleted the feature/streaming-api branch August 9, 2025 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@nrednav