Skip to content

IO Streaming#47

Merged
tsunaminoai merged 1 commit into
mainfrom
30-file-streaming
Mar 20, 2026
Merged

IO Streaming#47
tsunaminoai merged 1 commit into
mainfrom
30-file-streaming

Conversation

@sc2ben
Copy link
Copy Markdown
Contributor

@sc2ben sc2ben commented Mar 20, 2026

Added IO streaming for large documents.

Resolves #30

This pull request introduces streaming IO support to the Markdown parser and all built-in renderers, allowing efficient handling of large documents and direct output to any writer (such as files, sockets, or stdout) without intermediate allocation. It also fixes memory safety issues with dangling slices in the AST, improves documentation and usage examples, and updates performance benchmarks.

Streaming IO and Renderer API Enhancements:

  • All six built-in renderers (HTMLRenderer, ASTRenderer, AIRenderer, TerminalRenderer, MarkdownRenderer, TypstRenderer) now support a renderToWriter(Allocator, *std.Io.Writer, AST.Document) method for zero-allocation, streaming output. The Renderer interface and documentation are updated accordingly. (README.md, CHANGELOG.md, src/main.zig, src/markdown/renderers/*.zig, [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]

  • The parser now exposes parseFromReader(*std.Io.Reader), enabling parsing directly from streams (files, stdin, pipes, sockets) without requiring the entire input in memory. The returned AST is fully self-contained. (README.md, CHANGELOG.md, src/markdown/parser.zig, [1] [2] [3] [4]

Memory Safety and Bug Fixes:

  • Fixes dangling slice bugs in FencedCodeBlock.language and FootnoteDefinition.label by ensuring these fields are owned allocations, properly freed in their respective deinit methods. (CHANGELOG.md, src/markdown/ast.zig, src/markdown/parser.zig, [1] [2] [3] [4] [5]

Documentation and Examples:

  • Adds detailed streaming usage examples and updates the README to reflect the new streaming parser and renderer APIs, including code samples for both parsing from and rendering to streams. (README.md, README.mdR428-R466)

Performance Benchmarks:

  • Updates benchmark results in the README to reflect the new streaming architecture and increased input size, showing current memory and speed characteristics. (README.md, README.mdL669-L718)

Changelog and Versioning:

  • Updates the changelog to summarize the new streaming IO support, bugfixes, and infrastructure changes. (CHANGELOG.md, [1] [2]

@sc2ben sc2ben linked an issue Mar 20, 2026 that may be closed by this pull request
@tsunaminoai tsunaminoai added this to the V1.0.0 milestone Mar 20, 2026
@tsunaminoai tsunaminoai merged commit f27e572 into main Mar 20, 2026
2 checks passed
@tsunaminoai tsunaminoai deleted the 30-file-streaming branch March 20, 2026 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

File Streaming

2 participants