-
Notifications
You must be signed in to change notification settings - Fork 0
Use streaming I/O for markdown output instead of loading entire file into memory #4
Copy link
Copy link
Open
Description
Problem
readSymbolMarkdown currently reads the entire markdown cache file into memory before writing it to stdout. This was a workaround for a Zig stdlib bug on macOS where reader.stream() / reader.streamRemaining() fails with EBADF (errno 9).
The root cause is that Zig's std.Io.Writer.sendFile uses fcopyfile on macOS, which doesn't support pipes as the destination fd. When stdout is a pipe (e.g., gdoc Node3D | less), fcopyfile fails.
Current workaround
const content = try file.readToEndAlloc(allocator, 1024 * 1024);
defer allocator.free(content);
try output.writeAll(content);This works but allocates O(n) memory instead of streaming through a fixed buffer.
Desired solution
Use reader.streamRemaining(output) (or equivalent) to stream file content to the output writer with constant memory usage. This requires either:
- A fix upstream in Zig's
sendFile/fcopyfilehandling for pipe destinations on macOS - A custom streaming loop that bypasses
sendFile(read into fixed buffer, write to output, repeat)
Upstream
No existing issue found on codeberg.org/ziglang/zig for this specific bug. Consider filing one.
References
src/cache.zig:readSymbolMarkdownfunction- Zig stdlib:
std/fs/File.zigsendFile,std/Io/Writer.zigsendFile,std/Io/Reader.zigstream/streamRemaining
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels