-
Notifications
You must be signed in to change notification settings - Fork 3
Add --stream flag to ls/grep for SAX-style incremental XML output #4
Copy link
Copy link
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Description
When listing directories with thousands of files or running recursive grep on large codebases, aict currently buffers the entire result set in memory before writing XML. This can cause high memory usage and delayed output for the consumer. Adding a --stream flag would emit XML elements incrementally as they're discovered, similar to SAX-style event streaming.
Current Behavior
$ aict ls /usr/lib --xml
# Buffers all entries, then writes the complete <ls>...</ls> document at onceProposed Behavior
$ aict ls /usr/lib --stream --xml
# Writes <ls ...> opening tag immediately
# Writes each <file .../> element as it's discovered
# Writes </ls> closing tag when doneImplementation Notes
- Use
xml.NewEncoder(writer)instead ofxml.Marshalto stream elements - Open the root element with its attributes before processing begins
- Flush after each element so the consumer receives data incrementally
- Close the root element on completion or error
- For error cases mid-stream, emit an
<error>element before closing the root tag - Apply to
ls,grep, andfindinitially (the tools most likely to produce large result sets)
Example Streaming Output
<ls timestamp="1234567890" total_entries="unknown" path="/usr/lib" absolute="/usr/lib" stream="true">
<file name="libc.so" .../>
<file name="libm.so" .../>
<file name="libpthread.so" .../>
<!-- elements appear in real-time as files are discovered -->
<error code="13" msg="permission denied" path="/usr/lib/secret"/>
</ls>Note: total_entries would be unknown in stream mode since we don't know the count upfront.
Acceptance Criteria
-
--streamflag available onls,grep, andfind - First
<tool>opening tag is written before any processing begins - Each result element is flushed to stdout immediately
- Root element is properly closed on success and error
- Memory usage remains constant regardless of result set size
-
--streamis incompatible with--json(JSON requires knowing the array length upfront) — either error or fall back to buffered mode - Tests verify: streaming output, error mid-stream, empty results with stream flag
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request