What's New
JSON Lines Streaming Support
- Memory-efficient bulk operations:
--all flag now streams results in JSON Lines format instead of buffering
- New
--output flag: Write output directly to .json or .jsonl files
- Pipe-friendly: Stream results to
grep, jq, and other CLI tools
Changes
list_documents.py: Async generator pattern for streaming documents
list_tags.py: Same streaming improvements for tag listing
utils.py: New output_jsonlines() utility function
Usage Examples
# Stream all documents to grep
python scripts/list_documents.py --all | grep "python"
# Fetch all and save to file
python scripts/list_documents.py --location archive --all --output archive.jsonl
# Process with jq
python scripts/list_documents.py --all | tail -n +2 | head -n -1 | jq '.title'