Skip to content

v0.2.1

Choose a tag to compare

@xinthink xinthink released this 22 Mar 07:25
· 3 commits to main since this release

What's New

JSON Lines Streaming Support

  • Memory-efficient bulk operations: --all flag now streams results in JSON Lines format instead of buffering
  • New --output flag: Write output directly to .json or .jsonl files
  • Pipe-friendly: Stream results to grep, jq, and other CLI tools

Changes

  • list_documents.py: Async generator pattern for streaming documents
  • list_tags.py: Same streaming improvements for tag listing
  • utils.py: New output_jsonlines() utility function

Usage Examples

# Stream all documents to grep
python scripts/list_documents.py --all | grep "python"

# Fetch all and save to file
python scripts/list_documents.py --location archive --all --output archive.jsonl

# Process with jq
python scripts/list_documents.py --all | tail -n +2 | head -n -1 | jq '.title'