Skip to content

v1.4.5

  • v1.4.5
  • 3df3bbc
  • Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
  • Choose a tag to compare

  • v1.4.5
  • 3df3bbc
  • Choose a tag to compare

  • Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
@ikreymer ikreymer tagged this 25 Jun 22:17
- Fix a bug where SpooledTemporaryFile() did not have a max buffer size, resulting in reading everything into memory!

- Optimize sort: also add max size (currently 32MB) for amount of CDXJ lines to hold in memory. If size is exceeded, write to temp files and use heapq.merge to merge temp files into final output.

- Allow setting max sort buffer size with 'max_sort_buff_size' param, set to lower number for testing

- Add encoding=utf-8 to cdxj output
Assets 2
Loading