Skip to content

Release v0.0.3
Compare
Choose a tag to compare
@fracpete fracpete released this 10 Nov 03:31
· 108 commits to main since this release
v0.0.3
0986819
  • added the record-window filter
  • added the llm-registry tool for querying the registry from the command-line
  • added the replace_patterns method to ldc.text_utils module
  • added the replace-patterns filter
  • added -p/--pretty-print flag to to-alpaca writer
  • added pairs-to-llama2 and llama2-to-pairs filter (since llama2 has instruction as part of the string, it is treated as pretrain data)
  • added to-llama2-format filter for pretrain records (no [INST]...[/INST] block)
  • now requiring seppl>=0.0.8 in order to raise Exceptions when encountering unknown arguments
Assets 3