You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
pairs-to-llama2 filter now has an optional --prefix parameter to use with the instruction
added the pretrain-sentences-to-pairs filter for generating artificial prompt/response datasets from pretrain data
requires seppl>=0.0.11 now
the LDC_MODULES_EXCL environment variable is now used for specifying modules to be excluded from the registration process (e.g., used when generating help screens for derived libraries that shouldn't output the base plugins as well)
llm-registry and llm-help now allow specifying excluded modules via -e/--excluded_modules option
to-alpaca writer now has the -a/--ensure_ascii flag to enforce ASCII compatibility in the output
added global option -u/--update_interval to convert tool to customize how often progress of # records processed is being output in the console (default: 1000)
text-length filter now handles None values, i.e., ignores them
locations (i.e., input/instructions/output/etc) can be specified now multiple times
the llm-help tool can generate index files for all the plugins now; in case of markdown it will link to the other markdown files