Skip to content

v0.5.1

Latest

Choose a tag to compare

@Azure99 Azure99 released this 31 Jan 07:23

New Features

  • Parquet I/O Across Engines: Added read_parquet/write_parquet support for Local/MultiProcess/Ray/Spark DataFrames and dataset loader.
  • LLM-Based Sample Filtering: Added LLMJudgeFilter for guidance-driven keep/drop decisions with metrics output.

What's Changed

  • refactor: Improve JSON extraction from markdown by using regex
  • feat: Add AGENTS.md contributor guide outlining and operator generator skill
  • feat: add ChatContextRoundTrimmer to trim chat context by rounds
  • feat: add dynamic max_rounds support to ChatMultiTurnSynthesizer
  • feat: add persona-based chat synthesis example
  • feat: implement LLM-based sample filter
  • feat: add MetadataClearer to clear metadata
  • test: add pytest suite and initial unit tests
  • test: add more unit tests
  • test: add unit tests for dataframe & dataset & provider
  • test: align test filenames with source modules
  • docs: update operator generate SKILL.md to include unit testing guidelines
  • fix: correct ray dataframe union and add spark/ray tests
  • feat: add parquet support across engines