Skip to content

v1.6.0-rc1

Choose a tag to compare

@nelly-hateva nelly-hateva released this 15 Jan 06:40
· 29 commits to main since this release
c136234

What's Changed

  • #307:
    • Update cim-subset-pretty.ttl to the latest one
    • Changes in the system prompt related to the N-Shot tool usage:
      • we tell the LLM to retrieve 10 samples, not at least 10
      • we tell it to call the tool multiple times for questions consisting of
        multiple independent parts
      • we tell it to add the class prefixes when doing the parametrization,
        because the prefixes are indexed

Evaluation results with gpt-5.2-2025-12-11

Split Micro Macro (Micro) Mean Tokens (Micro) Mean Time
Dev 0.9105 0.8960 307456.3 18.96
Test 0.8333 0.84933 312005.4 23.17

by @nelly-hateva in #79

Full Changelog: 1.5.0-rc1...v1.6.0-rc1