v2.0.0

Breaking Changes

Heretic 2.0 is a major release that breaks backwards compatibility:

• Configuration files made for Heretic 1.x are not compatible with Heretic 2.0! See the migration guide for what you need to change in order to make your old configurations work.
• Research features have been removed. The intended replacement is Lophius.

Changes

• fix: correct total trial count when adding additional trials by @kali113 in #385
• feat: let the optimizer disable MLP ablation via a 0 max_weight floor by @rocker-zhang in #387
• fix: resolve UnicodeEncodeError on Windows during model evaluation by @umran666 in #389
• feat: headless operation + end-to-end tests by @p-e-w in #392
• fix: use W_org matrix only where needed by @Vinay-Umrethe in #398
• feat: generic plugin system by @red40maxxer in #53
• fix: validate scorer instance names in config by @Bortlesboat in #407
• feat: Add alternate CI hash in the valid list (for qwen2.5) by @Vinay-Umrethe in #412
• docs: quote pip extras install examples by @nyxst4ck in #395
• feat: add reproducible property to plugins by @red40maxxer in #414
• docs: update model count in README by @Vinay-Umrethe in #424
• chore: remove Gemini style guide by @p-e-w in #427
• chore: bump version to 2.0.0.dev0 by @p-e-w in #428
• feat: Show a summary at end of tests by @Vinay-Umrethe in #425
• feat: add benchmark scorer by @p-e-w in #444
• feat: support specifying a dataset's specific config/subset by @Vinay-Umrethe in #445
• feat: A better logic for detecting the thinking prefix in the response. by @Vinay-Umrethe in #423
• feat: Add base model information in the README.md model card by @Vinay-Umrethe in #459
• feat: modifier plugins + ARA by @p-e-w in #446 (contributions to ARA by @kabachuha, @joninco, and @coder3101)
• chore: update dependencies by @p-e-w in #469
• chore: add new Windows test hashes by @Vinay-Umrethe in #470
• fix: stop lora_A from carrying over between trials by @benthecarman in #476
• fix: prevent empty layer range and too many neighbors in ARA by @benthecarman in #478
• fix: escape prompts and responses printed with print_responses by @jayhemnani9910 in #482
• fix: silence warning spam from PyTorch's C++ logging by @p-e-w in #485
• fix: skip non-metrics in benchmark output by @p-e-w in #486
• fix: report a save or upload before re-ablating the model by @HamedShams in #488
• chore: add pull request template by @p-e-w in #502
• fix: say which check found the Chain-of-Thought prefix by @HamedShams in #489
• fix: restore stored plugin settings as they are on reproduce and resume by @jayhemnani9910 in #484
• fix: support models whose chat template rejects the system role by @varunbln in #498
• feat: make upload tags configurable by @officialayush5839-arch in #506
• ci: add Python 3.14 and 3.15 to CI matrix by @p-e-w in #511
• fix: reject blank refusal keyword markers by @jovial-liu in #512
• Support models with Hyper-Connections by @benthecarman in #519
• fix: Improve response prefix logic, handle default behaviors, and clarify documentation by @Vinay-Umrethe in #523
• fix: support flattened hyper-connection residual streams by @clee in #525
• perf(ara): compute closure outputs via low-rank update instead of dense weight by @varunbln in #499
• feat: add config.nosyco.toml by @UnstableLlama in #349
• fix: rename reproduce checksum file to SHA256SUMS.txt by @mvanhorn in #479
• feat: make adjective describing the type of model produced configurable by @p-e-w in #539
• feat: Generic BERT Classifier scorer plugin by @Vinay-Umrethe in #500
• docs: prepare for 2.0 release by @p-e-w in #531

New Contributors

• @kali113 made their first contribution in #385
• @Bortlesboat made their first contribution in #407
• @nyxst4ck made their first contribution in #395
• @benthecarman made their first contribution in #476
• @jayhemnani9910 made their first contribution in #482
• @HamedShams made their first contribution in #488
• @varunbln made their first contribution in #498
• @officialayush5839-arch made their first contribution in #506
• @jovial-liu made their first contribution in #512
• @clee made their first contribution in #525
• @mvanhorn made their first contribution in #479

Full Changelog: v1.4.0...v2.0.0