v0.8.0

Progress bars with rich

Progress tracking is now available for all metric computation and generation steps using rich progress bars. For the applicable methods, these progress bars can be turned on/off using the show_progress_bar boolean parameter.

Adversarial evaluations

We now offer two out-of-the-box adversarial evaluations with LangFair, for scenarios when users wish to explore worst-case generations for their LLM of choice. These evaluations, which include toxicity and counterfactual fairness assessments, are implemented using the AdversarialGenerator class.

1. Adversarial toxicity evaluation

This implementation generates responses with the provided LLM to prompts from the RealToxicityPrompts dataset.

2. Adversarial counterfactual evaluation

This implementation generates counterfactual responses with the provided LLM to adversarially designed counterfactual prompt sets. Off-the-shelf templates are available for a wide variety of attributes, including gender, race, age, etc.

What's Changed

• v0.7.1 updates by @dylanbouchard in #210
• Za/adversarial generator by @zeya30 in #205
• 171 adversarial generator by @dylanbouchard in #213
• chore(deps): bump actions/setup-python from 5.6.0 to 6.0.0 by @dependabot[bot] in #204
• add unit tests for adversarial generator by @zeya30 in #217
• Enable rich progress bars by @dylanbouchard in #216
• Minor release: v0.8.0 by @dylanbouchard in #231
• Release v0.8.0 (version bump) by @dylanbouchard in #232

Full Changelog: v0.7.1...v0.8.0