Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Release/1.5.0 #778

Merged
merged 196 commits into from
Sep 19, 2023
Merged

Release/1.5.0 #778

merged 196 commits into from
Sep 19, 2023

Conversation

ArshaanNazir
Copy link
Collaborator

Description

Release 1.5.0

📢 Overview

LangTest 1.5.0 Release by John Snow Labs 🚀: Debuting the Wino-Bias Test to scrutinize gender role stereotypes and unveiling an expanded suite with the Legal-Support, Legal-Summarization (based on the Multi-LexSum dataset), Factuality, and Negation-Sensitivity evaluations. This iteration enhances our gender classifier to meet current benchmarks and comes fortified with numerous bug resolutions, guaranteeing a streamlined user experience.

A heartfelt thank you to our unwavering community for consistently fueling our journey with their invaluable feedback, questions, and suggestions 🎉

Make sure to give the project a star right here ⭐

🔥 New Features & Enhancements

🐛 Bug Fixes

alytarik and others added 30 commits September 4, 2023 12:57
@alytarik
add simpler LLM evaluation for some datasets.
dfc0676
@alytarik
add other datasets
771c919
@alytarik
make comparison case insensitive
c6cfeb3
@RakshitKhajuria
chore(langtest.py): Updating the data when bias dataset loaded
ea50b4d
@ArshaanNazir
Add Wino-Bias data
1720cf8
@ArshaanNazir
update datasource
3bc503e
@ArshaanNazir
Add WinoBias Sample and default prompt
dd03dd3
@ArshaanNazir
update datasource
7dd9927
@ArshaanNazir
Update modelhandler
578f25b
@ArshaanNazir
Update sample and prompt
959ae40
@ArshaanNazir
Add WinoBiasTestFactory
89e2640
@ArshaanNazir
fix datasource and add wino default config
b25852f
@ArshaanNazir
fix wion data
2b8e9d7
@ArshaanNazir
Update datasource, sample and prompt
fbc9e0c
@ArshaanNazir
fix linting
636da2c
@RakshitKhajuria
chore(datasource.py): Loading Bias/XSum-bias data
37f2d17
@RakshitKhajuria
chore(langtest.py): Filter bias data in generate
3fcb10a
@RakshitKhajuria
chore(datasource.py): Warning added
09b011f
@Prikshit7766
task(sample.py): added SensitivitySample
225d0b3
@Prikshit7766
task(datasource.py): support for senstivity test
d9b50c7
@Prikshit7766
task(hf_model_n_tokenizer): load model and tokenizer
e43bc5b
@RakshitKhajuria
task(hf_model_n_tokenizer): Updated get model-tokenizer function
fe9d023
@RakshitKhajuria
task(tf_modelhandler): Added PretrainedModelForSensitivityTest
59dd0fa
@RakshitKhajuria
task(modelhandler.py): call PretrainedModelForSensitivityTest class
805c74a
@Prikshit7766
task(transform\__init__.py): added SensitivityTestFactory
cfb0940
@RakshitKhajuria
task(sensitivity.py): Added Abstract base class for implementing sens…
9ee2628 
…itivity measures.
@Prikshit7766
task(sensitivity.py): added SensitivityNegation
17637f6
@RakshitKhajuria
task(config): added sensitivity config
159e22a
@Prikshit7766
task(langtest.py): load senstivity config
7738757
@RakshitKhajuria
unittest(sensitivity): Setup SensitivityTestCase suit
a16bda9
ArshaanNazir and others added 26 commits September 19, 2023 10:56
@ArshaanNazir
update test.md
bcdf976
@ArshaanNazir
update task/one-liners/NB
0df9729
@ArshaanNazir
update one-liners
dad8449
@alytarik
remove unnecessary transformers
ef2bc4e
@ArshaanNazir
update transformers dependency
8dbcb76
@RakshitKhajuria
Chore(notebook): Removed langchain from pip install
8a799c7
@RakshitKhajuria
Chore(website): Removed langchain -> model/landing page
6d1fac3
@RakshitKhajuria
Chore(website): Removed langchain -> OneLiners
2166fd9
@alytarik
fix accuracy hf bug
050d2b2
@ArshaanNazir
Merge pull request #774 from JohnSnowLabs/fix/dependencies
f2df4c7 
update transformers dependency
@ArshaanNazir
Merge pull request #773 from JohnSnowLabs/limit-sensitivity-dataset
ed9c23a 
Limit sensitivity dataset
@ArshaanNazir
Merge pull request #775 from JohnSnowLabs/fix/accuracy-fairness-huggi…
1b1e5be 
…ngface-bug

fix accuracy hf bug
@RakshitKhajuria
Merge branch 'release/1.5.0' of https://github.com/JohnSnowLabs/langtest
4510eba 
 into docs/website-changes
@ArshaanNazir
Merge pull request #770 from JohnSnowLabs/docs/website-changes
fa5e67d 
Docs/website changes
@Prikshit7766
update jsl_modelhandler
b92d230
@Prikshit7766
update tutorials.md
40ba18b
@ArshaanNazir
Merge pull request #776 from JohnSnowLabs/entities-bug
b670710 
update jsl_modelhandler
@ArshaanNazir
update landing page
27c3cb4
@ArshaanNazir
update one-liner
a3cfaed
@Prikshit7766
updated notebook
e1a7396
@RakshitKhajuria
updated wino bias nb
279522d
@RakshitKhajuria
Merge branch 'chore/website_update' of https://github.com/JohnSnowLab…
704ba08 
…s/langtest into chore/website_update
@Prikshit7766
update one_liner.md
98a7444
@ArshaanNazir
update README
40505cd
@ArshaanNazir
Merge pull request #777 from JohnSnowLabs/chore/website_update
b83cfe3 
updating Website/Nbs
@ArshaanNazir
Change release version
10a4c24
@ArshaanNazir ArshaanNazir self-assigned this Sep 19, 2023
chakravarthik27
chakravarthik27 approved these changes Sep 19, 2023
View reviewed changes
Copy link
Collaborator

@chakravarthik27 chakravarthik27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 😄

@ArshaanNazir ArshaanNazir merged commit a630c5d into main Sep 19, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chakravarthik27 chakravarthik27 chakravarthik27 approved these changes

Assignees

@ArshaanNazir ArshaanNazir

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@ArshaanNazir @chakravarthik27 @alytarik @RakshitKhajuria @Prikshit7766