John Snow Labs NLP Test 1.5.0: Amplifying Model Comparisons, Bias Tests, Runtime Checks, Harnessing HF Datasets for Superior Text Classification and Introducing Augmentation Proportion Control
📢 Overview
NLP Test 1.5.0 🚀 comes with brand new features, including: new capabilities to run comparisons between different models from same/different hubs in a single Harness for robustness, representation, bias, fairness and accuracy tests. It includes support for runtime checks and ability to pass custom replacement dictionaries for bias testing. Also added support for HF datasets for text classification task and many other enhancements and bug fixes!
A big thank you to our early-stage community for their contributions, feedback, questions, and feature requests 🎉
Make sure to give the project a star right here ⭐
🔥 New Features & Enhancements
- Adding support for Model Comparisons #514
- Adding support for passing custom replacement dictionaries #509
- Adding support for hf datasets for text classification task #511
- Adding support for runtime checks #515
- Adding support for Augmentation Proportion Control #506
- Adding new tutorial notebooks #526
🐛 Bug Fixes
- Review issues with add-context for QA #507
❓ How to Use
Get started now! 👇
pip install nlptest
Create your test harness in 3 lines of code 🧪
# Defining a dictionary to run model comparisons
models = {
"ner.dl": "johnsnowlabs",
"en_core_web_sm": "spacy"
}
# Import and create a Harness object
from nlptest import Harness
h = Harness(task='ner', model=models, data='/Path-to-test-conll')
# Generate test cases, run them and view a report
h.generate().run().report()
📖 Documentation
❤️ Community support
- Slack For live discussion with the NLP Test community, join the
#nlptestchannel - GitHub For bug reports, feature requests, and contributions
- Discussions To engage with other community members, share ideas, and show off how you use NLP Test!
We would love to have you join the mission 👉 open an issue, a PR, or give us some feedback on features you'd like to see! 🙌
♻️ Changelog
What's Changed
- Fix/context-issue by @RakshitKhajuria in #507
- supports custom proportions for augument by @chakravarthik27 in #506
- Feature/ Add option to pass custom replacement dictionaries for bias tests by @RakshitKhajuria in #509
- feature/Add support for hf datasets for text classification task by @Prikshit7766 in #511
- test/hf-load-dataset by @Prikshit7766 in #517
- Features/model comparisons by @ArshaanNazir in #514
- Docs/nb docs update by @RakshitKhajuria in #518
- Feature/add runtime tests by @chakravarthik27 in #515
- Restructure quac dataset by @Prikshit7766 in #508
- Fix/runtime compare conflict by @alytarik in #522
- fix bug for runtime tests by @alytarik in #523
- fix coloring by @alytarik in #524
- support of hf dataset for jsl and spacy by @RakshitKhajuria in #521
- Chore/website updates by @ArshaanNazir in #519
- updated time unit in report() by @chakravarthik27 in #520
- augmentation and runtime tests nb by @chakravarthik27 in #525
- Chore/tutorial nbs and website updates by @ArshaanNazir in #526
- Release/1.5.0 by @ArshaanNazir in #527
Full Changelog: v1.4.0...v1.5.0
