John Snow Labs NLP Test 1.1.0: Announcing Support for Testing LLMs

[luca-martial]

---

📢 Overview

NLP Test 1.1.0 🚀 comes with brand new features, including: new capabilities for testing Large Language Models on Question Answering tasks, with support for testing OpenAI-based LLMs and support for robustness tests on the BoolQ and Natural Questions datasets!

A big thank you to our early-stage community for their contributions, feedback, questions, and feature requests 🎉

Make sure to give the project a star right here ⭐

---

🔥 New Features & Enhancements

• Support for testing OpenAI LLMs on Question Answering Adding support for openAI model testing for question-answering on several benchmark datasets #361
• Support for BoolQ and Natural Questions datasets Adding support for openAI model testing for question-answering on several benchmark datasets #361
• Improved layout for configuring tests Adding support for openAI model testing for question-answering on several benchmark datasets #361
• Improved warning and error messaging Adding support for openAI model testing for question-answering on several benchmark datasets #361

---

🐛 Bug Fixes

• Fixed overlapping and mis-formatted country names in dictionaries fix country names #347

---

❓ How to Use

Get started now! 👇

pip install nlptest

Create your test harness in 3 lines of code 🧪

# Set OpenAI API keys
os.environ['OPENAI_API_KEY'] = ''

# Import and create a Harness object
from nlptest import Harness
h = Harness(task='question-answering', model='gpt-3.5-turbo', hub='openai', data='BoolQ-test', config='config.yml')

# Generate test cases, run them and view a report
h.generate().run().report()

---

📖 Documentation

• NLP Test: Documentation
• NLP Test: Notebooks
• NLP Test: Test Types
• NLP Test: GitHub Repo

---

❤️ Community support

• Slack For live discussion with the NLP Test community, join the #nlptest channel
• GitHub For bug reports, feature requests, and contributions
• Discussions To engage with other community members, share ideas, and show off how you use NLP Test!

We would love to have you join the mission 👉 open an issue, a PR, or give us some feedback on features you'd like to see! 🙌

---

♻️ Changelog

What's Changed

• fix country names by @alytarik in fix country names #347
• Fix/country names by @alytarik in Fix/country names #348
• Adding support for openAI model testing for question-answering on several benchmark datasets by @chakravarthik27 in Adding support for openAI model testing for question-answering on several benchmark datasets #361
• update boolQ prompt by @ArshaanNazir in update boolQ prompt #366
• Chore: Website updates for LLM release by @luca-martial in Chore: Website updates for LLM release #369
• Update notebooks by @alytarik in Update notebooks #368
• Release/1.1.0 by @luca-martial in Release/1.1.0 #367

Full Changelog: v1.0.2...v1.1.0

---

This discussion was created from the release John Snow Labs NLP Test 1.1.0: Announcing Support for Testing LLMs.