BUG How To notebook fails on instantiation SelfAskGptClassifier, chat_target invalid keyword

[corticalstack]

Describe the bug

Getting started with PyRit, fresh conda environment setup, install of pyrit with pip today.

Looking at the How To notebook:
https://github.com/Azure/PyRIT/blob/main/doc/how_to_guide.ipynb

Fails on trying the scoring (classification) example when:

classifier = SelfAskGptClassifier(content_classifier=str(SENTIMENT_CLASSIFIER), chat_target=target_llm)

As chat_target is not a valid keyword.

Versions

OS: Ubuntu 23.04 (Windows 11, WSL 2)
Python : 3.10.14

System:
python: 3.10.14 (main, Mar 21 2024, 16:24:04) [GCC 11.2.0]
executable: /root/miniconda3/envs/lhg_py310_pyrit/bin/python
machine: Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.37

Python dependencies:
pyrit: 0.1.2
Cython: None
numpy: 1.26.4
openai: 1.23.2
pip: 23.3.1
scikit-learn: 1.4.2
scipy: 1.13.0
setuptools: 68.0.0
tensorflow: None
torch: 2.1.2
transformers: 4.40.0

[romanlutz]

Thanks for filing an issue! Did you make sure to use the notebook from the repo version of PyRIT that corresponds to the release version of PyRIT?

You've got 0.1.2 so you need to match the notebook accordingly. I'm fairly sure that argument was changed since the latest release. So you're probably using the notebook from main. Please let me know if my guess is correct 🙂

[corticalstack]

Now that's interesting, and I wasn't aware there were diff versions of notebooks and diff branches. I installed Pyrit using pip, as per instructions today. Is PyPi release not updated? How would you suggest I can install latest version?

[corticalstack]

@romanlutz I've uninstalled, and installed latest from repo. Now have pyrit: 0.1.3.dev0. I'm now trying to complete a chat with an OpenAIChatTarget (not Azure), and get the following error:

'OpenAIChatTarget' object has no attribute 'complete_chat'

Looks like codebase changed since this example in feature request for OpenAI support, Would you have an example that works to test a prompt completion with OpenAI? Thanks!

[romanlutz]

We create releases every few weeks based on progress with features. I can't say when the next one will be yet but likely sooner rather than later.

Still, the main branch reflects the latest state of development. You can always assume that there's something new and it might not work with the latest release. That's why the repo has release tags. This is not unique to PyRIT but applies to every python project.

See contributor guide for Installation instructions for the dev version.

[romanlutz]

[Sorry your response came in while I was writing mine so it's slightly out of order.]

@rlundeen2 you may want to look into this. Sounds like recent refactoring missed something.

@corticalstack we should be able to get you a response pretty quickly.

Still, the main branch is expected to be more prone to such errors since it's not as polished as a release version. The latest pypi release with the corresponding notebook version should work. You can browse the repo at that version using the branch selector on the repo landing page.

[corticalstack]

No worries. Would use an earlier version but have prerequisite for OpenAIChatTarget, which looks like not available with earlier release 0.1.2?

[romanlutz]

It was already included in v0.1.2, see

https://github.com/Azure/PyRIT/blob/releases/v0.1.2/pyrit/prompt_target/openai_chat_target.py

[corticalstack]

Yep, I've had a brain burp day, thanks for your support!