feat: `GPT35Generator` #5714

ZanSara · 2023-09-04T14:53:16Z

Related Issues

related to OpenAI LLM Generators for Haystack 2.0 #5623

Proposed Changes:

Add ChatGPTGenerator
Add unit tests
Add end to end tests

How did you test it?

Local tests run
CI

Notes for the reviewer

openai already added to haystack-ai: deepset-ai/haystack-preview-package#2

Checklist

I have read the contributors guidelines and the code of conduct
I have updated the related issue with new insights and changes
I added unit tests and updated the docstrings
I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
I documented my code
I ran pre-commit hooks and fixed any issue

mathislucka · 2023-09-05T18:06:33Z

A small note on naming, a ChatGPTGenerator that does not support chat feels a bit confusing to me.

ZanSara · 2023-09-06T07:30:12Z

Good point 👀 Things should clear up as soon as the Agents proposal takes shape. At that point we will rework this very simple component into something more powerful and/or clarify better what it can and can't do. It is in no way feature complete, final or anything. It's just a simple component to make RAG possible in v2. Let's keep iterating!

@mathislucka @sjrl I might rename it GPT35Generator if that helps, but it looks a bit uglier to me. Wdyt?

mathislucka · 2023-09-06T08:19:01Z

@mathislucka @sjrl I might rename it GPT35Generator if that helps, but it looks a bit uglier to me. Wdyt?

That sounds better I think.

dfokina · 2023-09-06T10:19:17Z

@ZanSara hey I see there are some more places with ChatGPT name, like the name of the .py file or in docstrings 🙂

ZanSara · 2023-09-06T10:27:59Z

Thank you! I'm having another pass 🙌

releasenotes/notes/chatgpt-llm-generator-d043532654efe684.yaml

anakin87

Great work!

I took a general look and I have some high-level doubts.
(Probably many of these points are about legitimate implementation choices or future developments to be done in separate PRs...)

Although it has already been discussed, the naming sounds strange to me. I would not expect that I can use the GPT35Generator to run GPT4. Other options: OpenAIGenerator, OpenAIGPTGenerator,...
(Partly related to the previous point?) This component does not support the legacy Completions API, supported in Haystack 1.x. Although declared legacy, the Completions API will be probably retired in 2024. Don't we want to support it in Haystack 2.x? (It may well be a reasonable choice)
What happens if the user enters a wrong model name? Is it ok to let the OpenAI SDK raise an error or do we want to intercept it in Haystack?
What happens if the user sends a prompt that fills/exceeds the context window?
Maybe the OpenAI organization should be declared in __init__ and then used. See feat: support OpenAI-Organization for authentication #5292.
Azure ChatGPT Generator (corresponding to this invocation layer) will be addressed in another PR. Right?

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

ZanSara · 2023-09-06T14:45:51Z

Ok let's take them one by one:

Although it has already been discussed, the naming sounds strange to me. I would not expect that I can use the GPT35Generator to run GPT4. Other options: OpenAIGenerator, OpenAIGPTGenerator,...

I plan to make a subclass called GPT4Generator that simply sets another default model. I thought about OpenAIGenerator, but there are models that have a very different interface (think about davinci, if we ever want to support it) and in any case if tomorrow OpenAI releases another model we'll have naming issues. In any case rest assured that at least the GPT4 alias is coming asap.

(Partly related to the previous point?) This component does not support the legacy Completions API, supported in Haystack 1.x. Although declared legacy, the Completions API will be probably retired in 2024. Don't we want to support it in Haystack 2.x? (It may well be a reasonable choice)

If we do, I'd rather support them into a dedicated generator that we will be able to deprecate easily when the time comes.

What happens if the user enters a wrong model name? Is it ok to let the OpenAI SDK raise an error or do we want to intercept it in Haystack?

Good point! I'll check how the error looks like. I expect it to be quite clear, if it is I'll let it go across. Otherwise I'll figure out a way to communicate it to the user better..

What happens if the user sends a prompt that fills/exceeds the context window?

Right! I'll add a test!

Maybe the OpenAI organization should be declared in __init__ and then used. See feat: support OpenAI-Organization for authentication #5292.

Ah right! It got lost in the last refactoring. Let me bring it back.

Azure ChatGPT Generator (corresponding to this invocation layer) will be addressed in another PR. Right?

Yes Azure support is for the next PR 🙂

anakin87

👍

releasenotes/notes/chatgpt-llm-generator-d043532654efe684.yaml

haystack/preview/components/generators/openai/gpt35.py

* chatgpt backend * fix tests * reno * remove print * helpers tests * add chatgpt generator * use openai sdk * remove backend * tests are broken * fix tests * stray param * move _check_troncated_answers into the class * wrong import * rename function * typo in test * add openai deps * mypy * improve system prompt docstring * typos update * Update haystack/preview/components/generators/openai/chatgpt.py * pylint * Update haystack/preview/components/generators/openai/chatgpt.py Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com> * Update haystack/preview/components/generators/openai/chatgpt.py Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com> * Update haystack/preview/components/generators/openai/chatgpt.py Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com> * review feedback * fix tests * freview feedback * reno * remove tenacity mock * gpt35generator * fix naming * remove stray references to chatgpt * fix e2e * Update releasenotes/notes/chatgpt-llm-generator-d043532654efe684.yaml Co-authored-by: Daria Fokina <daria.fokina@deepset.ai> * add another test * test wrong model name * review feedback --------- Co-authored-by: Daria Fokina <daria.fokina@deepset.ai> Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

ZanSara added 6 commits September 4, 2023 15:12

chatgpt backend

a243cae

fix tests

f59abe8

reno

5f70a65

remove print

ffb1a8f

helpers tests

853f29d

add chatgpt generator

f0c5a8d

github-actions bot added the topic:tests label Sep 4, 2023

ZanSara changed the title ~~add chatgpt generator~~ feat: ChatGPTGenerator Sep 4, 2023

github-actions bot added the type:documentation Improvements on the docs label Sep 4, 2023

use openai sdk

0a25414

ZanSara mentioned this pull request Sep 4, 2023

feat: ChatGPTBackend #5711

Closed

ZanSara changed the base branch from chatgpt-llm-backend to main September 4, 2023 16:55

ZanSara added 3 commits September 4, 2023 18:58

remove backend

5105ae8

tests are broken

7d0c8e6

fix tests

de46d10

ZanSara marked this pull request as ready for review September 5, 2023 09:45

ZanSara requested review from a team as code owners September 5, 2023 09:45

ZanSara requested review from dfokina and vblagoje and removed request for a team September 5, 2023 09:45

ZanSara added 5 commits September 5, 2023 11:47

stray param

28d83f4

move _check_troncated_answers into the class

30b4bc3

wrong import

1b744e4

rename function

ab0e45c

typo in test

fc7dc05

ZanSara requested a review from silvanocerza September 5, 2023 09:54

add openai deps

3e43dcd

github-actions bot added the topic:dependencies label Sep 5, 2023

ZanSara added 2 commits September 5, 2023 19:02

freview feedback

835fd0c

reno

0eb43f9

ZanSara requested a review from silvanocerza September 5, 2023 17:09

remove tenacity mock

e8d92dd

gpt35generator

0aeb875

ZanSara changed the title ~~feat: ChatGPTGenerator~~ feat: GPT35Generator Sep 6, 2023

fix naming

9167e05

ZanSara requested a review from anakin87 September 6, 2023 09:30

ZanSara added 3 commits September 6, 2023 15:12

remove stray references to chatgpt

941cc66

fix e2e

04ec229

Merge branch 'main' into chatgpt-llm-generator

4eece1e

dfokina reviewed Sep 6, 2023

View reviewed changes

releasenotes/notes/chatgpt-llm-generator-d043532654efe684.yaml Outdated Show resolved Hide resolved

anakin87 reviewed Sep 6, 2023

View reviewed changes

Update releasenotes/notes/chatgpt-llm-generator-d043532654efe684.yaml

8fb06ae

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

ZanSara added 2 commits September 6, 2023 17:28

add another test

46385ac

Merge branch 'main' into chatgpt-llm-generator

812e8b9

anakin87 approved these changes Sep 6, 2023

View reviewed changes

releasenotes/notes/chatgpt-llm-generator-d043532654efe684.yaml Outdated Show resolved Hide resolved

haystack/preview/components/generators/openai/gpt35.py Outdated Show resolved Hide resolved

ZanSara added 3 commits September 6, 2023 17:48

test wrong model name

3ca3f73

review feedback

1015424

Merge branch 'main' into chatgpt-llm-generator

b79c7c1

ZanSara merged commit 63cbde7 into main Sep 7, 2023
72 of 73 checks passed

ZanSara deleted the chatgpt-llm-generator branch September 7, 2023 08:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: `GPT35Generator` #5714

feat: `GPT35Generator` #5714

ZanSara commented Sep 4, 2023 •

edited

mathislucka commented Sep 5, 2023

ZanSara commented Sep 6, 2023 •

edited

mathislucka commented Sep 6, 2023

dfokina commented Sep 6, 2023

ZanSara commented Sep 6, 2023

anakin87 left a comment

ZanSara commented Sep 6, 2023

anakin87 left a comment

feat: GPT35Generator #5714

feat: GPT35Generator #5714

Conversation

ZanSara commented Sep 4, 2023 • edited

Related Issues

Proposed Changes:

How did you test it?

Notes for the reviewer

Checklist

mathislucka commented Sep 5, 2023

ZanSara commented Sep 6, 2023 • edited

mathislucka commented Sep 6, 2023

dfokina commented Sep 6, 2023

ZanSara commented Sep 6, 2023

anakin87 left a comment

Choose a reason for hiding this comment

ZanSara commented Sep 6, 2023

anakin87 left a comment

Choose a reason for hiding this comment

feat: `GPT35Generator` #5714

feat: `GPT35Generator` #5714

ZanSara commented Sep 4, 2023 •

edited

ZanSara commented Sep 6, 2023 •

edited