-
Notifications
You must be signed in to change notification settings - Fork 13.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug/4299 cant instantiate simplechatmodel subclass without defining agenerate #4300
Conversation
@@ -197,6 +197,14 @@ def _generate( | |||
generation = ChatGeneration(message=message) | |||
return ChatResult(generations=[generation]) | |||
|
|||
def _agenerate( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR!
This will have to be an async method. Will defer to @agola11 on whether it makes sense to have it default to
await asyncio.get_event_loop().run_in_executor(self._generate(messages, stop=stop, run_manager=run_manager)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, this needs to be async and we should not be calling blocking code in async methods. A good solution is having the default implementation for async def _agenerate
call run_in_executor, as @vowelparrot suggested
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how does this look
…chatmodel-subclass-without-defining-agenerate
…chatmodel-subclass-without-defining-agenerate
…chatmodel-subclass-without-defining-agenerate
Coming back to this tomorrow |
believe this was resolved in #4701, let me know if i'm missing something! |
Fixes #4299