Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python Bindings: Improved unit tests, documentation and unification of API #1090

Merged
merged 13 commits into from
Jun 30, 2023

Conversation

AndriyMulyar
Copy link
Contributor

@AndriyMulyar AndriyMulyar commented Jun 28, 2023

Testing, documenting and refactoring the python bindings to:

  • isort and black
  • include inference unit tests for orca model
  • expose writable c api parameters
  • change streaming to return an iterable, remove top level generator method in favor is just generate
  • introduces a chat session context manager for fast inference during chats with prompt templates.
  • bug fix to top_k and top_p to make them actually work
image

@AndriyMulyar AndriyMulyar marked this pull request as ready for review June 30, 2023 18:53
find . | grep -E "(__pycache__|\.pyc|\.pyo$\)" | xargs rm -rf

black:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since these run in the venv perhaps make them depend on the venv target so the venv gets created when running them if it doesn't exist? (and either rename the dir or the target so that actually gets detected properly) / make sure the venv has these deps?

Copy link
Member

@apage43 apage43 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking a good bit friendlier to use :)

@AndriyMulyar AndriyMulyar merged commit 46a0762 into main Jun 30, 2023
2 of 5 checks passed
@AndriyMulyar AndriyMulyar deleted the python-bindings-love branch June 30, 2023 20:02
rlancemartin added a commit to langchain-ai/langchain that referenced this pull request Jul 4, 2023
Running `GPT4All` per the
[docs](https://python.langchain.com/docs/modules/model_io/models/llms/integrations/gpt4all),
I see:

```
$ from langchain.llms import GPT4All
$ model = GPT4All(model=local_path)
$ model("The capital of France is ", max_tokens=10)
TypeError: generate() got an unexpected keyword argument 'n_ctx'
```

It appears `n_ctx` is [no longer a supported
param](https://docs.gpt4all.io/gpt4all_python.html#gpt4all.gpt4all.GPT4All.generate)
in the GPT4All API from nomic-ai/gpt4all#1090.

It now uses `max_tokens`, so I set this.

And I also set other defaults used in GPT4All client
[here](https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/gpt4all/gpt4all.py).

Confirm it now works:
```
$ from langchain.llms import GPT4All
$ model = GPT4All(model=local_path)
$ model("The capital of France is ", max_tokens=10)
< Model logging > 
"....Paris."
```

---------

Co-authored-by: R. Lance Martin <rlm@Rs-MacBook-Pro.local>
@rlancemartin
Copy link

We should add unit test for LangChain in GPT4All.

Also, is a backwards compatible fix advised? My PR will force LangChain users to upgrade GPT4All. I welcome an alternative fix for users that upgrade LangChain, but do not have the upgraded GPT4All clinet.

@AndriyMulyar
Copy link
Contributor Author

AndriyMulyar commented Jul 4, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants