Skip to content

Commit

Permalink
local llm guide - pytest
Browse files Browse the repository at this point in the history
  • Loading branch information
pchalasani committed Jun 9, 2024
1 parent 8d2cab6 commit 62945ff
Showing 1 changed file with 11 additions and 0 deletions.
11 changes: 11 additions & 0 deletions docs/tutorials/local-llm-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,3 +164,14 @@ llm_config = lm.OpenAIGPTConfig(
)
```

## Quick testing with local LLMs
As mentioned [here](https://langroid.github.io/langroid/tutorials/non-openai-llms/#quick-testing-with-non-openai-models),
you can run many of the [tests](https://github.com/langroid/langroid/tree/main/tests/main) in the main langroid repo against a local LLM
(which by default run against an OpenAI model),
by specifying the model as `--m <model>`,
where `<model>` follows the syntax described in the previous sections. Here's an example:

```bash
pytest tests/main/test_chat_agent.py --m ollama/mixtral
```
Of course, bear in mind that the tests may not pass due to weaknesses of the local LLM.

0 comments on commit 62945ff

Please sign in to comment.