LLM

A CLI utility and Python library for interacting with Large Language Models, both via remote APIs and models that can be installed and run on your own machine.

Run prompts from the command-line, store the results in SQLite, generate embeddings and more.

Full documentation: llm.datasette.io

Background on this project:

Installation

Install this tool using pip:

pip install llm

Or using pipx:

pipx install llm

Detailed installation instructions.

Getting started

If you have an OpenAI API key you can get started using the OpenAI models right away.

As an alternative to OpenAI, you can install plugins to access models by other providers, including models that can be installed and run on your own device.

Save your OpenAI API key like this:

llm keys set openai

This will prompt you for your key like so:

Enter key: <paste here>

Now that you've saved a key you can run a prompt like this:

llm "Five cute names for a pet penguin"

1. Waddles
2. Pebbles
3. Bubbles
4. Flappy
5. Chilly

Read the usage instructions for more.

Installing a model that runs on your own machine

LLM plugins can add support for alternative models, including models that run on your own machine.

To download and run Llama 2 13B locally, you can install the llm-mlc plugin:

llm install llm-mlc
llm mlc pip install --pre --force-reinstall \
  mlc-ai-nightly \
  mlc-chat-nightly \
  -f https://mlc.ai/wheels
llm mlc setup

Then download the 15GB Llama 2 13B model like this:

llm mlc download-model Llama-2-13b-chat --alias llama2

And run a prompt through it:

llm -m llama2 'difference between a llama and an alpaca'

You can also start a chat session with the model using the llm chat command:

llm chat -m llama2

Chatting with mlc-chat-Llama-2-13b-chat-hf-q4f16_1
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
>

Using a system prompt

You can use the -s/--system option to set a system prompt, providing instructions for processing other input to the tool.

To describe how the code a file works, try this:

cat mycode.py | llm -s "Explain this code"

Help

For help, run:

llm --help

You can also use:

python -m llm --help

License

petersenjoern/llm

ErrorLooks like something went wrong!

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Error
Looks like something went wrong!

Packages