API with Python

The get_completion() function below sits on top of the endpoint description at API Specs. It is a thin layer and does not implement all the packet options described in the Configurable parameters section of the documentation. But you can easily add those by extending the data dictionary near the top of the function.

To use this get_completion() function, you must have installed local.ai, downloaded an LLM and started the local.ai server.

import requests
import json

def get_completion(prompt, max_tokens=512, temperature=0.6):
    global response
    url = 'http://localhost:8000/v1/completions'
    headers = {
        'Content-Type': 'application/json'
    }
    data = {
        "prompt": prompt,
        "max_tokens": max_tokens,
        "temperature": temperature,
    }

    response = requests.post(url, headers=headers, data=json.dumps(data))

    completion = ''
    if response.status_code == 200:
        content = response.content.decode('ISO-8859-1')
        lines = content.split('\n')
        for line in lines:
            if line.startswith('data:'):
                if line.endswith('[DONE]'):
                    break
                else:
                    choice = json.loads(line[6:])
                    txt = choice['choices'][0]['text']
                    completion += txt
        return(completion)
    else:
        return(None)

Calling the LLM it then trivial:

val = get_completion('What are the capitals of Texas and Arizona?')
print(val)

to which the LLM responds:

The capitals of Texas is Austin, while that of Arizona is Phoenix.