A simple proxy to send API requests to free llm sites with the right headers. Goal is to be a fake openAI API. Unicode is supported.
- Add other openai API options ✅
- Also send old messages ✅
- Return in openai json format ✅
pip install -r requirements.txt
python3 llama2-api.py
or
python3 blackbox-api.py
The API will start on http://127.0.0.1:5000/api
Model Options For llama2:
meta/llama-2-70b-chat
meta/llama-2-13b-chat
meta/llama-2-7b-chat
yorickvp/llava-13b
nateraw/salmonn
Model Options For blackbox:
blackbox-code
blackbox-chat
Other Options:
max_tokens (1-4096)
temperature (llama2 only)
top_p (llama2 only)
You can try the api with:
python3 terminal-test-api.py
or without interactivity:
python3 test-api.py
The prompts and commands were copied off of the openai docs. (link and model was changed to use proxy)