Gecholog is an open-source LLM Gateway and Endpoint Manager, originally developed and contributed by the team at teneo.ai. This project is available under the MIT License.
Gecholog is written in Go, and it is recommended to run it containerized. You can find the latest container build in the public container repository on Docker Hub.
From the root of the source tree, run:
VERSION=$(git describe --tags | cut -d'-' -f1 | sed 's/^v//')
NATS_TOKEN=not_set
GUI_SECRET=not_set
docker build --secret id=NATS_TOKEN,env=NATS_TOKEN --secret id=GUI_SECRET,env=GUI_SECRET --no-cache --build-arg VERSION=$VERSION -f ./Dockerfile -t gecholog .
docker run -d -p 8080:8080 -p 5380:5380 -e GUI_SECRET=set_a_password --name gecholog gecholog
Then click http://localhost:8080/login and log in using the password you set. Start by running through the tutorials to learn the basic features of Gecholog.
docker run -d -p 8080:8080 -p 5380:5380 -e GUI_SECRET=set_a_password -e AISERVICE_API_BASE=https://your.openai.azure.com/ --name gecholog gecholog
To use environment variable references such as ${AISERVICE_API_BASE} in the configuration files.
docker run -d -p 8080:8080 -p 5380:5380 -e GUI_SECRET=set_a_password -v ./conf:/app/conf -v ./log:/app/log -v ./certs:/config/certs --name gecholog gecholog
To mount one directory for config files, one for log writing and one for cert files.
The Gecholog container contains the following services (see cmd/ folder)
entrypointprepares configuration files and starts theginitservice. Read more...ginitis the process supervisor and monitors configuration file changes. Read more...glis the actual API call routing service. Read more...guiruns the configuration UI. Read more...healthcheckis a simple service to confirm if a service is running. Read more...nats2loglistens to thenats-serverbus for log entries and writes them to a file, rest api, elastic or azure log analytics. Read more...nats-serveris the internal lightweight service bus. Kudos to the team at nats.io.tokencounteris a processor to standardize the token consumption fields in the logs. Read more...
If you want to manipulate (process) the payload, headers or response body you can connect your own processors to Gecholog for real-time or post-processing. Visit gecholog-resources for examples on how to mock an LLM API, load balancing or run natural language processing via Gecholog.
This project is licensed under the MIT License. See the LICENSE file for the full license text.
The Gecholog container build includes and utilizes NATS server, an open-source messaging system. Find license at /usr/share/doc/nats-server/LICENSE. We use NATS server under the terms of the Apache License 2.0. Our team acknowledges and appreciates the contributions of the developers and community behind NATS server to the open-source ecosystem. For details on the license, please find the Apache License 2.0 here.
Looking for an overview of the interface? Check it out!
