Project Lightspeed is a live streaming system with very low latency. Project Lightspeed Chat provides a chat server to this ecosystem. The objective is to have a simple yet fully functional chat system that integrates with Lightspeed.
- Authentication: currently guests and authenticated users are supported, authentication is only supported via an Open ID Connect provider
- Persistence: messages and translations are persisted on the file system using BuntDB or SQLite, other persistence backends may be supported in the future
- Built-in translation support: dynamic message translations are fully supported by the chat server, a plugin may provide the actual translation text
- Plugins: Hashicorps' go-plugin is used to provide a generic plugin interface. Plugins can process incoming messages and emit messages and/or translations based on those, or they can emit messages regularly. Two plugins are part of this repository: "Base commands" for basic commands and a translation plugin using the Google translate API (separate configuration and setup of a Google Cloud Platform project is required)
- Configuration: viper is used for reading configuration files in TOML format
Maybe the whole system may one day be replaced by a Matrix server and an embeddable Matrix client.
For live video streaming, the parts of Project Lightspeed are required, that is Lightspeed-ingest and Lightspeed-webrtc.
As a frontend, grab the branch feature-external-chat from this fork of Lightspeed-react.
In order to try the chat, actually only the frontend is required.
Note: Building and running has only been tested on Linux platforms using Go v1.16. So: YMMV.
Run build-binaries-prod.sh in the main directory (note that due to the SQLite dependency, the resulting binary may not run on every glibc version).
It is also possible to use the vagrant docker image (Ubuntu 20.04) for building via build-binaries-prod-vagrant.sh, so the executables run on Ubuntu 20.04.
Alternatively:
cd cmd/lightspeed-chat
go build .
cd ../lightspeed-chat-admin
go build .Note to developers:
If the protobuf definition for communicating with the plugins in proto/message.proto has been changed, the corresponding go-files need to be regenerated:
go generateNote that a change in the protobuf definition is likely to require code changes in the main source as well as in the plugins.
Execute go build . in the directory where the source code of the plugin is located, f.e.
cd plugins/lightspeed-chat-google-translate-plugin
go build .
cd ../lightspeed-chat-base-commands-plugin
go build .Copy the binaries cmd/lightspeed-chat/lightspeed-chat and cmd/lightspeed-chat-admin/lightspeed-chat-admin to a convenient location:
sudo cp cmd/lightspeed-chat/lightspeed-chat /usr/local/bin
sudo cp cmd/lightspeed-chat-admin/lightspeed-chat-admin /usr/local/binCopy the binary of the plugin (
f.e. plugins/lightspeed-chat-google-translate-plugin/lightspeed-chat-google-translate-plugin) to a convenient
location:
mkdir -p ~/.config/lightspeed-chat/plugins
cp plugins/lightspeed-chat-google-translate-plugin/lightspeed-chat-google-translate-plugin ~/.config/lightspeed-chat/plugins
cp plugins/lightspeed-chat-base-commands-plugin/lightspeed-chat-base-commands-plugin ~/.config/lightspeed-chat/pluginsLightspeed-chat requires an extended version of the React frontend of the Lightspeed project (branch feature-external-chat found here).
git clone https://github.com/tcriess/Lightspeed-react.git
cd Lightspeed-react
git checkout feature-external-chat
npm install
npm run-script build # production built is in the "build/" directoryTo test locally:
npm startLightspeed-chat uses HCL as configuration language. The configuration of lightspeed-chat and its plugins can either be put into one HCL file or split into multiple HCL files in the same directory.
./cmd/lightspeed-chat/lightspeed-chat -c CONFIGURATIONwhere CONFIGURATION points to either a single configuration file or a directory, in the latter case all *.hcl files in this directory are combined.
[[oidc]]
name = "google"
client_id = "YOUR-CLIENT-ID"
provider_url = "https://accounts.google.com"
[history]
history_size = 1000
[persistence]
[persistence.buntdb]
global_name = "default.buntdb"
room_name_template = "room_{{ .RoomId }}.buntdb"
[persistence.sqlite]
dsn = "file:global.db?_fk=true"Optionally, lightspeed-chat can use an Open ID Connect provider to authenticate a user.
OIDC providers are configured in oidc-blocks, the required attributes are client_id and provider_url.
Note that currently the claim preferred_username is required and used as the user id and nick.
This may change in the future.
The immediate chat history is kept in memory, the length of the ring buffer for all events (messages, translations, etc) is provided in the history-block, the attributes are called history_size.
Note that one translation for each configured language is generated per chat message.
As a persistence backend, currently BuntDB and SQLite are supported.
BuntDB is an in-memory key/value store persisting the contents to (almost) plain-text files.
The block to configure persistence backends is called persistence, the sub-block buntdb requires the attribute global_name defining the file name (relative to the working directory or absolute path)
of the global database (containing users and rooms), and the attribute room_name_template for a file name template of the individual room databases (text/template, where {{.RoomId}} can be used as a placeholder for the room id).
SQLite is a file-based SQL database, the only required configuration parameters is the DSN connection string.
Note that if both SQLite and BuntDB are configured, SQLite is used. Potentially, BuntDB support will be removed in the future.
Lightspeed-chat ships with a plugin which provides google-translate translations.
In order to use it in a live system, you need to register a google cloud project, activate access to the google translate API and set up an OAuth2.0 client id for web applications.
Make sure to include the correct redirect URL(s) for your system (for local testing, this can be something like http://localhost:3000).
[[plugin]]
name = "google-translate"
project_id = "YOUR-PROJECT-ID"
languages = [
"de-DE",
"es-ES",
"en-US"]
cron_spec = "@every 60m"
cache_size = 10000Plugins are configured in a plugin-block labelled with the name of the plugin.
That is, the file name of the binary stripped of the leading lightspeed-chat- and the trailing -plugin.
Each plugin defines its own configuration requirements which have to be in a config-block.
The google translate plugin requires the google cloud project ID (string), the languages to translate into (list of strings), a cron specification (string) - the plugin sends "alive" chat messages according to this cron spec -, and a cache_size, as all translations are cached in-memory in an LRU-cache.
Note that in order to actually use the google translate API, the API credentials are also required, the environment variable GOOGLE_APPLICATION_CREDENTIALS needs to point to the corresponding JSON-file provided by google.
In the Lightspeed-react directory (branch feature-external-chat)
npm startThe frontend is now served on localhost:3000.
Without plugins, single configuration file:
./cmd/lightspeed-chat/lightspeed-chat -c config/config.hclWith the google translate plugin, multiple configuration files in a directory:
./cmd/lightspeed-chat/lightspeed-chat -p plugins/lightspeed-chat-google-translate-plugin/lightspeed-chat-google-translate-plugin -p plugins/lightspeed-chat-base-commands-plugin/lightspeed-chat-base-commands-plugin -c configFor (limited) administration of users and rooms, lightspeed-chat-admin is provided.
tbd