This repository contains bblfsh daemon (bblfshd), which includes the runtime that runs the driver in containers and the bblfshctl, a cli tool used to control the installed drivers and query the status of the daemon.
See the Getting Started guide.
The recommended way to run bblfshd by itself is using Docker:
docker run -d --name bblfshd \ -p 9432:9432 \ -v /var/lib/bblfshd:/var/lib/bblfshd \ -v /proc:/newproc \ --security-opt seccomp=./bblfshd-seccomp.json \ bblfsh/bblfshd
On macOS, use this command instead to use a Docker volume:
docker run -d --name bblfshd \ -p 9432:9432 \ -v bblfsh-storage:/var/lib/bblfshd bblfsh/bblfshd \ -v /proc:/newproc \ --security-opt seccomp=./bblfshd-seccomp.json \ bblfsh/bblfshd
To understand the flags
-v /proc:/newproc and
bblfshd-seccomp.json is a file present in this repo, and check
further requirements, please refer to rootless.md.
bblfshd is based on
and interacts with the kernel at a low level. It exposes a gRPC server at the port
9432 by default
which is used by the clients
to interact with the server. Also, we mount the path
all the driver images and container instances will be stored.
We advise against it, but if you prefer to run
privileged mode to skip configuration steps of
rootless.md, you could do, in Linux:
docker run -d --name bblfshd --privileged -p 9432:9432 -v /var/lib/bblfshd:/var/lib/bblfshd bblfsh/bblfshd
docker run -d --name bblfshd --privileged -p 9432:9432 -v bblfsh-storage:/var/lib/bblfshd bblfsh/bblfshd
Now you need to install the driver images into the daemon, you can install the official images just running the command:
docker exec -it bblfshd bblfshctl driver install --all
You can check the installed versions by executing:
docker exec -it bblfshd bblfshctl driver list
+----------+-------------------------------+---------+--------+---------+-----+-------------+ | LANGUAGE | IMAGE | VERSION | STATUS | CREATED | GO | NATIVE | +----------+-------------------------------+---------+--------+---------+-----+-------------+ | python | //bblfsh/python-driver:latest | v1.1.5 | beta | 4 days | 1.8 | 3.6.2 | | java | //bblfsh/java-driver:latest | v1.1.0 | alpha | 6 days | 1.8 | 8.131.11-r2 | +----------+-------------------------------+---------+--------+---------+-----+-------------+
To test the driver you can execute a parse request to the server with the
bblfshctl parse command,
and an example contained in the Docker image:
docker exec -it bblfshd bblfshctl parse /opt/bblfsh/etc/examples/python.py
If your system has SELinux enabled (which is the default in Fedora, Red Hat, CentOS
and many others) you'll need to compile and load a policy module before running the
bblfshd Docker image or running driver containers will fail with a
permission denied message in the logs.
To do this, run these commands from the project root:
cd selinux/ sh compile.sh semodule -i bblfshd.pp
If you were already running an instance of bblfshd, you will need to delete the
docker rm -f bblfshd) and run it again (
Once the module has been loaded with
semodule the change should persist even
if you reboot. If you want to permanently remove this module run
semodule -d bblfshd.
Alternatively, you could set SELinux to permissive module with:
echo 1 > /sys/fs/selinux/enforce
(doing this on production systems which usually have SELinux enabled by default should be strongly discouraged).
For local development of bblfshd, first make sure Go is properly installed and
that a GOPATH has been set. You will
also need to add
$GOPATH/bin to your
Next, using Git, clone this repository into
$GOPATH/src/github.com/bblfsh/bblfshd. All the necessary dependencies are
automatically installed, so you just need to type
make. This will compile the
code and then run the tests. If this exits with exit status 0, then everything
Ensure you have
ostree and development libraries for
Debian, Ubuntu, and related distributions:
$ apt-get install libostree-dev
Fedora, CentOS, RHEL, and related distributions:
$ yum install -y ostree-devel
Arch and related distributions:
$ pacman -S ostree
Building From Source
$ make build
Run tests with:
$ make test
BBLFSHD_MAX_DRIVER_INSTANCES- maximal number of driver instances for each language. Default to a number of CPUs.
BBLFSHD_MIN_DRIVER_INSTANCES- minimal number of driver instances that will be run for each language. Default to 1.
Bblfshd supports OpenTracing that can be used to profile request on a high level or trace individual requests to bblfshd and/or language drivers.
To enable it, you can use Jaeger. The easiest way is to start all-in-one Jaeger image:
docker run -d --name jaeger \ -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \ -p 5775:5775/udp \ -p 6831:6831/udp \ -p 6832:6832/udp \ -p 5778:5778 \ -p 16686:16686 \ -p 14268:14268 \ -p 9411:9411 \ jaegertracing/all-in-one:1.8
For Docker installation of bblfshd add the following flags:
--link jaeger:jaeger -e JAEGER_AGENT_HOST=jaeger -e JAEGER_AGENT_PORT=6831 -e JAEGER_SAMPLER_TYPE=const -e JAEGER_SAMPLER_PARAM=1
For bblfshd running locally, set following environment variables:
JAEGER_AGENT_HOST=localhost JAEGER_AGENT_PORT=6831 JAEGER_SAMPLER_TYPE=const JAEGER_SAMPLER_PARAM=1
Run few requests, and check traces at http://localhost:16686.
For enabling tracing in production, consult Jaeger documentation.
GPLv3, see LICENSE