Babelfish driver SDK
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

sdk Build Status codecov license GitHub release

Babelfish SDK contains the tools and libraries required to create a Babelfish driver for a programming language.



The Babelfish SDK has the following dependencies:

Make sure that you've correctly set your GOROOT and GOPATH environment variables.


Babelfish SDK gets installed using either Go:

$ go get -t -v

or make command:

$ make install

These commands will install both bblfsh-sdk and bblfsh-sdk-tools programs at $GOPATH/bin/.


The SDK provides scaffolding templates for creating a new language driver. These templates are converted to Go code that ends up in bblfsh-sdk tool. Use make to update these templates:

$ make
go get -v
go get -v
cat protocol/internal/testdriver/main.go | sed -e 's|\([[:space:]]\+\).*//REPLACE:\(.*\)|\1\2|g' \
	> etc/skeleton/driver/main.go.tpl
chmod -R go=r ${GOPATH}/src/; \
go-bindata \
	-pkg build \
	-modtime 1 \
	-nocompress \
	-prefix ${GOPATH}/src/ \
	-o assets/build/bindata.go \
chmod -R go=r ${GOPATH}/src/; \
go-bindata \
	-pkg skeleton \
	-modtime 1 \
	-nocompress \
	-prefix ${GOPATH}/src/ \
	-o assets/skeleton/bindata.go \

You can validate this process has been properly done before submitting changes:

$ make validate-commit

If the code has not been properly generated, this command will show a diff of the changes that have not been processed and will end up with a message like:

generated bindata is out of sync
make: *** [Makefile:66: validate-commit] Error 2

Review the process if this happens.

On the other hand, If you need to regenerate proto and proteus files, you must run go generate from protocol/ directory:

$ cd protocol/
$ go generate

It regenerates all proto and proteus files under protocol/ and uast/ directories.


Babelfish SDK helps both setting up the initial structure of a new driver and keeping that structure up to date.

Creating the driver's initial structure

Let's say we're creating a driver for mylang. The first step is going to the location where we want the repository for the driver to be bootstrapped:

$ cd $GOPATH/src/

Now the driver should be bootstrapped with bblfsh-sdk. This will create a git repository, and some directories and files required by every driver. They will be overwritten if they exist, like the file in the example below.

$ bblfsh-sdk init mylang alpine
initializing driver "mylang", creating new manifest
creating file "manifest.toml"
creating file "Makefile"
creating file "driver/main.go"
creating file "driver/normalizer/normalizer.go"
creating file ".git/hooks/pre-commit"
creating file ".gitignore"
creating file ".travis.yml"
creating file ""
creating file "driver/normalizer/normalizer_test.go"
creating file "Dockerfile.tpl"
creating file "LICENSE"
managed file "" has changed, discarding changes
$ git add -A
$ git commit -m 'initialize repository'

Note that this adds a pre-commit git hook, which will verify these files are up to date before every commit and will disallow commits if some of the managed files are changed. You can by-pass this with git commit --no-verify.

You can find the driver skeleton used here at etc/skeleton.

Keeping managed files updated

Whenever the managed files are updated, drivers need to update them. bblfsh-sdk can be used to perform some of this updates in managed files. For example, if the README template is updated, running bblfsh-sdk update will overwrite it.

$ bblfsh-sdk update
managed file "" has changed, discarding changes

bblfsh-sdk doesn't update the SDK itself.

For further details of how to construct a language driver, take a look at Implementing the driver section in documentation.

Testing the driver

bbflsh-sdk also includes a testing framework for a driver. In order to run test for a particular dirver, change to it's directory and run:

$ bblfsh-sdk test

This will:

  • compile a "test binary" that parses content of the ./fixtures directory of the driver
  • create a docker image with all dependencies, native driver and a test binary
  • run this test binary inside a Docker container, using that image

Overall, SDK supports 3 different kind of tests for a driver:

  • UnitTests, parsing content of ./fixtures and applying UAST transformations. Described above.
  • Integration tests, using content of ./fixtures/_integration*
  • Benchmarks, using content of ./fixtures/bench_*

First two always run, benchmarks are only triggered by bblfsh-sdk test --bench.