Please add GTIRB

[eschulte]

A Common Lisp front end to the GrammaTech Intermediate Representation for Bianries (GTIRB). GTIRB is serialized using Google's protocol buffers. This library wraps the raw protocol buffer serialization with a more Lispy interface.

Git repository: https://github.com/grammatech/gtirb
Manual: https://grammatech.github.io/gtirb/

The ASDF file is located in cl/gtirb.asd in the git repository.

Please let me know if I can provide any additional information.

Thanks!

[quicklisp]

I get this: protoc-gen-lisp: program not found or is not executable. What should I do?

[eschulte]

See the relevant documentation at gtirb/cl/README.md. This is going to be an issue for anything that tries to use the Common Lisp protobuf package. Following the installation instructions from the protobuf package results in the creation of the protoc-gen-lisp executable, which processes protobuf .proto files and emits Common Lisp .lisp files. This all happens automatically on load via some nice ASDF functionality in the protobuf package.

[eschulte]

Actually, I notice that the protobuf package is in quicklisp. Is there a precedent for what a quicklisp installation does for a package like protobuf that requires an external non-lisp executables for subsequent use of the package?

[quicklisp]

I try to install non-lisp prerequisites for building if it isn't a big hassle. protoc-gen-lisp is a hassle - not sure if big or small yet. The directions in the protobuf Lisp source are complicated compared to a simple apt-get install foo.

[eschulte]

Yeah, it's a pain. And I believe it needs protobuf 3.7 which is newer than that installed on Ubuntu 18 by default. FWIW the following should work on Ubuntu18 (extracted from our CI setup; note that it expects to do all installs for the whole system).

# Install Protobuf
git clone https://github.com/google/protobuf -b v3.7.0 && \
    cd protobuf && \
    sh ./autogen.sh && \
    ./configure --prefix=/usr/local --enable-shared && \
    make && \
    make install && \
    cd .. && \
    rm -rf protobuf

# Common-Lisp specific setup.
sbcl --eval '(ql:quickload :protobuf)' --eval '(sb-ext:exit)'
git clone https://github.com/brown/protobuf
cd protobuf/protoc/lisp && INSTALL_ROOT=/usr/ PROTOC_ROOT=/usr/ make install

[quicklisp]

That is too much for me, sorry. For people who want to use the project, cloning the git repo is trivial compared to setting up the (non-Lisp) prerequisites.

[phoe]

@eschulte when it comes to complex foreign libs, @Shinmera's qt-libs solves the problem of buildinf foreign binaries by pre-building Windows/Linux/macOS libraries and then downloading/unzipping them when the respective ASDF system is loaded. I think this can work for both Google's protobuf binaries and the CL-specific protobuf system. Perhaps it will simplify the process of building your software to the point of making the system available for Quicklisp.

See https://github.com/Shinmera/qt-libs for details.

[Shinmera]

For smaller binaries I include the precompiled libraries directly in the repo. This isn't the case for qt-libs because the Qt binaries are over a hundred megs for each platform.

[eschulte]

I've mentioned some of the options here in a related issue on the protobuf repository: brown/protobuf#10.

Another option (mentioned in the above issue) would be to commit the .lisp files generated by ASDF by running this protoc-gen-lisp executable to the repository. Ideally ASDF could be configured to run protoc-gen-lisp if that program is found in the path, but to fall back to using the committed files otherwise. (I would suggest ASDF check mtimes on all files and decide whether or not to recompile that way, but git checkouts give random relative mtimes.)

[phoe]

Another option (mentioned in the above issue) would be to commit the .lisp files generated by ASDF by running this protoc-gen-lisp executable to the repository.

I don't think it's required. AFAIR, cl-unicode (included in Quicklisp) generates Lisp files from Unicode text file sources as a part of compiling the system, and these generated Lisp files are then compiled and loaded when someone loads the system.

What I mean is - the fact that Lisp files are generated by the system is not yet a no-go for including that system in Quicklisp.

[eschulte]

I just created a quicklisp branch of this package which will directly includes the generated lisp files. I think this is the best option for now as it should be trivial to load and I like the idea of maintaining an explicit quicklisp branch anyway because it ensures that we're more intentional about a stable version of the package being public.

Please use this branch as the source fo Quicklisp: https://github.com/GrammaTech/gtirb/tree/quicklisp

Longer-term it might be nice for either the protobuf package to distribute pre-compiled versions of proto-gen-lisp or for the protobuf asdf system to provide a way to use pre-compiled .lisp files.