API for protocol buffers using modern Haskell language and library patterns.
Haskell Shell
Clone or download
judah Expose bindings for empty.proto. (#206)
We've found it useful to use that generic type, for example in RPCs that don't
return a value.
Latest commit 591a8fa Aug 10, 2018
Permalink
Failed to load latest commit information.
discrimination-ieee754 Bump lower-bounds to `base-4.9`. (#197) May 22, 2018
docs Scalar type refs (#204) Jul 30, 2018
google Initial version. May 3, 2016
lens-labels Bump lower-bounds to `base-4.9`. (#197) May 22, 2018
proto-lens-arbitrary Bump lower-bounds to `base-4.9`. (#197) May 22, 2018
proto-lens-benchmarks Split out proto-lens-protoc into three packages. (#203) Jul 16, 2018
proto-lens-combinators Split out proto-lens-protoc into three packages. (#203) Jul 16, 2018
proto-lens-discrimination Split out proto-lens-protoc into three packages. (#203) Jul 16, 2018
proto-lens-optparse Bump lower-bounds to `base-4.9`. (#197) May 22, 2018
proto-lens-protobuf-types Expose bindings for empty.proto. (#206) Aug 10, 2018
proto-lens-protoc Split out proto-lens-protoc into three packages. (#203) Jul 16, 2018
proto-lens-runtime Split out proto-lens-protoc into three packages. (#203) Jul 16, 2018
proto-lens-setup Split out proto-lens-protoc into three packages. (#203) Jul 16, 2018
proto-lens-tests-dep Split out proto-lens-protoc into three packages. (#203) Jul 16, 2018
proto-lens-tests Split out proto-lens-protoc into three packages. (#203) Jul 16, 2018
proto-lens-tutorial Split out proto-lens-protoc into three packages. (#203) Jul 16, 2018
proto-lens Bump lower-bounds to `base-4.9`. (#197) May 22, 2018
.gitignore Add instructions and a script for releasing proto-lens packages. (#186) May 7, 2018
.gitmodules Initial version. May 3, 2016
.travis.yml Fix #185: Support and use autogen-modules. (#196) May 22, 2018
CONTRIBUTING.md Update format links (#164) Dec 15, 2017
LICENSE Initial version. May 3, 2016
README.md Scalar type refs (#204) Jul 30, 2018
TYPES.md Scalar type refs (#204) Jul 30, 2018
bootstrap.hs Split out proto-lens-protoc into three packages. (#203) Jul 16, 2018
stack-bootstrap.yaml Field comments (#172) Jan 23, 2018
stack.yaml Split out proto-lens-protoc into three packages. (#203) Jul 16, 2018
travis-cabal.sh Split out proto-lens-protoc into three packages. (#203) Jul 16, 2018

README.md

Build Status

The proto-lens library provides an API for protocol buffers using modern Haskell language and library patterns. Specifically, it provides:

  • Composable field accessors via lenses
  • Simple field name resolution/overloading via type-level literals
  • Type-safe reflection and encoding/decoding of messages via GADTs

This is not an official Google product.

Tutorial

You can find tutorial documentation in the proto-lens-tutorial subdir.

There is also a reference document, TYPES.md, showing the protobuf scalar type -> haskell type mappings used by the generated lenses.

Instructions

Setup

First, install the "protoc" binary somewhere in your PATH. You can get it by following these instructions.

Building from HEAD

To build and test this repository from HEAD, run:

git submodule update --init --recursive
stack test

Note: building this repository requires stack-1.7.1 or newer.

Using in a Cabal or Stack package

proto-lens is available on Hackage and Stackage. Cabal and Stack projects can use it to auto-generate Haskell source files from the original protocol buffer specifications (.proto files).

Note: if using Stack, these instructions require v1.4.0 or newer.

First, edit the .cabal file of your project to:

  • Specify build-type: Custom, and add a custom-setup clause that depends on proto-lens-setup.
  • List the .proto files in extra-source-files. Note that the field belongs at the top level of the .cabal file, rather than once per library/executable/etc.
  • List the generated modules (e.g. Proto.Foo.Bar) in exposed-modules or other-modules of the rule(s) that use them (e.g. the library or executables).
  • Add proto-lens-runtime to the build-depends of those rules.
  • Add a custom-setup clause to your .cabal file.

For example, in foo-bar-proto.cabal:

...
build-type: Custom
extra-source-files: src/foo/bar.proto
...
custom-setup
  setup-depends: base, Cabal, proto-lens-setup

library
    exposed-modules: Proto.Foo.Bar, Proto.Foo.Bar_Fields
    autogen-modules: Proto.Foo.Bar, Proto.Foo.Bar_Fields
    build-depends: proto-lens-runtime, ...

(Note: if you do not have proto-lens-{runtime/setup}, you are probably using a version earlier than 0.4 and should replace those packages with proto-lens-protoc.)

Next, write a Setup.hs file that uses Data.ProtoLens.Setup and specifies the directory containing the .proto files. For example:

import Data.ProtoLens.Setup

main = defaultMainGeneratingProtos "src"

Then, when you run cabal build or stack build, Cabal will generate a Haskell file from each .proto file, and use it as part of building the library/executable/etc.

See the proto-lens-tests package for some more detailed examples.

Manually running the protocol compiler

Suppose you have a file foo.proto. Then to generate bindings, run:

protoc --plugin=protoc-gen-haskell=`which proto-lens-protoc` \
    --haskell_out=. foo.proto

This will generate a file Proto/Foo.hs which contains Haskell definitions corresponding to the messages and fields in the protocol buffer file.

Use --haskell_out to control the location of the output file.

Use --proto_path to specify the location of input .proto files. For example, suppose we have the files src/project/{foo,bar}.proto, and bar.proto has the line

import "project/foo.proto";

Then running:

protoc --plugin=protoc-gen-haskell=`which proto-lens-protoc` \
    --haskell_out=. \
    --proto_path=src \
    src/project/foo.proto src/project/bar.proto

will generate the haskell files Proto/Project/{Foo,Bar}.hs.

Current differences from the standard

  • Services are not supported.
  • Extensions (proto2-only) are not supported.
  • Unknown proto2 enum values cause a decoding error, instead of being preserved round-trip.
  • Messages with proto3 syntax preserve unknown fields, the same as for proto2. This behavior tracks a recent change to the specification.
  • Files with import public statements compile correctly, but don't explicitly reexport the definitions from those imports.

Troubleshooting

Rebuilding

Due to stack issue #1891, if you only change the .proto files then stack won't rebuild the package (that is, it won't regenerate the Proto.* modules).

Loading into ghci with Stack

stack ghci can get confused when trying to directly load a package that generates Proto.* modules (for example: stack ghci <proto-package>). To work around this issue, run instead:

stack exec ghci --package <proto-package>

And then manually import the generated modules within ghci, for example:

Prelude> import Proto.Foo.Bar
Prelude Proto.Foo.Bar>
...

Alternately, you can make those modules available at the same time as another local package, by running:

stack ghci <another-package> --package <proto-package>

Linking errors

Due to the limitations of how we can specify the dependencies of Setup files, stack may try to link them against the terminfo package. You may get an error from stack build similar to:

/usr/bin/ld: cannot find -ltinfo

On a Debian based system (like Ubuntu), the remedy is:

sudo apt-get install libncurses5-dev