Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .formatter.exs
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@

[
inputs: ["mix.exs", "{config,lib,test}/**/*.{ex,exs}"],
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"],
locals_without_parens: [field: 2, field: 3, oneof: 2, extend: 4, extensions: 1],
export: [
locals_without_parens: [field: 2, field: 3, oneof: 2, extend: 4, extensions: 1]
Expand Down
8 changes: 4 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ erl_crash.dump
# Also ignore archive artifacts (built via "mix archive.build").
*.ez

protoc-gen-elixir
# Ignore package tarball (built via "mix hex.build").
protobuf-*.tar

# QuickCheck artifacts
# Misc.
protoc-gen-elixir
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a QuickCheck artifact, is it? It's the protoc compiler plugin

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a QuickCheck artifact, is it? It's the protoc compiler plugin

I will put under # Misc..

.eqc-info
*.eqc

.elixir_ls
107 changes: 59 additions & 48 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,22 @@
# protobuf-elixir

[![Hex.pm](https://img.shields.io/hexpm/v/protobuf.svg)](https://hex.pm/packages/protobuf)
[![Build Status](https://travis-ci.org/tony612/protobuf-elixir.svg?branch=master)](https://travis-ci.org/tony612/protobuf-elixir)
[![CI](https://github.com/elixir-protobuf/protobuf/actions/workflows/main.yml/badge.svg)](https://github.com/elixir-protobuf/protobuf/actions/workflows/main.yml)

A pure Elixir implementation of [Google Protobuf](https://developers.google.com/protocol-buffers/)
A pure Elixir implementation of [Google Protobuf](https://developers.google.com/protocol-buffers/).

## Why this instead of exprotobuf(gpb)?

It has some must-have and other cool features like:

1. A protoc [plugin](https://developers.google.com/protocol-buffers/docs/cpptutorial#compiling-your-protocol-buffers) to generate Elixir code just like what other official libs do, which is powerful and reliable.
2. Generate **simple and explicit** code with the power of Macro. (see [test/support/test_msg.ex](https://github.com/tony612/protobuf-elixir/blob/master/test/support/test_msg.ex))
2. Generate **simple and explicit** code with the power of Macro. See [test/support/test_msg.ex](https://github.com/tony612/protobuf-elixir/blob/master/test/support/test_msg.ex).
3. Plugins support. Only [grpc](https://github.com/tony612/grpc-elixir) is supported now.
4. Use **structs** for messages instead of Erlang records.
5. Support Typespec in generated code.

## Installation

The package can be installed by adding `protobuf` to your list of dependencies in `mix.exs`:
The package can be installed by adding `:protobuf` to your list of dependencies in `mix.exs`:

```elixir
def deps do
Expand Down Expand Up @@ -51,40 +50,47 @@ end

### Generate Elixir code

1. Install `protoc`(cpp) [here](https://github.com/google/protobuf/blob/master/src/README.md) or `brew install protobuf` on MacOS.
2. Install protoc plugin `protoc-gen-elixir` for Elixir . NOTE: You have to make sure `protoc-gen-elixir`(this name is important) is in your PATH.
```
$ mix escript.install hex protobuf
```
3. Generate Elixir code using protoc
```
$ protoc --elixir_out=./lib helloworld.proto
```
4. Files `helloworld.pb.ex` will be generated, like:
1. Install `protoc`(cpp) [here](https://github.com/google/protobuf/blob/master/src/README.md) or
`brew install protobuf` on MacOS.

```elixir
defmodule Helloworld.HelloRequest do
use Protobuf, syntax: :proto3
2. Install protoc plugin `protoc-gen-elixir` for Elixir . NOTE: You have to
make sure `protoc-gen-elixir`(this name is important) is in your PATH.

@type t :: %__MODULE__{
name: String.t
}
defstruct [:name]
```bash
$ mix escript.install hex protobuf
```

field :name, 1, type: :string
end
3. Generate Elixir code using protoc

defmodule Helloworld.HelloReply do
use Protobuf, syntax: :proto3
```bash
$ protoc --elixir_out=./lib helloworld.proto
```

@type t :: %__MODULE__{
message: String.t
}
defstruct [:message]
4. Files `helloworld.pb.ex` will be generated, like:

field :message, 1, type: :string
end
```
```elixir
defmodule Helloworld.HelloRequest do
use Protobuf, syntax: :proto3

@type t :: %__MODULE__{
name: String.t
}
defstruct [:name]

field :name, 1, type: :string
end

defmodule Helloworld.HelloReply do
use Protobuf, syntax: :proto3

@type t :: %__MODULE__{
message: String.t
}
defstruct [:message]

field :message, 1, type: :string
end
```

### Encode and decode in your code

Expand Down Expand Up @@ -118,32 +124,38 @@ $ protoc --elixir_out=plugins=grpc:./lib/ *.proto

### Tips for protoc

- Custom protoc-gen-elixir name or path using `--plugin`
```
Custom protoc-gen-elixir name or path using `--plugin`:

```bash
$ protoc --elixir_out=./lib --plugin=./protoc-gen-elixir *.proto
```
- Pass `-I` argument if you import other protobuf files
```

Pass `-I` argument if you import other protobuf files:

```bash
$ protoc -I protos --elixir_out=./lib protos/hello.proto
```

### Custom options

Since extensions(`Protobuf.Extension`) is supported now, some options are defined, like custom module_prefix.
Since extensions(`Protobuf.Extension`) is supported now, some options are
defined, like custom module_prefix.

1. Copy src/elixirpb.proto to your protos path
2. Import elixirpb.proto and use the options
1. Copy `src/elixirpb.proto` to your protos path.

```proto
syntax = "proto2";
2. Import `elixirpb.proto` and use the options.

package your.pkg;
```proto
syntax = "proto2";

import "elixirpb.proto";
package your.pkg;

option (elixirpb.file).module_prefix = "Foo.Bar";
```
3. Generate code as before
import "elixirpb.proto";

option (elixirpb.file).module_prefix = "Foo.Bar";
```

3. Generate code as before

More options will be added in the future, see elixirpb.proto comments for details.

Expand All @@ -163,7 +175,6 @@ mix test

<img src="https://user-images.githubusercontent.com/1253659/84641850-3f163d80-af2e-11ea-98a2-cfb854180222.png" height="80">


## Acknowledgements

Many thanks to [gpb](https://github.com/tomas-abrahamsson/gpb) and
Expand Down
18 changes: 11 additions & 7 deletions lib/protobuf.ex
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
defmodule Protobuf do
@moduledoc """
`protoc` should always be used to generate code instead of wrting the code by hand.
`protoc` should always be used to generate code instead of writing the code by hand.

By `use` this module, macros defined in `Protobuf.DSL` will be injected. Most of thee macros
are equal to definition in .proto files.
Expand All @@ -25,10 +25,10 @@ defmodule Protobuf do
Except functions in "Callbacks", some other functions may be defined:

* Extension functions when your Protobuf message use extensions. See `Protobuf.Extension` for details.
* put_extension(struct, extension_mod, field, value)
* get_extension(struct, extension_mod, field, default \\ nil)
"""
* `put_extension(struct, extension_mod, field, value)`
* `get_extension(struct, extension_mod, field, default \\ nil)`

"""
defmacro __using__(opts) do
quote location: :keep do
import Protobuf.DSL, only: [field: 3, field: 2, oneof: 2, extend: 4, extensions: 1]
Expand Down Expand Up @@ -73,7 +73,9 @@ defmodule Protobuf do
"""
@callback new() :: struct

@doc "Build and update the struct with passed fields."
@doc """
Build and update the struct with passed fields.
"""
@callback new(Enum.t()) :: struct

@doc """
Expand All @@ -97,19 +99,21 @@ defmodule Protobuf do
@callback decode(binary) :: struct

@doc """
It's preferable to use message's `decode` function, like
It's preferable to use message's `decode` function, like:

Foo.decode(bin)

"""
@spec decode(binary, module) :: struct
def decode(data, mod) do
Protobuf.Decoder.decode(data, mod)
end

@doc """
It's preferable to use message's `encode` function, like
It's preferable to use message's `encode` function, like:

Foo.encode(foo)

"""
@spec encode(struct) :: binary
def encode(struct) do
Expand Down
4 changes: 2 additions & 2 deletions lib/protobuf/json.ex
Original file line number Diff line number Diff line change
Expand Up @@ -74,15 +74,15 @@ defmodule Protobuf.JSON do
Currently, the `protoc` compiler won't check for field name collisions, this library either.
So make sure your field names will be unique when serialized to JSON. For instance, this
message definition will not encode correctly, it will emit just one of the two fields, and
the problem might go unoticed:
the problem might go unnoticed:

message CollidingFields {
int32 f1 = 1 [json_name = "sameName"];
float f2 = 2 [json_name = "sameName"];
}

According to the specification, when duplicated JSON keys are found in maps, the library
should raise a decoding error. It currently ignores duplicates and keeps the last occurence.
should raise a decoding error. It currently ignores duplicates and keeps the last occurrence.
"""

alias Protobuf.JSON.{Encode, EncodeError, Decode, DecodeError}
Expand Down
5 changes: 3 additions & 2 deletions lib/protobuf/protoc/cli.ex
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
defmodule Protobuf.Protoc.CLI do
@moduledoc """
protoc plugin for generating Elixir code
protoc plugin for generating Elixir code.

See `protoc -h` and protobuf-elixir for details.
NOTICE: protoc-gen-elixir(this name is important) must be in $PATH

**NOTICE:** protoc-gen-elixir(this name is important) must be in `$PATH`.

## Examples

Expand Down
19 changes: 15 additions & 4 deletions mix.exs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
defmodule Protobuf.Mixfile do
use Mix.Project

@source_url "https://github.com/elixir-protobuf/protobuf"
@version "0.8.0-beta.1"

def project do
Expand All @@ -17,7 +18,8 @@ defmodule Protobuf.Mixfile do
description: description(),
package: package(),
aliases: aliases(),
preferred_cli_env: ["test.integration": :test]
preferred_cli_env: ["test.integration": :test],
docs: docs()
]
end

Expand All @@ -36,7 +38,7 @@ defmodule Protobuf.Mixfile do
{:jason, "~> 1.2", optional: true},
{:dialyxir, "~> 1.0", only: [:dev, :test], runtime: false},
{:credo, "~> 1.5", only: [:dev, :test], runtime: false},
{:ex_doc, "~> 0.23", only: :dev, runtime: false},
{:ex_doc, ">= 0.0.0", only: :dev, runtime: false},
{:stream_data, "~> 0.5.0", only: [:dev, :test]}
]
end
Expand All @@ -53,9 +55,18 @@ defmodule Protobuf.Mixfile do
[
maintainers: ["Bing Han"],
licenses: ["MIT"],
links: %{"GitHub" => "https://github.com/tony612/protobuf-elixir"},
files:
~w(mix.exs README.md lib/google lib/protobuf lib/*.ex src LICENSE priv/templates .formatter.exs)
~w(mix.exs README.md lib/google lib/protobuf lib/*.ex src LICENSE priv/templates .formatter.exs),
links: %{"GitHub" => @source_url}
]
end

defp docs do
[
extras: ["README.md"],
main: "readme",
source_url: @source_url,
source_ref: "v#{@version}"
Comment on lines +64 to +69
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

]
end

Expand Down
2 changes: 1 addition & 1 deletion mix.lock
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"file_system": {:hex, :file_system, "0.2.9", "545b9c9d502e8bfa71a5315fac2a923bd060fd9acb797fe6595f54b0f975fd32", [:mix], [], "hexpm", "3cf87a377fe1d93043adeec4889feacf594957226b4f19d5897096d6f61345d8"},
"jason": {:hex, :jason, "1.2.2", "ba43e3f2709fd1aa1dce90aaabfd039d000469c05c56f0b8e31978e03fa39052", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "18a228f5f0058ee183f29f9eae0805c6e59d61c3b006760668d8d18ff0d12179"},
"makeup": {:hex, :makeup, "1.0.5", "d5a830bc42c9800ce07dd97fa94669dfb93d3bf5fcf6ea7a0c67b2e0e4a7f26c", [:mix], [{:nimble_parsec, "~> 0.5 or ~> 1.0", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "cfa158c02d3f5c0c665d0af11512fed3fba0144cf1aadee0f2ce17747fba2ca9"},
"makeup_elixir": {:hex, :makeup_elixir, "0.15.0", "98312c9f0d3730fde4049985a1105da5155bfe5c11e47bdc7406d88e01e4219b", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}, {:nimble_parsec, "~> 1.1", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "75ffa34ab1056b7e24844c90bfc62aaf6f3a37a15faa76b07bc5eba27e4a8b4a"},
"makeup_elixir": {:hex, :makeup_elixir, "0.15.1", "b5888c880d17d1cc3e598f05cdb5b5a91b7b17ac4eaf5f297cb697663a1094dd", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}, {:nimble_parsec, "~> 1.1", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "db68c173234b07ab2a07f645a5acdc117b9f99d69ebf521821d89690ae6c6ec8"},
"nimble_parsec": {:hex, :nimble_parsec, "1.1.0", "3a6fca1550363552e54c216debb6a9e95bd8d32348938e13de5eda962c0d7f89", [:mix], [], "hexpm", "08eb32d66b706e913ff748f11694b17981c0b04a33ef470e33e11b3d3ac8f54b"},
"stream_data": {:hex, :stream_data, "0.5.0", "b27641e58941685c75b353577dc602c9d2c12292dd84babf506c2033cd97893e", [:mix], [], "hexpm", "012bd2eec069ada4db3411f9115ccafa38540a3c78c4c0349f151fc761b9e271"},
}