Idea: API for generating code on a cargo build script.

[jansegre]

Cargo has updated the way build scripts work, with the latest changes it makes sense to have a simple function for use on a build.rs that given a .proto file will generate the .rs file on build time.

Usage would be something like this, inside build.rs:

let protos_dir = Path::new(os::getenv("CARGO_MANIFEST_DIR").unwrap()).join("my_protos");
let out_dir = Path::new(os::getenv("OUT_DIR").unwrap());
protobuf::build::generate(protos_dir.join("foo.proto"), out_dir);

Main advantage would be no need for either versioning generated .rs files or depending on on protoc.
The drawback would be doubled dependency on this crate, as a dependency and build-dependency, which could probably be solved by splitting protobuf on something like protobuf-gen and protobuf-runtime.

This looks fairly simple to implement and naturally it would be completely optional to use it that way. However I wanted to share the idea before trying to implement, partially because I'm having little time to spare lately and partially because I'd like to hear what other people think about it.

[stepancheg]

It is not that simple. Problem is in line:

protobuf::build::generate(..)

.rs files are generated by protobuf protoc program, that parses .proto files and passes binary data to protoc-gen-rust, which is a plugin to protoc.

On the other hand, protoc can just output binary data for .proto when called with --descriptor_set_out=FILE. So protobuf::build::generate may call protoc to generate FILE, and then read its contents, and generate .rs files. It is an unorthodox way, and pitfalls are possible.

[jansegre]

I hadn't realised that the parsing is made by protoc. So, that means it's not easy and may not make much sense to get rid of depending on protoc. Maybe makes sense to have a protoc-sys crate for the sake of only having cargo dependencies.

[jakerr]

In case you haven't seen it, capnpc-rust does something like suggested by calling out to the system capnp binary to do the parsing.

https://github.com/dwrensha/capnpc-rust/blob/master/src/lib.rs

[jansegre]

@jakerr I have seen similar, but what I had in mind was both on-build generation and no out of cargo dependencies, and I think the latter is more important. Thus without a rust implementation of the missing functionality of protoc, the best approach so far is having the .rs versioned.

[tamird]

@jakerr 👍 for providing something akin to what capnpc-rustprovides - together with the include! macro it is super handy, e.g.: given a file src/log.proto you can write a file src/log_proto.rs:

include!(concat!(env!("OUT_DIR"), "/log.pb.rs"));

and have your build script look something like this capnpc example:

extern crate capnpc;

use std::path::Path;

fn main() {
    let prefix = Path::new("src/");

    ::capnpc::compile(
      &prefix,
      &[
        &prefix.join("log.capnp"),
      ],
    ).unwrap();
}

This way shipping crates with proto files becomes easy and natural.

[plietar]

I've made a small crate rust-protobuf-build which links to both libprotoc on the C++ side and rust-protobuf on the Rust side.
This removes the need for protoc-gen-rust to be in the user's PATH, which simplifies the build process a lot, since everything is handled by cargo.

It doesn't have any documentation yet, but you can find a usage in my librespot project.
Generating the rust code
Importing the generated modules

@stepancheg It uses protobuf::codegen::gen to generate the rust code from the FileDescriptorProto.
Would you be OK to commit to relative stability of that API ?
Also I currently generate the files one by one, by giving gen only the descriptor for that file. I'm not really sure if it matters, I haven't tried with files with imports yet.

[stepancheg]

@plietar yes, you can rely on stability of gen.

Also, could you please add readme file to the rust-protobuf-build repository? So I could add a link to it from rust-protobuf readme.

[plietar]

Sorry it took me so long, I've finally added a README to rust-protobuf-build.

[stepancheg]

FYI, I've just created simple protoc crate which is a wrapper around protoc command.

Can be used like this: build.rs. protoc and protoc-gen-rust must be in $PATH though.

Next step would be invoking protoc to parse descriptors without invoking plugin.

[stepancheg]

OK, I've implemented protoc-rust crate.

Closing the issue now. Please reopen if you disagree.