commonjs support with import_style option

[zaucy]

Created for #197

Works in tandem with #210

[stanley-cheung]

Can you add a <space> after each if, while, and switch for consistency please?

[stanley-cheung]

You might have to rebase this as well. Thanks

[zaucy]

@stanley-cheung The uint is causing an error for me. Should this not be size_t?

[stanley-cheung]

Ah right. Sorry please change.

[zaucy]

Done!

[stanley-cheung]

Does this mean that the proto messages classes have to be generated into one single file named [some name]_pb.js, and can't be generated into one file per message?

[zaucy]

Yeah I'm matching how the protoc compiler outputs it's message with import_style set to commonjs. They output a js file per proto file with the name <proto-file-name>_pb.js.

[stanley-cheung]

@zaucy Do you mind doing a rebase here?

I am still trying to run the full npm + commonjs example. Just to make sure: you are using the same Echo backend gRPC c++ server + Envoy proxy right? So the only thing I need to swap out is how the frontend HTML calls the JS code right? I suppose one easy way is just set my webserver to point to a directory with index.js + node_modules/ generated from PR #210?

[zaucy]

@stanley-cheung Done!

Yeah all you should have to do is change how the frontend imports the client code. You'll need a frontend that works with commonjs imports. Anything based off webpack would work. Do you want me to write an example in webpack, angular and react? I can do that in another PR early next week.

[stanley-cheung]

@zaucy Yea such a complete example would greatly benefit the project! Thanks again for your great contributions!

[stanley-cheung]

Ha, I was able to (well, use a very roundabout way) finally verify that the CommonJS generated code works end-to-end!

Hack 1: I put all the generated code echo_pb.js and echo_grpc_pb.js into the same directory where package.json is, in the packages/grpc-web folder. And manually hack all the require to be like require('./echo_pb.js');

Hack 2: This line in the grpc-web generated code doesn't work
grpc.web = require('grpc-web');

But because I ran npm run build before, I changed that line to
grpc.web = require('./index.js');

And then I make client.js

const {EchoRequest} = require('./echo_pb.js');                                                                
const {EchoServiceClient} = require('./echo_grpc_pb.js');                                                     
                                                                                                              
const client = new EchoServiceClient('http://localhost:8080', null, null);                                    
                                                                                                              
const request = new EchoRequest();                                                                            
request.setMessage('Hello World!');                                                                           
                                                                                                              
client.echo(request, {}, (err, response) => {                                                                 
  console.log(response.getMessage());                                                                         
});  

And then ./node_modules/.bin/browserify client.js > out.js

And finally an index.html file that has a <script> tag that runs out.js

Is it because I haven't "published" the grpc-web module to npm that I need to hack require('grpc-web'); to require('./index.js');?

Hack 3: npm link (which install the local grpc-web module to my global node_modules and then npm link grpc-web to link the grpc-web module back to my current folder packages/grpc-web

[stanley-cheung]

@zaucy I noticed a potential discrepancy on where the generated file is placed for the grpc plugin vs the normal (message) protoc JS plugin:

Let's say my protoc command is like this:

~/grpc-web$ protoc ./net/grpc/gateway/examples/echo/echo.proto --js_out=import_style=commonjs:generated --plugin=protoc-gen-grpc-web=./javascript/net/grpc/web/protoc-gen-grpc-web --grpc-web_out=import_style=commonjs,mode=grpcweb,out=echo_grpc_pb.js:generated

The grpc generated file is placed in ./generated/echo_grpc_pb.js, but the messages (generated by the standard protoc JS plugin) file is placed in ./generated/net/grpc/gateway/examples/echo/echo_pb.js.

Should they be both placed in the ./generated/net/grpc/gateway/examples/echo directory?

But then I do noticed that in echo_grpc_pb.js, including the messages classes work this way:

proto.grpc.gateway.testing = require('./net/grpc/gateway/examples/echo/echo_pb.js'); 

So it does work as is. But it feels weird that the generated files are in a different location.

---

So perhaps it makes sense to 1. place both files into ./generated/net/grpc/gateway/examples/echo and 2. have echo_grpc_pb.js include the messages via proto.grpc.gateway.testing = require('./echo_pb.js');?

[lalomartins]

Off the top of my head generators for other languages (I've worked with Python and C# at least, maybe C++) put all output relative to where the command is run, it this case ./generated. Given the way I'd expect to automate generating bindings for multiple languages, my gut feeling is that's the right behavior, and the pb.js file is in the wrong.

And the reasoning is simple: if you want it in the same place as the proto file, that's easy to achieve by running the command with that cwd. In fact, the command line explicitly asks for :generated as the output path, but that's ignored, so I'd call it a bug.

[zaucy]

@stanley-cheung I'll take a look later today and make sure it's consistent.

[stanley-cheung]

@lalomartins sorry I was a little ambiguous. The _pb.js still honors the ./generated. I edited my post.

One is ./generated/echo_grpc_pb.js. The other is ./generated/net/grpc/gateway/examples/echo/echo_pb.js

[lalomartins]

Ah, I see what you mean. That's especially important if there are multiple files and they're not all in the same place (like you have a subdirectory — we had a layout like this in a recent project).

[stanley-cheung]

Nevermind, I can manipulate the protoc command a bit and get the generated files into the same directory.

protoc -I=net/grpc/gateway/examples/echo echo.proto
--plugin=protoc-gen-grpc-web=./javascript/net/grpc/web/protoc-gen-grpc-web
--js_out=import_style=commonjs:generated
--grpc-web_out=import_style=commonjs,mode=grpcweb,out=echo_grpc_pb.js:generated