Feature request: Support multiple packages in gRPC microservice

[zry656565]

I'm submitting a...

[ ] Regression 
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

Currently, I setup a gRPC microservice, and created several packages in protobuf files. But it didn't work.

Cause there's only a little of documents about gRPC in Nest, so I read some Nest codes about gPRC server initialization:

nest/packages/microservices/server/server-grpc.ts

Lines 53 to 74 in bf04704

	const packageName = this.getOptionsProp<GrpcOptions>(
	this.options,
	'package',
	);
	const grpcPkg = this.lookupPackage(grpcContext, packageName);
	if (!grpcPkg) {
	const invalidPackageError = new InvalidGrpcPackageException();
	this.logger.error(invalidPackageError.message, invalidPackageError.stack);
	throw invalidPackageError;
	}
	// Take all of the services defined in grpcPkg and assign them to
	// method handlers defined in Controllers
	for (const definition of this.getServiceNames(grpcPkg)) {
	this.grpcClient.addService(
	// First parameter requires exact service definition from proto
	definition.service.service,
	// Here full proto definition required along with namespaced pattern name
	await this.createService(definition.service, definition.name),
	);
	}
	}

What I found is that there's only one package that I could config in GrpcOptions. ONLY ONE. It does no good to construct my *.proto files in a well structure.

Minimal reproduction of the problem with instructions

Sample codes are like below:

// grpc-server.option.ts
import { GrpcOptions, Transport } from '@nestjs/microservices'
import { join } from 'path'

export const grpcServerOptions: GrpcOptions = {
  transport: Transport.GRPC,
  options: {
    url: 'localhost:5577',
    package: 'payment',
    protoPath: join(__dirname, './proto/payment.proto'),
    loader: {
      includeDirs: [
        join(__dirname, 'proto'),
      ],
    }
  },
} 

// bootstrap.ts
import { NestFactory } from '@nestjs/core'

import { ApplicationModule } from './app.module'
import { grpcServerOptions } from './grpc-server.options'

export async function bootstrap() {
  const app = await NestFactory.create(ApplicationModule)
  app.connectMicroservice(grpcServerOptions)
  await app.startAllMicroservicesAsync()
}

Expected behavior

I need to configure several packages when setup the server, like this:

const grpcServerOptions: GrpcOptions = {
  transport: Transport.GRPC,
  options: {
    url: 'localhost:5577',
    package: ['product', 'order'],
    protoPath: join(__dirname, './proto/payment.proto'),
    loader: {
      includeDirs: [
        join(__dirname, 'proto'),
      ],
    }
  },
} 

What is the motivation / use case for changing the behavior?

I need to separate individual services into different proto files (different packages).

Environment

Nest version: 5.7.4

 
For Tooling issues:
- Node version: v10.15.0  
- Platform:  Mac 

Others:

[zry656565]

proto files

Here're more sample codes.

.
└── proto
    ├── common
    │   └── entity.proto
    ├── modules
    │   ├── order.proto
    │   └── product.proto
    └── payment.proto

// entitiy.proto
syntax = "proto3";

package entity;

enum Currency {
  USD = 0;
}

message Product {
  string id = 1;
  string slug = 2;
  string name = 3;
  string description = 4;
  Currency currency = 5;
  int32 price = 6;
}

message Order {
  string id = 1;
  string subject = 2;
  string body = 3;
  int32 totalPrice = 4;
  Currency currency = 5;
  string channel = 6;
}

// entitiy.proto
syntax = "proto3";

import "../common/entity.proto";

package product;

service ProductService {
  rpc FindOne (ProductRequest) returns (ProductResponse) {}
  rpc CreateOne (CreateProductRequest) returns (CreateProductResponse) {}
  rpc FindByCategory (ProductByCategoryRequest) returns (ProductByCategoryResponse) {}
}

message ProductRequest {
  string slug = 1;
}

message ProductResponse {
  entity.Product product = 3;
}

message CreateProductRequest {
  string slug = 1;
  string name = 2;
  int32 price = 3;
  Currency currency = 4;
  string description = 5;
  string categorySlug = 6;
}

message CreateProductResponse {
  string productId = 3;
}

message ProductByCategoryRequest {
  string categorySlug = 1;
}

message ProductByCategoryResponse {
  repeated entity.Product products = 1;
}

// order.proto
syntax = "proto3";

import "../common/entity.proto";

package order;

service OrderService {
  rpc FindOne (OrderRequest) returns (OrderResponse) {}
  rpc FindByUser (OrderByUserRequest) returns (OrderByUserResponse) {}
  rpc CreateOne (CreateOrderRequest) returns (CreateOrderResponse) {}
}

message OrderRequest {
  string id = 1;
}

message OrderResponse {
  entity.Order order = 1;
}

message OrderByUserRequest {
  string userId = 1;
}
message OrderByUserResponse {
  repeated entity.Order order = 1;
}

message CreateOrderRequest {
  
}

message CreateOrderResponse {
  entity.Order order = 1;
}

// ./payment.proto
syntax = "proto3";

import "./modules/product.proto";
import "./modules/order.proto";

package payment;

message None {}

[kamilmysliwiec]

Thanks for reporting @zry656565! Would you like to create a PR?

[diepnghitinh]

in 5.7.4
microservice-configuration.interface line 18
package: string; is not working, should be string[] :(

[zry656565]

@kamilmysliwiec I created a related PR, please have a look when u'r free.

[chrisenytc]

@kamilmysliwiec What is the state of this feature request? What can I do to help?

I am using Nestjs to create a microservices ecosystem using gRPC but I am stuck with this issue. I need every gRPC service to run along with a gRPC health check service on the same port using different proto files.

Example:

account.session.v1.proto
grpc.health.v1.proto

syntax = "proto3";

package account.session.v1;

service SessionService {
  rpc FindAll(SessionFindAllRequest) returns (SessionFindAllResponse);
}

message Session {
  int32 id = 1;
  string token = 2;
}

message SessionFindAllRequest {}

message SessionFindAllResponse { repeated Session sessions = 1; }

syntax = "proto3";

package grpc.health.v1;

message HealthCheckRequest { string service = 1; }

message HealthCheckResponse {
  enum ServingStatus {
    UNKNOWN = 0;
    SERVING = 1;
    NOT_SERVING = 2;
  }
  ServingStatus status = 1;
}

service Health {
  rpc Check(HealthCheckRequest) returns (HealthCheckResponse);

  rpc Watch(HealthCheckRequest) returns (stream HealthCheckResponse);
}

I will be using Kubernetes so I need to use terminus to do health checking. I have time to contribute to this because I like Nestjs and want to use it in production at my new startup.

This is the non-breaking solution I am proposing:

  app.connectMicroservice({
    transport: Transport.GRPCMultiplePackages,
    options: {
      url: 'localhost:5000',
      services: [
        {
          package: 'account.session.v1',
          protoPath: 'account.session.v1.proto',
        },
        {
          package: 'grpc.health.v1',
          protoPath: 'grpc.health.v1.proto',
        },
      ],
    },
  });

[zombiQWERTY]

+1

[AnthonyMouallem]

+1

[vilkatis]

+1

[selginho]

+1

[Jeff-Tian]

+1

[Jeff-Tian]

Before the PR get merged, I created a pacakge: grpc-health that can enable standard grpc health check to nestjs grpc application. I improved the hero sample app to show that: https://github.com/Jeff-Tian/nestjs-hero-grpc-sample-with-health-check, the main change to the original sample is here: Jeff-Tian/nestjs-hero-grpc-sample-with-health-check@b76249c

[kamilmysliwiec]

Published as 6.11.0. More information here #3418