Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(microservice)There is a concurrent bug in microservice. #505

Closed
whtiehack opened this issue Mar 13, 2018 · 6 comments
Closed

(microservice)There is a concurrent bug in microservice. #505

whtiehack opened this issue Mar 13, 2018 · 6 comments
Labels
scope: microservices type: bug 😭

Comments

@whtiehack
Copy link

whtiehack commented Mar 13, 2018 •
edited
Loading

I'm submitting a...


[ ] Regression 
[X] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

Current behavior

[Nest] 17131   - 2018-3-13 18:56:06   [NestFactory] Starting Nest application...
[Nest] 17131   - 2018-3-13 18:56:06   [InstanceLoader] ApplicationModule dependencies initialized +6ms
[Nest] 17131   - 2018-3-13 18:56:06   [InstanceLoader] CatsModule dependencies initialized +2ms
[Nest] 17131   - 2018-3-13 18:56:06   [InstanceLoader] MicroClientModule dependencies initialized +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RoutesResolver] CatsController {/cats}: +27ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/, POST} route +1ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/micro, GET} route +1ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/micro2, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/microBuffer, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/:id, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [NestApplication] Nest application successfully started +1ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:196,102,138,162,96,result:531 +28628ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:149,191,43,199,184,result:531 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:172,173,129,100,34,result:531 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:78,73,184,99,46,result:531 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:139,101,64,195,156,result:632 +9ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:153,109,96,9,105,result:632 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] count:50 +1ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:31,156,77,50,185,result:632 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:128,22,188,27,2,result:632 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:152,69,56,3,181,result:448 +11ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:73,197,170,75,105,result:448 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:38,4,169,178,1,result:656 +6ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:137,174,56,115,25,result:401 +9ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:12,99,37,139,113,result:401 +1ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:191,41,152,122,132,result:401 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:186,106,14,27,186,result:401 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:142,60,16,127,165,result:472 +11ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:136,179,67,184,79,result:472 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:117,89,133,67,95,result:472 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:165,179,192,9,85,result:472 +1ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:132,191,146,85,128,result:567 +11ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:91,34,66,18,29,result:567 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:82,67,30,52,161,result:567 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:189,94,104,137,82,result:567 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:89,143,95,74,126,result:236 +6ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:150,136,38,131,87,result:437 +4ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:77,34,156,55,97,result:437 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:95,197,162,80,178,result:369 +5ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:11,194,108,198,106,result:369 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:190,94,154,195,148,result:617 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] data:196,41,193,170,151,result:617 +0ms
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end
[Nest] 17131   - 2018-3-13 18:56:34   [ClientTCP] Error: write after end

Expected behavior

[Nest] 17131   - 2018-3-13 18:56:06   [NestFactory] Starting Nest application...
[Nest] 17131   - 2018-3-13 18:56:06   [InstanceLoader] ApplicationModule dependencies initialized +6ms
[Nest] 17131   - 2018-3-13 18:56:06   [InstanceLoader] CatsModule dependencies initialized +2ms
[Nest] 17131   - 2018-3-13 18:56:06   [InstanceLoader] MicroClientModule dependencies initialized +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RoutesResolver] CatsController {/cats}: +27ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/, POST} route +1ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/micro, GET} route +1ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/micro2, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/microBuffer, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [RouterExplorer] Mapped {/:id, GET} route +0ms
[Nest] 17131   - 2018-3-13 18:56:06   [NestApplication] Nest application successfully started +1ms
[Nest] 17131   - 2018-3-13 18:56:34   [CatsController] count:50 +1ms

Minimal reproduction of the problem with instructions

Client

@Component()
export class MicroClientService {
    @Client({ transport: Transport.TCP, port: 3006 })
    public testMicroClient: ClientProxy

}


@Controller('cats')
export class CatsController {
    logger:Logger;
    count:number;
    constructor(private readonly catsService: CatsService, private readonly microClientService: MicroClientService) {
        this.logger = new Logger("CatsController",true);
        this.count = 0;
    }

    static getRandom():number{
        return Math.floor(Math.random()*200);
    }
    @Get('/micro')
    async testMicroService() {
        const data = [CatsController.getRandom(),CatsController.getRandom(),CatsController.getRandom(),CatsController.getRandom(),CatsController.getRandom()];
        const result = await this.microClientService.testMicroClient.send<number>({cmd: 'sum'}, data).toPromise();
        if(result!=data.reduce((a,b)=>a+b)){
            this.logger.error(`data:${data.toString()},result:${result}`);
        }
        this.count++;
        if(this.count%50==0){
            this.logger.log(`count:${this.count}`);
        }
        return result;
    }
}

Server

  const app = await NestFactory.createMicroservice(ApplicationModule,{
    transport: Transport.TCP,
      port:3006
  });

@Controller()
export class MathController {
  @MessagePattern({ cmd: 'sum' })
  sum(data: number[]): number {
  //  throw new RpcException('~~server error~');
    return (data || []).reduce((a, b) => a + b);
  }
}

test code

function fun(){
    $.ajax({
	url:'http://127.0.0.1:3000/cats/micro',
	method:'GET',
	success: function(res){
		
	},
	error: function(err){
		console.error('err',err);
  }});
}

// !!!!!!
for(var i=0;i<60;i++){fun()}

What is the motivation / use case for changing the behavior?

i think the problem is ..

nest/src/microservices/server/server.ts

Lines 25 to 36 in 7bb35c0

public send(
stream$: Observable<any>,
respond: (data: MicroserviceResponse) => void,
): Subscription {
return stream$
.catch(err => {
respond({ err, response: null });
return Observable.empty();
})
.finally(() => respond({ disposed: true }))
.subscribe(response => respond({ err: null, response }));
}

nest/src/microservices/server/server.ts

Line 34 in 7bb35c0

.finally(() => respond({ disposed: true }))

client:

nest/src/microservices/client/client-tcp.ts

Lines 40 to 63 in 7bb35c0

protected async sendSingleMessage(msg, callback: (...args) => any) {
const sendMessage = socket => {
socket.sendMessage(msg);
socket.on(MESSAGE_EVENT, buffer =>
this.handleResponse(socket, callback, buffer),
);
};
if (this.isConnected) {
sendMessage(this.socket);
return Promise.resolve();
}
const socket = await this.init(callback);
sendMessage(socket);
}
public handleResponse(socket, callback: (...args) => any, buffer) {
const { err, response, disposed } = buffer;
if (disposed || err) {
callback(err, null, true);
return socket.end();
}
callback(err, response);
}

nest/src/microservices/client/client-tcp.ts

Lines 58 to 59 in 7bb35c0

callback(err, null, true);
return socket.end();

If i want to re-implement ClientProxy and CustomTransportStrategy will modify server.ts and client-proxy.ts.

i want use my own implement , how can i do?
https://github.com/whtiehack/pinus/blob/master/packages/pinus-rpc/lib/rpc-server/acceptors/tcp-acceptor.ts

https://github.com/whtiehack/pinus/blob/master/packages/pinus-rpc/lib/rpc-client/mailboxes/tcp-mailbox.ts

https://github.com/whtiehack/pinus/blob/master/packages/pinus-rpc/lib/util/composer.ts

Environment


Nest version:4.6.6

 
For Tooling issues:
- Node version: 8.9.4 
- Platform:  Mac 

Others:
@whtiehack
Copy link
Author

We are looking for a new framework to start a new project. I would like to know if this framework has been stress tested.
Because of the large amount of traffic in our project, it is very cautious to choose what framework to use.
Is there any data to look at?
Our focus is on gateway(websocket) and microservices(tcp).😭

@kamilmysliwiec
Copy link
Member

kamilmysliwiec commented Mar 14, 2018 •
edited
Loading

Thanks for reporting, I'm on it.

However, Nest microservices strategies are good for prototyping. Still, I'd recommend using a mature solution like, for example, NATS and plug it into Nest as a Custom Transport Strategy (which should be very easy). It will change in the incoming release (v5.0.0) where microservices will offer much more than before.

@whtiehack
Copy link
Author

@kamilmysliwiec Thanks for the answer, I thought of a solution.Thanks.

@kamilmysliwiec
Copy link
Member

The microservices package has been completely rewritten under the hood. With v5.0.0 this bug is fixed (v5 is available already as 5.0.0-beta.1 in NPM, docs are here https://docs.nestjs.com/v5/)

@zuohuadong
Copy link
Contributor

zuohuadong commented Apr 23, 2018 •
edited
Loading

image
like it: https://github.com/Netflix/Hystrix

When the load is too high, the VIP access is guaranteed.
What am I supposed to do?

@kamilmysliwiec

@lock
Copy link

lock bot commented Sep 25, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked as resolved and limited conversation to collaborators Sep 25, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
scope: microservices type: bug 😭
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@zuohuadong @whtiehack @kamilmysliwiec