Node++ ASYNC
Node++ is – like Node.js – an asynchronous, single threaded machine. This is the fastest web server design that exists today. For the simple application that doesn't use any external resources, it's perfectly fine. Each request should take no more than 10–100 µs. But when we wanted to call a remote service, like i.e. Facebook API, we should expect the call to take seconds rather than microseconds. And this would be far too long to block the main server thread, because each client would need to wait until the external call returns:
To solve this, we need some kind of parallel processing. It can either be multithreading or multiprocessing. Node++ uses multiprocessing. Communication between processess is realized via standard POSIX message queues. Apart from convenient data exchange, message queues provide natural load balancing: the first npp_svc process that is able to receive the message, atomically removes it from the queue. CALL_ASYNC macros prepare the request, call mq_send and return without blocking of the main thread. The remaining part of the application logic will be executed by npp_svc. After calling CALL_ASYNC, npp_app process continues serving incoming requests and with every loop pass will check the response message queue whether npp_svc call has returned. When it has, it will write the response back to the client:
npp_app can then serve as a gateway or just handle those request that won't take too long. The exact treshold will depend on the required latency and the traffic, however we mostly look at the range of 1–10 ms.
POSIX message queues use fixed size messages. The default value for both the request and the response is 8 kB. Each request consists of the header and the data part. The sizes are reported in the beginning of the log file:
NPP_ASYNC_REQ_MSG_SIZE = 8192 B
ASYNC req.hdr size = 5320 B (5 KiB / 0.01 MiB)
G_async_req_data_size = 2872 B (2 KiB / 0.00 MiB)
ASYNC req size = 8192 B (8 KiB / 0.01 MiB)
NPP_ASYNC_RES_MSG_SIZE = 8192 B
ASYNC res.hdr size = 4480 B (4 KiB / 0.00 MiB)
G_async_res_data_size = 3712 B (3 KiB / 0.00 MiB)
ASYNC res size = 8192 B (8 KiB / 0.01 MiB)
The message sizes can be increased in npp_app.h with NPP_ASYNC_REQ_MSG_SIZE and NPP_ASYNC_RES_MSG_SIZE.
The data part is used to transfer the payload. The request message header will contain a flag indicating whether payload is in the message or – if it's too large – it passed over through a shared memory segment. The shared memory segment will only be created at the first time it's needed, and only once, and reused. Its size will be of NPP_MAX_PAYLOAD_SIZE bytes.
The data part of the response message is used to transfer the content generated by the application logic (using OUT macros). If the content requires more space than a single data part can contain, npp_svc will continue sending response messages and will flag the last one with the response header field chunk ASYNC_CHUNK_LAST bit set.
To enable ASYNC:
#define NPP_ASYNC- Add
ASYNCSvcProcesses=<number>to npp.conf with number > 0. For the test purposes 1 will be enough. For production it will depend on the usage profile and the expected load. For typical usage it can be equal to average number of user sessions active at the same time. Keep in mind that every process will have its own log file.
Every CALL_ASYNC is assigned its unique call_id. It is always passed to the service process. You can grep or search logs for the call_id. If logLevel is 4, the request starts with the similar line in npp_app log:
Sending a message on behalf of ci=1, call_id=123, service [get_inpost]
error: size of array ‘data’ is too large or
error: size of array 'data' is negative
This happens when app_session_data_t (SESSION_DATA) structure is larger than the remaining space in async_req_t. There are two options:
Increase NPP_ASYNC_REQ_MSG_SIZE and NPP_ASYNC_RES_MSG_SIZE.
Remove NPP_ASYNC_INCLUDE_SESSION_DATA from npp_app.h.