NethCTI 3: server does not work properly if some ports are busy during the boot

[alepolidori]

During
On a customer installation, during the boot of the machine, an uncaught exception happens and this cause the break of authentication service (that listen on 50113 port). So the users are not able to login to cti client.

This behavior can happens even if a port is already taken by other service during the boot.

Into the log an "Uncaught exception" is written.

Steps to reproduce

• I do not find any pattern to reproduce the problem. In testing environment all boot actions work properly.

Expected behavior

After the machine boot, cti server have to work properly.

Actual behavior

In the customer installation clients cannot do the login.

Components

cti server

Fix

Wrap into a try/catch block all local services that listen on a specific port on localhost (in all server files of all com... components). Ports are set into the /etc/nethcti/services.json.

In case of Uncaught exception, thing if a process.exit(1); is correct: upstart will restart the service.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[alepolidori]

Some logs related to the problem:

2018-10-24T15:35:08+02:00 - error: [nethcti] UncaughtException !!!
2018-10-24T15:35:08+02:00 - error: [nethcti] Error: connect ECONNREFUSED 127.0.0.1:50113
    at Object.exports._errnoException (util.js:1020:11)
    at exports._exceptionWithHostPort (util.js:1043:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1099:14)
2018-10-24T15:35:08+02:00 - error: [nethcti] UncaughtException !!!
2018-10-24T15:35:08+02:00 - error: [nethcti] Error: socket hang up
    at createHangUpError (_http_client.js:258:15)
    at Socket.socketCloseListener (_http_client.js:290:23)
    at emitOne (events.js:101:20)
    at Socket.emit (events.js:188:7)
    at TCP._handle.close [as _onclose] (net.js:509:12)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[alepolidori]

I've added a shutdown system:

• when the process is stopped, nethcti server start the shutdown operation
• all components is responsible to stop its operations and release all resources (in this case is important to release network resources)
• after all components has been stopped, nethcti-server stop itself after 2 sec (to allow the writing of log entries)
• if the shutdown process fails, after 8 seconds the nethcti server stop itself in a force manner

[alepolidori]

The fix has been applied only to com_authentication_rest component to now.
Evaluate to apply it to other components if needed in the future.

[nethbot]

in 7.6.1810/nethesis-testing:

• nethcti-server3-3.5.0-1.3.ge7e4e4a.ns7.x86_64.rpm
• nethcti-server3-debuginfo-3.5.0-1.3.ge7e4e4a.ns7.x86_64.rpm

[alepolidori]

How to test

• open 2 instances of cti client and login
• restart the server several times, waiting some seconds from a reboot and the subsequent
• when the log print the following warning:

2018-12-17T10:56:49+01:00 - warn: [astproxy] asterisk connected
2018-12-17T10:56:49+01:00 - error: [http_proxy] Error: connect ECONNREFUSED 127.0.0.1:50113
    at Object.exports._errnoException (util.js:1020:11)
    at exports._exceptionWithHostPort (util.js:1043:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1099:14)

stop the restart and verify that the authentication component is listening on port 50113 and that the clients successfully login (use netstat -lanp|grep 50113)

[nethbot]

in 7.6.1810/nethesis-testing:

• nethcti-server3-3.5.0-1.5.gb391538.ns7.x86_64.rpm
• nethcti-server3-debuginfo-3.5.0-1.5.gb391538.ns7.x86_64.rpm

[nethbot]

in 7.6.1810/nethesis-testing:

• nethcti-server3-3.5.0-1.7.gb7a1157.ns7.x86_64.rpm
• nethcti-server3-debuginfo-3.5.0-1.7.gb7a1157.ns7.x86_64.rpm

[nethbot]

in 7.6.1810/nethesis-testing:

• nethcti-server3-3.5.0-1.8.g5094fac.ns7.x86_64.rpm
• nethcti-server3-debuginfo-3.5.0-1.8.g5094fac.ns7.x86_64.rpm

[nethbot]

in 7.6.1810/nethesis-testing:

• nethcti-server3-3.5.0-1.10.g77aa745.ns7.x86_64.rpm
• nethcti-server3-debuginfo-3.5.0-1.10.g77aa745.ns7.x86_64.rpm

[nethbot]

in 7.6.1810/nethesis-updates:

• nethcti-server3-3.6.0-1.ns7.x86_64.rpm
• nethcti-server3-debuginfo-3.6.0-1.ns7.x86_64.rpm