This document describes how to install, configure, and deploy ACE Direct.
-
The acedirect-kurento signaling server has been tested with JsSIP library version
3.5.1. All interoperability tests have been completed with this version. IMPORTANT: ACE Direct is tied to JsSIP3.5.1. To upgrade the JsSIP library version, the files injssip-modificationsmust be ported/developed. -
Acquire[] domain names for the servers. Domain names must be three-level domain names with no special characters, for example:
acenode.domain.com,acestun.domain.com,aceturn.domain.com,aceproxy.domain.com,acesip.domain.com,acekms.domain.com, andportal.domain.com. -
Deploy all servers as Amazon Web Services (AWS) EC2 instances. Use the Amazon Linux 2 operating system, unless specified otherwise. Other Linux flavors may work with slight or no changes to the instructions.
-
Set all servers to the UTC timezone. Use
chronydorntpto synchronize time across all servers. -
On
acenode.domain.com, modify SE Linux:sudo setsebool -P httpd_can_network_connect 1. You will need to do this after any reboot. -
The only public facing server is
portal.domain.com. All other servers only have private IP addresses on the same subnet. -
Add all server names and private IP addresses to
/etc/hostson all servers. -
Create A records. Connect your IP addresses to host names with A records (link). Create private A records for all servers. The NGINX server (
portal.domain.com) also requires a public A record. Some other servers require public A records. See the installation instructions for those servers for more details. -
Update provider peering lists. For video/softphones, contact providers to update their peering lists.
-
Acquire website certificates
- Certificates should be wildcard certs to allow domain name flexibility.
- Name the certificates
cert.pemandkey.pem. - Place certificates in the expected folders on all the servers:
/etc/ssl/. Certificates must have644permissions. - For new certificates, you may need to execute
restorecon, for example:restorecon -R -v cert.pem
-
Internet access. An Internet connection is required to install, build, and update the ACE Direct software. This allows the build processes to download external software and other dependencies.
Install the STUN server on acestun.domain.com. See ./docs/installation/STUN.md.
Install the TURN server on aceturn.domain.com. See ./docs/installation/TURN.md.
This is the NGINX server for ACE Direct. The server acts as a reverse proxy, hiding internal Node.js servers from public access.
Install NGINX on portal.domain.com.
See ./nginx/README.md for installation instructions. There is an installation script there.
There are several components to install on acesip.domain.com:
-
Clone the
asteriskrepo, and follow the installation instructions. Additionally:- Acquire an Asterisk certificate and copy it to
/etc/ssl/asterisk.csrto enable the signaling server demo/test page. Then update thecommon.https.csrvariable indat/config.jsonto be/etc/ssl/asterisk.csr. Finally, execute:sudo chmod 644 ~/ace-direct/dat/asterisk.csr. Note that the demo/test page is for development only. It is disabled by default.
- Acquire an Asterisk certificate and copy it to
-
To enable ITRS lookups in ACE Direct:
- Copy the
asterisk/scripts/itrslookup.shfrom theasteriskrepo toace-direct/scripts/itrslookup.shin this repo. - Configure the
itrslookup.shscript for the desired providers. - ITRS lookups are disabled by default in the ACE Direct consumer portal.
- Copy the
-
Clone the
acequill-servicerepo and follow the instructions to enable captioning and language translation.
aceproxy is the SIP proxy server. Log into aceproxy.domain.com, clone the kamailio repo, and follow the installation instructions.
acekms is the Kurento media server. Log into acekms.domain.com, clone the kurento repo, and follow the installation instructions.
Copy the kms-share folder from this repo to acekms and follow the README.md to install it.
Install a strongSwan server. See STRONGSWAN.md.
acenode hosts the Redis, MongoDB, MySQL, and Node.js application servers. Log into acenode.domain.com and follow the directions below.
⚠️ This installer assumes that you already installed:: Asterisk, NGINX, STUN, TURN, Kurento, and Kamailio.
For convenience, the automatic installer installs all the core components on acenode. The recommended instance size for acenode is t3a.medium. acenode should not have a public FQDN.
The automatic installer script is install.sh. This script will perform the following on acenode:
- Install prerequisites
- Install Redis, MongoDB, MySQL, and Node.js app servers
- Populate databases
- Deploy ACE Direct
- Perform status checks
To use the script:
$ # clone this repo to the ACE Direct user account (e.g., /home/ec2-user) on the acenode server
$
$ cd /home/ec2-user/ace-direct
$ ./install.sh # see the usage, then run with the required parameters
.
.
.For example...
$ ./install.sh -u ec2-user \
-s acestun.domain.com \
-t aceturn.domain.com \
-m "acenode.domain.com 1.0.0.1" \
-n "portal.domain.com 1.0.0.1" \
-k "acekms.domain.com 1.0.0.2" \
-a "acesip.domain.com 1.0.0.3" \
-c /etc/ssl/cert.pem \
-y /etc/ssl/key.pemAfter executing this script, try to access the portals. See Accessing the websites .
For manual installation and customization of your ACE Direct deployment, continue to Manual installation of acenode.
Here are detailed installation instructions, if you do not use the overall installer above.
Complete these prerequisite prior to installation:
- An Internet connection is required during the build process.
- Log into the
acenodeserver. - Create/identify an ACE Direct user account, for example
/home/ec2-user. Select thebashshell for the user. - Install in the home folder
/home/ec2-user. - Make sure Git is installed:
git --version, otherwise, install it:sudo yum install git -y - Make sure
ccis present:which cc, othwerise, install Development Tools:sudo yum groupinstall "Development Tools" - Copy/clone this
ace-directrepo to the ACE Direct user home folder:/home/ec2-user. - Make sure that the ACE Direct home user, e.g.,
/home/ec2-user, hassudoprivileges.
-
Install Node.js locally:
-
Amazon Linux 2 example:
$ cd /home/ec2-user # go to the ACE Direct user home folder $ rm -rf .nvm >/dev/null 2>&1 $ rm -rf .n >/dev/null 2>&1 $ $ # Install NVM: get curl from https://nvm.sh - Installing and Updating $ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash $ $ *CLOSE and REOPEN your termninal* $ $ nvm install 16 $ npm install -g n $ npm install -g pm2 $ $ which node ; which n ; which pm2 # should all be local
-
Add the following lines to the end of the
~/.bash_profilefile:N_PREFIX=$HOME/.n PATH=$N_PREFIX/bin:/usr/local/bin:$PATH export PATH N_PREFIX
-
Select Node version:
$ source ~/.bash_profile $ $ n 16.15.1 # required Node version
-
Install and configure Redis. For an overview, read Redis Quick Start. Follow these instructions to install Redis on acenode.domain.com:
-
Download and install Redis:
$ cd $ $ sudo rm -rf redis-stable >/dev/null 2>&1 $ wget http://download.redis.io/redis-stable.tar.gz $ tar xvzf redis-stable.tar.gz $ cd redis-stable $ sudo make distclean $ sudo make $ sudo make install $ ls /usr/local/bin/redis-server /usr/local/bin/redis-cli # both folders should exist $ sudo yum install -y tcl $ $ sudo mkdir -p /var/lib/redis $ sudo mkdir -p /etc/redis $ sudo cp redis.conf /etc/redis/redis.conf $ sudo chmod 666 /etc/redis/redis.conf
-
Configure Redis by editing
/etc/redis/redis.conf. Enable and set the fields below, selecting your own value for the secret Redis password:myRedisPassword:supervised systemd logfile "/var/log/redis.log" requirepass myRedisPassword -
Enable Redis as a service by creating
/etc/systemd/system/redis.service:[Unit] Description=Redis In-Memory Data Store After=network.target [Service] User=root Group=root ExecStart=/usr/local/bin/redis-server /etc/redis/redis.conf ExecStop=/usr/local/bin/redis-cli shutdown Restart=always [Install] WantedBy=multi-user.target
-
Reload the Redis service and make it start on reboot:
$ sudo systemctl daemon-reload $ $ sudo systemctl enable redis.service -
Managing the Redis service:
$ sudo service redis start $ $ sudo service redis status $ sudo service redis stop
ACE Direct uses a MongoDB database for call statistics. Follow the instructions below to install ACE Direct on acenode.domain.com.
-
Create a
/etc/yum.repos.d/mongodb-org-4.4.repofile with the following contents:[mongodb-org-4.4] name=MongoDB Repository baseurl=https://repo.mongodb.org/yum/amazon/2/mongodb-org/4.4/x86_64/ gpgcheck=1 enabled=1 gpgkey=https://www.mongodb.org/static/pgp/server-4.4.asc
-
Install MongoDB and make it start on reboot:
$ sudo yum install -y mongodb-org $ $ sudo systemctl start mongod # if it fails: sudo systemctl daemon-reload $ sudo systemctl status mongod # check status $ sudo systemctl enable mongod # start at boot time $ $ sudo systemctl stop mongod # in case you need to stop MongoDB
-
MongoDB uses port
27017by default.
ACE Direct uses a MySQL database for application data. Install MySQL locally on acenode.domain.com or deploy an Amazon AWS RDS service.
The instructions below describe how to install MySQL locally on acenode.domain.com.
-
Install MySQL Server Version
5.6.37or a similar version and note the database root user and password:$ sudo yum remove mysql mysql-server > /dev/null 2>&1 # remove old version $ $ sudo rm -rf /var/lib/mysql >/dev/null 2>&1 $ sudo rm -rf /etc/mysql >/dev/null 2>&1 $ sudo rm mysql57-community-release-el7-11.noarch.rpm* >/dev/null 2>&1 $ sudo yum update -y $ wget https://dev.mysql.com/get/mysql57-community-release-el7-11.noarch.rpm $ sudo yum localinstall mysql57-community-release-el7-11.noarch.rpm $ sudo yum install mysql-community-server $ sudo rm mysql57-community-release-el7-11.noarch.rpm* >/dev/null 2>&1
-
Enable MySQL as a service and start it on reboot:
$ $ sudo systemctl start mysqld.service # if it fails: sudo systemctl daemon-reload $ sudo systemctl status mysqld.service # check status $ sudo systemctl enable mysqld.service # start at boot time $ $ sudo systemctl stop mysqld.service # in case you need to stop MySQL
-
With MySQL started secure the installation::
$ # get the temporary root password $ sudo grep 'temporary password' /var/log/mysqld.log # get the temporary root password $ $ mysql_secure_installation # reset the root password (REMEMBER IT!); configure security options $
-
On
acenode.domain.com, modify the~/ace-direct/dat/acedirectdefault.sqlscript:- Globally replace
_EXTENSION_PASSWORD_with the actual extension password from Asterisk. See thepassword=field in/etc/asterisk/pjsip.confonacesip.domain.com. - Change
_ACEDIRECT_PASSWORD_to the desired password for theacedirectdatabase user. - Change
_ASTERISK_PASSWORD_to the desired password for theasteriskdatabase user.
- Globally replace
-
Execute the
~/ace-direct/dat/acedirectdefault.sqlscript to create the ACE Direct databases and user accounts. You will need your MySQLrootuser and password. Here is an example, assuming a root userroot:$ mysql -u root -p -h localhost < acedirectdefault.sql # you will be prompted for the password $
-
MySQL uses port
3306by default. -
The ACE Direct database users are:
acedirectandasterisk.
The ACE Direct application servers are Node.js servers.
⚠️ Important: All previous installation steps must be completed before installing the application servers.
-
Log into
acenode.domain.com. -
For a new ACE Direct deployment, create the initial global configuration:
$ cp ~/ace-direct/dat/config.json_TEMPLATE ~/ace-direct/dat/config.json $
-
Edit all values in the
~/ace-direct/dat/config.jsonglobal configuration file to match your environment.- Review all lines in the file and make necessary edits.
- Many of the default values will work as-is for a default ACE Direct installation. The installation scripts in this repo assume default values.
- View
~/ace-direct/dat/parameter_desc.jsonfor a description of each configuration variable. - Supply FQDNs, IP addresses, etc. for the ACE Direct components that were installed in the previous steps.
-
Ensure SSH access to external libraries. This will avoid very long build times. Edit your
~/.gitconfigfile to make sure it has this entry:[url "https://"] insteadOf = git:// -
Create initial
fognitousers. See fognito/README.md. Use the same agent and manager usernames found in dat/acedirectdefault.sql when creatingfognito/db/data/users.csv.
-
Build the application servers:
$ cd ~/ace-direct $ $ # full build $ npm run clean $ npm run build $ npm run config # only needed once $ $ # other useful commands $ npm run test # automated tests, make sure all Node servers are down $ npm run lint # run linting tests $ npm run clean # remove all external libs $ npm run clean:logs # remove log files
-
Deploy the application servers - ACE Direct services use pm2 for process management:
$ cd ~/ace-direct $ $ # starting $ pm2 start dat/process.json # first time or clean build $ pm2 status # check status of app servers $ $ # other commands $ $ pm2 start all # ongoing $ pm2 restart all # ongoing $ pm2 restart 0 # restart just ID 0, for example $ $ # stopping $ pm2 stop all $ pm2 stop 0 # stop one, ACE Direct server $ $ # reset counters $ pm2 reset all # reset all $ pm2 reset 0 # reset just ID 0 $ $ # deleting services $ pm2 stop all $ pm2 delete all
-
Make app servers start on reboot:
$ cd ~/ace-direct $ $ pm2 start dat/process.json # start all node servers $ pm2 save $ pm2 startup $ # now node.js servers will start on boot
-
Check ACE Direct system status (requires
sshaccess fromacenode.domain.comto the other servers):$ cd ~/ace-direct $ $ pm2 start all # start ACE Direct $ npm run status # self-test; requires ssh between internal servers
-
Another way to build and deploy the application servers the script:
$ cd ~/ace-direct $ $ scripts/build.sh # build and deploy
The BusyLight is an optional visual ring indicator that you can install on the Agent laptop.
See the ./obusylight/README.md for instructions on how to install the BusyLight server and device on the Agent laptop.
🏁 This completes ACE Direct installation, configuration, and deployment.
The enterprise service bus (esb) is an optional component. Use it to integrate with your existing CRM systems. See the esb/README.md file for more information.
🏁 This completes ACE Direct installation, configuration, and deployment.
Create agent and manager users: fognito/README.md#creating-initial-users
The URLs depend on your ~/ace-direct/dat/config.json settings, specifically nginx:fognito_path, nginx:ad_path, nginx:mp_path, nginx:agent_route, and nginx:consumer_route.
Default Consumer portal numbers are 111-111-1111, 222-222-2222, ... 999-999-9999. However, if ITRS mode is enabled ('user_service:itrs_mode'), you will need a valid VRS phone number.
The default ACE Direct URLs are:
- Universal login:
https://portal.domain.com/ace - Agent portal:
https://portal.domain.com/ACEDirect/agent - Consumer portal:
https://portal.domain.com/ACEDirect/call - Management portal:
https://portal.domain.com/ManagementPortal
See the docs folder for the official ACE Direct user guide, installation manual, and other useful documentation.
See the RELEASE notes for ACE Direct version information.
- Check the status of application servers on
acenode.domain.com:
-
Run
pm2 status:- Are all
statusfieldsonline (OK)? If not, errors are present. View all~/ace-direct/*/logs/*.logfiles for errors. - Are any
restartcounts? Are they increasing? If so, errors are present.
- Are all
- To restart the application servers:
pm2 restart all - Perform an ACE Direct self-test on the
acenodeserver (requiressshaccess to all servers):
$ cd ~/ace-direct
$
$ npm run status # should have all green checkmarks-
Set the logging level. Change the
common:debug_levelparameter in~/ace-direct/dat/config.jsontoALLto see all messages in the application server log files. Runpm2 restart allto enable the new log level. -
Check the
logsfolder in each application folder for errors or warnings:~/ace-direct/*/logs/*.log. -
Verify that all back-end servers (e.g., Redis, MongoDB, NGINX, MySQL, Asterisk, SIP Proxy, Kurento, ...) are running.
-
Verify that there are no firewalls blocking internal ports (e.g.,
firewalldonacenode.domain.comcould block access to some internal ports). -
Does the BusyLight device respond? Try the self-test mode on the BusyLight server app.
-
Verify that the
/etc/hostsfile is configured correctly on all back-end servers. -
Verify that the NGINX
/etc/nginx/nginx.conffile is configured correctly. -
Verify that the global
~/ace-direct/dat/config.jsonfile is configured correctly. This must be configured before building the application. -
When rebooting servers, the order is:
-
aceturn- TURN -
acestun- STUN -
acesip- Asterisk and ACE Quill -
aceproxy- SIP proxy -
portal- NGINX -
acekms- media server -
acenode- Node.js- Set SE Linux variable:
sudo setsebool -P httpd_can_network_connect 1 - Start Redis:
sudo service redis start - Start MongoDB
- Start MySQL
- Start all Node.js servers:
pm2 start ~/ace-direct/dat/process.jsonorpm2 start all
- Set SE Linux variable:
- To use a custom videomail intro video for Consumer videomail, put the video in
acedirect/public/media/. Set the filename indat/config.json:web_videomail.introVideo. - To use custom Consumer portal intro videos, put them in
acedirect/public/mediaand update thecomplaint_videossection indat/config.json.
-
ISSUE: Management portal build has
lodasherrors. SOLUTION: Installing thelodashlibrary globally as root:sudo npm install lodash -gand rebuild. -
ISSUE: NGINX cannot proxy to the
acenodeserver. SOLUTION: When using FQDNs for ACE Direct in/etc/nginx/nginx.conf, the FQDNs may force traffic through a proxy. To resolve this, map the FQDN to the private IP instead, using a private host zone. Or, simply use private IP addresses in place of FQDNs in/etc/nginx/nginx.conffor all the paths. -
ISSUE: There are no CDR records in the management portal. SOLUTION: Make sure Asterisk (
acesip.domain.com) is configured to have the MySQL database credentials, CDR database name, and CDR table name (see~/ace-direct/dat/config.jsonfor these values). Onacesip.domain.com, check the/etc/odbc.iniand/etc/asterisk/*odbc.conffiles. Make sure that the ODBC C library is installed onacesip.domain.com. -
ISSUE: The consumer portal cannot reach Asterisk (
acesip.domain.com);ERR_CONNECTION_REFUSED. SOLUTION: Make sure Asterisk is configured to use valid certificates. -
ISSUE: Server certificates may have expired. SOLUTION: Check certificate expiration:
openssl x509 -enddate -noout -in cert.pem -
ISSUE: Cannot connect to the agent, consumer, or management portals from a browser. SOLUTION: Remap the AWS elastic IPs. Run
nslookupon the NGINX FQDN and verify its public FQDN and public IP. -
ISSUE: NGINX errors occur when trying to connect to the portals, but all servers are up and running. SOLUTION: Make sure all servers have the correct time, synced with each other.
-
ISSUE:
acedirect-kurentofails to build. SOLUTION:$ cd ~/ace-direct/acedirect-kurento $ $ npm run clean $ npm run build2 $ npm run config $ $ pm2 restart all # restart all application servers
-
ISSUE:
acedirect-kurentotakes too long to build. SOLUTION:$ cd ~/ace-direct/acedirect-kurento/vendor/reconnect-ws $ $ rm package-lock.json $ npm run clean $ npm run build $ $ cd ~/ace-direct/acedirect-kurento $ npm run clean $ npm run build $
-
ISSUE: The Node builds on
acenodeare taking too long. SOLUTION: See the instructions above for creating the~/.gitconfigfile. -
ISSUE: A specific node server is failing to build or start. SOLUTION: The
package-lock.jsonfile may be outdated. Delete the file and rebuild. For example, forvideomail-service:$ cd ~/ace-direct/videomail-service $ $ rm package-lock.json $ npm run build $ pm2 restart all # restart the node servers $
-
ISSUE: The Node server builds on
acenodeare failing. SOLUTION: Verify thenpmversion and update it if necessary:$ npm -v 7.19.1 $ $ # to update to 7.19.1 (for example): $ npm install -g npm@7.19.1 $ -
ISSUE: All Node services appear to be working, but calls are not queuing. SOLUTION: The signaling server (
acedirect-kurento) may have trouble connecting to Asterisk. There may be reconnect messages in the signaling server log file. Asterisk may have a successful status, but it could be unresponsive. Try restarting the Asterisk service:sudo service asterisk restart. -
ISSUE: On an incoming consumer portal call, the agent portal sees the consumer extension instead of the VRS number as the caller ID number. Also, the agent has no incoming video after answering the call. SOLUTION: Try restarting REDIS:
sudo service redis restart. Also make sure the agent URL environment matches the consumer portal URL environment. -
ISSUE: This message appears: [agent] has an account but ACE Direct Agent extensions have not yet been configured by a Manager. SOLUTION: Make sure the MySQL
agent_tablehas been updated forv6.1:ALTER TABLE agent_data ADD profile_picture varchar(50);. -
ISSUE: Node servers have this MySQL error:
ER_NOT_SUPPORTED_AUTH_MODE: Client does not support authentication protocol requested by server; consider upgrading MySQL client. SOLUTION: Grant full privileges to the primary database user has on all databases, for example:GRANT ALL PRIVILEGES ON acedirect.* TO 'acedirect'@'%'; GRANT ALL PRIVILEGES ON media_server.* TO 'acedirect'@'%'; GRANT SELECT ON asterisk.* to 'acedirect'@'%';
-
ISSUE: Database queries are failing in MySQL8. SOLUTION: Disable strict mode:
USE acedirect; SET GLOBAL sql_mode='';. Note, when strict mode is on, fognito/db/create-user.sh may fail to add MySQL users. -
ISSUE: When the agent logs in, an error appears in the agent portal browser console:
WebSocket connection to 'wss://...' failed:. SOLUTION: Make sure thecommon.https.csrfile specified indat/config.jsonactually exists on the Node server. It can be an empty file. It is created in the dat folder by the build process. -
ISSUE: Some incoming VRS provider calls get into queue, but some don't. SOLUTION: Make sure the SIP Proxy server database has the appropriate IP entries for each provider.
-
ISSUE: Consumer web calls are not queueing. The acedirect-kurento error log shows a silent error:
'media_server.ace_direct_webrtc_session' doesn't exist. SOLUTION: Runcd ~/ace-direct; npm run configto create the media_server tables at the start of a deployment. Restart nodepm2 restart alland try again. -
ISSUE: Agents do not appear in the management portal. SOLUTIN: Make sure the
/etc/asterisk/agents.conffile has the correct agent names and extensions. -
ISSUE: Inbound VRS calls are not connecting. There is no IVVR shown. The call just disconnects. SOLUTION: Make sure the call center is open. Use the management portal to open the call center.
-
ISSUE: Inbound web calls are connecting, but there is no incoming video on either side. SOLUTION: Make sure the STUN and TURN servers are running. Make sure the STUN and TURN servers are properly configured in
~/ace-direct/dat/config.json.
fin.
