MGateway Service Integration Gateway (SIG) for InterSystems Cache/IRIS and YottaDB.
Chris Munt cmunt@mgateway.com
3 June 2024, MGateway Ltd http://www.mgateway.com
- Current Release: Version: 3.1; Revision 103 (27 May 2023).
- SuperServer (%zmgsi routines) Release: Version: 4.5; Revision 32 (3 June 2024).
- Release Notes can be found at the end of this document.
Contents
- Overview
- Pre-requisites
- Installing the DB Superserver
- Starting the DB Superserver
- Installing the SIG
- Starting the SIG
- Using the SIG
- Resources used by the DB Superserver
- License
The MGateway Service Integration Gateway (SIG) is an Open Source network-based service developed for InterSystems Cache/IRIS and the YottaDB Database Servers. It will also work with the GT.M database and other M-like Databases Servers. Its core function is to manage connectivity, process and resource pooling for M-like DB Servers. The pooled resources can be used by any of the client-facing technologies in this product series (for example mg_php and mg_go etc ...).
An M-based client to the SIG is included (%zmgsic). This module provides easy access to global data, functions and InterSystems classes held on remote systems. Documentation for this component can be found here. The client and server do not need to be of the same type. You can use %zmgsic on InterSystems Cache/IRIS to access data on YottaDB and vice versa.
Also included in this package is a message/work queue manager for YottaDB and InterSystems Cache/IRIS. This is implemented as a stand-alone module written in M code, documentation for which can be found here.
There are two parts to a complete SIG installation:
- The DB Superserver. This component resides inside the DB Server and is independently required by other MGateway Open Source products.
- The SIG itself. This is effectively a client to the DB Superserver.
InterSystems Cache/IRIS or YottaDB (or similar M DB Server):
https://www.intersystems.com/
https://yottadb.com/
The DB Superserver consists of two M routines (%zmgsi and %zmgsis).
Log in to the %SYS Namespace and install the Superserver zmgsi routines held in /isc/zmgsi_isc.ro.
do $system.OBJ.Load("/isc/zmgsi_isc.ro","ck")
Change to your development Namespace and check the installation:
do ^%zmgsi
MGateway Ltd - Service Integration Gateway
Version: 4.5; Revision 32 (3 June 2024)
The instructions given here assume a standard 'out of the box' installation of YottaDB (v3.8) deployed in the following location:
/usr/local/lib/yottadb/r138
The primary default location for routines:
/root/.yottadb/r1.38_x86_64/r
Copy all the Superserver routines (i.e. all files with an 'm' extension) held in the GitHub /yottadb directory to:
/root/.yottadb/r1.38_x86_64/r
Change directory to the following location and start a YottaDB command shell:
cd /usr/local/lib/yottadb/r138
./ydb
Link all the zmgsi routines and check the installation:
do ylink^%zmgsi
do ^%zmgsi
MGateway Ltd - Service Integration Gateway
Version: 4.5; Revision 32 (3 June 2024)
Note that the version of zmgsi is successfully displayed.
The default TCP server port for the DB Superserver (zmgsi) is 7041. If you wish to use an alternative port then modify the following instructions accordingly.
In YottaDB, the DB Superserver can either be started from the DB (i.e. M) command prompt or Superserver processes can be started by the xinetd daemon.
- For InterSystems DB servers the concurrent TCP service should be started in the %SYS Namespace.
Start the DB Superserver using the following command:
do start^%zmgsi(0)
To use a server TCP port other than 7041, specify it in the start-up command (as opposed to using zero to indicate the default port of 7041).
DB Superserver version 4.4 (and later) can accept secured connections from clients over TLS. This facility is only available with InterSystems DB Servers. To use an InterSystems TLS Server configuration, specify this configuration name as the second argument to the DB Superserver start function.
Do start^%zmgsi(0,"MyInterSystemsTLSServerConfiguration")
Network connectivity to YottaDB is managed via the xinetd service. First create the following launch script (called zmgsi_ydb here):
/usr/local/lib/yottadb/r138/zmgsi_ydb
Content:
#!/bin/bash
cd /usr/local/lib/yottadb/r138
export ydb_dir=/root/.yottadb
export ydb_dist=/usr/local/lib/yottadb/r138
export ydb_routines="/root/.yottadb/r1.38_x86_64/o*(/root/.yottadb/r1.38_x86_64/r /root/.yottadb/r) /usr/local/lib/yottadb/r138/libyottadbutil.so"
export ydb_gbldir="/root/.yottadb/r1.38_x86_64/g/yottadb.gld"
$ydb_dist/ydb -r xinetd^%zmgsis
Note that you should, if necessary, modify the permissions on this file so that it is executable. For example:
chmod a=rx /usr/local/lib/yottadb/r138/zmgsi_ydb
Create the xinetd script (called zmgsi_xinetd here):
/etc/xinetd.d/zmgsi_xinetd
Content:
service zmgsi_xinetd
{
disable = no
type = UNLISTED
port = 7041
socket_type = stream
wait = no
user = root
server = /usr/local/lib/yottadb/r138/zmgsi_ydb
}
- Note: sample copies of zmgsi_xinetd and zmgsi_ydb are included in the /unix directory.
Edit the services file:
/etc/services
Add the following line to this file:
zmgsi_xinetd 7041/tcp # zmgsi
Finally restart the xinetd service:
/etc/init.d/xinetd restart
There are three parts to a complete SIG installation and configuration.
- The SIG executable (a UNIX Daemon or Windows Service) (mgsi or mgsi.exe).
- The DB Superserver described previously: the %zmgsi routines.
- A network configuration to bind the former two elements together.
The SIG (mgsi or mgsi.exe) is written in standard C. The GNU C compiler (gcc) can be used for Linux systems:
Ubuntu:
apt-get install gcc
Red Hat and CentOS:
yum install gcc
Apple OS X can use the freely available Xcode development environment.
Windows can use the free "Microsoft Visual Studio Community" edition of Visual Studio for building the SIG:
- Microsoft Visual Studio Community: https://www.visualstudio.com/vs/community/
There are built Windows x64 binaries available from:
Having created a suitable development environment, Makefiles are provided to build the SIG for UNIX and Windows.
Invoke the build procedure from the /src directory (i.e. the directory containing the Makefile file).
make
Invoke the build procedure from the /src directory (i.e. the directory containing the Makefile.win file).
nmake /f Makefile.win
The SIG executable can be installed in a directory of your choice. When started, it will create a configuration file called mgsi.ini. The event log file will be called mgsi.log.
Starting the SIG:
./mgsi
Stopping the SIG:
./mgsi -stop
Starting the SIG:
mgsi -start
Stopping the SIG:
mgsi -stop
When the SIG is started for the first time it will register itself as a Windows Service. Thereafter it can be managed from the Windows Services Control Panel if desired.
When the SIG is up and running its services are immediately available to participating clients. The SIG provides a web-based user interface for the purpose of maintaining the configuration and service management. By default the SIG listens on TCP port 7040. The web-based management suite may be accessed as follows.
http://[server]:7040/mgsi/mgsisys.mgw
The zmgsi server-side code will write to the following global:
- ^zmgsi: The event Log.
Copyright (c) 2018-2024 MGateway Ltd,
Surrey UK.
All rights reserved.
http://www.mgateway.com
Email: cmunt@mgateway.com
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
- Initial Release
- Updates to the documentation.
- Introduce a native concurrent TCP server for YottaDB.
- The Superserver can be started from the M command prompt using d start^%zmgsi().
- Invocation of Superserver processes from the xinetd daemon is still supported.
- Introduce commands to support M Transaction Processing.
- Review the Superserver code base and remove unnecessary/defunct code.
- Improve the performance of network connectivity for YottaDB.
- Correct a fault in the $$nvpair^%zmgsis() web helper function. It wasn't coping with unescaped '=' characters in values.
- Introduce improved support for InterSystems Objects for the standard (PHP/Python/Ruby) connectivity protocol.
- Add functionality to parse multipart MIME content for mg_web.
- Related to Superserver v4.2.20: move multipart section headers into a separate array.
- Create the infrastructure to allow mg_web to handle request payloads that exceed the maximum string length of the target DB Server.
- Introduce support for TLS secured connectivity for InterSystems DB Servers.
- Do start^%zmgsi([port],[name of InterSystems TLS server configuration])
- Correct an occasional 'undefined %payload' error in the content^%zmgsis() function.
- Remove the 'incoming connection ...' event log message unless the global node ^%zmgsi("loglevel") is set to a value greater than 1.
- Introduce support for native Unicode (UTF16) for InterSystems DB Servers.
- Correct a regression introduced in v4.4.23 that led to %zmgsis processes spinning for mg_web applications.
- Reinstate support for native Unicode (UTF16) for InterSystems DB Servers - this enhancement is mainly for mg-dbx.
- Add the functionality to support clients that implement an 'idle timeout' facility to close down inactive connections after a specified time.
- Introduce an options mask to allow the type of requests serviced by the DB Superserver to be restricted.
- Request types include: http, https, globals, functions, and classes
- The allowed request types are listed as the third parameter in the start-up function: e.g. Do start^%zmgsi(0,"","http,functions").
- The default behaviour is for all request types to be accepted by the DB Superserver.
- Suppress the recording of 'uci error: ...' messages in the event log unless the log level is set to 1 (or higher).
- Add the network commands for global node lock and unlock (used by mg_pwind and mg-dbx).
- Add the network command for a 'close object reference' operation (used by mg_pwind and mg-dbx).
- Allow the initial worker initialization message for the DBX protocol to be resent to an open/active connection. This change was made primarily to allow the M client to seamlessly reconnect to an existing (open) connection to the Superserver.
- Documentation update.
- Correct a fault affecting the return of Unicode data to Node.js through
- SQL.
- Correct a fault in the operation to get previous global node with data.
- Avoid occasional mgsql failures in YottaDB by always reloading the generated code before execution.
- Introduce support for Server-Sent Events (SSE) in mg_web.