A GO Extension for InterSystems Cache/IRIS and YottaDB.
Chris Munt cmunt@mgateway.com
23 June 2023, MGateway Ltd http://www.mgateway.com
- Current Release: Version: 1.2; Revision 4.
- Two connectivity models to the InterSystems or YottaDB database are provided: High performance via the local database API or network based.
- Release Notes can be found at the end of this document.
Contents
- Overview
- Pre-requisites
- Installing mg_go
- Connecting to the database
- Invocation of database commands
- Invocation of database functions
- Transaction Processing
- Direct access to InterSystems classes (IRIS and Cache)
- License
mg_go is an Open Source GO extension developed for InterSystems Cache/IRIS and the YottaDB database. It will also work with the GT.M database.
The mg_go extension connects to these databases using their high performance C-based APIs. There is also the option of connecting to the database over the network.
Go installation:
https://golang.org/
InterSystems Cache/IRIS or YottaDB (or similar M database):
https://www.intersystems.com/
https://yottadb.com/
Install the core database interface module (mg_dba.so for UNIX and mg_dba.dll for Windows) in a directory of your choosing.
The mg_go extension is a module written in Go and this is included in your Go project. mg_go dynamically loads the mg_dba library (written in C) and this latter module is responsible for connecting mg_go to the database either via the database's API or over the network.
UNIX (in the /src/ directory):
make
Windows (in the /src/ directory):
nmake -f Makefile.win
Install the GO extension (essentially a GO package) in your GO source directory.
.../go/src/mg_go/
In this directory you will find mg.go.unix and mg.go.windows. Rename the appropriate one for your OS as mg.go
The DB Superserver is required for:
- Network based access to databases.
Two M routines need to be installed (%zmgsi and %zmgsis). These can be found in the Service Integration Gateway (mgsi) GitHub source code repository (https://github.com/chrisemunt/mgsi). Note that it is not necessary to install the whole Service Integration Gateway, just the two M routines held in that repository.
Log in to the %SYS Namespace and install the zmgsi routines held in /isc/zmgsi_isc.ro.
do $system.OBJ.Load("/isc/zmgsi_isc.ro","ck")
Change to your development UCI and check the installation:
do ^%zmgsi
MGateway Ltd - Service Integration Gateway
Version: 4.5; Revision 28 (3 February 2023)
The instructions given here assume a standard 'out of the box' installation of YottaDB (version 1.30) deployed in the following location:
/usr/local/lib/yottadb/r130
The primary default location for routines:
/root/.yottadb/r1.30_x86_64/r
Copy all the routines (i.e. all files with an 'm' extension) held in the GitHub /yottadb directory to:
/root/.yottadb/r1.30_x86_64/r
Change directory to the following location and start a YottaDB command shell:
cd /usr/local/lib/yottadb/r130
./ydb
Link all the zmgsi routines and check the installation:
do ylink^%zmgsi
do ^%zmgsi
MGateway Ltd - Service Integration Gateway
Version: 4.5; Revision 28 (3 February 2023)
Note that the version of zmgsi is successfully displayed.
The default TCP server port for zmgsi is 7041. If you wish to use an alternative port then modify the following instructions accordingly.
- For InterSystems DB servers the concurrent TCP service should be started in the %SYS Namespace.
Start the DB Superserver using the following command:
do start^%zmgsi(0)
To use a server TCP port other than 7041, specify it in the start-up command (as opposed to using zero to indicate the default port of 7041).
- For YottaDB, as an alternative to starting the DB Superserver from the command prompt, Superserver processes can be started via the xinetd daemon. Instructions for configuring this option can be found in the mgsi repository here
GO code using the mg_go functions will, by default, expect the database server to be listening on port 7041 of the local server (localhost). However, mg_go provides the functionality to modify these default settings at run-time. It is not necessary for the GO installation to reside on the same host as the database server.
To use the mg_go extension you should include it in the list of packages required for your project. For a very basic GO project this might look something like:
import (
"fmt"
"mg_go"
)
In the following examples, modify all paths (and any user names and passwords) to match those of your own installation.
Assuming Cache is installed under /opt/cache20181/
db := mg_go.New("Cache")
db.APImodule = "../bin/mg_dba.so" // this will be mg_dba.dll for Windows
db.Path = "/opt/cache20181/mgr"
db.Username = "_SYSTEM"
db.Password = "SYS"
db.Namespace = "USER"
result := db.Open()
Assuming IRIS is installed under /opt/IRIS20181/
db := mg_go.New("IRIS")
db.APImodule = "../bin/mg_dba.so" // this will be mg_dba.dll for Windows
db.Path = "/opt/IRIS20181/mgr"
db.Username = "_SYSTEM"
db.Password = "SYS"
db.Namespace = "USER"
result := db.Open()
Assuming an 'out of the box' YottaDB installation under /usr/local/lib/yottadb/r130.
db := mg_go.New("YottaDB")
db.APImodule = "../bin/mg_dba.so"
db.Path = "/usr/local/lib/yottadb/r130"
db.EnvVars = db.EnvVars + "ydb_dir=/root/.yottadb\n"
db.EnvVars = db.EnvVars + "ydb_rel=r1.30_x86_64\n"
db.EnvVars = db.EnvVars + "ydb_gbldir=/root/.yottadb/r1.30_x86_64/g/yottadb.gld\n"
db.EnvVars = db.EnvVars + "ydb_routines=/root/.yottadb/r1.30_x86_64/o*(/root/.yottadb/r1.30_x86_64/r /root/.yottadb/r) /usr/local/lib/yottadb/r130/libyottadbutil.so\n"
db.EnvVars = db.EnvVars + "ydb_ci=/usr/local/lib/yottadb/r130/cm.ci\n"
db.EnvVars = db.EnvVars + "\n"
result := db.Open()
Assuming the server (Cache in this example) is listening on port 7041 on host localhost
db := mg_go.New("Cache")
db.APImodule = "../bin/mg_dba.so" // this will be mg_dba.dll for Windows
db.Host = "localhost"
db.TCPPort = 7041
db.Username = "_SYSTEM"
db.Password = "SYS"
db.Namespace = "USER"
result := db.Open()
If the MGateway Service Integration Gateway (MGWSI) is available, mg_go can connect to the database via this facility.
Assuming the MGWSI Gateway is listening on port 7040 on host localhost and the target Server (Cache in this example) is named as LOCAL in the Service integration Gateway configuration.
db := mg_go.New("Cache")
db.APImodule = "../bin/mg_dba.so" // this will be mg_dba.dll for Windows
db.Host = "localhost"
db.TCPPort = 7040
db.Server = "LOCAL"
db.Username = "_SYSTEM"
db.Password = "SYS"
db.Namespace = "USER"
result := db.Open()
db.Timeout := <time in seconds>
This setting applies to network based connectivity to the DB Server. The default value is 30 seconds.
Example: set timeout to 60 seconds:
db.Timeout := 60
db.InputBufferSize = <size in Bytes>
The buffer size must be large enough to hold the maximum size of the request or response data from the DB Server. The default value is 32767 Bytes. However for newer InterSystems databases the maximum string size can be up to 3,641,144 Bytes.
Example: set the buffer size to the maximum allowed for InterSytems databases:
db.Timeout := 3641144
version := db.Version()
Example:
fmt.Printf("\nVersion of mg\_go: %s\n", db.Version())
namespace := db.GetNamespace()
Example:
namespace := db.GetNamespace()
fmt.Printf("\nCurrent Namespace ns=%v\n", namespace)
result := db.SetNamespace(<namespace>)
Example:
result := db.SetNamespace("USER")
db.Close()
global := db.Global(<global_name>)
Example (using a global named "Person"):
person := db.Global("Person")
result := <global>.Set(<key>, <data>)
Example:
person.Set(1, "John Smith")
result := <global>.Get(<key>)
Example:
result := person.Get(1);
fmt.Printf("\nName : %s\n", result.Data.(string))
result := <global>.Delete(<key>)
Example:
result := person.Delete(1)
result := <global>.Defined(<key>)
Example:
result := person.Defined(1)
result := <global>.Next(<key>)
Example:
id := ""
for r := person.Next(id); r.OK; r = person.Next(id) {
id = r.Data.(string)
fmt.Printf("\nPerson ID: %s, Name: %s", id, person.Get(id).Data.(string))
}
result := <global>.Previous(<key>)
Example:
id = ""
for r := person.Previous(id); r.OK; r = person.Previous(id) {
id = r.Data.(string)
fmt.Printf("\nPerson ID: %s, Name: %s", id, person.Get(id).Data.(string))
}
result := <global>.Increment(<key>)
Example (increment the ^Person global by 1 and return the next value):
result := person.Increment(1)
result := db.Function(<function>, <arguments>)
Example:
M routine called 'math':
add(a, b) ; Add two numbers together
quit (a+b)
Go invocation:
result := db.Function("add^math", 2, 3)
fmt.Printf("\nFunction result: %v\n", fr)
M DB Servers implement Transaction Processing by means of the methods described in this section.
result := db.TStart()
- On successful completion this method will return zero, or an error code on failure.
Example:
result := db.TStart()
tlevel := db.TLevel()
- Transactions can be nested and this method will return the level of nesting. If no Transaction is active this method will return zero. Otherwise a positive integer will be returned to represent the current depth of Transaction nesting.
Example:
result := db.TLevel()
result := db.TCommit()
- On successful completion this method will return zero, or an error code on failure.
Example:
result := db.TCommit()
result := db.TRollback()
- On successful completion this method will return zero, or an error code on failure.
Example:
result := db.TRollback()
To illustrate these methods, the following simple class will be used:
Class User.customer Extends %Persistent
{
Property number As %Integer;
Property name As %String;
ClassMethod MyClassMethod(x As %Integer) As %Integer
{
// do some work
Quit result
}
Method MyMethod(x As %Integer) As %Integer
{
// do some work
Quit result
}
}
class := db.Class(<class_name>)
Example:
customer := db.Class("User.customer")
result := <class>.ClassMethod(<classmethod_name>, <arguments>)
Example:
result := customer.ClassMethod("MyClassMethod", 3)
Example (using instance/record #1):
result := customer.ClassMethod("%OpenId", "1")
result := <class>.GetProperty(<property_name>)
Example:
result = customer.GetProperty("name")
fmt.Printf("\nCustomer name: %s\n", result.Data.(string))
result := customer.ClassMethod("MyClassMethod", 3)
result := <class>.SetProperty(<property_name>, <value>)
Example:
result = customer.SetProperty("name", "John Smith")
result := <class>.Method(<method_name>, <arguments>)
Example:
result := customer.ClassMethod("MyMethod", 3)
Copyright (c) 2018-2023 MGateway Ltd,
Surrey UK.
All rights reserved.
http://www.mgateway.com
Email: cmunt@mgateway.com
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
- Initial Release
- Indroduce the option of connecting to the M server over the network.
- Verify that mg_go works with the latest version of Go: 1.15.6
- Restructure the documentation.
- Introduce support for M transaction processing: tstart, $tlevel, tcommit, trollback.
- Allow the DB server response timeout to be modified via the db.Timeout property.
- db.Timeout = [timeout]
- Allow the input buffer size to be modified via the db.InputBufferSize property.
- db.InputBufferSize = [size]
- Improved exception handling for DB connectivity errors.
- Introduce support for YottaDB Transaction Processing over API based connectivity.
- This functionality was previously only available over network-based connectivity to YottaDB.
- Documentation update.