Home

Overview

DxToolkit is a set of scripts, which are delivered by the Delphix professional services team. DxToolkit scripts look and feel like UNIX executables, following the typical conventions of using flags for arguments. DxToolkit is written in Perl, but no knowledge of Perl is required unless you want to extend it. In fact, no programming experience whatsoever is required to use the DxToolkit.

Working with DxToolkit

• Installation
• Configuration
• Password encryption procedure
• Connecting to Delphix engines
• DxToolkit commands
• Usage examples

Installation

Dxtoolkit is provided as an archive package of precompiled scripts, Docker container or as a source code.

Compiled version

Archive packages for most popular operating systems can be downloaded from a releases page.

Docker image

Run dxtoolkit using a docker image:

1. Create configuration file dxtools.conf based on dxtools.conf.example or a Wiki page.
2. Run a docker image with the following parameters:

• path redirection: -v /path/to/your/configfile:/config
• image name: pioro/dxtoolkit:develop
• dxtoolkit_command dxtoolkit_command_params

ex: docker run -v /configdir:/config pioro/dxtoolkit:latest dx_get_appliance -d myengine

Source version

Prerequisites

Perl version 5.16 or higher

Required packages

• JSON
• Date::Manip
• DateTime::Event::Cron::Quartz
• DateTime::Format::DateParse
• Crypt::CBC
• Crypt::Blowfish
• Text::CSV
• Try::Tiny
• LWP::UserAgent
• Net::SSLeay
• IO::Socket::SSL
• LWP::Protocol::https
• Term::ReadKey
• Log::Syslog::Fast

Source code can be downloaded from a main page of dxtoolkit

Configuration

DxToolkit reads the configuration file in JSON format. By default it's looking for dxtools.conf file in location of dxtoolkit script. Configuration file location can specified by -configfile parameter or value of environment variable DXTOOLKIT_CONF.

Field description

• hostname - Delphix identifier
• ip__address - Delphi appliance IP address or hostname
• username - Delphix username
• password - Delphix username password [ if password is blank or not specified script will ask for it and create a session ]
• port - Delphix Engine port number
• default - is this Delphix Engine, a default one - true | false
• encrypted - is password encrypted - true | false
• protocol - what protocol will be used for communication - http|https
• timeout - API call timeout - default 60 sec
• clientid - API access client id - default ''
• clientsecret - API access client secret - default ''
• passwordvar - Environment variable with password (see External password support)
• passwordscript - Script name to retrieve password (see External password support)
• additionalopt - Additional parameters for external script
• prevalidate - Flag to control when environment variable or password script is checked (see External password support)

Example

    {
    	"data":[
    		{
    			"hostname" : "Landshark",
    			"ip_address" : "delphix01",
    			"username" : "admin",
    			"password" : "delphix",
    			"port" : "80",
    			"default" : "false",
    			"encrypted" : "false",
    			"timeout" : "60"
    		},
    		{
    			"hostname" : "Landshark2",
    			"ip_address" : "delphix02",
    			"username" : "admin",
    			"password" : "delphix",
    			"port" : "443",
    			"default" : "true",
    			"encrypted" : "false",
    			"protocol" : "https"
    		},
    		{
    			"hostname" : "Delphix32",
    			"ip_address" : "192.168.1.32",
    			"username" : "admin",
    			"password" : "#papapaewiru32472931u12",
    			"port" : "80",
    			"default" : "true",
    			"encrypted" : "true"
    		},
    		{
    			"hostname" : "Delphix32",
    			"ip_address" : "192.168.1.32",
    			"username" : "admin",
    			"passwordvar" : "DELPASS",
    			"port" : "80",
    			"default" : "true",
    			"encrypted" : "false"
    		},
    		{
    			"hostname" : "Delphix32",
    			"ip_address" : "192.168.1.32",
    			"username" : "admin",
    			"passwordscript" : "/home/delphix/givemepass.sh",
    			"additionalopt" : "safefordelphix32",
    			"prevalidate" : "false",
    			"port" : "80",
    			"default" : "true",
    			"encrypted" : "false"
    		},
    		{
    			"hostname" : "marcinapi",
    			"ip_address" : "192.168.1.32",
    			"clientid" : "0oa6rs0js1f1zrueye",
    			"clientsecret" : "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
    			"port" : "80",
    			"default" : "true",
    			"encrypted" : "false",
    			"timeout" : "60"
    	  }
    	]
    }

Password encryption procedure

Password encryption is based on an encryption key build as combination of user name, Delphix Engine IP and from version 2.4.X a host name where dxtoolkit is installed. If dxtoolkit configuration file will be shared between different host, password should be encrypted with -shared option and in that case a host name of dxtoolkit installation is excluded from encryption key.

To encrypt a password inside a configuration file the following steps have to be executed:

• create a configuration file with plain text password and set a flag encrypted to true
• run a dx_encrypt command to generate a configuration file with an encrypted password.
• delete a configuration file with a plain passwords

Example of encryption command:

dx_encrypt -plainconfig dxtools.conf.plain -encryptedconfig dxtools.conf
New config file dxtools.conf created.

Example of encryption command with a shared key

dx_encrypt -plainconfig dxtools.conf.plain -encryptedconfig dxtools.conf -shared
New config file dxtools.conf created.

Access using API keys

Customers using a Central Management can use API keys to access Delphix Engine. Details how to generate clientid and clientsecret can be found here Using API Keys

Delphix is using this URL "https://delphix.okta.com/oauth2/default/v1/token" to generate an access token and dxtoolkit commands need access to this URL to be able to use API keys.

Connecting to Delphix engines

DxToolkit can run an operation on one or all engines configured in configuration file. After successful connection using username and password to the Delphix Engine a session is saved for a future usage. Default expired time for a session is set to 30 min and it's a Delphix Engine not a DxToolkit parameter. An existing user session can be destroyed using dx_logout command.

If access to Delphix Engine is controlled by API keys, session is not saved and keys are validated for each run of any dxtoolkit script.

If there is no -d nor -all option specified dxtoolkit command will connect to all engines defined with default=true in configuration file.

• -d name or –engine name Run command for one Delphix Engine specified by name
• -all Run command for all Delphix Engines defined in configuration file
• -dever Set version of API to particular Delphix Engine version. Supported values are 4.1, 4.2, 4.3, 5.0, 5.1, 5.2, 5.3

Common parameters

• -configfile file Location of configuration file
• -version Display version
• -debug n Enable debug mode at level n
• -help Display help
• -format Format ( [pretty] | csv | json )
• -nohead Turns off header

Usage examples

• Delphix replication – copy additional metadata from the source to the target engine
• Delphix metadata backup
• Delphix engine initialization

DxToolkit commands

Dxtoolkit commands can be dived into three different groups. First group contains scripts to perform actions on the datasets levels. Examples are: adding a dSource, provisioning a VDB, refreshing a VDB, etc. Second group contains scripts to create and control a Self Service objects from a command line. The last group is a set of commands working on the engine level. This group contains scripts to create or delete users, gather performance statistics, etc.

Dataset commands:

• dx_get_db_env - to list dSources and VDB's from engine(s) using filters
• dx_ctl_dsource - to control dSource operations
• dx_get_dsourcesize - list a size of dSources without Delphix compression ( can be used for ingestion model license check )
• dx_provision_vdb - to provision a new VDB
• dx_refrsh_db - to refresh a VDB using snapshot or point in time
• dx_rewind_db - to rewind a VDB using snapshot or point in time
• dx_get_hierarchy - to list hierarchy dependency between VDB's and dSource on single engine or between primary and replica engines
• dx_get_dbhooks - to list or save hooks from dSource or VDB
• dx_ctl_dbhooks - to create or change hooks on dSource or VDB

Self service commands:

• dx_get_js_bookmarks - to list a list of Self Service bookmarks
• dx_ctl_js_bookmarks - to add or remove Self Service bookmarks
• dx_get_js_snapshots - to list space consumed by Self Service bookmarks and operations like restore/branch/refresh
• dx_get_js_containers - to list Self Service containers
• dx_ctl_js_container - to create, remove, refresh or recover Self Service containers

Engine commands:

• dx_get_analytics - to export an analytis from Delphix Engine into csv file
• dx_ctl_analytics - to restart analytics collectors and create collector for NFS per IP
• dx_get_jobs - to list jobs from engine
• dx_get_replication - to list replication specification from engine

A full list of DxToolkit command can be found here