Files layout and configuration

Directories

Directories descriptions:

• actiontracker - packet (package) deployment tracking module (create and manage of dbc_packets, dbc_steps, dbc_actions and dbc_locks tables)
• conf - directory with configuration files db_converter.conf and db_converter_test.conf
• dbccore - threads management and packet execution logic
• matterhook - module for interaction with mattermost
• packets - contains packages whose names are specified in the parameter --packet-name
• psc - auxiliary modules for working with logs and database
• tests - contains unit tests
• log - deployment logs. A separate file will be created for each package and database.

 

 

The configuration file db_converter.conf can contain connection strings for all servers. This allows to centrally manage migrations from a single host. There is also a separate configuration file for tests db_converter_test.conf.

Config parameters

db_converter.conf consists of the following sections:

• [databases] - list of databases with connection strings. Databases names (aliases) must be unique. If password contains special characters, then use URL encoding
• [main] - section contains the main parameters of the db_converter
• • application_name is a name of the application connected to database. This value is automatically postfixed with the name of the runned packet.
• • lock_observer_sleep_interval - is a time interval in seconds after which the lock observer checks the status of active connections from db_converter and other active connections to the database
• • execute_sql - enable execution of sql queries on the database. If false, queries are not executed on the database
• • conn_exception_sleep_interval - is a time interval in seconds after which the connection to the database is re-established and the query is repeated if the connection was closed from the database side (by user, by timeout, or by lock_observer)
• • cancel_wait_tx_timeout - cancel a step's transaction if it has been waiting for another query to release the lock longer than the specified time (valid values correspond to the interval format, for example: '3 minutes', '60 seconds')
• • cancel_blocker_tx_timeout - cancel the active transaction of a step if another query waits for its completion longer than the specified time (valid values correspond to the cancel_wait_tx_timeout parameter)
• [log] - logging settings
• • log_level - logging level. Possible values: Debug, Info, Error
• • log_sql - if set to 1 then all SQL queries are logged
• • file_maxbytes - is a maximum size of the log file in megabytes, upon reaching which the current one is renamed and a new one is created, in which messages continue to be written
• • file_backupcount - is a maximum number of .log files, upon reaching which the rotation is performed
• [matterhook] - settings for interaction with the mattermost messenger
• • url - hostname or IP with port of the messenger server, for example https://mattermost.company.org:1443
• • chat_keys - list of chats and keys separated by comma, for example dbc-alert/xaqxcj3axjb,dbc-dba/8ajz1z6y8
• [postgresql] - session variables. This section is optional. May contain the following parameters:

 

Parameter	Default value
deadlock_timeout	100ms
statement_timeout	1h
vacuum_cost_limit	3000
work_mem	200MB
maintenance_work_mem	1GB
timezone	UTC

   

For boolean parameters (such as execute_sql, log_sql) values can be specified from the list: 0, 1, false, true, False, True.

Packets naming convention

For a more convenient perception of the purpose of the packets (its name in the .log files and in application_name), it is recommended to adhere to the following naming format:

 

Format	Description
dba_task_name	Administration utilities
test_some_test	Test packages
alert_notification_name	Notifications
maint_task_name	Periodic database maintenance
YEARMONTH[DAY optional]_TASK_NUMBER_short_descr	Typical migration

  

All packets with dba_, alert_ and test_ prefixes are automatically used in unit tests.