MHA Manager looks for relay-log.info in wrong location

[seattlegaucho]

Hi there,

I just found a bug as described in the title.

We store relay-log.info in a non-standard location (mostly for legacy reasons), which seems to confuse your scripts:

mysql -e "show global variables like '%relay_log%'"
+-----------------------+------------------------------+
| Variable_name | Value |
+-----------------------+------------------------------+
| max_relay_log_size | 0 |
| relay_log | /db_log/mysql/relay-bin |
| relay_log_index | /db_log/mysql/relay-index |
| relay_log_info_file | /db_log/mysql/relay-log.info |
| relay_log_purge | ON |
| relay_log_space_limit | 0 |
+-----------------------+------------------------------+

I'm using the following configuration for MHA:

Default:
cat /etc/masterha_default.cnf
[server default]
user=adm_mha
password=******
ssh_user=mysql
master_binlog_dir=/var/lib/mysql
remote_workdir=/var/log/masterha
ping_interval=3

Application:
cat /etc/masterha.d/test.cnf
[server default]
manager_workdir=/var/log/masterha/test
manager_log=/var/log/masterha/test.log
multi_tier_slave=1
master_binlog_dir=/db_log/mysql/

The results from masterha_check_repl are (only relevant section):
...
Mon Sep 26 17:44:05 2011 - [info] Checking SSH publickey authentication and checking recovery script configurations on the current master..
Mon Sep 26 17:44:05 2011 - [info] Executing command: save_binary_logs --command=test --start_file=mysql-bin.000114 --start_pos=4 --binlog_dir=/db_log/mysql/ --output_file=/var/log/masterha/save_binary_logs_test --manager_version=0.52
Mon Sep 26 17:44:05 2011 - [info] Connecting to mysql@xdc-tst-mysql-003(xdc-tst-mysql-003)..
Creating /var/log/masterha if not exists.. ok.
Checking output directory is accessible or not..
ok.
Binlog found at /db_log/mysql/, up to mysql-bin.000114
Mon Sep 26 17:44:05 2011 - [info] Master setting check done.
Mon Sep 26 17:44:05 2011 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Mon Sep 26 17:44:05 2011 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user=adm_mha --slave_host=xdc-tst-mysql-004 --slave_ip=10.55.210.155 --slave_port=3306 --workdir=/var/log/masterha --target_version=5.1.56-community-log --manager_version=0.52 --relay_log_info=/db_data/mysql//db_log/mysql/relay-log.info --slave_pass=xxx
Mon Sep 26 17:44:05 2011 - [info] Connecting to mysql@10.55.210.155(xdc-tst-mysql-004)..
Checking slave recovery environment settings..
Opening /db_data/mysql//db_log/mysql/relay-log.info ...Could not open relay-log-info file /db_data/mysql//db_log/mysql/relay-log.info.
at /usr/bin/apply_diff_relay_logs line 274
Mon Sep 26 17:44:05 2011 - [error][/usr/lib/perl5/vendor_perl/MHA/MasterMonitor.pm, ln129] Slaves settings check failed!
...

In bold are the parameters / output that illustrate the settings and bug. I know that this can be fixed by moving the master.info and relay-log.info files, but that would require a server bounce which is out of the question for now.

Please let me know if you need additional info or if there is an easy work around for now.

[seattlegaucho]

Moving everything under the same data directory doesn't seem to help either.

As long as relay-log-info is configured with a fully qualified path, it always fails. If left to the default value or only a file name, it works fine.

It seems like the agent should prepend the datadir variable if the full path to the relay-log.info file is specified.

[yoshinorim]

Thank you for the bug report.
This is a bug in lib/MHA/SlaveUtil.pm::get_relay_log_info_path() in mha4mysql-node package. It does not aware of absolute path of relay_log_info parameter. I'll fix soon.

[yoshinorim]

Fixes are committed.
Node package: yoshinorim/mha4mysql-node@250564a
Manager package: e798039

If you can try development tree, please pull the latest manager and node branches and install on all hosts.

[seattlegaucho]

Testing the patch, will report the results ASAP.

[seattlegaucho]

The tests seem to be working just fine. Feel free to close the issue and thank you once more for your time on this project.

[yoshinorim]

Thanks for the report. The fix passes our published test cases, too.