Skip to content
This repository has been archived by the owner on May 9, 2023. It is now read-only.

Make possible to pass startup options to PS servers for --test_mode #119

Closed
ShahriyarR opened this issue Oct 17, 2017 · 6 comments
Closed

Comments

@ShahriyarR
Copy link
Owner

The idea is:

  • Make possible to pass startup/initialize options for PS servers. For eg,
innodb_buffer_pool_size = 1G 2G 3G
innodb_log_file_size = 1G 2G 3G
innodb_page_size = 4K 8K 16K 32K 64K

So the work flow should be:

  • Start/initialize PS server from the option combinations which will be generated:
innodb_buffer_pool_size = 1G
innodb_log_file_size = 1G
innodb_page_size = 4K

Run becnhmark, Take backup of this server and prepare and restore.

Then continue such:

innodb_buffer_pool_size = 1G
innodb_log_file_size = 2G
innodb_page_size = 8K

And so on. This will cover possible MySQL options interactions with XtraBackup.

@ShahriyarR
Copy link
Owner Author

I have changed an idea for passing MySQL options:
--innodb_buffer_pool_size=1G 2G 3G,--innodb_log_file_size=1G 2G 3G,--innodb_page_size=4K 8K 16K 32K 64K -> this one actually more robust solution.

@ShahriyarR
Copy link
Owner Author

ShahriyarR commented Oct 20, 2017

Basically, what I am going to do, to create list of option combinations then:

  • go through this list,
  • initialize/start MySQL with those options
  • backup
  • prepare
  • recover

And repeat this action for all option combinations

@ShahriyarR
Copy link
Owner Author

Here is the draft code for option combinations creation:
https://repl.it/MxCc/6

ShahriyarR added a commit that referenced this issue Oct 20, 2017
@ShahriyarR
Copy link
Owner Author

Options are generated properly now, based on PS version:

$ cat xb_2_3_ps_5_6.conf | grep mysql_options
mysql_options = --innodb_buffer_pool_size=1G 2G 3G,--innodb_log_file_size=1G 2G 3G,--innodb_page_size=4K 8K 16K

$ cat xb_2_4_ps_5_6.conf | grep mysql_options
mysql_options = --innodb_buffer_pool_size=1G 2G 3G,--innodb_log_file_size=1G 2G 3G,--innodb_page_size=4K 8K 16K

$ cat xb_2_4_ps_5_7.conf | grep mysql_options
mysql_options = --innodb_buffer_pool_size=1G 2G 3G,--innodb_log_file_size=1G 2G 3G,--innodb_page_size=4K 8K 16K 32K 64K

@ShahriyarR
Copy link
Owner Author

I have decided to change the logic of calling tests.
For this purpose, I have created new bats file called prepare_env.bats, which is going to prepare whole test environment with much more granular output.
After running this file, we could run tests on this.

@ShahriyarR
Copy link
Owner Author

So according test run and the results this issue implemented fully.

$ ls backup_dir/
ps_5_6_x_2_3  ps_5_6_x_2_4  ps_5_7_x_2_4

Merging to release_v1.4.9

ShahriyarR pushed a commit that referenced this issue Oct 24, 2017
* Preliminary fix for issue #119

* Removed redundant code

* Added new prepare_env.bats file for preparing full test env

* Added new test for options combination generator method

* Fixed typo

* Added assert statement

* Trying to spot fail reason here

* Adding another test for return tupe from method

* Added new tests for final usage testing

* Added print message for showing function output

* Fixing option parsing and passing

* Removed redundant print message

* commented some code portions

* Calling staticmethod with passing config

* Added redundant print statement

* Added another print statement here

* Added some debug messages

* Changed test function names

* Added TODO
ShahriyarR pushed a commit that referenced this issue Oct 25, 2017
* Bugfix issue99 (#100)

* Preliminary fix for issue #99

* Trying to fix AtributeError

* Fixing path issue

* Added another test for method

* Trying to fix an issue with git command

* Added test for build_server method

* Added all tests for available methods

* Implemented different class structure added sysbench run class; now testing PyTest

* Fixing pytest conftest.py

* Added pytest for run_benchmark.py

* Fixing the issue with class method calling

* Fixing class name

* Masking something for running awk command

* Trying to fix failed thing

* Trying to fix N2

* Trying POPEN

* Trying to path basedir

* Failing thing is too annoying :)

* Again failed here

* Failed again here N111

* Using subprocess.PIPE

* Trying to use Pythonic way to extract socket file

* fixing sql statement

* Trying to fix sql statement

* Trying to print the output from sysbench

* Failed socket name for sysbench

* Fixing socket file usage in tests also

* Finalizing the feature implementation

* Bugfix issue101 (#102)

* Preliminary fix for issue #101

* Trying to fix fail reason

* Trying to spot fail reason

* Fixing the latest fail about path issue

* Fixing all script calling

* Using all_no_cl script

* get_basedir() function a bit

* Fixing the all_runner() usage

* Fixing latest fail

* Trying to get full path for basedir

* Trying to print statement

* Slightly changed the get_basedir() function

* Fixed the failed path

* Removed redundant print statement

* Added unit test for get_basedir()

* Trying to use regex search

* Trying to get the basedir using regex

* another commit

* Fixed failed PS cloning

* Added some more debug messages

* Added new test

* New debug message

* Finalizing this feature implementation

* Bugfix issue104 (#105)

* Added method for getting xb package and unit test for this method

* Passed the unit test

* Added method for extracting from archive

* Trying to fix the failed test

* Trying to extract to right folder

* Again something failed here

* Trying to fix some subtask

* Added main function for generating config files with pytests

* logging the failed message

* Trying fix interesting error

* Trying to fix again this error

* Failed again with this option values must be strings error

* Changed a bit file handling

* Trying to raise an exception

* Calling function from file

* Trying to spot the fail reason

* Removed redundant code

* Added [Backup] section generation for config

* Finalizing the config file generation

* Bugfix issue103 (#106)

* Preliminary fix for issue #103

* Trying to access testpath from GeneralClass

* It seems to be the wrong key generated in config generator

* Trying to fix some issues

* Changed conftest.py

* Trying to fix class inheritance issue

* Calling Backup class directly here

* Fixing failed format function

* converting to int

* Trying to fix check_mysql_conf() methdod

* Trying to fix check_mysql_conf() again

* Trying to fix check_mysql_conf() again and again

* Fixing mysql_connection_flush_logs() method

* Added another check here

* Trying to fix config generator

* Trying to catch SystemExit

* Commented exit()s

* Finalizing this feature implementation

* Added this to RunnerTestMode class

* Bugfix issue107 (#108)

* Preliminary fix for issue #107

* Finalizing the feature implementation

* Preliminary fix for issue #109 (#110)

* Bugfix issue111 (#112)

* Preliminary fix for issue #111

* Added new files to percona-qa and started to use them based on PS versions

* Added new test for printing result

* Changed a bit get_basedir() method to return list of basedirs

* prepare_startup() method also affected by these changes changed test a bit

* Changed test_start_server() and test_wipe_server_all() methods

* Bugfix issue113 (#114)

* Preliminary fix for issue #113

* Trying to fix failed test

* Wrong config file generated for 2.3 xb version

* Trying to fix 2.3 pxb config creation

* Trying to spot why the thing is failed

* Trying to fix again

* Changed for loop execution

* Changed a bit config generation

* Changed test function

* Again faile to generate 5.6 version of xb 2.4 config

* Again faile to generate 5.6 version of xb 2.4 config N2

* Trying to print statements

* Changed a bit for loop

* It seems to be I am a bit sleepy and made a pretty mistake

* Finalizing the feature implementation

* Bugfix issue115 (#118)

* Preliminary fix for issue 115

* Preliminary fix for issue #115

* Bugfix issue122 (#123)

* Changed test_get_sock() method

* Changed test_get_mysql_conn() and test_create_db() methods

* Changed test_run_sysbench()

* Finalizing the fix

* Bugfix issue120 (#124)

* Preliminary fix for issue #120

* Fixed the test for splitting xb_configs option

* Changed config generation for backup folders

* Changed xb folder checking

* Bugfix issue125 (#126)

* Preliminary fix for issue #125

* Added another test for copy_back actions

* Finalizing the issue implementation

* Bugfix issue119 (#127)

* Preliminary fix for issue #119

* Removed redundant code

* Added new prepare_env.bats file for preparing full test env

* Added new test for options combination generator method

* Fixed typo

* Added assert statement

* Trying to spot fail reason here

* Adding another test for return tupe from method

* Added new tests for final usage testing

* Added print message for showing function output

* Fixing option parsing and passing

* Removed redundant print message

* commented some code portions

* Calling staticmethod with passing config

* Added redundant print statement

* Added another print statement here

* Added some debug messages

* Changed test function names

* Added TODO

* Bugfix issue128 (#129)

* Preliminary fix for issue #128

* Trying to override new paths

* Trying to figure out why new path is not passed here

* Trying to fix path passing for environment check class

* Trying to pass inc_dir and full_dir path to CheckEnv() class

* Bugfix issue130 (#131)

* Preliminary fix for issue #130

* Finalizing this issue fix

* Preliminary fix for issue #132 (#133)

* Preliminary fix for issue #116 (#134)

* Preliminary fix for issue #135 (#136)

* Bugfix issue138 (#139)

* Preliminary fix for issue #138

* Fix for failed class call

* Added new test and fixed previous one for issue #140 (#141)

* Bugfix issue137 (#142)

* Added new test mode doc.
Also updated few things in previous docs for issue #137

* Added few more notes

* Added few more things into DOC

* Version bump for autoxtrabackup
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant