Here is a summary of the improvements to MTR test tool and suite in 5.7.4
Added 69 new tests and enhanced several existing tests in the MTR suite.
Test Suite migration
Test suite migration activity is continuing and in 5.7.4 we completed migration of replication suites rpl/binlog and about 75% of main suite. Relevant WLs are:
- WL#6921 Migrate rpl suite to run with innodb engine
- WL#6922 Migrate binlog suite to run with innodb storage engine
- WL#7263 Migrate myisam specific tests
- WL#7405 Migrate partition tests in the main suite
- WL#7402 Migrate tests in main suite for federated, blackhole, merge and csv engine
- WL#7411 Migrate tests that do not show any result difference when run with innodb engine
- WL#7410 Migrate authentication tests in main suite
- WL#7404 Migrate ctype tests in the main suite
Test Suite for replication (rpl) made GTID agnostic
Global Transaction Identifiers (GTID) was introduced in 5.6 and MTR tests were added specifically for testing GTID. To improve test coverage we wanted to run all tests in rpl suite with GTID ON. This was not possible because many will have result differences and fail. We have addressed this in 5.7.4 and have added regular regression runs of rpl suite with GTID turned on. Here is an example command line that is run regularly:
perl mysql-test-run.pl –force –timer –debug-server –parallel=auto –experimental=collections/default.experimental –comment=rpl_gtid-debug –vardir=var-rpl_gtid-debug –suite=rpl –mysqld=–enforce-gtid-consistency –mysqld=–log-slave-updates –mysqld=–gtid-mode=on –skip-test-list=collections/disabled-gtid-on.list –big-test –testcase-timeout=60 –suite-timeout=360
Refer WL#7205 for more details.
Moved InnoDB compression tests to a separate suite
Tests for InnoDB compression were part of the InnoDB suite and hence it was not easy to run only these tests with different compression options. Compression tests are now moved to a new suite called innodb_zip. This change is done from 5.5 onwards and following command lines were added to regression runs.
- perl mysql-test-run.pl –vardir=var-compressed_log0 –force –big-test –comment=compressed_log0 –testcase-timeout=60 –debug-server –parallel=auto –experimental=collections/default.experimental –mysqld=–innodb-log-compressed-pages=0 –suite=innodb_zip
- perl mysql-test-run.pl –vardir=var-compressed_log1 –force –big-test –comment=compressed_log1 –testcase-timeout=60 –debug-server –parallel=auto –experimental=collections/default.experimental –mysqld=–innodb-log-compressed-pages=1 –suite=innodb_zip
- perl mysql-test-run.pl –vardir=var-compressed_log0_level1 –force –big-test –comment=compressed_log0_level1 –testcase-timeout=60 –debug-server –parallel=auto –experimental=collections/default.experimental –mysqld=–innodb-log-compressed-pages=0 –mysqld=–innodb-compression-level=1 –suite=innodb_zip
- perl mysql-test-run.pl –vardir=var-compressed_log1_level9 –force –big-test –comment=compressed_log1_level9 –testcase-timeout=60 –debug-server –parallel=auto –experimental=collections/default.experimental –mysqld=–innodb-log-compressed-pages=1 –mysqld=–innodb-compression-level=9 –suite=innodb_zip
- perl mysql-test-run.pl –vardir=var-compressed_log0_level9_4k –force –big-test –comment=compressed_log0_level9_4k –testcase-timeout=60 –debug-server –parallel=auto –experimental=collections/default.experimental –mysqld=–innodb-log-compressed-pages=0 –mysqld=–innodb-compression-level=9 –mysqld=–innodb-page-size=4k –suite=innodb_zip
- perl mysql-test-run.pl –vardir=var-compressed_log1_level1_8k –force –big-test –comment=compressed_log1_level1_8k –testcase-timeout=60 –debug-server –parallel=auto –experimental=collections/default.experimental –mysqld=–innodb-log-compressed-pages=1 –mysqld=–innodb-compression-level=1 –mysqld=–innodb-page-size=8k –suite=innodb_zip
Minor enhancement to mysql-test-run.pl
A new option “–do-test-list was added to mysql-test-run.pl . This will take a file name as an argument and run the tests listed in the file.
eg: perl mysql-test-run.pl do-test-list=mytests.list
Above command will run all tests listed in mytests.list. This is useful in development when we are working on a feature that will have impact on tests spread across multiple suites. The relevant tests can now be grouped together and run with a simple command line.