Percona’s Technical Director of Quality Assurance Roel Van de Paar shares his findings on the quality of MySQL 8.0.3 RC.
On 21 September 2017, our upstream friends at Oracle released MySQL 8.0.3 RC as the first MySQL 8.0 Release Candidate.
I tested the MySQL 8.0.3 Release Candidate branch with selected Percona bugfixes applied and built it as debug using the pquery QA framework (freely available here). Percona releases Percona Server for MySQL – an improved fork of the MySQL server – with additional features and bug patches.
Any QA engineer would enjoy seeing the many bugs discovered:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | ================ [Run: 079328] Sorted unique issue strings (11258 trials executed, 246 remaining reducer scripts) err == DB_SUCCESS .. err == DB_IO_NO_PUNCH_HOLE (Seen 1 times: reducers 5336) error == DB_SUCCESS (Seen 7 times: reducers 3896,3897,3902,5330,5331,5334,5425) .field->prefix_len .. field->fixed_len == field->prefix_len(Seen 8 times: reducers 442,1323,3990,7492,7586,8940,10101,10454) has_prelocking_list .. thd->mdl_context.owns_equal_or_stronger_lock. MDL_key..TABLE, rt->db(Seen 10 times: reducers 49,242,264,1251,4294,5170,6257,6261,7704,7732) length == 3 (Seen 1 times: reducers 9603) max_length >= length (Seen 10 times: reducers 418,2100,2304,4094,5171,5603,5695,6530,7526,8335) .m_fatal (Seen 6 times: reducers 4540,4878,6091,6143,6824,7157) mon > 0 .. mon < 13 (Seen 10 times: reducers 4,135,185,230,261,266,343,353,367,412) m_thd->get_transaction (Seen 2 times: reducers 4468,5674) MYSQL_BIN_LOG..handle_binlog_flush_or_sync_error (Seen 1 times: reducers 3114) MYSQL_BIN_LOG..new_file_impl (Seen 10 times: reducers 559,563,695,1202,1729,2046,4029,5578,6938,7212) new_table .= nullptr (Seen 1 times: reducers 5033) new_value >= 0 (Seen 1 times: reducers 7913) next_insert_id == 0 (Seen 4 times: reducers 3227,7917,9935,10945) num_codepoints >= scanner.get_char_index (Seen 10 times: reducers 962,1992,4829,5032,6835,7112,7117,7984,8209,9788) purge_sys->n_stop == 0 (Seen 10 times: reducers 337,512,606,709,856,1028,1106,1187,1257,1270) recv_recovery_is_on.. .. Log_DDL..is_in_recovery.. (Seen 10 times: reducers 688,1372,1499,4183,5233,5964,6478,7386,8556,8705) Sql_cmd_call..prepare_inner (Seen 5 times: reducers 3665,6965,7777,9775,10064) static_cast<ulint>.slot->n_bytes. < slot->original_len - n_bytes(Seen 10 times: reducers 3749,3892,3893,3898,3900,5333,5335,5423,5424,5897) String..ptr (Seen 10 times: reducers 1946,2487,2619,2965,3559,4533,5696,6007,6457,6806) sys_var_pluginvar..global_update (Seen 1 times: reducers 9359) ticket->m_lock->m_obtrusive_locks_granted_waiting_count .= 0(Seen 10 times: reducers 1696,1942,2253,2398,2501,2752,3343,3449,3679,7039) trans_table .. .changed .. thd->get_transaction (Seen 10 times: reducers 14,424,484,529,694,1694,2125,2993,3106,3569) trn_ctx->is_active.Transaction_ctx..SESSION (Seen 6 times: reducers 18,415,4512,6310,6773,9989) type (Seen 2 times: reducers 5522,10692) val <= 4294967295u (Seen 3 times: reducers 2298,3837) * TRIALS TO CHECK MANUALLY (NO TEXT SET: MODE=4) * (Seen 10 times: reducers 11,15,27,45,60,66,69,72,94,96) |
However, these results seem to show a lack of quality in the MySQL 8.0.3 RC release.
It looks like there are many new regressions, and I logged no less than 40 crashing, asserting and other bugs (security bugs are hidden from this list) to the MySQL bug tracker at Oracle. For each bug found, the test case is reduced (using reducer.sh, also available in the pquery QA framework) where possible and the bug is verified against upstream. If the bug is found upstream also, we log it there. No bugs were logged against Percona Server for MySQL.
Upstream quality is very important for us for several reasons. Firstly, any bug present upstream means we see the bug too. We then need to figure out whether it is upstream, or in our own code. Secondly, the better upstream quality is, the better our eventual product is as well. We care about quality.
We have seen the upstream developers do some great things in the past when it comes to dealing with a large influx of bugs, and we hope it will be no different this time. There is hope that MySQL 8.0 becomes another successful release from the great team at Oracle!
As soon as Oracle releases 8.0.4 RC (the second release candidate), and our team merges it in the Percona Server for MySQL patches (and perhaps features by then), I will test it again!
If you like to stay tuned for more on MySQL 8.0 tested and reviewed by the world’s leading database experts, just leave us a comment below and tick the “Notify me of new posts via email” subscription box.
All numbers in front of reducers are open defects ?
Hi Abe,
Each line is a new regression – i.e. a new issue not previously seen. Previously seen issues (i.e. open defects already logged) were/are removed by our filters. Also, this list lists a maximum of 10 trials per issue, even though there may have been more.
Technical sidenode; All this is done by pquery-go-expert.sh which is an advanced way of running the pquery framework. It 1) preps testcase reducers (reducer.sh) ready-to-run for each trial, 2) removes >10 ‘issue duplicates’ leaving 10 trials per issue (10 trials is usually more then sufficient to get a good test case), 3) filters known issues (using pquery-clean-known.sh which in turns uses known_bugs.strings – our bug filter list). If you ever have a crash, run text_string.sh against it and compare it with the list in known_bugs.strings (even old bugs are left, but remarked when fixed) – it’s a great way to find out if a bug is already been logged for the crash you’re seeing. For all these scripts, just clone percona-qa (https://github.com/Percona-QA/percona-qa) from Github.
In other words/in summary, the list above is substantial proof of a lot of new regressions in 8.0. Still, I have trust in Oracle to resolve most of these before GA!
Notify me of new posts via email.
Hi Yuval, When leaving a comment you have the option of ticking ‘Notify me of new posts via email.’