Comments on: How Percona Develops Open-Source Software

By: Vadim

Vadim — Mon, 12 Jan 2009 23:27:29 +0000

Giuseppe, Ivan.

Ok, we are using mysql-test suite for everyday testing, and adding our own tests to suite.
After significant changes we run stress-test as: sqlbench, sysbench, dbt2, tpcc, tpce benchmarks. There is no code coverage measurements to prove full covering of changes yet, something to think about.
Before patches are going to main repository we do mandatory patch-review.

As criteria to ship patch – it is also not hard defined yet. Obviously there “no serious production bugs”, like crashes, changes in behavior, wrong results. And criteria is simple – does it pass above mentioned tests or not.

After that we setup release on some our production boxes and sometimes on customer boxes (if allowed). After few days of work we consider we can show it to public.

By: Ivan

Ivan — Sun, 11 Jan 2009 18:48:49 +0000

Hi,

Also would be interested in seeing what test cases you are using to validate your server.

Thanks,
Ivan

By: http://datacharmer.blogspot.com/

http://datacharmer.blogspot.com/ — Sat, 10 Jan 2009 08:26:41 +0000

Baron,
Thanks for describing the creation process. There is something missing, though.
What about QA? I don’t believe you are just applying a patch and shipping the binaries to your customers.

* Do you have a test suite that you run after every commit, to make sure your change is not breaking anything?
* Do you have code coverage measurements to make sure that your tests are covering all the code?
* Do you enforce code reviews?
* What is your criteria for shipping a patch? When it is alpha, beta, RC, GA?

Cheers

Giuseppe