Search the MySQL manual:

5.1.4 The MySQL Benchmark Suite

This should contain a technical description of the MySQL benchmark suite (and crash-me), but that description is not written yet. Currently, you can get a good idea of the benchmark by looking at the code and results in the `sql-bench' directory in any MySQL source distributions.

This benchmark suite is meant to be a benchmark that will tell any user what things a given SQL implementation performs well or poorly at.

Note that this benchmark is single threaded, so it measures the minimum time for the operations. We plan to in the future add a lot of multi-threaded tests to the benchmark suite.

For example, (run on the same NT 4.0 machine):

Reading 2000000 rows by index Seconds Seconds
mysql 367 249
mysql_odbc 464
db2_odbc 1206
informix_odbc 121126
ms-sql_odbc 1634
oracle_odbc 20800
solid_odbc 877
sybase_odbc 17614
Inserting (350768) rows Seconds Seconds
mysql 381 206
mysql_odbc 619
db2_odbc 3460
informix_odbc 2692
ms-sql_odbc 4012
oracle_odbc 11291
solid_odbc 1801
sybase_odbc 4802

In the above test MySQL was run with a 8M index cache.

We have gathered some more benchmark results at http://www.mysql.com/information/benchmarks.html.

Note that Oracle is not included because they asked to be removed. All Oracle benchmarks have to be passed by Oracle! We believe that makes Oracle benchmarks very biased because the above benchmarks are supposed to show what a standard installation can do for a single client.

To run the benchmark suite, you have to download a MySQL source distribution, install the Perl DBI driver, the Perl DBD driver for the database you want to test and then do: 

cd sql-bench
perl run-all-tests --server=#

where # is one of supported servers. You can get a list of all options and supported servers by doing run-all-tests --help.

crash-me tries to determine what features a database supports and what its capabilities and limitations are by actually running queries. For example, it determines:

We can find the result from crash-me on a lot of different databases at http://www.mysql.com/information/crash-me.php.

User Comments

Posted by Matt Walsh on Wednesday December 18 2002, @5:28pm[Delete] [Edit]

I had trouble with this.

0) Couldn't get the MySQL-bench-3.23.51-1.rpm to
install as it complains about MySQL-DBI-perl-bin.
But I'm sure I have that installed from the default
redhat 7.2 install as I can do queries from Perl all day
long. Sigh. So instead I worked off the source
distrubution.
1) The scripts end in .sh. The instructions and
readme assume they don't
2) The automake won't run since there is no
configure.in. Not sure what I need automake for,
anyway...
3) At least on my (linux) system, the @PERL@
statement at the top didn't work. I strongly suspect
my Perl inexperience is to blame and that there's a
smart way to fix this, but I went ahead and had to
replace all occurences with #!/usr/bin/perl (which
obviously won't work for everybody depending on
perl's location)
4) Even with all this, the run-all-tests script did
nothing and said everything was ok. I could run all
the tests individually, though.
5) The instruction might helpfully mention that
you
need a 'test' database. I had deleted mine. Or
better yet, the script could create the database if it
doesn't exist.

I documented my experience <A
HREF="http://www.mattwalsh.com/twiki/bin/view/M
ain/HowToRunTheMysqlBenchmarks">on my site
here:</A>

Posted by Djun Kim on Monday October 28 2002, @11:52pm[Delete] [Edit]

Problem 0) reported by Matt (regarding MySQL-bench
not installing from the RPM) persists through version
3.23.53a-1. The problem seems to be that
the MySQL-DBI-perl-bin package required by
MySQL-bench is actually called Mysql-DBI-perl-bin
(case matters)

Workaround: repackage Mysql-DBI-perl-bin as
MySQL-DBI-perl-bin. ALternatively, you can
do a search and replace on Mysql-DBI-perl-bin.rpm
and replace all instances of Mysql-DBI with MySQL-DBI.

Djun

Add your own comment.