Many times am I asked what privileges do i need for this user. Explaining them can be confusing so I made this to share with everyone to help them understand MySQL users and privileges. Hopefully this will help you in deciding proper permissions for users. Click image below to download the pdf.

The attached pdf has a  table that anticipates that you want to know two things about privileges:

• What action is permitted: Read either the “What action is permitted” column, or the “Privilege Column Name” column
• At what level of granularity can it be set: For the privilege you’re interested in, note the “X”‘s, and read up the table to see which privilege tables provide that privilege, and then inspect their Key columns to understand the level of granularity.

These are some settings values I look at when I start up a new MySQL instance. Not all instances of MySQL will have the same, so I tweak until I get optimal performance out of MySQL. Also remember not to go into swap when allocating memory.

General MySQL Settings
The Most important ones are:
Key Buffer
The key buffer holds the indexes of tables in memory and a bigger key buffer results in faster row lookups. Adjust according to your own needs. Bigger is better, but prevent swapping at all costs. A good rule of thumb seems to be to use 1/4 of system memory.

Query Cache
This is where the magic happens. Well, not magic really, just plain old caching. Keeping the result of queries in memory until they are invalidated by additional writes enhances performance by magnitudes. The query_cache_size, as the name suggests, is the total size of memory available to query caching. The value query_cache_limit is the maximum number of kilobytes one query may be in order to be cached. Setting this value too high might prevent a lot of smaller queries to be cached. Setting it too low will result in bigger queries to never be cached, and the smaller queries not being able to completely fill the cache size, which would be a waste of resources. Adjust according to your own needs and memory available:

Table Cache
An important variable if your application accesses many tables. It is the number of tables a thread can keep open at the same time. A value of 512 should do no harm.

Sort Buffers
sort_buffer_size (the variable previously known as sort_buffer), used for grouping and sorting and is a per-thread buffer. If the buffer can not hold the data to be sorted, a sort is performed on disk. Watch out for making this too large as the buffer is allocated for every thread that needs sorting and with many sorts it can easily consume all your memory.

Binary Logging
MySQL has a few powerful features. Replicating data changes to a second server is one of them. MySQL keeps a log file of data changes which is used for this purpose. If you do not use replication or use the file as incremental backup, you can disable it. This will save you expensive disk write actions for every change to your data. For applications that have a lot of frequently updated data, this can be quite a performance boost. According to the official docs, this will generally result in just a 1% boost but it’s an easy gain if you do not need the log. Read more about the binary log here. Comment the following line:
log-bin = /var/log/mysql/mysql-bin.log

Temporary Tables
Temporary tables are used for sorting and grouping. The buffer is created on demand so watch out for setting this too high here as well. If the buffer cannot accomodate the data, a temp file is used on disk instead.

Delayed Writing
This setting can greatly improve writing or updating data to a table. Instead of directly committing data to the disk, MySQL queues writes and returns write queries immediately. Be very very careful with this, because this also means that in case of a power failure or crash, you lose data. You can use this for logging if you don’t mind losing a couple of rows in case of a crash.

Connection Timeout
This is a little tweak that determines the closing of sleeping connections. The default is one hour and is often too long for practical purposes. I often set this at one minute instead (60).
wait_timeout = 60
The above settings are just to make mysql a little faster in general. You can get much better speed improvements by optimizing the database itself. Setting the correct indexes on tables can be a life-saver.

MySQL Innodb Settings
The most important ones are:

innodb_buffer_pool_size
70-80% of memory is a safe bet. I set it to 12G on 16GB box.

innodb_log_file_size
This depends on your recovery speed needs but 256M seems to be a good balance between reasonable recovery time and good performance

innodb_log_buffer_size=4M
4M is good for most cases unless you’re piping large blobs to Innodb in this case increase it a bit.

innodb_flush_log_at_trx_commit=2
If you’re not concern about ACID and can loose transactions for last second or two in case of full OS crash than set this value. It can dramatic effect especially on a lot of short write transactions.

innodb_thread_concurrency=8
Even with current Innodb Scalability Fixes having limited concurrency helps. The actual number may be higher or lower depending on your application and default which is 8 is decent start

innodb_flush_method=O_DIRECT
Avoid double buffering and reduce swap pressure, in most cases this setting improves performance. Though be careful if you do not have battery backed up RAID cache as when write IO may suffer.

innodb_file_per_table
If you do not have too many tables use this option, so you will not have uncontrolled innodb main tablespace growth which you can’t reclaim. This option was added in MySQL 4.1 and now stable enough to use.

Also check if your application can run in READ-COMMITED isolation mode – if it does – set it to be default as transaction-isolation=READ-COMMITTED. This option has some performance benefits, especially in locking in 5.0 and even more to come with MySQL 5.1 and row level replication.

Even a query that is checking against an good index will result in a table scan if it is poorly written.  It seems some people think that HAVING can be a substitute for WHERE.  For example:

SELECT word_id
FROM forum.wordmatch
GROUP BY word_id
HAVING COUNT( word_id ) =53805;

This seems pretty straightforward; the query is trying to display how many entries have a word_id (indexed) greater than 53805.  But an explain shows that this is a table scan:

| table | type | possible_keys | key | key_len | ref | rows | Extra |
| wordmatch | index | NULL | word_id | 3 | NULL | 6374529 | Using index |

The problem is there is no WHERE clause, and HAVING just controls output.  So rewriting the query like so improves the performance by a factor of 10:

SELECT count( word_id )
FROM forum.wordmatch
WHERE word_id =53805;

| table | type | possible_keys | key | key_len | ref | rows | Extra |
| wordmatch | range | word_id | word_id | 3 | NULL | 641340 | Using where; Using index |

To quote mysql.com:

“Do not use HAVING for items that should be in the WHERE clause”. For example, do not write the following:

SELECT col_name FROM tbl_name HAVING col_name = 0;

Write this instead:

SELECT col_name FROM tbl_name WHERE col_name = 0;

The HAVING clause can refer to aggregate functions, which the WHERE clause cannot:

SELECT user, MAX(salary) FROM users
GROUP BY user HAVING MAX(salary) ;

SELECT DISTINCT ac2.category, ac0.name
FROM snaggy.a2_category ac1, snaggy.a2_category ac2, snaggy.a2_catalog ac0
WHERE ac1.category = 'GAME'
AND ac2.vendor_code = ac1.vendor_code
AND ac2.item_id = ac1.item_id
AND ac0.category = ac2.category
ORDER BY ac0.name

| table | type | possible_keys | key | key_len | ref | rows | Extra |
| ac0 | ALL | NULL | NULL | NULL | NULL | 145 |
| ac1 | ALL | category | NULL | NULL | NULL | 2886 | Using where; Using temporary; Using filesort
| ac2 | ALL | category | NULL | NULL | NULL |2886 | Using where |

CREATE TABLE `a2_category` (
`vendor_code` varchar(4) default NULL,
`item_id` int(5) default NULL,
`category` varchar(8) default NULL,
FULLTEXT KEY `category` (`category`)
) TYPE=MyISAM

Just adding an index helps makes the same query complete in a realistic timeframe:

ALTER TABLE a2_category ADD INDEX (item_id);

| table | type | possible_keys | key | key_len | ref | rows | Extra |
| ac0 | ALL | NULL | NULL | NULL | NULL | 145 | Using where |
| ac1 | ALL | item_id,category | NULL | NULL | NULL | 2886 | Using where; Using temporary; Using filesort |
| ac2 | ref | item_id,category | item_id | 5 | ac1.item_id | 3 | Using where |

I added a page with a reg expression cheat sheet, so I wanted to post how MySQL can use reg expressions in select queries. I use these and they can be powerful. Using reg expressions in selects is about as basic as it comes.

The format for reg expression for a SELECT is:

SELECT something FROM table WHERE column REGEXP 'regexp'

For example, to select all columns from the table events where the values in the column id end with 5309, use:

SELECT * FROM events WHERE id REGEXP '5309$'

A more elaborate example selects all columns of the table reviews where the values in the column description contain the word “mysql”:

SELECT * FROM reviews WHERE description REGEXP '[[:<:]]mysql[[:>:]]'

MySQL allows the following regular expression metacharacters:. match any character

• ? match zero or one
• * match zero or more
• + match one or more
• {n} match n times
• {m,n} match m through n times
• {n,} match n or more times
• ^ beginning of line
• $ end of line
• [[:<:]] match beginning of words [[:>:]] match ending of words
• [:class:] match a character class
• i.e., [:alpha:] for letters
• [:space:] for whitespace
• [:punct:] for punctuation
• [:upper:] for upper case letters
• [abc] match one of enclosed chars
• [^xyz] match any char not enclosed
• | separates alternatives

MySQL interprets a backslash (\) character as an escape character. To use a backslash in a regular expression, you must escape it with another backslash (\\).

I have written a little c program that will rotate tables and keep older tables for specified time. I use this program in a few environments where we have data retention requirements. This is to allow the tables not to grow absurdly large. This is my first C program please leave me feedback thank you.

USAGE:
Set up cron job to call the script once a week passing 3 variables
  1. Weeks to keep older tables
  2. Database Name
  3. Table Name
0 0 * * 0 /bin/lt_rotate_tables 3 testdb testing

// This code written by Lee Thompson with debugging by Dave Allmon//
// This program will rotate definded table and keep backup tables for n ammount of weeks //
// The inital table must exist first!! //
// to compile **** gcc lt_rotate_tables.c -o lt_rotate_tables  `mysql_config --cflags --libs` ***//
// Cron ***  *** //

#include <time.h>
#include <my_global.h>
#include <mysql.h>

#define ONE_WEEK (86400 * 7)

// This code will create a new backup and delete the backup that is n weeks old.
// Ex: This is week 21. Running this code with weeks=4 will create a table_21 table and delete a writes_17 table.

char tableNameNew[64];
char tableNameDrop[64];
char tmpTable[64];
char tmpDBname[64];

// Creates two suffixes based on the week of the year:
//  1. This week for new backup.
//  2. (weeks) ago to be deleted.

void CreateTargetDateStrs( int weeks )
{
  time_t thisTime;
  struct tm * lt;

  memset( tableNameNew, 0, sizeof ( tableNameNew ));
  memset( tableNameDrop, 0, sizeof ( tableNameDrop ));

  thisTime = time( NULL );
  lt = localtime( &thisTime );
  strftime(( char * ) & tableNameNew, sizeof ( tableNameNew ), "%B_%d", lt );

  thisTime -= ( ONE_WEEK * weeks );
  lt = localtime( &thisTime );
  strftime(( char * ) & tableNameDrop, sizeof ( tableNameDrop ), "%B_%d", lt );
}

int main(int argc, char **argv)
{
  char tmpStr[256];
  MYSQL *conn;

  if ( argc == 4 )
  {
    CreateTargetDateStrs( atoi( argv[1] ));
    sprintf(tmpTable,(argv[3]));
    sprintf(tmpDBname,(argv[2]));
  }
  else
  {
    printf( "\nUsage: ./lt_rotate_tables <weeks> <database name> <table name>\n" );
    exit( 1 );
  }
  conn = mysql_init(NULL);
  /////////////// Make only change here ////////////////////////
  //Set connection here change <user_name>  <password>
  mysql_real_connect(conn, "localhost", "<user_name>", "<password>", tmpDBname, 0, NULL, 0);

  // Drop oldest saved table.
  sprintf( tmpStr, "DROP TABLE %s_%s", tmpTable, tableNameDrop );
  mysql_query(conn, tmpStr );

  // Rename current table to new name.
  sprintf( tmpStr, "RENAME TABLE %s to %s_%s",tmpTable, tmpTable, tableNameNew );
  mysql_query(conn, tmpStr );

  // Create a new table like the last table.
  sprintf( tmpStr, "CREATE TABLE %s like %s_%s", tmpTable, tmpTable, tableNameNew );
  mysql_query(conn, tmpStr );

  mysql_close(conn);
}

Many environments I manage are large and span over multiple data centers across the world. We replicate data across and sometimes need to have a visual aid to help look at the replication flow. This simple PhP script allows me to see the replication flow starting from master to all slaves connected. You can start from any server but the results will be master first.

<?

#################################
######  Mysql Replication Mapper #########
######     Lee Thompson ##############
######      Version 1.0   ###############
#################################

$user = "<username>";
$pass = "<password>";

$masterserver = $_REQUEST['master_server'];

$masterport = $_REQUEST['master_port'];
if ($masterport == '')
$masterport = '3306';

echo "Please enter a master server";
echo "<form method='post' action='$php_self'>";
echo "Server: <input type ='text' name='master_server' value='" . $masterserver . "'><br />";
echo "Port: <input type ='text' name='master_port' value='3306'><br />";
echo "<input type ='submit'><hr />";

$serverlist = array();

// fill data
$serverlist = query_slave_server($masterserver.":".$masterport);

// print data
print "<b>Master Server: </b>".$serverlist." on Port: ".$masterport . '<br />';

$array = query_server($serverlist);
print_server($array, 0);

function query_slave_server($server)
{
  global $user, $pass, $masterport, $masterserver;

  $sql ="show processlist";
        $foundmaster = FALSE;
        while ( $foundmaster === FALSE )
        {
    $con = mysql_connect($server.":".$masterport,$user,$pass) or die($server. mysql_error());
                $result = mysql_query($sql,$con);
                while ($row = mysql_fetch_array($result))
                {
                        $slaveuser = $row['User'];
                        if ($slaveuser=='system user')
                        {
                                $sql2 = "show slave status";
                                $result = mysql_query($sql2,$con);
                                while ($row2 = mysql_fetch_array($result))
                                {
                                        $smasterserver = $row2['Master_Host'];
                                        $smasterport = $row2['Master_Port'];
                                        $server = $row2['Master_Host'];
                                        $masterport = $row2['Master_Port'];
                                }
                                break;
                        }
                }
                if ( $slaveuser != 'system user' )
                {
                        $foundmaster = TRUE;
                }
        }
        return $server;
}

function query_server($mserver)
{
  global $user, $pass, $masterport;

  $sql ="show processlist";

  $con = @mysql_connect($mserver.":".$masterport,$user,$pass) or die(mysql_error());
  $result = @mysql_query($sql,$con);
  while ($row = @mysql_fetch_array($result))
  {
    $slave = $row['Host'];
    $slavehost = ereg_replace(":.*", "", $slave);
    $slavecmd = $row['Command'];
    if ($slavecmd=='Binlog Dump')
    {
      $current[$slavehost] = query_server($slavehost.":".$masterport);
    }
  }
  return $current;
}

function print_server($current, $depth)
{
  global $user, $pass;

  foreach ($current as $slave => $ary)
  {
    $conn2 = mysql_connect($slave,$user,$pass);
    $sql2 = "show slave status";
    $result2 = mysql_query($sql2, $conn2);
    while ($row2 = mysql_fetch_array($result2))
    {
      $slavecmd = $row2['Slave_IO_Running'];
      $slavecmd1 = $row2['Slave_SQL_Running'];
    }

      if ($slavecmd=='Yes' && $slavecmd1=='Yes')
      {
        $slaverunning = "Yes";
      }
      else
      {
        $slaverunning = "<font color ='red'>No</font>";
      }
    $local = "localhost";
    $lconn = mysql_connect($local,$user,$pass);
    $lsql = "select server_name from myman.server where primary_ip = '".ereg_replace(":.*", "", $slave)."'";
    $lresult = mysql_query($lsql, $lconn);
    while($lrow = mysql_fetch_array($lresult))
    {
      if ($depth > 0) print str_repeat('&nbsp;',10 * ($depth - 1));
      echo "&#160---&gt;&#160; ";
      echo $lrow[0]. " (" . $slave . ") Slave Running:" . $slaverunning . "(depth: $depth)<br />\n";
      if ($ary) print_server($ary, $depth + 1);
    }
  }
}
?>

Example Output:

Master Server: 172.1.1.102 on Port: 3306
 ---> testdb02 (172.1.1.103) Slave Running:Yes(depth: 0)
 ---> testdb03 (172.1.1.104) Slave Running:Yes(depth: 0)
 ---> testdb04 (172.1.1.105) Slave Running:Yes(depth: 0)
     ---> testdb04-02 (172.1.1.106) Slave Running:Yes(depth: 1)
     ---> devdb01 (172.1.1.107) Slave Running:Yes(depth: 1)
       ---> dc2testdb (172.1.1.108) Slave Running:Yes(depth: 2)
	  ---> dctestdb-02 (172.2.2.101) Slave Running:Yes(depth: 3)
	  ---> dctestdb-03 (172.2.2.102) Slave Running:Yes(depth: 3)
       ---> dc3testdb (172.1.1.108) Slave Running:Yes(depth: 2)
	 ---> dctestdb-02 (172.2.2.103) Slave Running:Yes(depth: 3)
---> devdb02 (172.3.3.101) Slave Running:Yes(depth: 0)
---> devdb03 (172.3.4.101) Slave Running:Yes(depth: 0)

MySQL is a relational database management system. A relational database stores data in separate tables rather than putting all the data in one big storeroom. This adds speed and flexibility. The SQL part of “MySQL” stands for “Structured Query Language.” SQL is the most common standardized language used to access databases and is defined by the ANSI/ISO SQL Standard. The SQL standard has been evolving since 1986 and several versions exist. In this manual, “SQL-92” refers to the standard released in 1992, “SQL:1999” refers to the standard released in 1999, and “SQL:2003” refers to the current version of the standard. We use the phrase “the SQL standard” to mean the current version of the SQL Standard at any time.