RAID 1
RAID 1 mirrors the contents of the disks, making a form of 1:1 ratio realtime backup. The contents of each disk in the array are identical to that of every other disk in the array. A RAID 1 array requires a minimum of two drives. Although RAID 1′s writing process copies the data identically to all drives, a RAID 1 mirror would not be suitable as a permanent backup solution for businesses, since RAID architecture by design allows for certain failures to take place (e.g. vandalism or accidental file deletion). However for home or other applications, where vandalism is very unlikely and accidental file deletion is put up with, RAID 1 offers a good backup solution.

RAID 3/4
RAID 3 or 4 (striped disks with dedicated parity) combines three or more disks in a way that protects data against loss of any one disk. Fault tolerance is achieved by adding an extra disk to the array, which is dedicated to storing parity information; the overall capacity of the array is reduced by one disk. A RAID 3 or 4 array requires a minimum of three drives: two to hold striped data, and a third for parity. With the minimum three drives needed for RAID 3, the storage efficiency is 66 percent. With six drives, the storage efficiency is 87 percent. The main disadvantage is poor performance for multiple, simultaneous, and independent read/write operations.

RAID 5
Striped set with distributed parity or interleave parity requiring 3 or more disks. Distributed parity requires all drives but one to be present to operate; drive failure requires replacement, but the array is not destroyed by a single drive failure. Upon drive failure, any subsequent reads can be calculated from the distributed parity such that the drive failure is masked from the end user. The array will have data loss in the event of a second drive failure and is vulnerable until the data that was on the failed drive is rebuilt onto a replacement drive. A single drive failure in the set will result in reduced performance of the entire set until the failed drive has been replaced and rebuilt.
RAID 6
RAID 6 (striped disks with dual parity) combines four or more disks in a way that protects data against loss of any two disks.
RAID 10
RAID 1+0 (or 10) is a mirrored data set (RAID 1) which is then striped (RAID 0), hence the “1+0″ name. A RAID 1+0 array requires a minimum of four drives: two mirrored drives to hold half of the striped data, plus another two mirrored for the other half of the data. In Linux MD RAID 10 is a non-nested RAID type like RAID 1, that only requires a minimum of two drives, and may give read performance on the level of RAID 0.

RAID 01
RAID 0+1 (or 01) is a striped data set (RAID 0) which is then mirrored (RAID 1). A RAID 0+1 array requires a minimum of four drives: two to hold the striped data, plus another two to mirror the first pair.

RAID can involve significant computation when reading and writing information. With traditional “real” RAID hardware, a separate controller does this computation. In other cases the operating system or simpler and less expensive controllers require the host computer’s processor to do the computing, which reduces the computer’s performance on processor-intensive tasks (see Operating system based (“software RAID”) and Firmware/driver-based RAID below). Simpler RAID controllers may provide only levels 0 and 1, which require less processing.
RAID systems with redundancy continue working without interruption when one (or possibly more, depending on the type of RAID) disks of the array fail, although they are then vulnerable to further failures. When the bad disk is replaced by a new one the array is rebuilt while the system continues to operate normally. Some systems have to be powered down when removing or adding a drive; others support hot swapping, allowing drives to be replaced without powering down. RAID with hot-swapping is often used in high availability systems, where it is important that the system remains running as much of the time as possible.
Note that a RAID controller itself can become the single point of failure within a system.

We will talk about nested raid levels in the upcoming days as I start to benchmark for which ones have better performance with out risk

Every 60 days I add a user on the database that has the only needed privileges for the application to run. I show the current privileges and change user and password and add the new user. I record the old one in case I need to fail back. I also have changed the default user “root” to  another username, and I have deleted the testdb that is installed when you setup mysql initially.

I use the CLI to do all my mysql work but you can easily do this yourself using phpmyadmin, make sure the user you make the changes with has GRANT OPTION.

1) SHOW GRANTS FOR ‘<username> ‘@’<host>’;

GRANT USAGE ON *.* TO ‘olduser’@'%’ IDENTIFIED BY PASSWORD ‘*BCF0C51505BF07C0AC46B9AEBB7F9726EB4677B8′;
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE ON `databasename`.* TO ‘olduser’@'localhost’;

2) Now you need to change the use and password from GRANT. You have 4 otptions for setting password by calling the password function or by using plain text. Using plain text will put the plain text in the binary log!!!

Option A: Generate the password hash using mysql

SELECT PASSWORD(‘somepassword’);

this will out put
——————————————-+
| password(‘somepassword’)                  |
+——————————————-+
| *DAABDB4081CCE333168409A6DB119E18D8EAA073 |
+——————————————-+
1 row in set (0.00 sec)
Now that we have the has of the password “somepassword” we can use that to update the password when applying grants.

GRANT USAGE ON *.* TO ‘newuser’@'%’ IDENTIFIED BY PASSWORD ‘*DAABDB4081CCE333168409A6DB119E18D8EAA073′;
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE ON `databasename`.* TO ‘newuser’@'localhost’;

Option B: You can use the password function to update the password during grant option.

GRANT USAGE ON *.* TO ‘newuser’@'%’ IDENTIFIED BY PASSWORD(‘somepassword’);
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE ON `databasename`.* TO ‘newuser’@'localhost’;

Option C: You can set the password by removing the password from the old grant.

GRANT USAGE ON *.* TO ‘newuser’@'%’ IDENTIFIED BY ‘somepassword’;
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE ON `databasename`.* TO ‘newuser’@'localhost’;

Option D: I do not reccomend this as it is not a proper way to update grants for a user, but you can update the mysql.user table to change the user and passwords.

UPDATE mysql.user set user = ‘newuser’, password = PASSWORD(‘newpassword’) where user = ‘olduser’;

This is not a proper way of updateing user but you can use it as long as you have rights to the mysql database.

The next step for any of these changes is to FLUSH PRIVILEGES: This flushes the new user to disk and refreshes the memory.

The final step will be to update the wp-config.php file in your www directory. In your www directory open your wp-config.php file with a text editor I use VIM for mine and look for these lines:
/** MySQL database username */
define(‘DB_USER’, ‘olduser’);
/** MySQL database password */
define(‘DB_PASSWORD’, ‘oldpassword’);
Update the user and password with your new information.

To try to secure my website I have changed my wp-config.php adn took the above lines out and put them in a seperate file and moved it to a directory outside of the www directory. I have my own server so I can put it anywhere as long as I know the absolute path. I have also encrypted the file using gpg which I will talk about in the near future. Once I make all the password and user updates I remove the old user from mysql and flush privileges once again.

Hopefully this article will help those who want to try to secure their site more effeciently. Please let me know if you have any questions or suggestions to secure a site.

The format for reg expression for a SELECT is:

SELECT something FROM table WHERE column REGEXP 'regexp'

For example, to select all columns from the table events where the values in the column id end with 5309, use:

SELECT * FROM events WHERE id REGEXP '5309$'

A more elaborate example selects all columns of the table reviews where the values in the column description contain the word “mysql”:

SELECT * FROM reviews WHERE description REGEXP '[[:<:]]mysql[[:>:]]'

MySQL allows the following regular expression metacharacters:. match any character

• ? match zero or one
• * match zero or more
• + match one or more
• {n} match n times
• {m,n} match m through n times
• {n,} match n or more times
• ^ beginning of line
• $ end of line
• [[:<:]] match beginning of words [[:>:]] match ending of words
• [:class:] match a character class
• i.e., [:alpha:] for letters
• [:space:] for whitespace
• [:punct:] for punctuation
• [:upper:] for upper case letters
• [abc] match one of enclosed chars
• [^xyz] match any char not enclosed
• | separates alternatives

MySQL interprets a backslash (\) character as an escape character. To use a backslash in a regular expression, you must escape it with another backslash (\\).

DSR Load Balance

To set up Loop-back
Create a second lo file Make sure there is no ARP

cp /etc/sysconfig/network-scripts/ifcfg-lo /etc/sysconfig/network-scripts/ifcfg-lo:0
vi /etc/sysconfig/network-scripts/ifcfg-lo:0

DEVICE=lo:0
IPADDR=1
#### Make sure you use nor arp
ARP=no
NETMASK=255.255.255.255
# If you're having problems with gated making 127.0.0.0/8 a martian,
# you can change this to something else (255.255.255.255, for example)
ONBOOT=yes

Now bring up your loop-back

 /sbin/ifup lo:0

#!/bin/bash
user=<user name>
pass=<password>

for i in /tmp/*.sock ;
do
for j in $(mysql -u$user -p$pass -S $i  -e 'show processlist' | grep 'sleep' | awk '{print $1}' )
do
mysql -u$user -p$pass -S $i  -e "kill $j"
done
done

Our system runs on Linux and has a gig network connection so times may vary depending on your setup.

First test was using rsync. Using this method is great if you can recover with existing data as it can make the changes using incremental differences. If you are building from newer hardware or cannot use the existing data then the time is drastically longer. The first rsync will have to move the files over by using the scp method(later explained). You can use this method if you want to minimize downtime. You will need to set up ssh keys in order to proceed with rsync which I remove once the data transfer is complete. The step below are the tests I timed using this method.

1.Create a ssh key from the server running rsync
ssh-keygen -t rsa
2.cat the newly created key
cat ~/.ssh/id_rsa_pub
3.Copy the text and paste on the remote server.
Vi /root/.ssh/authorized_keys
4.Execute rsync on the server to be rebuilt
/usr/bin/rsync -avzpogt –exclude=”*any files*” –delete -e ssh root@:/some/path* /some/path
This test was conducted while MySQL is stopped(you can lock the tables dont forget to set wait_timout and interactive timeout to 28000).

Timed Results first run
real    677m28.259s
user   228m48.514s
sys     47m40.737s

Timed Results first (Final)
real    2m21.259
user   0m38.259s
sys     1m00.252s
This test was conducted while MySQL was running, but you will need to stop MySQL on the final sync. I also uses an expect script to continue to run the rsync until I was ready to do the final transfer.

Timed Results first run
real    877m28.845s
user   248m54.214s
sys     30m22.231s

expect average time

real    48m32.156s
user   22m59.781s
sys     35m43.376s

expect script ran until we locked table and ran the final

real    22m45.101s
user   8m09.563s
sys     6m15.785s

The second method was introduced to me by A friend of mine and his blog is Koopman.me There are 2 methods and I chose the first one. With this method You will need to sop MySQL so transfer the files. This method will compress the files over the network and decompress on the remote host. Yes I could have used the lets tar up the data files on sending server, but this is expensive and will take time to do. Not to mention that you can run out of disk space quickly.

This is the basic commands:
A. tar cf – data files(*) | ssh user@remotehost ” ( cd /some/path ; tar xf – ) ”
B. ssh user@remotehost “( cd /some/path ; tar cf – data file(*) ) ” | tar xf -

Timed Results:
real    177m41.929s
user   108m52.797s
sys     47m40.737s

The last method was using straight scp. This method is just like the first step of rsync, but simplified. Not a lot of explaining on this one.
Command from the sending server.
scp data files(*) user@somehost:/some/path
Please not that you will need to have ownership or a tmp directory on the remote server.

Timed Results:
real    599m48.729s
user   203m52.992s
sys     68m40.569s

The next test to be conducted will be using scp with the -C flag for compression some state this will be the fastest. At this time the tar over ssh seems to be the fastest thanks koopman.me

]]> http://www.mysqlhow2.com/2009/09/01/transferring-large-files-between-servers/feed/ 1