If you push play, the video should scoot out to about the 14m40s mark where MySQLTuner appears on one of the slides. Thanks to Trent Hornibrook for letting me know!

MySQLTuner mentioned at the O'Reilly MySQL Conference is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

pv allows a user to see the progress of data through a pipeline, by giving information such as time elapsed, percentage completed (with progress bar), current throughput rate, total data transferred, and ETA.

The usage certainly isn't complicated:

To use it, insert it in a pipeline between two processes, with the appropriate options. Its standard input will be passed through to its standard output and progress will be shown on standard error.

A great application of pv is when you're restoring large amounts of data into MySQL, especially if you're restoring data under duress due to an accidentally-dropped table or database. (Who hasn't been there before?) The standard way of restoring data is something we're all familiar with:

# mysql my_database < database_backup.sql

The downside of this method is that you have no idea how quickly your restore is working or when it might be done. You could always open another terminal to monitor the tables and databases as they're created, but that can be hard to follow.

Toss in pv and that problem is solved:

# pv database_backup.sql | mysql my_database
96.8MB 0:00:17 [5.51MB/s] [==>                                ] 11% ETA 0:02:10

When it comes to MySQL, your restore rate is going to be different based on some different factors, so the ETA might not be entirely accurate.

Monitor MySQL restore progress with pv is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

The guide: Redundant Cloud Hosting Guide

As always, if you find anything in the guide that needs improvement, I'm all ears.

A simple guide to redundant cloud hosting is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

ERROR 1206 (HY000): The total number of locks exceeds the lock table size

MySQL is trying to tell you that it doesn't have enough room to store all of the row locks that it would need to execute your query. The only way to fix it for sure is to adjust innodb_buffer_pool_size and restart MySQL. By default, this is set to only 8MB, which is too small for anyone who is using InnoDB to do anything.

If you need a temporary workaround, reduce the amount of rows you're manipulating in one query. For example, if you need to delete a million rows from a table, try to delete the records in chunks of 50,000 or 100,000 rows. If you're inserting many rows, try to insert portions of the data at a single time.

Further reading:

• MySQL Bug #15667 - The total number of locks exceeds the lock table size
• MySQL Error 1206 » Mike R's Blog

MySQL: The total number of locks exceeds the lock table size is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

ERROR 1206 (HY000): The total number of locks exceeds the lock table size

InnoDB stores its lock tables in the main buffer pool. This means that the number of locks you can have at the same time is limited by the innodb_buffer_pool_size variable that was set when MySQL was started. By default, MySQL leaves this at 8MB, which is pretty useless if you're doing anything with InnoDB on your server.

Luckily, the fix for this issue is very easy: adjust innodb_buffer_pool_size to a more reasonable value. However, that fix does require a restart of the MySQL daemon. There's simply no way to adjust this variable on the fly (with the current stable MySQL versions as of this post's writing).

Before you adjust the variable, make sure that your server can handle the additional memory usage. The innodb_buffer_pool_size variable is a server wide variable, not a per-thread variable, so it's shared between all of the connections to the MySQL server (like the query cache). If you set it to something like 1GB, MySQL won't use all of that up front. As MySQL finds more things to put in the buffer, the memory usage will gradually increase until it reaches 1GB. At that point, the oldest and least used data begins to get pruned when new data needs to be present.

So, you need a workaround without a MySQL restart?

If you're in a pinch, and you need a workaround, break up your statements into chunks. If you need to delete a million rows, try deleting 5-10% of those rows per transaction. This may allow you to sneak under the lock table size limitations and clear out some data without restarting MySQL.

To learn more about InnoDB's parameters, visit the MySQL documentation.

MySQL: The total number of locks exceeds the lock table size is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

Fatal error: Call to undefined method wpdb::has_cap() in 
/path/to/wordpress/wp-admin/includes/schema.php on line 22

I was perplexed at the error, so I restored from a backup and began upgrading manually. The manual upgrades have always worked well for me in the past, so I figured this would probably fix the problem. After the upgrade, I went to /wp-admin/upgrade.php and saw:

Fatal error: Call to undefined method wpdb::has_cap() in 
/path/to/wordpress/wp-admin/includes/schema.php on line 22

What the heck is going on? I restored from a backup, tried the manual upgrade again, and it still failed. I took a look at the lines causing the problem in schema.php:

if ( $wpdb->has_cap( 'collation' ) ) {
	if ( ! empty($wpdb->charset) )
		$charset_collate = "DEFAULT CHARACTER SET $wpdb->charset";
	if ( ! empty($wpdb->collate) )
		$charset_collate .= " COLLATE $wpdb->collate";
}

I figured I could comment out the if statement and probably still be safe:

// if ( $wpdb->has_cap( 'collation' ) ) {
	if ( ! empty($wpdb->charset) )
		$charset_collate = "DEFAULT CHARACTER SET $wpdb->charset";
	if ( ! empty($wpdb->collate) )
		$charset_collate .= " COLLATE $wpdb->collate";
// }

Success? I could make it through the upgrade.php part fine at this point, but whenever I tried to add a tag to a post, it wasn't saving to the database. I caught this error in my apache logs:

[Fri Jun 12 23:45:03 2009] [error] [client 72.183.200.144] WordPress database error Duplicate entry 'debian' for key 'slug' for query INSERT INTO wp_terms (`name`,`slug`,`term_group`) VALUES ('debian','debian','0') made by wp_insert_term, referer: http://rackerhacker.com/wp-admin/post.php?action=edit&post=877

Frustration quickly ensued. I moved my /wp-content/ folder out of the way and replaced it with the standard WordPress stuff, but that didn't help. I moved plugins out of the way, one by one, but that didn't fix it either. Then I spotted a strange file sitting in /wp-content/ called db.php. When I opened it, I found a lot of database setup classes for mysqli.

I renamed it to db.pleasedonteverrunthisphp and I was able to save tags properly. So far, I haven't found any issues after I made chat change.

Does anyone know where that file might have come from? I don't remember adding it myself, so I'm wondering if it was ever packaged with a WordPress plugin or a WordPress installation. I hope this helps someone else!

Ugly upgrade path from WordPress 2.7.1 to 2.8 is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

If you don't want to downgrade to PHPMyAdmin 2.x, just add the following line to the top of your config.inc.php file:

$cfg['InitialSlidersState'] = 'open';

This will cause the indexes to be displayed when you click Structure for a certain table. By default, they are hidden.

Sidenote: Some of you might be thinking: "Hey, you're a DBA, you should know MySQL queries without needing PHPMyAdmin." You're right. I do know how to get the job done without PHPMyAdmin, but I enjoy the way PHPMyAdmin allows me to visualize my table structures. Also, it's a handy way to present data to others very quickly.

PHPMyAdmin 3.x hides the table indexes is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

While it won't include all situations, it will give ideas to those who are just starting down the path of MySQL optimization. If you find any corrections or if you have suggestions for improvements, please let me know!

UPDATE: I've received a lot of positive feedback via twitter and e-mail about the flow chart idea. I'll try to work on some more charts soon and get them onto the site. Let me know if there are specific topics that you're interested in.

MySQL performance flow chart is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

If you find yourself needing to write a Rails application without a database, just do the following:

For Rails 1.0 and up:
config/environment.rb:

config.frameworks -= [ :active_record ]

test/test_helper.rb

class Test::Unit::TestCase
  self.use_transactional_fixtures = false
  self.use_instantiated_fixtures  = false
  def load_fixtures
  end
end

For Rails 2.1 and up: Comment out both of the lines that begin with ActiveRecord::Base in config/initializers/new_rails_defaults.rb:

if defined?(ActiveRecord)
  # Include Active Record class name as root for JSON serialized output.
  # ActiveRecord::Base.include_root_in_json = true
 
  # Store the full class name (including module namespace) in STI type column.
  # ActiveRecord::Base.store_full_sti_class = true
end

For more details, review the full article on rubyonrails.org.

Writing a Ruby on Rails application without using a database is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

As always, I'd like to personally thank everyone who has made this release possible. Your efforts have provided benefits from MySQL users worldwide, and they have shown me how rewarding it is to maintain open source software.

MySQLTuner 1.0.0 is now available is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

If you want to know when the latest releases are available, there are multiple options:

• Follow RackerHacker on Twitter
• Sign up for the MySQLTuner mailing list
• Use the --checkversion option when you run the script

MySQLTuner 0.9.9 is now available is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

Easy download!
You can download it simply by running wget mysqltuner.pl. You will automatically be redirected to the script.

Access servers remotely
If you have multiple servers that you need to tune and monitor, simply download MySQLTuner onto one machine. You can run it against other servers by specifying a hostname, port, and login credentials. Two new options are available: --host and --port.

Pass login credentials on the command line
Two new options are available: --user and --pass. You can pass a username and password on the command line to log in quickly. This can be especially helpful in conjunction with cron.

Manually set the amount of RAM and swap memory installed
I've received a few bug reports where users on certain virtual environments saw incorrect memory calculations when they ran MySQLTuner. You can now set the current amount of RAM and swap memory installed with --forcemem and --forceswap respectively. (Thanks to Jason Gill for the patch and the bug report!)

Checking for updates is now optional
Some operating system distributions and environments can't allow for automatic update checking due to security concerns, so I've made update checks optional. The --skipversion option has been removed and it was replaced with --checkversion. As a side note, the only data that I collected was the version number being run and the IP from which the update originated. This data has only been used for aggregate statistical purposes.

As you might imagine, MySQLTuner 1.0 is just around the corner. I've been holding out for the MySQL 5.1 GA release, but I may release the script sooner. Don't worry - as soon as MySQL 5.1 becomes a GA release, I'll be hard at work to support any new optimization options which it provides.

MySQLTuner 0.9.8 is now available is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

# cat /etc/fedora-release
Fedora release 9 (Sulphur)
# yum info mysqltuner
Loaded plugins: fastestmirror, priorities, refresh-packagekit
Loading mirror speeds from cached hostfile
* updates: mirrors.usc.edu
* fedora: mirror.unl.edu
* livna: mirrors.tummy.com
Available Packages
Name : mysqltuner
Arch : noarch
Version : 0.9.1
Release : 4
Size : 11 k
Repo : updates
Summary : MySQL high performance tuning script
URL : http://mysqltuner.com/
License : GPLv3+
Description: MySQLTuner is a MySQL high performance tuning script written in perl that will provide you with a snapshot of a MySQL server's health. Based
: on the statistics gathered, specific recommendations will be provided that will increase a MySQL server's efficiency and performance. The
: script gives you automated MySQL tuning that is on the level of what you would receive from a MySQL DBA.

In addition to Ville, I'd like to thank Jason Tibbitts for reviewing and approving the new package.

MySQLTuner is now a Fedora 9 package! is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

MySQLTuner 0.9.1 available for Mandriva users is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

MySQLTuner now checks for fragmented tables
When deletes or updates are made on tables, MySQL will often leave holes behind that it hopes to fill in later. If the size and quantity keep climbing, the holes can cause performance degradation for writes and reads. Fragmentation can be corrected with OPTIMIZE TABLE, and the script recommends it if needed.

Fixed a bug where zero-length passwords cause authentication to repeat
The script will now allow you to have a zero-length password, and it won't re-prompt for the password over and over again.

Fixed a wget 1.11 timestamp bug
This can cause the version check to fail if the .wgetrc has timestamps enabled.

Corrected a math error in the temporary table calculation
The script should now be able to more accurately determine the relative quantity of temporary tables created on disk.

Fixed an error when the status variable Open_tables returned zero
The divide by zero error has been corrected.

Added table cache changes in preparation for MySQL 6
It's still in the early stages, but MySQLTuner should have full support for MySQL 6 by the time it reaches RC status.

Thanks for the contributions!
This update would not have been possible without help from Ville Skytta, Trent Hornibrook and Luuk Vosslamber.

To download the latest copy, visit the MySQLTuner page.

MySQLTuner v0.9.1 is now available is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

I've discovered one of the most awesome applications for a Mac called ScreenFlow (#3 on my list after #1 TextMate and #2 Adium and I decided to put it to use for my first screencast.

This screencast will teach you how to write a basic PHP script that will query data in MySQL. It is designed for users who are familiar with Linux but are just getting started with PHP. Here are the basic parts of the screencast:

» Import a MySQL dump file
» Connect to MySQL with PHP
» Select a default database
» Issue a query and retrieve results
» Style the rows retrieved into a table with alternating row colors

Download the screencast in Quicktime format (80.9MB)

NEW: Uploaded the video to Vimeo as well.

As always, your comments are welcome. This is my first screencast, and I'm sure I can make some improvements on future videos. Please let me know what I can change to make these screencasts better for you.

Screencast: Write a PHP script to query MySQL is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

MySQLTuner is the Debian Package of the Day! is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

Plesk may display an error on a white page that looks something like:

Cannot initialize InnoDB

This could mean that InnoDB support was disabled when MySQL was started. To correct this issue, search through the /etc/my.cnf for this line:

skip-innodb

If you find it in your configuration, remove it, and then restart MySQL. To test that InnoDB is enabled, you can refresh the Plesk page, or you can log into MySQL and run SHOW ENGINES. The output from the SHOW ENGINES statement should show YES on the line with InnoDB.

Should DISABLED appear instead, you may have an issue with your InnoDB configuration in your /etc/my.cnf. Be sure to check for innodb_data_file_path and make sure that it is set to an appropriate value.

A value of NO is not a good sign. This means that your version of MySQL was compiled without InnoDB support. This means that it cannot be enabled at runtime because MySQL wasn't built with any support for InnoDB. Be sure to recompile MySQL with --with-innodb or obtain a new package for your operating system which includes InnoDB support.

If you suspect that your MySQL InnoDB configuration is incorrect, you may want to review this documentation on MySQL's site:

For MySQL 5: 13.2.3. InnoDB Configuration
For MySQL 4/3.23: 13.2.4. InnoDB Configuration

After Plesk upgrade, "Cannot initialize InnoDB" is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

So, I have two questions for the general techy community:

1. What tools/applications would you recommend on a Mac for making high-quality screencasts that have a professional feel?

2. Would you pay for these screencasts (if they are really good), and if so, how much would you want to spend in total to get all of the screencasts for the DBA certification?

Feel free to add comments to this post, or you're welcome to drop me an e-mail at major at mhtx dot net. Your feedback is greatly appreciated!

Proposed MySQL DBA Screencasts is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

PHP and MySQL Web Development by Luke Welling, Laura Thomson
Barnes & Noble: http://snurl.com/265xp

Why do I like this book so much?

• Teaching by application - The book teaches fundamentals by showing how to apply techniques to an active website. There's not a ton of theory to wade through, and you feel like you're learning the material faster.
• Intertwined strategies - You learn how to get PHP working with MySQL, and then you learn how to optimize your code. It's important to know which work is best done by PHP and which is best done by MySQL. This book teaches both.
• Lots of examples - The CD-ROM comes with tons of code examples that actually relate to something you can use.

I'd be happy to loan my copy, but I've loaned it out and it never returned.

Best PHP and MySQL development book is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

Some of you may be thinking, "Okay, this makes sense." If that's you, skip the next two paragraphs. If not, keep reading.

Analogies can help understand topics like these. Here's one that I came up with during a training class. Consider two people sitting in front of a notepad on a table. Let's say that a sentence like "The quick brown fox jumps over the lazy dog" is already written on the notepad. If both people want to read the sentence simultaneously, they can do so without getting in each other's way. A third or fourth person could show up and they could all read it at the same time.

Well, let's say one of the people at the table is writing a screenplay for Cujo, and they want to change "lazy" to "crazy". That person erases the "l" in "lazy" and then adds a "cr" to the front to spell "crazy". So if the other person is reading the sentence while the first person is writing, they will see "lazy" turn into "azy", then "c_azy", and then finally, "crazy". This isn't a big issue in real life, but on the database level, this could be dangerous. If the person who was reading the sentence showed up during the middle of the letter changes, they would think that the dog was "azy", and they'd walk away wondering what the adjective "azy" means. To get around this, MySQL uses locking to block clients from reading data while it's being written and it blocks clients from writing data simultaneously.

Now that we're all familiar with what locks are, and why MySQL uses them, let's talk about some ways to reduce the delays caused by locking. Here's some situations you might be running up against:

Writes are delayed because reads have locked the tables
This is the most common occurrence from the servers that I have seen. When you run a SHOW PROCESSLIST, you may see a few reads at the top of the queue that are in the status of "Copying to tmp table" and/or "Sending data". On optimized servers running optimized queries, these should clear out quickly. If you're finding that they are not clearing out quickly, try the following:

• Use EXPLAIN on your queries to be sure that they are optimized
• Add indexes to tables that you query often
• Reduce the amount of rows that are being returned per query
• Upgrade the networking equipment between web and database servers (if applicable)
• Consider faster hardware with larger amounts of RAM
• Use MySQLTuner to check your current server's configuration for issues
• Consider moving to InnoDB to utilize row-based locking

Reads and writes are delayed because writes have locked the tables
Situations like these are a little different. There's two main factors to consider here: either MySQL cannot write data to the disk fast enough, or your write queries (or tables) are not optimized. If you suspect a hardware issue, check your iowait with sar and see if it stays at about 10-20% or higher during the day. If it does, slow hardware may be the culprit. Try moving to SCSI disks and be sure to use RAID 5 or 10 for additional reliability and speed. SAN or DAS units may also help due to higher throughput and more disk spindles.

If you already have state-of-the-art hardware, be sure that your tables and queries are optimized. Run OPTIMIZE TABLES regularly if your data changes often to defragment the tables and clear out any holes from removed or updated data. Slow UPDATE queries suggest that you are updating too many rows, or you may be using a column in the WHERE clause that is not indexed. If you do a large amount of INSERT queries, use this syntax to enter multiple rows simultaneously:

INSERT INTO table (col1,col2) VALUES ('a','1'), ('b','2'), ('c','3');

This syntax tells MySQL to hold off on updating indexes until the entire query is complete. If you are updating a very large amount of rows, and you need to use multiple queries to avoid reaching the max_allowed_packet directive, you can do something like this:

ALTER TABLE table DISABLE KEYS;
INSERT INTO table (col1,col2) VALUES ('a','1'), ('b','2'), ('c','3');
~~~ many more inserts ~~~
ALTER TABLE table ENABLE KEYS;

This forces MySQL to not calculate any new index information until you re-enable the keys or run OPTIMIZE TABLE. If all of this does not help, consider using InnoDB as your storage engine. You can benefit from the row-level locking, which reduces locking in mixed read/write scenarios. In addition, InnoDB is able to write data much more efficiently than MyISAM.

Reducing locking delays in MySQL is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

Fixed a bug in the enumeration/sizing of tables in MySQL 5
In MySQL 5 on some distributions, a NULL is returned for the storage engine and data length. Luuk Vosslamber quickly e-mailed me about the bug yesterday and it has been fixed.

MySQLTuner version checking
MySQLTuner will now check to see if a new version is available when the script runs. You can disable the check with the --skipversion option if you do not wish to perform the check.

The version check does not submit any information about the server, the MySQL installation, or any of your MySQL data. It simply queries a page on mysqltuner.com with the version number of your currently running script. Based on the value returned by the page, MySQLTuner will alert you if a new version is available.

To download the new version right now, please go to the project page and use the download links.

MySQLTuner v0.9.0 is now available is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

Table enumeration and sizing can now be skipped
I've received reports that MySQLTuner will stall while enumerating tables on servers that contain a lot of tables or a lot of large tables.  You can now use the --skipsize option to skip over the table enumeration process and let the script finish.  However, you will not be able to receive some recommendations since the script was unable to determine which storage engines are enabled.

New table enumeration and sizing method for MySQL 5
The script now uses information_schema to enumerate tables and their sizes for MySQL 5 servers.  This has provided a drastic improvement in performance.

Readability improvements
The recommendations for query_cache_limit and max_heap_table_size/tmp_table_size have been improved.

As always, I welcome your suggestions, bug reports, and questions!  Please feel free to drop a comment on this blog posting or send me an e-mail (it's in the script).  Also, don't forget to sign up for the MySQLTuner version announcement list.

MySQLTuner v0.8.9 is now available is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

ERROR 1118 (42000) at line 1686: Row size too large. The maximum row size for the used table type, not counting BLOBs, is 65535. You have to change some columns to TEXT or BLOBs

You really don't have any options in this situation. You'll need to adjust your table on the 64-bit server for good and then make a new dump file, or you will just have to live with the fact that it can't be imported into a 32-bit instance of MySQL.

Importing MySQL dumps made on 64-bit servers is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

Newlines are placed between the sections for increased readability
Each section now contains extra lines to set the sections apart. It makes the output a little longer, but easier to read as well.

Storage engine status color bug
Even if the --nocolor option was passed, the storage engine statuses were being shown in color.

Excluded information_schema from storage engine calculations
The information_schema database was causing extra MEMORY tables to show up in the calculations. They're now excluded when the calculations are being made.

Shawn Ashlee has also been added as a contributor as he's been a constant help for the project. He's recommended implementation ideas and he has worked to create internal MySQLTuner RPM's for use at Rackspace. Thanks, Shawn!

MySQLTuner v0.8.6 is now available is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.