Racker Hacker

NOTE: This is a sub-page off the main redundant cloud hosting configuration guide. If you've arrived at this page first, I recommend reviewing the parent page first.

End goal of this step

When you've completed this portion of the guide, you should have the following:

• a failover haproxy configuration for MySQL and memcached
• GlusterFS client mounts

Configuring haproxy

As of this post's writing, Fedora 13 does not yet have haproxy 1.4.x packaged. I recommend haproxy 1.4.x because the support for checking MySQL is vastly improved. To get it running, we'll need to compile it. I like to install Fedora's package anyway as it gives you some helpful stuff out of the bag (I hate writing init scripts).

yum -y install gcc gcc-c++ make haproxy
wget http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.8.tar.gz
tar xzf haproxy-1.4.8.tar.gz
cd haproxy-1.4.8
make TARGET=linux26
make install

You'll need to flip one line in the /etc/init.d/haproxy file to get your compiled haproxy to start up:

exec="/usr/sbin/haproxy"          [original]
exec="/usr/local/sbin/haproxy"    [change to this]

Drop in the new haproxy configuration file:

# cat /etc/haproxy/haproxy.cfg
global
    log         127.0.0.1 local2
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon
    # stats socket /var/lib/haproxy/stats
 
defaults
    mode                        http
    log                         global
    option                      dontlognull
    timeout connect             1000
    timeout client              300000
    timeout server              300000
    maxconn                     60000
    retries                     3
 
listen  admin_stats 0.0.0.0:5432
    mode                http
    stats uri           /stats
    stats realm         HAPROXY
    stats auth          youruser:htpasswd-password-hash
    stats refresh       5
 
listen mysql 127.0.0.1:3306
    mode tcp
    option mysql-check
    balance roundrobin
    server mysql1 10.1.100.10:3306 check port 3306
    server mysql2 10.1.100.15:3306 check port 3306 backup
 
listen memcached 127.0.0.1:11211
    mode tcp
    balance roundrobin
    server memcached1 10.1.100.10:11211 check port 11211
    server memcached2 10.1.100.15:11211 check port 11211 backup

Start haproxy and ensure it comes up on boot:

/etc/init.d/haproxy start
chkconfig haproxy on

Once it's running, test your connectivity from the web servers to the database servers using the MySQL client utility. You should be able to do something like this:

mysql --host=127.0.0.1 --port=3306 --user=root -p

To test memcached, just try telnetting to the port that haproxy opened for you on each web node:

# telnet localhost 11211
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
MEOW
ERROR

At this point, you may want to reboot one of your database servers to ensure that your haproxy failover is working properly. You can check the haproxy stats page on port 5432 to see the current status.

GlusterFS client

As with the previous guide for the database servers, you'll want to snag the most recent GlusterFS RPM's from Gluster's FTP site. Once they're installed, make your GlusterFS client mountpoint and configure your /etc/fstab:

mkdir -p /mnt/glusterfs
echo "/etc/glusterfs/glusterfs.vol	/mnt/glusterfs	glusterfs	defaults,noatime,log-level=DEBUG	0 0" >> /etc/fstab

When you ran glusterfs-volgen during the database server steps, a file with "client" in the name should have been created. Copy that file to /etc/glusterfs/glusterfs.vol on both of your web nodes. You should be able to mount your GlusterFS volume now:

mount /mnt/glusterfs

On some distributions, you may need to run modprobe fuse to load the FUSE kernel module first (Fedora's mount scripts take care of this for you). When you're done, you should end up with a mounted GlusterFS volume:

# mount | grep gluster
/etc/glusterfs/glusterfs.vol on /mnt/glusterfs type fuse.glusterfs (rw,allow_other,default_permissions,max_read=131072)

You can test it out by dropping a file into your GlusterFS mount on the primary node and then check for it on the secondary node. If it appears on both, you're all set.

Conclusion

After finishing this step, you should have haproxy running on both web nodes providing a way for you to communicate with MySQL and memcached on both database nodes. Also, you should have replicated and highly available storage via GlusterFS for your web content and shared configuration files.