How To Install Repcached (Memcached Replication) For High-Availability Over 2 Nodes On Ubuntu 11.04

This is a short tutorial based on MarcusSpiegel's howto found here. This how to will cover the missing parts for installing Repcached on Ubuntu server 11.04. This is a copy/paste friendly tutorial so using putty will make this a lot easier.

This how to will show you how to install and build memcached with replication, create a startup script and configure PHP to use memcache for sessions rather than storing them as files. This tutorial assumes you have set up 2 servers to replicate the information over.

1. My Setup

In this demo I will be setting up repcache on 2 hosts, they will have both lighttpd and repcache installed on them.

• Web Server 1: server1.example.com, IP Address: 10.10.20.10; This will be refered to as server1.
• Web Server 2: server2.example.com, IP Address: 10.10.20.11; This will be refered to as server2.

2. Installing repcache

First you need to visit http://repcached.lab.klab.org/ and download the lastest version (latest at time of writing: 2.2-1.2.8). After downloading the tar file, we will need to install some extra packages. Do this on both nodes.

apt-get install libevent-dev g++ make

From here we can continue the installation.

tar xvf memcached-1.2.8-repcached-2.2.tar
cd memcached-1.2.8-repcached-2.2/
./configure --enable-replication
make
make install

3. Configuring repcache

Repcache is now installed, binary location is /usr/local/bin/memcached. From here we will create the config file and startup script. Config file and init script are the ones found on MarcusSpiegel's howto. Do this on both nodes.

First open the file with vi:

vi /etc/memcachedrep

Then enter the following, remember to change the IP address at the end. This IP will be the other server you wish to replicate to.

## extra commandline options to start memcached in replicated mode
# -x < ip_addr > hostname or IP address of the master replication server
# -X < num > TCP port number of the master (default: 11212)
DAEMON_ARGS="-m 64 -p 11211 -u root -P /var/run/memcachedrep.pid -d -x 10.10.20.11"

Once the file is saved we need to create the init script. Once again this is the script found on MarcusSpiegel's howto with some minor changes for config location only.

vi /etc/init.d/memcachedrep

#! /bin/sh
### BEGIN INIT INFO
# Provides:             memcached
# Required-Start:       $syslog
# Required-Stop:        $syslog
# Should-Start:         $local_fs
# Should-Stop:          $local_fs
# Default-Start:        2 3 4 5
# Default-Stop:         0 1 6
# Short-Description:    memcached - Memory caching daemon replicated
# Description:          memcached - Memory caching daemon replicated
### END INIT INFO
# Author: Michael 
#
# Please remove the "Author" lines above and replace them
# with your own name if you copy and modify this script.
# Do NOT "set -e"
# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="memcachedrep"
NAME=memcached
DAEMON=/usr/local/bin/$NAME
DAEMON_ARGS="--options args"
PIDFILE=/var/run/memcachedrep.pid
SCRIPTNAME=/etc/init.d/$DESC
VERBOSE="yes"
# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0
# Read configuration variable file if it is present
[ -r /etc/$DESC ] && . /etc/$DESC
# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh
# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions
#
# Function that starts the daemon/service
#
do_start()
{
	# Return
	#   0 if daemon has been started
	#   1 if daemon was already running
	#   2 if daemon could not be started
	start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
		|| return 1
	start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
		$DAEMON_ARGS \
		|| return 2
	# Add code here, if necessary, that waits for the process to be ready
	# to handle requests from services started subsequently which depend
	# on this one.  As a last resort, sleep for some time.
}
#
# Function that stops the daemon/service
#
do_stop()
{
	# Return
	#   0 if daemon has been stopped
	#   1 if daemon was already stopped
	#   2 if daemon could not be stopped
	#   other if a failure occurred
    start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
    RETVAL="$?"
    [ "$RETVAL" = 2 ] && return 2
	# Wait for children to finish too if this is a daemon that forks
	# and if the daemon is only ever run from this initscript.
	# If the above conditions are not satisfied then add some other code
	# that waits for the process to drop all resources that could be
	# needed by services started subsequently.  A last resort is to
	# sleep for some time.
	start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON
	[ "$?" = 2 ] && return 2
	# Many daemons don't delete their pidfiles when they exit.
	rm -f $PIDFILE
	return "$RETVAL"
}
#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
	#
	# If the daemon can reload its configuration without
	# restarting (for example, when it is sent a SIGHUP),
	# then implement that here.
	#
	start-stop-daemon --stop --signal 1 --quiet --pidfile $PIDFILE --name $NAME
	return 0
}
case "$1" in
  start)
	[ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
	do_start
	case "$?" in
		0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
		2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
	esac
	;;
  stop)
	[ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
	do_stop
	case "$?" in
		0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
		2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
	esac
	;;
  #reload|force-reload)
	#
	# If do_reload() is not implemented then leave this commented out
	# and leave 'force-reload' as an alias for 'restart'.
	#
	#log_daemon_msg "Reloading $DESC" "$NAME"
	#do_reload
	#log_end_msg $?
	#;;
  restart|force-reload)
	#
	# If the "reload" option is implemented then remove the
	# 'force-reload' alias
	#
	log_daemon_msg "Restarting $DESC" "$NAME"
	do_stop
	case "$?" in
	  0|1)
		do_start
		case "$?" in
			0) log_end_msg 0 ;;
			1) log_end_msg 1 ;; # Old process is still running
			*) log_end_msg 1 ;; # Failed to start
		esac
		;;
	  *)
	  	# Failed to stop
		log_end_msg 1
		;;
	esac
	;;
  *)
	#echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2
	echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
	exit 3
	;;
esac
:

Save this then type in the following.

chmod +x /etc/init.d/memcacherep
update-rc.d memcachedrep defaults

This will make sure repcache is started everytime the server is restarted.

4. Testing

Before we move on we will test that the 2 nodes are replicating.

server1:

telnet 127.0.0.1 11211

Then type in

set foo 0 0 3
bar

You should see the word STORED appear under it. Type in

quit

and it will return to the console. Go to your other server and type in the following.

server2:

telnet 127.0.0.1 11211

Then type in

get foo

and it should return the value you entered on the first server. If it has then the replication is working.

5. Setting up PHP

Here I will show you how to setup PHP to use memcache. Because I am using lighttpd the directories may be different to those using apache2 or nginx. Type the following into the console and repeat for the secondary server.

vi /etc/php5/cgi/php.ini

Search down the file till you find the following:

[Session]
; Handler used to store/retrieve data.
; http://php.net/session.save-handler
session.save_handler = files

You will need to change these to look like this:

[Session]
; Handler used to store/retrieve data.
; http://php.net/session.save-handler
extension = memcache.so
session.save_handler = memcache
session.save_path = "tcp://127.0.0.1:11211"

Once these have been saved simply type in (remember to change this to the software you are using for the hosting):

/etc/init.d/lighttpd force-reload

Or simply reboot both servers. You can now test out that php is using them correctly by creating a session on one and reading that session from the other.