Install Openldap From Source And Configure Multi-Master Replication

Author: Nitin Bhadauria
Version: 1.0

Going forward to my previous document on setting up a Qmail Server with Openldap, I am now sharing a doc on how we can set up an Openldap in multi-master replication mode. So If you want more than one ldap-server for redundancy here we go:

1 Preliminary Note

In this tutorial I will use two demo servers, just to make steps simple, But you can replicate the same setup on more then two servers.

server1.example.com: IP address 192.168.0.100
server2.example.com: IP address 192.168.0.101

Because we will run all the steps from this tutorial with root privileges, you can either pretend all commands in this tutorial with the string sudo, or we become root right now by typing:

sudo su

Two servers should be able to resolve the other systems' hostnames. If this cannot be done through DNS, you should edit the /etc/hosts file so that it looks as follows on all three systems:

vi /etc/hosts

127.0.0.1      localhost.localdomain   localhost
192.168.0.100  server1.example.com     server1
192.168.0.101  server2.example.com     server2
 

2 Prerequisites

a. Before compiling we will install some dependencies:

yum install libacl-devel libblkid-devel gnutls-devel readline-devel python-devel autoconf gcc-c++ gcc glibc-devel glibc-headers kernel-headers libgomp libstdc++-devel openssl-devel e2fsprogs-devel keyutils-libs-devel krb5-devel libselinux-devel libsepol-devel libtool-ltdl-devel

b. Before installing Openldap we need to install latest Oracle Berkeley DB.

wget http://download.oracle.com/berkeley-db/db-4.7.25.tar.gz
tar xvf db-4.7.25.tar.gz
cd db-4.7.25
cd build_unix
../dist/configure
make
make install
ls /usr/local/BerkeleyDB.4.7/
cp -p /usr/local/BerkeleyDB.4.7/bin/db_* /usr/bin/
cp -p /usr/local/BerkeleyDB.4.7/lib/* /usr/lib
cp -p /usr/local/BerkeleyDB.4.7/include/* /usr/include/
mv /usr/lib/libdb-4.7.so /usr/lib/libdb-4.7.so.0.0.0
ln -s /usr/lib/libdb-4.7.so.0.0.0 /usr/lib/libdb-4.7.so
ldconfig

3 Install Openldap

a. install the OpenLDAP server from source:

wget ftp://ftp.openldap.org/pub/OpenLDAP/openldap-release/openldap-2.4.26.tgz
tar xvf openldap-2.4.26.tgz
cd openldap-2.4.26
./configure --prefix=/usr/local/openldap --enable-overlays=mod --enable-modules --enable-bdb --enable-accesslog --enable-auditlog --enable-collect --enable-memberof --enable-syncprov
make depend
make
make install

Try to start the service and check it started without any error. To make sure we are good to go.

/usr/local/openldap/libexec/slapd -d 5

Note: Press "ctrl + c"  to exit.

Now we will follow the same steps to install the Openldap on other server.

4 Configure Multi-Master Replication

Now we will configure the replication although i will include most of the important configuration just to make sure that the configuration are in correct order (which is important because you can’t just put any line anywhere in file).

On server1 (192.168.0.100):

vi /usr/local/openldap/etc/openldap/slapd.conf

pidfile /usr/local/openldap/var/run/slapd.pid
argsfile /usr/local/openldap/var/run/slapd.args

# Load the required modules
moduleload syncprov.la
moduleload accesslog.la
moduleload back_bdb.la

#Define the server ID.
serverID 1

# Make sure you change below configuration as your need

database bdb
suffix " dc=example,dc=com "
rootdn "cn=ldadmin,dc=example,dc=com "
rootpw {SSHA}MxGntcb+QdYimYqbly7IOCY2ZJ0SxqCZ # Generate password using "slappasswd"
directory /usr/local/openldap/var/openldap-data

# These are basic performances configuration required
checkpoint 10240 720 # check point whenever 10M data bytes written or 24Hr has elapsed whichever occurs first
cachesize 50000 # LDAP maintains 50,000 entries in memory

# These configurations are to set the default database parameters
dbconfig set_cachesize 0 524288000 1 # Set the database in memory cache size to 500 MB, Tuning this value can greatly effect your database performance.
dbconfig set_lk_max_locks 3000
dbconfig set_lk_max_objects 1500
dbconfig set_lk_max_lockers 1500
dbconfig set_lg_regionmax 262144
dbconfig set_lg_bsize 2097152

# Replication configuration, only things you may have to change here are provider, binddn, credentials and searchbase.
syncrepl rid=001
    provider=ldap://server2.example.com:389
    binddn="cn=ldadmin,dc=example,dc=com "
    bindmethod=simple
    credentials=secret
    searchbase=" dc=example,dc=com "
    type=refreshAndPersis
    interval=00:00:00:10
    retry="5 5 300 5"
    timeout=1

index objectClass eq

#Rest replication configuration goes to end of the file.
mirrormode TRUE
overlay syncprov
syncprov-nopresent TRUE
syncprov-reloadhint TRUE
syncprov-checkpoint 1000 60 

Now configure server2 (192.168.0.101):

vi /usr/local/openldap/etc/openldap/slapd.conf

pidfile /usr/local/openldap/var/run/slapd.pid
argsfile /usr/local/openldap/var/run/slapd.args
moduleload syncprov.la
moduleload accesslog.la
moduleload back_bdb.la
serverID 1

database bdb
suffix " dc=example,dc=com "
rootdn "cn=ldadmin,dc=example,dc=com "
rootpw {SSHA}MxGntcb+QdYimYqbly7IOCY2ZJ0SxqCZ # Generate password using “slappasswd”
directory /usr/local/openldap/var/openldap-data

checkpoint 10240 720
cachesize 50000
dbconfig set_cachesize 0 524288000 1
dbconfig set_lk_max_locks 3000
dbconfig set_lk_max_objects 1500
dbconfig set_lk_max_lockers 1500
dbconfig set_lg_regionmax 262144
dbconfig set_lg_bsize 2097152

syncrepl rid=002
    provider=ldap://server1.example.com:389
    binddn="cn=ldadmin,dc=example,dc=com "
    bindmethod=simple
    credentials=secret
    searchbase="dc=example,dc=com "
    type=refreshAndPersist
    interval=00:00:00:10
    retry="5 5 300 5"
    timeout=1

index objectClass eq

mirrormode TRUE
overlay syncprov
syncprov-nopresent TRUE
syncprov-reloadhint TRUE
syncprov-checkpoint 1000 60 

5 Configure Startup Script File

First create an account to run ldap service:

groupadd ldap
useradd -g ldap -d /usr/local/openldap/var/openldap-data/ -s /bin/false -p'*' ldap

As openldap didn't include any scripts in package we will create one:

vi /etc/init.d/ldap

#!/bin/bash
#
# ldap	This shell script takes care of starting and stopping
#	ldap servers (slapd and slurpd).
#
# chkconfig: - 27 73
# description: LDAP stands for Lightweight Directory Access Protocol, used \
#              for implementing the industry standard directory services.
# processname: slapd
# config: /usr/local/openldap/etc/openldap/slapd.conf
# pidfile: /usr/local/openldap/var/run/slapd.pid

# Source function library.
. /etc/init.d/functions

# Source networking configuration and check that networking is up.
if [ -r /etc/sysconfig/network ] ; then
	. /etc/sysconfig/network
	[ ${NETWORKING} = "no" ] && exit 1
fi

# Source an auxiliary options file if we have one, and pick up OPTIONS,
# SLAPD_OPTIONS, SLURPD_OPTIONS, SLAPD_LDAPS, SLAPD_LDAPI, and maybe
# KRB5_KTNAME and SLURPD_KRB5CCNAME.
if [ -r /etc/sysconfig/ldap ] ; then
	. /etc/sysconfig/ldap
fi

#slapd=/usr/sbin/slapd
slapd=/usr/local/openldap/libexec/slapd
slurpd=/usr/sbin/slurpd
slaptest=/usr/local/openldap/sbin/slaptest
[ -x ${slapd} ] || exit 1
[ -x ${slurpd} ] || exit 1

RETVAL=0

#
# Pass commands given in $2 and later to "test" run as user given in $1.
#
function testasuser() {
    local user= cmd=
    user="$1"
    shift
    cmd="$@"
    if test x"$user" != x ; then
	if test x"$cmd" != x ; then
	    /sbin/runuser -f -m -s /bin/sh -c "test $cmd" -- "$user"
	else
	    false
	fi
    else
	false
    fi
}

#
# Check for read-access errors for the user given in $1 for a service named $2.
# If $3 is specified, the command is run if "klist" can't be found.
#
function checkkeytab() {
    local user= service= klist= default=
    user="$1"
    service="$2"
    default="${3:-false}"
    if test -x /usr/kerberos/bin/klist ; then
	klist=/usr/kerberos/bin/klist
    elif test -x /usr/bin/klist ; then
	klist=/usr/bin/klist
    fi
    KRB5_KTNAME="${KRB5_KTNAME:-/etc/krb5.keytab}"
    export KRB5_KTNAME
    if test -s "$KRB5_KTNAME" ; then
	if test x"$klist" != x ; then
	    if LANG=C $klist -k "$KRB5_KTNAME" | tail -n 4 | awk '{print $2}' | grep -q ^"$service"/ ; then
		if ! testasuser "$user" -r ${KRB5_KTNAME:-/etc/krb5.keytab} ; then
		    true
		else
		    false
		fi
	    else
		false
	    fi
	else
	    $default
	fi
    else
	false
    fi
}

function configtest() {
	local user= ldapuid= dbdir= file=
	# Check for simple-but-common errors.
	user=ldap
	prog=`basename ${slapd}`
	ldapuid=`id -u $user`
	# Unaccessible database files.
	for dbdir in `LANG=C egrep '^directory[[:space:]]+[[:print:]]+$' /usr/local/openldap/etc/openldap/slapd.conf | sed s,^directory,,` ; do
		for file in `find ${dbdir}/ -not -uid $ldapuid -and \( -name "*.dbb" -or -name "*.gdbm" -or -name "*.bdb" -or -name "__db.*" \)` ; do
			echo -n $"$file is not owned by \"$user\"" ; warning ; echo
		done
	done
	# Unaccessible keytab with an "ldap" key.
	if checkkeytab $user ldap ; then
		file=${KRB5_KTNAME:-/etc/krb5.keytab}
		echo -n $"$file is not readable by \"$user\"" ; warning ; echo
	fi
	# Unaccessible TLS configuration files.
	tlsconfigs=`LANG=C egrep '^(TLS_CACERT|TLSCACertificateFile|TLSCertificateFile|TLSCertificateKeyFile)[[:space:]]' /usr/local/openldap/etc/openldap/slapd.conf /usr/local/openldap/etc/openldap/ldap.conf | awk '{print $2}'`
	for file in $tlsconfigs ; do
		if ! testasuser $user -r $file ; then
			echo -n $"$file is not readable by \"$user\"" ; warning ; echo
		fi
	done
	# Check the configuration file.
	slaptestout=`/sbin/runuser -m -s "$slaptest" -- "$user" "-u" 2>&1`
	slaptestexit=$?
	if test $slaptestexit == 0 ; then
		if echo "$slaptestout" | grep -v "config file testing succeeded" >/dev/null ; then
			echo -n $"Checking configuration files for $prog: " ; warning ; echo
			echo "$slaptestout"
		fi
	else
		echo -n $"Checking configuration files for $prog: " ; failure ; echo
		echo "$slaptestout"
		if /sbin/runuser -m -s "$slaptest" -- "$user" "-u" &>/dev/null ; then
			for directory in `LANG=C egrep '^directory[[:space:]]+[[:print:]]+$' /usr/local/openldap/etc/openldap/slapd.conf | sed s,^directory,,` ; do
				if test -r $directory/__db.001 ; then
					echo -n $"stale lock files may be present in $directory" ; warning ; echo
				fi
			done
		fi
		exit 1
	fi
}

function start() {
	configtest
	# Define a couple of local variables which we'll need. Maybe.
	user=ldap
	prog=`basename ${slapd}`
	if test x$SLAPD_LDAP = xyes ; then
	    harg="ldap:///"
    fi
	if grep -q ^TLS /usr/local/openldap/etc/openldap/slapd.conf || test x$SLAPD_LDAPS = xyes ; then
	    harg="$harg ldaps:///"
	fi
	if test x$SLAPD_LDAPI = xyes ; then
	    harg="$harg ldapi:///"
	fi
	# Start daemons.
	echo -n $"Starting $prog: "
	ulimit $ULIMIT_SETTINGS > /dev/null 2>&1
	daemon --check=$prog ${slapd} -h \"$harg\" -u ${user} $OPTIONS $SLAPD_OPTIONS
	RETVAL=$?
	echo
	if [ $RETVAL -eq 0 ]; then
	    if grep -q "^replogfile" /usr/local/openldap/etc/openldap/slapd.conf; then
		prog=`basename ${slurpd}`
		echo -n $"Starting $prog: "
		if [ -n "$SLURPD_KRB5CCNAME" ]; then
			export KRB5CCNAME="$SLURPD_KRB5CCNAME";
		fi
		daemon ${slurpd} $OPTIONS $SLURPD_OPTIONS
		RETVAL=$?
		echo
	    fi
	fi
	[ $RETVAL -eq 0 ] && touch /usr/local/openldap/var/lock/subsys/ldap
	return $RETVAL
}

function stop() {
	# Stop daemons.
	prog=`basename ${slapd}`
	echo -n $"Stopping $prog: "
	killproc -d $STOP_DELAY ${slapd}
	RETVAL=$?
	echo
	if [ $RETVAL -eq 0 ]; then
	    if grep -q "^replogfile" /usr/local/openldap/etc/openldap/slapd.conf; then
		prog=`basename ${slurpd}`
		echo -n $"Stopping $prog: "
		killproc ${slurpd}
		RETVAL=$?
		echo
	    fi
	fi
	[ $RETVAL -eq 0 ] && rm -f /usr/local/openldap/var/lock/subsys/ldap /usr/local/openldap/var/run/slapd.args
	return $RETVAL
}

# See how we were called.
case "$1" in
    configtest)
	configtest
	;;
    start)
	start
	RETVAL=$?
	;;
    stop)
	stop
	RETVAL=$?
	;;
    status)
	status ${slapd}
	RETVAL=$?
	if grep -q "^replogfile" /usr/local/openldap/etc/openldap/slapd.conf ; then
	    status ${slurpd}
	    RET=$?
	    if [ $RET -ne 0 ] ; then
		RETVAL=$RET;
	    fi
	fi
	;;
    restart)
	stop
	start
	;;
    condrestart)
	if [ -f /usr/local/openldap/var/lock/subsys/ldap ] ; then
	    stop
 	    start
	    RETVAL=$?
	fi
	;;
    *)
	echo $"Usage: $0 {start|stop|restart|status|condrestart}"
	RETVAL=1
esac

exit $RETVAL

You may have to make some aditional directory required:

mkdir -p /usr/local/openldap/var/{run,lock/subsys}
chown ldap: -R /usr/local/openldap

Now, make this script executable and change its default permissions:

chmod 700  /etc/init.d/ldap
chkconfig --add ldap
chkconfig --level 345 ldap on

Start your OpenLDAP Server manually with the following command:

/etc/init.d/ldap start

6 Migration

Only follow these steps if you are setting replication on already running Openldap server:

a. First copy all the schema used in your running setup, I would suggest sync the whole directory.

rsync -av /usr/local/openldap/etc/openldap/schema root@server2:/usr/local/openldap/etc/openldap/schema

Now remember to include the these schema in your configuration:

vi /usr/local/openldap/etc/openldap/slapd.conf

include         /usr/local/openldap/etc/openldap/schema/core.schema
include         /usr/local/openldap/etc/openldap/schema/cosine.schema
include         /usr/local/openldap/etc/openldap/schema/inetorgperson.schema
include         /usr/local/openldap/etc/openldap/schema/nis.schema
include         /usr/local/openldap/etc/openldap/schema/qmail.schema

b. Now import all the data from server1 to server2.

on server1:

ldapsearch -x -b 'dc=example,dc=com' > master.ldif
scp master.ldif root@server2:

on server2:

ldapmodify -cvx -D'cn=ldadmin,dc=example,dc=com' -W -f /root/master.ldif

Enter LDAP Password:

Now restart Ldap service simultaneously on both servers:

/etc/init.d/ldap restart

7 Configure Qmail And IMAP To Use Both servers

Now we will edit a qmail control file to define multiple ldap servers:  

vi /var/qmail/control/ldapserver

server1.example.com:389
server2.example.com:389

And courier-auth configuration to make imap use both the servers:

vi /etc/courier/authldaprc

LDAP_URI    ldap://server1.example.com, ldap://server2.example.com

Now just restart the services and all should be good :)

I will be publishing a doc to configure replication over SSL very soon...