Using ATA Over Ethernet (AoE) On Ubuntu 12.04 (Initiator And Target)

Version 1.0
Author: Falko Timme
Follow me on Twitter

This guide explains how you can set up an AoE target and an AoE initiator (client), both running Ubuntu 12.04. AoE stands for "ATA over Ethernet" and is a storage area network (SAN) protocol which allows AoE initiators to use storage devices on the (remote) AoE target using normal ethernet cabling. "Remote" in this case means "inside the same LAN" because AoE is not routable outside a LAN (this is a major difference compared to iSCSI). To the AoE initiator, the remote storage looks like a normal, locally-attached hard drive.

I do not issue any guarantee that this will work for you!

1 Preliminary Note

I'm using two Ubuntu 12.04 servers here:

• server1.example.com (Initiator): IP address 192.168.0.100
• server2.example.com (Target): IP address 192.168.0.101

2 Loading The aoe Kernel Module On Both Systems

server1/server2:

Before we start, we must make sure that the the kernel supports AoE:

grep ATA_OVER /boot/config-`uname -r`

This should display something like this:

root@server1:~# grep ATA_OVER /boot/config-`uname -r`
 CONFIG_ATA_OVER_ETH=m
 root@server1:~#

This means that AoE was built as a kernel module. Let's check if the module is already loaded:

lsmod | grep aoe

If you get nothing back, this means it's not loaded. In this case we can load it as follows:

modprobe aoe

Let's check again if the module is loaded:

lsmod | grep aoe

root@server1:~# lsmod | grep aoe
 aoe                    26960  0
 root@server1:~#

To have the module loaded automatically when the system boots, we add the aoe module to /etc/modules:

vi /etc/modules

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.

loop
lp
rtc
aoe

3 Setting Up The Target (server2)

server2:

First we set up the target (server2):

apt-get install vblade

We can use unused logical volumes, image files, hard drives (e.g. /dev/sdb), hard drive partitions (e.g. /dev/sdb1) or RAID devices (e.g. /dev/md0) for the storage. In this example I will create a logical volume of 20GB named storage1 in the volume group vg0:

lvcreate -L20G -n storage1 vg0

(If you want to use an image file, you can create it as follows:

mkdir /storage
 dd if=/dev/zero of=/storage/storage1.img bs=1024k count=20000

This creates the image file /storage/storage1.img with a size of 20GB.

)

Now we export our storage device as follows:

vbladed 0 1 eth0 /dev/vg0/storage1

The first number (0) is the shelf number (major), the second (1) the slot number (minor), change these numbers to your liking. Each AoE device is identified by a couple major/minor which must be unique (if you are exporting multiple devices), with major between 0-65535 and minor between 0-255. The eth0 part tells vbladed which ethernet device to use (if you ethernet device is eth1, then use eth1 - you can find out about your ethernet devices by running

ifconfig

).

To start the export automatically whenever you boot the target, open /etc/rc.local...

vi /etc/rc.local

... and add the following line to it (before the exit 0 line):

[...]
vbladed 0 1 eth0 /dev/vg0/storage1
[...]