Running ZFS With CAM-based ATA On FreeBSD 8.1
As was mentioned in last Friday's article, Which Is Faster: Debian Linux or FreeBSD, tests of FreeBSD atop the ZFS file-system (rather than UFS2+S) are currently underway and those results are expected to be published in full later this week as the ZFS disk performance is compared directly to UFS2+S, UFS2+J, and also Ubuntu Linux with the EXT4 and Btrfs file-systems. Today though we have a few ZFS performance numbers to share as we look at the performance of the new CAM-ATA sub-system on FreeBSD.
FreeBSD's CAM-based infrastructure was introduced in the FreeBSD 8.0 kernel last year and deemed as experimental support for ATA/SATA devices, but with the FreeBSD 8.1 release its support is much more mature and considered by many to be production ready even though it is not yet enabled by default. By the FreeBSD kernel interfaces manual in FreeBSD 8.1, cam is described as:
"The CAM SCSI subsystem provides a uniform and modular system for the implementation of drivers to control various SCSI devices, and to utilize different SCSI host adapters through host adapter drivers. When the system probes the SCSI busses, it attaches any devices it finds to the appropriate drivers. The pass driver, if it is configured in the kernel, will attach to all SCSI devices."
Alexander Motin of FreeBSD described the CAM-based ATA infrastructure to us as "up to double performance difference in some random workloads" due to this new infrastructure supporting NCQ (Native Command Queuing). The CAM-ATA infrastructure also provides FreeBSD with support for Serial ATA Port Multipliers, better hot-plug support, and less code duplication between the different sub-systems and applications.
Until the CAM-based ATA infrastructure is enabled by default (perhaps in FreeBSD 8.2), FreeBSD's AHCI (Advanced Host Controller Interface) kernel module needs to be loaded, which can be done by adding ahci_load="YES" to /etc/loader.conf on a FreeBSD 8.0/8.1 system. The siis and mvs kernel modules also need to be loaded (via siis_load="YES" and mvs_load="YES" in the same loader.conf file) depending upon your hardware for supporting some Serial ATA host controllers from Silicon Image and Marvell, respectively. The /etc/fstab may also need to be updated to change the disks from /dev/ad0 to /dev/ada0 (assuming it is the first disk). From the motherboard's BIOS, the Serial ATA controller setting also needs to be set to operate in AHCI mode rather than any IDE / compatibility mode that is often the default option on many motherboards. To see whether CAM-ATA is being used, the camcontrol command can be used and/or the dmesg output can be checked.
Testing of the CAM-based ATA infrastructure was done on the Lenovo ThinkPad T61 used in last week's FreeBSD, Debian GNU/kFreeBSD, and Debian GNU/Linux performance comparison. This notebook has an Intel Core 2 Duo T9300 (Penryn) processor, 4GB of system memory, a 100GB Hitachi HTS72201 7200RPM Serial ATA 2.0 hard drive, and a NVIDIA Quadro NVS 140M graphics processor. For this ZFS and new ATA infrastructure benchmarking we used PC-BSD 8.1, which provides FreeBSD 8.1. Unlike the FreeBSD installer, the PC-BSD installer easily supports installing to a ZFS file-system without any hassles in the PC-BSD 8.1 release that has upgraded ZFS support. For today's tests and next week's FreeBSD/Linux file-system testing we used a root ZFS file-system using a basic ZFS pool. FreeBSD 8.1 provides version 14 of ZFS. The software stack being tested consists of the FreeBSD 8.1-RELEASE x86_64 kernel, KDE 4.4.45, X.Org Server 1.7.5, and GCC 4.2.1.
In these benchmarks we compared the performance of the traditional ATA infrastructure in FreeBSD/PC-BSD 8.1 to that of the new CAM-based ATA infrastructure when using ZFS. The tests included LZMA compression, Gzip compression, Compile Bench, PostMark, and the Threaded I/O Tester. Testing was done as usual by the Phoronix Test Suite.