The Mess That is Linux Volume Management

Posted by hkwint on Jul 24, 2008 7:11 AM EDT
LXer Linux news; By Hans Kwint, The Netherlands 		Mail this story
Print this story

LXer Feature: 24-Jul-2008

The GNU/Linux operating system is blessed to have sound partition management tools like GParted which are very easy to use. However, when it comes to the management of 'virtual partitions' known as volumes, things are quite different. There is Logical Volume Management, or LVM for short, however it can only really be used from the command line. Also, it doesn't integrate software RAID - except for striping. I was quite optimistic when I started using volume management some four years ago, but not anymore. Let me explain why I'm disappointed.

The Problem

It all started when I installed Gentoo for the first time. Or actually, my friend did it for me - since it was more difficult than installing OpenBSD, as the latter can be done in two minutes. After Slackware, it was the second Linux distribution to touch the platters of my harddrive. I was quite used to the rather rigid partition schemes of OpenBSD, in which /usr, /var, /tmp, /opt and /home all have their own partitions. Not being experienced at all, I remember I thought "1GB must be enough for /usr" only to find out it wasn't if I installed a lot of applications. Normally, such a thing would require re-installation of the OS if it was too difficult to re-partition the disk without losing data.

That's when my friend said to me: "You know, in Linux there's some way to prevent this kind of problem, by using a system in which you can dynamically resize partitions without even having to reboot your computer!".

The Solution

That's when I learned about LVM, and I immediately wanted to try it. I was encouraged when I found a guide how to install Gentoo on an LVM-system. I found out the most important thing to make LVM work was the device mapper. The device mapper makes virtual block devices; which basically means virtual partitions. On this virtual partitions, one can put a filesystem and mount that filesystem to use it. Or - and that's what makes Linux volume management so flexible - one can put another virtual partition on a virtual partition, in other words layer the virtual devices. Using LVM wasn't really hard: I just typed over the commands given in the HOWTO while changing some parameters to my liking, and in a short amount of time, Gentoo booted using my new LVM environment. However, because I was using LVM1 back then, my system wouldn't boot at first. Running lvchange -a y when the boot halted halfway cured this.

Not much later, my friend and I were once again in a contemplating mood, and my friend told "Did you know you can encrypt your harddisk in such a way that even most secret services (almost) can't decrypt them?". I was eager to find out, and in no time I stacked up an encrypted virtual block device on top of the LVM virtual block devices. OK, I had to type 32 characters everytime /home was mounted, but it gave a great feeling of privacy. But I did came across a problem: The Gentoo-init scripts assumed you would encrypt a physical partition and put virtual partitions on top of that encryped partitions. I did it the other way around: I put my encrypted partition on top of virtual partitions, so I had to write a bash-script to mount my encrypted partition. It was in the time of Fedora 2 probably (though I hadn't heard of Fedora back then), and I remember even the 'cryptsetup' CLI-frontend wasn't available back then. I had to use the dmsetup command, which took lot and lots of parameters. Yeah, I remember being glad when Christopher wrote cryptsetup. I tried to hack the Gentoo-init scripts but failed. Only one year later or so the Gentoo-init scripts were changed exactly in the way I tried: Making it possible to stack encrypted virtual block devices on LVM virtual block devices or the other way around both worked. However, having Gentoo without experimenting doesn't make much sense, so when buying a new computer I decided to take my volume management to the next level.

Finally, a GUI!

About the same time I learned about volume management, I learned about software-RAID too. However, mastering the Linux' mdadm commando looked quite difficult. I was rather scared of the mdadm tool. That doesn't make sense at all for somebody who used the dmsetup commando to setup encrypted partitions, but choices are not always rational. However, I found out about Enterprise Volume Management System, a graphical frontend which handles LVM2, software RAID and even much more. "That's what I've been looking for!" I thought. About reading some more, I even thought "Phew, this EVMS thing is going to save me a lot of time!". Indeed it did, when setting up my new volume management scheme. However, I couldn't have foreseen the way it would screw up that very same scheme two years later.

The Scheme

The scheme was quite complex for a home-PC; but I'll try to explain. I had two 80Gb / ~73GB hard drives. Both Lilo and GRUB are not capable of booting from EVMS. Therefore, one needs a non-EVMS partition to boot from, or an initrd, but I detest the latter. However I wanted to use software-RAID on my two drives, which works best if the two partitions used are equal of size. Therefore, I reserved the first 2GB of both drives for my small root-partition and I used the other 71GB for my software RAID-drive. That very same decision would cost me a lot of trouble later, but at that moment it seemed a good idea to do it that way. EVMS put the two 71GB partitions used for RAID in one software-RAID device named md0. One can directly make a filesystem on such an md-device, but instead I added all space on md0 to my LVM-container named c0. That LVM container can be seen as an aquarium with water: I can take out glasses of water of any size out of it, and put water from my glasses back in to the aquarium and after that into other glasses of water as much as I want.

Off course, there's a maximum of water I can use: The amount in the aquarium. The glasses were my LVM virtual block devices, which contained the /usr, /opt and so on partitions mentioned earlier. On top of those LVM virtual block devices I could have put another mapped virtual device, for example an encryption layer, but I was rather tired of typing 32 character passphrases every morning when I boot my computer. I had to boot my computer because software-suspend didn't support crypto layers back then, and I wouldn't be surprised if it still doesn't. My new scheme worked really well. In fact, it worked so good, when I built my second computer I decided to use two 80GB drives also and used exactly the same EVMS scheme. I even used the same names for the LVM containers and virtual block devices. Not smart, as would be proven later.

Like always: Windows is the forerunner of trouble

So, Everything worked like I envisioned, until I needed Windows. Like usual, Windows also meant problems this time. I needed AutoCAD which wouldn't run on Wine because the freakin' thing needs SP2. Therefore, I tried to install Windows on my 2GB. That was quite hard, but eventually I managed to do it by using space-saving tricks. Being really economic, like a genuine Dutchman is ought to, I only used 300MB of my other 2GB partition for Linux' root partition. That meant I had 1.7GB left, which was exactly enough to install AutoCAD. All worked quite well, but then I wanted to install AutoDesk Inventor. A real nightmare, (also) when it comes to hard disk requirements. I figured out I needed 3.5+GB to install it, so of course my almost filled two 2GB partitions were not enough. "I need to add some of my EVMS-space to my Windows FAT partitions" I decided, but how would I do that?

First I thought "EVMS is meant for easy volume management, so this can't be hard at all; probably it is even very simple'. Boy was I wrong! In theory, it goes like this:

    • - First, you shrink your filesystems to add unused space to the LVM-container c0.
        - Then, since not all space of c0 is used, you can shrink c0 and at the 'end' of c0 some GB of md0 are unused. However, Windows can't use space on a Linux software-RAID disk.
          - So therefore, that space should be freed from md0 also. After that, some space on my physical partitions hda2 and hdb2 isn't used for md0 anymore.
            - So I should be able to shrink hda2 and hdb2. With the space left, I should be able to.
              - Use gparted to partition that space into new FAT-partions, possibly some E:, G:, K: or 'what else have you'? drive in Windows.


              Simple, huh? Well, until shrinking the LVM container all worked well. But after I successfully shrunk my LVM container, I couldn't figure out how to shrink the md0 software-RAID partition. Turned out it wasn't possible in EVMS. And worse, IBM - the maintainer of EVMS - stopped to maintain it. Now, here's what would have made a lot of sense: Buy a big external harddisk, take my two boxes with the two EVMS schemes, and backup everything to the external drive. After that, build a better volume management scheme, preferably without the unmaintained EVMS. Yes, that would have made a lot of sense; it would also enable me to make sure both EVMS arrays would have exactly the same content. But did I mention I'm Dutch? Why would you buy something if by fiddling some extra hours you could save the bucks? Most of my files were duplicated on both disks anyway.

              Screwing it up

              Instead of buying an external disk, I decided to take the two EVMS arrays, take the files of the smallest EVMS array which were not on the second and backup those to the second. Then, all my files would be on the second array, and I could start over with my first EVMS array. After that, I could move my data to the by then 'reschemed' first array, and 'rescheme' my second array.
              Nonetheless, this required me two connect all four hard drives to my computer at the same time. That was no problem; my mobo supported that, and I booted from a Gentoo Minimal CD. I thought the fun could start, when I noticed a problem: Both LVM containers were named c1, so I only could use one of those at the same time. Stupid. Really stupid. So I tried to rename one of them to c1. And failed. EVMS wouldn't run my arrays anymore. Instead of EVMS I tried using LVM2 directly now, but that didn't work and made things worse; and as you can see when following the link help didn't turn up.

              I finally decided to use mdadm to restart my software-RAID arrays but because it was the first time had I used mdadm; I screwed up even more dramatically by using the force switch and -assume- clean and after that mounting my software-RAID device. It only used one of the two RAID-partitions, so all my files were halved. Mounting the failed RAID-device caused some important EVMS meta-data to screw up in such a way that finally I considered my 80GB of files on that array as lost. Sadly, it was the array that was the most filled with files. On the bright side of things, this enabled me to wipe it all out and start again. This time using LVM2 and mdadm from the start on. Again, as in the 'old days', from the command line...

              Alternatives and Conclusion

              Not much later, I saw somebody on LXer asking for a GUI for LVM. That would be a very logical thing to ask. EVMS is just that and much more, however I can't recommend it to anyone while it's not supported anymore; and as you can't shrink software-RAID partitions. Then there is system-config-lvm, but it's something from RedHat which is not in the repository of Gentoo, meaning Gentoo doesn't support it. Update: I was wrong, it is in the Gentoo repository, but hardmasked because it needs testing.There's also the Aperi project, but it's not finished yet. SunsZFS would be a solution, but GNU/Linux's license is not compatible with the ZFS license, so that's not an option for Linux either. Which means, at this moment, you have to use RedHat or the CLI if you want to use LVM. Kinda sad, I'd say. For now we can only hope someone makes a simple GUI for LVM available on all distro's, or - like open source works - and do it ourselves of course!

    		•

      Nav
    » Read more about: Story Type: Editorial, LXer Features; Groups: Gentoo, GNU, Kernel, Linux, Red Hat

    « Return to the newswire homepage

    Subject Topic Starter Replies Views Last Post
    A problem of expectations, not of Linux volume Management texastwister 3 3,508 Aug 3, 2008 11:15 AM
    LVM2 and mirroring jylenhof 1 5,656 Aug 3, 2008 5:55 AM
    Grub and lvm jylenhof 1 2,728 Jul 26, 2008 2:26 AM
    system-config-lvm jylenhof 1 2,556 Jul 25, 2008 5:36 PM
    Gentoo & system-config-lvm robbat2 1 5,942 Jul 25, 2008 1:12 AM
    Bad luck and some mistakes jhansonxi 2 3,097 Jul 25, 2008 12:47 AM

    You cannot post until you login.