Advanced

Linus calls Linux 'bloated and huge'

Posted by Scott_Ruecker on · 37 comments
The Register; By Austin Modine
Mail this story
Print this story

Linux creator Linus Torvalds says the open source kernel has become "bloated and huge," with no midriff-slimming diet plan in sight. During a roundtable discussion at LinuxCon in Portland, Oregon this afternoon, moderator and Novell distinguished engineer James Bottomley asked Tovalds whether Linux kernel features were being released too fast, before the kernel is stabilized. Citing an internal Intel study that tracked kernel releases, Bottomley said Linux performance had dropped about two per centage points at every release, for a cumulative drop of about 12 per cent over the last ten releases. "Is this a problem?" he asked. "We're getting bloated and huge. Yes, it's a problem," said Torvalds.

Full Story

  Nav
» Read more about: Story Type: News Story; Groups: Linux

» Get new stories by e-mail: nightly or as they are published

« Return to the newswire homepage

  Related stories
» More like this

Comments (37)

kernel

Right at the top of the article it says:

"LinuxCon 2009 Linux creator Linus Torvalds says the open source kernel has become "bloated and huge," with no midriff-slimming diet plan in sight."

"open source kernel" not "OS" why is the description of this story different from the actual article? Linux is the kernel, not the OS, and the article is about the kernel, not the OS (which Linus has no say in). Things like these should be corrected before they are posted.
You're right. Thanks for noticing. I hope the issue is fixed.

These things may happen however when posting 13k stories, like Scott did. And we don't have the money / people to make a nice QA team, so to quote Linus: while it's unacceptable it's also unavoidable.
If I remember correctly Linux refers to the kernel. So if the kernel is bloated Linux is bloated. I picked it up pretty fast....
Wonderful, they yap about it when it is used 'incorrectly' then yap about it when it is used 'correctly'.

*sigh* locksteppers unite, the FSF [(ph)Fony Software Freedoms] are endangered.
I guess I'm being a little facetious eh azerthoth. Good call.
'Bloat' is a loaded word with little useful meaning. More functionality = bigger kernel. More functionality is good. Does having a big kitchen sink kernel hurt anyone? Probably not on a desktop system, it's just more disk space, and not all that much. On my PCLinuxOS system the kernel is 1.7 megabytes. /lib/modules is over 27 MB, and if I wanted to I could remove most of that to save space on my 160GB hard drive.

On embedded systems the kernel is stripped down to the essentials for that particular system, and what's cool is the same kernel supports a raft of different architectures and features. Just use the bits you want and discard the rest.

I don't like it when these 'bloat' comments get flung around because they're usually getting all excited over nothing-- did Linus say the kernel is full of non-functional bloaty code? There is probably a bit of real Crisco, some bits where one line of code can replace two. But let's get real, this is not a problem. Vista at over 14 GB and none of it functional, that is bloat.

*edit* ntoskrnl in Vista is 3.4 MB.

I wonder what this means exactly

Bottomley refers to an internal Intel benchmark which I assume amounts to a test of each kernel with all the default kernel switches set. From my experience of compiling the kernel there is an awful lot of choice there (intimidatingly in fact).

So I am wondering if you were so inclined and technically competent enough, could you run through this compilation process and turn off most of the bloat? Or is Linus actually saying that the basic bare bones part of the kernel is becoming bloated because of the free for all nature of how contributions are assembled?

Would be interesting to know where the biggest problem was. I suspect with no evidence that it is all the options being turned on to handle the ever increasing diversity of modern hardware but that is a guess on my part.
What I think would be cool is an easier way to manage modules, and not by default end up with every video and networking interface driver in the world. I seem to recall the PCLinuxOS has an installation option to strip all the video drivers you don't need.
Yes that would be great. The qt interface at present is just a prettyfication of the usual text or ncurses interfaces along with a whole bunch of way too technical comments about what each option means.

I guess the kernel developers leave this sort of thing to the distributors and distributors are just interested in assembling a blackbox kernel that meets most users demands for hardware and stability.
The problem is that Linux has too much hardware support. What we need is a "Windows mode", where Linux only supports hardware that is supported by Windows. That would probably solve most of the problems with bloat.
I tink Mandriva also has a post-install option that removes all the unneeded drivers. I left them in though because I was a little uncertain what it would do when I plonk a USB device in there that was not detected at the time of install.
I wonder what we're talking about exactly. Just how big is this "kernel," anyway. Maybe it's trivial when taken in context.

I do agree that an easy way to manage modules in a given installation's kernel is something that should be available (and probably is; I just don't know about it)
> I seem to recall the PCLinuxOS has an installation option to strip all the video drivers you don't need.

Debian has meta-packages for "xorg-input-all" and "xorg-video-all", and then Xorg autodetects (or finds in the xorg.conf) which specific one to use.

All it takes is disk space, and a considerate user can go back in and remove the unused drivers once he knows which one is in use. If PCLinuxOS's install checks first and then only installs the working driver, that would cut down on work. That's one good side-effect of installing from a LiveCD rather than a generic install disk.

For optimization of a working system, this is what I would like:

0) A tool that parses the loaded kernel modules and unloads what is not in use. While it would do little or nothing other than free up memory, I think it would be a public relations coup.

1) Automagic configuration of the configuration file for building a kernel, selecting only what is presently in use to be compiled in, everything else as loadable modules (just in case, a minimal waste of disk space and compile time, and the whole reason for loadable modules in the first place). Then building and installing of the custom kernel.

2) Optimization of the initrd.img for said custom kernel, making the assumption that catastrophic hardware changes have not been made since last time. Or would the optimized build above make the use of an initrd.img unnecessary?

3) Versioning of the above custom build so that if hardware changes have been made (or the magical build doesn't work), the "generic" kernel build remains available to start the process again.

Bigg, that would do pretty much what you're saying, cut the "supported hardware" to a minimum in the running kernel, while keeping module auto-loading "just in case". Like when plugging in a USB thumbdrive and needing FAT16 for the first time in years.

The last time I compiled a kernel was 2003, v2.4.xx, and even then the number of hardware modules to go through was awful. I can only imagine what it would be like now, so I'm right there with Monti in being intimidated. What that must be like for someone who has never compiled a kernel before....ugh! Serious intimidation.
Bob,

It would be a nice project to do what you suggest and then use the Intel benchmark to verify speedups. If you then did it for two quite different versions of the kernel (2.6.28 versus 2.6.14 say) you could check whether the bloat was coming from the additional unneeded drivers or from Linus and team being less than optimal with the "base" kernel.....
Well, the Phoronix comparisons (to my eye) show some variations, but I don't know how to isolate file system module speed-ups from core kernel functionality.
> Bigg, that would do pretty much what you're saying, cut the "supported hardware" to a minimum in the running kernel, while keeping module auto-loading "just in case". Like when plugging in a USB thumbdrive and needing FAT16 for the first time in years.

What I have in mind is either (a) allowing the user to keep the installation CD around in case you install new hardware, or (b) automatically downloading available drivers, assuming you have a working internet connection.

I do not recall that I've ever compiled a kernel. It sounds scary. I'd much rather install a smaller kernel and add to it on an old machine.
> A tool that parses the loaded kernel modules and unloads what is not in use.

My understanding was that a kernel module was only loaded if you had hardware which needed it. Of course, that ignores things like the alsa-oss compatibility module and such, which alsa also loads.
> My understanding was that a kernel module was only loaded if you had hardware which needed it. Of course, that ignores things like the alsa-oss compatibility module and such, which alsa also loads.

From my experience, it is also possible to load a dynamic module that is not actually being used, if you deliberately do it "modprobe modulename" and its been included in the compile as a loadable module.

The alsa-oss and a LOT of other sound modules get loaded, true, just by turning sound on. I've enabled enough sound cards in the past by hand. More than enough.

So maybe my list of optimizations is redundant. Good! More demonstration that the Linux kernel environment is doing better than the sheer size of the source code might suggest.
Quoting:Automagic configuration of the configuration file for building a kernel


This was added to Linux in August:

make localmodconfig

(See http://lwn.net/Articles/347713/)
It would be safer to say that the 'kitchen sink' kernels are indeed in need of a diet plan, then you have to add in the initrd,initramfs, or what have you on top of it. You plug it in and it works has a cost, and size/performance is that cost. What we really need is something that says, you only have this hardware and only need these controllers for your system, would you like to build an optimized kernel.

I do it by hand, and with one exception without an initrd or variant. Nice, small, and fast.
> This was added to Linux in August:

Coolness!
jezuch,

That would help (a lot) with speeding up module compilation but I'm not sure it would make a lot of difference with the slowdown noted by the intel benchmarks. After all when all is complete you will still be loading the same set of modules it's just that the rest won't be around.... I guess you could unload all the "useless" ones using rmmod before execution but my suspicion is that the biggest problem is with the builtin (non-modules) part of the kernel.

Edit: I notice Alan Cox links to Lxer. Maybe he (or someone like him) could comment instead of klutzes like myself who are speculating. ;-)
> ...it is also possible to load a dynamic module that is not actually being used, if you deliberately do it "modprobe modulename" and its been included in the compile as a loadable module.

Probably, but unless you expect the hardware to be present at least part of the time, why would you want to do that? On the other hand, a number of modules I've used in the past have checked for applicable hardware and wouldn't load unless they found it (mostly ethernet cards, from memory).
Quoting:That would help (a lot) with speeding up module compilation but I'm not sure it would make a lot of difference with the slowdown noted by the intel benchmarks.


I'd like to see those benchmarks.
> why would you want to do that?

Ah, you see, I _don't_ want to do that. But I can, and have, because of hunting for the right driver for something before the days of semi-reliable auto-detection.

As Monte points out it's not just a problem of modules, but of which modules.

So something that merely reads what modules are presently loaded can make the error of over-provisioning. I really did mean "presently in use" in wish-list item 1, and that would also include un-selecting those modules which are compiled in the pre-mixed kernel but are unused.

I guess that's why it's called a "wish" list, it doesn't mean it's practical or workable.

Great movie BTW, "Frequently Asked Questions About Time Travel". Highly recommended.
>I'd like to see those benchmarks.

I also. It's a bit hard saying anything definitive on this issue until you know those details. Relevancy is all in benchmarks.

Kernel "bloat", efficiency of new versions.

First let's define bloat.

Is the Linux kernel source huge? Yes indeed, for all the reasons stated except sloth. The simple number of supported hardware architectures and kernel modules, and don't forget the good programming practice of copious comments, makes the source itself grow as it is changed and added to.

Is the Linux kernel slower than it used to be? Could be, because I see the bar-graphs on Phoronix kernel comparisons go up a little bit sometimes. I only retired my 2003 laptop a couple months ago, and in those 6 years of staying with the latest kernels I didn't notice any slowdown.

So let's narrow this down a little. Otherwise, I'm just writing Microsoft marketing copy.

The Debian "everything and the kitchen sink" kernel that I'm using right now:

-rw-r--r-- 1 root root 1506032 2009-08-19 05:48 vmlinuz-2.6.26-2-686

That's 1.5MB. Yes, compressed, but I have to wonder, "this is bloat?"

Silly question. This kernel is bigger than Linux used to be, even if it's tiny in comparison to other "popular" kernels.

Is the bloat in the modules? Certainly the NUMBER of modules has exploded, mostly due to hardware makers and their infinite variations on a theme. To my jaded opinion, I also think too many are being loaded, thus making for a larger loaded "kernel" when they're not used. Again my unoptimized system:

$ lsmod | wc -l 86

Certainly no high performance server is going to need the sound and video drivers. Would it be nice to unload unused drivers even if they are compiled in? Would it be nice if there were an app that could be run, unloading all modules not in use? Sure, but first one has to wonder just how much such unused modules actually slow the running kernel.

Here's what I think is going on: There is bloat, and the system is getting slowed down, but it's because of multiple areas of possible bloat added together.

The kernel itself, the scheduler, memory management, the basic things that constitute the fundamental kernel are constantly being tweaked. The entire "Fair Scheduler" controversy, to me, demonstrates that even the most fundamental elements of the kernel get a share of the attention.

Kernel modules. Well, the prime motivation is to get hardware working. After that, is all that much attention paid to optimizing? According to "Greg Kroah Hartman on the Linux Kernel", changes happen everywhere, across the board.

Not sure how to sum this up. With as many different hardware architectures, maybe the core kernel elements cannot be as "optimized" as it used to be when i386-specific tweaks could be inherently used in the code, not relying on compiler optimizations. Maybe bloat at the statistical level is inevitable, and over the course of years those statistics add up.

I've tried Menuet OS, and of course something optimized to a specific architecture and limited hardware seems very fast. So I wonder why such "hardware specific" OSs have such small penetration on the Top500 supercomputers?

Could it be that, stripped of all unused modules and compiled for specific hardware, Linux isn't actually all that bloated?
To give you an idea of how, to answer your last question, Linux isn't all that bloated... My netbook died and is going back for warranty repair. I've put my next best laptop back into service, a Toshiba purchased in 2002. It's was a low end machine then: a 1GHz Celeron processor with 512MB of RAM (maxed out). I'm running Vector Linux Light at the moment but it's in a dual boot configuration with Slackware 13.0. Neither OS Is slow on my machine.

We're talking low end hardware from seven years ago and performance is good...

Just in case I am rebuilding my ancient Toshiba Libretto SS1010, from 1998. It can still run Linux 11 years later. Yes, I'm restricted to lightweight apps and a lightweight window manager but it can still do work. It is not intolerably slow, either. Specs: Pentium MMX 233MHz processor with 64MB of RAM. A modern Linux kernel can run on 11 year old hardware.

That's bloated? Gee... I wish other things were "bloated" like that.
The whole "bloat" quote has been blown waaay out of proportion anyway. It's the headline du jour, and nearly every report gets it wrong. God forbid an intrepid "tech""journalist" should spend a few minutes watching the actual video of the actual kernel panel from yesterday, and hear first-hand what Linus said. It was a throwaway quip from an hour discussion on a number of aspects of kernel development.

Yep, me too. 128MB, 300MHz, 1998 laptop ran 2.6.18 kernel and current applications from Debian Sid last summer when I finally retired it. And, as you mention, all I had to do was use one of the smaller window managers and only one of the "big" applications at a time, like OpenOffice.

But it worked, and as you say it worked just fine.

If this be "bloat", let's make the most of it.
@Caitlyn and Bob:

I have an old laptop (1 GHz purchased in 2002). It has only 256 MB RAM. Debian Etch ran great, Lenny not so much. Slackware 12.1 didn't run worth a darn. Vector Light OTOH is very responsive. I assumed that Vector Light has a stripped-down kernel, but I may be wrong.
Bigg, I've noticed on a 1GHz Athalon server, that several months ago it could play full motion video just fine, using Lenny. XINE had to be told to "just drop frames", but it was smooth and ran just fine even with H264 files.

I had a Grub2 problem and reinstalled, again with Lenny, but now XINE won't play the same video files smoothly at all, and builds up enough problems to stop itself even playing miniscule .avi files.

What changed? I have NO freaking idea. But what I can say is that I don't think it's the fault of the Linux kernel itself. Certainly applications do keep getting bigger, and bigger, and bigger...

I've been thinking of getting an "obsolete" nVidia PCI card to replace the onboard video, see if that frees up enough resources to make the difference.
@Bob_Robertson You are probably on the right track with trying a separate nVidia card. Not only are even the "obsolete" low-end PCI or AGP cards that you can often find as cast-offs or the $19 loss leader at Fry's usually significantly faster than most built-in video you will probably also be able to re-claim some system memory because most on-board video uses shared memory. Both of those things should help not only video performance but may even give a slight boost to overall system performance.
It could be Xine. At least VLC is having a similar problem, where the newer versions are so heavy they choke on some files. 0.86b, on the other hand, works fine.
'Kernel Bloat' is not a problem when it comes to desktops (except for having to configure your 2.6 kernel starting with an empty .config - may take a few hours). It may a problem be when it comes to embedded systems.

That's why embedded systems run 'trimmed down' Linux kernels, uclibc, busybox and dasBoot, and why ARM joined the Linux foundation - I guess.
I've put Slackware 3.2 (yes, three point two) and Slackware 12.2 on the same machine (different disks, however), a 400 MHz K-6 Gateway. I did this during the current calendar year.

I had to upgrade the 3.2 distro's kernel to 2.0.40 to get it to support a 3c905 ethernet card, and I had to recompile Xfree86 4.99 to get the ATI "Rage" video card to work. 3.2 also didn't include SSH.

Kernel 2.0.40 did seem slightly snappier than the 2.6.xx kernel in Slackware 12.2, but overall, there's not that much performance difference.
I personally haven't seen any performance "regressions" between Debians Etch and Lenny. What I have seen is a lot more functionality in Lenny just by virtue of the kernel and all the ancillary tools being able to do more.

I just rolled out a Debian Lenny laptop with Xfce that has VERY low specs: 144 MB RAM, 233 MHz Pentium II CPU. I had Etch on it quite a while ago, and it seems to run as well or better with Lenny. Of course it ran pretty well with OpenBSD 4.2 as well; I did try to reinstall (having failed at upgrading another box with OpenBSD) ... this time I screwed it up in a 10th the time, hence the return of Debian.

Probably the best distro I tried on this old laptop was Puppy Linux, with 2.13 being quite a bit faster than 4.1.x (can't remember the exact version number). But both were great running from live CD.

But Lenny is more than adequate on it, and the ability to update/upgrade easily and keep it patched means I'll probably be sticking with Lenny on it.

Post a comment

A free account is needed to post. Write your comment here first: your text is kept in this browser while you sign up or log in, and is waiting for you when you come back.

Your message:
|||||
10
Markdown: **bold**, *italic*, [text](https://link), - lists, > quotes, $x^2$ for math. Formatting help

Sign up and post (it is free)  or  Log in and post