The 2009 Linux and free software timeline - Q1

[Posted November 24, 2009 by jake]

Here is LWN's twelfth annual timeline of significant events in the Linux and free software world for the year.

2009 offered few surprises to those that have been following Linux and free software for as long as we have. As expected, there were new releases of many of the tools and underlying infrastructure that we use on a daily basis. There were also lawsuits over software patents, arguments over licensing, and various security flaws found and fixed. Distributions were packaged up and released, more phones and other devices with Linux and free software were sold, and so forth. All part of the march to "world domination". We look forward to 2010—and beyond.

This year we will be breaking things up into quarters, and this is our report on January-March 2009. Over the next month or so, we will be putting out timelines of the other three quarters of the year.

This is version 0.8 of the 2009 timeline. There are almost certainly some errors or omissions; if you find any, please send them to timeline@lwn.net.

LWN subscribers have paid for the development of this timeline, along with previous timelines and the weekly editions. If you like what you see here, or elsewhere on the site, please consider subscribing to LWN.

For those with a nostalgic bent, our timeline index page has links to the previous eleven timelines and some other retrospective articles going all the way back to 1998.

January

I will just note wryly that it used to be that I could compile 0.9x kernels on a 40 MHz 386 machine in 10 minutes. Some 15 years later, it still takes roughly the same amount of time to compile a kernel, even though computers have gotten vastly faster since then. Something seems wrong with that....

-- Ted Ts'o

One Laptop Per Child (OLPC) restructures, laying off half the staff and "refocusing" in various ways. (OLPC blog) [Valgrind logo]

Valgrind releases version 3.4.0 of the popular program analysis tool for finding memory and other errors. (review).

Nokia announces the release of Qt under LGPLv2.1 for the upcoming 4.5 release. (announcement).

linux.conf.au is held in Hobart, Tasmania. (LWN coverage, 2, 3, 4, and 5)

The word "Python" was also catchy, a bit edgy, and at the same time, it fit in the tradition of naming languages after famous people, like Pascal, Ada, and Eiffel. The Monty Python team may not be famous for their advancement of science or technology, but they are certainly a geek favorite.

-- Guido van Rossum on how Python got its name

Red Hat Enterprise Linux 5.3 is released. (announcement) [Moonlight]

Moonlight developers work overtime to make President Obama's inauguration viewable on Linux, because the streams were only made available in Silverlight form. (article)

GCC and FSF announce a GPLv3 exception to allow for GCC plugins; the exception is for the GCC runtime library and will allow free software plugins, while preventing proprietary plugins. This particular incarnation of the exception is not adopted. (announcement)

The government ought to mandate open source products based on open source reference implementations to improve security, get higher quality software, lower costs, higher reliability - all the benefits that come with open software.

-- Scott McNealy

KNOPPIX 6.0 is released. (announcement, review)

KDE 4.2 is released. (announcement)

AMD releases 3D register reference guide for R6xx/R7xx chips, which will help with the development of free software drivers for devices using those chips. (announcement)

The Linux Foundation kicks off the "We're Linux" video contest. (press release)

February

Zope 3.4 is released after two years of development on the Python-based web application server.(announcement)

Open source is not a lawless frontier at all. There are clear license terms that have to be followed, even though open source generally offers more freedoms than proprietary software. It's true, that many organisations are still struggling to understand open source and its license terms.

-- Martin Michlmayr

Red Hat hires former Mandriva community manager Adam Williamson to drive community involvement in Fedora QA. (introduction)

Miro internet TV version 2.0 is released. (announcement)

RPM version 4.6.0 released; the package manager used by Red Hat, Mandriva, SUSE, and others. (announcement)

Debian 5.0 ("Lenny") is released after "22 months of constant development". (announcement) The release is dedicated to Thiemo Seufer, a community member who died in a car accident. [Debian]

DragonFly BSD 2.2 is released—now with a production-ready HAMMER filesystem. (announcement)

At this point, DRM seems intended to accomplish a very different purpose: giving some industry leaders unprecedented power to influence the pace and nature of innovation and upsetting the traditional balance between the interests of copyright owners and the interests of the public.

-- EFF Staff Attorney Corynne McSherry

Kurt Roeckx is appointed as Debian project secretary, after the previous secretary resigned in late 2008. (announcement)

Red Hat moves from Xen to KVM for virtualization in future releases, as expected by many after its acquisition of Qumranet. (press release)

Microsoft launches patent suit against TomTom, for patents on the VFAT filesystem among other things. (LWN coverage)

BASH 4.0 is released.; BASH is the Bourne-Again SHell (announcement)

X server 1.6.0 released. (announcement)

March

There's no easy fix for this - you need to be aware of what is right and what is wrong, but you cannot look at existing code to determine this.

-- Andrew Morton on kernel code

The Linux Foundation acquires the Linux.com domain, which they will turn into a community news and collaboration site. (announcement)

MontaVista starts Meld community site for embedded Linux developers. (announcement)

The "ext4 data loss" controversy heats up. (first LWN article) [Firefox]

Firefox 3.1 renamed to 3.5 to better reflect the scope of the changes. (announcement)

The Linux kernel gets a new logo for one release; "Tuz" is a reminder of the plight of the Tasmanian devil. (LWN coverage)

Linux leaders have a problem. Ever since Microsoft adopted the 'let's get along' strategy of licensing and interoperating, it has been hard to get people to volunteer their time for the platform, and interest seems to be waning.

-- Rob Enderle grasping at straws

GNOME 2.26 released. (announcement)

Parrot 1.0.0 released; Parrot is a "virtual machine aimed at running all dynamic languages". (announcement, LWN article)

Linux 2.6.29 is released with an experimental Btrfs, squashfs, kernel mode setting for Intel graphics hardware, and more. (announcement, KernelNewbies coverage)

SUSE Linux Enterprise 11 is released in both desktop (SLED) and server (SLES) varieties. (press release) [Rails]

Rails 2.3 released—aka Ruby on Rails, the Ruby-based web framework. (announcement)

In Europe we had the habit of reading Slashdot, and reading about all the crazy patents in the USA, and we all had a good laugh. Then, very suddenly, we were faced with our own software patent problem.

-- Ciarán O'Riordan of End Software Patents

GNOME switches to Git, from Subversion, for version control. (announcement)

Microsoft vs. TomTom comes to an end, via a settlement, but, before that, TomTom joins the Open Invention Network and countersues Microsoft. (Groklaw settlement article)

Fedora issues report on August 2008 intrusion, seven months after it occurs. (report)

Python starts switch to Mercurial for distributed version control. (Guido van Rossum's announcement)

(Log in to post comments)

The 2009 Linux and free software timeline - Q1

Posted Nov 25, 2009 0:38 UTC (Wed) by mato (guest, #964) [Link]

> I will just note wryly that it used to be that I could compile 0.9x kernels on a 40 MHz 386 machine in 10 minutes. Some 15 years later, it still takes roughly the same amount of time to compile a kernel, even though computers have gotten vastly faster since then. Something seems wrong with that....

Hear, hear!

Kernel build performance

Posted Nov 25, 2009 7:20 UTC (Wed) by mingo (subscriber, #31122) [Link]

The kernel source has gotten larger, a bit faster than Moore's law - so every year there's more stuff to build.

Kernel v0.96 was 35 thousand lines of code, v2.6.32 will be more than 11 million lines of code. The size of the source code has increased more than 300-fold in 16 years.

There's other factors as well, like GCC optimizing much more aggressively (and producing better code as a result) - at the expense of runtime overhead, which too caused a systemic slowdown in kernel build performance over the years.

(If it's any consolation, the Linux kernel still builds faster on decent hardware than some bigger OSS projects finish their initial auto-tools pass ;-)

Kernel build performance

Posted Nov 25, 2009 10:27 UTC (Wed) by epa (subscriber, #39769) [Link]

It's bigger, but not all of the new code is built. Otherwise the kernel image, which took a few hundred kilobytes back in the old days, would now take 300 times as much space.

Still, I remember leaving the computer running all night to build a new 1.2.x kernel, even though it was a 486. I think disk thrashing is the biggest factor.

Kernel build performance

Posted Nov 25, 2009 10:36 UTC (Wed) by mingo (subscriber, #31122) [Link]

Not all new code is built - but the bit you do build did get comparatively larger - and GCC got comparatively slower.

It roughly evens out for the configs i'm using - my kernel build times stayed more or less constant for the last 10 years. When i go back to older kernels and build them again they get built quite a bit faster than on older hardware. (no surprise there)

My point is that it's all pretty natural and the build times are acceptable. The kernel grows, GCC gets smarter and slower, and hardware gets faster - on a long-term trend basis these factors manage to cancel out each other. (which is pure chance but that's what's happening)

Kernel build performance

Posted Nov 25, 2009 11:21 UTC (Wed) by tialaramex (subscriber, #21167) [Link]

It's not /pure/ chance, it's partly because people adjust their behaviour in order for things to come out as expected. When you're a bit late you run for the train, and when you're early you stop to look at the clouds in the sky.

To drag in the inevitable car analogy, we know that making cars safer has a muted effect, the drivers compensate somewhat for (some kinds of) safety improvements by driving less carefully. Wider lanes result in faster driving. Better brakes result in more tail-gating. There is some hidden 'right' amount of danger that we're instinctively comfortable with, and it's calibrated way above the level we would accept if it was presented as a conscious decision.

If your new PC and compiler seem lightning fast, you may be tempted to put off that idea you had to speed up build times, because it doesn't seem to matter. On the other hand, when compiling your program seems to always take too long, you may spend the time thinking of ideas to make compilation faster, reduce code size, etc.

I've read somewhere the idea that this same thing applies to Moore's law. When engineers see that the new product is right in line with Moore's prediction, they relax, take some holiday, goof off at work. But when it's 10% short they fear the competition will overtake them, their bosses insist they work overtime, they worry about the problem at home, and so on. So in practice Moore's law may work just because we think it works, if we'd expected a doubling every three years, that could work fine too.

Kernel build performance

Posted Nov 25, 2009 12:08 UTC (Wed) by mingo (subscriber, #31122) [Link]

Yeah it's not pure chance, but note that none of the factors i cited are really any 'macro behavioral' items. People don't speed up or slow down the kernel build via a single act - their micro-changes have _way_ too little effect on it as a whole. It literally needs a thousand changes for anything like this to show up in any wall-clock measurement.

(Sometimes there's feedback in terms of 'hey you made the kernel build slower' - but these aren't efforts that stabilize it - these just affect the basic parameters and the combination (end result) is random.)

But i'll certainly agree that people wouldnt accept 30+ minutes kernel build times - nor would they stop from bringing its speed from 10 seconds to 20 secods (halving build performance, without anyone really complaining). So there's a certain psychology driven behavior that keeps it somewhat within a given "band" of performance - but the fact that kernel build times are pretty stable over the past decade is pure chance i think.

But i think i digress :-)

Kernel build performance

Posted Nov 25, 2009 14:13 UTC (Wed) by nix (subscriber, #2304) [Link]

people wouldnt accept 30+ minutes kernel build times

Your enormous hardware budget is showing :) until I upgraded this year I had never owned a machine nor even seen a machine which could build a kernel in less time than that. Multi-hour-long build times are not unknown, though admittedly you need to use 1999-vintage hardware for that, which is pushing obsoleteness even among those of us with near-nil hardware budgets.

Kernel build performance

Posted Nov 25, 2009 15:41 UTC (Wed) by mingo (subscriber, #31122) [Link]

Your enormous hardware budget is showing :)

You are making assumptions :-)

I regularly build the kernel on stock laptops and desktops (which are typically 1-2-3 generations older than state-of-the-art) and the build times are well below 30 minutes. (usually within 10 minutes.)

The oldest system on which i still build the kernel is a 833 MHz P3 laptop with 512 MB of RAM, there a typical kernel takes 45 minutes to build. (But that's ~5 generations old and kernel developers/testers rarely use such old systems.)

I never build distro kernels though - i always use (and used) .config's tailored to the specific hardware. You can certainly waste a lot of time by building generic kernels.

Kernel build performance

Posted Nov 25, 2009 12:23 UTC (Wed) by nix (subscriber, #2304) [Link]

I don't know. For me, kernel build times stayed roughly constant (on those machines I upgraded), until I went multicore. Then they *plunged*. The first machine I ran Linux on in 1997 took half an hour to build a kernel, and that stayed true (it takes a bit over an hour for a 900MHz PIII nowadays). My current single-socket Nehalem desktop takes two minutes, and that's over NFS: do it on the NFS server and it takes 57 seconds. And that's *with* debugging information!

I think that could be considered a speedup. :)

Kernel build performance

Posted Dec 5, 2009 4:06 UTC (Sat) by adobriyan (subscriber, #30858) [Link]

> I think disk thrashing is the biggest factor.

Kernel build is CPU bound unless you do something like not using make -j even on UP or turning CONFIG_DEBUG_INFO on (where so-called thrashing becomes much more noticeable).

Kernel build performance

Posted Nov 26, 2009 13:51 UTC (Thu) by intgr (subscriber, #39733) [Link]

(If it's any consolation, the Linux kernel still builds faster on decent hardware than some bigger OSS projects finish their initial auto- tools pass ;-)

Not anymore... 2.6.32 fixes some cpufreq problems that previously made autoconf scripts run in the lowest-frequency state on some quad-core CPUs. On my computer, autoconf scripts are now almost 4 times faster. ;)

Kernel build performance

Posted Nov 26, 2009 15:31 UTC (Thu) by nye (guest, #51576) [Link]

Clearly the solution is for the kernel to fix this 'regression', so we can once again be confident that the kernel builds faster than a large autotools pass :P.

The 2009 Linux and free software timeline - Q1

Posted Nov 25, 2009 13:43 UTC (Wed) by amk (subscriber, #19) [Link]

The last item might be better phrased as "Python decides to switch to Mercurial", since the transition hasn't happened yet (it's currently stalled on cross-platform handling of LF vs. CR-LF line endings).

Gut feelings

Posted Nov 26, 2009 7:23 UTC (Thu) by man_ls (guest, #15091) [Link]

So the "gut feeling" of van Rossum in the linked article was not right after all. I wonder why he says

At PyCon, Brett already announced that Git was no longer being considered -- while it has obviously many fans, it also provokes strong antipathies.

I don't see the problem, neither for hating git nor for banning it because it is hated by some. It is a wonderful tool IMHO. Now if the reason is just to use a DVCS written in Python it would be understandable, but for git it should just be "see if it fits our use case -- and works". Deciding on hate and gut feelings may lead to unnecessary stalls such as the one you mention.

Gut feelings

Posted Nov 26, 2009 16:47 UTC (Thu) by anselm (subscriber, #2796) [Link]

The problem with Git, AFAIK, is that it isn't quite there yet on Windows (at least for a big project such as Python). It is also likely that Git, being much more Unix-centric by design than Mercurial, would run into the same sort of problem concerning line endings that is holding back the Mercurial adoption, so it isn't even clear that Git would have been the better choice to begin with.

That, and the fact that it looks better for the Python project to be using a Python-based DVCS for PR reasons -- Git is probably a very good DVCS, but so is Mercurial, and, all other things being equal, you want to be seen eating your own dog food.

Gut feelings

Posted Nov 27, 2009 1:59 UTC (Fri) by dlang (guest, #313) [Link]

git has long since grown options for dealing with line endings across all platforms.

deciding that he wants to use a tool written in python instead of C is his privilage, but if that's the reason he should say so, not take the 'gut feeling' position

Gut feelings

Posted Nov 27, 2009 2:27 UTC (Fri) by foom (subscriber, #14868) [Link]

Mercurial has support as well, and a casual bystander would be hard pressed to understand what
was wrong with it for Python's use...

http://mercurial.selenic.com/wiki/Win32TextExtension

The 2009 Linux and free software timeline - Q1

Posted Nov 26, 2009 14:57 UTC (Thu) by briangmaddox (guest, #39279) [Link]

Adam was THE community at Mandriva before they canned all their non-French employees. I remember being so happy to see him get picked up by Fedora and it looks like he's still doing a great job there.