We few, we happy few: Big boys dominate early stage OpenStack

The large numbers of you who watched our recent Regcast All about OpenStack (catch it in the on-demand version if you missed it) show how much interest there is in the project. But no one pretends that OpenStack is anywhere near mature. It is a work in progress and in the short term it will mostly be visible in the service provider community.

How do we measure the degree of practical commitment to OpenStack from the wider community? We learned in the Regcast that HP has placed OpenStack technology at the core of its cloud plans, and that IBM among others is moving in the same direction.

This doesn’t convince everyone. For example, recently TheStreet.com got involved. Dell abandoned its OpenStack-based public cloud service, it argues, and the way that proprietary public cloud vendors have dropped prices will mean that OpenStack-based public clouds have neither a price nor a functionality advantage in the short term.

The conclusion of the doubter is that if users don’t care about lock-in, or the long run, then momentum will falter.

OpenStack has not yet reached the open-source nirvana where we can be largely indifferent to the question of who is contributing to development, because if one organisation or developer drops out, then someone else will pick up the slack.

In the short term a lot of the commitment comes from large vendors whose immediate commercial plans are built around OpenStack, and who can donate chunks of code in the belief that the community can get it to market more quickly than they can.

At this stage, it is not surprising that the profile of developers is very different to their profile during the recent evolution of the Linux Kernel.

Let’s compare.

In 2012 a long and detailed Linux Foundation report into post-2005 development (Linux Kernel Development: How Fast it is Going, Who is Doing It, What They are Doing, and Who is Sponsoring It ) concluded: “Since 2005, over 7,800 individual developers from almost 800 different companies have contributed”.

There’s a patch roughly every 13 minutes, it added.

Download the report for yourselves, because there are a lot of useful statistics in it, but some basic crunching of the numbers shows that Linux development is broad-based and sustained.

First, the number of developers has been steadily increasing since the release of 2.6.11 in 2005.

The number of contributors per million lines of code, though volatile, is broadly consistent.

And although many contributors do not identify as working for a company (therefore this is a rule of thumb measure), the number of developers divided by the number of identified companies shows a gradual decline.

The top employer of contributors (Red Hat) supplied just 11.9 per cent of the changes in the last release before the report was published. This implies that, if anything, a more diverse group is contributing to development.

Compare this to OpenStack development. At the moment, it is much more centralised. There are several good reasons for this. Initially, it is hardly surprising that development was dominated by Rackspace and Nebula, because they had a head start.

Second, it is still a development in its early stages: by 2005, Linux is already a mature product, with commercial support and broad adoption.

Third, this is a multi-component infrastructure development. Expertise and motivation are unlikely to be widespread and development is a constant process of opening up new projects.

Mirantis offers the Stackalytics analysis tool, which shows who is active in the OpenStack community, and what they contribute.

In the past six months Red Hat, Rackspace, IBM, Mirantis and HP dominate the commits, with 63 per cent of all commits made in that period. Dell had made only 12 in total, so while its public cloud fail gathered negative commercial publicity, it had little effect on the technical evolution of OpenStack.

If we make a graph of the Big Five contributors over the four releases, it shows that the initial contribution by Rackspace and Nebula has diminished as the other major contributors to the Foundation start to extend OpenStack.

Of course, the number of commits doesn’t reflect the significance of the work, but at least it is measurable. I have shaded HP’s commitment.

So distribution of commits over the four releases shows an equalisation of effort among the major backers. But compared with Linux development, after the first seven or eight contributors there’s a long tail.

Over time the top one or two contributors are contributing proportionally less in each release. The slack is being taken up by the groups immediately below, such as HP, not by a large community.

This is good news if you believe that OpenStack will fly only if the major vendors in the OpenStack foundation drive it forward by placing it in the middle of their product set, as HP has done, and also contributing the insights that it gives them. If, on the other hand, you want the pattern of activity to mirror the development of the Linux kernel, you will be disappointed.

This might not indicate anything more than the fact that OpenStack and Linux are fundamentally different projects. OpenStack may form the core of a non-proprietary cloud environment in time.

The fact that it hasn’t got there yet, and that its development may always be centred more on large vendors than that of other open-source initiatives, is not a cause for panic. ®