Puppets, chefs, and community competition
Benefits for LWN subscribers The primary benefit from subscribing to LWN is helping to keep us publishing, but, beyond that, subscribers get immediate access to all site content and access to a number of extra site features. Please sign up today! |
There are many criticisms that one can make of the applications offered by the free software community, but lack of choice is generally not one of them. Our community thrives on competition while our licensing makes it hard to keep secrets from competitors. A recent episode in the Puppet community shows that, while this competition can sometimes take unwelcome forms, there is often little to do but to welcome it anyway.
Puppet is an automated configuration management system intended to make life easier for system administrators; it can be seen as a competitor to venerable tools like cfengine. Over time, Puppet has attracted an active community of users and developers; it would appear to be a tool which is growing in capability and popularity. Puppet is managed by Reductive Labs, which has a clear commercial interest in providing training and support services for Puppet users.
Recently (January, 2009), a project named Chef announced its existence. Chef's developers, who have previously worked with the Puppet code, set out to solve a similar problem. Chef is not a fork of Puppet, though; it's a new project developed from the beginning. Among other things, the Chef developers decided to use Ruby as the configuration language and they chose the Apache License (Puppet, instead, is distributed under the GPL). This project claims to be in active, production use, but its community, at this point, is clearly small. As of this writing, the chef-dev mailing list shows a total of four messages over its entire history.
Initially, the Puppet developers responded confidently to the Chef announcement:
More recently, though, Puppet developer Luke Kanies posted to the project's user list that Chef wasn't competing entirely fairly:
My take is that if your participation in our community is *solely* for purposes of shrinking it by drawing people into your community at the expense of ours, then you should be kicked from our community.
In particular, it is said that one developer from the Chef project has been sending private mail to Puppet users - especially those experiencing problems with Puppet - suggesting that they should switch to Chef. Luke, clearly, sees this activity as a threat to his livelihood; every Puppet user who deserts is one less potential customer. Even without that incentive, though; it can be hard to stand by and watch as others try to woo users away from your project. One need only think back to the days when "Ubuntu is better" posts were a semi-regular feature of the Fedora mailing lists to see how galling it can be.
In this case, a cooler perspective quickly won over and it became clear that there was little to be done. If nothing else, the objectionable messages were private email; there is little that the project could do to stop them even if it wanted to. Beyond that, though, certain things are inherent in the running of a free software project, including:
- There will be competition, in some form or other. Somebody,
somewhere, is sure to decide to scratch an itch, even if that itch is
no more substantial than "I want to run my own project." This is both
a strength and a weakness in our community. The ability for new and
different ideas to develop into functioning projects is the source for
much of the great software we now have, but it also leads to a certain
amount of duplication of effort and confusion of users.
- Some Puppet users expressed dissatisfaction that the Chef developers
had clearly drawn a lot of inspiration and knowledge from the Puppet
project. But, again, that's how our community works. Anybody who
wants to hide the ideas that go into an application would be well
advised to keep their software proprietary and closed. In the free
software world we learn from each other - at least some of the time.
- In a community which values freedom, attempts to silence or banish inconvenient characters will not get very far. When inappropriate or unethical behavior is seen (and spamming users of a competing project to urge them to switch is certainly pushing the boundary), shining light on that behavior is usually the best thing to do. In this case, the discussion made it clear that this email campaign did not inspire respect; it would not be surprising to learn that the pro-Chef emails have already stopped.
Andrew Shafer summed up the situation nicely:
Projects which are focused on "awesome" tend, over the long term, to be rather more successful than projects which worry about what others might be saying about them. They are also likely to be more successful than projects which put their effort into trying to poach another project's users. Puppet appears to have good code and an active and engaged user community. If it can stay focused on that code and that community, this project need not fear what its competitors are doing.
(Thanks to Friedrich Clausen for calling our attention to this discussion).
(Log in to post comments)
Puppets, chefs, and community competition
Posted Mar 10, 2009 21:54 UTC (Tue) by dberkholz (guest, #23346) [Link]
In this case, the discussion made it clear that this email campaign did not inspire respect; it would not be surprising to learn that the pro-Chef emails have already stopped.Cute idea. In my experience, people like that don't have the same morals as everyone else.
Puppets, chefs, and community competition
Posted Mar 10, 2009 22:31 UTC (Tue) by drag (guest, #31333) [Link]
Puppets, chefs, and spam
Posted Mar 12, 2009 21:49 UTC (Thu) by giraffedata (guest, #1954) [Link]
The solicitations here aren't really spam, because spam isn't just unsolicited solicitations. The defining characteristic of spam is that it's blasted indiscriminantly to so many people that the chance of any particular recipient being interested in its message is miniscule. It is thus bad because it takes more from the recipients than it gives.But sending an advertisement for product A to users of a similar product B is quite focused advertising, and there's a significant chance the user of B really will prefer A, and be glad he got the ad. It's not spam, and I'm all for it.
Puppets, chefs, and community competition
Posted Mar 10, 2009 22:37 UTC (Tue) by branden (guest, #7029) [Link]
Keith denied undertaking such poaching (and since I am unaware of any contradictory information, I believe him), but I suspect most people can agree that everyone has benefitted from the realignment of loyalties that followed. Even the XFree86 diehards themselves are happy to all appearances, now that they have a nice, small, quiet sandbox to play in, untrammeled by the distractions of actual users.
Apart from Keith's legendary reputation and proven resume, what distinguishes these cases? I mean socially, not in terms of the technology used.
Was it simply that the XFree86 leadership didn't obviously care about "awesome" first and foremost?
Puppets, chefs, and community competition
Posted Mar 10, 2009 23:23 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link]
Puppets, chefs, and community competition
Posted Mar 11, 2009 14:16 UTC (Wed) by njd27 (subscriber, #5770) [Link]
Is this free software or not?
Posted Mar 11, 2009 23:02 UTC (Wed) by xoddam (subscriber, #2322) [Link]
Puppets, chefs, and community competition
Posted Mar 12, 2009 1:41 UTC (Thu) by Ze (guest, #54182) [Link]
You can't stop this unless you close up the support/development process.
Realistically the only way you can combat this is by providing a better option for your users than the alternative the lurkers are pushing. This includes guidance , support and user experience.
Puppets, chefs, and community competition
Posted Mar 11, 2009 19:04 UTC (Wed) by PO8 (guest, #41661) [Link]
As someone who was peripherally involved with the transition to X.Org, I am quite confident that had the leaders of XFree86 agreed to open governance and open development there would have been no X.Org. This is quite a different situation than what is described in the article for Puppet / Chef.
Puppets, chefs, and community competition
Posted Mar 11, 2009 9:49 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]
I hate this new trend of using Turing-complete languages for configurations. It makes automatic processing of configurations (using Augeas, for example) impossible.
Puppets, chefs, and community competition
Posted Mar 11, 2009 11:44 UTC (Wed) by nix (subscriber, #2304) [Link]
Puppets, chefs, and community competition
Posted Mar 13, 2009 4:20 UTC (Fri) by njs (subscriber, #40338) [Link]
Puppets, chefs, and community competition
Posted Mar 11, 2009 11:50 UTC (Wed) by angdraug (subscriber, #7487) [Link]
Puppets, chefs, and community competition
Posted Mar 11, 2009 13:47 UTC (Wed) by alankila (guest, #47141) [Link]
Puppets, chefs, and community competition
Posted Mar 11, 2009 14:00 UTC (Wed) by clugstj (subscriber, #4020) [Link]
Puppets, chefs, and community competition
Posted Mar 11, 2009 14:09 UTC (Wed) by alankila (guest, #47141) [Link]
In general case the configuration often needs to be updated whenever the tool changes, too. New fields appear, old fields disappear, some are reinterpreted...
Puppets, chefs, and community competition
Posted Mar 11, 2009 16:12 UTC (Wed) by drag (guest, #31333) [Link]
Personally I prefer to seperate data from code as much as possible. Configuration is data, not code, and should not be treated as such.
Also configuration files are user interfaces and should be treated consciously as such, too. So configuration files should be clean, concise, and well documented. This is how you have good usability for a program that is configured through text files.
Even XML can be good, even though it does make your life harder when it comes to good interfaces.
Puppets, chefs, and community competition
Posted Mar 12, 2009 2:54 UTC (Thu) by flewellyn (subscriber, #5047) [Link]
It's not as "easy" as simply evaluating a text file, granted, but as you said, config files are a form of user interface. Still, there's no reason that, with a bit of care, you can't have the flexibility of a full Turing-complete language for config, while maintaining security.
Puppets, chefs, and community competition
Posted Mar 12, 2009 22:42 UTC (Thu) by rwmj (subscriber, #5474) [Link]
/etc/xen configuration files are a good example: It's useful to be able to parse and manipulate
them from outside Xen itself, but because they are really Python code this isn't possible to do in the
general case.
Puppets, chefs, and community competition
Posted Mar 13, 2009 1:31 UTC (Fri) by flewellyn (subscriber, #5047) [Link]
I think one thing about configuration-via-programming that should be adhered to, is that the configuration code should only consist of variable assignments. In a language which has first-class functions, closures, lambdas, and so forth, this still allows for a good bit of customizing, via binding functions to hooks. But, combined with the sandboxing approach, this could keep the application's config secure, while still allowing a good deal of flexibility.
Puppets, chefs, and community competition
Posted Mar 11, 2009 16:14 UTC (Wed) by man_ls (guest, #15091) [Link]
There is also the issue of security. If your config file can contain code then any user with write permissions can make you run arbitrary code. The config file is probably not executable and yet it is executed. The same is true for .bashrc though, so perhaps it's not that important.
Puppets, chefs, and community competition
Posted Mar 11, 2009 18:06 UTC (Wed) by bart (guest, #466) [Link]
configuration files, but also packages, users, cron jobs, etc.: everything you need to get a fully
configured server. By providing a language they allow you to do things like: only install this file on
the server if the operating system is Debian.
Augeas is something Puppet and Chef can use to actually manage the configuration files on the
system. (And in fact, Puppet can use Augeas for doing just that do that.)
Turing complete configs
Posted Mar 11, 2009 23:07 UTC (Wed) by prl (guest, #44893) [Link]
It's perfectly reasonable to want to have either a static config file or a scripting language or both. But what I REALLY object to is being forced to use ONLY the author's favourite language or config format - especially if the author has an interest in selling support for that format...
Turing complete configs
Posted Mar 12, 2009 1:17 UTC (Thu) by jimparis (guest, #38647) [Link]
I use mod_perl and I still need to set up my virtual hosts in a text file (or generate that text file externally with a script).
Turing complete configs
Posted Mar 12, 2009 12:02 UTC (Thu) by Klavs (guest, #10563) [Link]
Puppets, chefs, and community competition
Posted Mar 13, 2009 3:41 UTC (Fri) by jamtur01 (guest, #48573) [Link]
Puppets, chefs, and community competition
Posted Mar 14, 2009 0:22 UTC (Sat) by dmag (guest, #17775) [Link]
But the Puppet "config language" had to be extended in a number of different directions. People need to be able to express complex things like "All servers must have this program installed, except the staging/test servers". Worse, some configuration is repetitive (think vhosts for a webhost). Puppet has a limited way of dealing with that. You can quickly devolve into generating your Puppet config, which is the wrong answer.
Chef started copying the resource declaration style of Puppet. But they implemented the config files as Ruby with extensions. Unlike Python or Perl, you can write very simple to understand config files that are actually valid Ruby. But writing in Ruby allows you to DRY (Don't Repeat Yourself) your configuration. Not just simple variable substitutions, but loops anywhere you want them, calling out to external services (databases, parsing custom config files), etc. Like Rails, Chef gives you a pre-made directory structure (definitions, attributes, recipes, etc.). It looks complex, but it's nice because "everything has a place" and it keeps you organized.
Chef is very useful when building complex apps "in the cloud" (i.e. on Amazon EC2 where servers come and go).
Self-promotion
Posted Mar 12, 2009 14:03 UTC (Thu) by pboddie (subscriber, #50784) [Link]
This kind of thing gets old very fast. It's like someone phoning in a problem with the CD player in a Honda half way (or further) through a journey, and then being asked why they don't sell their car, return home, buy a Toyota instead, and set off once more.
OT: What would be the Python equivalent to cfengine/puppet/chef?
OT: What would be the Python equivalent to cfengine/puppet/chef?
Posted Feb 25, 2010 14:52 UTC (Thu) by keithlard (guest, #50464) [Link]
There is a Python-based clone of Chef available, called Kokki.
However, I'm not sure what it means to ask about a 'Python equivalent of Puppet'. Puppet is written in Ruby, but the configuration language that you use to describe the resources on your servers is a dedicated language designed especially for the job. So you don't need to know Ruby to use Puppet (though you do if you want to use Chef).
There is an interesting discussion about the merits, or otherwise, of dedicated configuration languages (cfengine, Puppet) as opposed to an embedded DSL (Chef, Kokki) here:
Puppet vs Chef
OT: What would be the Python equivalent to cfengine/puppet/chef?
Posted Jun 12, 2012 9:40 UTC (Tue) by baijum (guest, #85078) [Link]
Puppet vs Chef: configuration management showdown
Posted Jan 26, 2010 14:30 UTC (Tue) by keithlard (guest, #50464) [Link]
Clearly 'Puppet vs Chef' is a debate which will run and run - I've addressed some of the divisions between the two projects in my article Puppet vs Chef: 10 reasons Puppet wins. It's clear from the article that many of Puppet's current advantages are directly due to its large user base - itself due to Puppet being the first mover in this market.
It seems that the Chef community know this as well, hence their efforts to recruit Puppet users, especially disillusioned ones. I don't think that's inherently bad - when Puppet and Chef have equal market share, then they will be able to compete directly on technical merit, which can only be good for both projects.