LWN.net Weekly Edition for November 23, 2005
The end of USENET
Your editor, ancient relic that he is, first discovered the wonders of global email around 1981, thanks to a BSD-running VAX with a blazingly fast 1200-baud uucp connection. A USENET addiction was quick to follow; on the net, it was possible to converse with a few thousand people on literally hundreds of computers! It was an eye-opening introduction to what a global conversation could be like, both good and bad; hopefully some of those ill-advised, youthful conversations on net.singles and net.politics are lost forever.As it happens, your editor was late to the party, and the old-timers were busily worrying about how the whole thing was going to collapse under the load of all these new, clueless users. USENET proved to be resilient, however, to the point that the "death of the net" idea became a sort of running joke. It survived its rapid growth, thanks to faster modems, better software (including a thing called "rn" posted by a young Larry Wall), and user education. USENET survived the loss of the central "seismo" hub, in the process (as seismo's connections were shifted over to a new host called "uunet") kicking off the commercial ISP industry. It survived the abrupt arrival of AOL, initially connected via a uucp link of its own (here's a classic posting on how the AOL folks were perceived at that time). It even survived the beginning of the spam onslaught - the famous "green card spam" was carried via USENET, not email.
USENET was a useful medium for a long time. Among other things, much of the very early Linux development conversation happened over USENET; your editor decided to go for Linux after noting that the relevant groups had much more going on than the BSD groups. When LWN was first launched, the announcement went to comp.os.linux.announce - the news source for Linux users at that time. Many years earlier, Richard Stallman's first GNU Manifesto posting happened on USENET. The next time you complain about your distributor's repository, think back to the joy of receiving GNU emacs over USENET - as a large number of 50KB chunks which you got to piece back together yourself.
The legacy of USENET also surrounds us in other forms. Many of the features in your fancy mail client which allow you to deal with your incoming flood were first worked out for netnews reading. News clients still have their uses; your editor would have a hard time keeping up with so many lists if it weren't for the highly useful, NNTP-based Gmane repository.
The Globe and Mail has recently declared the death of USENET, as a result of Rogers Communications deciding to stop providing netnews access to its customers. Others might have noted the death of USENET earlier this year, when AOL disconnected its customers. But the fact of the matter is that USENET has been dead as a medium for useful conversations for some years now. It is too open, too easy to flood with spam, too easy to forge control messages for. The signal-to-noise ratio of USENET - often not all that high to begin with - sunk to a point that most people had no remaining desire to deal with it.
So it is not surprising that the commercial service providers are pulling the plug on USENET. A news feed requires significant bandwidth, and its contents seem to be mostly spam and porn. Few customers care anymore. There are much better alternatives out there now; the global conversation has moved on to different forums. USENET is dead, and, at this point, few of us miss it. But USENET played an important role in the history of the net as a whole. Those of you who were there: raise a glass to the memory of USENET at your next opportunity.
Open document formats and the path to world domination
It is almost ten years to the day that Bill Gates made his "Pearl Harbor" speech, which placed the Internet at the heart of everything Microsoft did. The recent announcements of Windows Live and Office Live may not be quite so epoch making, but it nonetheless represents a major change of direction for Microsoft, and has interesting implications for free software.The parallels between Microsoft's two strategy shifts are striking. Both were triggered in part by spectacular IPOs: Netscape's in 1995, Google's in 2004. Both sought to head off the same threat of OS-independent computing. Back in 1995, Gates was worried that Netscape's software might create a "Webtop" platform, where Java applets would be downloaded over the Internet into the browser to provide word processors, spreadsheets and the rest. In 2005, another Net-based approach software services of the kind popularized by Google not only allows the browser to provide those same functions, but comes with a flourishing ad-based revenue model to sustain it.
Gates's response is also similar in both cases: to embrace the basic idea so as to reduce the appeal of rival offerings, and then, ultimately, to use it to tie users more closely to his products. The success of that technique can be seen in the dominance of Internet Explorer, which not only replaced Netscape Navigator as the most popular browser, but managed to subvert Web standards to such an extent that Navigator was ultimately perceived as inferior since it was unable to work with the huge number of IE-specific sites.
One lesson to be learned from this history is that Microsoft should never be underestimated, even perhaps especially - when it seems to be wrong-footed and forced to adopt technologies that apparently threaten its empire. Fear has always given the company focus. The new Windows Live system may look innocuous and even conciliatory it can not only be accessed from GNU/Linux machines, but also explicitly supports Firefox - but the back-end hooks into Microsoft's products are likely to be deep.
The second and probably more important lesson to be drawn is that the much talked-about Google Office service if and when it does come is not going to be the Microsoft Office killer that many seem to imagine. Whatever Google or anyone else might do in this sphere, Microsoft can simply match it, at least in terms of functionality.
But one thing that Microsoft is unlikely to offer is support for truly open file formats, its recent announcement of the "open standardization" of Office formats notwithstanding. The technical and legal details of this will need to be examined closely to see whether it is yet another case of Microsoft apparently promising much, but in reality delivering considerably less. After all, if it did support a completely open file format, the barrier to switching to other office suites would disappear.
Until the approval of the new OpenDocument Format (ODF) standard by OASIS, there were many alternatives to Microsoft's office file formats, but none around which other manufacturers or major users could rally. With ODF, there is now not only an official standard, but a real choice of software that supports (or will support) it.
The key role that ODF will play in tomorrow's battles between open and proprietary approaches is already evident in the furore surrounding the Commonwealth of Massachusetts's decision to adopt ODF as an official file format. The rather forced logic of Microsoft's comments on this move is an indication of the company's difficulties in neutralizing this threat. Moreover, Massachusetts may turn out to be no simple loss of business, but a tipping point that could lead to large-scale defections from Microsoft's proprietary formats to open standards. Anyone who doubts that such a shift is possible should bear in mind that WordPerfect and Lotus 1-2-3 once dominated their respective sectors as totally as the programs that displaced them - Microsoft Word and Excel - do now.
An even more serious blow to Microsoft's grip on the office market could come from Europe. The European Union (EU) is keen to promote what it calls open document exchange formats. One of its technical subcommittees approved a series of recommendations that effectively back ODF provided it becomes a recognized standard. Bizarrely, OASIS does not count as a standards body in this context, and so ODF has been submitted to the better-known International Organization for Standardization (ISO). ODF could emerge as an ISO standard sometime next year. At that point, the EU may well throw its considerable weight behind ODF by specifying it as the preferred format for public sector communications in Europe. Microsoft is acutely aware of this threat: it is no coincidence that it announced the standardization of its Office formats in Paris, not Redmond.
Private sector support is gathering momentum, too. The original donor of the OpenOffice.org code, Sun, has naturally adopted ODF in its StarOffice 8.0, and also offers a grid-based service for bulk conversion of Microsoft Office documents into ODF files. Another major player in this area is IBM, which uses OpenOffice.org formats for its groupware product Workplace, likely to be the successor to Lotus Notes.
The strength of both of these companies' commitment is shown by the fact that, despite their other differences, Sun and IBM jointly hosted an ODF summit at the beginning of November; those attending included Google, Nokia, Novell, Oracle and Red Hat. One of the items discussed was the creation of a formal ODF Foundation to promote the standard. An Open Document Fellowship bringing together individuals interested in the development of ODF (including the present writer) already exists.
ODF is fast emerging as one of the most important recent developments in the software world had it not existed, Microsoft would surely never have embarked on its "open standardization" process. In time, its appearance in May this year might even turn out to be as pivotal as Bill Gates' Pearl Harbor Day speech. At the very least, it represents a rich new vein that can be mined by open source programmers keen to make their mark. As a young standard, there are still gaps in its software support. Items on the wish list include:
- A plug-in that would allow Microsoft Office users to read and write
ODF files (a server-based
approach is already under development).
- Improved accessibility for disabled users (one of the issues that is
threatening to derail the Massachusetts decision).
- A simple ODF reader,
along the lines of Adobe's Acrobat, that would enable users to read
ODF documents without installing an entire office suite.
- A lightweight
ODF editor even smaller than Abiword, say that would allow
simple changes to ODF text files.
- A Wiki-like collaborative editing system based around ODF Work on OpenFormula, which complements and extends ODF
In the browser wars of the late 1990s, Bill Gates was able to wrest control of the web from Netscape because of the latter's short-sighted attempts to beat Microsoft at its own game notably by adding proprietary twists to HTML. Today, as Microsoft re-invents itself in the image of Web 2.0, the situation is rather different. The importance and power of open standards is more evident, and the free software community is no longer a small and apparently marginal group but, instead, the most important counterpoise to Microsoft, well placed to resist any moves to "de-commoditize" key technologies like Ajax.
And this time, there is a chance to go on the offensive. The open source world has long had the desire to end Microsoft's dominance on the desktop; with ODF not GNU/Linux, as many have believed it may finally have the means.
(Glyn Moody is author of Rebel Code: Linux and the open source revolution.).
A SonyBMG update
One might think that the SonyBMG rootkit story would start to fade away, but that is not, yet, the case. Here's an update on the last week's developments.Those of you who have not yet read Bruce Schneier's Wired article on this episode may want to give it a look. He points out that one might have reasonably expected all of those security and anti-virus companies to say something about SonyBMG's software, given that it has been in circulation for over a year, has arguably infected hundreds of thousands of computers, and even phones home. Most of these companies have yet to explain why they missed such an obvious security compromise for so long.
Meanwhile, the EFF has launched a class-action suit against SonyBMG. As Ed Felten points out, the EFF is taking an interesting approach by putting the spotlight on SonyBMG's other DRM software: Sunncomm's MediaMax. MediaMax lacks some of the rootkit features found in XCP, but it is still highly unpleasant software which, among other things, phones home.
Widening the focus to other invasive DRM software is an important step to take if we want to win the larger battle, rather than just punishing SonyBMG for the XCP episode.
The state of Texas has also filed suit, charging SonyBMG with violations of the Texas anti-spyware act.
What is perhaps most interesting - and hopeful - about this incident is how
it has expanded the debate on DRM schemes. A quick news search shows just
how widely the mainstream, non-technical press has covered this story.
CERT has highlighted it for its November 15
Current Activity Report, offering some valuable advice: "Use
caution when installing software. Do not install software from sources that
you do not expect to contain software, such as an audio CD.
" Even
the Gartner Group has chimed in,
pointing out that the software is easily circumvented, and suggesting that
the music industry is now likely to push (even more) for legislation
requiring that DRM features be incorporated into computer products.
A legislative attack seems like a fairly safe prediction - such attacks have been ongoing for some time, after all. But the climate, which was not entirely favorable to legally-mandated DRM even before, has become harsher. SonyBMG's nasty DRM code has not impeded file sharers or commercial "pirates" in any way - it was, instead, an attack on the people who chose to actually buy the CD for themselves. DRM schemes are an attack on paying customers, and those customers are now figuring that out. More encouragingly, there are occasional signs that the industry is getting a clue as well.
Even more to the point, though, is that the SonyBMG rootkit has raised the question of whether we have the right to control our own computers. The nearly unanimous answer is that, yes, we have that right, and the entertainment industry cannot take that right away from us in the name of stopping copyright infringement - or, in the case of SonyBMG's software, simply keeping their customers from putting music onto their iPods. Your editor once heard Jim Gettys say, at some conference or other, that the DRM fight would be like the encryption battle: we would win, but there would be a decade or two of pain to endure first. SonyBMG, by making the issue so incredibly clear, may have done us the favor of shorting out several of those years of pain. Looking back some years from now, we might just find ourselves thanking them.
Page editor: Jonathan Corbet
Inside this week's LWN.net Weekly Edition
- Security: The Senate takes on spyware; New vulnerabilities in fuse, kernel, openswan, netpbm, inkscape, ...
- Kernel: Dynamic USB device IDs; Making notifiers safe; PG_reserved, VM_RESERVED, and VM_UNPAGED.
- Distributions: Gentoo Linux 2005.1-r1; Ubuntu Flight CD 1; Aurora Sparc Linux Build-2.0 Beta2; Zeroshell; SLAMPP
- Development: Inkscape, a Scalable Vector Graphics Editor, Managing Samba, new versions of Simple Grid Protocol, MySQL, Midgard, QjackCtl, Rivendell, Dropline GNOME, Xfce, SQL-Ledger, Cyphesis, SPTK, FreeMED, GTick, ruby/audio.
- Press: Richard Stallman's Tin-Foil Hat, Linux on the Line, Slackware in Sardinia, ODF Battle Gets Ugly, Run Python on Nokia, papers on real-time and embedded Linux, Ubuntu on a Laptop, Free Patent Online Course.
- Announcements: Flash Media Server 2, Sun supports PostgresSQL, EFF on Diebold, GNOME foundation election questions, Guide for Student Bloggers, Wireless HotSpot HowTo, GIMP Splash Contest, LPI passes 100K exam mark, Euro OSCon wrap-up, openlab openday London, ShmooCon 2006, SNORT conf, Medical Reference Wiki.