Device drivers and non-disclosure agreements
The situation is better now than it often has been in the past; free operating systems support a wide variety of hardware. In many cases, the vendors have given in and simply released programming information required for anybody to write a driver. In many others, however, this information is provided to a specific company or developer under a non-disclosure agreement (NDA), with the understanding that the resulting driver would then be released under a free license. This approach has, beyond a doubt, made more drivers available for use with our systems; it has become a common way of doing things, especially in the Linux world.
Not everybody is happy with this state of affairs, however. OpenBSD founder Theo de Raadt has started a campaign against the practice of writing drivers under NDA; in the process, he has stepped on, if anything, more than the usual number of toes, to the point that some of the people involved are now refusing to talk to him. Theo's tactics are never subtle, but he does have a point which is worth listening to.
At a first glance, a driver developed under NDA seems like a good thing. It is free software, after all, and it makes the device work under the target operating system. But these drivers can be problematic for the simple reason that they do not document the hardware the way the specification does. Without that documentation, many of the benefits of free software are lost.
In many cases, only the original author can maintain a driver developed under NDA. Nobody else has the documentation required to make any real changes to how the driver operates; nobody else really understands the device. Whenever a new version of the hardware comes out, or whenever somebody needs a feature that the original author didn't see fit to implement, one can only hope that said author is still around and in a mood to work on that driver.
This situation can be worse yet if the author who signed the NDA writes poor quality code, full of constants whose meaning is clear to nobody. In some cases, the vendor may require that the driver be written in that way in order to expose as little information about the hardware as possible. It's worth noting that this is a problem associated with poor hardware documentation in general. Your editor recently had cause to dig into the OmniVision OV7x20 sensor driver. The data sheet for this device can be found by anybody with access to a search engine, but that data sheet is little help for anybody trying to understand this code:
/* Settings for (color) OV7620 camera chip */ static struct ovcamchip_regvals regvals_init_7620[] = { { 0x12, 0x80 }, /* reset */ { 0x00, OV7620_DFL_GAIN }, { 0x01, 0x80 }, { 0x02, 0x80 }, { 0x03, OV7620_DFL_SAT }, { 0x06, OV7620_DFL_BRIGHT }, { 0x07, 0x00 }, { 0x0c, 0x24 }, { 0x0c, 0x24 }, { 0x0d, 0x24 }, /* ... 45 lines of this stuff removed ... */ { 0x74, 0x00 }, { 0x75, 0x8e }, { 0x76, 0x00 }, { 0x77, 0xff }, { 0x78, 0x80 }, { 0x79, 0x80 }, { 0x7a, 0x80 }, { 0x7b, 0xe2 }, { 0x7c, 0x00 }, { 0xff, 0xff }, /* END MARKER */ };
It's not clear that anybody really knows what all those register settings do; they involve a number of bits and registers which are marked "reserved" in the documentation. For all practical purposes, they constitute a form of opaque firmware which must be loaded into the device for it to operate correctly. Pain will come to anyone who attempts anything more than the most trivial tweaks to these values.
Similar issues (in an entirely different context) recently led Linus Torvalds to exclaim:
Without complete hardware documentation, we will not understand what our peripherals are doing.
Finally, a big problem with drivers written under NDA is that they only work on one system, and they can be very little help for anybody trying to make the device work on a different kernel. That, of course, has a lot to do with why there is a lot of criticism of this approach coming from the BSD world while the Linux community tends to be more accepting of it. It is probably safe to say that most developers who are able to get this sort of access to documentation are working on Linux drivers. If we were pounding our heads against our monitors in an attempt to reverse-engineer hardware by way of obscure BSD drivers written under NDA, we might see the situation in a different light.
Theo has picked out two targets for special attention: Intel and the One Laptop Per Child (OLPC) project. Intel has gotten a fair amount of good press supporting its hardware under Linux. The truth of the matter, however, is that a number of drivers for Intel hardware are written in-house, with little or no hardware documentation provided to the community. As long as Intel remains interested in maintaining those drivers, things will work well enough - for Linux users. BSD users are not so lucky, however, and we may all be out of luck if a change of management or focus at Intel causes the company to drop its Linux drivers. If Intel truly wants to be known as an open-source friendly company, it would do well to make its hardware truly open. The OpenBSD developers are currently running a campaign aimed at pushing Intel in that direction.
Disclosure time |
---|
Readers of this article should be aware that your editor is in the final stages of writing a GPL-licensed driver for the OLPC camera controller - and that he signed an NDA to obtain the requisite hardware documentation. As a result, he is, according to Theo de Raadt, "part of the problem." |
In the OLPC case, Theo's criticism has been centered upon (but not limited to) the driver for the Marvell wireless networking chip. Some very special things are being done with wireless on the OLPC, with the result that it will be able to function as a mesh network router with the CPU powered down. Enabling this involves a lot of close work with the chipset manufacturer - and a driver written under NDA. There are other NDA-covered drivers on the OLPC as well.
Theo is unhappy that the OLPC will be, as he sees it, a closed system for
OpenBSD. [Mr. de Raadt has taken exception to the previous
sentence, consider it removed].
But Theo is even more unhappy because, in his view, the OLPC
project has squandered an opportunity to use its economic power
with the manufacturers to force the hardware documentation out into the
open. This failure is not just a lost opportunity; to Theo it also sends a
message to other vendors that they need not worry about releasing hardware
documentation. So, he says, the OLPC folks have not only failed to do the
best they could; they have also actively made things worse for the free
software community as a whole.
The OLPC folks have several responses to this criticism. The arrangement they have now, they say, is the best they could achieve within their particular set of goals - which, it should be remembered, is the provision of economical computers to children worldwide. OLPC was not founded with the primary goal of helping the free software community, though, in fact, that has been the result of much of its work. OLPC developers make the point that this computer will be one of the most open systems built in many years. The BIOS is free software, as is the VSA microcode which implements x86 emulation on the Geode CPU. The system's SD controller was redesigned (by Marvell) for the express purpose of allowing a driver to be written for it without having to sign the SD Association's particularly unpleasant NDAs. Even the firmware blob which runs on the wireless processor is slated for replacement with free software - though that code does not exist at this point.
Meanwhile, work continues on getting the hardware documentation released. It should be remembered, however, that much of this hardware does not actually exist yet. It would be rare indeed for a manufacturer to openly release this sort of information for a product which is not yet generally available. OLPC's plan appears to be to continue to work with the vendors to get the documentation released as the hardware comes onto the market. Heavy-handed pressure tactics, they feel, would be counterproductive in the end.
The crux of the matter, thus, is this: if we accept that the community needs open hardware documentation to function as it should, what is the best way to get vendors to release that documentation? Some groups encourage ongoing engagement with these companies, with the intent of guiding them toward open source enlightenment. Under this line of thought, these companies will come to realize that the community will do great things with their hardware - growing the market - given the right information; they will see that it is in their economic interest to make the documentation available.
The contrary argument is that this approach has never worked well, that hardware companies will never be brought around in this way. What is required, instead, is an intransigent insistence that the documentation must be released from the outset, and a refusal to sign NDAs to get it. Only when the vendors see themselves locked out of the free software market entirely will they realize that their interest lies in openness, not secrecy. Until that time, there is no reason to cooperate with uncooperative vendors; the preferred approach, instead, would appear to be to attempt to shame them publicly.
There has been enough history of drivers written under NDA that it should
be possible to come to some sort of conclusion as to which approach is more
effective. The OpenBSD camp has arguably had some high-profile success
with the public shame approach. Corporate conversions through quiet
engagement tend to be more, say, quiet, however. Your editor would be most
interested to hear about examples of companies changing their approach to
hardware documentation as a result of working with free software developers
under NDA. The question is not just academic: if we want to bring about an
improvement in the hardware documentation situation, it behooves us to
understand which tactics work best.
Index entries for this article | |
---|---|
Kernel | Development model |
Kernel | Device drivers |
(Log in to post comments)
Device drivers and non-disclosure agreements
Posted Oct 9, 2006 22:17 UTC (Mon) by cventers (guest, #31465) [Link]
I loathe Theo's brutal personality but I have to say his heart is atleast in the right place on the whole free drivers thing. I have to say
that I sympathize with OLPC here and don't think Theo should have picked
them as a target; but I do think that the market, in general, needs to
universally adopt open and unencumbered documentation.
Device drivers and non-disclosure agreements
Posted Oct 9, 2006 23:02 UTC (Mon) by nix (subscriber, #2304) [Link]
Indeed. Jim Gettys makes the cogent point that there is no alternative tothat part, that Marvell have been as helpful as they could, and that (and
this is what Theo seems to have missed) there is *no* documentation that
is complete enough to write drivers: instead, `the documentation is the
code', and that code is licensed under nasty licenses for reasons (as I
understand it) partially out of Marvell's control.
I'm not sure what Theo would prefer: surely not that the OLPC project die
or be crippled due to the absence of the Marvell part?
Device drivers and non-disclosure agreements
Posted Oct 9, 2006 23:35 UTC (Mon) by drag (guest, #31333) [Link]
Why not sign a NDA with Marvell in order to develop the documentation needed for software development?!
I've seen this with the reverse engineered documentation for the Broadcom BCM43xx devices. Of course that was all reverse engineered from Linux binary-only drivers, but I think the end result would be the same.
The results of the drivers being developed based on that are actually very nice. The results are superior to say that of the Ralink drivers who work to improve on code released by ralink. (they can't work on SMP machines, it's not 64bit safe, it's not usefull for non-x86 platforms, etc etc. (The rt2x00 is working on the devicescape branch and drivers fro those devices which will be superior.. but nothing realy usefull has materialized yet)).
So if there is no documentation then get the NDA to develop documentation!! I am sure that since it's OS agnostic you should be able to get developers from OpenBSD, FreeBSD, and maybe even Apple to praticipate.
That way I figure then it's a win-win. Your doing the NDA to get more knowledge out there and make the hardware usefull for a wider audiance.
Of course then this doesn't help for hardware support for people who wish to keep everything secret... But that suites me. If a manufacturer wasn't to obscure how to use their own hardware then I don't want it, as long as a alternative is aviable.
(and not even speaking about internal hardware.. just the software interfaces)
The way I figure:
Open hardware specs + open code > Open hardware specs > Open code > closed code > no support.
Device drivers and non-disclosure agreements
Posted Oct 9, 2006 23:44 UTC (Mon) by drag (guest, #31333) [Link]
If your curious the reverse engineered bcm43xx specs can be found at:http://bcm-specs.sipsolutions.net/
Reverse engineering takes longer than a product cycle
Posted Oct 10, 2006 0:43 UTC (Tue) by gdt (subscriber, #6284) [Link]
Reverse engineering simply doesn't work. I own a Dell Latitude D600 with a Broadcom wireless MiniPCI card. The machine has reached the end of its extended warrranty, so is for essentially obsolete for use by enterprises. The wireless card still does not have a native Linux driver which will authenticate against an enterprise wireless network.
As fine as the technical work reverse engineering the Broadcom chip is, the fact that it needed to be reverse engineered at all is a systemic failure. Compounding this, the Broadcom 802.11a/b/g card is on the edge of being superceeded by a 802.11n card.
And no, I won't be buying machinery with Broadcom parts again.
Reverse engineering takes longer than a product cycle
Posted Oct 10, 2006 0:58 UTC (Tue) by drag (guest, #31333) [Link]
That's not quite what I ment.
I ment that if you have to sign a NDA because of non-existant documentation then sign it so that you develop documentation for other people to use with the hardware.
The BCM43xx documentation was just a example of what is possible for this sort of thing.. Athough that was from reverse engineering Linux drivers and not done under NDA.
In other words.. If it's true that the OLPC have to sign the NDA to work with the Marvell engineers because there is no documentation and they have to get help from the engineers then they should be making the nessicary documentation for other people, like the OpenBSD developers, to use.
If they can't do that then the lack of documentation is just a BS excuse and Marvell has no intention on having their hardware open. (and again, not completely open. Just documentation on what is needed to program software to run on them)
I wasn't saying that people should avoid NDAs completely and work on reverse engineering stuff.
Oh, and the bcm43xx drivers are working fine for me. They were included in the 2.6.17 kernel branch by default although you have to get the firmware seperately. The ralink drivers don't work because I am using a non-x86 machine, although I still very much suggest buying Ralink products instead of the Broadcom stuff. When the devicescape stack gets finalised then those drivers are going to be very very good and full featured. There is just some bugs standing in their way..
Reverse engineering takes longer than a product cycle
Posted Oct 10, 2006 11:10 UTC (Tue) by arafel (guest, #18557) [Link]
I think you'll find most NDAs forbid you from signing them in order to develop documentation to allow people to circumvent the NDA. The lawyers aren't *that* stupid.
Reverse engineering takes longer than a product cycle
Posted Oct 10, 2006 11:20 UTC (Tue) by drag (guest, #31333) [Link]
""I think you'll find most NDAs forbid you from signing them in order to develop documentation to allow people to circumvent the NDA. The lawyers aren't *that* stupid.""
YES. Now you are starting to see my point.
Theo says that Linux devs are screwing up by falling for NDA traps and it's not helping anybody other then themselves. Linux developers say they are doing it because it allows them to work with engineers because documentation is non-existant. The company can't release something that is non-existant.
I say, then if it is true then they can produce the documentation themselves so that other developers can use it to improve and develop their own drivers. However if the NDA forbids stuff like that.. Then the Linux developers are full of shit and are just saying that so they don't look bad. Corporations like that wouldn't release the documentation anyways even if it existed.
I don't have a problem with them replying something like: "Tough shit Theo. The ONLY way we are going to get good drivers in a reasonable time span is by signing NDAs. Next time make a more popular OS so you too can sign NDAs otherwise shut up and go back to reverse engineering with help from the Linux drivers code and bitching about the lack of quality in said code.".
If that is the truth then that is the truth. What can anybody do about it?
Reverse engineering takes longer than a product cycle
Posted Oct 10, 2006 23:26 UTC (Tue) by lambda (subscriber, #40735) [Link]
I think you don't understand what NDA stands for. It stands for "Non-Disclosure Agreement". Thatmeans that you can't talk about what you were working on. Writing documentation, and releasing it
as open documentation, would surely violate most NDAs. And the Linux developers certainly could
be telling the truth when they say there are no docs, but their NDA doesn't let them write docs. In
many cases, there may be closed firmwares, or the files they use for designing their hardware, that
they give the Linux developers access to. NDAs are usually quite broad, to make sure someone
doesn't find a way around, and so they can give you just the code to their closed firmware
while still not even allowing you to write documentation for interfacing with said firmware.
Reverse engineering takes longer than a product cycle
Posted Oct 11, 2006 2:19 UTC (Wed) by bronson (subscriber, #4806) [Link]
It's perfectly possible to sign an NDA to write documentation. In fact, it's happened in the past. Typically the NDA will include a stipulation that any materials produced must be authorized by the company before they can be distributed.
For example, let's say you enter into an agreement with Motorola. They can give you their Verilog code, and ASIC floorplan, or any other highly proprietary information that might be relevant. You would then write the documentation being careful to avoid writing about anything that would be proprietary. Once both you and Motorola are are satisfied with what you've produced, your work can be distributed publicly.
But, if you're under NDA anyway, why not just write code and some extremely well-commented header files? I've written a few drivers in my day and I found that even crappy reference drivers tend to be more useful that the most perfect English documentation. (Bad documentation: S3. Good documentation: Philips. But S3's part was much easier to bring up because they included code)
Reverse engineering takes longer than a product cycle
Posted Oct 11, 2006 8:54 UTC (Wed) by drag (guest, #31333) [Link]
Yep.
NDA is a contract and contracts differ.
I would expect that if it's true that the problem is that there is simply a lack of documentation then that can easily be overcome. Either through self documenting code or other things.
I figure a company may just need help or lack resources and a Linux developer or two can then work with them in a consulting role as a sort of ambassador to figure out the best way to get their hardware open enough to allow other developers write code for it.. but not to violate any prior patent or copyright agreements with other companies and not to risk revealing hardware design more then nessicary. If a Linux developer was to follow that role then it would probably be nessicary for them to know more about the company and the hardware design then is nessicary or desirable and a NDA were the company gets to review documentation and code before release would be appropriate and safe for everybody involved.
It would be a insurance policy to protect not only the ass of the hardware company in question, but also make sure that no third party 'IP' (horrid word) makes it's way into the Linux kernel or other people's software were it could cause legal troubles down the road.
However if the company isn't going to allow you to open up the hardware then the likelihood is that the real reason that there is no documentation or reveiling code or whatever the hell people want is because _they_had_no_intention_in_releasing_anything_at_all_ and why waste resources on something like that? They'll force developers to obsicate the code and thus it will be unmaintable by anybody that hasn't signed the NDA. In that case it's not realy any that much better then closed source. Free software, Open Source software just doesn't work under those conditions. This is what Theo was bitching about.
You'd just end up with something like the 'NV' driver were it's unmaintanable and pretty buggy and just encourages people to use close source drivers.
Reverse engineering takes longer than a product cycle
Posted Nov 14, 2009 17:49 UTC (Sat) by vleo (guest, #32027) [Link]
And if THEY refuse to sign it like that - so be it. Look for another chip vendor. There are enough chips designed to find one that meets your requirements, in both technological and legal areas.
Reasons partially out of $COMPANY's control.
Posted Oct 10, 2006 0:20 UTC (Tue) by shapr (subscriber, #9077) [Link]
"Reasons partially out of $COMPANY's control." - All the hardware vendors say this. Nvidia is the first one that comes to mind, but I'm tired of hearing it. I don't want to pay money to any hardware vendor to get a device that I don't control. This is like DRM, "We'll sell this to you as long as we can choose how you use it."Recently I've been hacking on the driver sources for my Nokia 770. One serious problem is that the Bluetooth firmware has bug(s) where rfcomm keyboards cause the chip to die in such a way that only a reboot fixes the problem. This problem has been known for months, and some of the smartest and most productive coders I know have been having this problem. If they had the firmware source, it would already be fixed.
But they won't get the source, and at some point the bluetooth chip vendor might get off their ass and fix it. But the has no motivation to fix it, they don't get more money for drivers. Users have a motivation to fix it because it affects them directly. Obvious conclusion? Get the users to write the drivers.
Similarly, I want to do cool and nifty tricks with the cx3110x 802.11 chip in the 770, but it also has a binary firmware blob that gets in the way.
When will someone start making hardware and hiring {Linux,BSD,etc} device driver authors to write, release and maintain the drivers?
I would cheerfully pay $5000+USD for my next system if it came with all the source for everything in the system. Is there such an option? I'm not picky about the arch...
Reasons partially out of $COMPANY's control.
Posted Oct 10, 2006 0:28 UTC (Tue) by cventers (guest, #31465) [Link]
I, too, would pay a premium for a truly and fully open system to which allsource was available.
But I wish to simultaneously observe that this discussion we are now
having should not even be necessary! It is downright obnoxious that
computer manufacturing has degraded to this point. Frankly, although we
hear arguments against fully open specs or drivers all the time, I think
that all of them, while possibly true in and of themselves, are even
collectively no good reason not to have open specs!
So I do think Theo is fighting the good fight. He may just be barking up
the wrong trees.
Reasons partially out of $COMPANY's control.
Posted Oct 10, 2006 7:15 UTC (Tue) by nix (subscriber, #2304) [Link]
I concur, for what it's worth. It's a very silly first-strike target,PR-wise.
Reasons partially out of $COMPANY's control.
Posted Oct 20, 2006 3:22 UTC (Fri) by Nimbleno (guest, #41230) [Link]
First strike? Theo and the OpenBSD folks have been fighting this battle for years. And they've had some good sucesses, such as with many wireless drivers. This is hardly a first strike.
Reasons partially out of $COMPANY's control.
Posted Oct 25, 2006 23:03 UTC (Wed) by nix (subscriber, #2304) [Link]
True.
It's a very badly-chosen fiftieth-strike target, for that matter. `Hey,
let's go after a project aimed at... very poor children!'
Theo de Raadt, unsung comedic genius?
There are fully open systems
Posted Oct 10, 2006 2:33 UTC (Tue) by wookey (guest, #5501) [Link]
Depends what you mean by system, but this board comes with everything, including CPLD/FPGA code, jtag programming code, bootloader, drivers, kernel, rootfs, schematics, gerbers. http://balloonboard.org/Of course, whilst you can run Debian on it, it is pretty feeble in comparison to a modern x86 machine, so is not a great desktop-replacement, but it is an example showing that fully open systems can and do exist.
Reasons partially out of $COMPANY's control.
Posted Oct 10, 2006 4:36 UTC (Tue) by danm628 (guest, #5995) [Link]
The companies may not be able to release the documentation even if they they want to.
I worked at a startup (since bought by Intel which is where I now work though on a totally different product -- just making full disclosure here). We developed a wireless product which unfortunately never shipped. But even if we had released it there were limits on what we could have publicly documented. Obviously we had the right to document the portions of the chip that we designed. But we purchased licenses for a couple of large RTL blocks and licensed a couple of large hunks of C code from other companies. These companies exist solely to sell licenses to the RTL modules and code they develop which means they have no desire to publish their code under any of the open source licenses.
Our product would have been "open", we planned to document the PC side API's to the firmware completely. You could have written a driver for Linux (in fact we did for internal use) or OpenBSD. But we could *NEVER* release the source code to the firmware or even document the internal registers that the firmware used. In a single register there could be IP from multiple companies and we can only publish our companies IP. So we couldn't have written documentation that would have allowed outside developers to write code unless they were under the same NDAs that we were under.
And all of this ignores regulatory issues. The FCC really doesn't like software controlled radios to be documented. You have to get approval to sell you chip. Our solution was to put the FCC stuff into our firmware. This made our driver (Windows, Linux, etc.) was very thin.
Many chips are like the one we designed. A small amount of IP designed by the company, the "key" feature of the product. A lot of off the shelf modules that may or may not have publicly available documentation. The company can try to select modules with public documentation but they may not exist. So you have to license what is available, whether or not it has public documentation. The company can try to develop all of the modules itself so it can release the documentation but that takes more money and takes more time. Both of which are in short supply on any project.
I'm not sure what the solution is. Theo does have a point; he usually does. Without public documentation of hardware we can't have a fully open OS. But getting fully open hardware requires changes to how chip IP vendors work, which will take time to achieve.
Reasons partially out of $COMPANY's control.
Posted Oct 10, 2006 6:25 UTC (Tue) by bignose (subscriber, #40) [Link]
> The companies may not be able to release the documentation even if they they want to.
Their choices are what led them to be in possession of hardware which they are not allowed to describe.
> Obviously we had the right to document the portions of the chip that we designed. But we purchased licenses for a couple of large RTL blocks and licensed a couple of large hunks of C code from other companies. These companies exist solely to sell licenses to the RTL modules and code they develop which means they have no desire to publish their code under any of the open source licenses.
This just begs the question one further level. If companies find themselves in a position to obtain part of their hardware design from another party, we need to let them know -- with noise, and real repercussions when they ignore us -- that if they don't get it under conditions that allow them to describe it fully, they are *choosing* to lock themselves out.
> But getting fully open hardware requires changes to how chip IP vendors work, which will take time to achieve.
So, by encouraging vendors when they *can* release the documentation on their hardware, and -- more importantly -- making it plain that we *don't* accept hardware without this documentation, we enlist the hardware manufacturers in pressuring *their* suppliers.
Reasons partially out of $COMPANY's control.
Posted Oct 10, 2006 6:34 UTC (Tue) by cventers (guest, #31465) [Link]
Yes. I believe this to be our challenge and goal in getting the free specsand drivers that we need. All manufacturer statements of "but it can't be
free" must be met with "It must, for we are the consumer that pays your
salary and rent."
I _do_ think that this is already a functional and working strategy.
Certain free software friendly companies have already engaged in coercion
of other business partners for the release of specs or drivers. What we
need is a bigger and more comprehensive effort, and a little bit of time.
But I want to caution at the same time that while we must always be firm
with our demands, we must be careful not to eat or young in the process.
Reasons partially out of $COMPANY's control.
Posted Oct 10, 2006 8:22 UTC (Tue) by shapr (subscriber, #9077) [Link]
How about a startup that writes this sort of free/oss RTL so that companies can release everything about their drivers? Maybe this would be better as a bounty?
If this existed, companies could sell their chips without drivers and users would have to deal with FCC discovered misuse themselves maybe?
How does the GNU USRP get sold? What does the FCC think about it?
Anyway, hardware specs and open firmware is of value to me, so I'd pay a startup to do it.
The earlier post that mentioned the balloon board is the sort of thing I'd like to have, but with desktop sized power. Maybe the Movidis x16 with totally open drivers?
Reasons partially out of $COMPANY's control.
Posted Oct 10, 2006 18:18 UTC (Tue) by flewellyn (subscriber, #5047) [Link]
These companies exist solely to sell licenses to the RTL modules and code they develop which means they have no desire to publish their code under any of the open source licenses.
That's as it may be, but there are now projects like OpenCores that are working on developing free IP blocks for hardware development. The project may not have existed at the time your old startup was working on this card, of course. But now that it does, I would think it'd be in the interest of many manufacturers, as well as the community at large, to start using that resource.
This assumes there *is* documentation.
Posted Oct 9, 2006 23:11 UTC (Mon) by pizza (subscriber, #46) [Link]
For documentation to be released, there has to be documentation in the first place. It is fairly common that there is no "programming document" to be released. The internal docs may take the form of a series of web pages, comments in their source code, or other such things.
Producing (useful) documentation takes a lot of time, effort, and money, and when it comes down to it, not many people actually want to be able to hack the hardware directly -- they generally want something higher-level instead.
It's also a question of support. For chips intended to be designed into custom products, there's a lot of documentation, as that helps you sell more chips. But will having open specs help Intel (for example) sell more wireless modules? nope -- but an open driver might.
There are arguments to be made on both sides of the NDA fence, and no simple answer.
This assumes there *is* documentation.
Posted Oct 11, 2006 1:22 UTC (Wed) by smitty_one_each (subscriber, #28989) [Link]
What hasn't happened yet is a hardware vendor set up a wiki (with reasonable editorial control) and let the community do the documentation.The reason that hasn't happened yet is that no business case has been made.
I'm pretty sure that if a company felt there was profit in doing so, it would have since been done.
I, for one, wouldn't attack Mr. Corbet for writing a driver under NDA. Spreading computational power to other corners of the planet, even under sub-optimal conditions, is likely to help create better market conditions for hardware vendors to move openly documented chips.
Or just grow a legion of reverse-engineers.
Something is better than nothing.
Device drivers and non-disclosure agreements
Posted Oct 9, 2006 23:26 UTC (Mon) by airlied (subscriber, #9104) [Link]
Also having hacked on graphics drivers with and without NDAs, in most cases the documentation is very substandard, only tells you about 1/2 the features, and without engineering resource access (i.e. talking to the guy who designed the hardware) or the Windows driver source, you will never find out about most of the lowlevel errata the affect the device.. these usually aren't documented anywhere but in the Windows driver in any case....
Device drivers and non-disclosure agreements
Posted Oct 10, 2006 1:04 UTC (Tue) by mikov (guest, #33179) [Link]
Amin. My experience exactly.
Device drivers and non-disclosure agreements
Posted Oct 10, 2006 2:33 UTC (Tue) by shemminger (subscriber, #5739) [Link]
Agreed. I have Marvell gigabit doc's under NDA. They don't tell you the intersting bit about bugs and errata's. Someone who has verbal contact with the hardware developers has the best chance of success. If one can walk down the hall and talk to the chip designer, then problems are solved much quicker.
Device drivers and non-disclosure agreements
Posted Oct 11, 2006 9:02 UTC (Wed) by smitty_one_each (subscriber, #28989) [Link]
One wonders if at some point this all won't come to a head.What if the FSF, the OpenBSD community, and a few other suspects ran a fundraiser to:
a) buy the rights to some existing hardware, or
b) design something from scratch
and just have done with it?
Device drivers and non-disclosure agreements
Posted Oct 12, 2006 2:26 UTC (Thu) by drag (guest, #31333) [Link]
Well it's already started, sorta.
Open spec'd hardware design for a video card. The actual logic and such won't be realy open, but everything about the card is going to be well documented.
http://wiki.duskglow.com/tiki-index.php?page=Open-Graphics
And Solaris has openned their Sparc CPU design. They licensed it under the GPL even!
http://news.com.com/Sun+to+release+open-source+Sparc+desi...
Then a couple other places people have pointed out open designed for that Balloon system.
http://www.linuxdevices.com/news/NS8608269433.html
And there is lots of stuff like that.
Neuros technology is interesting.
http://www.linuxdevices.com/news/NS4532837874.html
They actively engage Linux and open source folk as part of their hardware design. They are very open about everything and even have things like beta programs for people to hack around with hardware before the general public release.
Open source doesn't realy apply well to hardware often. Hardware by definition is propriatory, there is no way around it. And there is no real reason around it either as long as everything is interchangable parts.
But I think that some elements would probably be benifitial.
I think that maybe Linux inroads into embedded devices may be a good thing. As fast hardware gets smaller and smaller hardware gets faster it may help convince hardware developers not only to be open with how to program their devices, but also affect the way they design it to avoid having to expose trade secrets to third parties while still embracing open source software.
Unfortunately right now in embedded-land they (the hardware makers) seem very paraniod about commodizing embedded designes like the 'IBM compatable' pcs did for the personal computer. They are trying to avoid that by going ultra-propriatory.
Device drivers and non-disclosure agreements
Posted Oct 12, 2006 11:22 UTC (Thu) by smitty_one_each (subscriber, #28989) [Link]
>Hardware by definition is propriatory, there is no way around it.
This is my point: if you don't like the definition, and find it insurmountable, *then change it*.
While I wouldn't contend that it's likely that the F/OSS community can beat nVidia or ATI head-on in a graphics shoot-out, hopefully the Open-Graphics effort will culminate in something that runs 2d graphics at reasonable resolution.
I realize that some people are deeply into games, and there will always be an nVidia to push them 31337 hardware.
Others, however, see the compiler as the ultimate game...
Device drivers and non-disclosure agreements
Posted Oct 12, 2006 13:44 UTC (Thu) by grahammm (guest, #773) [Link]
Hardware (or rather the interface to it) does not have to be proprietary. Take UARTS for example. There are a vast number of UARTs, whether discrete or emulated in FGPA etc, or even the interfaces to GSM etc modules, which use the register interface originally used by the NatSemi 8250 - often with the 16550 FIFO extensions. Similarly up to and including VGA, the register interface to video cards was standard across manufacturers.
Device drivers and non-disclosure agreements
Posted Oct 19, 2006 12:03 UTC (Thu) by renox (guest, #23785) [Link]
About the spec for the SPARC: they are totally useless for SW developpers, what is needed is the spec of the interface for the *chipset*.
And Sun didn't open this part, I remember it quite well: Theo complained quite loudly that Sun tried to look cool by open-sourcing the SPARC, but they didn't open-source the important part..
Understanding and choice of tactics
Posted Oct 10, 2006 2:33 UTC (Tue) by jreiser (subscriber, #11027) [Link]
If we want to bring about an improvement in the hardware documentation situation, it behooves us to understand which tactics work best.That is too simplistic. Achieving effectively free and maintainable drivers for much or most of the interesting hardware is a game. Both the payoff matrix and the cost matrix are likely to have time-varying, non-linear, non-positive entries. Until there are effective feedback systems involved, then the best strategy is to play the game as many times as possible [produce drivers for many different "adverse" devices, including simultaneously by multiple teams per device], develop a catalog of histories, and adapt dynamically based on observation and history. Insisting on knowing and using only the best tactics can be counter-productive. Play, and play again, and keep on playing; observe, record, and share the history; adapt. Play enough times to find and/or create one or more feedback systems with positive, non-epsilon gain. Then exploit those positive feedback systems vigorously [and _still_ keep on playing as many times as possible.]
The code is the documentation often
Posted Oct 10, 2006 8:32 UTC (Tue) by arjan (subscriber, #36785) [Link]
(disclaimer: I work for Intel but don't speak for Intel nor do I speak about Intel below; my comments are generic and from a broad experience outside Intel in a previous life)
It's nice to request documentation for hardware, some people do it politely, others try to use unfriendly ways such as verbal abuse or blackmail. And for many things, documentation is actually available. If you look at it, most things that have good driver-writer documentation available are those which are around for a long time, with slightly different revisions over time (e1000 is such an example from Intel, other companies have similar examples; especially in the server/enterprise space you will find this kind of hardware with good docs).
However that's only one half of the story. For many pieces of hardware, no such driver-writer level documentation exists! At all. Often the driver (often only the Windows one) gets developed together with the hardware, and the driver ends up BEING the documentation. (sometimes there is a document that claims to be documentation but is utterly useless for writing an actual driver from; and often even that doesn't exist).
Theo and others can jump up and down the vendors throat all they want, but what doesn't exist doesn't exist. No amount of begging, pleading, violence, threats and PR changes that.
In addition, with a consumer product shelf life of 3 to 6 months, companies that make such consumer electronics usually don't want to waste the time (and time to market) to create detailed and good enough documentation. And once the product is on the shelves, they're no longer going to care, they know it's gone in 3 months so spending resources is only a waste of money. That's not bad will, that's simple economics of the cutthroat market. Again the server space is better for this, and you'll see companies care more there.
Now it's not all bleak: there is documentation! Just not in the traditional PDF form. It exists in the form of source code that is released for, say, Linux. If you're lucky that is. The sucky cases are when there isn't any open driver at all (hi Nvidia). Let me repeat that: A good open source driver is usually worth 100 times more than the shoddy PDF that's called "documentation"; if it exists at all.
A note about your webcam example: Even the best and most open driver-documentation will have tables like "to make this work, put these 20 values in the registers". Sometimes a vendor can explain what each value means, but often it's just a case of "these 20 worked and we tested that. anything else we don't know". It's nice to know each individual value and what it means, but reality is that often an individual value has no meaning on it's own. This is especially true for calibration and such settings, but often also for cases where multiple analogue components work together.
The code is the documentation often
Posted Oct 10, 2006 10:01 UTC (Tue) by nix (subscriber, #2304) [Link]
A previous life? What do Red Hat *do* to people who leave their employ? ;)
(come to that, I know several people who've gone around the rotisserie from RH to somewhere else and then back to RH... should we call it the Wheel of Karma?)
(sorry, I'm feeling rather silly today. Coffee overdose, perhaps.)
The code is the documentation often
Posted Oct 10, 2006 10:06 UTC (Tue) by nix (subscriber, #2304) [Link]
(Speaking a little more seriously than in my last post)
Another reason why code is often preferable to docs is that the code is, as it were, self-validating. The docs often, how shall we say, *drift* with respect to differing versions of the hardware, but there's no way you can tell if that's happened. If the code drifts with respect to the hardware, it's immediately obvious, because the driver doesn't work any more.
Also, check FCC registration info
Posted Oct 10, 2006 17:10 UTC (Tue) by AJWM (guest, #15888) [Link]
>In addition, with a consumer product shelf life of 3 to 6 months, companies that make such consumer electronics usually don't want to waste the time (and time to market) to create detailed and good enough documentation.
But, if the thing emits any kind of RF at all (as almost any digital device does), they do have to register it with the FCC. Which can include some pretty detailed filings.
> Now it's not all bleak: there is documentation! Just not in the traditional PDF form.
FCC registration info is, if available at all (it may be sealed), is available in PDF, but non-traditional. This will include anything from cover letters to photographs of the device, to test reports, to block diagrams to user manuals.
If you have the device's FCC registration ID (maybe on a label on the case, or on a sticker inside, or may not be present) you can just enter it into the FCC search page (http://www.fcc.gov/oet/fccid/). They also have a new page that lets you search on all kinds of different fields (https://gullfoss2.fcc.gov/prod/oet/cf/eas/reports/Generic...).
No guarantee that you'll find any info about your particular device (I don't know what the rules are about registering, or how long it takes the information to become available or stay available on FCC's site), let alone programmer-friendly information, but it can be worth a look.
Also, check FCC registration info
Posted Oct 10, 2006 19:37 UTC (Tue) by tetromino (guest, #33846) [Link]
I don't see why the FCC docs would help. I mean, how does knowing how many watts of RF the device emits help a programmer who is trying to figure out what random bit of hex needs to written to which register to make the device emit those watts?
Also, check FCC registration info
Posted Oct 10, 2006 23:07 UTC (Tue) by AJWM (guest, #15888) [Link]
As I said, the FCC info may not help. But sometimes it does -- it might help you discover that device Foo contains an Xyz1234 chip, for which specs or a driver is available. (I got an old cheap digital camera whose USB ID was unrecognized to work this way.) Sometimes the info is more useful to hardware hackers than driver writers.
It doesn't cost anything but a little time to look.
The code is the documentation often
Posted Oct 12, 2006 1:33 UTC (Thu) by bignose (subscriber, #40) [Link]
> For many pieces of hardware, no such driver-writer level documentation> exists! At all. Often the driver (often only the Windows one) gets
> developed together with the hardware, and the driver ends up BEING the
> documentation.
This doesn't change the request. If the hardware vendor wants their hardware to work with free operating systems, they should be providing documentation in whatever best form they have. If their best documentation is the source code for a driver, then that is what they can release.
If they *choose* to restrict this best form of documentation under a non-free license (either by choosing that license, or accepting it from another party), then they choose to lock themselves out of free operating systems.
It is *not* within the power of a free software developer to create that documentation under a free license. It *is* within the hardware vendor's power to provide the documentation as requested; if they choose not to participate, that is their decision, and they must wear the consequences.
The code is the documentation often
Posted Oct 12, 2006 6:59 UTC (Thu) by arjan (subscriber, #36785) [Link]
and guess what... Intel (one of the attacked parties here) does provide drivers for just about anything under open source Licenses....
also you "and they must wear the consequences" sentence starts to smell like we open source people have real power other than "your device is not supported"; guess what, we don't. It's not a small power, given the market size of open source today (esp Linux) in the server world, but in the desktop world... that power is near zero unfortunately.
The code is the documentation often
Posted Oct 17, 2006 4:51 UTC (Tue) by bignose (subscriber, #40) [Link]
> also you "and they must wear the consequences" sentence starts to smell> like we open source people have real power other than "your device is
> not supported"; guess what, we don't.
That seems like power enough. Linux is the kernel with the most hardware support, ever, out of the box. Customers with lots of money are realising this.
If the hardware vendors want in on that, consistently and with a minimum of maintenance costs, they need to play by the rules. The rules say that proprietary drivers in the kernel are illegal, unworkable, and unethical.
http://www.kroah.com/log/linux/ols_2006_keynote.html
Device drivers and non-disclosure agreements
Posted Oct 15, 2006 2:38 UTC (Sun) by Baylink (guest, #755) [Link]
A fairly believable argument has been made, I think about either ATI or Nvidia, that the reason that they don't provide documentation sufficient to write FOSS drivers is that it would reveal that their hardware internals violate other people's patents and, for obvious reasons, that's not a business course they're interested in pursuing.In lieu of a better argument, that's the one *I'm* running with...
Device drivers and non-disclosure agreements
Posted Oct 18, 2006 4:24 UTC (Wed) by svkelley (guest, #37299) [Link]
Partially the case. But also the fact that NDA documentation can include detail on the underlying RTL. That makes it an attractive target for competitors.
A good example are two DAC suppliers that I have worked with on embedded products. I have heard from folks at company X irritated over the degree to which company Y had mattched features pre-announced ahead of product launch. Stupid, I know. Never announce early. That is without NDA level access. Imagine how much could be learnt and done with that level of detail.
That being said. An NDA is only worth the paper it is written upon. I deal with countless NDAs through out a given year from CPU to software. The reality is that most NDA detail leave much to be desired. However, there are nuggets of knowledge in there that could be used by the competition or ambitious lawyers for pre-emptive patent law suits.
Sean
Device drivers and non-disclosure agreements
Posted Nov 24, 2011 12:08 UTC (Thu) by dag- (guest, #30207) [Link]
So while Theo likes to keep Linux developers to stand strong against vendors, that only makes sense because Linux developers didn't do so in order to gain market share. If Linux developers would have followed Theo's strategy from the start, there would be nothing to leverage today and there would be no economic interest from vendors in ways that there is today.
Even OLPC would not have been possible. It's possible that Theo is blind to this reasoning.
But while I share Theo's goals, sometimes a straight path makes it impossible to get there.