Understanding the Linux and open software Development Model
Companies spread falsehoods about Open Source methods of development
out of fear that it threatens their revenue stream. I saw this first hand in a Senate committee
meeting when a member of the BSA testified against a Bill I supported. Others simply
conjure up images of wild-eyed hackers breaking into computer systems
and writing bugs. Simply put, anyone under that impression suffers from
hysterical propaganda.
I thought to myself, "Someone at Microsoft has lost their grip. How embarrassing for them to place a two page ad about saving less than a million dollars at Rayovac?" But does it surprise you? Microsoft group vice president Jim Allchin reportedly had to walk into chairman Bill Gates' office in July 2004 and tell him that Longhorn was horribly behind schedule and would never get caught up. "It's not going to work," he said, according to a report in "The Wall Street Journal." The problem was that Vista was too complicated, and Microsoft's age-old methods for developing software just weren't going to be good enough. What's the Difference?I learned my stock and trade when we wore expensive suits and polished our lace-up shoes. Anyone who saw my Open Source style computer labs knew that we managed them professionally, securely and properly. We also deployed Open Source and Standards-based methodology. We put out quality products and did so using quality management and continuous improvement. None of this should seem strange to professional business people. I have wanted to clear up some misconceptions and help people transition from a closed development environment into an open one that shortens time to market while cutting costs. I would like you to walk away from this article with the notion that Open Source Software bears a mark of professionalism. Open Standards UnderstoodI began a short career in public accounting when eight large firms dominated the profession. At that time, each firm used a self-developed proprietary method for auditing companies' financial statements. Firms coveted their proprietary methods and we all signed non-disclosure and non-compete agreements to protect those business secrets. Shortly before joining the profession, some congressmen concluded that the Big Eight didn't do enough to protect against fraud. Congress along with the Securities Exchange Commission ultimately teed off against accountants who fought back arguing that the profession should police itself. To avoid a new law and regulation the profession agreed to form the Financial Accounting Standards Board (FASB). The FASB allowed the accounting profession to police itself while giving the SEC more say in how auditing firms did their work. Today, I notice something similar occurring in the field of information systems. The proprietary model of software development has started giving way to a standards-based model. Today, Congress faces issues with the nation's technology infrastructure that are similar to the ones that prompted accounting reform. Congress will also face intense lobbying and fanatical opposition to change like that put forth by the Big Eight in the early 1970's. The Precipitating Accounting EventPeople consider the fraud at Equity Funding Corp. of America as the event that prompted creation of "standards" in accounting. Like the Enron scandal, Equity Funding ranks as a landmark event. One could easily call it the main event in the history of cooking the books. In 1972, this seller of mutual funds and insurance captivated Wall Street. Fortune ranked Equity Funding as the fastest-growing financial conglomerate in America. A year later, the company had virtually collapsed amid the revelation that earnings were completely fabricated and the soundness of its business a sham. A computer programmer had created software especially for concocting policies out of thin air. And at the time the fraud collapsed, the company was developing additional software that would easily "kill off" a certain number of phantom policyholders without arousing the suspicions of the reinsurers. By the time the company collapsed, more than half of the insurance policies on the company's books were fake. At the time -- this is the early 1970's -- accountants did not know how to audit computer systems and lacked the desire to learn. IBM held 98% of the computer market for hardware and software. IBM lived by a carefully crafted proprietary model and ruled the market. If someone wanted to learn anything about computers, they had to learn it the IBM way. People found the cost of learning prohibitive. If someone wanted to work at IBM to learn the business, their father had to work at IBM. Standards Making BodiesOn their web page, the FASB writes: Since 1973, the Financial Accounting Standards Board (FASB) has been the designated organization in the private sector for establishing standards of financial accounting and reporting. Those standards govern the preparation of financial reports. They are officially recognized as authoritative by the Securities and Exchange Commission (Financial Reporting Release No. 1, Section 101) and the American Institute of Certified Public Accountants (Rule 203, Rules of Professional Conduct, as amended May 1973 and May 1979). Such standards are essential to the efficient functioning of the economy because investors, creditors, auditors and others rely on credible, transparent and comparable financial information. You might notice that the quote above does not specify who designated the FASB as the "organization" in the private sector even though they acknowledge the need for standards. When I took the CPA exam, the FASB had not made an impression on the landscape. So, my examination covered process based auditing methods. We saw little about "standards" on that exam. Today, the CPA exam covers standards extensively as if that's the way things always existed. As an audit manager, I learned a discipline for managing projects. As I became trained in the IBM software development model, I learned how to manage development in a strict and closed manner. In many ways, I worked in a rigid proprietary model similar to the pre-FASB days of accounting. As I changed the direction of my career, I decided to learn about the Internet and discovered the Internet Engineering Task Force (IETF) which promulgated "standards". Once again, I went through a personal transition from a proprietary model to a standards based model. I saw this change as significant since as an early webmaster, I worked alone and needed help to get my job done. Luckily, webmasters found each other in the same predicament and usenet became our saving grace. We had to share information, code and protocols. The Lunacy of Closed Proprietary ModelsIn the early 1980's, a CPA firm known as Arthur Andersen threatened me with litigation and censorship from the State Board and the AICPA. They claimed I had solicited one of their clients by practicing an unrelated business activity they called "Computer Consulting". At the time, the profession considered soliciting of clients as a crime. Lucky for me the FTC pressed the State Boards to change their rules. The proprietary model embraced repression in the work place. When I worked as a rookie in both accounting and software development, I learned that one shouldn't ask questions. First, it demonstrated ones lack of knowledge and lowered ones standing in the department. Secondly, if you asked a question people considered you a pest. So much for mentoring. Without formal training and reference material, I wondered how anyone could do their job. Luckily, I excelled at going through previous audit and development files to find answers I needed. I also borrowed manuals I found in various places. When I began my own firm, I enjoyed the freedom of daily briefings. My team posed questions to the department. In those sessions, people answered questions and shared information. I remember how remarkable it felt when I discovered that we had subject matter specialists within the team who people previously considered worthless. That may not seem like a big revelation today, but at the time it seemed as big as a lunar landing. The fight for political standing within an organization depended on how one made relationships with management. Today, flatter organizations view accomplishment as the main factor for advancement and compensation. Misconceptions for All to SeeRecently, I attended a meeting at the University of North Texas where one of the heads of a department briefed me on the latest disinformation metaphor. She said that opponents of Open Source told her that downloading a Linux application equated to someone giving you a free puppy. For those who have never received a free puppy, it winds up costing you a lot of money after the puppy arrives. The jokester uses the combination of free and puppy to create an oxymoron. I remember thinking how absurd that analogy sounded. Since then, Sam Hiser and I came up with our own analogy: What would you prefer for Christmas - a subway token or cash? (For those who have missed subway travel lately, tokens have all but disappeared.) With Open Source software you have the equivalent of cash - which you can spend freely on anything you want. With the subway token you have limited options, just like the closed and rigid model of proprietary software development. Recently, the CIO of a large organization and I discussed Open Source development processes. His organization has started transitioning from a closed model to a standards based model. I decided to include one of his early letters here to help clear up some misconceptions. Below you will find his questions and comments followed by my comments. " I have not seen anywhere a definition of the 'Open Source method'. I believe (and could be 100% wrong) it has several components: "1. Reuse. The reliance on the Open Source toolset. Before I write anything, I check to see if it has been done before. Reuse existing code, or at least contact the developer of the closest match and get some ideas. "2. Collaborative Development. The registration of projects in Source Forge or Fresh Meat, the submission of enhancements, the scrutiny of submissions, the process of selecting the successful submissions and rolling it into base code. The potential to get work done in parallel with the help of many. Instead of months/years, projects could take weeks or days. If this is true, or close enough, this is the most attractive feature for Local Governments. This could give us the main advantage in leveraging each other's good ideas and contributions. In this sense, I am thinking of Open Source as a Best Practice in Collaborative Software Development. If I am off base, please let me know! "The approach: "3.a. Ad-hoc. Anything goes, no real structure or vision of how the code may come together. If you are lucky, this may work for small projects. Will never work for large ones. Probably 80%+ of open source projects done this way. "3.b. Iterative. Combining #2 and some control - your way of doing development. The 80/20. I would guess 15% done this way. "3.c. Structured. The remaining 5%. "The above is my perception of how Open Source gets it done. Am I even close? "I have an IT staff of 65, out of which about 40 are the programming types. 25 are exclusively mainframe, 5 are Microsoft and the other 10 can work with most environments, including Linux, Unix, Java, PHP, etc. All of them however are new to the concept of Collaborative Development through Open Source. They are asking very basic questions about the process of developing applications in an Open Source, collaborative way. "Are there any books that you know which may be useful to them? Of course I recommended they check out the Open Source Initiative site, Source Forge, and Fresh Meat. But, none of those really cater to the new and inexperienced programmer. "What would you say is the best place for them to start?" Addressing MisconceptionsFalse Notion of Ad Hoc DevelopmentIn answering the letter I explained that I knew of very few successful "ad hoc" projects in either Open Source or proprietary development. Perhaps some projects start with an idea where a solitary person writes some code and then develops it further. To say that 80% of Open Source projects have no structure or vision, and that the process has an "anything goes" nature, escapes me. If one has had any experience in application development, he or she knows the necessity for order, organization and discipline. Open Source software development uses a different process, a different sense of order than closed source; but it's not anarchy. Most free software projects use an iterative approach to development while using open source programming tools. For example, at a project in which I participate we hold meetings using realtime messaging called IRC and have a central server to control the computer code submissions. The entire team can view all the code but cannot add code randomly. We focus on functionality first and polish second. Prior to beginning our project, we published our requirements and our technical specifications. We also selected the standards we wanted to use for every function of the project. We found specifications published by a project at a major university and used those to assist us in writing our own. I let the contributors of a project develop the requirements and specifications by holding a meeting or several meetings. I make sure that the functionality goes in first, which is about 80% of the work and takes about 20% of the time. When the team has a say in writing the specifications, they tend to own the project and act like stakeholders. In the project I'm referencing, we wrote a project plan and established deliverables after we agreed on the specifications. We selected existing Open Source technology from the Gentoo Linux project to allow us to expedite our development. If We Used a Closed Development ProcessIn a closed, proprietary development shop our project would continue to exist as an idea. Instead of people downloading the application and using it to build eGovernment web sites, the project team would still be writing specifications. Business analysts would have engaged in interviewing "users" and the whole project would have just crawled along. In a closed development shop, once we completed the user interviews, a software engineer would begin writing specifications. Once he or she completed writing the document, it would sit on someone's desk for approval for weeks and maybe months. The document would then make its way to a project manager who would fit it into his schedule and translate those specifications into tasks. To emphasize how much waste occurs in the proprietary model, think of how long and how much money each step can cost. Recently, I interviewed an academician working on a grant from the Federal Government using a closed model of development. The government funded $500,000 for the specifications document and gave the applicant one year to complete the project. In an Open Source world, we would have consulted the existing standards base, looked for existing code or projects, found a database vocabulary and tool kit and opened the specs to our team. People would never consider taking a year to generate a specification document. When I reviewed the specifications, I continued to ask myself why anyone would spend that kind of money on a whole project much less the specifications. Unfortunately, in the closed proprietary development model people function as in the example. They don't even question the methods. In fact, while the world moves from the old model to the new, many practitioners just continue as if nothing changed. Back to Open SourceOpen Source methods have evolved as the economy has evolved. The speed of development and the sheer numbers of people working on Internet standards have forced rapid development methods to arise. We find ourselves in a race against time. With digital convergence and a need for national security as cultural priorities, we can no longer linger in a closed development environment. Control of the source code arises as a main issues between proprietary development and closed development. Let's look at the issue from the Open Source perspective: Code ControlIn the Open Source model, a project manager controls the storage of code in a concurrent versioning system called CVS. The whole team and in this case the public can view the code and comment on it. People can build developer snapshots and make suggestions. Members of the team submit code to the project manager and he or she reviews it before adding it to CVS. If the project manager doesn't have the expertise to evaluate the code, someone on the team will evaluate it. Otherwise, the team will test the code to determine if it works. If it does the manager will commit it to the CVS server. During a project, a development team consists of project managers and maintainers, code contributors, release technicians, reviewers and testers. Instead of one or two people having total control of the software code, everyone has their eyeballs on the code. Everyone can view it while the maintainers control the code committed to the repository. Public or PrivateManagement can determine if they want to make an Open Source project available to the public or keep it private. Two of the three projects I managed in the last two years ran privately. The development team had access to the code in those private projects. Once the project went "live", we built binaries or executable code and released it to the public for testing. If an organization has enough people to test their own software, they wouldn't need to involve the public. The Open Source tools do not require that someone release their code to the public. Somehow and somewhere someone got the notion that Open Source development meant that everything had to use one of the many open source licenses. Again that's a misconception. Anyone can use Open Source development tools and processes to create proprietary software. You do not have to release the code or the executables. How the Open Source Development Provides EfficiencyYou can see the Open Source development model depicted below. In this depiction the project team begins by consulting the standards base which includes the IETF's RFC's, ISO and other organizational standards, existing projects, license requirements and the on-line community.  The maintainers and/or project managers organize the project and recruit contributors, technicians, reviewers, documentation specialists and miscellaneous code contributors. Projects often hire specialists for specific tasks. The code maintainers establish the repository also called CVS and have authority over code commitments. Teammates will often review and test each others code prior to a release. The efficiency of this model lies in the use of standards, the large on-line community from which to draw assistance, team members and help from the public. Instead of having to send an analyst to interview users the team starts with a standards base which has already consulted the user community. Once the project team has generated early code they can accept or reject feature requests. People refer to Open Source development as a collaborative model. We can also refer to the model as dynamic in that you can characterize it by continuous change, activity, or progress. Projects continue to improve and growth. Projects feed on each other so that the collective knowledge of the group expands exponentially. Projects which typify the efficiency and effectiveness of Open Source development include Linux, the Apache Web Server, Samba, Open LDAP, Cyrus IMAP, NASA's Beowulf clusters, OpenOffice, the KDE and Gnome Desktop projects, the Globus Grid Super Computer project, MySQL and PostgreSQL relational databases, the GCC complier, Mozilla and the Netscape Web Browser, Ximian's Evolution mail and personal management system and many more. How Open Source Tools Improve the Software Development EnvironmentThe transition to Open Source and Standards-based software development from the closed proprietary model has gained irreversible momentum. The paradigm has changed and those invested in the old paradigm will continue to hang on until someone drags them kicking and screaming from the field of play. In the simpliest of terms, the duplication of effort built into the old paradigm has negative value in today's economic environment. Let's look at the closed development model depicted below. One can easily see that the time-consuming progression of activities cannot keep pace with the rapid development techniques of the Open Source model. Open Source contains a growing body of knowledge. It's similar to Henry Ford's automated production line in contrast to the hand building process that it replaced.  In the closed model above, software development begins with an analysis of a user's needs without consulting existing standards. The analyst prepares a requirements document for a software engineer who writes specifications for a project manager. The project manager breaks the specifications into tasks for developers. The developers write the code assigned to them by the project manager. The code goes through several stages of development and testing until deployed. The user approves the functionality along the way or once delivered. The development team turns the code over to a maintenance organization charged with making bug fixes, patches and changes. The next iteration of the application starts again from scratch and begins the cycle again. This model has the disadvantage of slow development, high initial costs and even costlier maintainence. Compare it to the approach where the team participates in every step of the process without one person becoming a bottleneck. What do Existing Development Squads Need to TransitionFrom my experience with traditional development shops, the learning curve begins and ends with utilizing the on-line resources available to developers. A standard body of knowledge exsts for Open Source developers. The first area with which to get acquainted lies in the archives of the IETF's RFC's. Wander into the world of RFC's and discover how the Internet evolved. Once you gain an understanding of the Internet protocols, you'll have a clear line of sight to the development communities on the Internet. Developers should also learn to tap into on-line communities by searching Google groups. Many have found that they can find an answer to any question by going to the group search site and trying one or more combinations of key words in the search pattern. I know of no one on the project leopard development team that calls a help desk. Instead, they search the web and search the discussion-lists of projects dealing with the same issues with which they require assistance. You'll also discover the value of belonging to a mailing-list discussion forum. I belong to several and learn something new every day. Concluding RemarksI wrote this article to help instill a sense of confidence about Open Source development in those who have not experienced it. I once did a rope course as part of a six-day seminar and felt a sense of terror in watching people rappel a cliff. When my time came, I felt panic as I backed down the hill. After the first leap, the fear disappeared and I found rappelling fun and nothing like I imagined. Trying anything new often provides us with a sense of anxiety.Open Source development provides us with a way to use our existing talents and skills while increasing our productivity. Many people with whom I work consider this development model as mainstream. I experienced personal growth as a result of my involvement with Open Source projects and find it as acceptible at a cocktail party as working as an auditor for a major accounting firm. I believe you will find your own analogy and enjoy your work in this emerging mainstream technology. |
|
This topic does not have any threads posted yet!
You cannot post until you login.