Advanced

Build a "Virtual SuperComputer" with Process Virtualization

Posted by Scott_Ruecker on · 5 comments
Linux Journal; By John Grogan
Mail this story
Print this story

Build and release is a complicated process. I really don’t need to tell anyone that…but I did anyway. But a rapid, precise feedback cycle that notifies whether the latest build of software passed successfully or not, and one that tells you WHAT failed specifically instead of just “operation failed,” can mean the difference between being able to quickly fix defects, or not discovering those defects until later in the project’s lifecycle, thereby increasing exponentially the cost and difficulty of fixing them. IncrediBuild has been addressing precisely this with Microsoft TFS for years, and is now reinventing their tool for Linux developers.

Full Story

  Nav
» Read more about: Story Type: News Story; Groups: Community, Linux, Virtualization

» Get new stories by e-mail: nightly or as they are published

« Return to the newswire homepage

  Related stories
» More like this

Comments (5)

When I read this article I wasn't sure if I wanted to roll on the floor in uncontrollable laughter or launch a tirade in utter disgust. I'll let you decide which route I chose.

So now if I understand this correctly, Microsoft has invented cluster computing and is so generous as to present it as a gift to the Linux community.

(sorry, I'm still rolling on the floor, give me a few minutes to compose myself)

Ok thank you for your patience, now I'll try to collect myself and continue.

If you were working in computing around 1994 you will remember the Linux-based Beowulf cluster project from Thomas Sterling and Donald Becker who were working at NASA at the time. They put together the concepts that launched HPC, High Performance Computing, concepts which were quickly adopted by the rest of NASA as well as many or most academic and research organizations around the world. The idea was that a multitude of similar or dissimilar interconnected computing nodes could be tied together in a cluster and could be used to solve big problems fast. It is no accident that virtually all of the world's largest supercomputers are running Linux and are based on these ideas with roots back to 1994.

So now Microsoft is advancing the idea (with no acknowledgment to the idea's true originators) that big problems can run faster if they are run on a multitude of interconnected computing nodes. They have even given THEIR concept a name, which they have undoubtedly trademarked, and to heap insult on top of injury, they have the sheer audacity to offer this NEW freshly trademarked idea back to the Linux community with Microsoft portrayed as The Great Benefactor.

Excuse me... I am laughing so hard at this point that I can no longer type...
Microsoft has a long history of claiming innovation, when they take the creations of others and pass it off as something they invented. There are too many to list. This wouldn't bother me so much, except that they are doing it with patents now and trying to extort protection money with them.

Microsoft assumes its users are computer illiterates.
DEC invented cluster computing under VMS. I used a VAXcluster in the late 80's.
IncrediBuild is developed (v1.0 was released in 2002) by Xoreax Software who, according to their website, have Adobe, Microsoft, Google, Disney, IBM, and Motorola for clients. It's a dedicated build accelerator rather than a more generic cluster platform.

From the Wikipedia's description:
Quoting: Incredibuild offers the following acceleration solutions:

1) IncrediBuild for Visual Studio, which enables using IncrediBuild as a full Visual Studio integration to speed up build time for all Visual Studio versions, including Visual Studio 2012. Certified Visual Studio Industry Partner.[1]

2) IncrediBuild for Make and Build Tools, which accelerates build platforms including Make, MSBuild, Gmake, VSimake, Jam, nAnt, Jom, SH, Python, VsiMake, BJam, Jam+, and more. Some builds available for acceleration include Microsoft, Intel, NVidia, CUDA (nvcc), Sony PlayStation, Nintendo 3DS, Wii U, GCC, and ccppc.[2]

3) IncrediBuild for Dev Tools, which allows use of IncrediBuild's distribution technology to distribute processes managed by build tools (make variants, SCons, JAM and others), scripts, and custom applications. Virtually any potentially distributable MS Windows-based process can be accelerated through this technology.

4) IncrediBuild for the Gaming Industry.

The cornerstone technology behind IncrediBuild has the ability to create a unique virtual environment. Regardless of which node executes a particular task, IncrediBuild's grid computing technology guarantees that computational tasks always generate reliable results. This is achieved without having to create and manage system images using different types of tasks.[3]

When distributed jobs are initiated, all tasks comprising that specific project are executed by initiating the node's environment (file system, registry, etc.).). For example, if a C++ compiler task runs on a remote node, it uses the initiating node's file system, registry, etc. to ensure that correct copies of the source code files are being compiled.


As a commercial product I expect their 'Linux' development somehow services their existing customers rather than the open-source community. Perhaps just to let Linux systems act as nodes in the 'Xoreax Grid Engine'. The article doesn't mention any specifics of how this is relevant to Linux and haven't looked for a more relevant source yet.
I got the impression that this is more like SETI@home or distributed.net than it is like a Beowulf cluster. Of course it is for building code so it is more along the lines of distcc. There is also Electric Cloud as a competing proprietary distributed computing solution.

Post a comment

A free account is needed to post. Write your comment here first: your text is kept in this browser while you sign up or log in, and is waiting for you when you come back.

Your message:
|||||
10
Markdown: **bold**, *italic*, [text](https://link), - lists, > quotes, $x^2$ for math. Formatting help

Sign up and post (it is free)  or  Log in and post