Xoreax grids up Windows to goose apps

Grid computing is nothing new, but Xoreax, a company that created a set of tools to distribute the job of building and compiling applications across a network of PCs and servers, has come up with a new twist on the idea. It's called process virtualization, and when implemented in a more general purpose "virtual supercomputer" service, the company calls it Xoreax Grid Engine, or XGE for short.

Everyone remembers the SETI@Home screen saver, which starting in 1999 harvested the unused CPU cycles on PCs running the program to sift through the static of the universe around us looking for signs of intelligent life. IBM similarly runs the World Community Grid to harvest unused CPU cycles for do-gooder scientific causes. The two projects have a combined 5 million devices donating their cycles. And of course, long before we were talking about clouds, we were talking about grids of systems based on the open-source Globus Toolkit, Sun Microsystems' Grid Engine (now owned by Oracle), or the open-source Condor project at the University of Wisconsin (now championed by Red Hat). To a certain extent, even Windows HPC Server can be thought of as gridding software, at least it is for Excel workbooks running on Windows 7 workstations.

The XGE grid software of Xoreax, founded in 2002 in Tel Aviv, draws on some aspects of these earlier grid approaches and then takes a different approach, says Uri Mishol, co-founder and chairman of the company. Perhaps most significantly, it gets IT departments who want to use gridding techniques to distribute work across a network of machines or just the option of harvesting unused cycles the same set of tools and gets the system administrators out of the task of maintaining server images and keeping them in lockstep.

"Whenever there is a complex computational workload that could use a grid, there is sufficient idle processing power to meet the organization's need," says Mishol. "The problem is that there is no easy way to access that capacity."

With traditional grid approaches, you generally have to make modifications to the applications you want to spread across the grid so it can run those applications in parallel across PCs and servers linked by a network, and the cost of modifying applications generally makes this impractical, says Mishol. You can also virtualize the server nodes, dropping the grid software inside of virtual machines and making it easier to maintain those nodes, but now you have dedicated hardware and you are not harvesting unused cycles.

XGE, by contrast, isn't a gridding tool so much as a distributed runtime environment for processes in a Windows operating system. Basically, you have a single machine that is doing a particular task, called an initiating node, and a process virtualization runtime, which runs in an XGE agent that you put on all the machines in the grid, called helper nodes. The processes that comprise the running application are parceled out across the network to the helper nodes, which are fed data from the master machine so they can do its computations. Think of it as having an unlimited number of McFlys for a big Biff.

Workloads that have a lot of computation and a little bit of data transfer work best, says Mishol. You wouldn't use this for weather modeling or crash simulations. The XGE agent software uses data compression on the data transferred and has file caching to try to get the right data and keep it local on a host where processing is done.

The software can also be used internally and scale out across the internet to public clouds running Windows nodes – XGE won't see the difference between local and remote nodes (aside from latency, of course) and doesn't care. It just sees another place it can throw processes to run. The software works particularly well for rendering and various kinds of imaging, and has been extended to accelerate CAD/CAM programs, financial analytics like derivatives trading, geo-seismic analysis, and even large data conversion, parsing, and text analysis jobs.

The XGE gridding software is an offshoot of a product developed by Xoreax called IncrediBuild, was which created expressly to parallelize and accelerate the application-building process. The product was launched in 2003 and has had nine years' worth of upgrades and tweaking, and is now at the 4.0 level. It now has over 100,000 users at 2,000 companies; it is not clear what the term "user" means in this context. Google, eBay, IBM, Microsoft, Hewlett-Packard, Nvidia, Electronic Arts, Cisco Systems, Autodesk, and Adobe are among its top clients.

Mishol says that three years ago, companies that saw IncrediBuild accelerating their build processes wanted to know if they could tweak it to grid-up other workloads, and thus the XGE spinoff was started. It is being launched today at the 2.0 release after being used by selected customers in the 1.0 release for the past year. Customers using the 1.0 release have seen their applications accelerate by as much as a factor of 30 times: jobs that used to take days take hours and jobs that used to take hours take minutes. This is provided you have the right kind of job, of course. XGE can run on any Windows XP, Vista, or 7 client and on any Windows Server 2003 or 2008 server.

At the moment, Xoreax is only selling XGE through its software partners on an OEM basis. The pricing is set by the OEMs themselves, but generally speaking it is based on the number of agents you put out on the network and costs on the order of hundreds of dollars per agent. That's a lot less expensive than spending millions of dollars for a dedicated HPC cluster to do work.

If you want to accelerate your own homegrown Windows applications, Xoreax can help and is willing to entertain the idea of selling end users licenses and support to use XGE, but at the moment OEMs are its primary route to market. ®