nav search
Data Center Software Security Transformation DevOps Business Personal Tech Science Emergent Tech Bootnotes BOFH

This time it's SO REAL: Overcoming the open-source orgasm myth with TODO

If the web giants need it to work, hey, maybe it'll work

By Gavin Clarke, 30 Oct 2014

What can the world learn from Google, Twitter and Facebook - apart from how to make millions through ads flinging? How to run a successful open-source project.

The trio in September announced TODO, to make open-source project “easier.” Joining them are Dropbox and Box and code-site GitHub, payment providers Square and Stripe, US retailer WalMart Labs and a body called the Khan Academy.

Open source advocates and open-source failure is a bit like men and faked female orgasm: Most men are sure they've never been involved with one but most women at some time or another have faked it, according to When Harry Met Sally.

Just like successful mutual climax, successful open source is something most in this sector believe they are well acquainted with - and yet there are plenty of failed open-source projects.

Failure is important because open source has become a way of life spanning thousands of projects and committers.

A 2013 Black-Duck survey reckoned on the existence of more than one million open-source projects with a projected two million by 2014. Drupal alone consists of 613 projects and 25,146 committers. The Linux kernel has just over 3,000 committers.

Sixty two per cent of those surveyed believe half of purchased software will be open-source in five years' time, up from 26 per cent in 2009.

Software developed using open-source methods and released under open license runs servers, supercomputers, clouds, devices, cars – even drones: the Linux Foundation recently started the Dronecode Project, just to get an industry-grade open-source platform that’s stable and cost-effective for UAV makers.

Million-dollar firms and billion-dollar markets are now dependent upon open source.

But let’s come back to failure. For some reason, Facebook, Google, Twitter and Co feel they need to step up with some kind of consortium to get open-source right.

First, of course, you need to define your metric – what is “success”? For most, you’re talking about a project with fresh code and lots of active committers that is widely used.

By that definition, Google, Twitter and Facebook are successful.

Google has released more than 900 projects and 20 million lines of code, including Android, Chromium, V8 and Go. Facebook claims at least 90 projects slung to open source, and Twitter’s contributions to the pool include Boot Strap.

Failure however, by definition, is a party that nobody attends.

But even these giants get things wrong - and not always for the obvious tactical reasons.

Remember WebM? Mm?

WebM was Google’s streaming media technology, intended to liberate online video from proprietary codecs – chiefly H.264.

It was a political gambit, bigger than the technology itself, designed to mobilize developers and outflank H.264 through a grass-roots takeover of online video.

The idea officially crashed last year when Google signed a deal with MPEG LA to license patents used in the VP8 video codec portion of WebM.

WebM as a vehicle to supplant H.264-based video online has failed, with the vast majority of web video today coded and played using the proprietary format.

Code purists might hate it, but a project depends on more than just good code and features to survive contact with the real world – it needs good politics, too.

And giants don’t always get it right when it comes to the nitty gritty of projects, either.

Facebook hit the barriers on its 320 Project, building an iOS framework for designing web interfaces. The project was sent into the wild and largely ignored.

Let’s be honest: Most open-source won’t ever hit the giddy industry-changing heights of Android. Success is more humble – worthy and widespread adoption.

And, between the extremes of success and failure there’re a lot of grey areas – you can have projects with lots of contributors (considered a success by our definition) but that are criminally lacking in documentation and project controls, which mean they depend on key leaders and are hard for outsiders to maintain or fix.

Documentation and process are the least sexy bits of open source and get ignored the most, as people want to wade into code. Look at OpenStack.

Until now, much opensourciness has been going on in and around big flagship wranglers – operations like Apache, the Linux Foundation, or Mozilla.

But ODO’s founders reckon open-source is becoming corporate as more companies are now starting to build tools themselves.

And not all authors have the time, resources of interest in getting their projects up to the grade required for acceptance by the likes of Apache.

There are many reasons open-source projects fail. Two of the obvious are, firstly, abandonment – hurling your code over the wall and expecting people to come – and secondly forking, by releasing code to the community but then taking that code and building your own version internally. The risk is that without outside informed help, authors will go on making the same mistakes.

The result will be projects with potential that evolve slowly or require too much heavy lifting and engineering work from those downstream who’d use them.

But TODO says it will now tackle the bread-and-butter subjects where people go wrong: release cycles, community management, how to shepherd a project from inside a company to a community, tooling and instrumentation.

With the right advice TODO reckons projects can grow faster than they would otherwise and bugs will be quickly fixed.

This is not an entirely selfless act: Google, Facebook and Twitter are some of the web’s biggest consumers of open-source code – from languages to frameworks and databases to operating systems. Getting code for their production systems that’s of decent quality and is self-sustaining is a self-evident plus.

Two years ago, Facebook did something similar on the data center with Open Compute, to share server and data center designs with the industry. Facebook wants OEMs to build the systems it wants rather have it the other way around - of having to take what it gets given.

The plus is, Open Compute theoretically delivers more standardized parts and reduces proprietary designs and custom tweaks that lock everybody in. It also means vendors can standardize on more common components, cutting their costs.

GitHub’s director of outreach Chris Kelly told The Reg TODO is “building on the shoulders of giants” with Google & Co.

He reckoned that anybody using an open-source project that’s adhered these these guidlines will know what to expect in terms of good community steward ship, resources attention to testing bug fixing.

“If you are developing a new company and production system you can take one of these projects and put it into a critical system,” Kelly said.

Weeks into TODO, the members have started by sharing information to try and spot emerging patterns.

“We will publish a few things and see how the community responds, and focus on working on those problems first," Kelly said.

Kelly promised the end result will be credible and practical.

“There are projects that come out of these companies that have taken on their own life,” he said. "Google, Facebook and Twitter are very fast organisations, used to iterating very fast.” ®

The Register - Independent news and views for the tech community. Part of Situation Publishing