This article is more than 1 year old
Reader suggestion: Using HDFS as generic iSCSI storage
Good idea, bad idea? Speak your brains, commentards
Comment El Reg reader Greg Koss reckons the Hadoop filesystem can be used to build an iSCSI SAN. Er, say what?
Here's his thinking: we could use Hadoop to provide generic storage to commodity systems for general purpose computing.
Why would this be a good idea? Greg notes these points:
- Commodity storage pricing is roughly $60/TB in hardware costs
- Software costs can be zero with open source tools such as Hadoop
- Hadoop provides all the resiliency and management necessary for basic storage infrastructure
- Hadoop also provides DR capability to alternate sites
He proposes HDFS being used as a backing store to an iSCSI client. It will manage the replication (1,2, or 3 copies within the cluster), and the IO across the nodes. HDFS will also manage any DR replication between clusters across a network.
Thus a very inexpensive, highly resilient storage can be used by a client as an iSCSI volume. The performance is good enough for most department-level storage needs.
Greg has a working prototype and wonders what other people think of his notion. ®
Biting the hand that feeds IT
