Good things come in threes, right? So this poor software guy is happy that he's had some great opportunities to learn about hardware over the last few days.
The third of three is Robin Harris's fascinating article: The post-RAID era begins.
Harris starts out by pointing out that simple RAID 5 algorithms are failing to scale due to the immense growth in disk capacity.
He proceeds to discuss the substantial progress that is being made in "erasure coding", which involves new ways of encoding stronger parity checks into your data.
In the ’90s a new form of erasure coding was developed that enabled developers to create codes with an arbitrary level of redundancy – survive 4 failures? 10? Pick a number! – called fountain or rateless erasure codes.
Harris suggests that in the (near) future, storage systems will no longer be using simple RAID techniques, but instead will be using the more sophisticated erasure coding technology, and references a number of startups who are active in this area.
Harris also links to a nice presentation by Professor Jim Plank of the University of Tennessee: Erasure Codes for Storage Applications.
The Point of This TutorialProfessor Plank is not kidding about the coding needs being "not straightforward": by the fifth page of his paper A Tutorial on Reed-Solomon Coding for Fault-Tolerance in RAID-like Systems we're into arithmetic over Galois Fields...
- To introduce you to the various erasure coding techniques.
- Reed Solomon codes.
- Parity-array codes.
- LDPC codes.
- To help you understand their tradeoffs.
- To help you evaluate your coding needs.
- This too is not straightforward.
Professor Plank's web site is loaded with lots of additional information about erasure coding.
Thus ends our short three-part series of the software guy learning more about hardware...
No comments:
Post a Comment