Stuff I'm reading, Thanksgiving edition

Gearing up for all that time I'm gonna spend on the couch watching football, I gotta find something good to read...

• The Mature Optimization Handbook

  Knuth’s famous quote about premature optimization was never meant to be a stick to beat people over the head with. It’s a witty remark he tossed off in the middle of a keen observation about leverage, which itself is embedded in a nuanced, evenhanded passage about, of all things, using gotos for fast and readable code. The final irony is that the whole paper was an earnest attempt to caution against taking Edsger Dijkstra’s infamous remark about gotos too seriously. It’s a wonder we risk saying anything at all about this stuff.

• UDT: UDP-based Data Transfer

  UDT is a reliable UDP based application level data transport protocol for distributed data intensive applications over wide area high-speed networks. UDT uses UDP to transfer bulk data with its own reliability control and congestion control mechanisms. The new protocol can transfer data at a much higher speed than TCP does. UDT is also a highly configurable framework that can accommodate various congestion control algorithms.

• Solar at Scale: How Big is a Solar Array of 9MW Average Output?

  The real challenge for most people is in trying to understand the practicality of solar to power datacenters is to get a reasonable feel for how big the land requirements actually would be. They sound big but data centers are big and everything associated with them is big. Large numbers aren’t remarkable. One approach to calibrating the “how big is it?” question is to go with a ratio. Each square foot of data center would require approximately 362 square feet of solar array, is one way to get calibration of the true size requirements.

• DEFLATE performance improvements

  This patch series introduces a number of deflate performance improvements. These improvements include two new deflate strategies, quick and medium, as well as various improvements such as a faster hash function, PCLMULQDQ-optimized CRC folding, and SSE2 hash shifting.

• How long do disk drives last?

  The chart below shows the failure rate of drives in each quarter of their life. For the first 18 months, the failure rate hovers around 5%, then it drops for a while, and then goes up substantially at about the 3-year mark. We are not seeing that much “infant mortality”, but it does look like 3 years is the point where drives start wearing out.

• Farming hard drives: 2 years and $1M later

  In the last 30 years the cost for a gigabyte of storage has decreased from over $1 million in 1981 to less than $0.05 in 2011. This is evidenced by the work of Matthew Komorowski. In addition, the cost per gigabyte also declined in an amazingly predictable fashion over that time.

  Beginning in October 2011 those 30-years of history went out the window.

• How to be a Programmer: A Short, Comprehensive, and Personal Summary

  To be a good programmer is difficult and noble. The hardest part of making real a collective vision of a software project is dealing with one's coworkers and customers. Writing computer programs is important and takes great intelligence and skill. But it is really child's play compared to everything else that a good programmer must do to make a software system that succeeds for both the customer and myriad colleagues for whom she is partially responsible. In this essay I attempt to summarize as concisely as possible those things that I wish someone had explained to me when I was twenty-one.

• What It's Like to Fail

  During the nearly 18 months I spent homeless off and on, and during the ensuing years, I learned that I am more resourceful than I ever imagined, less respectable than I ever figured, and, ultimately, braver and more resilient than I ever dreamed. An important tool in my return to life has been Craigslist. It was through Craigslist that I found odd jobs -- gigs, they often are called -- doing everything from ghost-writing a memoir for a retired Caltech professor who had aphasia to web content writing jobs to actual real jobs with actual real startups.