Catching some rays on the bay

Did I mention that it was hot this weekend?

Although it really wasn't that unbearable here at the house, we were still feeling the get-out-and-go-somewhere urge, so we packed a few things and headed for the water.

Specifically, we headed for a favorite spot: Tomales Bay State Park, located off Pierce Point Road adjacent to the Point Reyes National Seashore.

Although we've experienced it numerable times, the temperature gradiant on the Point Reyes peninsula is always startling: it was 102 degrees in Petaluma and Novato, but a mere 15 miles west, it was a delightful 77 degrees.

Although we got a fairly early start, we made a pit stop at the Nicasio Valley Cheese Company, and by the time we got to the State Park, the Vista Point parking lot was nearly full! We were, lucky, since once the parking lot fills, Mr. Ranger closes the park gate, and no more visitors may enter.

But we found a spot, and found a table, and spent a delightful hour catching up on the events of the week.

After lunch, we stowed the gear and grabbed our walking shoes and headed off down the Johnstone trail to Pebble Beach.

Now, this is not the Pebble Beach with the $500 green fees, but rather the secluded and spectacularly gorgeous beach just south of the main picnic area. (Sure enough, just as predicted, a family was grilling fresh oysters at the picnic site next to ours during our lunch!)

Aside from a pair of kayakers who'd pulled ashore for a rest, we had Pebble Beach entirely to ourselves for nearly 45 minutes, digging our toes into the sand and looking for That Perfect Shell.

A surprising splash in the water redirected our attention offshore, and we realized that, not 75 feet from the water's edge, a group of 5-10 Bat Rays were feeding in the shallow waters of the cove.

Bat Rays are common across the entire West Coast coastline, and are a well-known inhabitant of Tomales Bay, but we'd never been lucky enough to see them before. It turns out our timing was superb:

In May-July, 2-10 pups are live born after a ten to twelve month gestation period. During mating season, June and July, the rays can be seen in Drakes Estero and near the oyster beds north of Hog Island in Tomales Bay. The rays move into the estero for birthing just as the harbor seals are wrapping up their birthing season.

We weren't anywhere near close enough to be able to see if we were watching adults or juveniles, males or females.

It seems that the best way to watch the rays is from a kayak on the bay:

But then at the eelgrass beds near Walker Creek, we saw some interesting shapes arising in pairs of rounded-off shark fins, and had to investigate. YES! It was Tomales Bay’s own bat rays (Myliobatis californica), feeding in a herd in the eelgrass beds, maybe 20 or 30 strong. Oh joy! Since the tide was way down, and the bay is quite shallow there, the rays’ "wings" emerged from the water as they raised them, and slipped back under as they propelled themselves through their orchard. The wings are actually pectoral fins, and they look like shark fins because rays are reasonably closely related to sharks. They use those fins for self-propulsion, and also to move the sand underneath them and unearth delicious crustaceans. They crush the crustaceans with their plate-like teeth, and spit out the shells.

Apparently the Bat Rays once had a bit of a bad reputation, but their relationship with the human co-inhabitants of Tomales Bay has improved:

Ray predation was once considered a major threat to commercial oyster beds. Growers killed thousands every year in Humboldt Bay alone. "That’s not a problem anymore," says John Finger, whose Hog Island Oyster Farm raises the tasty bivalves in Tomales Bay. "The cultivation technique changed a long time ago. Everything is in floating plastic mesh bags the rays can’t get into." Ironically, larger Humboldt Bay rays are major predators of red rock crabs, which do prey on oysters.

There's much to learn about Bat Rays, but more importantly (to me), they're fascinating to watch.

About a year ago we took a wonderful kayak trip with our granddaughter at Blue Waters in Inverness; as we headed back up the trail from Pebble Beach to return home, we both agreed that we need to do that again, and see if we can't see catch some more rays on the bay, soon.

Soleil

It's going to be a scorcher!

At 8:00 AM, temperatures in the center of the state are already nearing the mid-80's. The East Bay exurbs (Antioch, Fairfield, etc.) are forecast to be in the 106-108 range for their highs today.

Even closer to the coast it will be flaming. Guerneville is looking for 102 degrees, as is Glen Ellen.

The heat is more to the north than the south: the San Mateo coastline won't be so bad, as Half Moon Bay expects an 84 degree max, and San Gregorio only 82 degrees. But Santa Cruz is forecast to hit 95!

Closer to home, it will hit 86 at my house, which is very rare (we see about one 90 degree day a year).

And we're not done: the forecast for Chico is to hit 104 degrees today, but they're looking at an astonishing 113 degree forecast for Tuesday!

I hope the weather is beautiful where you are. For us, we're going to take the dog for her walk as early as possible, and plan to be out on our shady, breezy front porch in the heat of the afternoon.

Bloggin' for the man

I was on hiatus, but after some reflection I've resumed writing the occasional blog article for my day job.

The articles I write for the company blog tend to be more narrowly focused, and are specific to the technologies that I am directly involved with at the company.

They also go through a more careful editing process so I don't put my foot in my mouth quite so much.

That also slows the publication rate down, dramatically.

Anyway, if you're one of those people that reads Every Single Word I Have Ever Written, mostly because you've been doing it ever since I was three years old, and can't stop now (Hi Mom!), then you can find my company-related writings at the corporate site.

There, now you know.

The Trial

Among his many accomplishments, the German author Franz Kafka is credited with bringing the modern understanding of the organizational creation known as "the bureaucracy" into being with his 1925 novel Der Process (The Trial).

A century later, American healthcare and the American insurance industry have nearly-completely perfected the behaviors documented by Kafka.

To wit, my wife's recent experience trying to resolve a claim, now six months old and still dragging on.

In the following, HealthCare Co. and BigInsurer have been used in place of the actual healthcare company and insurance company, and the names of the representatives have been changed to M and J, but otherwise the experience is exactly as described:

68 minutes on hold and counting ...

Called HealthCare Co., explained the letter from BigInsurer and was told to call the HealthCare Co. billing department.

Called them and then used my noggin and said, I’d like to do a conference call with BigInsurer as this is my second call regarding the same matter.

Got on the conference call and the two reps (M at HealthCare Co. and J at BigInsurer – both very nice thankfully) reviewed the claim.

HealthCare Co. billing said to call the HealthCare Co. office because they need to resubmit the claim with the correct codes --- J called the HealthCare Co. office and they said to talk to the billing department and who then transferred J back to M in billing.

It’s all about codes. The HealthCare Co. office billed my visit as an out-patient visit and not a preventative care visit.

But to add to the complication and confusion of billing codes … they kept talking about vitamin D – I said: "I didn’t get any vitamins." J said, "no no they are talking about blood work."

OH! Yes, I was sent next door (in the same building) to the HealthCare Co. labs, to get my blood work done.

HealthCare Co. Labs sent it to the hospital who in turn billed BigInsurer.

Yes, still on hold while I type the saga and updating the number of minutes as I hold. J has been checking back in with me for the past 58 minutes of the number listed at the start of this epic email.

She came back on line and said ‘not to say anything bad about that office, but wow there is a serious disconnect over there”. I can’t agree more. She finally was told to call the hospital, was put on hold three times and then disconnected.

I am now off the phone and waiting for her to call back hopefully indicating that she was able to get this resolved. Oh, the wonders of the medical industry!

Kafka would surely have understood, when he wrote:

Try to realize that this vast judicial organism remains, so to speak, in a state of eternal equilibrium, and that if you change something on your own where you are, you can cut the ground out from under your own feel and fall, while the vast organism easily compensates for the minor disturbance at some other spot – after all, everything is interconnected – and remains unchanged, if not, which is likely, even more resolute, more vigilant, more severe, more malicious.

In the cloud, you never know where your server is

I think one of the things that was confusing me about Feedly is that:

http://www.feedly.com

is not the same as

http://cloud.feedly.com

I've been trying to access Feedly from about 4 different computers, and it kept seeming like it was showing me the same articles over and over.

I think that part of the problem (but perhaps not all of the problem) is that I was sometimes using the one address and sometimes the other.

I'm going to try going to http://cloud.feedly.com exclusively, and see if that makes the overall behavior better.

Some mysteries last for decades

I've felt a strange affinity for Amelia Earhart.

In 1937, the year my parents were born, Earhart took off from Oakland's North Field, just a mile from our house, on her around-the-Equator circumnavigation. All my children attended Amelia Earhart Elementary School, and learned her inspiring story at school assembly.

And more: Earhart herself attended Hyde Park High School in Chicago, where my wife was raised and my eldest daughter was born; the four years I spent in Hyde Park were certainly as formative in my life as Earhart's years were for her.

Earhart, of course, never completed that circumnavigation; her plane lost contact with the various Navy and Coast Guard ships she was communicating with, and was declared lost after a massive search effort.

But nothing definite was ever found, which, as the Christian Science Monitor observes, has left a unresolved mystery:

As the search ground on, a new brand of Earhart fixation began, says Susan Ware, author of "Still Missing: Amelia Earhart and the Search for Modern Feminism."

“There are really two Amelias, the Amelia of history and the Amelia locked in this mystery of the missing plane,” she says. “Hers is a fascinating story, but it’s one without an ending, and people can’t seem to get over that.”

One of the organizations trying to understand Earhart's final hours and minutes is a group called TIGHAR, which stands for The International Group for Historic Aircraft Recovery.

Around the 50th anniversary of Earhart's loss, TIGHAR began investigating her disappearance; they've been working on the problem for 25 years now, pursuing the Gardner Island theory intensely almost from the start of their work:

The broad, flat expanse of hard coral which surrounds the island’s shore dries at low tide to provide a very attractive surface upon which to make a forced landing. However, a disabled aircraft on that reef-flat would, at high tide, be partially afloat in 3 to 4 feet of water. Over a period of a few days tidal cycles would move the aircraft inexorably toward and ultimately over the edge of the fringing reef. From there it’s a steep plunge to depths of 2 to 4 thousand feet.

This spring, there's been great excitement in this area, as TIGHAR's 7th Nikumaroro expedition appears to have made a major breakthrough: Niku VII Analysis Update

A higher-resolution copy of the image captured from the raw sonar data reveals details that suggest another explanation for the sonar tail. There appears to be a break in the prominent part of the anomaly, corroborated by a break in the shadow. Also, some of the tail is casting a low shadow so it must have some elevation. Something that always strikes us about Electra wrecks or repair shops where Electras are being restored or rebuilt is the incredible amount of “junk” that came out of the airplane. Rather than a ground scar, a more likely hypothesis is that the tail is a debris field of fuselage wreckage, internal components, cables, crushed fuel tanks, etc. strewn behind the eviscerated center section.

There are some wonderful pages at the TIGHAR site; I particularly enjoy reading The Object Formerly Known as Nessie

What we now call The Bevington Object is a tiny feature in one photo among two hundred fifty-three pictures in the collection of a minor British colonial official. Is this incredibly small speck in an impossibly obscure photograph the longsought conclusive proof that the Earhart/Noonan flight ended on Gardner Island? Is there other evidence that supports the idea that an object in that place at that time might be wreckage from the Earhart aircraft? What can experts see in such a tiny picture that allows them to identify it so specifically?

The recent revelations from TIGHAR have brought lots of attention to the group, as well as some controversy: Did team find Amelia Earhart's plane in 2010 but keep it a secret? Millionaire claims wreck hunters hid the discovery of aviator's crash site 'to get more money'

The suit claims the aircraft recovery group intentionally misrepresented the status of its exploration to Mellon last year, telling him a discovery of Earhart's plane was yet possible if he supported the search.

The lawsuit states Mellon contributed stock worth more than $1 million to the 2012 search and accuses the organization of engaging in a pattern of racketeering to defraud him.

There's surely a lot at stake in the resolution of the Earhart mystery, and, although it's sad, it's no surprise that the lawyers have emerged, as they always seem to do when money is involved.

Still, for the time being, the TIGHAR web pages are a delight to read. They are clearly-written, lavishly illustrated, and have all the pulse-pounding excitement of any Dan Brown thriller.

What will their research reveal next? I don't know, but I know I'll be eagerly watching and waiting to see.

The third dimension

I think I'm slowly starting to grok the idea of a 3D Printer.

I've been reading about 3D printers for years. I know several friends and co-workers who actually have such a printer, or who have access to one at work. I've read about many of the suggested reasons why such a device would be useful, but many of those scenarios have seemed unlikely, remote from my daily life, not something I saw myself doing.

This week, however, we happened to be in Yosemite Valley.

As everyone does, we stopped at Lower Yosemite Falls for the glorious, iconic view.

The Lower Yosemite Falls trail is a short, mostly-level, mostly-paved trail which takes you up to the base of the falls, where you can feel the mist on your face, scramble around on the rocks, eat your sandwich, take some pictures. You can bring young children there; you can bring children of all ages there (though it's a bit steep for a wheelchair). It's the perfect destination for the Yosemite beginner.

Now, when you see the falls from the valley floor, you automatically look up, to the top of Upper Yosemite Falls.

And it's not long before you learn that there is a trail that leads to the overlook at the top of Upper Yosemite Falls.

And you think to yourself: Wow! I bet the view from there is awesome! I should take that trail and see that view!

But: the Lower Yosemite Falls trail is nothing like the Upper Yosemite Falls trail. The one is short, the other is long. The one is shaded, the other is fully-exposed. The one is level, the other is an unrelenting uphill slog which ascends 2,700 feet in 3.6 miles, and then returns the same way you came. People die on the Upper Yosemite Falls trail.

All of this is hard to convey.

So, as part of a major rework of the Lower Yosemite Falls Trail, somebody at the National Park Service (or at Lawrence Halprin's architecture firm) at some point had an absolutely brilliant idea: they designed and installed a gorgeous bronze relief sculpture of the falls, showing the Upper Yosemite Falls trail.

I've been reading topographic maps for 40 years, and I'm pretty good at reading them. I go on lots of hikes, and I spend a lot of time trying to understand what a topographic map is telling me about my upcoming trail. Learning to read a topo is hard. It takes practice, and it's easy to make a mistake, and it's easy to mis-read your topo, thinking that a ridge is a canyon, or that a dome is a valley.

But the new bronze sculpture is immediately comprehensible. Everybody just walks up to it and understands it, with no training or explanation necessary.

So, bringing an overly-long story to a close, it occurred to me, as I re-visited the sculpture and watched both my granddaughter and my mother-in-law interact with it, that now I want to do this with every trail I visit, and this is a perfect scenario for how I would actually use a 3D Printer:

1. Find some trail I'm interested in
2. Bring up the corresponding topo map on my computer and highlight the trail of interest
3. Press a button, and have the 3D Printer print me out a relief sculpture of my trail, topographically accurate, so I can see what it's going to be like

OK, not everybody will want to do this. And I realize that the basic technology is not quite ready for this. But I think I'm only slightly ahead of my time in visualizing this.

And I wonder if I will be alive (and will still be an active trail hiker) in 5, 10, or 15 years, when this becomes not only possible, but routine.

Shear Keys

We're supposed to be in the final home stretch for opening the new Bay Bridge, and this weekend the New York Times carried this story: In California, Bolts May Hold Up a Bridge in More Ways Than One

The new bridge, whose foundation will reach the bedrock underneath, will be the world’s largest self-anchored suspension bridge, an unusual design that contributed to its high cost. Equipped with the latest antiseismic technology, the bridge was designed to last 150 years by remaining intact in a major earthquake.

That is why the failure of some high-strength steel bolts attaching shock-absorbing devices called shear keys to a concrete crossbeam under the roadway raised alarms. When workers tightened the 17- to 24-foot-long bolts in March, 32 in a batch of 96 snapped. Engineers blamed hydrogen-assisted cracking, in which atoms of hydrogen infiltrate steel and make it brittle, but they have yet to determine its cause.

Don't miss the nifty infographic: Solving a Structural Issue, which explains the concept of the Shear Keys with a nice diagram.

Shear keys and bearings allow limited and controlled movement at the joints of different sections of the bridge during seismic events.

Of course, since the whole point of the new bridge is to address seismic issues, it's natural that engineers are intently focused on the seismic aspects of the new design.

I've long been fascinated with this part of civil engineering, since I live in such a seismically active area. Ten years ago, I was privileged to have a window seat for the construction of Oakland's Cathedral of Christ the Light, a beautiful building, and once again a structure which was constructed to replace a structure that collapsed in the same great Loma Prieta earthquake.

Among the interesting aspects of the Cathedral of Christ the Light is the use of enormous wooden beams for the main space, since the natural flexibility of the wooden beams is supposed to make a more earthquake-friendly structure. Interesting, since as the NY Times article about the bridge observes,

the eastern half, a cantilever design that runs from the island to Oakland, was deemed beyond retrofitting. Its main problem was that it was anchored in treated Douglas fir trees that were used as pilings in clay and mud.

Poor trees: sometimes they are the blessing; sometimes they are the curse.

A more common earthquake technique, used in both the bridge and the cathedral, is the use of special seismic bearings. In the cathedral, as the manufacturer points out,

The new cathedral is located just 4.7 km from the Hayward fault and uses Friction PendulumTM seismic isolation bearings to protect it from damage during a 1000 year earthquake. The bearings support a 120 foot tall main sanctuary, which incorporates glulaminated timber, reinforced concrete, high strength steel, and aluminum and glass to create a light-filled structure.

I remember watching the bearings being installed; they were quite fasinating to observe.

And, of course, don't find the language confusing: "a 1000 year earthquake" is not an earthquake which lasts for 1000 years (though in fact the faults in California are nearly constantly moving, and it's isn't an inaccuracy to describe such continual movements in such a way), but rather an earthquake of such severity that we might expect it would occur once every 1000 years.

Meanwhile, back over the water, the engineers are pointing out that nothing is ever perfectly safe, and rather than ask whether the new bridge is flawless, we should be asking whether it is (a) safe enough, and (b) safer than the current bridge: Bay Bridge: Open new span with or without bolt repairs, experts say

If a moderate temblor were to strike after traffic moves onto the new bridge but before the retrofit is done or any other bolts are replaced, the new bridge is still at least twice as safe as the 77-year-old span motorists use today, Seible and Fisher said.

And some engineers just love the fact that the Bay Area is building new bridges: My Word: Stop all the negativity about beautiful bridge project

We in the Bay Area are renowned for our creativity, our leadership in technology and our eagerness to move forward. This bridge exemplifies all of these strengths. And let's not forget that Caltrans originally proposed a conservative, although arguably bland, simple viaduct structure to replace the existing bridge.

We rejected that and are much better off for it.

Although there are no guarantees in seismic engineering because temblors are inherently random, computer modeling and analysis has become so sophisticated that we have a truly remarkable understanding of the predicted performance of large structures in seismic events.

There's no doubt that the attention on seismic design has brought amazing improvements in construction techniques. But I'm also pleased to see that there's always room for improvement, and people are already actively studying the results of these recent large public projects, trying to learn from them and figure out how to do still better in the future.

Tom Fassbender's JMT hike

I'm completely loving Tom Fassbender's awesome series of articles about his three week epic hike on the John Muir Trail in June, 2012.

His writing style is great; I love the links to gear, locations, and background information, and the pictures are a lot of fun, too!

Here are the articles in his "2012 John Muir Trail Adventure" series:

• John Muir Trail Adventure Journal: Day One

  I talked briefly to a group of hikers headed toward Yosemite. They were complaining of the mosquitoes, which I didn’t think were that bad. (But they were going to get a lot worse before this hike was over).

• John Muir Trail Adventure Journal: Day Two

  I thought for a moment that maybe it was a bear until it heard me and looked up. It was just a mule deer. The deer along the trail, especially the does, proved to be almost totally unafraid of humans. It was kind of like being in a petting zoo.

• John Muir Trail Adventure Journal: Day Three

  A bit about trail names.

  There are rules to these things.

  1. You can’t name yourself.
  2. You can’t decline a trail name.
  3. It should have some meaning and/or describe you, be it a good thing, a bad thing, or just a humorous thing.
  4. Once you have one, you’re stuck with it for life and it follows you from trail to trail

  Not all who have trail names abide by these rules, but everyone I talked to who used a trail name earned them by these rules.

• John Muir Trail Adventure Journal: Day Four

  As I crossed a bridge into the heart of the monument, I ran into a day hiker from the nearby campsite. He was the first person I’d seen in about 24 hours. It had been a strange kind of solitude that I didn’t even think about until I saw this guy.

• John Muir Trail Adventure Journal: Day Five

  After a relentless climb and countless downed trees, I made it to as series of lakes named for what white people thought of as traditional roles for the Native American people.

• John Muir Trail Adventure Journal: Day Six

  I talked with other hikers. We talked trail conditions, the gear we used, crazy things we’d seen, other hikers we’d met. This is when I learned about Tuna Helper‘s bid to set a PCT speed record. There was also talk of a hiker who wore different wedding dresses along the trail. He’d wear an old wedding dress for a certain amount of distance depending on how much you paid him. You’d get a picture of course. Somehow we missed each other on the trail. I would have enjoyed seeing that.

• John Muir Trail Adventure Journal: Day Seven

  I skipped my usual lunch, instead enjoying Fritos and Peanut M&Ms. This is the point when the meals started blending together, and I’d switch it up (like soup for breakfast and oatmeal for dinner) to keep things from getting too boring.

  A bit about Fritos

  One of the greatest hiking foods of all time! Per bag, 1,760 calories, 990 of that from fat. Plus a good amount of sodium. These things keep your motor running! And they’re great kindling.

• John Muir Trail Adventure Journal: Day Eight (I've hiked parts of this section myself!)

  Seriously. I can’t even begin to describe how beautiful this place is. And I’m not a good enough photographer (I’m sure you’ve realized that by now) to capture it in photos. But this place was majestic, easily the most beautiful place I’ve ever seen with my own eyes.

• John Muir Trail Adventure Journal: Day Nine

  It’s hard to get here, about two (if you’re fast) or three days worth of hiking from any sort of civilization. Most people won’t ever see this with their own eyes.

• John Muir Trail Adventure Journal: Day 10

  Ah yes, the Golden Staircase. This was the last section of the John Muir Trail to be finished, and it wasn’t complete until 1938, some 23 years after construction of the trail had officially begun.

  I don’t know why it’s called the Golden Staircase. It’s not golden, and it’s not really a staircase. Instead, it’s a arduous stretch of rocky trail that switchbacks its way up to the Palisade Lakes.

• John Muir Trail Adventure Journal: Day 11

  He asked where I was going. I told him. He asked if I knew where the pass was. I told him not exactly, but it was up there somewhere. I pointed to the ridge way up ahead and high above us.

  He nodded. Yeah, see than snow field? Yep. That dark line across it, that’s footprints in the snow that lead right up to the pass.

• John Muir Trail Adventure Journal: Day 12

  At Wright Creek, I ran into a group of about five really stoned hikers. Some of them were in their underwear. They quickly pointed out to me that it was Hike Naked Day (it is also Hike Naked Day when this post was published). I didn’t know there was a thing, and I’m glad they weren’t totally naked, but I’m glad they told me about it. I realized then that I had no idea what had happened for the past week, and even then, the news was sporadic at best. The world could have been overrun by zombies and I wouldn’t have known.

• John Muir Trail Adventure Journal: Day 13

  And then it was over. It was about 2:30 in the afternoon. I got to the Whitney Portal Store, ordered myself a giant hamburger with fries and drank a 22-ounce bottle of Stone Smoked Porter.

What a wonderful report; thank you so much, Tom!

I can't wait to read more of your "trail adventures" in the future.

I'm starting to get nervous about losing Google Reader

We're down to the last two weeks of availability of Google Reader.

Like tens of thousands of you, I'm trying to figure out what to switch to.

I tried The Old Reader, and it looks pretty nice, but it hasn't been very reliable. I frequently get the pictures of kittens, with a polite message apologizing for their inability to show me actual data. So they seem to be polite, and hard-working, but over-whelmed.

I tried NewsBlur, and it looks appealing, but they won't let me add even a single feed without giving them my credit card. It's not all that expensive ($2/month), but I was really hoping I'd get to try it for free and see if it worked or not first.

I guess maybe the next ones I'll try are Feedly and NetVibes.

Am I nuts? Is this working better for anyone else? What is everyone switching to?

I continue to be baffled by e-book pricing

I felt like I wanted to re-read Watership Down; it's been 35 years since I read it and I'd like to re-explore it.

On Amazon, I can find:

• New ("mass market") paperback editions for $6.50
• Used paperback editions for less than $1.00 (+shipping)
• New paperback edition from Amazon is $13.00
• Kindle edition is $11.00

Does anybody actually pay $11.00 for the Kindle edition of a book you can get for half that price?

I thought about just getting the pirated version from the Internet, but decided instead to buy a used copy for $3.00.

I find the whole Kindle thing bizarre. It's such a gorgeous device, but in practice I'm buying almost no content for it.

As the kids say, "just saying."

What I'm reading

Well, Dell Service has visited and the new computer appears to be booting successfully. Time to get back to surfing the web!

• The Cray Files

  These pages contain my project notes on how to build a Cray supercomputer at home. It was – and is – a fascinating journey to me into the history of computers. I’m writing my story up in hopes that others (well, you, dear reader) will find it interesting. It is somewhat of a living document, a journal of sorts. Though the story starts several years ago – and that part I recount from memory – I intend to add new pages to the files as I progress. When I tell this story I leave out some of the dead-ends and my wanderings in the digital desert, so the story-line feels more linear than it actually was. I don’t think you want to hear about all the dozens of failed attempts at solving something. I’ll only leave some of the most spectacular failures in for entertainment value.

• Practical HTTP Host header attacks

  There are two main ways to exploit this trust in regular web applications. The first approach is web-cache poisoning; manipulating caching systems into storing a page generated with a malicious Host and serving it to others. The second technique abuses alternative channels like password reset emails where the poisoned content is delivered directly to the target. In this post I'll look at how to exploit each of these in the presence of 'secured' server configurations, and how to successfully secure applications and servers.

• Bobby Tables: A guide to preventing SQL injection

  There is only one way to avoid Bobby Tables attacks

  • Do not create SQL statements that include outside data.
  • Use parameterized SQL calls.

  That's it. Don't try to escape invalid characters. Don't try to do it yourself. Learn how to use parameterized statements. Always, every single time.

• How tcmalloc Works

  tcmalloc is a memory allocator that's optimized for high concurrency situations. The tc in tcmalloc stands for thread cache — the mechanism through which this particular allocator is able to satisfy certain (often most) allocations locklessly. It's probably the most well-conceived piece of software I've ever had the pleasure of reading, and although I can't realistically cover every detail, I'll do my best to go over the important points.

• How to Build a Highly Available System Using Consensus

  This paper explains how to build efficient highly available systems out of replicas, and it gives a careful specification and an informal correctness proof for the key algorithm. Nearly all of the ideas are due to Leslie Lamport: replicated state machines [5], the Paxos consensus algorithm [7], and the methods of specifying and analyzing concurrent systems [6]. I wrote the paper because after I had read Lamport’s papers, it still took me a long time to understand these methods and how to use them effectively. Surprisingly few people seem to know about them in spite of their elegance and power.

• Distributed Locking: When to use it? How?

  An obvious question to ask yourself before venturing into any of the approaches above is: Do you need a lock service after all? Sometimes the answer is you don’t need to as executing the same workflow twice may not be an issue. Whereas there are many other cases where you need such an abstraction. So, please make this decision carefully.

• Building a Distributed Messaging System

  MessageBus has become an important component in Groupon’s infrastructure, with over 100 topics, 200 subscribers deployed in production. 5 million messages are published to two clusters with 9 nodes daily, and spiky traffic up to 2 million per hour.

• Network Congestion and Web Browsing

  Despite the rejection of this patch, Google’s servers have been leveraging this kernel modification to experiment with increasing the initial congestion window for SPDY connections. Beyond just experimenting with statically setting the initial congestion window, Google has also experimented with using SPDY level cookies (see SETTINGS_CURRENT_CWND) to cache the server’s congestion window at the browser for reuse later on when the browser re-establishes a SPDY connection to the server.

• Scaling Storage Is Hard To Do

  But clustering only goes as far as the interconnect will allow. Systems that relied on Fibre Channel, IP/Ethernet, and iSCSI for inter-node communication could only scale to a handful of nodes before node coordination latency got in the way.

• Atomic I/O operations

  Quite a few workloads — including a lot of database workloads — are especially sensitive to the latency imposed by waits in the filesystem. If the number of waits could be somehow reduced, latency would improve. Fewer waits would also make it possible to send larger I/O operations to the device, with a couple of significant benefits: performance would improve, and, since large chunks are friendlier to a flash-based device's garbage-collection subsystem, the lifetime of the device would also improve. So reducing the number of wait operations executed in a filesystem transaction commit is an important prerequisite for getting the best performance out of contemporary drives.

That's all for now. Have fun!

Network failures and system resilience

On his Aphyr blog, Kyle Kingsbury has been doing some superb work.

First, there's his recent article surveying the ways that networks can fail, defeating our well-intentioned-but-inadequate attempts to survive such failures: The Network is Reliable

This post is meant as a reference point–to illustrate that, according to a wide range of accounts, partitions occur in many real-world environments. Processes, servers, NICs, switches, local and wide area networks can all fail, and the resulting economic consequences are real. Network outages can suddenly arise in systems that are stable for months at a time, during routine upgrades, or as a result of emergency maintenance. The consequences of these outages range from increased latency and temporary unavailability to inconsistency, corruption, and data loss. Split-brain is not an academic concern: it happens to all kinds of systems–sometimes for days on end. Partitions deserve serious consideration.

And don't stop there; make sure you read Kingsbury's series of articles on system architectures for handling network partitions:

• Call me maybe: Carly Rae Jepsen and the perils of network partitions

  This article is part of Jepsen, a series on network partitions. We're going to learn about distributed consensus, discuss the CAP theorem's implications, and demonstrate how different databases behave under partition.

• Call me maybe: Postgres

  Previously on Jepsen, we introduced the problem of network partitions. Here, we demonstrate that a few transactions which “fail” during the start of a partition may have actually succeeded.

• Call me maybe: Redis

  Previously on Jepsen, we explored two-phase commit in Postgres. In this post, we demonstrate Redis losing 56% of writes during a partition.

• Call me maybe: MongoDB

  Previously in Jepsen, we discussed Redis. In this post, we'll see MongoDB drop a phenomenal amount of data.

• Call me maybe: Riak

  Previously in Jepsen, we discussed MongoDB. Today, we'll see how last-write-wins in Riak can lead to unbounded data loss.

• Call me maybe: Final Thoughts

  Previously in Jepsen, we discussed Riak. Now we'll review and integrate our findings.

• Asynchronous replication with failover

  In response to my earlier post on Redis inconsistency, Antirez was kind enough to help clarify some points about Redis Sentinel's design.

I've been really enjoying and learning from these articles; I hope Kingsbury continues to write and publish more great work!

Naval contemplations

Several interesting articles recently, all with a common marine theme.

• Drowning Doesn’t Look Like Drowning

  So if a crew member falls overboard and everything looks OK—don’t be too sure. Sometimes the most common indication that someone is drowning is that they don’t look like they’re drowning. They may just look like they are treading water and looking up at the deck. One way to be sure? Ask them, “Are you all right?” If they can answer at all—they probably are. If they return a blank stare, you may have less than 30 seconds to get to them. And parents—children playing in the water make noise. When they get quiet, you get to them and find out why.

• The Ghost Fleet of Suisun Bay. I didn't realize that the Glomar Explorer was once a Mothball Fleet member.

• When Billionaire Sets Rules, It’s an Exclusive Race

  In San Francisco, an increasingly unaffordable city where critics say the booming tech industry has been coddled at the expense of the less affluent, criticism has been rising. Much of it has been leveled at Mr. Ellison, who recently appeared at a red-carpet premiere of “The Wind Gods,” a laudatory documentary about his 2010 victory that was produced by his son, David.

• Critical decision on America’s Cup could come this week

  If Artemis withdraws from the race, it would leave just two teams — Emirates Team New Zealand and Italy’s Luna Rossa Challenge — battling for the right to take on Oracle Team USA, the current defender of the America’s Cup.

Retweet this!

The Onion nails it: Local Laundromat Employs Social Media Coordinator.

So, so true.

Hoops

Now that the NBA finals are here, you should read this nice article about my favorite basketball player: Mysterious, Wise, and At Peace: Duncan is just Duncan

He’s skipping. This is how he gets his body loose, skipping around, and it’s a sight. He’s 37 years old, he’s at the end of his 16th season, he is playing in his 204th playoff game, and he’s skipping around joyfully like my 8-year-old daughter at a playground she’s never played in before.

Pinnacles National Park

To which season does June 1st belong?

Some people believe that summer begins with Memorial Day and ends with Labor Day. Others see the calendar year divided into quarters: winter, spring, summer, and fall, and thus conclude that summer is June, July, and August. Others are astronomically minded, and see spring as the time from the Vernal Equinox to the Summer Solstice, while still others, most commonly those with small children, see summer as the the time when school is not in session, and thus track it from the last day of one school year to the first day of the next.

So June 1st becomes an ambiguous day; it might be summer, or it might not be; it all depends.

We experienced many of those ambiguities during our June 1st weekend this year.

We took advantage of the brilliant California sunshine to make a short camping trip to Pinnacles National Park. Pinnacles is the newest of America's National Parks, having been upgraded from Pinnacles National Monument in January (in these days of budget-tightening, most of the signs in the park still read "Pinnacles National Monument", but we knew where we were!).

As compared with many of the big name National Parks, Pinnacles is small, unknown, and uncrowded. Even though we visited on a summer Saturday, there were campsites unfilled at night and parking spaces available until well into the morning. Try that at Yosemite!

The main attraction at Pinnacles are the California Condors, a most remarkable bird. We've visited the Pinnacles over a dozen times and have yet to actually see a condor, and we were no luckier this time. Part of the problem is that, in order to have the best chance of seeing a condor, you need to be in the highest and most remote parts of the park at sunrise or sundown, which is rather a challenge for us.

Pinnacles is also known for rock-climbing, for its pair of unusual Talus Caves, and for its superb star-gazing.

We took a hike to the Bear Gulch Cave, but it's still closed to protect the Townsend's Large Eared Bats, so we took the bypass trail and walked up to the reservoir and back. It's a beautiful walk, the perfect length and challenge level for a nine-year-old.

We were lucky to be able to take our walk early, as June 1st this year found the park already reaching temperatures of 104 degrees! We were on the trail by 8:45 AM and done by 11:00, and as I watched the late arrivals beginning their hikes, I was extremely grateful that we had chosen to walk as early as we could.

Happily, Pinnacles Campground features an (unheated) swimming pool, and so we spent mid-day and afternoon by the pool, enjoying the beauty of the park while keeping pleasantly cool. The camp store also sells ice cream sandwiches and bags of cooler ice, a service that we made ample use of.

The trails, the Bear Gulch Dam, the Visitor Centers and Ranger Stations, and many other of the park improvements were all built by the Civilian Conservation Corps. Eighty years later, their work is still holding up great; those "CCC Boys" did fine work!

There was no moon for our trip, and the extremely warm weather made it quite pleasant to sit outside by our campsite until nearly 11:00 PM, watching and enjoying the stars and matching the skies up with Google SkyMap.

My granddaughter has become quite a skilled Uno player, but I can still beat her at Yahtzee.

And our simple camping gear continues to hold up well; although it was a tight squeeze, all four of us fit into the car and we made it there and back quite nicely.

So, to what season does June 1st belong? The bats are still nesting and the cave is closed, which means it must still be spring. However, 104 degrees certainly counts as a summer day. School is still in session, so we had to leave on Friday after school (and slog through the rush hour traffic) and be back in time to prepare for Monday morning classes. But we were clearly in a summer frame of mind, lying by the pool and playing cards, so I'll choose to adopt the "Memorial Day through Labor Day" definition, and label this a wonderful mini-vacation to start our summertime.