Farewell to the Calaveras Pioneer Cabin tree

It was a big storm: Historic Pioneer Cabin Tree toppled in California storm

The historic Pioneer Cabin Tree, a former "drive-through" giant sequoia in Calaveras Big Trees State Park in Calaveras County, was felled in California's weekend storms.

The tree was hollowed out in the 1880s to allow tourists to pass through it, and even allowed cars, but more recently has hosted only hikers. It was part of the "Big Trees Trail" at the state park.

Jim Allday of Arnold is a volunteer at the park who was working there Sunday. He said the tree went down about 2 p.m. and "shattered" on impact.

I walked that trail in 2014, and if you ever get to the Sierras, Calaveras Big Trees State Park is a wonderful place.

But we humans should never have done that to that tree.

CHKDSK to the rescue!

Last Thursday evening, the 2 TB system disk on my primary Windows machine decided to fail, developing a number of unrecoverable media errors.

It wasn't immediately obvious that it was a hard disk failure: the machine would boot, run for a while (2 minutes? 5 minutes?) then it would lock up and stop running and eventually shut down.

I was able to bring up a document or two, but it wouldn't stay up long enough for me to bring up the Windows Event Log or any other sort of diagnostic tool.

Eventually I realized that the most likely cause was a media error on the system disk, and recognized that this wouldn't be a trivial task to recover from. Luckily, I have some other machines in the house that I can use, so I wasn't totally dead in the water.

Typically, when you develop bad blocks and other media errors on your hard disk, particularly to the point that the device itself can no longer quietly re-map those bad blocks to other sectors on the disk, you're looking at total data loss.

Which was unfortunate, because I didn't actually have a backup of this disk. Yes, I know: oops.

So over the weekend, I began trying to understand what was really wrong with the machine and what could be saved. I booted into the BIOS, and ran the various self-tests and diagnostics that come with this particular Dell machine. They confirmed that (a) the system disk was returning I/O errors, and (b) the rest of the computer seemed to be fine, from a hardware point of view.

It turns out that I had a second internal hard disk already installed in this computer, so I (with the power down of course) disconnected the main system disk from power and SATA, and popped my Windows 10 DVD into the DVD drive and installed Windows on the other hard disk.

Yay! Now I at least had a functioning computer again.

I had already placed an order for a replacement storage device, but I was intrigued by the fact that the machine had still been able to boot from the damaged system disk on Thursday: perhaps there was some faint hope that I could recover my data?

So I carefully re-connected the damaged disk to power and SATA, having made sure in the BIOS that it would NOT be the boot disk, and started back up again.

Then, from an Administrator CMD.EXE window, I fired up:

CHKDSK /F D:

And I went off, to take the dog for a walk.

And watched some football.

And my son came over and we talked about his life and how things are going.

And, eventually, after about 3 hours, the CHKDISK completed!

I was astonished; I hadn't really expected that.

So I plugged one of my external USB backup drives into the machine, and opened up the damaged drive in Windows Explorer, and lo and behold!

I was able to see files!

As quickly as I could, with my fingers crossed and my toes crossed, I selected all the most important files (4 years of digital pictures, various documents I'd written, some backup copies of data from my wife's machine, and so forth), and copied them all to the external drive.

Successfully.

Anyway, I'm mostly back up and running, although I've got a lot of software to re-install on this machine.

It's interesting that much of the stuff that I care about, is no longer the sort of problem that it would have been, even just 5 years ago: my email is all in the cloud, my Steam saved games are all in the cloud.

And, of course, my eight years of writings about my life and interests are in the cloud, as well: on this blog!

Other than some various files that I don't particularly want to save in the cloud (tax documents, etc.), the files that would have caused me the most trouble to lose were my VirtualBox images. I haven't yet had time to verify whether I successfully recovered those, but I'm hopeful.

In a way, completely starting over with a computer like this is a bit like moving house: it makes you think about what you really need to keep, and what you can throw away. There's a saying that, after you move into your new house, if should happen that there are some moving boxes that you haven't opened, and it's been 6 months, well, you can just throw those boxes away.

I'm not really that person; I find it very hard to throw things away.

But I understand the motivation behind the sentiment. Declutter!

Meanwhile, I'm sort of doing the same thing with my office environment. At my last job, I spent my entire 7 years in the same physical cubicle, never even moving once. That's really rare, at any company, and especially in the chaotic world of Silicon Valley high technology. Seven years was far and away the longest amount of time that I had ever spent in the same cubicle in my entire 35 year career; I think the second longest time I ever spent in a single cubicle is much closer to 2 years than to 7.

So, over those 7 years, I of course accumulated a vast amount of junk. I tried to throw most of it away, of course disposing of the company materials that had languished on my disk and bookcase and in filing cabinets: pile after pile after pile of documents full of hard, anguished work, which I was convinced I might need to refer back to, "sometime".

Even so, I still brought home more than a dozen shopping bags. Many were full of books, my professional library that I've accumulated over the years. But there were pictures, children's art, coffee cups, backup eyeglasses, spare clothing and jackets and umbrellas and whatnot, and lots of odds and ends that didn't fit into any easy category but I couldn't just immediately throw away.

So here I am, entering the new year. I have piles of stuff in the garage, which I'll have to sort through, and decide which things to take to the new office. And I have piles of files on my recovery disk, which I'll have to sort through, and decide which things to retain on my new computer.

Check back with me in 6 months; I wonder if I'll have even opened them all, much less decided what to keep and what to (finally) throw away.

Change

Nobody is ever comfortable with change.

But, as John Lennon said,

Life is what happens to you while you're busy making other plans.

I confess I have the bad habit, when stressful news forces its way into my life, of closing my eyes and covering my ears and trying to pretend it isn't real. I fool myself; I think that if I simply re-double my efforts, refuse to acknowledge, be stubborn, that I can will away the change.

But change happens, and it's happening to me.

Sadly, over the holidays, I lost my "other mother" (I could never bring myself to call Kaye my mother-in-law; I've always hated that term).

Kaye had been ill for years, but she never let it conquer her. Every time I was with her she was a treasure: showering us with love, looking to the future, never complaining. Even as her cruel, cruel illness robbed her of mobility, trapping her in a tinier and tinier box, she was never bitter or harsh, but gentle and gracious as always.

It can be no easy thing for a mother to see her own child grow up, but when I showed up and took her daughter out of her life and into my own, moving us far, far, (too far) away, she never lashed out, nor showed any regret; instead, she welcomed me into her own life and gave me her love as if I was her own.

And, I suppose, at that point, and due to that, and from then on, I was.

If I could seek only one thing, it would be to incorporate her uncrushable spirit into my own life.

Here are the words it's too late to say now, because I can say them but you cannot hear them, but I say them anyway; let the world hear them.

Thank you, Kaye.

Thank you for having such a beautiful and wonderful daughter; thank you for letting me take her into my life; thank you for choosing to love me like you loved her; thank you for being brave; thank you for passing all your wonderful qualities on to your daughter; thank you for everything.

I meant to tell you all this; life happened while I was making those plans.

Meanwhile, other changes continue to happen. In a brutally short period of time we found out that we lost three neighbors: one next-door, one across the street, and one just a short ways away.

We weren't related to any of these folks, but we'd lived next door to them for 15 years and they were all wonderful, wonderful people as well: Harry, the Stanford football player turned mining engineer, Barry, the hotel manager who moved here from Michigan and had an unbelievable sense of humor, and Barbara the wonderfully-kind best friend of our close friend and neighbor Mark.

It's all a dirty, rotten shame, and none of it makes any sense.

But life goes on, and change continues. Just to keep me on my toes, there's still more change in my life: after seven wonderful years at my job, I got one of those "too good to refuse" offers, so I made the terribly hard decision to step away from a job I thought I'd never leave, and take the next step in my professional career.

I'll have more to say about that, some other time.

For now, I'm just dealing with change.

The gang

Perforce Server Development, January, 2017 (plus a couple alumni):

My math may be a bit off, but I think that's slightly more than 400 years of professional software development experience, all in one place.

Pretty great.

Atmospheric river ahead?

This forecast hasn't changed much in the last 36 hours.

It's still 72 hours away, but check this out:

By Friday night, another abrupt change in the forecast pattern is expected as deeper moisture begins to advect inland from the eastern Pacific. Light rain remains on track to spread inland late in the night into Saturday morning. During the day Saturday, PWAT values are forecast to increase as an atmospheric river (AR) sets aim at the central California coast. By Saturday night into Sunday, PWAT values are forecast to approach 1.50" and will likely result in widespread moderate to locally heavy rainfall. The AR is forecast to move very little from Saturday night through Sunday and will likely lead to 24 to 36 hours of steady rainfall across the region. At this time, it appears the focus for the heaviest precipitation will be south of San Francisco along the coastal ranges which have the potential to pick up more than 12.00" of rainfall through the weekend. Given the impressive amounts of precipitation recorded in the Santa Cruz and Santa Lucia Mountains from this past system (Tuesday into Wednesday morning) cannot rule out even greater rainfall totals exceeding 15.00" as the upcoming AR event has the potential to be much stronger. Meanwhile, many urban locations can expect anywhere from 2.00" to 5.00" of precipitation through the weekend. In addition, breezy to gusty winds are forecast from Saturday night into Sunday as surface low pressure approaches the coast and pushes inland to the north. This south/southeast flow will also help to enhance precipitation rates along the coastal ranges. Be sure to stay up to date with the latest information regarding this upcoming weekend`s storm system as we will continue to refine our forecast in the coming days.

5 inches of rainfall in urban areas in 3 days?

15 inches of rainfall in the mountains in 3 days?

Remember, Oakland CA averages only 24 inches of rainfall for the entire YEAR, so getting 20 percent of the yearly total in just 3 days is, what, astounding?

Check back here in a week for the outcome, unless I'm paddling around in a boat somewhere...

Topic matching and bitmaps

Some very old ideas remain new. Inverted bitmaps are a wonderful example.

In a very nice recent article, Tyler Treat surveys various search structures employed by message queueing systems to solve the Fast Topic Matching problem: Fast Topic Matching

The naive hashmap solution is generally a poor choice due to its prohibitively expensive lookup time. Inverted bitmaps offer a better solution. The standard implementation is reasonable if the search space is finite, small, and known a priori, especially if read latency is critical. The space-optimized version is a better choice for scalability, offering a good balance between read and write performance while keeping a small memory footprint. The trie is an even better choice, providing lower latency than the optimized inverted bitmap and consuming less memory. It’s particularly good if the subscription tree is sparse and topics are not known a priori. Lastly, the concurrent subscription trie is the best option if there is high concurrency and throughput matters. It offers similar performance to the trie but scales better. The only downside is an increase in implementation complexity.

Treat's survey is clear and well-illustrated, and I'm really pleased that he worked hard to evaluate the choices across many different metrics: not just search time and update time, but also memory usage, concurrency, etc.

But I was particularly pleased that he included Bitmap indexes in the survey, for they are a fascinating data structure, and they don't get the coverage they deserve, particularly for work that is now 45 years old.

Although the original Model 204 implementation was hand-coded in IBM 370 Assembly Language, there are now widely-available bitmap indexing libraries, including Concise bitmaps, EWAH bitmaps, and Roaring bitmaps.

One (small) disappointment I have with Treat's great article is that he doesn't really investigate these libraries, instead simply noting that

While inverted bitmaps allow for efficient lookups, they are quite wasteful for sparse sets, even when using highly compressed bitmaps like Roaring bitmaps.

While I have no reason to disbelieve him, it would have been great to see him explain that conclusion more deeply.

At any rate, since Model 204's bitmap indexes were what got me into the software field, some 35 years ago, it's great to see them still finding favor and fairly widespread usage, decades later.

Rogue One: a very short review

At their best, the Star Wars movies deliver story-telling in a very old-fashioned way, with emphasis on story, character, and emotion.

Rogue One: A Star Wars Story is an ideal demonstration of this.

It's a Star Wars movie, so of course there are gadgets and Sci-Fi crutches (androids! faster-than-light-travel! spaceships galore!)

But those elements are just sort of the scenery for a classic story of a group of ragamuffin wanderers who come together in a time of need and figure out how to work together as a team.

Moreover, there's something to the Star Wars style that works particularly well in Rogue One: our heroes are underdogs, scavengers and makers-do, swept up in a crisis not of their own making, trying to make the best decisions they can in situations of great pressure and uncertainty.

You know: life, as it's led.

Oh, and one last thing: if you go, be prepared to shed a tear or two.