Saturday, July 20, 2019

More trouble on the I-880 railroad overpass

Large Pothole Opens Up on I-880 in Oakland; Multiple Lanes Blocked

The pothole, over Jackson Street, opened up in the No. 3 lane on southbound I-880, the CHP said, but the Nos. 1 and 2 lanes also were closed for the repair work.

Lane number 3, in this section, is of course the only lane that the Port of Oakland container cargo trucks are allowed to use through that stretch of the highway.

They spent so much time and effort on the section of 880 in downtown Oakland where it crosses the Union Pacific Railroad Tracks. That was an *enormous* engineering project!

It was partly for seismic retrofit and partly for traffic issues; the southbound on-ramp from 5th & Oak onto southbound 880 is well-known as one of the most dangerous on-ramps in the entire California road system.

But that bridge (12 lanes wide in sections!!) is still very troubled.

Not only was there that giant hole in the southbound lanes yesterday, but the northbound section of the bridge is separating at the earthquake expansion joint near the apex of the bridge just by the Laney College parking lot, and 3 of the four northbound lanes have enormous steel plates covering the gap in the expansion joint.

I have no idea how or when this can be solved, but it's certainly a great illustration of just how hard it is to build adequate civil engineering structures, even when you try very very hard.

Of course, one of the huge problems in this area is that this is the only road that the Oakland port container cargo traffic can use, so it bears a *very* heavy burden of constant fully-loaded cargo traffic.

But still, that was all well-known when Caltrans built the bridge.

Anyone know if there's a Caltrans project page which covers the ongoing issues with this bridge?

Saturday, July 13, 2019

Fermat's Last Theorem

Somehow, I happened across a beat-up, weathered copy of Amir Aczel's Fermat's Last Theorem just before a long airplane trip, and so I happily accepted the gift.

Aczel has written a number of "popular mathematics" books; I think this was one of his first books.

Fermat's Last Theorem, of course, is a wonderful topic, and perhaps one of the best topics for a popular mathematics book, starting from its origin story in Pierre de Fermat's notebooks ("I have discovered a truly remarkable proof of this theorem which this margin is too small to contain") to the marvelous work of Andrew Wiles, who was a great source of quotations himself:

Perhaps I could best describe my experience of doing mathematics in terms of entering a dark mansion. You go into the first room and it's dark, completely dark. You stumble around, bumping into the furniture. Gradually, you learn where each piece of furniture is. And finally, after six months or so, you find the light switch and turn it on. Suddenly it's all illuminated and you can see exactly where you were. Then you enter the next dark room...

(Wiles's evocative quote is pretty much a spot-on description of writing computer software, too, by the way.)

Fermat's Last Theorem is only peripherally about Fermat's Last Theorem. The mathematics in this book stops, more or less, at the level that a strong high school student would encounter. Mostly, it is a sort of history-of-science book, about what the process of "doing mathematics" was like in the second half of the twentieth century.

Which is still a pretty interesting thing to read about.

And, really, the world needs more books about mathematics, of any sort, even if they are just books about mathematicians, rather than books about mathematics. It drives me absolutely crazy when I wander into a magazine rack in a store and I pick up a book full of Sudoku puzzles, and, emblazoned on the cover, it says "No mathematics skills needed!" as if that was something to be happy for. (And anyway they are so wrong: logic and deduction are absolutely mathematics skills. Oh well, what can you do?)

But as it turns out, I already knew most of the story of Fermat's Last Theorem, though perhaps I didn't know quite so much about the individual mathematicians at the heart of the story.

The most interesting part of the book for me, because it was a bit of history that I had somehow never learned, was the discussion of two Japanese mathematicians, Yutaka Taniyama, and Goro Shimura, who together formulated something called the Taniyama-Shimura conjecture when they were both just in their mid-20's; the details of the conjecture turned out to be the critical turning point in "cracking" Fermat's Last Theorem.

If none of this is any interest to you, or if you're not a history of mathematics sort, give Fermat's Last Theorem a pass. But if you think you might find it interesting, it's a fun little book, and perfect for a 6 hour cross-country plane flight!

Thursday, July 11, 2019

Variations in the behavior of the default Java maximum heap size

Since the last millennium, Java implementations have always had system-specific behavior for things such as the maximum amount of memory your Java program can use, also known as the "maximum java heap".

Many (most?) JVM implementations support the -Xmx flag, which allows you to specify the maximum heap size.

But if you don't set that flag, what maximum heap size do you get?

Implementations vary widely in this case.

The most commonly-used JVM implementation is the Oracle JDK, whose behavior is documented here: https://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/jrdocs/refman/optionX.html

In particular, see "Table 2-6 Default Maximum Heap Sizes", which shows that the actual value will

  • depend on your operating system platform,
  • on the choice of a 32-bit or 64-bit JVM,
  • and also on the amount of RAM on the machine,
  • BUT will never exceed 2GB.

However, the OpenJDK, (https://openjdk.java.net/faq/) behaves differently.

With some experimentation, you will find that the Default Maximum Heap Size on an OpenJDK JVM also depends on various variables, but is NOT hard-limited to 2GB, and instead appears to simply use 25% of the RAM on your machine!

I discovered this when I was trying to figure out why we were experiencing extreme memory pressure on a fleet of machines that I manage. It turns out that our administrators had moved us from OracleJDK to OpenJDK on those machines, and our workloads had a bunch of places where we were NOT specifying -Xmx.

For those workloads, each time we spawned a JVM without specifying the maximum heap size, it had (quietly) changed from using up to 2GB of memory, to using up to 25 % of the system memory, and since we routinely spawned 2 such JVMs, half of our system memory was devoted to those 2 JVMs! Since these were quite large machines, having typically 48GB to 64GB of memory, this meant that a very large memory usage change had occurred: instead of 2 JVMs consuming 4GB of RAM total, the 2 JVMs were now consuming 24-32GB of RAM total!

Big machine or small machine, 25% is a lot! Since our machines routinely run thousands of processes, it took me far too long to notice these two processes quietly occupying half the system ram.

So the overall impact was subtle and it took me a LONG time to notice it (many months, sadly), because all that really happened right away was that the workloads ran more slowly, and the machine performance seemed quite poor, and it took me far too long to understand that the reason was that 2 of the 1000+ processes on the machine were suddenly taking half of the memory on the machine.

Unfortunately, this particular behavior does not seem to be documented anywhere (at least, I haven't found the "openjdk documentation for the -Xmx flag"), although since it is open source you can always "use the source, Luke": https://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/87ee5ee27509/src/share/vm/runtime/arguments.cpp#l1655

I estimate that I lost, over all of 2019 so far, about 1.5 weeks of my life to this particular detail.

But, happily, I finally figured it out!

Monday, July 1, 2019

Up, up and away

SF’s Brand New Gondola Opens July 1st

One of the most fun ways to get to the Salesforce Rooftop Park isn’t via a boring old escalator or elevator, but by a brand new downtown gondola system with a 20-passenger glass cabin which will connect the Mission Square (the new plaza of the Salesforce Tower at Mission and Fremont) to the rooftop park.

Also,

Salesforce Park’s grand reopening will be a subdued affair

San Francisco’s new transit structure, which shuttered six weeks after opening following the discovery of two cracked steel beams in September, will reopen to the public July 1. And while bus service won’t recommence until mid- to late-July, the park level, arguably the public’s preferred portion of the $2.2-billion behemoth, will once again be accessible.

But unlike the first opening, the gondola, which whisks people up two stories from the corner of Mission and Fremont, will be in operation.

Saturday, June 29, 2019

Yes, I just keep looking at this picture

It's pretty much the perfect picture.

Few things are as abstract and complex as finance

In America, retirement support is roughly divided into two parts:

  • Pretty much everybody has access to the Social Security system, which provides basic support when you get old or if you become disabled and cannot work. Related to Social Security is Medicare, with similar goals and restrictions.
  • A substantial, but not complete, section of the population also has access to individual retirement accounts, which are intended to provide the bulk of your retirement support.

Put more briefly, retirement support is pretty much up to you.

Your primary tool for providing for your own retirement is, usually, your company's 401K plan, which gives you tax-advantaged access to investments, and can be as good or bad as your company wants it to be.

Some companies have very good 401K plans (I'm lucky enough to work at one such company), while others have pretty bad ones, frankly. Your only choices in this area are to try to encourage your company to have the best plan it can, and to consider your company's plan as part of your considerations when you choose where to work.

But then, once you have access to a company-provided 401K plan, you still have lots of work to do, as modern 401K plans are fiendishly complicated.

And they, periodically, send you notices like this:

The new investment options are collective investment trusts (CITs) or “commingled pools.” The commingled pools will offer you similar investment strategy and risk as the mutual funds they replaced, but the expenses will be lower.

Like mutual funds, a CIT combines the money of many investors who own a share of the pool and/or trust. A fund manager invests assets on behalf of all shareholders in accordance with the pool’s stated investment objectives.

Unlike a mutual fund, a CIT is available to investors only through their workplace savings plans. Because they are not publicly traded, some information (e.g., ticker symbols, CUSIP numbers, and Morningstar ratings) is not available. A CIT is not registered with the Securities and Exchange Commission (SEC). It is generally governed by state banking laws and by federal agencies, such as the Internal Revenue Service and the Department of Labor.

Sheesh.

Maybe you go somewhere else to learn about CITs, and you find a little bit more information:

CITs were first introduced in 1927. Early versions of CITs required investor purchases and withdrawals to be processed manually and were valued infrequently, typically only once per calendar quarter, providing investors little access to portfolio and performance data. For this reason, the early adopters of CITs were defined benefit plans.

But starting in 2000, CITs began operating in ways that many believed were more comparable to mutual funds, providing daily valuation and standardized transaction processing, which greatly increased adoption by defined contribution plans. Then, in 2006, the Pension Protection Act provided for a new default investment election for unallocated 401(k) participant assets. These Qualified Default Investment Alternatives (QDIAs) include certain types of “approved” investment strategies and may take the form of managed accounts, target risk funds and target date funds. Many target date funds—the most widely adopted QDIA—are implemented as CITs, and as their assets have grown, so have the assets of CITs generally. Recently, CIT coverage by database vendors such as Morningstar has increased as well, providing additional transparency and reporting capabilities.

CITs have become a popular alternative to mutual funds within qualified retirement plans. Since 2012, CIT use has grown by 56% within DC plans, while the usage of mutual funds has decreased1—a trend that we expect to continue.

Basically, the fact that my company's plan now offers me new investment options which have lower expenses is a Good Thing, but every time I try to learn enough about finance to understand what the heck all these notices and documents are trying to tell me, I feel overwhelmed by the abstraction and complexity of it all.

And, as a mathematician and software engineer, I'm actually pretty good with abstraction and complexity, I think.

Oh, well, on we go.

Tuesday, June 25, 2019

How to spend your June vacation

You're going to be tired after your very long airplane journey.

So you should have some Hand-rolled ice cream.

Or maybe have a cool drink on the patio at Les Crepes

Or even make a day trip to Charlottesville to take in the mountain air

Then, once everything is all set up

And you've had your rehearsal

Then it'll be time to put on your fancy clothes

(Looking good!)

And attend the real ceremony

(Even if your granddaughter finds part of it a little boring)

The end of the ceremony will involve handfasting!

And everyone will be cheering!

And then it's time for the party!

Which means spending time with family

Family, new and old

Three generations!

And spending time with friends

And other traditions, such as the father-daughter dance

And the second line parasol procession

Oh, a wonderful vacation indeed

Tuesday, June 11, 2019

Google have lost my old pictures, I think

It appears that, whatever Google did recently, they have discarded my old pictures that I posted on my blog.

Whatever I did to put my pictures on my blog, they went to something like:

https://lh6.googleusercontent.com/-iqY0yw3vYj8/UgRf2jYNj8I/AAAAAAAACH0/ijmLIzswgNU/w426-h320/IMG_1253.JPG

But those images just fail now.

Sigh.

Live by the cloud, die by the cloud.

Sunday, June 9, 2019

Some of this, some of that

The days are long and my mind wanders.

  • An update on Sunday’s service disruption
    Google’s engineering teams detected the issue within seconds, but diagnosis and correction took far longer than our target of a few minutes. Once alerted, engineering teams quickly identified the cause of the network congestion, but the same network congestion which was creating service degradation also slowed the engineering teams’ ability to restore the correct configurations, prolonging the outage. The Google teams were keenly aware that every minute which passed represented another minute of user impact, and brought on additional help to parallelize restoration efforts.
  • Our not-so-magic journey scaling low latency, multi-region services on AWS
    CQRS is seriously good. By isolating out concerns and clearly defining which use cases you want to optimise for, it becomes possible to invest the effort in the most mission-critical services without having to drag along other functionality at the same time.
  • CQRS
    The change that CQRS introduces is to split that conceptual model into separate models for update and display, which it refers to as Command and Query respectively following the vocabulary of CommandQuerySeparation. The rationale is that for many problems, particularly in more complicated domains, having the same conceptual model for commands and queries leads to a more complex model that does neither well.
  • The Labor Movement’s Newest Warriors: Grad Students
    Hannah Kim and Natalia Piland are not your typical labor organizers. Kim, 23, has a bleached mullet, and when we met at a cafe near campus last Friday, she was wearing baggy track pants and chunky dad shoes. Piland, 29, was wearing all black, other than an iridescent fanny pack. Both of them are graduate students at the University of Chicago.

    It’s the final week of classes, but the two women have not been consumed with schoolwork. Instead, they’ve been busy organizing their peers to fight for better work conditions: On Monday, many UChicago graduate students participated in a three-day walkout, refusing to teach or grade papers.

    “What is a way for graduates to actually have power and to actually be able to push what we want our work place to look like?” Piland said. “The union is the only way that seems feasible.”

    These women, both members of Graduate Students United at UChicago, are among the new faces of unionization in America. They’re organizing what were once stable, middle class professions, which have seen wages and benefits erode precisely as positions opened up to women and minority candidates.

  • Learning Chess at 40
    Denise Park, the director of research at the University of Texas’ Center for Vital Longevity, described what was happening to me in unsettling terms. “As you get older, you actually see clear degradation of the brain, even in healthy people. Your frontal cortex gets smaller, your hippocampus—the seat of the memory—shrinks.” My brain volume is atrophying annually, my cortical thickness dropping some 0.5 percent a year.

    Where my daughter’s brain was hungrily forming new neural connections, mine could probably have a used a few new ones. “You don’t want to be pruning synaptic connections, you want to be growing them,” Park told me. My daughter’s brain was trying to efficiently tame the chaos. “For older adults,” Park said, “there’s not nearly enough chaos.”

    Back at the board, there seemed to be plenty of chaos. For one, my daughter tended to gaily hum as she contemplated her moves. Strictly Verboten in a tournament setting, but I did not want to let her think it was affecting me—and it certainly wasn’t as bad as the frenetic trash talking of Washington Square Park chess hustlers. It was the sense of effortlessness that got to me. Where I would carefully ponder the board, she would sweep in with lightning moves. Where I would carefully stick to the scripts I had been taught—“a knight on the rim is dim”—she seemed to be making things up. After what seemed a particularly disastrous move, I would try to play coach for a moment, and ask: Are you sure that’s what you want to do? She would shrug. I would feel a momentary shiver of pity and frustration; “it’s not sticking,” I would think. And then she would deliver some punishing pin on the Queen, or a deft back rank attack I had somehow overlooked. When I made a move, she would often crow: “I knew you were going to do that.”

  • Open Source Game Clones
    This site tries to gather open-source remakes of great old games in one place. If you think that something is missing from the list - please go to our GitHub repository and create an issue or even a pull request!

    Since all these projects are open-source you can help them and make this world a better place. Or at least you can play something to appreciate the effort people put in them.

  • For Remote Communities In Scotland's Outer Hebrides, Mobile Libraries Are A Lifeline
    In Stornoway, the biggest town in Scotland's Outer Hebrides islands, a yellow van sits on a narrow, one-way street. The Gaelic word leabharlann is painted on the front, back and sides, with its English translation, "library," on the front and sides.

    Driver Iain Mackenzie has loaded his books in the van, organized his customers' orders and is preparing for his last run of the week on the island of Lewis and Harris. The 16-year-old van runs three days a week, covering more than 800 miles of rugged roads to deliver books to more than 800 residents.

  • Strawberry farmers open homes for country's sole village of books
    Until recently, the village, just a few kilometres from the picturesque hill stations of Mahabaleshwar and Panchgani, was known for growing the best strawberries in the country. Now, it is a unique ‘village of books’ with 25 villagers having given up a part of their homes to set up open libraries.

    The idea was inspired by the Welsh village of Hay-on-Wye, which is informally known as the ‘town of books’, and has scores of second-hand and antiquarian bookstores. But the Maharashtra government has made the concept their own and expanded its scope. Its Marathi language department’s experts meticulously put together a collection of over 30,000 books organised under various genres. These were then distributed among the home libraries, as well as public places like temples and schools. Each home is allotted books pertaining to one genre and identified with street signs and wall paintings.

Saturday, June 8, 2019

Wonder Valley: a very short review

Reading Ivy Pochoda's Wonder Valley is like ..., is like ..., is like what?

It's like being at a neighborhood 4th of July block party on a hot sweltering night, and somebody hands you a string of firecrackers and you look down at them and you see that they're already lit, and you realize: this is utterly fascinating, but I think it's about to blow up in my hands!

Wonder Valley is so startling, so vivid, so hard-core, that it may have been a long time since you read a book like this.

I found it to be simultaneously hilarious and heart-breaking, glorious and tragic all at once.

Pochoda's work reminds me of the best of Carl Hiaasen or Elmore Leonard in so many ways: larger than life characters, mixtures of all walks of life, gritty realism, and a marvelous sense of place.

Wonder Valley's main action centers around a varied cast of characters (desert rats, hippies, street preachers, entertainment executives suffering mid-life crises, failed college athletes, ex-cons, loners on the run) who end up in and around a cult compound in the Southern California desert, and then follows the light trails of these sparkling souls as they bounce around the Southern California scene following the events at the compound, illuminating the past in a blistering 48 hours of non-stop action.

What really makes the book work is that Pochoda totally groks desert rats.

Britt had met Cassidy and Gideon that morning at the Joshua Tree farmers' market where they were selling chickens inexpertly sealed in plastic. While they rambled on about the beauty of the soul and the health of the spirit to their customers, the birds' blood leaked over their forearms, running down their lariats and beads.

They were both the sort of dirty tan that comes from too much time in the desert, like the sand had worked its way into their skin. Their hair was long, dreaded in places with beads that often vanished in the tangled mess. Cassidy wore two necklaces -- one with a large feather, the other with a tooth. Gideon had a bird claw on a leather braid. Life is beautiful even in death, he'd said when he caught Britt looking at it.

And, later, as others arrive:

They drove north. The road rose and fell. The little cabins were spread out at almost regular intervals just out of sight of one another. Some had been duded up, fenced in and expanded, turned into compounds with jury-rigged satellites and dirt yards filled with old pickups and rusted trailers. But many sat empty, their windows boarded up or missing.

"Would you look at that?" Sam said as they passed a cinder-block cabin, its doors and windows gaping holes.

"Yeah," Blake said, "I'm looking."

There was nothing special about the jackrabbit homestead the Samoan settled on except that it was unremarkable -- a pale cinder-block structure with plywood for windows and a rusted chain-link fence. The interior was a single room with a battered mattress and a good-for-nothing bed frame. A black-and-white TV lay on its side, the screen spidered, the rabbit ears bent in on themselves.

I love "jackrabbit homestead," and I really love the way Pochoda quietly ties it together with the "rabbit ears" of the old television. Somehow I had never heard the phrase "jackrabbit homestead" before, but of course it's well-known: Jackrabbit Homestead: Artists, Off-Roaders, and the American Dream Writ Miniature:

One of the many land acts designed to dispose of "useless" federal lands from the public domain, the Small Tract Act authorized the lease of up to five acres of public land for recreational purpose or use as a home, cabin, camp, health convalescent, or business site to able-bodied U.S. citizens. If the applicant made the necessary improvements to his or her claim by constructing a small dwelling within three years of the lease, the applicant could file for a patent--the federal government's form of a deed--after purchasing the parcel for the appraised price (on average $10 to $20 an acre) at the regional land office. This highly popular mid-century homestead movement reflects the quintessential American desire to claim territory and own a piece of the land even if the property in question is virtually "worthless" from an economic perspective.

(Oh, man, check out the pictures at that link!)

But back to Wonder Valley. Marvelous though its sense of place is, Wonder Valley is mostly interested in how people become to be who they are, and how they decide where they want to be, and, above all, how one thing sometimes just leads to another:

It was a maddening tangle, a complex and horrifying puzzle, figuring out when she had first put her foot wrong, and which foot, and where. And how that led to all the things that put her in that car with Andy and that car rolling down the ravine and killing him or hurting him or nothing at all. Because there must have been a first error, something that set the whole disaster in motion.

You can find this moment in every blown match, that split-second what-if that might have sent the ship sailing in the right direction -- the return that didn't sail long, the second serve that clipped the line, the volley that skimmed the net, the look you didn't give your coach, the cheer from the stands you ignored. All of it discoverable, each wrong decision easily pinpointed, addressed, advised against on the next go-round. If you hadn't done that, you would have avoided the entire landslide. The descent into chaos. So it had to be there, that initial mistake.

Or, really, is that how it works? Is life just like training for your tennis match (Pochoda's bio lists her background as "a former professional squash player," surely not a common avenue to Next Great Novelist)?

I think not, and I think Pochoda knows not, herself:

"I promise you the first time I saw him was when he was running down the freeway. Then I got out of my car and chased him."

"Why?"

The same damn question he's been trying to answer for twenty-four hours. But the beer is helping. "Because I hate my job. Because I should have gone running over the weekend, but instead I drank too many beers and pretended to be interested in my daughter's friends' parents and their school fund-raiser. Because I have to attend the damn fund-raiser to make up for the fact that my wife and I are in the bottom tier of contributors to a school I already pay too much for my daughter to attend."

...

Tony's finally able to admit to himself that he had been unable to catch up to the runner. He didn't have the strength for that final acceleration, that last kick. Or maybe he did but he was too complacent, too happy to trail behind the guy instead of reaching him. Coasting, almost. Doing just enough but failing at the final hurdle. He still runs regularly but he's losing ground, letting go of the college runner he used to be. He's growing solid. Soon he'll be grounded like the commuters stuck in their cars he'd left behind on the 110.

"He's growing solid."

That's it, right there. It's that moment when what could be, what was dreamt, what was envisioned, shimmers and fades, and what is left is what is. Jackrabbit homesteads in Wonder Valley.

Pochoda's talent is immense and her vision is vivid. But most importantly, her stories and characters ring true.

I hope she writes many more books, each as wonderful in its own way as Wonder Valley

Sunday, June 2, 2019

Fifty years of Arcosanti

Here's a pretty nice article, with lots of links to chase, surveying the highs and lows of the first 50 years of the Arcosanti project: In Arizona, the Arcosanti desert utopia has become a kind of commune

In the sixties and seventies, he noted, it became somewhat common for people to see innovative or alternative thinkers as gurus or messiahs (though Soleri rejected those monikers). As late as the 90s, Bell said, people settled in Arcosanti just to be close to Soleri.

Today, that dynamic has changed.

"I think something that we understand as a generation, as millennials, is that everything is collaborative," Bell said. "Nothing gets done by one single person."

He added that Arcosanti residents are well aware that no single city or development solve every problem within our society.

"Soleri gave us a map and we followed that map to the edges," Bell said. "I don't think Arcosanti is the city of the future. ... There are a lot of places doing really innovative work."

Saturday, June 1, 2019

The Berlin Trilogy: a very short review

Somehow I hadn't heard of the Scottish writer Philip Kerr, who died last year at the far too young age of 62, but when I stumbled across an obituary I decided I'd give his books a try, and picked up Berlin Noir: March Violets; The Pale Criminal; A German Requiem.

Now, there's hard-boiled, and there's hard-boiled. My taste in hard-boiled has mostly tended toward American authors, such as Elmore Leonard, Michael Connolly, James Lee Burke, etc., for I was born a bit too late to ever really pick up Hammett or Spillane or Chandler.

But I've also enjoyed the European style of hard-boiled writing, such as Stieg Larsson's astonishing Millenium trilogy.

So what are we to make of the audacity of a Scottish writer setting his German protagonist squarely in the middle of WWII Berlin?

On the one hand, how much more noir can you possibly get? There's no shortage of villains, no lacking for drama, no need to dream up conditions any harsher or bleaker than these. And Kerr delivers the action! These books are roller-coaster rides of twist-a-minute thriller investigations, punctuated frequently with sex, violence, and way too many all night sessions stumbling from one back-alley location to the next.

On the other hand, what I found most compelling about The Berlin Triology was how tremendously atmospheric they were. Kerr really immerses you in a very specific place and time, and the effect is very powerful. Some of my favorite parts of the books were the places where Kerr manages to artfully take a simple description and slap you across the face in just the right way, letting you see things which are simultaneously beautiful and still cruel.

For example, this visit to a fairly upscale Berlin city square quickly darkens, vividly emphasizing the rapid militarization that was changing the entire literal landscape of the society:

The houses on Herbertstrasse, in any other city but Berlin, would each have been surrounded by a couple of hectares of shrub-lined lawn. But as it was they filled their individual plots of land with little or no space for grass and paving. Some of them were no more than the front-gate's width from the sidewalk. Architecturally they were a mixture of styles, ranging from the Palladian to the neo-Gothic, the Welhelmine and some that were so vernacular as to be impossible to describe. Judged as a whole, Herbertstrasse was like an assemblage of old field-marshals and grand-admirals in full-dress-uniforms obliged to sit on extremely small and inadequate camp stools.

Or this marvelous description of post-war Vienna, that neatly tucks in an acknowledgement of how thoroughly the war had affected the city, even to the extent of the urban plants and animals:

The morning was bright, clear and chilly. Crossing the park in front of the new town hall on my way to the Inner City, a couple of squirrels bounded up to say hello and check me out for breakfast. But before they got close they caught the cloud on my face and the smell of fear on my socks. Probably they even made a mental note of the heavy shape in my coat pocket and thought better of it. Smart little creatures. After all, it wasn't so very long since small mammals were being shot and eaten in Vienna. So they hurried on their way, like living scribbles of fur.

If you're after joy, light, and hope, avoid The Berlin Trilogy, for there is none of that to be found here.

But if you're after a vivid exploration of what it was like, and you're far too young to have been there, The Berlin Trilogy delivers over and over again.

Kerr wrote these three masterpieces in a whirlwind 18 month period from mid-1989 through early 1991, then apparently took a 15 year break, before returning to pick up Bernie Gunther's story again with 8 more books written after 2006, the last one published posthumously.

I'm not sure if I'm going to follow the rest of Gunther's adventures. Maybe I'll take a 15 year break. Lots of other stuff on the shelf, after all.

But I'm not disappointed that I read these.

Saturday, May 25, 2019

One Fine Day in Alameda

Pretty good article: One Fine Day in Alameda

With summer almost upon us, it’s time to start planning seaside outings. But you don’t have to head to Marin, the Peninsula, or Santa Cruz for an escape to the beach. The East Bay’s own Alameda features sandy shores and outdoor activities galore.

Not a clunker in their list of suggestions, I'd say; well done!

Wednesday, May 22, 2019

Up, up, and away

Let's try this whole thing all over again:

  • Salesforce Transit Center
    Thornton Tomasetti is the structural engineer-of-record and is also providing sustainability services for Salesforce Transit Center, a new transit terminal topped by a 5.4-acre public park. The project, awarded through a design and development competition, promises to transform the neighborhood and centralize transportation for the region, as part of an overall redevelopment plan for the city of San Francisco.
  • Contractor Gets Green Light to Fix Two Fractured Girders at Salesforce Transit Center
    The third-floor Fremont Street tapered plate girders help support the park directly above and the second-floor bus level—via a hanger at the midspan that thickens the web to 4 in. and slots through the bottom flange—directly below the girder level.

    ...

    As described by Bruce Gibbons, the TT managing principal in charge of the transit hub, the bolted fix would only repair the compromised region, at the 8-ft-deep midspan of each 80-ft-long girder, by bypassing the fractured area. The double splint consists of a sandwich of two Grade 70 steel plates, 2 in. thick, and a total of 20 in. wide, on both sides of the web. The bent plates will be a total of 14 ft long, centered at the girder's midspan. There will be 224 bolts, said Turchon. In addition, 8-in.-tall plates will be bolted to each girder’s vertical stiffener.

  • Update on Construction and temporary closure of the Salesforce Transit Center
    Actions taken since last Board meeting:
    • Fremont Street repair designed and material being procured; MTC Peer Review Panel (PRP) in concurrence
    • Same detail proposed at First Street with PRP concurrence
    • Load Shedding analysis completed and submitted to the PRP
    • Bolts in load path received Ultrasonic/Non-Destructive Testing; no damage detected
    • Initiated search for other areas susceptible to brittle fracture. PRP actively participating in review of the effort, supported by Ruby & Associates
  • Repair of Fractured Girders Complete at Shuttered Salesforce Transit Center
    To date, crews have removed all shoring from both sets of third-floor girders that span 87 ft and support both a public rooftop park above and hang the second floor bus level. All traffic lanes are now open during the day. Night street closures will continue throughout May to restore lights, MUNI overhead lines and to reinstall ceiling panels.

    Recommissioning of the 4.5-block-long facility will continue through the end of this month. The quality assurance process includes retesting and re-inspecting fire and life safety systems throughout the facility and retesting the building’s mechanical and electrical systems, reports TJPA.

  • Repairs on cracked beams finished at shuttered Transbay Transit Terminal
    The people behind the grinding effort to reopen the $2.2 billion Transbay Transit Center after its emergency closure last September is holding their breath and waiting to see what a peer review team will say about the inspections of the facility when it meets May 22.

    That’s because the say-so of the reviewers may be the last obstacle toward setting a date to finally reopen the building and let bus traffic roll once again.

    At Thursday’s meeting of the Transbay Joint Powers Authority (TJPA) board, Executive Director Mark Zabaneh said that it will take about a month from the time the review panel gives the okay for the building to open.

    “We would need about a four week period to get buses inside,” said Zabaneh, citing a need to get drivers used to navigating the facility once again.

Tuesday, May 21, 2019

I entirely concur ...

... with this philosophy: 60 Days of Light

The phrase “60 Days of Light” first entered my vocabulary a few years ago in St. Petersburg, Russia (I don’t recall the Russian expression, but thats the rough translation). They are so far north they have almost continuous day light during this period, the sun barely sets, and when it does finally dip below the horizon, it is well after midnight. From mid-May (now) until mid-July, the darkest it gets is between sunset and sunrise look like dusk; Even around 2 a.m., the effect of sunset is muted and short-lived.

Love that graph of St. Petersburg!

(Of course, we have recently been spending the first of our 60 days with this: the second half of May 2019 will be the wettest in CA history for many locations., but still.)

Sunday, May 19, 2019

Responses to the letters to the editor in the local paper

To the woman who wrote A Heartfelt Message of Gratitude:

Thank you for writing, and for sharing your experience.

You are very right; it's just stuff.

Various things I've been reading

A little bit of this, a little bit of that.
  • Privacy Rights and Data Collection in a Digital Economy (Senate hearing)
    But there is a second, more fundamental sense of the word privacy, one which until recently was so common and unremarkable that it would have made no sense to try to describe it.

    That is the idea that there exists a sphere of life that should remain outside public scrutiny, in which we can be sure that our words, actions, thoughts and feelings are not being indelibly recorded. This includes not only intimate spaces like the home, but also the many semi-private places where people gather and engage with one another in the common activities of daily life—the workplace, church, club or union hall. As these interactions move online, our privacy in this deeper sense withers away.

    Until recently, even people living in a police state could count on the fact that the authorities didn’t have enough equipment or manpower to observe everyone, everywhere, and so enjoyed more freedom from monitoring than we do living in a free society today. [Note: The record for intensive surveillance in the pre-internet age likely belongs to East Germany, where by some estimates one in seven people was an informant.].

    A characteristic of this new world of ambient surveillance is that we cannot opt out of it, any more than we might opt out of automobile culture by refusing to drive. However sincere our commitment to walking, the world around us would still be a world built for cars. We would still have to contend with roads, traffic jams, air pollution, and run the risk of being hit by a bus.

    Similarly, while it is possible in principle to throw one’s laptop into the sea and renounce all technology, it is no longer be possible to opt out of a surveillance society.

    When we talk about privacy in this second, more basic sense, the giant tech companies are not the guardians of privacy, but its gravediggers.

  • Solving Puzzles to Protect the Cloud: CTO Taher Elgamal on His Role at Salesforce and the Future of Cryptography
    Elgamal sees interesting challenges emerging in the next decade or so as quantum computing becomes a reality. He’s passionate about agility in cryptography, noting that, currently, when changes need to be made because an implementation has been shown to have weaknesses, it causes a big slowdown for security engineers. We can’t wait ten years, Elgamal says, to start the effort to protect against new technologies.
  • Amazon’s Away Teams laid bare: How AWS's hivemind of engineers develop and maintain their internal tech
    The Away Team model and generally easy access to source code means that investment can easily cross service boundaries to enhance the power of the entire system of services. Teams with a vision for making their own service more powerful by improving other services are free to execute.
  • OPP (Other People's Problems)
    If it’s your job (or the job of someone who reports to you), great. Go to it! Tend your own garden first. Make systems that are as robust as you believe systems should be. Follow processes that you believe are effective and efficient. If you are not leading by example, you have to start there. Stop reading now and go fix the things!

    If there’s no clear owner, do you know why? Is it just because no one has gotten around to doing it, or has the organization specifically decided not to do it? If no one’s gotten around to doing it, can you do it yourself? Can your org do it, just within your org?

    If it’s someone else’s job, how much does it affect your day to day life? Does it bother you because they’re doing it wrong, or does it actually, really, significantly make it harder for you to do your job? Really? That significantly? There’s no work around at all? If it is not directly affecting your job, drop it!

  • Strong Opinions Loosely Held Might be the Worst Idea in Tech
    The idea of strong opinions, loosely held is that you can make bombastic statements, and everyone should implicitly assume that you’ll happily change your mind in a heartbeat if new data suggests you are wrong. It is supposed to lead to a collegial, competitive environment in which ideas get a vigorous defense, the best of them survive, and no-one gets their feelings hurt in the process.

    On a certain kind of team, where everyone shares that ethos, and there is very little power differential, this can work well. I’ve had the pleasure of working on teams like that, and it is all kinds of fun. When you have a handful of solid engineers that understand each other, and all of them feel free to say “you are wrong about X, that is absolutely insane, and I question your entire family structure if you believe that, clearly Y is the way to go”, and then you all happily grab lunch together (at Linguini’s), that’s a great feeling of camaraderie.

    Unfortunately, that ideal is seldom achieved.

    What really happens? The loudest, most bombastic engineer states their case with certainty, and that shuts down discussion. Other people either assume the loudmouth knows best, or don’t want to stick out their neck and risk criticism and shame. This is especially true if the loudmouth is senior, or there is any other power differential.

  • People + AI Guidebook: Explainability + Trust
    Key considerations for explaining AI systems:
    1. Help users calibrate their trust. Because AI products are based on statistics and probability, the user shouldn’t trust the system completely. Rather, based on system explanations, the user should know when to trust the system’s predictions and when to apply their own judgement.
    2. Optimize for understanding. In some cases, there may be no explicit, comprehensive explanation for the output of a complex algorithm. Even the developers of the AI may not know precisely how it works. In other cases, the reasoning behind a prediction may be knowable, but difficult to explain to users in terms they will understand.
    3. Manage influence on user decisions. AI systems often generate output that the user needs to act on. If, when, and how the system calculates and shows confidence levels can be critical in informing the user’s decision making and calibrating their trust.
  • Decision Tree Learning
    The Decision Tree Learning algorithm adopts a greedy divide-and-conquer strategy: always test the most important attribute first. This test divides the problem up into smaller subproblems that can then be solved recursively. By “most important attribute,” we mean the one that makes the most difference to the classification of an example. That way, we hope to get to the correct classification with a small number of tests, meaning that all paths in the tree will be short and the tree as a whole will be shallow.
  • Awesome decision tree research papers
    A curated list of decision, classification and regression tree research papers with implementations.
  • Video Game Workers See Power in a Union
    This is the first labor-related walkout in the video game industry, but it likely will not be the last. The sector has been under scrutiny for years over exploitative practices, including lack of job security, mass layoffs and “crunch.” Short for “crunch time,” crunch is an industry-wide practice that requires employees, especially developers, to put in extra, unpaid hours—making for 60 to 80-hour work weeks—to deliver a game by its release date. One of the most egregious examples of crunch came in October of 2018 when reports surfaced that employees of Rockstar Games were working 100-hour weeks to finish the game Red Dead Redemption 2.
  • This World of Ours
    The worst part about growing up is that the world becomes more constrained. As a child, it seems completely reasonable to build a spaceship out of bed sheets, firecrackers, and lawn furniture; as you get older, you realize that the S.S. Improbable will not take you to space, but instead a lonely killing field of fire, Child Protective Services, and awkward local news interviews, not necessarily in that order, but with everything showing up eventually. Security research is the continual process of discovering that your spaceship is a deathtrap.

Saturday, May 18, 2019

A nice collection of git links

Cue the line about how a master craftsman needs to thoroughly understand his tools, as we explore another fascinating series from the great Raymond Chen (After we first share one unrelated bonus link)

  • Learn to change history with git rebase!
    One of Git's core value-adds is the ability to edit history. Unlike version control systems that treat the history as a sacred record, in git we can change history to suit our needs. This gives us a lot of powerful tools and allows us to curate a good commit history in the same way we use refactoring to uphold good software design practices. These tools can be a little bit intimidating to the novice or even intermediate git user, but this guide will help to demystify the powerful git-rebase.
OK, that was a good warm-up. Now on to the main feature!
  • Stupid git commit-tree tricks, Part 1: Building a commit manually out of a tree
    I take a snapshot of what’s in our internal staging repo and push it to the public repo. All of the intermediate steps are squashed out, so that the public repo isn’t cluttered with noisy history.
  • Stupid git commit-tree tricks, Part 2: Building a merge commit manually out of a tree
    I want the commit to be a merge of win10-1507 and the changes specific to that branch. To do this, I use the commit-tree command, but provide multiple parents. The first parent is the previous commit for the branch, and the second parent is the incoming changes from its ancestor branch.
  • Stupid git commit-tree tricks, Part 3: Building a throwaway commit in order to perform a combined cherry-pick-squash
    Suppose you have a series of commits you want to cherry-pick and squash onto another branch.

    The traditional way of doing this is to cherry-pick the series of commits with the -n option so that they all stack on top of each other, and then perform a big git commit when you’re done. However, this mechanism churns the local hard drive with copies of each of the intermediate commits, and if there are merge conflicts, you may end up having to resolve the conflict in the same block of code over and over again.

  • Stupid git commit-tree tricks, Part 4: Changing a squash to a merge
    you could hard reset the master branch back to M1 and redo the merge. But that means you have to redo all the merge conflicts, and that may have been quite an ordeal. And if that was a large merge, then even in the absence of conflicts, you still have a lot of files being churned in your working directory.

    Much faster is simply to create the commit you want and reset to it.

  • Stupid git commit-tree tricks, Part 5: Squashing without git rebase
    Since all of the commits we want to squash are consecutive, we can do all this squashing by simply committing trees.

    ...

    The point is that we were able to rewrite a branch without touching any files in the working directory.

  • Stupid git commit-tree tricks, Part 6: Resetting by reusing an earlier tree
    This tree-based merge is the trick I referred to some time ago in the context of forcing a patch branch to look like a nop. In that diagram, we started with a commit A and cherry-picked a patch P so we could use it to patch the master branch. Meanwhile, we also want a nop to go into the feature branch. We did it with a git revert, but you can also do it in a no-touch way by committing trees.
  • Stupid git tricks: Combining two files into one while preserving line history
    For best results, your rename commit should be a pure rename. Resist the tempotation to edit the file’s contents at the same time you rename it. A pure rename ensure that git’s rename detection will find the match. If you edit the file in the same commit as the rename, then whether the rename is detected as such will depend on git’s “similar files” heuristic.¹ If you need to edit the file as well as rename it, do it in two separate commits: One for the rename, another for the edit.

    Wait, we didn’t use git commit-tree yet. What’s this doing in the Stupid git commit-tree tricks series?

    We’ll add commit-tree to the mix next time. Today was groundwork, but this is a handy technique to keep in your bag of tricks, even if you never get around to the commit-tree part.

  • Stupid git commit-tree tricks, Part 7: Combining more than two files into one while preserving line history, manual octopus merging
    The problem is that octopus merges work only if there are no conflicts. We’re going to have to build our own octopus merge.
    cat dairy fruits veggies | sort >food
    git rm dairy fruits veggies
    git add food
    git write-tree
    The git write-tree creates a tree from the index. It’s the tree that a git commit would create, but we don’t want to do a normal commit. This is the tree we want to commit, but we need to set custom parents, so we’ll ask git write-tree for the tree that would be committed, so we can build our custom commit.
Any series that culminates in the phrase "manual octopus merging" is epic.

Sunday, April 28, 2019

Responses to the Letters to the Editor in the local paper

To the Traceys, who who wrote in with updates and thanks about their son Austin:

Thank you for writing. I am so very, very sorry.

Mental health issues are, in my opinion, the greatest unsolved tragedy of my time here on earth.

Friday, April 26, 2019

Thunderstruck: a very short review

Erik Larson is deservedly famed for his The Devil in the White City, which I enjoyed quite a lot when I read it years ago.

But Larson has written a number of books in addition to The Devil in the White City, and recently I happened to read Thunderstruck.

Thunderstruck is set at the turn of the 19th century, from about 1895 through about 1910, and focuses primarily on the unusual and turbulent life of Guglielmo Marconi, the Irish-Italian inventor who, at a very young age, moved to London and invented a set of devices which could send and receive messages using radio waves, something which quickly became known as the "Wireless Telegraph".

As a narrative device, Larson spins Marconi's story together with another story, that of H.H. Crippen, a physician of sorts and a peddler of the sorts of homeopathic "cures" that were popular at that time.

Alternating back and forth between the two stories, Larson brings things to a climax with a certainly entertaining depiction of a heinous crime and its unraveling by Scotland Yard.

But, overall, I found myself rather unaffected, for a variety of reasons.

Most importantly, the two stories didn't really have anything to do with each other, except for the rather boring observation that Scotland Yard were able to make use of the Wireless Telegraph during their apprehension of the murderer.

Furthermore, neither of the main characters are all that gripping. Marconi certainly had a vivid time in the world spotlight, rushing here and there to demonstrate his invention, build a company to deliver it, and grow it into a worldwide success. But Marconi (at least in Larson's telling) was a quiet, private, almost reclusive man, with not much more than a string of failed relationships to add background and color to the story of his tireless work on refining and improving the wireless transmitters and receivers.

Crippen, meanwhile, is even less appealing, and Larson is forced to tell Crippen's story primarily through the stories of those he came in contact with: wives, girlfriends, business associates, etc.

The most interesting character in the book, I thought, was the completely fascinating John Nevil Maskelyne: magician, entertainer, skeptic, and general gadabout; the best part of the entire book, in my opinion, is the wonderful story of how Maskelyne attempts to disrupt one of Marconi's public exhibitions of wireless technology by commandeering a wireless transmitter of his own and beaming risque limericks into the on-stage receiver. And, we must not forget, Maskelyne is one of those rare creatures whose own book is still read and enjoyed now, 125 years after it was first published!

As he always does, Larson does a fine job of painting the picture of the time, and the book is pleasant enough to read.

But I guess I think he tried to force his material to carry more than its fair share of a story, and the result felt to me like one of those situations where the enormous build-up with which Thunderstruck arrived led to an unavoidable disappointment when its reality sunk in.

Thursday, April 25, 2019

I've been out of date with my reading ...

... but here's a little snapshot, anyway.

  • How the Boeing 737 Max Disaster Looks to a Software Developer (Free registration required)
    The flight management computer is a computer. What that means is that it’s not full of aluminum bits, cables, fuel lines, or all the other accoutrements of aviation. It’s full of lines of code. And that’s where things get dangerous.

    Those lines of code were no doubt created by people at the direction of managers. Neither such coders nor their managers are as in touch with the particular culture and mores of the aviation world as much as the people who are down on the factory floor, riveting wings on, designing control yokes, and fitting landing gears. Those people have decades of institutional memory about what has worked in the past and what has not worked. Software people do not.

    In the 737 Max, only one of the flight management computers is active at a time—either the pilot’s computer or the copilot’s computer. And the active computer takes inputs only from the sensors on its own side of the aircraft.

    When the two computers disagree, the solution for the humans in the cockpit is to look across the control panel to see what the other instruments are saying and then sort it out. In the Boeing system, the flight management computer does not “look across” at the other instruments. It believes only the instruments on its side. It doesn’t go old-school. It’s modern. It’s software.

  • TRITON Actor TTP Profile, Custom Attack Tools, Detections, and ATT&CK Mapping
    The TRITON intrusion is shrouded in mystery. There has been some public discussion surrounding the TRITON framework and its impact at the target site, yet little to no information has been shared on the tactics, techniques, and procedures (TTPs) related to the intrusion lifecycle, or how the attack made it deep enough to impact the industrial processes. The TRITON framework itself and the intrusion tools the actor used were built and deployed by humans, all of whom had observable human strategies, preferences, and conventions for the custom tooling of the intrusion operation. It is our goal to discuss these adversary methods and highlight exactly how the developer(s), operator(s) and others involved used custom tools in the intrusion.

    In this report we continue our research of the actor’s operations with a specific focus on a selection of custom information technology (IT) tools and tactics the threat actor leveraged during the early stages of the targeted attack lifecycle (Figure 1). The information in this report is derived from multiple TRITON-related incident responses carried out by FireEye Mandiant.

    Using the methodologies described in this post, FireEye Mandiant incident responders have uncovered additional intrusion activity from this threat actor – including new custom tool sets – at a second critical infrastructure facility. As such, we strongly encourage industrial control system (ICS) asset owners to leverage the indicators, TTPs, and detections included in this post to improve their defenses and hunt for related activity in their networks.

  • RIP Joe Armstrong (See also this twitter thread)

    (Yes, he was the man who invented Erlang. And did a surprising amount of other stuff that you never knew about.)

  • France’s Yellow Vests Challenge
    Over lunch at an Avenue Matignon café in Paris, I asked a literate media entrepreneur and political expert to explain the Yellow Vests mystery. Who are they, what do they want, who leads them? He started by offering a metaphor. The interior of France, some call it La France Profonde (think of Red States), is like a forest that has been progressively desiccated by climate change. Then gasoline was poured on the trees. Sooner or later a spark would set it ablaze.

    I’ve seen the desiccation, the emptying of France. In an October 2017 Monday Note titled The Compostelle Walk Into Another France, I recounted how, with daughter Marie, we walked through empty villages, no café, no bakery, no garage. Barking dogs roamed the streets during the day, the inhabitants having rushed to their distant jobs. I had had a similar impression years before, driving small country roads in Northern Burgundy, but walking the Compostelle (Camino de Santiago) made it more vivid.

    Over the past three or fours decades, La France Profonde has been slowly hollowed out as jobs moved out of the country or to larger urban areas. This desiccation isn’t unique to France, but gasoline was poured on the forest in the form of an accumulation of laws and regulations that exasperated the remaining France Profonde population, creating a schism between “real people” and “those people in Paris”.

  • Google I/O 2019
    This year’s developer festival will be held May 7-9 at the Shoreline Amphitheatre in Mountain View, CA
  • How Common Are Data Center Meltdowns?
    We all know about catastrophic headline-generating failures like AWS East-1 region falling apart or a major provider being down for a day or two. Then there are failures known only to those who care, like losing a major exchange point. However, I’m becoming more and more certain that the known failures are not even the tip of the iceberg - they seem to be the climber at the iceberg summit.
  • “Users want control” is a shoulder shrug
    There are some cases when a person really does want control. If the person wants to determine their own path, if having choice is itself a personal goal, then you need control. That’s a goal about who you are not just what you get. It’s worth identifying moments when this is important. But if a person does not pay attention to something then that person probably does not identify with the topic and is not seeking control over it. “Privacy advocates” pay attention to privacy, and attain a sense of identity from the very act of being mindful of their own privacy. Everyone else does not.
  • Tesla Full Self Driving ASIC
    Overall, it’s a nice design. They have adopted a conservative process node and frequency. They have taken a pretty much standard approach to inference by mostly leaning on a fairly large multiple/add array. In this case a 96×96 unit. What’s particularly interesting to me what’s around the multiply/add array and their approach to extracting good performance from a conventional low-cost memory subsystem. I was also interested in their use of two redundant inference chips per car with each exchanging results each iteration to detect errors before passing the final plan (the actuator instructions) to a safety system for validation before the commands are sent to the actuators. Performance and price/performance look quite good.
  • Highlights from Git 2.21
    An important optimization for Git servers is that the format for transmitted objects is the same as the heavily-compressed on-disk packfiles. That means that in many cases, Git can serve repositories to clients by simply copying bytes off disk without having to inflate individual objects.

    But sometimes this assumption breaks down. Objects on disk may be stored as “deltas” against one another. When two versions of a file have similar content, we might store the full contents of one version (the “base”), but only the differences against the base for the other version. This creates a complication when serving a fetch. If object A is stored as a delta against object B, we can only send the client our on-disk version of A if we are also sending them B (or if we know they already have B). Otherwise, we have to reconstruct the full contents of A and re-compress it.

    This happens rarely in many repositories where clients clone all of the objects stored by the server. But it can be quite common when multiple distinct but overlapping sets of objects are stored in the same packfile (for example, due to repository forks or unmerged pull requests). Git may store a delta between objects found only in two different forks. When someone clones one of the forks, they want only one of the objects, and we have to discard the delta.

    Git 2.20 solves this by introducing the concept of “delta islands“. Repository administrators can partition the ref namespace into distinct “islands”, and Git will avoid making deltas between islands. The end result is a repository which is slightly larger on disk but is still able to serve client fetches much more cheaply.

Thursday, April 18, 2019

Babylon's Ashes: a very short review

It had been six months, so I picked up the next episode of The Expanse: Babylon's Ashes.

I'm not actually sure how many series I've read this far, i.e., all the way to Book Six. Patrick O'Brian's stories of Jack Aubrey and Stephen Maturin is the only other that comes to mind.

Of course, O'Brian's books were one of the great literary achievements of the 20th century; that's a high bar!

But clearly The Expanse has something going on.

I agree with those who say that Babylon's Ashes was not the strongest of the series so far. Book Five was much better. The Free Navy are not very interesting, and I'm not sure where I stand on the investigation into the soul of Filip Nagata.

However, one of the strongest parts of the overall series is the way that characters with dark pasts are developed into rich and fascinating stories. Think of Clarissa Mao, or Basia Merton, or even Joe Miller from Book One (Leviathan Wakes).

And, we get some great space battles, we get a very new interesting character in Michio Pa.

My pattern of late has been to take a bit of a break between books. After all, I have much else to read.

But I'm sure I'll be back for Book Seven. I suspect that, just as summer turns into fall, Persepolis Rising will be finding its way into my hands...

Wednesday, April 17, 2019

For historians of the birth of computer games

Here's a pretty amazing resource:

  • The Infocom Cabinet: Binders and Folders of Infocom, Inc. (1981-1987)
    In 2006, documentary filmmaker Jason Scott began production on GET LAMP, a video documentary about the realm of text adventures and interactive fiction. Shooting and research time was roughly 4 years, during which Jason interacted with a large variety of members of the various communities and companies that made up the story of text adventures. Among these was Steve Meretzky, developer at Infocom, Inc, arguably the largest and most influential of the 1980s adventure game companies.

    During his time at Infocom, Steve Meretzky meticulously gathered thousands of pages of notes, journals, maps, memos, forms and other printable materials related to all aspects of Infocom, and kept them in his basement for decades. During the GET LAMP production, Jason Scott scanned in roughly 9,000 pages of these documents across a number of months, borrowing the materials from Steve and scanning them as quickly as possible, at around 600dpi. From these scans, a portion was used in the GET LAMP movie to illustrate various scenes and descriptions by interviewees.

  • Historical Source
    A collection of historical source files, for education and perusal.
  • Source Code For A Ton Of Classic Infocom Games Appears
    A generous benefactor has very kindly offered retro fans and the wider internet a glorious gift this morning: a dump of source code from classic Infocom text games, including the original Zork adventures, Shogun, and Infocom's adaptation of the legendary Douglas Adams novel, Hitchiker's Guide To The Galaxy.

Now, about those other 16 hours per day that I requested in order to study all this...

Thursday, April 11, 2019

The (un)happy Medium

Generally speaking, I feel like I ought to like Medium

So why is it that I seem to be actively avoiding every medium link that shows up in my various feeds, nowadays.

I can't clearly express the unease I have about the platform.

What do you think? Is Medium to be avoided, embraced, or is it just "meh"?

Sunday, April 7, 2019

We are stardust, we are golden; we are billion year old carbon

Every year, as we approach Earth Day, it's good to remember, and good to consider, this magical aggregate of dust upon which we all survive:

  • This Woman Paddled 730 miles up the Green River - to save our water systems.
    I’m paddling the length of the river, to try and understand that risk, my own and other people’s, and to see, from river level, what we could stand to lose if we don’t change how we use and allocate water. “Throughout the whole last century, if you needed more water it always worked out somehow, but it doesn’t work when you get to the point where you’re storing every last drop,” Doug Kenney, Director of the Western Water Policy program at the University of Colorado, tells me before I set out on the river. “You have to talk people through it, and explain that for every new reservoir you try and fill you’re putting more stress on the other parts of the system. Things are changing and we should behave in a way that limits our risk.”

    (See also: Heather Hansman: The Dam Problem in the West)

  • Letter From a Drowned Canyon
    On a map, Glen Canyon before its submersion looks like a centipede: a 200-mile-long central canyon bending and twisting with a host of little canyons like legs on either side. Those side canyons were sometimes hundreds of feet deep; some were so cramped you could touch both walls with your outstretched hands; some had year-round running water in them or potholes that explorers had to swim across. Sometimes in the cool shade of side-canyon ledges and crevices, ferns and other moisture-loving plants made hanging gardens. Even the names of these places are beautiful: Forbidding Canyon, Face Canyon, Dove Canyon, Red Canyon, Twilight Canyon, Balanced Rock Canyon, Ribbon Canyon. Like Dungeon Canyon, they are now mostly underwater.

    When the Sierra Club pronounced Glen Canyon dead in 1963, the organization’s leaders expected it to stay dead under Lake Powell. But this old world is re-emerging, and its fate is being debated again. The future we foresee is often not the one we get, and Lake Powell is shriveling, thanks to more water consumption and less water supply than anyone anticipated. Beneath it lies not just canyons but spires, crests, labyrinths of sandstone, Anasazi ruins, petroglyphs, and burial sites, an intricate complexity hidden by water, depth lost in surface. The uninvited guest, the unanticipated disaster, reducing rainfall and snowmelt and increasing drought and evaporation in the Southwest, is climate change.

  • How the Flint River got so toxic
    Why did Flint’s river pose so many problems? Before processing, the water itself is polluted from four sources: natural biological waste; treated industrial and human waste; untreated waste intentionally or accidentally dumped into the river; and contaminants washed into the river by rain or snow. The river is also warmer than Lake Huron, and its flow is less constant, particularly in the summer. All of this raises levels of bacteria and organic matter and introduces a wide range of other potential contaminants, whether natural or human-made.

    In fact, while the Flint River had been improving thanks to the new regulations, the departure of heavy industry, and local cleanup efforts, it had long been known as an exceptionally polluted river. Until very recently, it had been repeatedly ruled out as a primary source for the city’s drinking water. It is hard to imagine why anyone familiar with the river’s history would ever decide to use it even as a temporary water source. But they did.

  • Looking Again at the Chernobyl Disaster
    A neglected step caused the reactor’s power to plunge, and frantic attempts to revive it created an unexpected power surge. Poorly trained operators panicked. An explosion of hydrogen and oxygen sent Elena into the air “like a flipped coin” and destroyed the reactor. Operators vainly tried to stop a meltdown by planning to shove control rods in by hand. Escaping radiation shot a pillar of blue-white phosphorescent light into the air.

    The explosion occurs less than 100 pages into this 366-page book (plus more than 100 pages of notes, glossary, cast of characters and explanation of radiation units). But what follows is equally gripping. Radio-controlled repair bulldozers became stuck in the rubble. Exposure to radiation made voices grow high and squeaky. A dying man whispered to his nurse to step back because he was too radioactive. A workman’s radioactive shoe was the first sign in Sweden of a nuclear accident 1,000 miles upwind. Soviet bigwigs entered the area with high-tech dosimeters they didn’t know how to turn on. Investigations blamed the accident on six tweakers, portrayed them as “hooligans” and convicted them.

Saturday, April 6, 2019

A sense of fullness

Here's one take: Trump, following border trip, says country is full: 'We can't do it anymore'

President Trump, fresh off a trip to the U.S. southern border, doubled-down on his message that “the country is full”

...

“The country is full. We have ... our system is full. We can't do it anymore,” Trump said

...

The president shared the same message earlier in the day at a roundtable with law enforcement and immigration officials, telling any potential migrants to “turn around” because the U.S. “can’t take you anymore.”

And here's another: Heartland Visas Report

  • U.S. population growth has fallen to 80-year lows. The country now adds approximately 900,000 fewer people each year than it did in the early 2000s.
  • The last decade marks the first time in the past century that the United States has experienced low population growth and low prime working age growth on a sustained basis at the same time.
  • Uneven population growth is leaving more places behind. 86% of counties now grow more slowly than the nation as a whole, up from 64% in the 1990s.
  • In total, 61 million Americans live in counties with stagnant or shrinking populations and 38 million live in the 41% of U.S. counties experiencing rates of demographic decline similar to Japan’s.
  • 80% of U.S. counties, home to 149 million Americans, lost prime working age adults from 2007 to 2017, and 65% will again over the next decade.
  • By 2037, two-thirds of U.S. counties will contain fewer prime working age adults than they did in 1997, even though the country will add 24.1 million prime working age adults and 98.8 million people in total over that same period.
  • Population decline affects communities in every state. Half of U.S. states lost prime working age adults from 2007-2017. 43% of counties in the average state lost population in that same time period, and 76% lost prime working age adults.
  • Shrinking places are also aging the most rapidly. By 2027, 26% of the population in the fastest shrinking counties will be 65 and older compared to 20% nationwide.
  • Population loss is hitting many places with already weak socioeconomic foundations. The share of the adult population with at least a bachelor’s degree in the bottom decile of population loss is half that in the top decile of population growth. Educational attainment in the fastest shrinking counties is on average equivalent to that of Mexico today or the United States in 1978.
  • Population loss itself perpetuates economic decline. Its deleterious effects on housing markets, local government finances, productivity, and dynamism make it harder for communities to bounce back. For example, this analysis found that a 1 percentage point decline in a county’s population growth rate is associated with a 2-3 percentage point decline in its startup rate over the past decade.

Happily for me, I live in one of those areas where immigrants are welcomed; nearly everyone that I spend my waking hours with is either an immigrant or a child of an immigrant, and my part of the country is experiencing the most breathtaking growth, both cultural growth and economic growth, since the pre-Civil-War "Gold Rush" years of 1849-1850.

But I understand that other areas of the country are different.

Monday, April 1, 2019

Alex Honnold breaks it down

Two great tastes that taste great together! Alex Honnold Breaks Down Iconic Rock Climbing Scenes.

Thanks very much, GQ, for making and sharing this very entertaining 15 minute short feature!

Wednesday, March 27, 2019

This seems pointless to me...

... I visit news.google.com.

... It shows me a story that looks interesting.

... I click on the link

... The web site loads, displays the same headline that I saw on news.google.com, and then puts up a pop-up saying that I've reached my article limit and would I like to subscribe?

Nowadays, I just scan news.google.com, and don't bother clicking on most of the links, unless I'm sure that they're to a site, like reuters or cnn or bbc, that isn't going to refuse to show me their article.

I realize that, for the most part, that means that I am that worst of news consumers, the headline-only reader.

So I compromise and I read the entire newspaper once a week, on Sunday.

It seems like not much happens between one Sunday and the next, actually, anyway.

So maybe I'm not missing much.

And I have more time to read books.

And play God of War.

Monday, March 25, 2019

Responses to the Letters to the Editor in the local paper

To the person who wrote the letter entitled: ‘Wellness center’ may end Crab Cove concerts, stating that free public concerts cannot be held near parks that "have a homeless shelter or offer homeless services next to or very close by", I would like to know:

Is this some sort of a legal requirement? Or are you just suggesting that most performers don't want to be seen anywhere close to a social services center?

For a spot of counter-evidence, such as it is, I offer: The Definitive Guide to San Francisco's Free Outdoor Concerts.

I'm not sure what the position of being "Alameda's Arts Advocate" is, but since you refer to "our Concerts at the Cove summer music concert series", I assume you are the person who schedules and invites local musicians to perform in these events, or it is your organization that does so?

If so, I'm sorry you feel that you'll have to move your concerts elsewhere, and I encourage you to reconsider, regardless of what happens in the surrounding areas of the city.

Wednesday, March 13, 2019

The Weight of Ink: a very short review

Who can resist a love story?

Who can resist a love story, set in a library?

It's two great tastes, that taste great together: Rachel Kadish's The Weight of Ink.

Well, I'm not really being fair. It's not set in a library, it's set (partially) in a Rare Manuscripts Conservation Laboratory.

In a library.

Well, it's also set in a kibbutz in Israel.

Oh, and it's also set in 17th century London, during the time of the Inquisition, and the Plague, and yet also, the time of the birth of modern Philosophy.

It's a book about Baruch Spinoza, who you might never have spent much time thinking about (certainly I never did), and it's a book about being Jewish in England during a time when that was only barely legal.

And it's DEFINITELY a love story.

But it's rather a non-traditional love story, not least because a lot of it is about People Who Love Books, both now and then, back in the days when a book was still a thing that People Who Love Books built by hand, with agonizing care.

Our heroes and heroines are the sort of people who know immediately what a rare thing it is to find a 350 year old book, or even writing of any sort:

Her eyes were on the book. "Iron gall ink," she said after a moment.

Following her gaze, he understood that the damage had been done before he ever touched the ledger. The pages were like Swiss Cheese. Letters and words excised at random, holes eaten through the page over the centuries by the ink itself.

And they are the sort of people who can survive the most horrible tortures and injuries, and yet the thing that pains them the most is the loss of books:

Before she knew what she was saying, she turned to the rabbi. "What do you see," she said, "behind the lids of your eyes?"

For the first time there was unease beneath his silence. She felt a hard, thin satisfaction she was ashamed of.

"I shall not, at this moment, answer this question," he said. "But I will tell you what I learned after I lost my sight, in the first days as I came to understand how much of the world was now banned from me -- for my hands would never again turn the pages of a book, nor be stained with the sweet, grave weight of ink, a thing I had loved since first memory. I walked through rooms that had once been familiar, my arms outstretched, and was fouled and thwarted by every obstacle in my path. What I learned then, Ester, is a thing that I have been learning ever since."

The literary technique of trying to tell two stories, one old and set in the past, and one new and set in the current time, is well-known, and although it can be powerful, it can also be a bit of a crutch.

It also leads to a situation in which the book is packed full of characters, and can be a tad confusing when you jump back and forth, although I felt like, overall, The Weight of Ink pulled this off well, and did not over-burden the reader.

Some of the characters are extraordinarily compelling, and front and center is surely Ester, the 17th-century orphan girl who comes to live in the household of a blind, dying rabbi.

Other characters are, well, not quite so gripping, such as the young heiress Mary, or the extremely annoying graduate student Aaron.

But for my money, my favorite was the aging scholar Helen, absorbed in the study of history, dragging herself out of bed every day, overcoming her advanced Parkinson's disease, to get into the library and spend her time with The Books:

For a long time, Helen sat in the silent laboratory. All around her, on shelves and tables, on metal trays and in glass chambers, lay a silent company of paper: centuries old, leaf after leaf, torn or faded or brittle. Pages inked by long-dead hands. Pages damaged by time and worse. But they -- the pages -- would live again.

The climactic scene in which Helen must go to face the Dean, who waits for her to deliver her requested resignation, is remarkably more vivid and compelling and heart-wrenching than you could possibly imagine.

A large part of The Weight of Ink is the painstaking detective story of the literary historians, discovering The Books, poring over their contents, and then, slowly, but surely, reading between the lines to understand what they really say.

But The Weight of Ink succeeded, for me, because it balances that detective story quite nicely with the fill-in-the-blanks story of Ester and her adventures in London.

Bit by bit, page by page, Aaron and Helen come to understand what Ester's life was like, and what she did and thought and felt.

And yet, how could they? How could any of us know what it was like to be a young girl, alone in a city of tragedies at a time of horrors, still consumed by those most elemental of human passions:

"No," she said. "No, it's not that way. I choose with my heart, and my heart is for you." As she said it she felt her heart insisting within her ribs -- indeed, for the first time in her life she almost could see her heart, and to her astonishment it seemed a brave and hopeful thing: a small wooden cup of some golden liquid, brimming until it spilled over all -- the rabbi breathing in his bed, the dim candlelight by which Ester had so long strained at words on the page, the dead girl with her father in the cart. All that was beautiful and all that was precious, all of it streaming with sudden purpose here -- to this place where they now stood.

And, of course, in and around it all, there is that Birth of Modern Philosophy business, with plenty of Hobbes and Descartes and Spinoza.

And whether that's your thing, or not, probably depends a lot on how you feel about Philosophy.

The folly of her own words astonished her. She pulled the papers back from over the water, and read more, and as she read she saw the enormity of her blindness. In her arrogance and loneliness she'd thought she understood the world -- yet its very essence had been missing from her own philosophy.

The imperative -- she whispered it to herself -- to live. The universe was ruled by a force, and the force was life, and live, and live -- a pulsing, commanding law of its own. The comet making its fiery passage across their sky didn't signify divine displeasure, nor did it have anything to say of London's sin; the comet's light existed for the mere purpose of shining. It hurtled because the cosmos demanded it to hurtle. Just as the grass grew in order to grow. Just as the disfigured woman must defy Bescos, who'd consider her unfit for love; just as Ester herself had once, long ago, written because she had to write.

But I suspect that most People Who Love Books are also people who are quite interested in Philosophy, so I suspect that it's actually a pretty fair bet that if you want to read a love story (actually four or five different love stories, as it turns out) set in a library (yes, yes, I know, a Rare Manuscripts Conservation Laboratory), then you probably want to read a fair amount about the early days of Rationalism and its conflicts with the major Religious Philosophies of the day.

Or maybe you just want to read a great love story!

Oh, to heck with it: go read The Weight of Ink. It's well worth your time.