Saturday, February 18, 2017

The Lost City of Z: a very short review

Several years ago, I picked up David Grann's The Lost City of Z: A Tale of Deadly Obsession in the Amazon.

I think I was looking for some summer vacation reading, or a book to take backpacking. As I recall, the book was "too heavy" to take on my backpacking trip: it's 400 pages, and printed on nice quality paper, so it actually weighs a full pound (14.4 ounces according to the publisher). And then it somehow slipped down on the stack and I forgot about it.

But it floated back to the top of the stack the other day, and I finally read it. Just in time, for I see they've now made a movie based on the book.

No surprise there; this is a book which could make quite a good movie, I think.

At times, I had to remind myself that everything in this story is true: a movie star really was abducted by Indians; there were cannibals, ruins, secret maps, and spies; explorers died from starvation, disease, attacks by wild animals, and poisonous arrows; and at stake amid the adventure and death was the very understanding of the Americas before Christopher Columbus came ashore in the New World.

The Lost City of Z is several books at once:

  • It's the story of Percy Fawcett:
    He was the last of the great Victorian explorers who ventured into uncharted realms with little more than a machete, a compass, and an almost divine sense of purpose. For nearly two decades, stories of his adventures had captivated the public's imagination: how he had survived in the South American wilderness without contact with the outside world; how he was ambushed by hostile tribesmen, many of whom had never before seen a white man; how he battled piranhas, electric eels, jaguars, crocodiles, vampire bats, and anacondas, including one that almost crushed him; and how he emerged with maps of regions from which no previous expedition had returned.
  • It's the story of Grann's own obsession with learning about Fawcett and how it seemed to bring out a different person in the author:
    Let me be clear: I am not an explorer or an adventurer. I don't climb mountains or hunt. I don't even like to camp.


    But when I'm working on a story, things are different. Ever since I was young, I've been drawn to mystery and adventure tales, ones that had what Rider Haggard called "the grip."


    While most of my articles seem unrelated, they typically have one common thread: obsession. They are about ordinary people driven to do extraordinary things -- things that most of us would never dare -- who get some germ of an idea in their heads that metastasizes until it consumes them.

  • And it is the story of the upper Amazon region itself, and of the people who live there:
    The region has generally been regarded as a primeval wilderness, a place in which there are, as Thomas Hobbes described the state of nature, "no Arts; no Letters; no Society; and which is worst of all, continuall feare, and danger of violent death." The Amazon's merciless conditions have fueled one of the most enduring theories of human development: environmental determinism. According this theory, even if some early humans eked out an existence in the harshest conditions on the planet, they rarely advanced beyond a few primitive tribes. Society, in other words, is a captive of geography.

It's that third story that I found the most interesting.

Fawcett himself, as it turns out, is rather an unpleasant person, and frankly I wasn't terribly interested in learning more about him.

Grann's modern journey of self-exploration and his own trip into the Amazon jungle is interesting enough, and I enjoyed reading about his own adventures.

But what was really enjoyable about the book was what Grann found there.

In the cave and at a nearby riverbank settlement, Roosevelt made another astonishing discovery: seventy-five-hundred-year-old pottery, which predates by more than two thousand years the earliest pottery found in the Andes or Mesoamerica. This means that the Amazon may have been the earliest ceramic-producing region in all the Americas, and that, as Fawcett radically argued, the region was possibly even a wellspring of civilization throughout South America -- that an advanced culture had spread outward, rather than vice versa.

Using aerial photography and satellite imaging, scientists have also begun to find enormous man-made earth mounds often connected by causeways across the Amazon -- in particular in the Bolivian floodplains where Fawcett first found his shards of pottery.


Heckenberger told me that scientists were just beginning the process of understanding this ancient world -- and, like the theory of who first populated the Americas, all the traditional paradigms had to be reevaluated.


"These people had a cultural aesthetic of monumentality," he said. "They liked to have beautiful roads and plazas and bridges. Their monuments were not pyramids, which is why they were so hard to find; they were horizontal features. But they're no less extraordinary."


"Anthropologists," Heckenberger said, "made the mistake of coming into the Amazon in the twentieth century and seeing only small tribes and saying, 'Well, that's all there is.' The Problem is that, by then, many Indian populations had already been wiped out by what was essentially a holocaust from European contact. That's why the first Europeans in the Amazon described such massive settlements that, later, no one could ever find."

The Lost City of Z is a great read; I'm glad it finally floated back up to the top of my pile.

I will probably never make it to the upper Amazon myself, but I'm quite glad that David Grann took me there, and that he took the time to open his own eyes to what really happened, and is happening, there.

Friday, February 17, 2017

Perhaps the forecast is wrong ...

Right now, they are forecasting FOUR INCHES of rain in the 36-hour period starting Sunday morning and continuing through Monday.

My Mediterranean climate has transmogrified into a rain-forest.

When the rain falls, and the snow melts, ...

... all the water west of the Sierras has only one place to go:

From: BayAlerts
Subject: Storm Debris Affects Ferry Service


Recent storms have washed an unprecedented amount of debris into the San Francisco Bay. On 4 occasions, debris has damaged vessel propellers, resulting in  delays and service cancelations. And, on numerous occasions, debris has been sucked into waterjet propulsion systems, requiring crews to clear the system before proceeding. We are doing our best to steer around major obstacles while adhering to published schedules. We thank you for your patience and understanding.

Wednesday, February 15, 2017

Up, up and away! But, carefully, please!

Not two hours earlier, I was walking along the sidewalk in front of this construction site...

Concrete wall threatening to fall from SF high-rise stabilized

At least 16 office buildings in the South of Market neighborhood of San Francisco were evacuated Wednesday afternoon as a malfunctioning crane in a 35-story skyscraper under construction threatened to topple a 2,000-pound wall of concrete, officials said.

People were evacuated from seven buildings on Howard Street, two on Tehama Street and one on Second Street shortly before 3 p.m. as a crane began leaning precariously into a concrete wall, putting the wall at danger of falling, said Lt. Jonathan Baxter, a spokesman for the San Francisco Fire Department.

The crane was located inside the 30th floor of 33 Tehama Street, between First and Second streets.

“If the crane falls, it could take the concrete wall down,” said Baxter. “The worst case scenario is we’re going to have some structural damage to one or more buildings below.”

He said another concern is that parts of the slab could ricochet and hit surrounding buildings.

Beware of walls of concrete which ricochet into surrounding buildings; yikes!

Sunday, February 12, 2017

Oroville Dam situation

Oh dear, this could be REALLY bad: BREAKING: Marysville, Yuba County evacuated as Oroville spillway collapse feared

Releases through the main spillway at Oroville Dam have been boosted to 100,000 cubic feet per second from 55,000 cfs in hopes of easing pressure on the emergency spillway before a failure occurs, officials said Sunday night.

Kevin Dossey, a Department of Water Resources engineer and spokesman said “it might help” to alleviate the pressure.

So far, Dossey said, the emergency spillway’s concrete lip at the top has not crumbled, although the hillside had “eroded to within several feet” of the big concrete structure.

Marysville and Yuba County ordered evacuated; officials unsure how Sacramento could be hit.

When asked how much water could be released should the spillway collapse, DWR spokesman Chris Orrock said, “It’s uncontrolled. It’s uncontrolled.”

Let's keep our fingers crossed for a little bit of luck, to help all those hard-working souls...

Saturday, February 11, 2017

Read, read, read, read, ...

... learn, learn, learn, learn

  • The Security Impact of HTTPS Interception
    As a class, interception products drastically reduce connection security. Most concerningly, 62% of traffic that traverses a network middlebox has reduced security and 58% of middlebox connections have severe vulnerabilities. We investigated popular antivirus and corporate proxies, finding that nearly all reduce connection security and that many introduce vulnerabilities (e.g., fail to validate certificates). While the security community has long known that security products intercept connections, we have largely ignored the issue, believing that only a small fraction of connections are affected. However, we find that interception has become startlingly widespread and with worrying consequences.
  • Facebook is terrifying
    Now imagine you take a selfie in a crowded place. Like an airport or a train station. There are some people walking on the background. Hundreds of them. Some of them facing the camera. Guess what: the Facebook’s AI has just spotted them.

    Even if you’re extremely cautious, even if you never post anything on Facebook, even if you have “location services” disabled on your phone at all times etc. etc. Facebook still knows where you are. You can’t stop other people from taking selfies in an airport.

  • On Deniability and Duress
    Deniable schemes let you lie about whether you’ve provided full access to some or all of the encrypted text. This is important because, currently, you can’t give the guard in the above example a fake password. He’ll try it, get locked out, and then proceed with the flogging.

    I’m convinced that there’s a sociotechnical blind spot in how current technology handles access to personal devices. We, in the infosec community, need to start focusing more on allowing users the flexibility to handle situations of duress rather than just access control. Deniability and duress codes can go a long way in helping us get there.

  • Backblaze Hard Drive Stats for 2016
    Backblaze has recorded and saved daily hard drive statistics from the drives in our data centers since April 2013. At the end of 2016 we had 73,653 spinning hard drives. Of that number, there were 1,553 boot drives and 72,100 data drives. This post looks at the hard drive statistics of the data drives we monitor. We’ll first look at the stats for Q4 2016, then present the data for all of 2016, and finish with the lifetime statistics for all of the drives Backblaze has used in our cloud storage data centers since we started keeping track. Along the way we’ll share observations and insights on the data presented.
  • Building, And Losing, A Career On Facebook
    Here's how the money part works: Just like Google and Facebook get paid to post advertisements (in your search and your news feed), Lawler gets paid to posts ads too — in his Facebook page — by a third party, an entity known as an "affiliate link" company. In the complex world of online advertising, these companies are middlemen between big brands like Home Depot and publishers. It's a standard practice for businesses on Facebook to post these advertising links. He'll share a link — it could be for a juice company or a news site — and every time a fan clicks on that link, he gets less than a penny.

    But the money adds up. Lawler made anywhere from a couple of hundred dollars a day, to $1,000.

  • Asking the wrong questions
    However, to me the interesting thing is how often the order is wrong. What we now know to be the hard problems were going to be solved decades before what we now know were the easy ones. So it might take until 2020 to 'fax' a newspaper to your home, and automatic wiretapping might be impossible, but automatic doctors, radar implants for the blind, household robots and machine translation would be all done by 1990 and a machine would be passing human IQ tests at genius level by 2000. Meanwhile, there are a few quite important things missing - there is no general-purpose computing, no internet and no mobile phones. There's no prediction for when everyone on earth would have a pocket computer connected to all the world's knowledge (2020-2025). These aren't random gaps - it's not just that they thought X would work and didn't know we'd invent Y. Rather, what's lacking is an understanding of the structural impetus of computing and software as universal platforms that would shape how all of these things would be created. We didn't make a home newspaper facsimile machine - we made computers.
  • Lessons from Real-Time Programming class
    This class has been around since at least the 80’s. Currently Bill Cowan teaches this class, and has been for over 20 years.

    The equipment have evolved since then, but the underlying challenges have not.

    For example, teamwork, setting priorities, and dealing with real (imperfect) systems.

  • Cardinality estimation done right: index-based join sampling
    The index-based sampling operator can cheaply compute a sample for a join result, but it is not a full solution by itself. We also need a join enumeration strategy which can systematically explore the intermediate results of a query using the sampling operator, while also ensuring that the overall sampling time is limited. If we sampled every possible combination, it would take too long for queries with many joins. In the Join Order Benchmark (JOB), queries with 7 joins have 84-107 intermediate results, and queries with 13 joins have 1,517-2,032. A time limit is set on the sampling phase, after which the algorithm falls back to traditional estimation.
  • Vim's 25th anniversary and the release of Vim 8
    2016 was a big year for project anniversaries. The Linux kernel, of course, turned 25. And Vim, that other iconic text editor, also celebrated its 25th anniversary.
  • Monitoring and Tuning the Linux Networking Stack: Sending Data
    This blog post explains how computers running the Linux kernel send packets, as well as how to monitor and tune each component of the networking stack as packets flow from user programs to network hardware.

    This post forms a pair with our previous post Monitoring and Tuning the Linux Networking Stack: Receiving Data.

  • MIT Lecture: Gödel Escher Bach; an Eternal Golden Braid
    MIT Open CourseWare videos investigating Doug Hofstadter's classic book.
  • Online migrations at scale
    Moving millions of objects from one database table to another is difficult, but it’s something that many companies need to do.

    There’s a common 4 step dual writing pattern that people often use to do large online migrations like this. Here’s how it works

  • Coronal Mass Ejections (again)
    A study published last month by the Cambridge Centre for Risk Studies estimates that a solar storm would have the potential to wipe between $140 billion to $613 billion off the global economy in a five-year time span, depending on the severity of the impact.
  • Most of the web really sucks if you have a slow connection
    When I was at Google, someone told me a story about a time that “they” completed a big optimization push only to find that measured page load times increased. When they dug into the data, they found that the reason load times had increased was that they got a lot more traffic from Africa after doing the optimizations. The team’s product went from being unusable for people with slow connections to usable, which caused so many users with slow connections to start using the product that load times actually increased.
  • Disaggregate: Networking recap
    At Facebook, we build our data centers with fully open and disaggregated hardware. This allows us to replace the hardware or the software as soon as better technology becomes available. Because of this, we see compute, storage, and networking gains that scale with our business. We spoke about our latest networking hardware and software — including Wedge 100, Backpack, Voyager, FBOSS and OpenBMC — at the event. We also heard from Apstra, Barefoot, Big Switch Networks, Canonical, Cumulus, and SnapRoute, who talked about their solutions and how they fit in with the rapidly growing ecosystem for open networking.
  • Back-to-Basic Weekend Reading: Monte-Carlo Methods
    The probabilistic approach may not result in the perfect result, but it may get you very close, and much faster than deterministic techniques (which may even be computationally impossible).
  • htop Explained Visually
    htop is an interactive process monitor.
  • Using tmux Properly
    What is a terminal multiplexer? A terminal multiplexer is a souped-up terminal. If you used a plain terminal for a few years and then someone said: "What features do you think we should add?", you'd end up with a multiplexer.
  • Against Storytelling
    Here at the slightly pretentious hotel (call it “P”) we went down to breakfast early and got eggs. The salt and pepper shakers were tallish, stainless steel, with little plastic windows to see the spice. Atop each is a plunger—meaning each one is a little grinder. Interesting! Except they didn’t work.
  • More on GVFS
    Looking at the server from the client, it’s just Git. All TFS and Team Services hosted repos are *just* Git repos. Same protocols. Every Git client that I know of in the world works against them. You can choose to use the GVFS client or not. It’s your choice. It’s just Git. If you are happy with your repo performance, don’t use GVFS. If your repo is big and feeling slow, GVFS can save you.
  • Considerations On Cost Disease
    might the increased regulatory complexity happen not through literal regulations, but through fear of lawsuits? That is, might institutions add extra layers of administration and expense not because they’re forced to, but because they fear being sued if they don’t and then something goes wrong?

    I see this all the time in medicine. A patient goes to the hospital with a heart attack. While he’s recovering, he tells his doctor that he’s really upset about all of this. Any normal person would say “You had a heart attack, of course you’re upset, get over it.” But if his doctor says this, and then a year later he commits suicide for some unrelated reason, his family can sue the doctor for “not picking up the warning signs” and win several million dollars. So now the doctor consults a psychiatrist, who does an hour-long evaluation, charges the insurance company $500, and determines using her immense clinical expertise that the patient is upset because he just had a heart attack.

    Those outside the field have no idea how much of medicine is built on this principle. People often say that the importance of lawsuits to medical cost increases is overrated because malpractice insurance doesn’t cost that much, but the situation above would never look lawsuit-related; the whole thing only works because everyone involved documents it as well-justified psychiatric consult to investigate depression. Apparently some studies suggest this isn’t happening, but all they do is survey doctors, and with all due respect all the doctors I know say the opposite.

  • A Very Comprehensive Guide to Getting Drunk at Disney World
    Animal Kingdom:

    Let’s give credit where credit is due: the staff at Animal Kingdom diligently sourced hard-to-find African beers. Unfortunately, most of them are mundane, flavorless lagers, but since you’ll likely not find these brews at home, they serve their purpose.

    Your choices at the Dawa Bar, located in the heart of Harambe Village near Kilimajaro Safari, are quite varied. There’s not much on tap, but the bottled beer selections are quite cheap and include standouts like the Hakim Stout and Bedele Pilsner, both procured from Ethiopia. Two solid American craft beers round out the menu, SweetWater IPA and Victory Golden Monkey Tripel Ale, and while I’m uncertain why they are present at an ostensibly African bar, they are tasty nonetheless. If you’re ambitious enough to visit in the morning, order an African Bloody Mary made with a spiced Ethiopian-style berbere sauce.

OK, now it has officially rained more than we can handle

There's trouble all around the state, but this is massive:

  • Inside the frantic fight to protect Oroville dam, nation's tallest, as spillway rapidly erodes
    Officials have stressed that the dam itself suffered no damage and that the spillway problems don’t pose a imminent threat to the public. Still, they have been frantically working to reduce the amount of water in the Lake Oroville reservoir, which is near capacity. It’s now at about 96% of capacity, and more water has been flowing in than is draining out.
  • Officials say Oroville Dam not compromised but are preparing for the worst
    As the concrete from the spillway falls into the Diversion Pool and Lake Oroville continues to fill, the California Department of Water Resources is sending about 35,000 cfs of water continually down the broken spillway.

    Some of the water in the Diversion Pool is routed through the Thermalito Forebay and Afterbay where it warms before being used for crops. The rest, about 40,000 cfs as of Thursday, goes directly into the Feather River.

    The river flows parallel to Montgomery Street, one of downtown Oroville’s main streets. Right now, the only thing separating the 40,000 cfs of water in the Feather River from flooding the downtown part of the city is about a hundred yards and a levee that holds back the water.

    Eric See, a public information officer with the Department of Water Resources, said 150,000 cfs went through the Feather River in Oroville during a storm in 1997.

  • Use of untested emergency spillway yet again a possibility at crippled Oroville Dam
    The Department of Water Resources announced it was dialing back water releases over the battered main spillway by about 15 percent to keep erosion along the side of the spillway from “compromising” the power line towers that fuel the dam’s power plant. That reduced releases to 55,000 cubic feet per second.

    With that, the possibility that the reservoir would crest the lip of the emergency spillway was raised anew. The slower releases “may keep the lake level below 901 feet,” the point at which water would start topping the emergency structure, the department announced. However, “there are many variables involved, and the public should not be surprised if some water flows into the emergency spillway.”

  • Damage to Oroville Dam spillway worsens — could cost $100 million
    “We’re going to lose a lot of the spillway,” said Chris Orrock, a spokesman for the California Department of Water Resources, which manages the nation’s tallest dam, about 75 miles north of Sacramento. “The director has said we are willing to lose the bottom of that spillway to make sure we maintain flood control for the downstream communities.”


    The good news, Orrock said, is that the larger spillway, made of reinforced concrete, was peeling downward and not threatening the integrity of the 770-foot-high dam itself. “If the erosion was moving up toward the dam, they would stop the flow,” he said.


    The dam’s spillway — and valves in the Edward Hyatt Power Plant at the bottom of the reservoir — were releasing 79,000 cubic feet per second of water Friday, but the flow was reduced overnight. About 130,000 cubic feet per second was flowing into the dam from the surrounding mountains.

    If it were not damaged, the spillway could usher out up to 200,000 cubic feet per second of water, though that flow would be too much for the Feather River, which can handle 150,000 cubic feet per second without flooding.

  • At Oroville Dam, a break in the storms gives engineers hope
    The break in storms and a drop in the volume of water pouring into the huge reservoir gave dam operators hope that they could keep lake levels from hitting an elevation of 901 feet — the point at which uncontrolled flows would start washing down an unpaved emergency spillway that has never been used in Oroville’s 48-year history.

    “The sun is coming out. The rain has stopped. The inflow has peaked,” said Eric See, a spokesman for the state Department of Water Resources. “We still don’t expect to use the auxiliary spillway.”

Indeed, here in the Bay Area, 250 miles away, the sun is shining and it's a beautiful day, and there's no rain in the forecast.

Well, no rain in the forecast for the next 4 days.

Then the next storm is looming.

It's going to be a very, very, very busy spring for the entire state.

After so many years of below-average rainfall, it's a vivid reminder that California's enormous flood control projects were built for a reason: the peculiar geography of the state positions the entire Central Valley as a giant bathtub, 500 miles long and 60 miles wide, draining a region of over 60,000 square miles.

There's nowhere we can put all that water at this point; all the reservoirs are full.

And the enormous snowpack in the mountains hasn't even begun to melt yet; the real flooding season typically starts in mid-March and runs well through April. We're a full month ahead of schedule; the worst is DEFINITELY yet to come.

So, for the next 3 months at least, as all that snow melts, and runs down from the mountains, it's going to be one flood-control crisis after another.

Hold on for a wild ride.

And best of luck to the hard-working folk at the DWR; pack some extra thermoses of coffee: you're going to be booking a LOT of overtime between now and Memorial Day.