Wednesday, November 30, 2016

And Carlsen it is

Today was the tie-breaker day, and Magnus Carlsen has retained the World Chess Champion title: Magnus Carlsen defeats Sergey Karjakin to retain World Chess Championship – as it happened.

Before the match, I spent a fair amount of time describing Carlsen's astonishing endurance and ability to sustain his concentration over a six, seven, or even eight hour chess game.

But his skill on shorter time frames is even greater.

And, although Karjakin was every bit Carlsen's equal during the standard time control games, today was all Carlsen.

So we move on. As I said, I don't think anyone is pleased that it had to go to tie breaks, but those are the rules and that's the way the match was organized, it was not a surprise that this was a possibility.

Everybody is going to have their own opinions about the match, but overall I was pleased. It was fun chess to watch, and I can't wait for the next match! (Of course, not everyone shares my opinion.)

Tuesday, November 29, 2016

The coming Atlantis

Evidently, the European Space Agency's Sentinel-1 satellites have accurate enough instrumentation that they can actually detect the movement of the Millenium Tower: Satellites confirm sinking of San Francisco tower

To probe these subtle shifts, scientists combined multiple radar scans from the Copernicus Sentinel-1 twin satellites of the same area to detect subtle surface changes – down to millimetres. The technique works well with buildings because they better reflect the radar beam.

Over the weekend, my wife and I were walking along the shore of the bay, approximately 7 miles from downtown, with a very clear view on the day after a big storm, and my wife wondered if it was possible to tell which tower was the Millenium Tower from our perspective.

I suspect not.

But if we had a satellite...

Monday, November 28, 2016

Carlsen-Karjakin, game 12: draw (6-6)

And so, the match is complete, with a fairly bloodless 30-move draw in the final game.

As I understand it, there will be a tie-breaking session on Wednesday:

If the score is tied after 12 games, rapid chess (4 games), then blitz chess (five 2-game matches), and possibly an Armageddon game will be used until the tie is broken (Regulations 3.7)

Of course, nobody will be satisfied with this result; tie break procedures are never satisfying.

But, one way or another, on Wednesday we will know for sure.

What I've been reading, late November edition

We had a lot of rain in November, and the reservoirs are filling up fast. New Melones and Pine Flat are still dramatic outliers at less than 25% capacity, but the Big Three (Shasta, Oroville, and Trinity) are filling up fast. Let's go, rain!

Meanwhile, there's always more to read...

  • A Crazy Miscalculation Doomed the Schiaparelli Lander
    As it was making its slow descent, Schiaparelli’s Inertial Measurement Unit (IMU) went about its business of calculating the lander’s rotation rate. For some reason, the IMU calculated a saturation-maximum period that persisted for one second longer than what would normally be expected at this stage. When the IMU sent this bogus information to the craft’s navigation system, it calculated a negative altitude. In other words, it thought the lander was below ground level.


    Encouragingly, this behavior was replicated in computer simulations, which means mission planners stand a good chance of correcting the anomaly. The exact cause of the IMU’s miscalculation was not disclosed, but if it was tripped by some kind of mechanical problem, that would be bad news. The ESA is planning a similar mission in 2020, which doesn’t leave much time for an engineering overhaul. A software glitch, on the other hand, would likely prove to be an easier fix.

  • My Distributed Systems Seminar's reading list for Spring 2017
    Below is the first draft list of papers I plan to discuss in my distributed systems seminar in the Spring semester.
  • Benchmarking Commit Logs
    The bigger question at this point is how NATS Streaming will tackle scaling and replication (a requirement for true production-readiness in my opinion). Kafka was designed from the ground up for high scalability and availability through the use of external coordination (read ZooKeeper). Naturally, there is a lot of complexity and cost that comes with that. NATS Streaming attempts to keep NATS’ spirit of simplicity, but it’s yet to be seen how it will reconcile that with the complex nature of distributed systems. I’m excited to see where Apcera takes NATS Streaming and generally the NATS ecosystem in the future since the team has a lot of experience in this area.
  • CockroachDB Stability Post-Mortem: From 1 Node to 100 Nodes
    We downsized the team working on core components (the transactional, distributed key-value store), composed of five engineers with the most familiarity with that part of the codebase. We even changed seating arrangements, which felt dangerous and counter-cultural, as normally we randomly distribute engineers so that project teams naturally resist balkanization.


    Relocating team members for closer proximity felt like it meaningfully increased focus and productivity when we started. However, we ended up conducting a natural experiment on the efficacy of proximity. First two, and then three, out of the five stability team members ended up working remotely. Despite the increasing ratio of remote engineers, we did not notice an adverse impact on execution.


    The smaller stability team instituted obsessive review and gatekeeping for changes to core components. In effect, we went from a state of significant concurrency and decentralized review to a smaller number of clearly delineated efforts and centralized review.

    Somewhat counter-intuitively, the smaller team saw an increase per engineer in pull request activity

  • Code Is Ephemeral, Concepts Are Eternal
    Concepts fight obsolescence. Even when ASP.NET inevitably dies, the concepts I've learned from programming in it for ten plus years will still be useful. Concepts have a longer shelf life than details, because details change. Languages are born and die, frameworks become unpopular overnight, companies go out of business, support will end. But the thoughts, the ideas, the best practices? They live forever.

    Learn about SOLID. Learn KISS, DRY, and YAGNI. Learn how important naming is. Learn about proper spacing, functional vs object-oriented, composition vs. inheritance, polymorphism, etc. Learn soft skills like communication and estimation. Learn all the ideas that result in good code, rather than the details (syntax, limitations, environment, etc.) of the code itself. Mastering the ideas leads to your mind being able to warn you when you are writing bad code (as you will inevitably do).

  • Cloud Trends — Where have we come from and where are we headed
    What do modern applications look like? We are seeing the combination of rapid cloud based provisioning, a DevOps culture transformation, and the journey from waterfall through agile to continuous delivery product development processes give rise to a new application architecture pattern called microservices. This shares the same principles as the service oriented architecture movement from 10–15 years ago, but in those days, machines and networks were far slower, and XML/SOAP messaging standards were inefficient. The high latency and low messaging rates meant that applications ended up composed of relatively few large complex services. With much faster hardware and more efficient messaging formats, we have low latency and high messaging rates. This makes it practical to compose applications of many simple single function microservices, independently developed and continuously deployed by cloud native automation.
  • AWS Well-Architected Framework
    The AWS Well-Architected Framework documents a set of foundational questions that allow you to understand if a specific architecture aligns well with cloud best practices. The framework provides a consistent approach to evaluating systems against the qualities you expect from modern cloud-based systems, and the remediation that would be required to achieve those qualities.
  • Where’s DevOps Innovation Headed? Reflections from the 2016 DevOps Enterprise Summit
    Something that stood out in almost all of the presentations at DOES16 was the vast number of tools and point solutions companies are using to achieve their goals. Many speakers at some point in their presentations even listed the hodge-podge of vendors and tools they have in their toolchains. Different teams within an organization use a variety of different tools, and the resulting complexity can become overwhelming for enterprises.
  • The Macintosh Endgame
    Instead of racing to the bottom as the market plummets, Apple appears to be taking the “high road”, in a sense: They’re taking refuge at the high end of the market by introducing new, more expensive MacBook Pros, with a visible differentiating feature, the Touch Bar. This is known, inelegantly, as milking a declining business, although you shouldn’t expect Apple to put it that way.
  • Why Dark Posts Are the Best Facebook Advertising Approach Right Now
    Facebook has allowed advertisers to create news stream ads with the option of not publishing to them to the news feed for some time, but it’s still a fairly untapped play for the moment.

    By employing this tactic the advertiser mentioned about could run all four product ads as sponsored posts, target different audiences, split test headlines and even create personalized messages for demographic and geographic targets – literally run dozens of ads all on the same day – without a single ad showing in their own news stream.

  • Managing the Machines: AI is making prediction cheap, posing new challenges for managers
    The modern era of autonomous driving began in the 1980s. The US and Germany were the two nations at the forefront in this line of research. In the US, the research was largely funded by DARPA (Defense Advanced Research Projects Agency). In Germany, large automotive companies such as Mercedes-Benz funded research. The leading projects utilized computer vision-based systems, lidar, and autonomous robotic control. The decision-making systems were essentially driven by optimizing if-then-else algorithms (e.g., optimize speed subject to not exceeding the speed limit and not hitting anything; “if is raining, then slow down…”, “if a pedestrian approaches within 5 feet on the left, then swerve right”). In other words, the systems were algorithmic, that is, codifying an algorithm describing the connection between road conditions and decisions related to speed and steering.

    However, to be able to drive on roads in unstructured and unpredictable environments, including ones where other drivers (human or autonomous) also exist, requires predicting the outcomes of a large number of possible actions. Codifying the set of possible outcomes proved too challenging. In the early 2000s, however, several groups began using primitive versions of modern machine-learning techniques. The key difference between the new method and the old method is that while the old method centered around optimizing a long list of if-then-else statements, the new method instead predicts what a human driver would do given the set of inputs (e.g., camera images, lidar information, mapping data, etc). This facilitated significant improvements in autonomous driving performance.

  • Some Simple Economics of the Blockchain
    Whereas the utopian view has argued that blockchain technology will affect every market by reducing the need for intermediation, we argue that it is more likely to change the scope of intermediation both on the intensive margin of transactions (e.g., by reducing costs and possibly influencing market structure) as well as on the extensive one (e.g., by allowing for new types of marketplaces). Furthermore, for the technology to have any impact in a specific market, verification of transaction attributes (e.g., status of a payment, identity of the agents involved etc.) by contracting third-parties needs to be currently expensive; or network operators must be enjoying uncompetitive rents from their position as trusted nodes above and beyond their added value in terms of market design.
  • Consumers can’t help health care. Customers can.
    Economically speaking, the American health care system is not built for patients, because patients aren’t the ones paying for it directly. Insurance companies are.

    See, health care in the U.S. is mostly a B2B business. It is only B2C where insurance doesn’t cover expenses to the patient. And even then, insurance still often pays for it when patients can’t, don’t or both.

Saturday, November 26, 2016

Carlsen-Karjakin game 11: draw (5.5 - 5.5)

The eleventh game is now decided, and it was a draw.

Carlsen, with the black pieces, seemed to me to have a lot of play, and managed to advance a pawn all the way to the 7th rank.

But Karjakin was careful, and was able to force a draw by perpetual check.

So Monday will be the final game of the scheduled 12 games. If either player wins that game, that player wins the entire match.

But if they draw, then they will decide the championship by playing some more games, on Wednesday, using different time controls (faster and faster, until someone loses).

On we go!

Thursday, November 24, 2016

Carlsen-Karjakin game 10: Carlsen win (5.0-5.0)

He's back!

Carlsen wins with white, and the match is back to even.

You could just sort of tell, I think, that Carlsen wanted this game, badly.

On move 19, Karjakin offered an exchange of bishops, and re-took with his f-pawn, to open up the possible lanes for an attack.

But Carlsen quickly exchanged off the queens, and by move 30 the game had entered what became an extraordinarily complex endgame. Both sides had a pair of rooks and a knight, and all 16 pawns were still on the board.

But one of those pawns was doubled: Karjakin's f-pawn, offered back on move 19, became the focus of the entire game.

Carlsen maneuvered and maneuvered, patiently and carefully, taking his sweet, sweet time, as he is so willing to do.

Karjakin defended superbly, for 50 more moves, as the game stretched past the second time control, and entered its seventh hour.

And then, in a blink, it was over.

So now there are just 2 games to go in the match. Both sides have demonstrated they can win.

To me, it seems like anyone's match.

But I think Carlsen may have recovered his form with this win.

We'll see soon enough!

Wednesday, November 23, 2016

Carlsen-Karjakin game 9: draw (4.0-5.0)

As we draw closer to the conclusion of the match, things have really heated up!

With his back against the wall, Carlsen is NOT going quietly.

In what appeared to be a very classic, very vanilla Ruy Lopez, Carlsen, with the black pieces, sacrificed a pawn in the opening for initiative, rapid development, and a quite threatening attack. By move 17, Karjakin's king was open and exposed, and Carlsen was lining up the big guns.

Moves 20 through 40 were, frankly, as exciting as chess ever gets, with the advantage see-sawing back and forth, pieces en prise all around, sacrifices, advanced passed pawns, each player's king being chased around the board, both barely avoiding disaster...

Then suddenly, just after both players had made the 40-move time control, all the pieces were suddenly off the board, leaving each player with just Queen and Bishop.

Karjakin had an extra pawn, but it was doubled, and Carlsen's pawns were connected, while Karjakin's were not.

Karjakin pressed hard, hard, hard for 30 more moves, but Carlsen was equally tenacious, and there was no breakthrough to be found by either player.

I'm really looking forward to the final three games. The match may have started slowly, but once blood was drawn, it's been as vibrant and vivid as I could have possibly hoped for.