Monday, November 28, 2016

What I've been reading, late November edition

We had a lot of rain in November, and the reservoirs are filling up fast. New Melones and Pine Flat are still dramatic outliers at less than 25% capacity, but the Big Three (Shasta, Oroville, and Trinity) are filling up fast. Let's go, rain!

Meanwhile, there's always more to read...

  • A Crazy Miscalculation Doomed the Schiaparelli Lander
    As it was making its slow descent, Schiaparelli’s Inertial Measurement Unit (IMU) went about its business of calculating the lander’s rotation rate. For some reason, the IMU calculated a saturation-maximum period that persisted for one second longer than what would normally be expected at this stage. When the IMU sent this bogus information to the craft’s navigation system, it calculated a negative altitude. In other words, it thought the lander was below ground level.


    Encouragingly, this behavior was replicated in computer simulations, which means mission planners stand a good chance of correcting the anomaly. The exact cause of the IMU’s miscalculation was not disclosed, but if it was tripped by some kind of mechanical problem, that would be bad news. The ESA is planning a similar mission in 2020, which doesn’t leave much time for an engineering overhaul. A software glitch, on the other hand, would likely prove to be an easier fix.

  • My Distributed Systems Seminar's reading list for Spring 2017
    Below is the first draft list of papers I plan to discuss in my distributed systems seminar in the Spring semester.
  • Benchmarking Commit Logs
    The bigger question at this point is how NATS Streaming will tackle scaling and replication (a requirement for true production-readiness in my opinion). Kafka was designed from the ground up for high scalability and availability through the use of external coordination (read ZooKeeper). Naturally, there is a lot of complexity and cost that comes with that. NATS Streaming attempts to keep NATS’ spirit of simplicity, but it’s yet to be seen how it will reconcile that with the complex nature of distributed systems. I’m excited to see where Apcera takes NATS Streaming and generally the NATS ecosystem in the future since the team has a lot of experience in this area.
  • CockroachDB Stability Post-Mortem: From 1 Node to 100 Nodes
    We downsized the team working on core components (the transactional, distributed key-value store), composed of five engineers with the most familiarity with that part of the codebase. We even changed seating arrangements, which felt dangerous and counter-cultural, as normally we randomly distribute engineers so that project teams naturally resist balkanization.


    Relocating team members for closer proximity felt like it meaningfully increased focus and productivity when we started. However, we ended up conducting a natural experiment on the efficacy of proximity. First two, and then three, out of the five stability team members ended up working remotely. Despite the increasing ratio of remote engineers, we did not notice an adverse impact on execution.


    The smaller stability team instituted obsessive review and gatekeeping for changes to core components. In effect, we went from a state of significant concurrency and decentralized review to a smaller number of clearly delineated efforts and centralized review.

    Somewhat counter-intuitively, the smaller team saw an increase per engineer in pull request activity

  • Code Is Ephemeral, Concepts Are Eternal
    Concepts fight obsolescence. Even when ASP.NET inevitably dies, the concepts I've learned from programming in it for ten plus years will still be useful. Concepts have a longer shelf life than details, because details change. Languages are born and die, frameworks become unpopular overnight, companies go out of business, support will end. But the thoughts, the ideas, the best practices? They live forever.

    Learn about SOLID. Learn KISS, DRY, and YAGNI. Learn how important naming is. Learn about proper spacing, functional vs object-oriented, composition vs. inheritance, polymorphism, etc. Learn soft skills like communication and estimation. Learn all the ideas that result in good code, rather than the details (syntax, limitations, environment, etc.) of the code itself. Mastering the ideas leads to your mind being able to warn you when you are writing bad code (as you will inevitably do).

  • Cloud Trends — Where have we come from and where are we headed
    What do modern applications look like? We are seeing the combination of rapid cloud based provisioning, a DevOps culture transformation, and the journey from waterfall through agile to continuous delivery product development processes give rise to a new application architecture pattern called microservices. This shares the same principles as the service oriented architecture movement from 10–15 years ago, but in those days, machines and networks were far slower, and XML/SOAP messaging standards were inefficient. The high latency and low messaging rates meant that applications ended up composed of relatively few large complex services. With much faster hardware and more efficient messaging formats, we have low latency and high messaging rates. This makes it practical to compose applications of many simple single function microservices, independently developed and continuously deployed by cloud native automation.
  • AWS Well-Architected Framework
    The AWS Well-Architected Framework documents a set of foundational questions that allow you to understand if a specific architecture aligns well with cloud best practices. The framework provides a consistent approach to evaluating systems against the qualities you expect from modern cloud-based systems, and the remediation that would be required to achieve those qualities.
  • Where’s DevOps Innovation Headed? Reflections from the 2016 DevOps Enterprise Summit
    Something that stood out in almost all of the presentations at DOES16 was the vast number of tools and point solutions companies are using to achieve their goals. Many speakers at some point in their presentations even listed the hodge-podge of vendors and tools they have in their toolchains. Different teams within an organization use a variety of different tools, and the resulting complexity can become overwhelming for enterprises.
  • The Macintosh Endgame
    Instead of racing to the bottom as the market plummets, Apple appears to be taking the “high road”, in a sense: They’re taking refuge at the high end of the market by introducing new, more expensive MacBook Pros, with a visible differentiating feature, the Touch Bar. This is known, inelegantly, as milking a declining business, although you shouldn’t expect Apple to put it that way.
  • Why Dark Posts Are the Best Facebook Advertising Approach Right Now
    Facebook has allowed advertisers to create news stream ads with the option of not publishing to them to the news feed for some time, but it’s still a fairly untapped play for the moment.

    By employing this tactic the advertiser mentioned about could run all four product ads as sponsored posts, target different audiences, split test headlines and even create personalized messages for demographic and geographic targets – literally run dozens of ads all on the same day – without a single ad showing in their own news stream.

  • Managing the Machines: AI is making prediction cheap, posing new challenges for managers
    The modern era of autonomous driving began in the 1980s. The US and Germany were the two nations at the forefront in this line of research. In the US, the research was largely funded by DARPA (Defense Advanced Research Projects Agency). In Germany, large automotive companies such as Mercedes-Benz funded research. The leading projects utilized computer vision-based systems, lidar, and autonomous robotic control. The decision-making systems were essentially driven by optimizing if-then-else algorithms (e.g., optimize speed subject to not exceeding the speed limit and not hitting anything; “if is raining, then slow down…”, “if a pedestrian approaches within 5 feet on the left, then swerve right”). In other words, the systems were algorithmic, that is, codifying an algorithm describing the connection between road conditions and decisions related to speed and steering.

    However, to be able to drive on roads in unstructured and unpredictable environments, including ones where other drivers (human or autonomous) also exist, requires predicting the outcomes of a large number of possible actions. Codifying the set of possible outcomes proved too challenging. In the early 2000s, however, several groups began using primitive versions of modern machine-learning techniques. The key difference between the new method and the old method is that while the old method centered around optimizing a long list of if-then-else statements, the new method instead predicts what a human driver would do given the set of inputs (e.g., camera images, lidar information, mapping data, etc). This facilitated significant improvements in autonomous driving performance.

  • Some Simple Economics of the Blockchain
    Whereas the utopian view has argued that blockchain technology will affect every market by reducing the need for intermediation, we argue that it is more likely to change the scope of intermediation both on the intensive margin of transactions (e.g., by reducing costs and possibly influencing market structure) as well as on the extensive one (e.g., by allowing for new types of marketplaces). Furthermore, for the technology to have any impact in a specific market, verification of transaction attributes (e.g., status of a payment, identity of the agents involved etc.) by contracting third-parties needs to be currently expensive; or network operators must be enjoying uncompetitive rents from their position as trusted nodes above and beyond their added value in terms of market design.
  • Consumers can’t help health care. Customers can.
    Economically speaking, the American health care system is not built for patients, because patients aren’t the ones paying for it directly. Insurance companies are.

    See, health care in the U.S. is mostly a B2B business. It is only B2C where insurance doesn’t cover expenses to the patient. And even then, insurance still often pays for it when patients can’t, don’t or both.

No comments:

Post a Comment