2016 has flown by.
- 15 Years of Concurrency
In a Tale of Three Safeties, we discussed three kinds of safety: type, memory, and concurrency. In this follow-on article, we will dive deeper into the last, and perhaps the most novel yet difficult, one. Concurrency-safety led me to the Midori project in the first place, having spent years on .NET and C++ concurrency models leading up to joining. We built some great things that I’m very proud of during this time. Perhaps more broadly interesting, however, are the reflections on this experience after a few years away from the project.
- AWS re:Invent 2016
There was no inside-the-data center ground fault and that is exactly my point. The facility did not have a problem but the switchgear incorrectly locked out the backup power. The customer called in the utility to investigate and they reported the facility experienced a switch fault that locked out the backup generator.
Under rare circumstances the switch gear incorrectly determines there is a problem and does not transfer the load to generator. When this happens, the generators are running but not taking load due to switch gear lock-out and the critical load is dropped when the UPSs are exhausted.
- Cherami: Uber Engineering’s Durable and Scalable Task Queue in Go
Cherami is a distributed, scalable, durable, and highly available message queue system we developed at Uber Engineering to transport asynchronous tasks. We named our task queue after a heroic carrier pigeon with the hope that this system would be just as resilient and fault-tolerant, allowing Uber’s mission-critical business logic components to depend on it for message delivery.
- GLB part 2: HAProxy zero-downtime, zero-delay reloads with multibinder
Unfortunately there’s a small period between when this process last calls accept() and when it calls close() where the kernel will still route some new connections to the original socket. The code then blindly continues to close the socket, and all connections that were queued up in that LISTEN socket get discarded (because accept() is never called for them)
For small scale sites, the chance of a new connection arriving in the few microseconds between these calls is very low. Unfortunately at the scale we run HAProxy, a customer impacting number of connections would hit this issue each and every time we reload HAProxy.
- Orchestrator at GitHub
The master is not an isolated entity. It has replicas. These replicas continuously poll the master for incoming changes, copy those changes and replay them. They have their own retry count/interval setup. When orchestrator looks for a failure scenario, it looks at the master and at all of its replicas. It knows what replicas to expect because it continuously observes the topology, and has a clear picture of how it looked like the moment before failure.
orchestrator seeks agreement between itself and the replicas: if orchestrator cannot reach the master, but all replicas are happily replicating and making progress, there is no failure scenario. But if the master is unreachable to orchestrator and all replicas say: “Hey! Replication is broken, we cannot reach the master”, our conclusion becomes very powerful: we haven’t just gathered input from multiple hosts. We have identified that the replication cluster is broken de-facto. The master may be alive, it may be dead, may be network partitioned; it does not matter: the cluster does not receive updates and for all practical purposes does not function.
- The Medium-Term Prospects for Long-Term Storage Systems
The fundamental problem is not storing bits safely for the long term, it is paying to store bits safely for the long term. With an unlimited budget an unlimited amount of data could be stored arbitrarily reliably indefinitely. But in the real world of limited budgets there is an inevitable tradeoff between storing more data, and storing the data more reliably.
Historically, this tradeoff has not been pressing, because the rate at which the cost per byte of storage dropped (the Kryder rate) was so large that if you could afford to keep some data for a few years, you could afford to keep it "forever". The incremental cost would be negligible. Alas, this is no longer true.
- Nobody ever implements sort
The place where this is used is performance-critical -- the "semi-sorted" (a type of compression) cuckoo filter has to sort the contents of a 4-element bucket any time an element is inserted. The sorting network works well on our target x86 platform because it exploits the inherent parallelism of modern processors (they can issue multiple instructions per cycle, if those instructions are independent). The entirely inlined implementation avoids a lot of unnecessary function call and setup overhead from a more general-purpose sorting algorithm.
- So what are Bloom indexes for Postgres?
Basically Postgres calculates a hash for each of your column values and then stores some bits out of each hash as one index entry, together with the row’s physical location info (as with every other index). And exactly this “merging” of many column values into one index entry, resulting in a signature in our Bloom context, is where the effectiveness of this index type comes to shine. In short – it can help you to save a lot of disk space! Thus instead of 10 separate normal B-tree indexes you can now have only one Bloom index that’s though lossy, meaning it won’t give you perfect accuracy as matched values need to be always re-checked from the table, but from a probabilistic viewpoint it is “good enough” to be useful.
- PostgreSQL "Compatible" Aurora
The language Amazon uses around Aurora is really wierd – they talk about “MySQL compatibility” and “PostgreSQL compatibility”. At an extreme, one might interpret that to mean that Aurora is a net-new database providing wire- and function-level compatibility to the target databases. However, in the PostgreSQL case, the fact that they are additionally supporting PostGIS, the server-side languages, really the whole database environment, hints strongly that most of the code is actually PostgreSQL code.
- What Kangaroos & Beachballs Can Teach You About Code
The problem is that class inheritance (by extension, the `extends` keyword in JavaScript) forces you to inherit everything from the parent class. This problem is easily avoided using object composition instead of class inheritance.
- Why Kakoune — The quest for a better code editor
Many vi users have an epiphany when they realize that vi does not just provide a set of modes making various text editing shortcuts easier to type, but actually provides a text editing language.
Commands are composable in order to express complex changes, dw in vi is not just a shortcut to delete a word, it is the combination of a verb: d for delete, with an object w for word. There are more complex objects like ib (inside block) refers to the content of the parenthesis surrounding the cursor, so yib would yank (copy) the text inside the surround parenthesis.
This language allows the programmer to express their intent much more closely than in other editors; most editors can express "delete the word after the next parenthesis", but more often than not, expressing that intent is more cumbersome than simply doing an ad-hoc edit. Text editing as a language changes that, by making clearly expressing your intent the fastest and easiest way to do your edit.
- Defending Accounts Against Common Attacks
While we often worry about sophisticated digital attacks, the most common attacks for accessing news organizations’ accounts depend on only a few simple weaknesses. These weaknesses are usually a combination of predictable passwords, phishing emails designed to steal login credentials, as well as malicious file attachments in email and elsewhere. While the attacks are simple, so are the defenses. This collection of resources and learning materials will walk you through practices recommended by security specialists for defending your newsroom against common attacks on your accounts.
- What Git got Right and Wrong
I still have real complaints with the software These include fundamentally different concepts merged into the same label and the fact that commands may do many different things depending on how you call them. The fact that the concepts are not clear means that it is worse than a learning curve issue. One cannot have a good grasp of what git is doing behind the scenes because this is not always clear.
- One More Sign World Is Shrinking – eBay is for Suckers
No one should be surprised that unscrupulous buyers use eBay to commit fraud on unsuspecting sellers. What surprised me was the extent to which eBay now facilitates this fraud through its “buyer protection program”. In October this year I listed a very slightly used iPhone 6S for sale on eBay and was quite satisfied when it eventually sold for $465. This satisfaction was short-lived, however, as I came to realize that I had been taken in by an eBay scammer.
- The Man Who Tried to Redeem the World with Logic
So impressed was Wiener that he promised Pitts a Ph.D. in mathematics at MIT, despite the fact that he had never graduated from high school—something that the strict rules at the University of Chicago prohibited. It was an offer Pitts couldn’t refuse. By the fall of 1943, Pitts had moved into a Cambridge apartment, was enrolled as a special student at MIT, and was studying under one of the most influential scientists in the world. It was quite a long way from blue-collar Detroit.
- KVH Industries Tour
KVH was founded as Sailcomp back in 1982 by Arent Kits van Heyningen and his sons Robert and Martin. Their initial product offering was a digital compass for use in racing sailboats. Nearby Newport, Rhode Island was at the time a hot-bed for racing sailboats and, although America’s Cup racing no longer takes place there, sail racing remains an important part of Newport.
KVH has evolved to focus on satellite communications/guidance and stabilization for both military and civilian applications but, while walking through the factory, the descendant of the original KVH product line remains a part of the now much broader product line.
- Northamptonshire in a Nutshell
Written and illustrated by Ron Mears
- Ships under Financial District
The shadowed area is roughly the outline of Yerba Buena Cove when the city was founded and there are more than 70 old wooden ships buried in the old cove that is now part of the city.
No comments:
Post a Comment