In my mind I imagine a Witcher 3 without Ciri. One which embraces Geralt's wandering nature, his inclination towards short-term contract work, and relies on these ten-hour plots. A world the size of Novigrad/Velen/Skellige but filled with all manner of quests from one-off contracts to—on the other end—these heavily character-driven vignettes of a witcher's life.
Saturday, October 31, 2015
Friday, October 30, 2015
Frankly, I had no idea that there was an annual awards ceremony called the Golden Joysticks.
Shows how much attention I've been paying.
The action, role-playing game won ultimate game of the year, best storytelling, best visual design and best gaming moment.
"Best gaming moment"?
Oh, I see:
Best gaming moment: Bloody Baron quest in The Witcher 3: Wild Hunt
Yes, that definitely deserved an award.
Sunday, October 25, 2015
We took a short trip to the mountains, to see if it was as dry as the reports all said.
It was even worse.
Still, the mountains are beautiful, even when they are parched.
Meanwhile, the reading list only grew while I was gone...
- In Defense of The New York Times
In other words, the job of The New York Times is no longer to produce "All the News That’s Fit to Print"; rather, it is to invest in stories that make a difference — stories that start a conversation — and trust that readers will be willing to pay for quality. The content follows from the business model.
- '10-second' theoretical hack could jog Fitbits into malware-spreading mode
The athletic-achievement-accumulating wearables are wide open on their Bluetooth ports, according to research by Fortinet. The attack is quick, and can spread to other computers to which an infected FitBit connects.
- The NSA and Weak-DH
They further observed that most servers using this for IPsec, a major Virtual Private Network protocol that encrypts a large amount of business traffic, commonly use the same p and g, and most of these systems are using 1024b Diffie-Hellman.
So with an NSA-style budget of a few hundred million dollars, one could build a supercomputer that can first perform a huge amount of work, running for months, in order to break a particular 1024b p and g and then, using the same supercomputer, quickly break any key exchange using that particular p and g. This wouldn’t work for longer keys (such as 3072b Diffie-Hellman), elliptic curve Diffie-Hellman, or RSA encryption.
- I'm Shocked, Shocked to Find There's Cryptanalysis Going On Here (Your plaintext, sir.)
There's also been unhappiness that IPsec uses a small set of Diffie-Hellman moduli. Back when the IETF standardized those groups, we understood that this was a risk. It's long been known that the discrete log problem is "brittle": you put in a lot of work up front, and you can solve each instance relatively cheaply. The alternative seemed dangerous. The way Diffie-Hellman key exchange works, both parties need to have the same modulus and generator. The modulus has to be prime, and should be of the form 2q+1, where q is also a prime. Where does the modulus come from? Presumably, one party has to pick it. The other party then has to verify its properties; the protocol has to guard against downgrades or other mischief just in agreeing on the modulus. Yes, it probably could have been done. Our judgment was that the risks weren't worth it. The real problem is that neither vendors nor the IETF abandoned the 1024-bit group. RFC 4307, issued ten years ago, warned that the 1024-bit group was likley to be deprecated and that the 2048-bit group was likley to be required in some future document.
- Fun with recreating an evil merge
A good news is that, when the evil merge is in a file that also has textual conflicts to resolve, "git rerere" will automatically take care of this situation. All you need to do is to set the configuration rerere.enabled to true before attempting the merge between X and B and recording their merge M, and then attempt a new merge between B and Y. Without even having to type "git rerere", the mechanism is invoked by "git merge" to replay the recorded resolution (which is where the name of the machinery "rerere" comes from). A bad news is that when an evil merge has to be made to a file that is not involved in any textual conflict (i.e. imagine the case where we didn't have "line added by A" vs "line added by X" conflict earlier in the same file in the above example), "rerere" does not even kick in. The question is what to do, knowing B, X, and M, to recreate N while keeping the adjustment needed for semantic conflicts to record M.
- Storage Technology Roadmaps
At the recent Library of Congress Storage Architecture workshop, Robert Fontana of IBM gave an excellent overview of the roadmaps for tape, disk, optical and NAND flash (PDF) storage technologies in terms of bit density and thus media capacity. His slides are well worth studying, but here are his highlights for each technology
- Stacking Up The Next Modern Platform
When we were building Google Compute Engine, I viewed virtual machines on GCE as a transitional technology. VMs will always be around but there is a better world out there if we get past the local maxima that we are at with the current infrastructure offerings.
- ARM Server Market
Clearly Intel still makes the CPU behind more than 90% of the world’s servers (even when taking a very generous interpretation of server). And, just as clear, Intel is a very competent company that has in the past responded quickly to competitive pressure. Intel has also gotten very good at working closely with its major customers and, unlike the bad old days, is actually very good to work with. I’m more impressed with what they have been bringing to market than ever. Nonetheless, there are factors that make it very likely that we are going to see some very good server parts based upon ARM in market in the near future. It’s hard to predict the pace of execution of any of the participants nor where this will end up but, generally, change and competition is good for the industry and great for customers.
- It's happening - OpenSSH for Windows...from Microsoft
Sure, it's late, and ya, it should have happened years ago, but it's happening and it'll be built in. SSH will be one less thing to worry about.
- Overcomplexifying, Underdelivering
While it is hard to draw definitive lessons from a handful of programs, it is nearly certain that IT modernization efforts will overrun their cost estimates by significant amounts. The chart also demonstrates the challenges of holding these projects accountable when they do: cost overruns, delays, and reduced functionality are so common that even self-proclaimed success-stories have them.
One solution is to try to make more realistic initial estimates. More data is required (and if you know of a project we're missing, please leave a comment), but trying to combine more than 50 nontrivial legacy government IT systems for less than $400 million to $500 million seems nearly impossible. So we should be skeptical when we see overoptimistic—or outright fraudulent—project estimates that claim to do just that.
- The Tide Turns on Big Outsourcing – on cloud, agile, and rebuilding skills
The idea that massive outsourcing contracts don’t suffer from scope creep and related, massive cost overruns, would be funny were it not for the fact that in the public sector at least, it’s our money, paid in taxes, being wasted. The UK government has wasted tens of billions of pounds on failed IT projects over the last 20 years or so, and one huge step forward under the last coalition government was a more sensible approach to citizen service provision.
- Negative Gross Margins
We have seen a tremendous number of high growth companies raising money this year with negative gross margins. Which means they sell something for less than it costs them to make it.
It can be an "on-demand" service provider that subsidizes the cost of the workers on its platform so that the service seems like it costs less than it actually does. Why would an on-demand startup take this approach? To build demand for the service, of course. The idea is get users hooked on a home cleaning service, a ridesharing service, a food delivery service, or a gym roaming service by bringing it to market at a price point that is highly attractive and then, once the users are truly hooked, take the price up.
- A Pulitzer is no guarantee
There is a problem with the article. It correctly credits the Internet Archive with its major contribution to Web archiving, and analogizes it to the Library of Alexandria. But it fails to mention any of the other Web archives and, unlike Jill Lepore's New Yorker "Cobweb" article, doesn't draw the lesson from the analogy. Because the Library of Alexandria was by far the largest repository of knowledge in its time, its destruction was a catastrophe. The Internet Archive is by far the largest Web archive, but it is uncomfortably close to several major faults. And backing it up seems to be infeasible.
- The Little-Known Story Behind Britain's Road Signs
Kinneir and Calvert created rules for traffic signs that have endured to this day. Consider the wide gaps in letter spacing typically seen on roadside signs: That spacing is derived from research the designers conducted on how type should scale according to the speed of traffic and the amount of information on display. For Transport, the unit of measure for spacing is based on the width of the capital letter ‘I’—a consistency in form which, over time, helped foster a sense of familiarity in drivers.
- Best Haka Ever
The dueling Sipi Tau and haka prior to the first-round match between Tonga and New Zealand at Newcastle’s St James Park was the most scintillating, intense, and beautiful performance of the dance in modern rugby history.
Not only was it breathtaking, the dueling dances grounded the extraordinarily physical tone of the 80 minutes of rugby to follow. “We’re going to tell the whole world that God and Tonga is our inheritance,” Tongan center Siale Piatau explained before Tonga took the turf.
It was impressive to see the heart with which Tonga—a tiny and impoverished nation—performed the Sipi Tau dressed in their traditional luminous red jerseys that contrasted sharply with New Zealand’s black. The All Blacks response was incendiary, 23 men moving with a razor-sharp unity and collective purpose. The resulting rugby was worthy of the display. Tonga’s superb first half was probably the best they’ve ever played. Meanwhile, the All Blacks, rebounded from an unconvincing victory over surprise darlings Georgia to claim the win, overcoming the Tongans with graft, flair, and legs, scoring seven tries.
- How Prison Architect Could Liberate Gaming
After being available for three years as an open, prerelease “alpha,” Prison Architect was officially released two weeks ago and appears destined for long-term cult success. With a current user base of more than 1 million players, many of whom have already been playing for months if not years, the release carried significantly less risk for both players and creators than most project launches, its slow launch limiting its vulnerability to the caprices of the market and the media. I spoke to Introversion Software’s Mark Morris about the company’s approach to crowdfunding, project management, and community relations.
Thursday, October 22, 2015
Somewhere along the way, the Book Fairy (i.e., my youngest) dropped off a few books, and I've been making my way through them.
Recently, the choice was Anuradha Roy's The Folded Earth.
I didn't know much about The Folded Earth, nor about Roy; this is not surprising, I gather, as she's a new writer and perhaps not that well known.
She may be new, but she's certainly worthy of being well known: The Folded Earth is a superb novel.
It's set in modern times, in a small town in the Himalayan foothills of northern India. Our narrator, Maya, has found herself here somewhat accidentally, and is part of an assemblage of eccentric characters from all sorts of different backgrounds and heritages who find themselves living their lives in a most vibrant, vivid way.
Things happen, people come and go, Major Events Occur, the tragedy of Maya's life is slowly revealed and explored and absorbed, and before you even notice you're at the end of this marvelous book.
As you go, there are small mysteries and big ones. The biggest, of course, is Maya's, but there are many others. There is an amusing sub-plot involving Nehru and Lord Mountbatten, various commentaries on the changes underway in Indian society, side-trips into cultural traditions, and more, but through it all the impression that stays with me is the effortless grace of Roy's writing.
She has that sort of natural sense of who people are, how they behave, what they say, and why they do what they do, that makes you feel like you're right there, sitting on a bench, watching it all unfold, listening to the same words she's hearing, breathing the same air they're breathing.
It would be too much to claim that The Folded Earth is perfect, but then again I'm not sure Roy would take offense. Perhaps better, it has that feeling of being "right", that it-could-happen-here-and-in-fact-something-like-it-just-did feeling that you get from reading a story and realizing that, in a way, this story, like all the best stories, starts and ends with the people just down the street from you, living and loving and making their way through their lives.
If you get a chance to pick up one of Roy's books, I think you'd enjoy it; I'm certainly looking forward to following her career and reading more.
Monday, October 12, 2015
Does everything look like a nail?
- Uber's Move to Oakland Will Test an Economy in Overdrive
For non-residents, this expansion by Uber, a company that has already strong-armed its way into multiple countries and won scores of battles against regulators, might not sound especially significant. But Oakland is one of the most diverse communities in the US, a West Coast arts mecca, and the heart of California counterculture, from the peace movement to the Black Panther Party—an ingrained anti-establishment culture of protest that survives to this day. In that respect, Oakland seems like an especially awkward fit for Uber, the company whose capitalist ideologies and characteristic stubbornness when it comes to barging into new places and bending governments to its will has made it such a lightning rod for controversy.
- This Is How Uber Takes Over a City
Although Uber promotes itself as a great disrupter, it’s quickly mastered the old art of political influence. Over the past year, Uber built one of the largest and most successful lobbying forces in the country, with a presence in almost every statehouse. It has 250 lobbyists and 29 lobbying firms registered in capitols around the nation, at least a third more than Wal-Mart Stores. That doesn’t count municipal lobbyists. In Portland, the 28th-largest city in the U.S., 10 people would ultimately register to lobby on Uber’s behalf. They’d become a constant force in City Hall. City officials say they’d never seen anything on this scale.
- Lunch with the FT: Travis Kalanick
Amazon, once just an online bookstore, is often cited as a template for how Uber might develop. Jeff Bezos, Amazon’s founder and chief executive, is an Uber investor, though Kalanick rejects comparisons. “I think Uber is just very different, there’s no model to copy,” he says. “It may be the reason why we’ve been a lightning rod in so many ways, because we don’t do anything conventional . . . And then I think also, as an entrepreneur, I’m a bit of a lone wolf.”
- Twitter Thanks You For Your Service
That these are impossible guidelines is a hint at a coming conflict that I’m not sure anyone has quite figured out: as publications assimilate into platforms, and as platforms, in an effort to capture some of the energy and attention garnered by said publications on their turf, attempt to redefine the role of an editor/curator/reporter in terms that are most beneficial to them, we’ll have to re-litigate old arguments about control, bias, voice and balance, with higher stakes and less of a sense of accountability; the defendant, now, is simply insisting it isn’t liable. A first look suggests that the Moments approach to editorial judgement is learned, too: Its vague and primitive sense of what is and isn’t political (I’m sure opponents would challenge the unquoted “right-to-die” language here, for example); its implicit distinguishing between explicitly supporting or criticizing something, which is bad, and merely celebrating it, which is… fine?
- What fintech companies can learn from Uber
There’s also the issue of money itself, which is far more emotionally fraught than, say, what car service you’re going to use. “Money is really hard for people because it’s not just about the money,” said von Tobel. “They don’t want to face the fact that for most of us, we’re not in a great spot.” Fintech companies must build trust and community to break through the emotional wall that some people put up around their finances, said panelists.
- How Do You Value A Company Like Uber?
Though not a direct input into valuation, it is unquestionable that when investing in a young business, you should be aware of the management culture in that business. With Uber, the news stories about its management team and the responses to these stories reflect your prior opinions on the company. If you are predisposed to like the company, you will view it as confident in its attacks on new markets, aggressive in defending its turf and creative in its counter-attacks. If you don’t like the company, the very same actions will be viewed as indicative of the arrogance of the company, its challenging a status quo will signal its unwillingness to play by the rules and its counter attacks will be viewed as overkill.
- The Chicago End-Times
Miner tried to spin the news with a positive light. He credited Knight with bringing twelve pages of national USA Today content to the print paper, of which a former Sun-Times employee said, “I don’t think they care about having national content. They just want more pages, they wanted to be thicker, because they probably heard some feedback saying the paper’s too small. It came right on the heels of us cutting a bunch of people, and then you expand the newshole. They’re trying to do the same thing with the Network, which is demand that everybody write a billion stories a day because that’s the panacea. That’s how we’re gonna get pageviews, that’s how we’re gonna get cheap CPM ads, that’s how we’re gonna fix everything, right?”
- Lyft: It wasn't our CTO who cracked Uber's database
It turns out that the company itself leaked a database login key, onto the code-sharing platform GitHub, and nobody noticed until months had passed and 50,000 records - including names and driver license numbers - had been slurped.
... a story that "leaves the player in a very interesting and complicated situation—and gives him/her a wide range of different feelings and emotions": The Personal Story Behind The Witcher 3's Bloody Baron Quest
And though the player’s immediate response may be to condemn the Baron for his actions, the game spends a huge amount of time showing his remorse. It’s up to you to decide if he’s worthy of forgiveness.
"Because everybody should have the right to choose," said Sasko. "And besides, while as a player you control Geralt, in the end it’s still you making the decisions. And you should have the opportunity to decide who Geralt really is—meaning how he acts and responds in different situations, what he decides to do. It’s a role-playing game—to make it right and real, the player has to identify with the playable character—and how can this be done, if not by giving them the freedom to act?
"It’s also important to add that 'choice' is the heart of the Witcher games," said Stachrya. "When the player has to make a decision regarding a character that he likes and dislikes at the same time, he will hesitate. He will think it through, analyze all the information he was given and then chose—but rarely will he be certain that it was a good decision."
Hearts of Stone: T minus 48 hours and counting...
Sunday, October 11, 2015
Derby releases tend to come about once every 18 months nowadays.
Right on the schedule that was proposed last spring, the Derby 10.12 release is now available from the Derby web site.
Many thanks to Rick for doing the heavy lifting to make the actual release occur, and thanks to the entire Derby community for their contributions and encouragement!
I had some minor changes included in this latest release, which marked my return to a more active role in the Derby community after having been somewhat on the sidelines during 2012 and 2013 as I was busy with many other things.
Derby is a remarkably powerful and sophisticated piece of software, probably among the most complex and intricate pieces of open source software out there. Derby is not as complex as the Linux kernel, surely, but if you want to learn about how a database system works, or how you can write amazingly powerful software entirely in Java, studying Derby is the way to go.
Hopefully the Derby community can continue producing releases like this for some time. Corporate support for open source software is a fickle thing, and a piece of software as complex as Derby needs a certain amount of resources behind it.
Friday, October 9, 2015
Zoom zoom go the Blue Angels!
- The Kunduz hospital bombing: Terrible accident, or war crime?
It’s understandably extremely hard to believe that any American would deliberately bomb a hospital: it’s the exact opposite of everything that America stands for. But wherever there is war, there are war crimes, no matter who is doing the fighting. Any investigation into this bombing has to start from the assumption that it’s at least possible a dreadful war crime was committed. Given that so far no American has even admitted the possibility, the U.S. self-investigation really does look like a joke.
- Four more carmakers join diesel emissions row
“The issue is a systemic one” across the industry, said Nick Molden, whose company Emissions Analytics tested the cars. The Guardian revealed last week that diesel cars from Renault, Nissan, Hyundai, Citroen, Fiat, Volvo and Jeep all pumped out significantly more NOx in more realistic driving conditions. NOx pollution is at illegal levels in many parts of the UK and is believed to have caused many thousands of premature deaths and billions of pounds in health costs.
- SHA-1 Freestart Collision
We've long known that SHA-1 is broken, at least theoretically. All the major browsers are planning to stop accepting SHA-1 signatures by 2017. Microsoft is retiring it on that same schedule. What's news is that our previous estimates may be too conservative.
- Freestart collision for full SHA-1
We present in this article a freestart collision example for SHA-1, i.e., a collision for its internal compression function. This is the first practical break of the full SHA-1, reaching all 80 out of 80 steps, while only 10 days of computation on a 64 GPU cluster were necessary to perform the attack. This work builds on a continuous series of cryptanalytic advancements on SHA-1 since the theoretical collision attack breakthrough in 2005
- What Happens Next Will Amaze You
Those who control the data gain enormous power over those who don't. The power is not overt, but implicit in the algorithms they write, the queries they run, and the kind of world they feel entitled to build.
- Haunted By Data
The data we're collecting about people has this same odd property. Tech companies come and go, not to mention the fact that we share and sell personal data promiscuously.
But information about people retains its power as long as those people are alive, and sometimes as long as their children are alive. No one knows what will become of sites like Twitter in five years or ten. But the data those sites own will retain the power to hurt for decades.
- The Whining Of The Online Ad Industry
People are only mildly averse to advertising. They tolerate it in many forms in many media. What people hate is the type of ultra-annoying, creepy advertising that has been enabled by online ad tech.
- 16th International Workshop on High Performance Transaction Systems (HPTS)
Every two years, HPTS brings together a lively and opinionated group of participants to discuss and debate the pressing topics that affect today's systems and their design and implementation, especially where scalability is concerned. The workshop includes position paper presentations, panels, moderated discussions, and significant time for casual interaction. The only publications are slide decks by presenters who choose to post them.
- Audio Testing - Automatic Gain Control
In this particular case, the problem was not packet loss since we have ideal network conditions (one machine, packets go over the machine’s loopback interface = zero packet loss). But how can we have clock drift? Well, recall the fake device I wrote earlier that reads a file? It never touches the sound card like when the sound comes from the mic, so it runs on the system clock. That clock will drift against the machine’s sound card clock, even when we are on the same machine.
- Summary of the Amazon DynamoDB Service Disruption and Related Impacts in the US-East Region
Over the last few months, customers have rapidly adopted a new DynamoDB feature called Global Secondary Indexes (“GSIs”). GSIs allow customers to access their table data using alternate keys. Because GSIs are global, they have their own set of partitions on storage servers and therefore increase the overall size of a storage server’s membership data. Customers can add multiple GSIs for a given table, so a table with large numbers of partitions could have its contribution of partition data to the membership lists quickly double or triple. With rapid adoption of GSIs by a number of customers with very large tables, the partitions-per-table ratio increased significantly. This, in turn, increased the size of some storage servers’ membership lists significantly. With a larger size, the processing time inside the metadata service for some membership requests began to approach the retrieval time allowance by storage servers. We did not have detailed enough monitoring for this dimension (membership size), and didn’t have enough capacity allocated to the metadata service to handle these much heavier requests.
- Irreversible Failures: Lessons from the DynamoDB Outage
The worst outages stem from irreversible failures, where there is no simple compensating action you can make. In this DynamoDB incident, the initial triggering event—the network disruption—was quickly repaired. However, the system was now stuck in a state where the metadata service was overloaded. Fixing the network didn’t eliminate the overload. Amazon’s engineers were forced to invent a lengthy procedure, on the fly, to steer the system back to a stable state.
- Job System 2.0: Lock-Free Work Stealing – Part 3: Going lock-free
This week, we will finally tackle the heart of the job system: the implementation of the lock-free work-stealing queue. Read on for a foray into low-level programming.
- JEP XYZ: Spin Loop Hint
Provide an API that would allow Java code to hint to the runtime that it is in a spin loop. The API would be a pure hint, and will carry no semantic behavior requirements (i.e. a no-op is a valid implementation). Allow the JVM to benefit from spin loop specific behaviors that may be useful on certain hardware platforms. Provide both a no-op implementation and an intrinsic implementation in the JDK, and demonstrate an execution benefit on at least one major hardware platform.
- Probability, Paradox, and the Reasonable Person Principle
In this notebook, we cover the basics of probability theory, and show how to implement the theory in Python. (You should have a little background in probability and Python.) Then we show how to solve some particularly perplexing paradoxical probability problems.
- Git from the bottom up
Welcome to the world of Git. I hope this document will help to advance your understanding of this powerful content tracking system, and reveal a bit of the simplicity underlying it — however dizzying its array of options may seem from the outside.
- Git Internals: Source code control and beyond
This book is aimed at the developer who does not particularly like Subversion, Perforce or whatever SCM system they are currently using, has heard good things about Git, but doesn’t know where to start or why it’s so wonderful. It is meant to explain Git as simply as possible in a clean, concise, easily readable volume. My goal is to help you understand Git internals as well as usage at a fundamental level by the time you finish this book.
The government, at least partly out of genuine curiousity and other good intentions, has decided to get to the bottom of the VW scandal, and so they have called the executives before Congress to explain what went on.
I'm not sure why people outside the software industry find it so appalling that the head doesn't know what the arms and legs are doing.
Volkswagen's US CEO testified Thursday that the decision to use emissions cheating software was not made at the corporate level. Instead, it was "software engineers who put this in for whatever reason," Michael Horn told a congressional panel that is investigating the scandal.
This is nothing new.
In 35 years in the industry, I've never been in an organization where managers more than 1 or (rarely) 2 levels above the engineers have any idea what the individual contributors are talking about in their meetings and hallway discussions and email threads.
When asked how the cheating software worked, Horn said he didn't know. "Personally, no. I'm not an engineer," he said.
Software engineering is an extraordinarily complex mental activity, requiring you to construct elaborate, extensive abstract models in your mind, and visualize and comprehend how alterations to those models and their algorithms will change the system's behavior, and then to transfer those mental constructs into concrete embodiment as lines of program code that are compiled and executed by actual physical machines.
It's a very specific skill, and although it can be refined and honed over the years, it's not something you can dabble in.
You either do it, or you don't.
If you don't, you say,
"Personally, no. I'm not an engineer."
And then, that's where it ends.
You don't understand what the software is.
You don't understand how it works.
You don't understand where the strong points are, and where the weak points are.
Rarely do you even have any idea which parts of the software are associated with which engineers, and who works on what, and what particular strengths and weaknesses your engineers have, and how that relates to the particular strengths and weaknesses that your software, and hence your products, have.
You just have no clue. For you, it ends at:
"Personally, no. I'm not an engineer."
That's how the software industry works, that's how it has always worked for the 4 decades I've been part of it, and that may be how it will always work.
I'm not sure what this means for the world, but it's a fact.
Tuesday, October 6, 2015
... it's 168 hours to go until Hearts of Stone is released!
"You're ... immortal?"
"Indeed! But it's not what folk imagine."
Friday, October 2, 2015
Q4 really snuck up on me; these years are passing much faster than I remember from the past.
- Bitcoin and Cryptocurrency Technologies
After this course, you’ll know everything you need to be able to separate fact from fiction when reading claims about Bitcoin and other cryptocurrencies. You’ll have the conceptual foundations you need to engineer secure software that interacts with the Bitcoin network. And you’ll be able to integrate ideas from Bitcoin in your own projects.
- Classic Bug Reports
A bug report is sometimes entertaining either because of the personalities involved or because of the bug itself. Here are a collection of links into public bug trackers
- Code Words Issue Four
Issue Four of Code Words, our quarterly publication about programming, is now online!
- In-Memory Performance for Big Data
we enable buffer pool designs to match in-memory performance while supporting the "big data" workloads that continue to require secondary storage, thus providing the best of both worlds. We introduce here a novel buffer pool design that adapts pointer swizzling for references between system objects (as opposed to application objects), and uses it to practically eliminate buffer pool overheads for memory-resident data. Our implementation and experimental evaluation demonstrate that we achieve graceful performance degradation when the working set grows to exceed the buffer pool size, and graceful improvement when the working set shrinks towards and below the memory and buffer pool sizes.
- Understanding Distributed Analytics Databases, Part 1: Query Strategies
New analytics databases are designed to run across a cluster of machines. Instead of one supercomputer, your analytics database can run on dozens of commodity machines at the same time. This lets you achieve greater performance at a lower cost.
However, distribution comes with a new performance bottleneck. When all the data is on the same machine, the rate at which you can read and process data is limited by the speed of your hard drive.
In a cluster, the network is the limiting factor. The nodes in your analytics cluster need to share information because no single node has all the data. And a hard drive is over 3x faster than gigabit ethernet
- When Limping Hardware Is Worse Than Dead Hardware
So why should we care about designing systems that are robust against limping hardware? One part of the answer is defense in depth. Of course we should have monitoring, but we should also have systems that are robust when our monitoring fails, as it inevitably will. Another part of the answer is that by making systems more tolerant to limping hardware, we’ll also make them more tolerant to interference from other workloads in a multi-tenant environment.
- Limplock: Understanding the Impact of Limpware on Scale-Out Cloud Systems
In this paper, we highlight one often-overlooked cause of performance failures: limpware – “limping” hardware whose performance degrades significantly compared to its specification. The growing complexity of technology scaling, manufacturing, design logic, usage, and operating environment increases the occurrence of limpware. We believe this trend will continue, and the concept of performance perfect hardware no longer holds.
- 25th ACM Symposium on Operating Systems Principles
The biennial ACM Symposium on Operating Systems Principles is the world's premier forum for researchers, developers, programmers, and teachers of computer systems technology. Academic and industrial participants present research and experience papers that cover the full range of theory and practice of computer systems software.
- Holistic Configuration Management at Facebook
configuration changes help manage the rollouts of new product features, perform A/B testing experiments on mobile devices to identify the best echo-canceling parameters for VoIP, rebalance the load across global regions, and deploy the latest machine learning models to improve News Feed ranking. This paper gives a comprehensive description of the use cases, design, implementation, and usage statistics of a suite of tools that manage Facebook’s configuration end-to-end, including the frontend products, backend systems, and mobile apps.
- Building Consistent Transactions with Inconsistent Replication
In this paper, we use a new approach to reduce the cost of replicated, read-write transactions and make transactional storage more affordable for programmers. Our key insight is that existing transactional storage systems waste work and performance by incorporating a distributed transaction protocol and a replication protocol that both enforce strong consistency. Instead, we show that it is possible to provide distributed transactions with better performance and the same transaction and consistency model using replication with no consistency.
- Existential Consistency: Measuring and Understanding Consistency at Facebook
We use measurement and analysis of requests to Face- book’s TAO system to quantify how often anomalies happen in practice, i.e., when results returned by eventually consis- tent TAO differ from what is allowed by stronger consistency models.
- How to Get More Value From Your File System Directory Cache
This paper identifies several design principles that can substantially improve hit rate and reduce hit cost transparently to applications and file systems. Specifically, our directory cache design can look up a directory in a constant number of hash table operations, separates finding paths from permission checking, memoizes the results of access control checks, uses signatures to accelerate lookup, and reduces miss rates through caching directory completeness.
- Cross-checking Semantic Correctness: The Case of Finding File System Bugs
We applied JUXTA to 54 file systems in the stock Linux kernel (680K LoC), found 118 previously unknown semantic bugs (one bug per 5.8K LoC), and provided corresponding patches to 39 different file systems, including mature, popular ones like ext4, btrfs, XFS, and NFS. These semantic bugs are not easy to locate, as all the ones found by JUXTA have existed for over 6.2 years on average.
- Read-Log-Update: A Lightweight Synchronization Mechanism for Concurrent Programming
This paper introduces read-log-update (RLU), a novel exten- sion of the popular read-copy-update (RCU) synchronization mechanism that supports scalability of concurrent code by allowing unsynchronized sequences of reads to execute concurrently with updates. RLU overcomes the major limitations of RCU by allowing, for the first time, concurrency of reads with multiple writers, and providing automation that eliminates most of the programming difficulty associated with RCU programming.
- The Beginner's Guide is a game that doesn't want to be written about
It's difficult to tell at first exactly what The Beginner's Guide is supposed to be: a tribute, a eulogy, a motivational speech. Wreden says several times that Coda stopped making games in 2011 and that he hopes one day his old friend will create again. It's an impulse we see a lot on the internet these days, particularly in fan culture: the desire to write a paean so beautiful that it can bring the things we've lost back from the dead. And make no mistake, Wreden is Coda's number one fan. There are parts of this game that feel uncomfortably grasping, that want very badly to be a resurrection spell of sorts, though it takes a while to figure out exactly what has died—or why.