Kinds of book reviews

There are a bunch of kinds of book reviews.

There are book reviews like the ones I write. When I write a book review, I'm just writing it for myself, because I feel the need to put down in words some thoughts about a book I read. I don't expect anybody else to read them, or find them worth reading, but sometimes they do. Many of the book reviews on a site like Amazon are like this.

But the majority of book reviews are part of the core functioning of the marketplace of books, to coin a phrase. Readers of books consume reviews because they are trying to find books they would like to read. Writers of books consume reviews because they are trying to understand what makes books work, or fail, and how they can incorporate that into their own writing. Publishers, distributors, catalogers and other middlemen and merchants in the marketplace of books consume (and sponsor) reviews because they are interested in the overall business of books: which books are popular and why? what needs are being met, and what are being overlooked? which opportunities for books are available, and which are over-saturated? etc.

I think that the vast majority, almost exclusively all, of the book reviews in the world, are one or the other of the above two kinds.

And increasingly, as the Internet dis-intermediates the marketplace of books, the above two categories are merging.

But there are, at least, two other kinds of book reviews.

One kind of book review is the "Important Person Making a Statement on Matters of Consequence, and Using a Book Review as the Vehicle." Think, say, of Seymour Hersh's recent Bin Laden bombshell. Or Leon Wieseltier's diatribe against technology.

Generally, these Deep Thoughts Disguised as a Book Review annoy me, and I give them a wide berth. But they are regular occupants of the world of book reviews, so I have to keep my eyes open for them and recognize them early enough to dodge out of the way.

A recent example of this (to me) is Cythia Ozick's observations on Harold Bloom's latest book: ‘The Daemon Knows,’ by Harold Bloom (titled "Shared Visions" in the print edition). Ozick clearly wants to be part of "the conversation" (the "Shared Vision") about Bloom's book:

If, as Emerson claims, the true ship is the shipbuilder, then is the true poem the critic who maps and parses and inhabits it? Can poet and critic be equal seers?

Ozick goes on:

It is through intoxicating meditations such as these that Bloom has come to his ­formulation of the American Sublime, and from this to his revelation of the daemon: the very Higgs boson of the sublime.

And later

So when Bloom tells us there can be no critical method other than the critic himself — meaning Bloom — we should not take it as blowhard hyperbole. With Emerson, he intends to pry open the unpossessed and to possess it, and to lead the reader to possess it too: a critical principle rooted in ampleness and generosity.

Uhm, I'm sorry: I'm sure there was something important being discussed here, but it was all "blowhard hyperbole" to me, like one of those dinner parties where, no matter how hard you try, the topic of conversation is entirely in an alien tongue (the "Higgs boson of the sublime"?).

Yet there is one final kind of book review.

This is the review where the reviewer is so passionate about the book, so entranced, so enthralled, so consumed, that all the reviewer can stand to do is to jump up, throw their arms wide, and shout out their joy at having found and read this book. Their passion and delight is brilliantly evident, and leaps out of the book review, drawing you in, infecting you with their own eagerness and joy.

These are the book reviews I always hope to find.

Such a review appeared recently, in the New Yorker, in Katheryn Schulz's review of Helen Macdonald's H is for Hawk: Rapt: Grieving with your goshawk.

Schulz describes her apprehension about the book, fretting that it would be, as Macdonald herself describes it, "the most appalling falconry bore":

Listen to her now, two pages in:

Maybe you’ve glanced out of the window and seen there, on the lawn, a bloody great hawk murdering a pigeon, or a blackbird, or a magpie, and it looks the hugest, most impressive piece of wildness you’ve ever seen, like someone’s tipped a snow leopard into your kitchen and you find it eating the cat.

So much for the falconry bore. Intellectually, Macdonald is unhurried—she pauses to point out whatever is interesting—but, stylistically, she is like this passage, all pounce. Over and over, her writing takes you by surprise: no sooner have you registered the kitchen than, whoa, there’s the snow leopard, its huge Himalayan paws leaving prints on the tile and half a domestic shorthair hanging from its mouth.

Yes! Exactly! This is how a reviewer grabs you by the shirt collar and shoves a book into your face, saying "Read! This! Now!"

And lest you worry that you'll be wasting your time with the book, Schulz is careful to alert you to the fact that this book is not just a joy to read, but worth reading, as well:

Macdonald, who is writing both kinds of book at once, makes neither mistake. She is intimate and moving on the anguish that carried her into the company of hawks, but the world of her book is like the world we really live in, crowded with humans and human ideas, and she turns on it all the triple perspicacity of a poet, a naturalist, and a historian. She dissects the cultural symbolism attached to hawks from Victorian England to the Third Reich; she catalogues the classic animal stories by gay authors, who could not write openly of their human relationships; she observes that when a species is endangered it suffers not only numeric but also semantic decline. “The rarer they get, the fewer meanings animals can have,” she observes. “Eventually rarity is all they are made of.”

It just so happened that I picked up a few other books first, but H is for Hawk is sitting there on my Kindle, waiting for me, and I know I'm not going to miss it; Schulz has made certain of that.

Some people say that the Internet is the end of reading and writing, and that all we can do now is formulate 140 character vocal burps and exchange pictures of kittens.

Which often seems to be closer to the truth than I'd like.

But then along comes Katheryn Schulz to arrest my despair, to keep me engaged, to draw me in, and to whet my appetite.

That, then, is the kind of book review that keeps me reading book reviews.

In which people discuss things I don't understand.

The discussion is certainly not boring.

• Socialize Uber: It’s easier than you think.

  Given that the workers already own all the capital in the form of their cars, why aren’t they collecting all the profits? Worker cooperatives are difficult to start when there’s massive capital needed up front, or when it’s necessary to coordinate a lot of different types of workers. But, as we’ve already shown, that’s not the case with Uber. In fact, if any set of companies deserves to have its rentiers euthanized, it’s those of the “sharing economy,” in which management relies heavily on the individual ownership of capital, providing only coordination and branding.

• How to Socialize Uber

  Uber promises investors it will soon be making mega-profits, but it also claims those profits just represent a return on its technology and risk-bearing. Certainly the money doesn’t come from exploiting Uber’s workers. What workers? No, no — you see, the drivers are merely Uber’s business partners, and you can’t exploit your business partner.

• Once a sure bet, taxi medallions becoming unsellable

  In an April letter to creditors, New York taxi commission officials and other stakeholders, Freidman's attorney, Brett Berman, called on industry regulators and medallion lenders to restructure and extend loans for his client and reform the industry.

• Billionaire hedge-fund manager says Uber told him it might cut driver pay ‘because we can'

  "'You've got happy employees, you've got happy customers, you've got happy shareholders. The holy triumvirate are all really excited about your company. Why are you going to risk that and push the employees salary down 5%?'"

  Callinicos simply responded "because we can."

• Elizabeth Warren: No Need to Stop Uber-ized Workforce, but Must Invest in Education

  she returned to her argument, made several times during the interview, that the government’s position should pour more investment into education and infrastructure. “We have to invest in the two places where it works,” she said. “We have to invest in brains and people who are willing to do the long, long arc research.”

• This lawyer fought for FedEx drivers and strippers. Now she's standing up for Uber drivers

  By using contractors instead of employees, companies are not responsible for things like payroll taxes, job expenses, anti-discrimination protections or overtime pay. For bootstrapped startups, it's a cost-saving measure that can mean life or death.

  But Liss-Riordan isn't drinking the same venture-capital bought kool-aid as the startups who have built businesses around the 1099 economy. Rather, she views it as another example of companies using contract workers as a way to skirt their obligations as an employers.

  "I don’t believe this industry needs to be built on a system whereby the workers don’t need to receive any of the protections that we have a society that workers need to receive," she said. "I just don’t know how Uber can argue with a straight face that as a $40 billion dollar company it can’t afford to insure its drivers, pay minimum wage or pay overtime, or be reimbursed for their expenses.

• Uber Isn’t the Problem

  If drivers on the Uber platform had better options available to them, if there were jobs that offered them higher wages and better working conditions, they’d presumably have already taken them. That means that if you’re appalled by Uber, your real problem is with every other option that the drivers who use it have for earning a living—which is entirely fair. But despair over the fact that many American workers aren’t commanding the wages and working conditions we’d want for them in an ideal world doesn’t seem like a sound reason for shutting Uber down, or regulating it out of existence.

Stuff to read as May turns to June

Mother Nature apparently thinks that May is turning to April, or maybe March. But that's OK, too.

• Internet Trends 2015

  Consumers’ Expectation That They Can Get What They Want With Ease & Speed Will Continue to Rise...

  This Changes Fundamental Underpinnings of Business & Can Create Rising Demand for Flexible Workers

• Using Computer Vision to Increase the Research Potential of Photo Archives

  Collaborating with the Frick Art Reference Library I utilized TinEye’s MatchEngine image similarity service and developed software to analyze images of anonymous Italian art in their photo archive. The result was extremely exciting: it was able to automatically find similar images which weren’t previously known and confirm existing relationships. Analysis of some of the limitations of image similarity technology was also conducted.

• A Toolkit to Measure Basic System Performance and OS Jitter

  To complement the great information I got on the “Systematic Way to Find Linux Jitter”, I have created a toolkit that I now used to evaluate current and future trading platforms.

  In case this can be useful, I have listed these tools, as well as the URLs to get the source code and a description of their usage. I am learning a lot by reading the source code, and the blog entry associated.

• Systematic Process to Reduce Linux OS Jitter

  Based on empirical evidence (across many tens of sites thus far) and note-comparing with others, I use a list of "usual suspects" that I blame whenever they are not set to my liking and system-level hiccups are detected. Getting these settings right from the start often saves a bunch of playing around (and no, there is no "priority" to this - you should set them all right before looking for more advice...).

• New C++ experimental feature: The tadpole operators

  Visual Studio 2015 RC contains a pair of experimental operators, nicknamed tadpole operators. They let you add and subtract one from an integer value without needing parentheses.

• The tadpole operators explained

  The __ENABLE_EXPERIMENTAL_TADPOLE_OPERATORS is just a red herring.

• ​The Counselor

  In March 2015, I&A commissioned authors to write a series of narratives that investigated near future concerns around intelligent systems in warfare, urban design, medicine, and labor. These stories served as the centerpiece of a two-day intensive forum bringing together participants to identify the core set of challenges that consistently arise in deploying intelligent systems regardless of arena. "The Counselor," by Robin Sloan (Mr. Penumbra's 24-Hour Bookstore) focuses on the persuasive qualities of these systems in the medical context.

• Is there a way with Git to make future merges ignore version number difference in a pom file between branches?

  I am trying to find a way to make Git ignore pom version differences between branches. This works well in Perforce and I'm not having any luck reproducing the behavior with Git.

• Post Traumatic Crash Disorder & the 1962 Flash Crash

  The enormous impact of 1929 and the Great Depression had outsize and lasting effects that haunted investors decades later.

• Soccer Superpower

  The economic might of the United States in international soccer has indeed been realized: American sponsors, broadcasters, marketers, and apparel companies have funneled billions of dollars into the game; a pro league is thriving; millions of Americans watch European matches on television; and hundreds of thousands will attend European exhibition games on U.S. soil this summer. But the soccer cronies who have fed at the revenue trough didn’t anticipate the consequences of courting all that American money: It gave America the power to bring them down.

• Everything You Need to Know About FIFA’s Corruption Scandal

  The Justice Department’s announcement primarily cites deals between FIFA, sports marketing groups, and broadcast corporations for the television rights to air the World Cup and other international soccer tournaments. Dating back to 1991, the indictment alleges, those involved conspired to receive bribes from marketing firms in exchange for exclusive television contracts—to the cumulative tune of more than $150 million. As Attorney General Loretta Lynch stated, “It spans at least two generations of soccer officials who, as alleged, have abused their positions of trust to acquire millions of dollars in bribes and kickbacks.”

Warriors advance again!

For the first time in the 30 years that I've lived in Northern California, the Golden State Warriors are in the NBA finals!

The series against Houston was funny: there were 3 very close games and 2 massive blowouts. For some reason it felt like the series against Memphis was more challenging for the Warriors; I think that was partly because Houston was exhausted after the effort they had to produce to get past the Clippers, and partly because the Warriors, being a very versatile team, match up better against Houston, who have trouble adjusting as rapidly to different styles as do the Warriors.

Now comes their hardest test, playing the Cleveland Cavaliers and LeBron James, who is as dominant and magnificent an individual player as has ever existed in the NBA, at least since Wilt Chamberlain was playing. (Pretty-well-known trivia fact: when Wilt Chamberlain scored 100 points that strange night in Hershey Pennsylvania, he was playing on the (Philadelphia) Warriors team.)

Although the media will be all about Stephen Curry vs. LeBron James, that's not really how the finals will play out.

The real issue will be LeBron James versus the Warriors defense, which means the players you need to start getting familiar with are the ones you might not have paid so much attention to: Draymond Green, Andre Iguadala, Harrison Barnes, Festus Ezeli, Klay Thompson.

There is no single player who can come close to guarding LeBron James, but the Warriors don't play defense with a single player, so I think there is hope.

We'll have to wait a week to see; the finals start Thursday, June 4!

A collection of John and Alicia Nash articles

Some of these, in particular the Peter Woit article, have some wonderful additional contributions in the comments sections.

And thank you to those authors who took the time to recognize that there were two very special people in the back seat of that taxi.

• John F. Nash Jr., Math Genius Defined by a ‘Beautiful Mind,’ Dies at 86

  Dr. Nash’s theory of noncooperative games, published in 1950 and known as Nash equilibrium, provided a conceptually simple but powerful mathematical tool for analyzing a wide range of competitive situations, from corporate rivalries to legislative decision-making. Dr. Nash’s approach is now pervasive in economics and throughout the social sciences and applied in other fields as well, including evolutionary biology.

• John Nash 1928-2015

  During the years I was a graduate student in Princeton, Nash was often to be seen, especially in the mathematics/physics library, and I talked to him a few times. The first time was when he stopped me one day, told me he had seen my name on the physics department picture board, and was curious about the origin of my last name.

• John and Alicia Nash, 1928,1933–2015

  Alicia’s life was one of boldness and courage from the day she emigrated with her family from El Salvador in 1944. Her father had followed her uncle fearing backlash against their aristocratic family from a popular insurrection. Hearing about the nuclear bomb on the radio inspired her to become an atomic physicist, and she worked hard to become one of only seventeen women in MIT’s class of 1955.

• John Nash, mathematician portrayed in A Beautiful Mind, dies in taxi crash at 86

  Nash describes thinking himself out of his illness.

  “After my return to the dream-like delusional hypotheses in the later 60s I became a person of delusionally influenced thinking but of relatively moderate behavior and thus tended to avoid hospitalization and the direct attention of psychiatrists,” Nash wrote.

  “Thus further time passed. Then gradually I began to intellectually reject some of the delusionally influenced lines of thinking which had been characteristic of my orientation. This began, most recognizably, with the rejection of politically oriented thinking as essentially a hopeless waste of intellectual effort.”

• John Forbes Nash, 1928–2015

  On the evening before the award ceremony John Nash was introduced to World Chess Champion Magnus Carlsen. You can watch a video of the encounter in this NRK report. Nash says that he encouraged his son (who was also diagnosed with schizophrenia, to play chess and asks Magnus whether he thinks the game could be good for mental health. "I think it keeps the mind active, I suppose," replies Magnus, who goes on to sign a chessboard for Nash.

• Eisgruber: Princeton saddened over reported deaths of John Nash and wife

  The University community is "stunned and saddened" upon hearing news reports that Princeton mathematician John Nash and his wife, Alicia, were killed in a traffic accident, President Christopher L. Eisgruber said Sunday.

• A Beautiful Mind, an Amazing Couple: Community Mourns the Loss of John and Alicia Nash

  It was common for Princeton area residents to see John Nash taking his beloved Dinky train between his home in Princeton Junction and downtown Princeton. At 86, he was still a daily fixture on the Princeton University campus and was often seen in the Princeton U Store.

• The Lost Years of a Nobel Laureate

  Then came what Professor Kuhn calls "a miraculous remission." And as happens, for reasons unknown, in the case of some people with schizophrenia, it was not, according to Mrs. Nash or Mrs. Legg, due to any drug or treatment.

  "It's just a question of living a quiet life," said Mrs. Nash.

  The most dramatic sign of that remission, perhaps, is that Mr. Nash was able to do mathematics again.

• Nash and the NSA

  Not only did John Nash think about computation and cryptography, there are many ideas in these letters that were a bit ahead of their time when Nash sent these letters in 1955.

  • Expressing a cryptographic process as a Boolean function with input bits.
  • Breaking the cryptographic system as a function of the key length.
  • Exponential in key length as computationally hard and polynomial in key length is computationally easy.

• National Cryptologic Museum Opens New Exhibit on Dr. John Nash

  The National Cryptologic Museum's newest exhibit, "An Inquisitive Mind: John Nash Letters," features copies of correspondence between Dr. Nash and the National Security Agency (NSA) from the 1950s when he was developing his ideas on an encryption-decryption machine.

I don't take taxi rides often, but I must try to train myself to put on my seat belt even in the taxi.

Sigh.

Stephen Curry is playing at an astonishing level

I was never a good basketball player, but I love to watch the game; I think it is the best televised sport.

And Stephen Curry is playing at a simply astonishing level.

The last time I saw a player playing at this level, it was 1991, and his name was Michael Jordan.

They're very different players, but Curry is really doing something remarkable.

Oh, and by the way: Curry's current coach, the one who seems to have figured out how to elevate Curry's play that final little bit until he's truly above and beyond anyone else around him on the court?

None other than Steve Kerr, who was Jordan's teammate during the great "3-peat" years of 1995-1997.

Astonishing talent, coupled with a superb coach: total joy to behold.

This is why sports, when it finds that magical something, is the most fascinating entertainment.

Stuff to read, late May edition

Happy Memorial Day, a little bit in advance.

• Silicon Valley Is a Big Fat Lie

  What was once a land of upstarts and rebels is now being led by the money-hungry and the unspirited. Which is why we have a start-up that mails your dog curated treats and an app that says "Yo." The brightest minds in tech just lately seem more concerned with silly business ideas and innocuous "disruption," all for the shot at an immense payday. And when our country's smartest people are working on the dumbest things, we all lose out.

  That gap between the Silicon Valley that enriches the world and the Silicon Valley that wastes itself on the trivial is widening daily. And one of the biggest contributing factors is that the Valley has lost touch with reality by subscribing to its own self-congratulatory mythmaking.

• Google systems guru explains why containers are the future of computing

  Inktomi and also early Google ended up using essentially a Unix process model and doing everything in terms of processes, running many processes on the same piece of hardware. In fact, Google didn’t use virtual machines really at all until it started doing some corporate stuff where it wanted to run third-party things. But all the internal stuff never used VMs.

• A Repository with 44 Years of Unix Evolution

  The evolution of the Unix operating system is made available as a version-control repository, covering the period from its inception in 1972 as a five thousand line kernel, to 2015 as a widely-used 26 million line system. The repository contains 659 thousand commits and 2306 merges.

• Microsoft, Salesforce talks fizzled over price: Sources

  The deal envisioned Microsoft using a significant portion of its $95 billion cash pile to pay for Salesforce, but there was discussion of allowing Benioff to roll his 5.7 percent stake in Salesforce into Microsoft stock, while other shareholders would have gotten paid in cash. Benioff would have had a management role at Microsoft under the deal, according to people close to the talks.

• Top 10 data mining algorithms in plain English

  Once you know what they are, how they work, what they do and where you can find them, my hope is you’ll have this blog post as a springboard to learn even more about data mining.

• Why should I have written ZeroMQ in C, not C++ (part I)

  While the possibility to handle the exceptions differently in different contexts may seem appealing at the first sight, it quickly turns into a nightmare.

  As you fix individual bugs you'll find out that you are replicating almost the same error handling code in many places. Adding a new function call to the code introduces that possibility that different types of exceptions will bubble up to the calling function where there are not yet properly handled. Which means new bugs.

• Why should I have written ZeroMQ in C, not C++ (part II)

  The real reason why any C++ programmer won't design the list in the C way is that the design breaks the encapsulation principle: The implementer of the "person" class has to know that its instances will be eventually stored in "people" list. Moreover, if a 3rd party chooses to store it in a different list, the implementation of the person would have to be change. This is an anti-pattern that OO programmers learned to avoid.

  However, if we can't place the "prev" and "next" fields into "person" class, we have to put them elsewhere. Thus, there's no other option but to allocate a helper object, the way std::list<> does.

• Don’t Stack Your Log On My Log

  Our work investigates the impacts to performance and endurance in flash when multiple layers of log-structured applications and file systems are layered on top of a log-structured flash device. We show that multiple log layers affects sequentiality and increases write pressure to flash devices through randomization of workloads, unaligned segment sizes, and uncoordinated multi-log garbage collection. All of these effects can combine to negate the intended positive affects of using a log.

• How Could a Flash Cache Degrade Database Performance Rather Than Improve It? Lessons to be Learnt from Multi-Tiered Storage

  Contrary to intuition, host-side flash caches can degrade performance rather than improve it. With flash write operations being expensive, cache hit-rates need to be relatively high to offset the overhead of writes. Otherwise, the end-to-end performance could be worse with flash cache.

Apache status changes

Among other things, I realized that I'm coming up on my 9th year as an Apache committer, my 8th year as a participant in the Google Summer of Code, and my 11th year, overall, being a part of the open source community at the Apache Software Foundation.

And now there's another change, as I'm now in the (largely administrative, but still interesting) new role: "Chair, Apache DB project."

I'm following in a long list of Apache DB chairs: Rick Hillegas, Jean Anderson, Kristian Waagan, Myrna van Lunteren, others whose history I'm unaware of).

As I said above, it's really just an administrative thing, but there you go: Who runs the ASF? (search for Apache DB)

Google Summer of Code 2015 is underway!

This is the 10th year of the Google Summer of Code, though I have only participated in 7 years, I believe. (It would have been my 8th year, but one year I messed up the paperwork and had to miss that year.)

This year, I'm pleased to be working with Abhinav Gupta, who is studying Information Technology at Indian Institute of Information Technology, Allahabad, India.

This summer, we will be working on fixing some of the open bugs in Apache Derby.

Derby is a very mature and sophisticated piece of software by now, and all the easy bugs have been fixed.

Which is fine with us; fixing the hard bugs is fun, too, and there's always more to learn!

Here's to a successful summer for all the Google Summer of Code participants.

Dean Potter died

Not a lot of information, but I guess it's pretty clear what happened, in broad strokes: Dean Potter Killed in BASE-Jumping Accident

Yosemite Search and Rescue (YOSAR) initiated a hasty search, but the rangers were unable to locate the pair overnight. Potter and Hunt had been attempting to fly along terrain that required them to clear a notch in a rocky ridgeline. “It’s kind of a trickier flight to go through this notch,” Gauthier says. On Sunday morning, a state police helicopter was able to spot both bodies from the air. No parachutes had been deployed. Two rangers were then airlifted to the site to perform the recovery.

It would appear that Clif Bar was accurate in their assessment of his activities.

There's more, if you don't already know who Dean Potter is, everywhere. The National Geographic Adventurer's Blog has this: Pioneering Climber Dean Potter Killed in Wingsuit BASE Jumping Accident

An observer shooting photos of Hunt and Potter’s flight reported hearing two disconcerting, loud sounds in succession that suggested impact—but also could have been parachute deployment.

What a shame.

Details, details

I pre-ordered Seveneves.

I thought it came on May 15th.

My bad.

It will arrive on May 19th.

Just a few more days to wait.

Too much piano jazz?

We played the Koln Concert at my wedding, and frankly I've been kind of a piano jazz convert ever since.

We listen to lots of Keith Jarrett, but recently I've been branching out a bit.

I completely love Joe Sample, and can never get enough.

Bill Evans may be my overall favorite, and I have many of his recordings.

When I was younger I loved Dave Brubeck, and I still listen to lots of Brubeck; he is from Berkeley after all.

And I've got Oscar Peterson, Chick Corea, and several others in the regular rotation.

Recently, though, it's been a lot of Kenny Barron, who I think doesn't get enough credit.

I'm not sure any of them can hold a candle to Beethoven's Op. 10, second movement, but then who cares?

It's all beautiful music, and good to listen to after a hard day of slinging code.

The wandering mind ...

... finds plenty to distract it on the Internet.

• Fabiano Caruana Returns to Team USA

  Current World #3 Fabiano Caruana announced today that he will switch federations to represent the United States in international competitions. Born in Miami and raised in New York City, the 22 year old and his family moved to Europe to further his career as a budding chess professional.

• Fabiano Caruana to Join U.S. Chess Federation

  The United States Chess Federation (USCF) today announced that Grandmaster Fabiano Caruana has initiated the application process to change chess federations, which, when approved, will allow him to play for the United States. Fabiano, who has dual citizenship in the United States and Italy, has been playing for the Italian Chess Federation for a number of years.

• 2015 Click Clean Report

  Greenpeace has been evaluating the energy demand of the Internet, and the energy choices made by individual Internet companies, since 2010.

  See the companies that host much of the Internet in their data centers, and whether they’re going green or not.

• Greenpeace, Renewable Energy, and Data Centers

  The best way to address these small deployments that make up the bulk of the problem is to move them to cloud computing. Suddenly the utilization goes up, they are immediately running on very current, high-efficiency technology, it frees up office space in their facility, and they no longer need to overdrive office cooling systems not designed to cool data centers efficiency to cool servers.

• Beekeepers see 42% of US honeybee colonies die off in a single year

  Since April 2014, beekeepers lost 42.1% of their colonies, the second-highest rate in nine years, according to an annual survey conducted by a bee partnership that includes the US Department of Agriculture.

• Chimamanda Ngozi Adichie: 'Fear of causing offence becomes a fetish'

  Using the contrast between Nigerian and American hospitals as an example, Adichie pointed out that Americans like to be “comfortable”. And she worried that the comfort has brought “dangerous silencing” into American public conversation. “The fear of causing offence, the fear of ruffling the careful layers of comfort, becomes a fetish,” Adichie said. As such, the goal of many public conversations in the United States “is not truth … [it] is comfort”.

Juve v. Messi

A very nice set of semi-finals, and now the final is set.

Barcelona somewhat coasted over an unexpectedly injured Bayern Munich, while Juventus gritted out a tough and spirited pair of matches with Real Madrid.

Surely Barcelona will be the massive favorites heading in to next month's final.

But the Juventus team is solid, confident, and capable. They played as a team and looked very strong in both matches.

And they weren't a boring team, either; they were fast and well-coordinated.

I am hopeful that Juve can give Barcelona a good competition, and surely after going up against the fearsome Real Madrid offense (at one point Madrid had Christiano Ronaldo, Gareth Bale, Javier Hernandez, James Rodriguez, and Marcelo all in attack at once!) Juve must feel that they can go head-to-head with the Messi/Neymar/Suarez line.

Make your plans now!

June 6th, Berlin.

Stuff I'm reading, mid-May edition

We've entered the seasonal pattern: heavy overcast until about 1 PM, then a glorious and beautiful afternoon.

• The Discovery of Apache ZooKeeper’s Poison Packet

  After many, many packet captures, we were able to find a single problem packet. It contained a scheme_len of 0x6edd0b51… or about 1.7GB. The lack of a bounds check resulted in ZooKeeper trying to allocate memory for the bogus length, which causes an OutOfMemory exception to be thrown, killing the thread. Cool. Well, not so cool, but now we’re starting to get somewhere. There are still so many questions, but the most pertinent problem is clear: if the leader is dead, why doesn’t it get re-elected?

• Weak Homegrown Crypto Dooms Open Smart Grid Protocol

  The weaknesses discovered by Jovanovic and Neves enabled them to recover private keys with relative ease: 13 queries to an OMA digest oracle and negligible time complexity in one attack, and another in just four queries and 2^25 time complexity, the paper said.

  "A different approach only requires one arbitrary valid plaintext-tag pair, and recovers the key in an average of 144 message verification queries, or one ciphertext-tag pair and 168 ciphertext verification queries," the researchers wrote.

• Dumb Crypto in Smart Grids: Practical Cryptanalysis of the Open Smart Grid Protocol

  The OMA digest algorithm presents multiple flaws. Firstly, it uses a simple zero byte message padding, which results in messages with any number of trailing zeroes sharing the same tag. Secondly, given a tuple (a, m, k) where a is the OMA digest’s state or authentication tag, m a message and k the OMAK, the function is fully reversible (see Algorithm 2) which is a very useful property for the attacks presented in Sections 3.1 and 3.2. Likewise, it is also possible to take an arbitrary internal state, and continue to process it as if to resume a partially digested message.

• Python Packaging: Hate, hate, hate everywhere

  What I learned of that personally is not so much anything about packaging Python code but to not make any attempts to replace existing infrastructure without understanding all the reasons that lead to its existence. Also since that happened in the past I think it's a good idea to write down a list of design decisions and use cases and why they exist when I make another open source project in the future. A lot of what went into setuptools can only be understood after a long time of using it because the design is not documented enough.

• go get considered harmful

  This is the practice (of which Godeps is a variant) of sticking all your dependencies in your source code repo and committing them. Which means that upstream changes need to be individually downloaded to each source repo. Godeps at least gives you a way to list your packages, which is a start.

  But you don’t want every upstream commit in your repo’s commit history, so this is typically done by just blowing away the commit history. Or you end up with git submodules, which are brittle as hell.

  And if there are bugs in the upstream, now you have to re-vendor that package for every case you use it across all your repos, instead of just bumping a version number somewhere and testing.

• Notes on Theory of Distributed Systems CS 465/565: Spring 2014

  These are notes for the Spring 2014 semester version of the Yale course CPSC 465/565 Theory of Distributed Systems.

• In Mathematics, Mistakes Aren’t What They Used To Be

  Voevodsky decided that the only way he could proceed would be to employ a "proof assistant"—a computer program that would prevent him from wandering too far down errant paths. He had already been investigating the prospects of a computer-assisted formalization of mathematics, since 2003. The error, he says, "was just like a little present for me, to make my argument stronger."

  The experience of working with a proof assistant, as he describes it, is a bit like playing a video game. He’s jousting with the computer. He instructs it to try this, try that—he can even instruct it to try something known to be wrong, just to see what happens. "It’s very interactive," he says. "It can be fun and exciting, and kind of addicting." When he makes a mistake, all the assertions he’s typed in, lines and lines and lines, vanish before his very eyes (though he saves the work in a file called "leftovers").

  This way, once Voevodsky has proved something, he knows for certain it’s true. He doesn’t wonder if there are any mistakes lurking within arguments, and he doesn’t worry about how to convince his colleagues. The proof assistant provides a stamp of authenticity.

• Fast Approximate Logarithms, Part I: The Basics

  You can find code for approximate logs on the web, but they rarely come with an evaluation of how they compare to the alternatives, or in what sense they might be optimal. That is the gap I'm trying to fill here. The first post in this series covers the basics, but even if you are familiar with this subject I think you will find some interesting nuggets. The second post considers rounding error, and the final post gives the code for a family of fast log functions.

• MIT alumni in their 50s

  The engineers who’d chosen to work in industry, however, were a varied lot. A woman who’d taken a job at a defense contractor was still there, 30 years later. The super-wizard Lisp Machine programmer was now in a senior technical, but non-supervisory role, at a multi-billion dollar dotcom (not necessarily getting paid more than a competent 30-year-old, however). About half of the engineers, however, talked about being pushed into a financially uncomfortable early retirement and/or not being able to find work.

• I would have hired Doug, but...

  Unlike Engelbart, I have re-tooled. I now work in JavaScript in the browser and on the server. I had to walk away from the codebase that I loved. I understood that the price of relevance is to give up fighting at some point and settle for a partial victory.

• Globalization unraveling

  There are myriad reasons why these markets have lagged, some of them unique to specific countries or regions. For instance, China’s one-child policy has produced a penurious generation of young adults who are the sole support for aging family members. And in parts of Southeast Asia and Africa the infrastructure in rural areas, where much of the population lives, is too primitive to support extensive retail activities. But equally problematic is that the growth of the middle class in China and most other developing economies has been slow. And these newly minted consumers face volatile, often expensive prices for housing, food and other staples.

• The Killing of Osama bin Laden

  The major US source for the account that follows is a retired senior intelligence official who was knowledgeable about the initial intelligence about bin Laden’s presence in Abbottabad. He also was privy to many aspects of the Seals’ training for the raid, and to the various after-action reports. Two other US sources, who had access to corroborating information, have been longtime consultants to the Special Operations Command. I also received information from inside Pakistan about widespread dismay among the senior ISI and military leadership – echoed later by Durrani – over Obama’s decision to go public immediately with news of bin Laden’s death.

A Star for Mrs. Blake: a very short review

April Smith, perhaps somewhat known for her "Special Agent Ana Gray" series, has tried her hand in a completely different genre with the unusual A Star for Mrs. Blake.

Not quite a romance, not quite historical fiction, not really a memoir, only nibbling at the edges of a detective story, not exactly a war novel, A Star for Mrs. Blake is something just a little different.

Taking inspiration from a real event (the "Gold Star Mothers" of World War I), Smith imagines the travels of Mrs. Cora Blake as she goes on A Great Adventure from a remote Maine island all the way to the Argonne, in France, to find and visit the grave of her son, killed in action in World War I.

But time has passed, it's not World War I any more. In fact, it's 1931, and things are changing. Smith uses this to her advantage, building a story that is about how we deal with change, with loss, with sorrow, with events that are big and complex and often beyond our control.

Frankly, I almost gave up on this book, about a quarter of the way through.

At first, Smith is a bit over-eager, not quite sure what sort of book she wants to write. At times, she seems unable to trust her reader, and has a tendency to walk up to us and club us over the head:

When they reached the hotel, she and Griffin Reed said goodbye, shook hands, and wished each other well, both of them satisfied that an exchange had been made that was of lasting consequence. Later she would mark that moment, when they passed through the gold-kissed gates of the Luxembourg Gardens and she trusted him enough to tell her story, as the one that changed her life.

But after a while Smith seems to relax, and grow more confident, and as she does so, she rewards our patience, and her book becomes much more fun to read:

Griffin Reed stayed home, barefoot and in his pajamas. When he was writing he never needed to pass beyond the white walls of the garden. He barely saw anything around him; his mind chased a progression of ideas that kept evaporating whenever he got close. Occasionally a phrase would emerge like a neon sign out of the mist and he'd grab for it with pen and paper, miss it completely, then toss the note aside and veer away. Soon there was a trail of notes blowing across the propery, each one urgent and forgotten. He snapped at the maid not to pick anything up. He'd find himself on the floor, on the bed, staring out the window, walking down the steps to make a cup of tea, clipping leaves in the potting shed, dropping the clippers, back in the kitchen, looking at the mail. He lived in a muffled corner of his mind that was not illuminated by any kind of logic or salvation.

Were there actually neon signs in 1931? Wikipedia says yes, they became quite common by 1920. But it doesn't matter, we're having a good time now, wandering around the muffled corners of our mind, trying to follow this trail of notes.

Mrs. Blake goes to France; things happen; she meets people; she has adventures; characters develop; conversations occur. We follow along, enjoying ourselves, being part of the moment, taking this opportunity to remember what was, and what undoubtedly will be again.

Silence entered her ears and pressed against her brain. Did anybody laugh here? Did anyone rejoice at happy memories? Or did that not serve our country? She stood up, running her fingers along the stone, noting the smooth edgework. She knew something that she hadn't known before. She'd always imagined Sammy falling alone in suspended space like a stage backdrop, but now she saw a marble forest of young men who were dead, and knew that Sammy was, had been, and always would be in their company. A spasm of grief almost doubled her up -- for her boy, for all the boys, and for the lives they never had. And then, a moment ago unbearable, it left her like the breeze.

I could wish Smith had fought a little harder, dug a little deeper, been more sure of her footing and more trusting of her readers, and maybe written a tighter book, a sharper book, a bolder book. I think she was nearly there; I think she had the elements of something truly great.

But I shouldn't pick, I shouldn't whine. Smith wrote the book she wanted to write, and she did a fine job. A Star for Mrs. Blake is a book I certainly didn't regret reading.

Those little games we play ...

It's somehow both fascinating and terrifying to read these three articles more-or-less together:

• Handcrafted Fraud and Extortion: Manual Account Hijacking in the Wild

  In this paper we focus on manual account hijacking—account hijacking performed manually by humans instead of botnets. We describe the details of the hijacking workflow: the attack vectors, the exploitation phase, and post-hijacking remediation.

• The Psychology of Pricing: A Gigantic List of Strategies

  Whether you’re launching a new product, selling items on eBay, or negotiating a deal on your house, you’ll learn how to choose a price that will maximize your profit.

• How Facebook’s Algorithm Suppresses Content Diversity (Modestly) and How the Newsfeed Rules Your Clicks

  Today, three researchers at Facebook published an article in Science on how Facebook’s newsfeed algorithm suppresses the amount of “cross-cutting” (i.e. likely to cause disagreement) news articles a person sees.

In the first article, I was fascinated by this description of how scammers manipulate your emotions to try to trick you into falling for their scheme:

Over time, we realized that scam schemes share a set of core principles that we were then able to formalize as follow:

• A story with credible details to limit the victim suspicion.
• Words or phrases that evoke sympathy and aim to persuade. E.g apologizing and providing distressing details such as "had a knife poking my neck for almost two minutes".
• An appearance of limited financial risk for the plea recipient, as financial requests are typically requests for a loan with concrete promises of speedy repayment.
• Language that discourages the plea recipient from trying to verify the story by contacting the victim through another means of communication, often through claims that the victim’s phone was stolen.
• An untraceable, fast and hard-to-revoke yet safe-looking money transfer mechanism. The payment also needs to be picked up anywhere and somewhat anonymously as the scammers might not be from the country they claim the victim what mugged in.

The second article is just full of such wonderful ideas. I was particularly taken by this:

When people compare your price to a reference price, you can influence them to pull a lower price into that comparison.

Why would people pull a lower price into the comparison? This strategy takes advantage of our brain’s laziness for encoding numerical values. Adaval and Monroe (2002) explain that: "...price information about a product is unlikely to be coded into memory in terms of exact numerical digits but, rather, is coded spontaneously in more general magnitude terms (e.g., "low," "high"). Thus the numerical price is susceptible to the influence of its original context when people attempt to reconstruct it later." (pp. 585)

With such a hazy memory, you can influence how people recall your price. How? You just need to reframe your price into a lower numerical value. Exposing people to that lower value will cause them to encode a smaller magnitude.

And then they follow up with such classic examples as

• Keep the Shipping and Handling separate
• Offer payment in installments

And the last article focuses mostly on placement, which is Facebook's algorithm for deciding what appears near the top of your display, and what is hidden below.

Sure enough, it matters:

Notice how steep the curve is. The higher the link, more (a lot more) likely it will be clicked on. You live and die by placement, determined by the newsfeed algorithm.

Well, I guess there's no surprise there. After all, don't bury the lead; keep it above the fold.

(Who even reads a newspaper anymore nowadays; much less worries about how it will be folded on the train?)

Well, I guess the more things change, the more things stay the same.

Wilder Mind: a very short review

We have the new Mumford & Sons now: Wilder Mind.

It is different.

The banjo is gone.

The sound is different.

But it is good.

Time passes.

What happened to spring?

All of a sudden, the last few days, we've had highs in the 50's, gray skies, even some drizzle.

The heater even switched on in the house this morning!

What happened?

We were going along so well.

Things that caught my interest over May Day weekend

Clear and breezy, just what you expect from early May.

• XML External Entity (XXE) Processing

  There exists a specific type of entity, an external general parsed entity often shortened to an external entity, that can access local or remote content via a declared system identifier. The system identifier is assumed to be a URI that can be dereferenced (accessed) by the XML processor when processing the entity. The XML processor then replaces occurrences of the named external entity with the contents dereferenced by the system identifier. If the system identifier contains tainted data and the XML processor dereferences this tainted data, the XML processor may disclose confidential information normally not accessible by the application.

  Attacks can include disclosing local files, which may contain sensitive data such as passwords or private user data, using file: schemes or relative paths in the system identifier. Since the attack occurs relative to the application processing the XML document, an attacker may use this trusted application to pivot to other internal systems, possibly disclosing other internal content via http(s) requests.

• XML External Entity (XXE) Vulnerabilities

  The Billion Laughs Denial-of–Service (DoS) attack consists of defining 10 entities, each defined as consisting of 10 of the previous entity, with the document consisting of a single instance of the largest entity, which expands to one billion copies of the first entity.

• Notes on indexes and index-like structures

  One of the best examples of a DBMS winning largely on the basis of its indexing approach is Sybase IQ, which popularized bitmap indexing. But when last I asked, some years ago, Sybase IQ actually used 9 different kinds of indexing. Oracle surely has yet more. This illustrates that different kinds of indexes are good in different use cases, which in turn suggests obvious reasons why clever indexing rarely gives a great competitive advantage.

• Why Are Geospatial Databases So Hard To Build?

  If your data model is inherently non-scalar, you enter an algorithm wasteland in the computer science literature. Paths, vectors, polygons, and other elementary aggregations of scalar coordinates used in spatial analysis are non-scalar data types. Computational relationships are topological instead of graph-like.

  Spatial data types, among a few other common data types, are interval data types. An interval data type cannot be represented with less than two scalar values of arbitrary dimensionality, like the boundary of a hyper-rectangle. These differ from scalar types in two important ways: sets have no meaningful linearization and intersection relationships are not equivalent to equality relationships. The algorithms that do exist in literature for interval data are poor.

• Space efficient indexes for the big data era

  An index is typical used to restrict access to only the relevant to the query parts of the data. By space efficient we emphasize that the index has to be significantly smaller that the original data and typically has to reside in at least one level higher in the memory hierarchy than the indexed data. During query evaluation, before accessing and transferring data from a slower memory to a faster, we consult the space efficient index which resides in much faster memory. The index will reveal which parts, if any, of the ”slower” data are relevant to the query and should be transferred.

• Distance Metrics for Fun and Profit: "People Who Like This Also Like ... "

  People building search engines have developed some pretty nice models for calculating similarity between query strings and text documents. These models can be easily adapted to our purposes, by treating each artist as a document and each user as term in those documents.

• Interop Liveblog: The Post-Cloud

  If the problem was “I want to deploy my web service”, the initial answer was the x86 server (which made computing power more accessible to people). The next answer was to use multiple virtual servers, for more density. Next we wanted workload mobility (live migration), and then deploying web services as a service (IaaS). That lead to “automagically configured” web services (PaaS). Then we wanted captured and immutable images of our web services (Docker), being able to turn them up extremely quickly (Linux containers and Docker) and easily integrated into our Continuous Development lifecycle. Finally, we wanted to be able to do all that on-demand, and quickly replaced on error (Mesos, CF Diego, Kubernetes). That, in turn, leads to wanting to be able to manage and place the workload intelligently based on data from any level—i.e., i just want to run a web service and have the data center do all the rest (the Post-Cloud).

• Beej's Guide to Network Programming Using Internet Sockets

  This document has been written as a tutorial, not a complete reference. It is probably at its best when read by individuals who are just starting out with socket programming and are looking for a foothold. It is certainly not the complete and total guide to sockets programming, by any means.

• Committers' FAQ

  This document is targeted at Apache committers. A committer is an individual who was given write access to the codebase of any Apache project.

• Why our future depends on libraries, reading and daydreaming

  The simplest way to make sure that we raise literate children is to teach them to read, and to show them that reading is a pleasurable activity. And that means, at its simplest, finding books that they enjoy, giving them access to those books, and letting them read them.

• Ladies and Gentlemen, The English Language…

  Place the word "only" anywhere on the sentence: "She told him that she loved him."

Learning about Derby

Abhinav and I were talking about Derby the other day, and I put together the following, which I thought was possibly interesting to others, too.

Apart from the bug that we are working on, is there any other resource/ documentation

Yes, definitely! In fact, there's so much, the big problem is knowing where to start.

There's all the manuals: Derby 10.11 Manuals.

If you haven't read through "Getting Started with Derby", that's the most important one to read at the beginning. It has a bunch of examples arranged a little bit like a tutorial that are good to get you going.

After a while, you're pretty comfortable running Derby, and you'like to know more about how it works.

For internals documentation, the most important resources are:

• Papers and presentations
• the Derby internals wiki

Of all of those, I think this is the best one to start with: Internals of Derby: An Open Source Pure Java Relational Database Engine.

Of course, there is the code, which can be tricky, but is very good to read. Anyone can read the code. You can download a source tarball, or check out a copy from Subversion.

Or you can use the Online ViewVC browser.

And another important resource is the JIRA database. Often, when I am trying to figure out how something works or why it is written the way it is, I end up looking at the Subversion history to learn about the history of the changes, and then looking in JIRA for a discussion of the change in question.