Ora et Labora: a very short review

We bought two new board games over the holidays.

One of them is Ora et Labora.

Ora et Labora is the latest game from world-famous board game designer Uwe Rosenberg, who is best known for his game Agricola, and whose other games include At The Gates Of Loyang and Le Havre.

Ora et Labora is a wonderful game. It takes many of the basic concepts of Agricola, and matches them with some of the best aspects of Le Havre, while also introducing a few innovations that simplify the operational aspects of the game.

From Agricola, we get the theme of medieval peasant farming, with its cultivation of grain and livestock and its progressive development of your farm into an estate. From the "farmers and moors" expansion of Agricola, we get the concepts of food versus energy, and the tree and peat cards as resource production.

From Le Havre, we get the various different types of basic and developed resources, and we get the introduction of buildings which can be constructed and developed on your estate.

An important mechanical improvement is the game wheel, which arranges for the introduction of basic goods into the game without the labor-intensive need to sprinkle game tokens onto the board at the start of each round, which occupies a lot of time in both Agricola and Le Havre.

As you would expect from a Uwe Rosenberg game, Ora et Labora is deep and complex, with lots of choices, and very sophisticated strategy and tactics. It is also superbly crafted: the pieces are well made, the rules are thorough and precise and clear.

And, like other Uwe Rosenberg games, Ora et Labora is designed to work well for two players, and also has a solo version.

Unfortunately, although we've had the game for six weeks now, we've not yet found the time to play an entire game through to conclusion; we're just too busy. But hopefully soon...

I suspect that Agricola will continue to hold its sweet spot in our hearts, as it was the game (together with Ticket to Ride) that got us back into board gaming about a decade ago after we had fallen off the board gaming habit.

But Ora et Labora seems to be a wonderful game and I'm looking forward to playing it more as time permits.

Cloud Computing reading list

Professor Murat Demirbas of SUNY Buffalo, whose blog I regularly follow, is teaching a seminar this spring entitled Cloud computing and distributed systems

In this course we will discuss and study most influential and timely research papers on the topics of cloud computing and scalable & reliable distributed systems.

Professor Demirbas has now posted the reading list for the seminar, and I am extremely impressed with the quality of the readings he has chosen.

These are excellent, fascinating papers, and they are definitely representative of the state of the art in distributed systems research in 2013.

I was already familiar with about 60% of the papers on this list, and I'm now happily digging into the other 40%, and have already hit upon several fascinating reports that were previously unknown to me.

I guess I'm somewhat surprised that Google's Spanner report is not included, nor is Yahoo's Zookeeper report, both of which I consider to be absolute must-reads, but of course with any such list you have to draw the line somewhere, and I'm not sure which papers I'd exclude in order to include the Spanner and Zookeeper works.

I hope Professor Demirbas's seminar goes well; I hope he is lucky enough to have a motivated group of students who dig into these papers and have some great discussions. It would be great to be a fly-on-the-wall during those sessions; I'm sure I'd learn a lot!

Here comes the sun!

It's starting to be light in the morning when I take the dog out for her 6:30 AM walk, and it's starting to remain light in the evening when I'm coming home at 5:00.

My wife's tulips have started to grow, and down the block the daffodils are poking their shoots through the ground.

I'm afraid it will be a few months yet until these poor fellows feel that winter is ending, but here in the temperate California coast, the signs are unmistakable.

My parents are planning an entire daffodil-crocus-tulip tour, starting in the far south at the Mexico border, and traveling slowly northward to Canada, stopping along the way, proceeding only as far and as fast as the spring blossoms do.

It sounds delightful!

A patient departure

The San Francisco Chronicle has a fascinating story in this weekend's paper: Bar pilot panel slow to act on complaint.

The story specifically involves an incident from February 18th, 2012, in which Captain David Chapman of the SF Bar Pilot's Association was commanding the Overseas Tampa, an oil tanker departing Richmond's massive Chevron refinery with a full load of diesel fuel destined for Hawaii.

The preferred method for Chevron pilots pulling away from the wharf, the state commission found, is to use two tractor tugs to help a ship make a pair of 180-degree turns and head to open sea.

On Feb. 18, however, Chapman persuaded the tanker's captain to agree to an alternative plan: The tugs would pull the tanker backward and north, then position the ship at an angle that would, with the force of the maximum ebb tide, swing it toward the Golden Gate. Chapman called the move a "patient departure," telling the ship's master and tug pilots that it would work "as long as we're patient."

When it left its berth, however, the Overseas Tampa began moving more swiftly than expected, complicating its ability to reach an intended 45-degree angle from the wharf. Chapman ordered the engines to run at full power in reverse, and the tanker narrowly missed shoals north of the wharf.

I'm very familiar with the area surrounding the Richmond oil tanker wharf; I've sailed past it a dozen times and can attest to the extremely tricky currents and winds in that area. If you don't know where this wharf is, you should check the pictures in the Chronicle story; they make visualization of the incident much easier.

The Board of Pilot Commissioners for the Bays of San Francisco, San Pablo, and Suisun investigated the incident; its report cites issues of judgement, and of communication, as well as some more primal questions about how and why humans decide to cooperate in situations where chain of command is not established by force:

In his defense, Chapman not only denied being reckless but also blamed the event on the failure of one of the tug pilots to obey his orders at a critical point. That pilot, he said, still resented that Chapman had passed the bar pilot test more than a decade earlier and that the tug pilot hadn't.

The Bar Pilots association has had a long and complex history here in the Bay Area, and the Chronicle story continues on to consider the larger question of how the bar pilots are assessed, regulated, and overseen, and whether the public's interests are being adequately protected.

Mike Jacob, vice president of the Pacific Merchant Shipping Association, a trade group representing shipping companies and terminal operators, said dealing with Chapman is the state commission's job.

"That is precisely why we rely on the state and the board of the pilot commissioners to regulate the licensees and ensure the safety of the ships that come into the bay," Jacob said.

...

Steve Knight, political director for the environmental group Save the Bay, said the commission's handling of the incident and the questions about Chapman's seamanship raise "serious concerns" about "the systems in place to protect San Francisco Bay."

"The public relies heavily on these pilots to keep our bay safe," he said. "Why we are reading about all this nearly a year later?"

It's not often that you see the captains of industry and the tree-huggers line up so clearly on the same side like this, and it indicates that the issues with the Bar Pilots are reaching a head. As the article notes, some shipping companies are now refusing to allow certain pilots aboard their ships, effectively taking the question of certification and validation of pilot skills into their own hands.

If the bar pilots arrangement were to break down, it's not clear what would replace it. As Wikipedia notes, the handling of port traffic on the California coast is a real hodge-podge:

Pilots on other California waters operate under the authority of their federal pilot’s license. Port of Los Angeles pilots are municipal employees. Port of Long Beach pilots work for a private contractor. Pilots in the ports of Humbolt Bay, San Diego, and Port Hueneme are commissioned or contracted with by their respective port authorities or districts.

I love living here in the Bay Area, and have enjoyed the beauty of the bay ever since we arrived. Commerce is necessary, though, and the Richmond refinery is just one aspect of the Bay's commerce; the Port of Oakland is the fifth busiest container port in the country, handling an enormous amount of traffic.

It's good to see the Chronicle keeping its attention on the issues with the Bar Pilots; the discussion of how we want to ensure the Bay's safety is important and needs to continue.

One Million Square Feet

Tracy, California, is kind of a funky place. Most people in my circles know it as

that place about 45 minutes east of here, on the way to Yosemite, where you can stop and grab a cup of coffee when you've left the Bay Area at 4:45 A.M. on your way to the mountains.

Tracy is located smack in the middle of California's Central Valley. Longitudinally, it is over on the west side of the valley; latitudinally, it's almost perfectly in the middle of the valley, halfway from Bakersfield to Redding.

In the summer, Tracy is hot and dry; in the winter, it is cool and dry. But, as the old saying goes, what Tracy has going for it is location, location, location:

1. It is in a large open space, where land is plentiful and cheap
2. It is abundantly served by large highways and major railroad lines
3. It is about as close to the Bay Area, and to the Port of Oakland, as you can get, and still satisfy rules #1 and #2.

All of which makes Tracy about ideal for that particular specimen of industrial development: the distribution center.

Tracy doesn't get much mention in the news, but it had another 15 seconds of fame this week, as Amazon rolled out some information about its newest project: Amazon Sets Up (Really Big) Shop to Get You Your Stuff Faster.

Amazon’s strategy hinges on getting its inventory as close as possible to as many people as possible. The closer Amazon can get its stuff to the people who want it, the faster and cheaper the company can get it to them.

As the article points out, Amazon previously located its distribution centers outside of the state, but an agreement with the state government over sales tax issues has changed Amazon's practice.

And Tracy isn't the only such center. In southern California, the same story is being repeated, in the SoCal version of Tracy:

Amazon’s first distribution center in the state went up in San Bernardino, which filed for bankruptcy last year. Amazon gets cheap land and cheap labor; hard-hit communities get hundreds of jobs; and in Northern California at least, everyone who orders from Amazon seems likely to have their online consumer cravings satisfied faster than ever.

I have a close friend who is a distribution center specialist; he designs and operates distribution centers for Nike. These facilities really are amazing: complex, high-tech, incredibly efficient.

And I'm sure it would be no surprise to Lathrop J. Tracy, if he were to see what's become of the city named for him one hundred years ago, since distribution is what gave Tracy its start:

The origins of Tracy are related to the mid-19th century construction of Central Pacific Railroad lines running from Sacramento through Stockton and to the San Francisco Bay Area. A number of small communities sprang up along these lines, including the one named for railroad director Lathrop J. Tracy.

I guess that the next time I find myself zooming past Tracy, flying down the super-speedway on my way to the Sierra Nevadas or points elsewhere, I'll try to remember to turn my head to the side and take a peek at the anonymous buildings as I pass by, and see if I can tell which one is the new One Million Square Feet.

Debugging: a very short review

Computer books come in all sorts of flavors. There are textbooks, users manuals, reference works, etc.

This short book by David Agans doesn't fit easily into any of the common categories: Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems.

Debugging is one of those funky parts of the programming profession that doesn't get talked about very much. I'm not sure why that is, because it's quite clear to me, after three decades of programming, that debugging skills are one of the crucial characteristics that distinguish successful programmers from failed ones:

Successful programmers are able to debug programs, whether or not they wrote them initially.

I've seen a lot of people who wanted to become programmers, and failed, and in many of those cases the part they struggled with was debugging. They could write programs that were quite clear, well-designed, neatly-expressed.

But no program ever works correctly the first time; as the great Donald Knuth said: "Beware of bugs in the above code; I have only proved it correct, not tried it."

So, to be a successful programmer, you have to learn how to be successful at debugging.

Many programmers think that debugging is some sort of magical ability; either you have it, or you don't. I don't think that is true; I think you can learn debugging skills, and, much more importantly, I think you can improve your debugging skills.

Agans's short, fun, highly-readable book is a great way for you to learn or improve your debugging skills. Here's what Agans says about the goal of his book:

In this book I'll introduce the nine golden rules of debugging, then devote a chapter to each. I'll start each chapter with a war story where the rule proved crucial to success; then I'll describe the rule and show how it applies to the story. I'll discuss various ways of thinking about and using the rule that are easy to remember in the face of complex technological problems (or even simple ones).

The rules themselves are online, at Agans's web site, and they are nice and short and to-the-point:

1. Understand the system
2. Make it fail
3. Quit thinking and look
4. Divide and conquer
5. Change one thing at a time
6. Keep an audit trail
7. Check the plug
8. Get a fresh view
9. If you didn't fix it, it ain't fixed

You can get a sense of Agans's light and slightly tongue-in-cheek style as you read the rules, but in fact I like all his rules and think they're great.

If I could add one rule of my own, it would be:

Don't be afraid to try something.

I see so many people get stuck in debugging where they know there's a bug, and they can reproduce the bug, but they don't know what to do next. When I see this happen, I say to them: "what have you tried?"

Often, just asking them what they've tried gets them going again, but other times people are paralyzed by a fear that changing something is going to make matters worse.

Well, it might. But, you can't let that stop you from at least trying something. It may not be the right thing, but try it, and you'll learn from your experiment, and that will help break the paralysis.

Anyway, regarding Agans's book, I was pleasantly surprised: even after 30 years of experience, I found it a rewarding read, and I recommend it to anyone who's considering spending a significant portion of their life as a computer programmer.

Northern California Perfection

We took advantage of a supremely beautiful winter weekend to escape to Big Sur for 48 hours.

Friday afternoon we drove down to Carmel, and treated ourselves to a nice dinner at Bistro Giovanni, where the people were very pleasant.

Saturday morning we grabbed some coffee and headed south to Point Lobos State Reserve. Arriving there shortly before 10 AM, I noticed that the entrance gate was already busy and the first parking lot was already full.

It turned out that we had had the extreme good luck to arrive at the reserve on a very special day: Underwater Parks Day at Point Lobos State Natural Reserve.

Filmmakers, photographers and explorers who have studied marine sanctuaries all over the world converged at Point Lobos State Natural Reserve on Saturday for Underwater Parks Day, an event to honor the implementation of a statewide network of marine protected areas and to raise ecological awareness.

We left the car at the Sea Lion Point trailhead, and hiked through the Allan Grove of Monterey Cypress, then along the North Shore Trail back to Whalers Cove, stopping frequently along the way for pictures and binocular gazing.

We reached the cove around 11:30, just in time to spend 45 minutes talking to the divers from the diving club who were exhibiting a fantastic array of animals they had found while diving in the cove. Normally, the divers in the reserve are allowed only to "look but not touch", but today only they were permitted to bring animals up to the surface for us all to see. (After the 90 minute viewing period, the animals were returned to their habitat by the divers.)

There were stars, crabs, nudibranchs, limpets, sea cucumbers, and many other creatures, attentively watched over by the divers and entertaining a steady supply of children of all ages (including us).

At the Whalers Cabin, the docents had set up a powerful telescope, trained precisely on one of the Southern Sea Otters basking in the sun. After admiring the otter from afar, we hopped aboard the park shuttle and rode back to our car.

We walked out Sea Lion Point Trail to Punta de los Lobos Marinos, where indeed several hundred of the well-known "Sea Wolves" for which the reserve is named were at their favorite perch on Sea Lion Rocks.

As we watched the Sea Lions cavorting and listened to them barking, we began to spot puffs of spray farther out to sea. First one, then another, and sometimes two or three at a time. Without really planning to do so, we had the tremendous good fortune to have arrived at the peak of the Gray Whale migration

Each year approximately 15,000 of an estimated total population of 17,500 gray whales swim south from Arctic feeding grounds, en route to their Baja California breeding and calving grounds.

From December through May, gray whales can be observed a few miles off California's shore.

At Point Lobos, the greatest numbers of migrating whales are usually seen in mid-January, so our timing was perfect!

As we grew accustomed to spotting the whale spouts on the horizon, we saw more and more of them; during the next twenty minutes, as we patiently waited and watched, we saw perhaps 35 spouts, as well as 5 flukes. As the Point Lobos docent guide notes

They usually blow 3 to 5 times in 15 to 30 second intervals before raising their tails (flukes) for a deep dive. A gray whale can stay underwater for up to 15 minutes and can swim 3 to 6 miles per hour.

As the day was crystal clear and the temperature was in the low 70's, we were as content as could be, but by now we were hungry, so we drove all the way down to the end of the road at the Bird Island trail head, where we parked again and had our lunch.

After a good lunch and a nice rest, we walked out the Bird Island trail where we spotted more otters, more whale spouts, and a nesting pair of Pelagic Cormorants, perched in a cliff niche some 150 feet above the crashing waves.

Sunday morning, before we hit the road home, we decided to take another hike, this time at Monterey County's Jacks Peak County Park.

This little-known park is named for David Jack, a Scottish immigrant who was an early cattle rancher in Monterey County. David Jack is much better known for the cheese he invented: "Monterey Jack Cheese".

Jacks Peak is the highest point on the Monterey Peninsula, just over 1,000 feet high. But the best part of the park is that the entrance road climbs nearly to the top; the trail head parking lot is at 925 feet. So, once you have paid your fee and parked, you have a short 0.75 mile walk and a 100 foot ascent to get to the peak with its stunning views of Monterey Bay, the cities of Monterey and Carmel, and the beautiful Carmel Valley and Carmel River.

Then it was time to get home; we had chores to do and a dog to feed. So we saddled back up and returned home.

Carmel and Big Sur was a wonderful weekend holiday; I am sure we will do that again before long.

Stuff I'm reading this weekend

OK, admit it: you've been spending way too much time reading about Manti Te'o's bizarre story and you're ready to move on and read about something else.

Well, here's a few things that you might find interesting...

• A nicely illustrated article by Josh More on the RJS Security blog describing some of the social engineering tricks that scammers do to try to fool you into giving them your money: Internet Theft and the Holidays

  However, it did puzzle me how the scam worked. After all, I hadn’t given them any useful data. How would they get my money? Were they just incompetent criminals? This was well outside the realm of photography and I now had a professional interest.

• Kevin Kelly highlights this bizarre story uncovered by a Verizon security audit: Case Study: Pro-active Log Review Might Be A Good Idea

  As it turns out, Bob had simply outsourced his own job to a Chinese consulting firm. Bob spent less that one fifth of his six-figure salary for a Chinese firm to do his job for him. Authentication was no problem, he physically FedExed his RSA token to China so that the third-party contractor could log-in under his credentials during the workday. It would appear that he was working an average 9 to 5 work day.

• I love the idea of this tool that helps you find free books for your e-reader: Freebook Sifter
• I hadn't been following Jeremy Kun's blog, but I will now. There's some great stuff there! For example check out Probability Theory — A Primer

  In this post, we will begin with a naive version of probability theory. That is, everything will be finite and framed in terms of naive set theory without the aid of measure theory. This has the benefit of making the analysis and definitions simple.

• Moxie Marlinspike offers some Career Advice

  Jobs at software companies are typically advertised in terms of the difficult problems that need solving, the impact the project will have, the benefits the company provides, the playful color of the bean bag chairs. Likewise, jobs in other fields have their own set of metrics that they use to position themselves within their domains.

  As a young person, though, I think the best thing you can do is to ignore all of that and simply observe the older people working there.

  They are the future you. Do not think that you will be substantially different. Look carefully at how they spend their time at work and outside of work, because this is also almost certainly how your life will look.

• Leonard Cohen is coming to the Paramount Theatre in early March. The show has received wildly rave reviews, so I went to see about buying tickets. Tickets start at $275/seat and range up to $600/seat! As my wife said, "we can take an awfully nice vacation, and still buy every one of his records, for less than $1200."
• Last December, Gary Brecher published another of his epic works: The War Nerd’s Twelve Days of 1812. As usual, his writing is solid and well-researched, while still being wildly entertaining.

  The British Army has had some wild ups and downs over the past 300 years, unlike their navy, which has been damn good straight through. The redcoats we faced in 1776 weren’t much of an army—the troops were seldom-fed unemployables and the officers mostly dim-bulb second sons. That was one of the reasons the US woofed so loud at the Brits leading up to 1812: we were bigger and stronger and figured if we beat them back in the 1780s it’d be a cakewalk now.

• A nice article about Industrial Light and Magic's latest work over at Wired: How ILM Built the Best Hulk Ever for The Avengers (And Earned an Oscar Nom)

  ILM’s Jeff White told Wired that job number one in building a better Hulk was building the elements of character within the CGI, and shared five key steps that helped them create the most impressive visualization of the Hulk to hit any screen, big or small

• For a solid breakdown of the Java vulnerability that's getting all the attention this month, try Esteban Guillardoy's writeup: Java MBeanInstantiator.findClass 0Day Analysis

  what is happening here is that they forgot to skip the frames related to the new Reflection API and only the old reflection API is taken into account.

• David Lang's short paper about wireless network efficiencies is very practical: Building a Wireless Network for a High Density of Users

  Wireless networks for conferences and schools tend to work very well when tested, and then collapse completely when all the users show up to use them. This pattern is repeated time and time again to the point where people tend to think that it's a fundamental limitation of Wi-Fi technology. There are real limitations that you have to deal with, but if you keep them in mind it is very possible to build a wireless network for thousands of people and have it be rock solid and reliable.

• I'm still mostly baffled by Locator/ID Separation Protocol, but this paper by a team at Cisco is helpful: Network-Based Protocol Innovations in Secure Encryption Environments

  The use of dynamic discovery of the routes to the secure networks the IVDs are protecting could increase demand on hardware resources and IVD functionality in order to process and hold a potentially larger number of IP prefixes being received from the protected network. This could prove challenging, particularly if the IVD hardware design was not originally intended to hold a large amount of IP prefixes.

• The Networking Nerd wonders what the IP address equivalent of the famous "555-nnnn" phone numbers are: IP Addresses in Entertainment

  Hollywood has been trying for some time to come up with IP addresses that look real enough to pass the sniff test but are totally false. Sometimes that works. Other times, you end up with Law and Order. In particular, the SVU flavor of that show has been known to produce IP address ranges that don’t even come close to looking real.

• Both Coursera and Udacity continue to roll out new material. Here's a couple upcoming classes that look intriguing:
  • Software Defined Networking
  • Cryptography II
  • Pattern-Oriented Software Architectures for Concurrent and Networked Software
  • Compilers
  • Networks: Friends, Money, and Bytes
  • Introduction to Parallel Programming
• The story about the way the Nokia mobile browser handles https traffic was well-covered, but still somehow didn't seem to get the attention it deserves, perhaps because the Java 7 exploit stole all the media attention. Anyway, start here: Nokia phone forcing traffic through proxy and then go here: Nokia’s MITM on HTTPS traffic from their phone

  Just upgraded my Nokia browser, the version now is 2.3.0.0.48, and as expected there is a change in HTTPS behaviour. There is a good news and a bad news. The good news is with this browser, they are no more doing Man-In-The-Middle attack on HTTPS traffic, which was originally the issue, and the bad news is the traffic is still flowing through their servers. This time they are tunneling HTTPS traffic over HTTP connection to their server.

• Lastly, I'm sure you've already seen it, but if not, don't miss this great story about code review in the open source world: 'SHUT THE F**K UP!' The moment Linus Torvalds ruined a dev's year.

  Last year Torvalds insisted he was a mild-mannered man of peace who is mischaracterised as angry because only his outbursts are reported.

  But the dad-of-three admitted it's not in his nature to be overly nurturing and gentle when dealing with Linux development matters and noted he tends to get involved in issues at the gasket-blowing stage.

  Here's the actual email traffic.
  • message 1
  • message 2
  • message 3
  • message 4
  Although it's strongly worded, observe that it's actually a highly important subject, and the strong language is due to passion, not to crudity.

  In other words, the reason people respect Linus, and want to work with him, learn from him, and use his software, is because he actually cares about making great software. That's something deserving of respect.

Jeff Hodges on distributed systems

I quite enjoyed this article by Jeff Hodges: Notes on Distributed Systems for Young Bloods.

Below is a list of some lessons I’ve learned as a distributed systems engineer that are worth being told to a new engineer. Some are subtle, and some are surprising, but none are controversial. This list is for the new distributed systems engineer to guide their thinking about the field they are taking on. It’s not comprehensive, but it’s a good beginning.

The entire article is very good, with lots of practical advice, but to get you motivated, here's Hodges's list:

• Distributed systems are different because they fail often.
• Writing robust distributed systems costs more than writing robust single-machine systems.
• Robust, open source distributed systems are much less common than robust, single-machine systems.
• Coordination is very hard.
• If you can fit your problem in memory, it’s probably trivial.
• “It’s slow” is the hardest problem you’ll ever debug.
• Implement backpressure throughout your system.
• Find ways to be partially available.
• Metrics are the only way to get your job done.
• Use percentiles, not averages.
• Learn to estimate your capacity.
• Feature flags are how infrastructure is rolled out.
• Choose id spaces wisely.
• Exploit data-locality.
• Writing cached data back to storage is bad.
• Computers can do more than you think they can.
• Use the CAP theorem to critique systems.
• Extract services.

I hadn't heard the term "Russian-doll Caching" before, so I looked it up, and it turns out it's a Ruby-on-Rails thing, and is the subject of much discussion right now:

• How key-based cache expiration works
• Russian Doll Caching & Cache Digests
• Advanced Caching: Part 1 - Caching Strategies

Overseas Reymar

I hadn't been paying much attention to the story of the Overseas Reymar accident, but I picked up the story today in the San Francisco Chronicle: Bridge-crash tanker's last-minute shift.

when Kleess left his anchorage spot south of the Bay Bridge, he was heading for an opening between the nautically termed C and D towers of the Bay Bridge, which is the second opening between towers from the west side of Yerba Buena.

"But then he switched and headed for Delta-Echo," McIsaac said, referring to the opening between the D and E towers, which stand on either side of the opening closest to the island. "I don't know why he changed direction, and that sort of information will have to come out in the investigation."

The "racon" radar beacon that helps guide ships between the C and D towers was not functioning on that day, according to Caltrans, which run the units, but the racon for the D and E tower opening was working. Coast Guard investigators said it is unsure how that may have affected the accident.

There seem like a lot of uncomfortable similarities with the crash of the Cosco Busan five years ago.

This investigation is just beginning; I'll be paying close attention to how it pans out.

Statistical evidence of fraud

Professor Ken Regan of SUNY Buffalo has written a detailed article about the recent allegations of fraud in a high-level chess tournament in Europe: The Crown Game Affair.

Regan describes the complexity of attempting to develop statistical evidence of fraud:

My report describes two main tests, which are partially independent. Presumably their combined confidence would be higher, though I have not yet worked out how to do this numerically. Several alternative specifications for the tests, such as using the player’s rating before rather than after the tournament as the main baseline, excluding one (or two) game(s) where public transmission of moves was switched off amid suspicion of him, and excluding moves after (say) move 70 in very long games when the time available to think might be too short for some cheating mechanisms, exhibit much higher deviations. Although the “Intrinsic Rating” component does not accompany a statement of odds, it indicates that the inherent quality of the moves, as judged by computers, was highly significantly beyond what goes with a 2700-level performance.

Although this is a challenge, Regan is confident that the results are definitive:

Here 2600 is a chess rating that usually distinguishes a “strong grandmaster,” while my own rating near 2400 is typical of the lesser title called “international master,” and Ivanov’s pre-tournament rating of 2227 is near the 2200 floor to be called any kind of master. Although Magnus Carlsen recently broke Garry Kasparov’s all-time rating record to reach 2861, my program for “Intrinsic Ratings” clocked Ivanov’s performance in the range 3089–3258 depending on which games and moves are counted according to supplementary information in the case, all higher than any by Carlsen and his two closest pursuers enumerated by me

The New York Times notes that statistical evidence may be the only evidence that exists in this case: A Quandary for the Game in a High-Tech Era.

A third-place finish at a tournament last month by a formerly obscure player was so startling that organizers searched his clothing and took apart his pen looking for evidence that he had outside help. They found nothing.

But the episode has again raised the question of how officials can monitor games in an era when technology is so advanced, and it set off concerns about how such suspicions will affect the game. If every out-of-the-ordinary performance is questioned, bad feelings could permanently mar the way professional players approach chess.

Chess, of course, is not the only sport that continually struggles with the problem of maintaining fair competition: just look to this week's cycling news for a much higher profile example, or to this fascinating story from last fall for a more intriguing example.

But chess is somewhat unique in the man-machine symbiosis aspect; indeed, a number of people have suggested that chess as a whole might benefit from a more open acceptance of computers, as in the recent development of Advanced Chess, also called Freestyle Chess, Cyborg Chess, and Centaur Chess.

Today, however, the humans had the day, as World Champion Vishy Anand played perhaps the most brilliant game of his decades-long spectacular career, a devastating annihilation of world #2 Levon Aronian, with the black pieces no less!.

The day of the computers draws ever closer, but Anand's scintillating performance today is one to treasure.

Kingdoms of Amalur: Reckoning: a very short review

I've been blogging less the past two weeks. One of the reasons for that is Kingdoms of Amalur: Reckoning.

Kingdoms of Amalur is a game in the style of Dragon Age, or of the Morrowind, Oblivion, Skyrim: it is a single-player RPG, in which your character explores the world, discovers new locations, talks to the characters that reside in the various towns, castles, caves, etc., and responds to the various situations that arise as you see appropriate.

Along the way there are quests and adventures to perform, skills to be developed, gear to be acquired and equipped, potions and spells to master, and many, many, many baddies and monsters to be defeated.

According to the in-game clock, I've spent 29 hours playing Kingdoms of Amalur since I started on New Year's Day. Which seems about right, although I think the game may be counting time while paused. I haven't encountered any major bugs or frustrations, although I confess I'm rather poor at the actual combat mechanics, and so the fight scenes are typically a bunch of screaming and button-mashing rather than something far more elegant. The young kids in the room stand around and watch this senior citizen flail, finding it amusing, but hey, at least I'm still alive!

Kingdoms of Amalur certainly compares well with the other games I've mentioned: the artwork is nice, the writing is very good, the game pace is just right; I can easily see myself spending several hundred more hours playing it this year.

However, when I was playing Skyrim a year ago, there would be times when I would be wandering through the world, and it was just so jaw-droppingly beautiful that I would find myself stopping, just standing there, looking at the scenery, listening to the music, immersed in the game. In Skyrim, the rain felt like rain, the sunsets looked like sunsets, and the dungeons made my skin crawl.

Kingdoms of Amalur is no Skyrim. But it's a very nice game, and I think it's unfair to condemn it for not achieving the heights that Skyrim did.

Do you like these sorts of games? If you do, you'll like Kingdoms of Amalur

Tata Steel 2013 has begun

The 75th running of the annual Tata Steel Chess Tournament begins today. As the folks at Chess Vibes note, this is one of the two or three most important tournaments of the regular chess calendar:

Six players out of the current top ten will be present in Wijk aan Zee. Besides Anand and Carlsen there's three times winner Levon Aronian, fast-rising star Fabiano Caruana and former winners Sergey Karjakin and Hikaru Nakamura.

The event has only recently been known as the Tata Steel tournament.

The event has been held since 1938 and until 1999, the year of the famous game Kasparov-Topalov, the event was called Hoogovens Tournament. Between 2000 and 2010 the name was Corus Tournament and since 2011 it is called Tata Steel tournament, after Tata Steel bought Corus in April 2007 for 8.1 billion Euros.

When I was younger many people just called it "the Wijk aan Zee tournament"; Wijk aan Zee is a small resort town on the Dutch coast where the tournament is always held. Follow all the action at the official site.

Aaron Swartz

Cory Doctorow describes the life and work of Aaron Swartz: RIP, Aaron Swartz. I never met Swartz, but I used to read a lot of his writings; amazingly, thinking back about it, I was reading his work when he was 17 years old. I remember being particularly struck by some of his writings during his brief studies at Stanford. I see that many of those essays were at some point removed from the web, though you can still read some of them here and here.

Greatly exaggerated rumors

I'm often not completely up to date on the pop culture scene, but is this related to this?

I suppose it's not that far a leap from:

Tupac Shakur was the star performer at the first weekend of this year's Coachella music festival in California — despite the fact that the hip-hop legend died more than 15 years ago. The late rapper surprised, delighted, and creeped out the crowd by appearing through a convincing, though slightly unsettling, hologram alongside the real-life Dr. Dre and Snoop Dogg.

to

The concerts will feature projected vintage video of Denver performing, backed live by members of his original touring band and an accompanying string section. For the first time in fifteen years, audiences will experience John Denver in concert, performing hit songs from throughout his career.

I can see how inquiring minds might make the connection, but it does lead me to wonder: do you have to wait 15 years first? Is there actually some legal expiration-of-rights reason why both events occurred exactly 15 years later?

Or did the unexpected success of the first just inspire the second?

And are there more such?

Is there a Michael Jackson hologram tour?

Virtualization

All of a sudden, everything is virtualization. What began in the operating systems arena with technologies like Xen and VMWare is now moving throughout the stack.

Curt Monash looks at the state of the virtualization world in the database arena: Data(base) virtualization — a terminological mess.

Data/database virtualization seems to be a hot subject right now, and vendors of a broad variety of different technologies are all claiming to be in the space. A terminological mess has ensued

By plowing through vendor press releases, technology presentations, and marketing materials, Monash provides a solid service by validating technology that represents a real breakthrough, while rejecting vendor attempts to re-brand existing technology under the latest trendy terminology; this is the best sort of independent analyst work and I'm grateful to Monash for his efforts.

Meanwhile, as virtualization continues to spread through the networking world, everybody seems to be jumping on board there, too. GigaOm highlights one: Alcatel-Lucent makes its network play for the cloud.

Alcatel Lucent, the grandaddy in telecommunications gear, has dreams of getting into the cloud infrastructure game with a new venture called Nuage Networks (nuage means cloud in French). The division was created a year ago and is prepping a new product aimed at creating flexible and programmable networks for cloud providers.

Unfortunately, the GigaOm team aren't as willing to call "BS" on vendor fluff, and so they publish an article like this while admitting that the vendor "didn’t want to tell me what, exactly, it is".

And who knows what it is, when you go to the vendor's home page and find:

By extending the Popek-Goldberg virtualization theorems to the network, we can create a new type of distributed network hypervisor that will enable nested overlays and will bridge the performance gaps between overlays and underlays. This yields an efficient, reliable and elastic Compute Service Fabric.

I have no idea what that says, if anything.

It's important to stay up to date. It's frustrating when it's so hard to do.

Performance behaviors can be very non-linear

One of the challenging things about working on performance problems is their fluid, unpredictable nature.

A system can be completely stable with a certain configuration and workload; it may be handling a high volume of requests, using the available resources very efficiently.

But a slight perturbation in that workload, or a tiny increase in the volume, or even the presence of some other unrelated activity that competes unexpectedly for resources, can turn a well-operating system in to a completely malfunctioning mess.

Performance behaviors have these "knees in the curve", places in their performance profiles where the first derivative of your performance graph changes dramatically.

Worse, this can be very hard to diagnose and adjust, because often these systems don't exhibit these behaviors under synthetic controlled workloads. Your existing performance benchmarks are probably mature and predictable; over the years, your system has become "grooved in" to these workloads, and doesn't exhibit these performance peculiarities under laboratory conditions.

So you are forced to diagnose and debug these performance anomalies under live conditions, in the field, under pressure, with voices raised and tempers flaring.

One lesson is to keep your diagnosis tools and skills ready at all times: have lots of monitoring points in your application, capture lots of information, and be familiar with the available tools for diagnosing that information.

But surely there are better ways to build software so it doesn't fall over in cases like these? Where is the body of research that talks about how to design algorithms and systems that decay gracefully under overload conditions?

Several years ago, I remember that people were talking about "chaos theory", claiming that it would provide techniques for understanding systems that exhibited large responses to small changes in input.

These performance knees seem to present a worthwhile venue for applying such theory, but I don't recall having seen actual results in this area.

Am I just looking in the wrong places?

SHRDLU is wrong!

It turns out it's not

ETAOIN SHRDLU

after all!

Peter Norvig reveals the New Truth:

there is a standard order of frequency used by typesetters, ETAOIN SHRDLU, that is slightly violated here: L, R, and C have all moved up one rank, giving us the less mnemonic ETAOIN SRHLDCU.

Given all the powerful tools that have come along in the last fifty years, it's actually amazing what a good job Mayzner did with the tools of his time.

What's a bit hard to tell is how the Google language corpus differs from Mayzner's " variety of sources, e.g., newspapers, magazines, books, etc."

Linguists of the world, enjoy!

How Complex Systems Fail

I quite enjoyed Richard Cook's presentation at the Velocity 2012 Conference: How Complex Systems Fail. The video is about 30 minutes long, but it moves right along and he is an excellent speaker.

Among the key concepts in the talk is this observation:

As systems developers, we design for reliability:

• stiff boundaries, layers, formalisms
• defence in depth
• redundancy
• interference protection
• assurance
• accountability

But what we actually want is resilience, which is different:

• withstand transients
• recover swiftly and smoothly from failures
• prioritize to serve high level goals
• recognize and respond to abnormal situations
• adapt to change

If you consider yourself a serious systems software engineer, or if you want to become one, you should listen to Cook's talk and go read some of his papers at his web site. He is a clear speaker and writer, and his proposals are sensible and grounded in real experience. Start by reading this concise summary: How Complex Systems Fail, and then move on from there to explore Cook's ideas about how to make systems safer by making them more resilient, for example in Operating at the Sharp End: The Complexity of Human Error.

Thankfully, the systems that I work on don't come close to approaching the safety-critical systems that Cook considers, but I'm quite grateful to him for sharing his experiences and observations, because even ordinary systems software can be made more stable, tolerant of errors, and adaptable, by considering these issues.

Update: Fixed the link to the CTLab site (thanks Anton!)

Update 2: Fixed the video link (is Blogger eating my links? Or am I just getting old...nevermind, don't answer that.)

Computer programming remains the province of experts

Every so often, you'll hear people bemoan the fact that computer programming is so intricate and obtuse and complex, and wonder when we'll arrive at a time when computers aren't so doggone hard to program, and everybody who wants to can simply program computers to do whatever they want.

Well, today's news is that, though that might be a laudable goal, we're still an incredibly long way away from that.

Two postcards from the front lines of today's most mature programming languages, C++ and Java, follow:

• Herb Sutter brings us a treat of a video: You Don’t Know const and mutable (the video is here).

  there's actually another major change that isn't being talked about anywhere, or even being listed as a change in C++11 at all as far as I know, because I and other key experts and committee members I've asked didn't fully realize that we altered the basic meaning of not one but two fundamental keywords in C++. It's a change that has profound consequences, that rewrites and/or invalidates several pieces of pre-C++11 design guidance, and that's directly related to writing solid code in a concurrent and parallel world. This isn't just an academic change, either — everyone is going to have to learn and apply the new C++11 guidance that we'll cover in this session.

• Meanwhile, from Java-land, Heinz Kabutz digs deeply into the meaning of Java's "final" keyword: Final Parameters and Local Variables

  I am not against marking fields final, as this has some useful concurrency semantics. I am also not against making methods and classes final if that is going to help to guide users of your classes on how they should be extended. This is only a rant against marking local variables and parameters as final, when it is not necessary to do so.

An interesting aspect of both of these situations is that they involve bits of the programming language (const for C++ and final for Java) which have been around for more than 20 years (I remember working with const in 1987 or so, and with final in 1995 or so, but they both probably pre-date those memories).

It's my job, nay my career, nay my entire professional life, to learn, keep tabs on, and otherwise eat, drink, and breathe incredibly annoying details like these.

If that is your life, too, then hopefully you found these links useful.

And if that is not your life, then maybe you know understand a little bit more about why it's going to be a long time before the fundamental aspects of why computer programming is hard cease to matter.

Happy New Year!

Don't just sit there watching football (except the Rose Bowl, which ought to be pretty good); get over to your computer and read, read read!

• A few of the accomplishments of 2012 that may have slipped by without you noticing: The Year’s Best in Mountaineering: The 5 Most Impressive Climbs of 2012

  Steck managed to string together a series of summits on the Jungfrau, Mönch, and Eiger by speed climbing each of the mountains and paragliding from one to the next. The expedition began early in the morning and Steck glided from the summit of the Eiger back to his car by 5:00 PM, getting him home in time for dinner.

• Like my nieces, you probably weren't able to watch Netflix on Christmas Eve. Netflix and Amazon attempt to explain why: A Closer Look At The Christmas Eve Outage and Summary of the December 24, 2012 Amazon ELB Service Event in the US-East Region

  At 12:24 PM Pacific Time on December 24 network traffic stopped on a few ELBs used by a limited number of streaming devices. At around 3:30 PM on December 24, network traffic stopped on additional ELBs used by game consoles, mobile and various other devices to start up and load lists of TV shows and movies. These ELBs were patched back into service by AWS at around 10:30 PM on Christmas Eve, so game consoles etc. were impacted for about seven hours.

  It's interesting to see Amazon's analysis: at the bottom, it was a human error, by an Amazon employee:

  The data was deleted by a maintenance process that was inadvertently run against the production ELB state data. This process was run by one of a very small number of developers who have access to this production environment. Unfortunately, the developer did not realize the mistake at the time.

  My co-worker, who has similar production authority, has a fairly simple technique to help himself avoid these sorts of mistakes. He has two accounts, with the high-privileged one named specially, and with a special command-line prompt, and only enabled on the production machine itself, to help re-inforce the mantra: "sign on, issue one command, sign back off".

• Dan Bricklin comments on the end of a podcasting error: the shutdown of Doug Kaye's IT Converstaions podcasts: End of a podcasting era but NYTimes #56,000

  Doug created the second podcast ever (the first was one recorded by Christopher Lydon posted by Dave Winer, if you use Doug's definition of podcasting starting with using the RSS enclosure tag). ITConversations.com started in 2003. The ongoing series of podcasts of many conferences, interviews, and "talk" shows like the Gillmor Gang, showed many of us the value of podcasting and also was of great educational value.

  Are podcasts truly mainstream now? I've fallen out of the podcast world, as I never found a decent podcast client for Linux, and I lost access to the Windows machine I used to use for podcast subscriptions.

• Two contrasting views on our machine-assisted future to start the new year:
  • A 2011 perspective: Why Workers Are Losing the War Against Machines

    The threat of technological unemployment is real. To understand this threat, we'll define three overlapping sets of winners and losers that technical change creates: (1) high-skilled vs. low-skilled workers, (2) superstars vs. everyone else, and (3) capital vs. labor.

  • A 2012 perspective, from the much more optimistic Kevin Kelly: Better Than Human

    This is the greatest genius of the robot takeover: With the assistance of robots and computerized intelligence, we already can do things we never imagined doing 150 years ago. We can remove a tumor in our gut through our navel, make a talking-picture video of our wedding, drive a cart on Mars, print a pattern on fabric that a friend mailed to us through the air.

• You'll want to read Bruce Sterling's annual take on the state of things: State of the World 2013: Bruce Sterling and Jon Lebkowsky

  Greed, like polarization, is a wicked problem. Jamais' response was reasonable, but was it practical? Can we eradicate greed? It's deeply embedded. Even the most enlightened struggle with it (if they say they don't, they're not enlightened).

• Thinking about your own more personal future? Wendy Nather comments on Victor Wong's fascinating What They Don’t Tell You About Promotions in her own Levelling up in the real world.:

  If you upset people inside or outside the team, it creates extra work for your boss, who has to smooth things over. When you create extra work for your boss, you are totally not getting rewarded for it.

• Still need more to read? Check out this great list: Free Datascience books
• This book should probably be on my own personal list: SSH Mastery: OpenSSH, PuTTY, Tunnels and Keys

  This book saves you from sifting a decade of obsolete online tutorials and quickly gets you running:SSH with the OpenSSH server and the PuTTY and OpenSSH clients.

Finally, how could we possibly end without noting that today is the 150th anniversary of one of the greatest days in the entire history of the human race: The Emancipation Proclamation: January 1, 1863:

On the first day of January, in the year of our Lord one thousand eight hundred and sixty-three, all persons held as slaves within any State or designated part of a State, the people whereof shall then be in rebellion against the United States, shall be then, thenceforward, and forever free; and the Executive Government of the United States, including the military and naval authority thereof, will recognize and maintain the freedom of such persons, and will do no act or acts to repress such persons, or any of them, in any efforts they may make for their actual freedom.

For this, then, God Bless America. Here's looking forward to 2013!