Tweeting the Vancouver games

Technology never stops evolving; everybody has to keep up.

This year, the International Olympic Committee found it necessary to post official guidelines regarding what was appropriate (and inappropriate) for athletes to post on their Twitter, Facebook, or blog pages during the Olympic Games.

Let's see, does

Just nailed that inverse 720, dude!

fit in an SMS message?

I think it does!

Some changes in my work life

I'm actually quite excited about the implications of the recent changes in my day job. Finally, the software I've been pouring my life into for the last 8.5 years will be backed by the resources and market reach of the largest software company on the planet!

At a personal level, there will be a number of changes. I'm sad that I'll no longer get to commute to work with my wife; it's been a wonderful six year run of our shared commute. And I'm sad that I'll lose my beautiful private window office. AmberPoint's executive team is a very enlightened group who really understand the importance of a great work environment for software engineering, and I've tremendously enjoyed being able to work in Greg's team.

But overall, I'm hopeful about the future, and looking forward to learning more as the details are revealed.

FAT32 filesystem limits, DRDA CMDCHKRM and DERBY-3729

As part of DERBY-3729, I've been diving back into DRDA, an area I hadn't visited in several years.

DRDA is IBM's Distributed Relational Database Architecture protocol. It is the client-server protocol which was chosen for the implementation of the Apache Derby network server functionality. DRDA is extremely rich and powerful, but unfortunately it is not simple. However, it is very thoroughly documented; the complete DRDA specification is available from The Open Group.

In the particular case of DERBY-3729, the issue involves a fairly simple question:

When the server runs out of disk space, or is otherwise unable to write any more data to the database, how should it notify the client?

In general, DRDA accomodates the return of error message information from the server to the client. However, in this case, the issue is complicated because of the severity of the error. In Derby, as in (probably) all standard SQL implementations, there are a range of severity levels:

• Warnings, which are simply returned to the client following the statement execution. For example, I think "value was truncated to fit" is a warning.


• Errors which cause the current statement to be aborted. For example, I think "unique key constraint was violated" is a statement-level error.


• Errors which cause the current transaction to be aborted. For example, I think "deadlock occurred and you were chosen as a victim" is a transaction-abort error.


• Errors which cause the current connection/session to be closed. These are not very common in Derby except due to internal errors. One example, though, is when you try to connect, but give invalid arguments for the connection parameters. Then you get an error and your (never-really-created) connection is closed.


• Errors which cause the current database to be closed. These usually involve I/O errors which are affecting Derby's storage engine. Note that since Derby can support multiple databases simultaneously, it is possible that one database is on a bad disk drive while other databases are still OK, so only the failing database is shut down.


• Errors which cause the entire system to shut down. These are extremely rare, and mostly involve internal logic errors that are detected in critical pieces of the Derby engine. For example, if an unexpected exception occurs while aborting a transaction, Derby concludes that something is horribly wrong and shuts the entire system down rather than risking further damage.

Back to the case at hand: DERBY-3729.

In this case, it wasn't that the disk was full; rather, since the user had configured their system using the FAT32 filesystem format, individual files are limited to 4 gigabytes in size. Once a file gets that large, attempts to make it larger are rejected, with an error that turns into an IOException in the Derby storage engine. The IOException is caught and treated as a database-severity error, which results in:

• The statement and its transaction are rolled back (if possible)


• The session and its connection are closed


• The database is shut down


• The client is informed that a "command check" has occurred"

A "command check" is the DRDA message which is used when an error of such severity has occurred that it caused the connection to be closed. It is conveyed as a CMDCHKRM, which is DRDA jargon for "command check response message".

Unfortunately, the normal Derby client-server error message communication mechanism requires that the connection remain open, because generally the Derby server just sends a message "code" to the client, which then requests the full error message details from the server by making additional protocol calls back-and-forth. In this case, since the connection is closed, we only get one shot to convey any error message information, which is via the CMDCHKRM message itself.

It turns out that the CMDCHKRM message always contains a SQLCARD, which is a SQL Communications Area Response Data object, which allows a small amount of error information to be carried inline.

So, to try to resolve DERBY-3729, I:

• Enhanced the server-side error message text to reflect that there are other possible causes of I/O errors besides the disk being full, such as a filesystem limit (FAT32) or a quota being reached.


• Enhanced the client-side error message text to reflect that, when a CMDCHKRM is received, there may be additional information available in the server-side derby.log file.


• Enhanced the client code which processes the CMDCHKRM message to look for the SQLCARD, and to fetch whatever summary error message text is present in that object, and include it in the client-side exception that is thrown.

Hopefully this will help the next user who runs into this problem.

I wonder how long the FAT32 filesystem format will still be in use?

I'm sorry you had to see me like that

This is perhaps the greatest TV review of all time.

Sure, it's just TV, and it's just Lost, and it's just entertainment, but what a wonderful review!

Salman Rushdie, and Gottfried Liebnitz, and Soren Kierkegaard, and Dante!

When reviewers are saying things like this, we've moved beyond simple entertainment; we're definitely in the realm of Art:

One wonders if the entire season 6 side ways story line will model the general thematic thrust of the castaway story, but with different incidents and events — a gritty, more down-to-earth version of the mythic, larger-than-life Island epic, like how Dorothy's adventure in Oz was a fantastical extrapolation of her life in Kansas. Lost also loves its Alice in Wonderland references, and so we recall that Lewis Carroll's sequel to Alice's Adventures In Wonderland was entitled Through The Looking Glass, which begins with Alice gazing into a mirror and wondering if it could be portal into a topsy-turvy Otherworld. The book itself is a cracked mirror reflection of the previous book — the same story in essence, sharing similar if not identical themes, just rendered with different incident and detail.

I left my weekly soccer game early; as I was walking out the door, my teammates asked: where are you going?

I'm going home, I said. I made a promise to myself, that I was going to watch the last season of Lost.

And so I am.

Tracer-T!

Raymond, you really made my day!

Joel and Jeff on programmers who blog

During episode 81 of the StackOverflow podcast, Joel and Jeff got into a fairly interesting discussion about programmers who write blogs.

Joel encouraged programmers who blog to focus on quality over quantity. He said that, when you write an article, you should try to narrow your focus down to some little tidbit of information that you are particularly interested in and familiar with, and you should attempt to write the world's greatest article on that subject. He used the example of "chocolate-covered strawberries", and suggested that you shouldn't just write another article talking about some neat chocolate-covered strawberries that you made, but you should really dig deeply into the particular details that fascinate you, like the particular type of cocoa powder to use. He said, your goal should be to write the article that will be the #1 hit on Google for "chocolate-covered strawberries".

Jeff differed with Joel (he often does), and offered a number of alternate perspectives:

• Firstly, he said, you should blog about what interests you, and not worry so much about what might interest others.


• Secondly, he said, you should make the barrier to blogging fairly low. There are already plenty of obstacles to blogging, and if you spent all your time worrying that your writing isn't good enough, or doesn't deserve to be posted, you'll never blog anything at all


• Thirdly, he said, writing is a skill, and like any skill it must be practiced, and so you should exercise your writing skills regularly and routinely. The more you write, the better you will get at writing, so write as often as you can.


• Fourthly, he said, simply writing about something will, often subconsciously, cause you to improve your knowledge about it. If you blog about X, in the process of doing so, you will find that you learn more about X. Partly this is because you'll find yourself doing some additional research about X; partly this is just because blogging about X will force you to organize your thoughts and organizing your thoughts is one of the ways you learn

Essentially, Joel said: programmers who blog should blog for their readers. While Jeff said: programmers who blog should blog for themselves.

I've only been seriously blogging for about 8 months, but I find myself strongly on Jeff's side in this area. I'm not trying to be some sort of A-list world-renowed blogger; instead, I'm trying to keep track of the particular topics that I'm interested in, practice my writing skills, record notes and observations about things that I learn, etc.

So, I'm sorry, Joel, but you're never likely to get to my blog from the top of a Google search, and if you're looking for information on chocolate-covered strawberries I've got nothing to offer. But I am feeling good about the return on investment by keeping this blog, so I'm intending to continue it more-or-less as it has been, as long as I can.

Ceremony vs Essence

I was listening to one of Scott Hanselman's Hanselminutes shows the other day. I can't remember whether it was show 196 or show 197 (probably it was the IronRuby show), but in one of those shows, the conversation turned to Ceremony vs Essence.

I hadn't heard of Ceremony vs Essence before, so I've been learning about it.

Ceremony vs Essence, as I understand it, is a framework for critiquing programming languages. It appears to have been invented by Neal Ford of Thoughtworks, and delivered in a series of conference keynotes in 2008. (Unfortunately, I can't find his keynote slides available online anywhere -- anybody have a link?)

The fundamental idea of the Ceremony vs Essence idea appears to be that, all other things being equal, programming languages should attempt to allow programmers to clearly express the essence of their programs without being caught up in excessive ceremony provided by the programming language.

That is, it appears to be a form of Occam's Razor: an argument for simplicity and clarity, and for avoiding unwanted extraneous clutter.

So, for example, some people look at a statically typed language like C or Java, and observe that the need to define your variables ahead of time, and specify exactly the type of the variable, is "ceremony", and distracts from the "essence" of the use of the variable. This point is made quite well by a pair of essays by Stuart Halloway:

• Ending Legacy Code in our Lifetime (see also his presentation slides)


• Refactoring from Ceremony to Essence

Clink Shank makes a similar point, from a testing perspective, in his essay: Essence over Ceremony in Unit Testing.

I think that, setting aside for a moment the language debates regarding Ruby-versus-Java-versus-JavaScript-versus-whatever, there is clearly a universal point to be made: simpler is better. Simpler is clearer, simpler means less code, simpler is easier to tune, to maintain, to document, to debug.

One of the best aspects of the code review process that occurs during the analysis of a patch proposal on the Derby project is that each successive pair of eyeballs will look at the code and, sometimes, will notice: "Say, you know, you could write this more easily if you did this, or did that, or did such-and-such".

Nobody is a perfect programmer, and no code is ever perfect. The ongoing study, refinement, and improvement of software has no end.

And is ever-enjoyable.