A Mac OS X firewall mystery

Where do I go to learn more about the implementation and behavior of the Mac OS X firewall?

I had a very strange situation recently; it will take a bit of time to explain, but maybe somebody can shed some light on what was happening?

Here's the situation:

• I run Mac OS X 10.6.6, with all the latest patches


• I also run VMWare Fusion version 3.1.2 on my Mac


• I have a variety of guest operating systems that I run in VMWare

I was running a suite of client-server networking tests, with the server on a VMWare guest, and the clients on my Mac host. The test harness is a script with lots and lots of client invocations; during a test case, we generally run a client which initiates a connection to the server, does some work, and then exits.

The behavior I saw was as follows:

• The tests would occasionally run to completion, but usually they would run partway, then the network connection between the host and the VMWare guest would be disrupted


• When the disruption occurred, the host could continue to talk to other networked machines, both near and far.


• And, the guest could continue to talk to other networked machines, both near and far.


• But the host and guest were having trouble talking to each other.


• Simply waiting for an hour or two, the network connection between the two machines would appear to "magically" repair itself. Or, re-booting the host would repair the connection (rebooting the guest was not enough).

After lots of configuration and experimentation, I discovered that disabling the Mac's built-in firewall software stops this problem from occurring.

But what I don't understand is: why? The firewall, as I understand it, is supposed to be controlling in-bound connections from other machines into my Mac. But in my test suite, all of the network connections that I was making were out-bound, from my Mac to my VMWare guest. So why was the firewall involved in that network processing at all?

For now, it's a mystery, although happily one that I care much less about since I've figured out this workaround.

But it does leave me with that initial question: where do I go to learn more about the implementation and operation of the built-in firewall on Mac OS X 10.6.6?

FUSE, CUSE, and UIO

Recently I've been spending a little bit of time learning about FUSE, CUSE, and UIO, which are related technologies for user space device driver implementations in Linux.

What ties these various technologies together is that they are Userspace techniques for implementing functionality that previously required kernel-mode programming. As the Userspace I/O HOWTO says:

For many types of devices, creating a Linux kernel driver is overkill. All that is really needed is some way to handle an interrupt and provide access to the memory space of the device. The logic of controlling the device does not necessarily have to be within the kernel, as the device does not need to take advantage of any of other resources that the kernel provides. One such common class of devices that are like this are for industrial I/O cards.

To address this situation, the userspace I/O system (UIO) was designed. For typical industrial I/O cards, only a very small kernel module is needed. The main part of the driver will run in user space. This simplifies development and reduces the risk of serious bugs within a kernel module.

The general UIO framework is now more than 4 years old and is quite well integrated into Linux. There is lots of information available about the general topic of User Space Interfaces in Linux.

FUSE techniques are also quite well-known and accepted, and there is a long list of interesting FUSE implementations, and lots of great documentation about how to build FUSE-based filesystems.

CUSE, which is an extension of FUSE to handle character devices, is younger and still evolving. There is still considerable controversy about how and when to use CUSE effectively.

Apparently there is even a BUSE, for Block device User Space drivers, although this appears to be sketchier still than the CUSE work. And you can certainly find people who still find the whole world of Linux device driver writing controversial, though you'd think that this program would have resolved most, if not all, of those complaints.

Luckily I don't have to write device drivers or filesystems myself, though I find them fascinating to study. I wandered down this path because I was trying to get a more clear understanding of the difference between the f_bsize and f_io_size fields in the statfs structure, and how those fields relate to the f_frsize and f_bsize fields in the statvfs structure. I'm still engaged in that investigation, but at least my side-trip into Linux user-space driver implementations was interesting.

Ken Olsen has died

Ken Olsen, the former head of Digital Equipment Corporation, died last weekend at the age of 84; here's his obituary in the Boston Globe.

I was a heavy user of DEC systems for many years. I worked on a VT100 emulator for IBM PC's back in 1988 at CCA in Boston. I had a VAXStation in 1990, and learned to program DBMS storage and recovery systems on it; we used to read the VMS microfiche to understand how the cluster lock manager worked. I once interviewed with a DEC team which was building an early workflow product to run on departmental mini-computers; that product eventually became Forte Fusion and was my first introduction to XML.

Later, I had an Alpha box, and ran Digital Unix; I remember that box as being the one where I learned about the various different techniques for synchronizing access to shared memory.

DEC rose, foundered, and then was bought by Compaq, which in turn was bought by HP.

Many brilliant engineers worked at DEC during its time; Mr Olsen, you certainly ran an influential and fascinating company.

Just how hot is the Google I/O conference?

Is it actually true that this year's Google I/O conference sold out in only 59 minutes?!! That is astonishing.

Recently this has been a very interesting conference, with lots of information about the latest topics. Here's the current list of sessions, but I think it's just the tip of the iceberg; there will be more sessions than these.

Source code and a paean to MSDN

It will surprise nobody to hear that, for me, it's all about the code. Clear requirements statements are a must, crisply-written design specifications are crucial, architecture diagrams can convey lots of information, but when it comes to truly understanding a system, service, or API, I want code.

Sample code, test code, prototype implementation code, patches, snippets: give me code.

So I find myself really surprised that, after all these decades, no major computer system comes close to MSDN when it comes to recognizing the importance of code to students. Microsoft does provide plenty of those other materials, but just wander around for a while in the MSDN library and you'll immediately see what I mean about MSDN and code. For instance, here's a page I pretty much randomly brought up: FindFirstFile. There is plenty of descriptive text, of course, but there, big and clear and right up front is the Examples section.

And Microsoft's examples are, almost always, excellent: they are short and clear, but realistic; they include just enough complexity to show things like error-handling, parameter values, possible gotchas you should be prepared for. And if that example wasn't enough, look just after it:

For another example, see Listing the Files in a Directory.

Heck, their HTML even includes a convenient Javascript "Copy" button which nicely copies all the source code into your system clipboard to paste into your editor.

Furthermore, there's just no substitution for source code for understanding complex examples. Try this: Enter overlapped into the MSDN search box. You'll find yourself guided to Synchronization and Overlapped Input and Output, which is a fine page by itself, but (gasp!) no source code! Yet fear not, for the MSDN tech writers know what they are doing; within another click or two you'll be checking out GetOverlappedResult, and what does it say under the Examples section? That's right, "see Testing for the End of a File".

Simple things are simple, complicated things are possible, but, everywhere you go, and everywhere you look, source code. It's what programmers eat, drink, and breathe, and Microsoft is fully aware of that, and gives you lots and lots and lots of code.

So why don't other systems do this? It's been 30 years now, guys! Why is it then when I try the same thing for (say) Linux, and I want to learn how to read all the files in a directory, I can pretty quickly find my way to the Linux opendir API and its manual page, and that of readdir. But where's the source code?

Yes, there are 80 jazillion web sites around that collect this sort of thing, and try to make it available, and if you search for "linux opendir readdir" you'll soon enough find something tolerable, albeit short, poorly-formatted, and cursory.

So please, guys: come into this century, and learn from Microsoft! Your developer documentation is targeted at developers, so show them what they want: source code. When I type

man readdir

in my terminal window, it's great that I get a quick summary of the API and its arguments and return values, and it's nice that you've accumulated some text over the years about some of the complexities of calling the function and dealing with the situations that may arise.

But please, include the Examples! It matters, it really does.

And it's not just operating systems that suffer from these problems. Try learning to program a DBMS in SQL: you know you'll be needing to write a SELECT statement, so let's see what some manual pages look like. Here's one; pages and pages and pages of descriptive text, "railroad"-style syntax diagrams, and, buried down at the bottom, a handful of examples. But here's Microsoft's page; it's short, and what's that down at the bottom of the short and simple page? YES IT'S THE SAMPLE CODE!

Or suppose you want to do some network programming, and you're working in Java, so you look up the Javadoc for the socket object. Groan: text, text, text, and more text. Yes, it's all hyperlinked, but it's just more stupid text! How does Microsoft do here? Well, here it is: yes, there's a lot of text this time (boo on you Microsoft), but there, again, at the bottom: Yay! Example code! Simple, clear, ready to compile and run!

So please, everybody: I know it's hard to admit that Microsoft just has the whole world beat here, but can't all you other folk just get over that, and couldn't you just start putting those examples into the developer docs? Please?

We programmers will thank you. Over and over again.

Individually controlling the power management behavior of your LAN adapter

A co-worker recently showed me another (yes, yet another) place in Windows 7 that I hadn't visited before:

1. Bring up Network and Sharing Center, then find your Local Area Connection


2. Click on Local Area Connection to bring up Local Area Connection Status, then click on the Properties button.


3. In the Local Area Connection Properties, click the button near the top labeled Configure...


4. You'll be taken to a new box labeled (something) Network Connection Properties.
   

   
   Bwaa-hah-hah-hah-HA! You thought you were working with your connection properties, but now you have found your connection properties!
   
   You are in a maze full of network properties, all alike...
   



5. In the Network Connection Properties dialog, you will now see a tab labeled Power Management


6. In the Power Management tab, you will find a checkbox:
   

   
   Allow the computer to turn off this device to save power
   

On my machine, at least, this box was checked by default, which surprised me. I have a desktop computer! Why does Windows think it would be convenient or useful to sometimes turn off this device? Does it use a significant amount of power to keep my LAN network adapter running? I guess so...

Anyway, the point of this post, other than possibly letting you know about Yet Another Corner Of Windows 7 That You Haven't Visited Before, is to note that, although I found this checkbox, and I unchecked the checkbox, I am still a bit puzzled: how can I tell if this has had had any effect?

That is, how can I tell when Windows 7 has turned off my network connection device to save power, and how can I tell when Windows 7 has turned my device back on?

The wheels of government grind slowly

Today I read that the SEC and the CFTC are moving closer to implementing one aspect of the new Dodd-Frank Wall Street Reform and Consumer Protection Act:

The Securities and Exchange Commission today voted unanimously to propose rules defining security-based swap execution facilities (SEFs) and establishing their registration requirements, as well as their duties and core principles.

Dodd-Frank requires security-based swap transactions that are required to be cleared through a clearing agency to be executed on an exchange or on a new trading system called a security-based swap execution facility.

The Dodd-Frank Act further requires security-based SEFs to be registered with the Commission and specifies that such a registered security-based SEF, among other things, must comply with 14 core principles.

The SEC press release calls out the 14 core principles in detail. I'm neither a lawyer nor a banker, but they seem like good solid principles to me.

It's great to see that this is finally occurring, but my it sure takes a long time:

• Today's announcement allows two months for public comment.


• The final rules are not expected to be in effect until July, 2011


• That's a full year after the Dodd-Frank act was signed into law

Moreover, it's nearly 3 years since the collapse of Bear Stearns, and 13 years since the famous collapse of Long-Term Capital Management. By the spring of 2008, even the financial industry itself was calling for regulations such as these:

He also wants new government oversight of the arcane world of credit default swaps, a business with a notional value and risk of $50 trillion. “Everyone is missing the elephant in the room,” he said.

It was the interlocking relationships between thousands of investors and banks over credit default swaps that pushed the Fed to help rescue Bear Stearns. In particular, Mr. Griffin wants the government to require the use of exchanges and clearing houses for credit default swaps and derivatives.

That way, instead of investment banks playing matchmaker between parties, an exchange will do it with strict rules in place, eliminating billions of dollars in exposure and creating more transparency.

“It’s not sexy, but it’s simple, it’s cost forward, its straightforward, and it’s what we should have done after 1998,” referring to the collapse of Long-Term Capital Management, a big hedge fund. He added that it “is a very sad commentary on where we are from a regulatory perspective” that such a move hasn’t happened already.

The world is a complicated place and I understand that these rules and regulations are complex and intricate. I'm pleased that progress is being made, just (slightly) dazed that it takes such an incredibly long time.