Wednesday, December 9, 2015

fsync and Virtual Machines

Oh, the wonders of low-level systems software.

The documentation for fsync() says:

fsync() transfers ("flushes") all modified in-core data of (i.e., modified buffer cache pages for) the file referred to by the file descriptor fd to the disk device (or other permanent storage device) so that all changed information can be retrieved even after the system crashed or was rebooted.

But what if you are running in a Virtual Machine?

An interesting article says:

Hypervisors outperforming the host machine is the most interesting to me. The results of this test clearly show that the hypervisors must be lying about synced writes for performance. This corroborates what I’ve seen with Packer as well, where if the virtual machine is not cleanly shut down, committed writes are lost. fsync() in a virtual machine does not mean that the data was written on the host, only that is is committed within the hypervisor.

And yet, another interesting article says:

ESX(i) does not cache guest OS writes. This gives a VM the same crash consistency as a physical machine: i.e. a write that was issued by the guest OS and acknowledged as successful by the hypervisor is guaranteed to be on disk at the time of acknowledgement. In other words, there is no write cache on ESX to talk about, and so disabling it is moot. So that’s one thing out of our way.

Is this more than a "he says, she says" thing? That is, is there a more definitive resolution of this question somewhere?

Note, btw, that the core behavior of Linux's fsync() command is not even clear, putting aside Virtual Machines:

By durable, I mean that fsync() should actually commit writes to physical stable storage, not just the disk write cache when that is enabled. Databases and guest VMs needs this, or an equivalent feature, if they aren't to face occasional corruption after power failure and perhaps some crashes.

I'm sure there are places where this information is definitively and clearly documented.

I'm just not sure where those places are.

1 comment: