About a week ago, we started to get some indications that something was wrong in our internal network. Specifically:
- Various automated jobs and test runs started to fail, with error messages indicating that accounts were locked out
- Various machines running Symantec Anti Virus File System Auto Protect started popping up dialogs indicating that AutoProtect had detected and removed an instance of W32.Downadup.B.
- Using tools such as the Conficker Eyechart, the Microsoft MSRT, and Symantec's scanner, we tried to determine which machines were actively infected. These machines we shut down and disconnected from the network.
- For those machines, we then removed the virus, verified that all Windows Updates were applied, re-scanned the machines, and then restored them to the network.
- We monitored network and security activity, looking for machines that we had missed.
- The Symantec Auto-protect pop-ups on other machines claimed that they had received the virus from my machine. Multiple machines detected this, so it's hard to dismiss it as a single outlier.
- On my machine, when we examined it in detail, the Automatic Updates and Background Intelligent Transfer Service services were disabled. This is one of the symptoms of the Conficker virus; it shuts these services down to try to prevent the infected host from running Windows Update.
- Multiple virus scans of the machine, by multiple virus scanners, failed to detect the virus, although the various virus scanners detected the virus on other machines successfully.
- None of the special registry entries that the virus is supposed to create were present.
- None of the mystery files in the system directory were present.
- The in-memory DNS hooks that the virus uses to disable Windows Update (and which are checked by the eyechart) were not present, and the eyechart displayed without errors.
- Our network scanners did not detect the suspicious network traffic that was present with other infected machines (e.g., the traffic which was trying to test for machine users with weak passwords).
It's possible that I disabled the system services myself. Since I routinely use this machine for complicated long-running performance tests, I occasionally do things like disable background system services for a while to avoid interference with the tests, then forget to re-enable them. Though I don't rememer doing that in this case.
At this point, the virus infection appears to have subsided, which is good.
But it's frustrating that I ended up from the experience not understanding some basic things such as:
- which machines were infected, and why? Many of the infected machines were unpatched, and were not running the AutoProtect scanner, so they were not well defended. But at least two machines which were infected should have been protected.
- how did the virus originally enter the network? None of the infected machines appear to have been the source of the virus.
- Was my machine infected? If so, how, and is it still infected?
Such is life in the modern world.
Update: Conficker is astonishingly sophisticated.