NYT article on "black-hat" SEO activity

The lead article in the Business section of today's New York Times is titled The Dirty Little Secrets of Search. It discusses some of the various techniques that web site operators use to try to influence the algorithms that Google's search engine uses to rank results.

Sadly, the article's headline promises much more than the story actually delivers; I thought the article was interesting, but ultimately frustrating.

Although the writer is able to secure interviews with Matt Cutts of Google, and with a mysterious "Mark Stevens" of an unnamed company (neither the company's name, nor "Mr. Stevens"'s name is revealed in the article, though it's hard to see why), the article ends up mostly teasing, and exposing very little about this seamy under-belly of the web search world.

We never find out exactly which company was gaming the Google results, nor exactly how.

We never find out whether Penney was aware of that company's techniques, or how they selected that company, or much of anything about Penney's relationship with the unnamed vendor that they used.

We never find out exactly what Google did about the situation, other than that "manual action" was taken.

The only concrete and specific piece of evidence that the article seems able to unearth is a bit about a web site in Switzerland which posted a strange and completely unrelated link to the Penney's web site, apparently via a company called TNX.net, but, as the article says: "Efforts to reach TNX itself last week via e-mail were not successful."

Clearly there is a thick curtain here, and the Times apparently decided that they needed to publish what they had, now, even though they had very little to actually write about, because the story, such as it was, was nearly over: "On Feb. 1, the average Penney position for 59 search terms was 1.3. On Feb. 8, when the algorithm was changing, it was 4. By Feb. 10, it was 52." The Times writer ultimately fails to provoke either Google or Penney to talk about the details of what happened; Google's Cutts flat-out refuses to say: "Mr. Cutts said he did not plan to write about the situation ... because Google's goal is to preserve the integrity of results, not to embarrass people." So, instead, the article wanders around, venturing into what the author himself notes is just "another hypothesis, this time for the conspiracy-minded."

This is clearly an important topic, and serious. There is real money at stake, and real questions about legality, and ethics, and transparency. It's frustrating that even as powerful an institution as the New York Times can't break through and bring some real sunlight into these hidden corners.