Quantitative funds, after all, are the "black boxes" of investing -- portfolios run by managers who generally try to generate profit with computer algorithms that they don't share with outsiders, or even their own investors.
When you put your money into one of them, you are trusting not only that the overall strategy is sound, but also that its algorithms make sense and, furthermore, that they have been translated properly into computer code.
The article describes how AXA Rosenberg, the quantitative analysis firm in question, uncovered the problem, and links to several fascinating documents from AXA Rosenberg describing how the bug was discovered, and what its impacts were. From their April 15th letter:
We have been working on a number of projects -- in particular an insight into how to model rapidly emerging, transitory risk (our new "state contingent model"). In this process, we discovered the coding error in the scaling of the common-factor risks in the optimization process associated with an earlier upgrade of our risk model.
Modeling and simulating real-world systems inside the data structures of a computer program is an extremely powerful technique, used throughout the software industry. As the systems become more complex, and the models become more complex, verifying the accuracy of the models becomes simultaneously more important and harder.
Review and open critique are about the only way to catch such problems, I think; you need to exploit Linus's Law: "with enough eyeballs, all bugs are shallow". Of course, this is very hard to square with the culture of secrecy cultivated by the quant community; as seen above, these are "algorithms that they don't share with outsiders, or even their own investors".
So it is particularly interesting that the Times article goes on to discuss the question of whether it was possible to detect the presence of the bug, even without having direct access to the algorithms and to their risk models:
One question is whether AXA Rosenberg itself -- or the various mutual fund groups, financial advisers and consultants that have used its services -- monitored its operations with sufficient rigor.
The Times article notes an essay by a principal at another quant firm, which directly addresses this question. In the essay, Michael Markov and Kushal Kshirsagar say:
Because the “coding error” apparently impacted risk controls, we examined two basic risk measures that are routinely used by both fund managers and investors to evaluate and monitor investment products: Beta and Tracking Error.
Their article presents their analysis, with charts showing that the effects of the bug were visible, with the right data, but were not visible if the same analysis was performed with less detailed data.
It is worth stressing that such an apparent aberration in the fund’s risk profile could be most clearly seen using daily data. Unfortunately, investors typically use monthly data even though daily returns are now easily available from data providers (e.g. Lipper), public sources (Yahoo, Google, etc.), funds and custodians. When using monthly data, a longer history needs to be used to have sufficient observations to estimate the regression. This longer history may cloud, or, as in the case of the Laudus Rosenberg fund, completely transform the picture.
It seems unlikely that we will see the rise of open source quantitative analysis. Trading algorithms are likely to remain proprietary, protected by patents, trade secrets, and other similar legal constructs. Would the world be a better place if open source techniques were adopted in more arenas, besides the low level of operating systems and databases where they are currently successful? I'm not sure. But I do feel like the financial industry could do with a very large dose of "openness" being delivered to them, and it would certainly be nice to feel like, in my lifetime, we will get to a point where these banks and traders stop holding the world hostage with their too-big-to-fail, too-complex-to-understand, too-buggy-to-operate-correctly computer based financial products.
No comments:
Post a Comment