Recently, I've been working with various Java code coverage tools, such as Emma and JaCoCo.
Generally, these tools are quite powerful and sophisticated.
However, I recently learned of a rather fundamental limitation: they only measure a block as "covered" if that block terminates normally; if the block of code terminates due to a thrown exception, the block is not marked as covered.
The Emma documentation discusses exceptions in sections 2.5, 2.6, and 2.7 of the FAQ:
Thus, EMMA does not attempt to track the success or failure of every single bytecode instruction: instead, it marks a basic block covered when the control reaches its last instruction. A covered basic block is thus guaranteed to have executed without failures at least once in a given coverage run session....
the motivation behind this has been that most legitimate code is written so that methods return normally (without throwing an exception).
I'm sure I wasn't the only person unaware of this limitation.
There doesn't appear to be a way around this. So, for the time being, just because your Emma or JaCoCo coverage report indicates that a particular block of code is uncovered, you can't immediately conclude that it isn't invoked at all.
It might have been invoked, but terminated due to a thrown exception.
How is this situation handled in other code coverage tools, for other languages in which exceptions are a first-class part of the language?
No comments:
Post a Comment