Implementing a cache in Java is quite simple, although there are refinements and elaborations which can make this quite complicated, too. The simplest sort of Java cache typically consists of:
- A collection object, such as a java.util.Hashtable or java.util.HashMap, to hold the cached items and allow them to be easily searched.
- Usually there is a companion data structure which supports the cache's replacement policy.
- Typically the cache is packaged up in some manager class, which also holds configuration information, which is responsible for the creation and management of the cache, and which has apis to fetch objects from the cache, and (depending on the cache's policy) also has apis to add objects to the cache, evict objects from the cache, and manage updates to cached objects.
Note that this is only interesting when the objects in the cache are of differing sizes. If all of the items in your cache are identically sized, this is not really a very interesting problem. But many Java programs have objects of all sorts of differing sizes, so it's actually quite common that a cache will find itself holding objects which have a variety of sizes.
So imagine that you are trying to draw a circle around the cache, such that the cache's own data structures, and all of the cached objects, are inside the circle, and the rest of your Java program's memory usage is outside of the circle, and you want to know how much memory is inside that circle ("this cache is currently using 65Mb").
I've been unable to figure out a clean way to do this.
Ideally, I'd like to find a solution which the cache could implement itself, in its own Java code, but I'd also be glad if I could just find any solution to the problem, for now.
Here's some ideas that I've thought of so far:
- Use a tool which does this for you. Do such tools exist? I've tried messing about with things like jmap/jhat/hprof, and it looks like the OQL support ought to allow me to figure out how much memory my cache is using, but I haven't yet figured out how to get it to work.
- Run a series of experiments, using a workload which is capable of causing the cache to be bigger or smaller, and adjust the JVM's memory limit using the -Xmx argument, such that for each various workload, the -Xmx argument is set to the smallest possible value that enables the test to pass. Then correlate the size of the workload with the size of the cache: "with 1000 entries in the cache, we can just run with -Xmx128m, and with 5000 entries in the cache, we can just run with -Xmx160m, so 4000 cache entries must use 32M of space".
- Call Runtime.freeMemory before creating the cache. Then put some entries into the cache and call Runtime.freeMemory again. Then put some more entries into the cache and call it again. And so forth. Then correlate the changes that are seen in the return from Runtime.freeMemory with the objects that were put into the cache: "After adding 1000 entries to the cache, Runtime.freeMemory returned a value that was 10M smaller, so 1000 cache entries must use 10M of space".
- Enumerate the cache, and use reflection to examine the entries in the cache, as well as the cache itself. For each instance of each object "inside the circle", figure out what the actual runtime type of that object is, and thus what fields the object has. For each such field, figure out its runtime type, recursively. Eventually, you'll get down to the point where the fields that you're looking at are of Java built-in types, such as String, int, boolean, Long, etc. For each one of those instances, you can determine its actual size (though this is certainly implementation dependent). For example, a String generally takes 2 bytes per character in the String, plus a certain amount of additional overhead imposed by the implementation of the String class, whereas an int generally takes 4 bytes total. Add up all of these sizes, for the object and all of its sub-objects, and you have some sort of first-level estimate of the amount of memory consumed by the cache. (Aside: a truly complete implementation of this concept should allow for the fact that there may be object sharing among the items in the cache, so if two entries share a sub-object, for example they both point to the same instance of the same String, then we want to be careful not to count the shared sub-object's size more than once. But that's just a detail, and the algorithm is I think still tractable.)
Ideas? Suggestions? Am I overlooking something really simple and easy here?