Code Cache, Compilation & Inlining
mail at nmichael.de
Tue Jan 13 03:45:49 PST 2009
we're running Java 1.6.0_07 (Hotspot 10.0-b23) 32bit on SPARC Solaris
(Client as well as Server VM).
Our JVM options:
-Xms1400M -Xmx2400M -XX:NewSize=512M -XX:MaxNewSize=512M
-XX:MinHeapFreeRatio=25 -XX:MaxHeapFreeRatio=40 -XX:PermSize=32m
-XX:CompileThreshold=1500 -XX:+PrintCompilation -XX:+PrintInlining
During the last weeks, I have spent some time looking closer at
compilation and inlining, partly triggered by some "changing/degrading
performance" we were observing in some scenarios. To trace compilation,
I have switched to the 1.7.0-ea-fastdebug JVM (Hotspot
14.0-b07-fastdebug) and ran some tests. I have some questions about what
I've noticed and hope to get some help here.
With the Server JVM, the code cache became full after 7827 compiles:
"Java HotSpot(TM) Server VM warning: CodeCache is full. Compiler has
Setting -XX:ReservedCodeCacheSize to a large value got rid of this error
and the JVM compiled 8809 methods in this run.
My questions are:
- How large is the code cache per default?
- How can I monitor the current occupancy of the code cache? Neither
jstat, jinfo nor a dump of all perfdata values seem to give any hint on
- Can I see the code cache as a memory segment with pmap? Is it an
- Is there any limit for the code cache? In my tests where I increased
it, I set it to 128m (which is much too large, I assume, but worked).
With the Server JVM and a large-enough code cache, I see some methods in
the log being compiled, then being "made not entrant", being "made
zombie" and then being compiled again, made not entrant, zombie, being
compiled, ... Afaik, this "not entrant" and "zombie" stuff is some
deoptimization, when the JIT thinks some of its original assumptions
about those methods weren't good enough, right? Unfortunately, I can't
find much info on the net about this. Could someone probably explain
what those really mean and why this might happen?
BTW, our CompileThreshold is set to 1500 also for the Server VM, and the
process has been running for about half an hour or so. I didn't yet make
any really long-running tests for hours or days with enabled
+PrintCompilation, so I don't know whether this is an infinite loop of
deoptimization and compilation or just happens "for some time". But
still, even after half an hour I would not expect such a thing any more.
Now I'm referring to tests that I have done with the Client VM.
I have noticed that not all methods are being compiled. A Sun Studio
profile for a 1h run (after 30min warmup) is showing that 2.2% of the
CPU time is due to Java methods being interpreted:
Excl. Incl. Name
User CPU User CPU
26363.141 26363.141 <Total>
605.223 1049.764 Interpreter
587.311 587.311 java.lang.String.equals(java.lang.Object)
I tried -XX:-DontCompileHugeMethods: This decreases the CPU time
consumed by the Interpreter to 435.505 (Excl. User CPU) -- but still,
some methods seem to be interpreted. We're running an application that's
doing the same stuff all the time and doesn't have any change in
behavior, so a CompileThreshold of 1500 should easily be reached quite
fast. I remember tests for 2-Tiered together with Steve Goldman about 2
years ago when he told me something like our application is a huge
application with lots of "lukewarm" methods. Anyway, is there a way to
find out *which* methods are not being compiled and *why*?
+PrintCompilation tells a lot of stuff about methods that are being
compiled, but nothing about methods that Hotspot considers *not* being
worth compiling ... ;-)
Also for the Client VM, same with inlining: We have a method (isTraceOn,
to determine whether tracing is enabled or not) that's being called
quite often. It's a small method, just 17 bytes:
499 xxx.xxx.xxx::isTraceOn (17 bytes)
@ 0 xxx.xxx.xxx::isTraceOnInProcess (4 bytes)
- @ 13 xxx.xxx.xxx::isTraceOnInternal (57 bytes) callee is
I know from the Sun Studio profile which are the callers of isTraceOn.
Those callers are getting compiled, e.g. the method traceSimple, but
don't even consider to inline isTraceOn (I would expect to find
something as "- @ xxx.xxx.xxx::isTraceOn" in the log with a note *why*
Hotspot decided to not inline this method -- but it doesn't appear
anywhere at all):
1330 xxx.xxx.xxx::traceSimple (91 bytes)
! - @ 29 xxx.xxx.xxx::prepareBuffer (196 bytes) callee is too
s @ 48 java.lang.StringBuffer::length (5 bytes)
s @ 82 java.lang.StringBuffer::toString (17 bytes)
- @ 13 java.lang.String::<init> (72 bytes) callee is too
What could be the reason that isTraceOn is not being inlined? BTW, I
also tried to increase MaxInlineSize to 64 which didn't help. (Server VM does
inline isTraceOn, but not the Client VM.)
I can of course also provide full log files and everything...
Thanks a lot,
More information about the hotspot-compiler-dev