Possibility of code cache unloading
Thomas.Rodriguez at Sun.COM
Mon Oct 12 12:45:19 PDT 2009
There are two main problems. First is how to identify nmethods which
aren't being used without penalizing execution too much and also
without introducing new races. Reclamation of nmethods is somewhat
tricky and reusing existing mechanisms would be good. Second is the
policy part. Reclamation of nmethods isn't prompt so you have to get
enough warning that the code cache is getting full to start
reclamation before you run out. It's basically a GC policy problem
though we can always simply delay compilation until some storage has
been reclaimed. A hiccup near the transition would still be better
than our current policy of simply stopping compilation.
For the first part, I'd always imagined we end up with some strategy
where we marked methods as speculatively not entrant and if they
weren't used within some time period we'd transition them fully to not
entrant and reclaim the nmethod storage. One problem with this is the
repatching of the nmethod back into a normal state. I guess if it's
done at a safepoint it wouldn't be a big deal.
On the other hand it occurred to me today that we could rely on the
same guarantees that the nmethod sweeper uses to do this without
requiring patching. There are only two ways to get into compiled
code. You either grab the nmethod entry point from the methodOop you
are invoking, which is how interpreter invokes and polymorphic
dispatch work, or you have a CompiledIC that jumps you there directly
which is the normal way compiled code works. If you break both of
these kinds of links then an nmethod can no longer be invoked. This
is essentially how making an nmethod not entrant works but since the
cleaning of inline caches is done at a safepoint we also patch the
entry point of the nmethod to dispatch back into the call resolution
code. This allows the cleaning of the CompiledICs to be done lazily
because any caller that still references the nmethod will still end up
back in the resolution code.
I think we could use the same strategy for identifying unused
nmethods. We simply break the reference from the methodOop to the
nmethod and start a cache cleaning but without patching the entry
point. If we ever invoke those nmethods again we'll end up going
through the call resolution code where we can detect that the nmethod
is really being used and simply restore the nmethod and continue using
it. If it's never referenced within some time period the nmethod can
simply be freed because we know that it's clean.
As far as policy goes we could simply mark every nmethod this way when
we get into trouble but I think we'd want a more GC like policy that
would begin identifying nmethods once the code cache started filling
up and reclaim them before we actually run out.
On Oct 12, 2009, at 9:53 AM, Eric Caspole wrote:
> We discovered some app servers do not have really well behaved class
> loader schemes, so that in a long running process with multiple re-
> deployments of the same web app, the old stuff never gets unloaded
> and the first thing to run out of space in this case was the code
> cache. In this event the compiler shuts itself off and does not turn
> itself on regardless of what happens in later GC/sweep cycles.
> We have been wondering what it would take to do some kind of code
> cache unloading so long running applications like this won't end up
> having the later redeployments running interpreter-only. Many users
> do not even know when this has happened and may or may not notice a
> gradual mysterious slowdown until they just restart the process.
> In various discussions ideas have popped up from marking some amount
> of existing nmethods non-entrant so they will get unloaded in the
> existing code cache, to more elaborate reallocation schemes for the
> whole code cache. If necessary, a short term slowdown before getting
> back to having all the hot methods recompiled seems better than
> restarting the process. Perhaps there could be some JMX notification
> for this situation.
> One idea that came out of our testing was the idea of a current
> working set of hot methods. We saw that if enough space could be
> regained in the code cache, the program's normal operation would
> require recompiles only of the current hot method set, which is
> hopefully a lot smaller than the whole code cache size. Then it
> would quickly resume normal operation, and only in the event of one
> or more web app redeployments as mentioned above would the code
> cache require another flush operation, hopefully days or weeks later.
> Lastly, it is probably desirable to have a fallback plan of giving
> up and shutting off the compiler if the flush cycles are happening
> too often, for example if the hot method working set size is too
> close to the whole code cache size, and the application performance
> won't be any worse than it gets today.
> We'd like to hear if anyone else has a strong opinion or great idea
> on this topic, and what corner case did we not think of. I remember
> a wise crack about this topic at the JVM Languages Summit a few
> weeks ago so it seems someone out there is thinking about it.
More information about the hotspot-compiler-dev