Project Jigsaw, Apache Tomcat and RMI related memory leaks
markt at apache.org
Thu Apr 21 11:58:05 UTC 2016
On 19/04/2016 16:25, Roger Riggs wrote:
> Hi Mark,
> It may take a bit of time to unwind and find a good solution.
Agreed. Your response has actually been a huge help. It gave me a few
more things to look at and I have been able break the problem down
I need to work on this some more before I come back with a much clearer
statement of the problem. I hope to be in a position to do that later
today or tomorrow.
I'll probably do that in a new thread.
Thanks for the pointers.
> On 4/18/16 11:58 AM, Mark Thomas wrote:
>> The Apache Tomcat community was asked by Rory O'Donnell for feedback on
>> JDK 9 + Project Jigsaw. Having provided that feedback we were directed
>> here so I have reproduced that feedback below.
>> I've started testing Tomcat trunk with JDK 9 + Project Jigsaw and it
>> looks like we are going to hit a bunch of problems related to Tomcat's
>> memory leak protection.
>> The short version is there are lots of JDK calls that can result in a
>> reference being retained to the current class loader. If that class
>> loader is the web application class loader it often ends up being pinned
>> in memory. This triggers a memory leak when the web application is
>> reloaded since the web application class loader is not eligible for GC.
>> Tomcat generally uses reflection to find these problems. It then does
>> one of two things:
>> - If the JRE provides an API the application developer should have used
>> to clean up the reference, Tomcat does this for them and then logs a
>> very loud error message telling the developer they need to fix their app.
>> - If there is nothing the developer could have done to avoid the
>> problem, Tomcat cleans it up. Usually this is via reflection again.
>> Reporting this second class of issues as JRE bugs has been on my TODO
>> list for a long time. It looks like Java 9 is going to bump this to the
>> top of the list.
>> The first problem we have hit is related to RMI. The memory leak is
>> triggered by a call to:
>> java.rmi.registry.Registry.bind() or
>> The problem occurs when an object of a class loaded by the web
>> application class loader is bound to the RMI registry.
>> There is no standard API (that I have found) that completely removes all
>> references. In particular
>> doesn't help.
> unbind just removes the mapping from name to an exported object;
> it does not un-export the object.
> Unless it was exported as permanent, the normal GC/distributed GC should
> clear it.
> I would think the normal application shutdown should both remove the
> and unexport it (perhaps forcibly).
> But you are working on the case where the application doesn't behave
> If you had a way to discover the exported remote objects, the normal
> forcible unexport mechanism
> should be sufficient. Handing out references to exports objects would
> need some kind of security
> It might be worth investigating an official shutdown mechanism though
> linking it to a classloader
> seems a bit special purpose.
> $.02, Roger
>> The code Tomcat uses to clean this up is at . Essentially, we need to
>> remove any reference to the web application's class loader from the RMI
>> With JDK 9 this fails with:
>> java.lang.reflect.InaccessibleObjectException: Unable to make member of
>> class sun.rmi.transport.Target accessible: module java.rmi does not
>> export sun.rmi.transport to unnamed module ...
>> I took a look at the JDK 9 API but I could not find a way to bypass
>> Possible solutions include:
>> 1. Some way that allows us to continue to use reflection as per the
>> current code.
>> 2. A new method somewhere in the RMI API that clears all references
>> associated with a given class loader from the cache.
>> 3. Modify Registry.unbind() so it removes all references.
>> 4. Something else...
>> I do have a concern with 3 on its own that, while that would allow
>> applications to clear their own references, it would mean Tomcat would
>> have no way to check for the problem.
>> Ideally, I'd like to see the API extended so a) applications are able to
>> clean up after themselves and b) Tomcat can check for leaked references
>> and generate error messages for the ones found.
>> Any and all suggestions gratefully received.
More information about the core-libs-dev