BTW: the load on the JVM is very light. <br><br><br clear="all">best regards,<br>hanzhu<br>
<br><br><div class="gmail_quote">On Mon, Dec 20, 2010 at 7:47 PM, Zhu Han <span dir="ltr"><<a href="mailto:schumi.han@gmail.com">schumi.han@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
The program running is apache cassandra 0.6.8[1]. Based on the report of cassandra community, most of them does not meet it even in their production environment. Some of the users are big web company.<br><br>I have no idea whether sun jdk has same problem. Maybe I should have a try.<br>
<br>Do you have any hints on how to get more diagnostic information?<br><br>[1] <a href="http://cassandra.apache.org/" target="_blank">http://cassandra.apache.org/</a><br><br clear="all">best regards,<br>hanzhu<div><div>
</div><div class="h5"><br>
<br><br><div class="gmail_quote">On Mon, Dec 20, 2010 at 6:04 PM, Andrew Haley <span dir="ltr"><<a href="mailto:aph@redhat.com" target="_blank">aph@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div><div></div><div>On 12/19/2010 02:59 AM, Zhu Han wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Hi,<br>
<br>
I might observe C-heap leak of IcedTea6 1.9.2 on Ubuntu 10.04. This bug was<br>
reported against cassandra[1]. But I'm afraid it is a general openJDK<br>
problem. Here is the symptoms. [1] may has more information on it.<br>
<br>
For a infrequently accessed openJDK node, the memory usage is climbed up<br>
every day, and I'm pretty sure it is not caused by leak on Java object heap.<br>
This is the GC settings:<br>
<br>
" /usr/bin/java -ea -Xms1G -Xmx1G -XX:+UseParNewGC -XX:+UseConcMarkSweepGC<br>
-XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1<br>
-XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly<br>
-XX:+HeapDumpOnOutOfMemoryError ..."<br>
<br>
So the size of JVM heap is constant. Use pmap against the JVM process for<br>
the last 3 days, I observed a region expanded every day. It can be expanded<br>
to multiple gigabytes finally[1].<br>
<br>
on 12/19: 00000000010c3000 178548K rw--- [ anon ]<br>
on 12/18: 00000000010c3000 110320K rw--- [ anon ]<br>
on 12/17: 00000000010c3000 39256K rw--- [ anon ]<br>
<br>
I notice the region is on lower address of process address space. So I<br>
suspect it is the c-heap of JVM. The map of object heap is as below, which<br>
is constant in the last few days.<br>
<br>
00002b58433c0000 1069824K rw--- [ anon ]<br>
<br>
Both the heap and non-heap memory usage of java code is quite normal:<br>
<br>
12/19/2010 10:56:13 +0800 org.archive.jmx.Client HeapMemoryUsage:<br>
committed: 1065025536<br>
init: 1073741824<br>
max: 1065025536<br>
used: 160743000<br>
<br>
12/19/2010 10:56:23 +0800 org.archive.jmx.Client NonHeapMemoryUsage:<br>
committed: 24313856<br>
init: 24313856<br>
max: 224395264<br>
used: 20477424<br>
<br>
<br>
There are hundreds of alive threads in the JVM.<br>
<br>
12/19/2010 10:52:20 +0800 org.archive.jmx.Client PeakThreadCount: 244<br>
<br>
<br>
Originally, I suspect it is bug 6824570[2]. But After I upgraded JDK from<br>
IceTea 1.8.2 to IceTea 1.9.2, the problem is still not fixed. [3] said it is<br>
fixed in IceTea 1.9.0.<br>
<br>
And there is another JVM instances running on the same node with ParNewGC<br>
enabled, I did not observe the memory leak. Although the peak thread count<br>
is quite low of that instance, which is 43.<br>
<br>
Do you have any hints to diagnostic this problem? I'd like to help.<br>
</blockquote>
<br></div></div>
This may, of course, be a leak in JNI code. What Java program was running?<br><font color="#888888">
<br>
Andrew.<br>
</font></blockquote></div><br>
</div></div></blockquote></div><br>