<div dir="ltr">Luca,<div><br></div><div>I believe these timeout messages are related to MQ. I remember we would get these quite a lot when we were using JBoss 4 with MQ, if I remember correctly they were being shown because we were keeping open connections and sessions to some JMS queues.</div>
<div><br></div><div>I don't think this has any relation with your problem, but it might be worth checking anyway.</div><div><br></div><div>Jose</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Mar 7, 2014 at 5:09 AM, luke <span dir="ltr"><<a href="mailto:luke.bike@gmail.com" target="_blank">luke.bike@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Thanks Jose,<br></div>I think I haven't in my application <span style="font-family:arial,sans-serif;font-size:13px">"System.gc()" calls, but I'm working on a very large application so I'll check if some explicit call to GC has been introduced.<br>
<br></span></div><span style="font-family:arial,sans-serif;font-size:13px">I think the problem is how you wrote that "</span><span style="font-family:arial,sans-serif;font-size:13px">GC is running but is unable to free space in the heap". Is it possible that GC can't use cpu native threads and so can't run correctly?<br>
<br>few minutes before this outOfMemory in my jboss log I see: </span><br>
<i>SocketTimeoutException: </i><i>Caused by: java.net.SocketTimeoutException: Read timed out<br><br></i></div>Could be related to my problem?<br></div>thanks<br>luca<i><br></i><div><div><div><div><span style="font-family:arial,sans-serif;font-size:13px"><br>
</span></div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-03-06 20:53 GMT+01:00 Jose Otavio Carlomagno Filho <span dir="ltr"><<a href="mailto:jocf83@gmail.com" target="_blank">jocf83@gmail.com</a>></span>:<div>
<div class="h5"><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">If I'm not mistaken, "<i style="font-family:arial,sans-serif;font-size:13px">GC overhead limit exceeded" </i><span style="font-family:arial,sans-serif;font-size:13px">means the GC is running but is unable to free space in the heap. </span><div>
<span style="font-family:arial,sans-serif;font-size:13px"><br></span></div><div><span style="font-family:arial,sans-serif;font-size:13px">In many cases, this is caused by the application repeatedly calling "System.gc()", which normally triggers a full GC. You should check your application code and remove these calls if they exist. </span></div>
<div><span style="font-family:arial,sans-serif;font-size:13px"><br></span></div><div><span style="font-family:arial,sans-serif;font-size:13px">Additionally, you can add "</span><font face="arial, sans-serif">-XX:+DisableExplicitGC" to your startup parameters, this way the GC will not run if your application calls "System.gc()".</font></div>
<div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif">Jose</font></div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On Thu, Mar 6, 2014 at 12:13 PM, luke <span dir="ltr"><<a href="mailto:luke.bike@gmail.com" target="_blank">luke.bike@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr"><div><div><div>Hi, <br><br></div>I'm java.lang.OutOfMemoryError in a java application working on a JBossAS.<br>
</div>It's strange that OutOfMemory happen when application is not so stressed. </div>In my application log I found this exception<br>
<div><br><div><div><div style="margin-left:40px"><i>WARN [org.jboss.mq.Connection] Connection failure, use javax.jms.Connection.setExceptionListener() to handle this error and reconnect</i><br><i>org.jboss.mq.SpyJMSException: Exiting on IOE; - nested throwable: (java.net.SocketTimeoutException: Read timed out)</i><br>
<i> at org.jboss.mq.SpyJMSException.getAsJMSException(SpyJMSException.java:72)</i><br><i> at org.jboss.mq.Connection.asynchFailure(Connection.java:423)</i><br><i> at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:174)</i><br>
<i> at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:466)</i><br><i> at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:395)</i><br><i> at java.lang.Thread.run(Thread.java:619)</i><br>
<i>Caused by: java.net.SocketTimeoutException: Read timed out</i><br></div><br></div><div>and after some minutes:<br><div style="margin-left:40px"><i>2014-03-06 01:09:32,173 WARN [org.jboss.mq.Connection] Exception listener ended abnormally: <br>
java.lang.OutOfMemoryError: GC overhead limit exceeded<br> at java.lang.ThreadLocal.createInheritedMap(ThreadLocal.java:217)<br> at java.lang.Thread.init(Thread.java:358)<br> at java.lang.Thread.<init>(Thread.java:445)<br>
at org.jboss.mq.SpyMessageConsumer.setMessageListener(SpyMessageConsumer.java:237)<br> at it.oneans.iemx.qf.ejb.QueueService$QueueServiceExceptionListener.onException(QueueService.java:193)<br> at org.jboss.mq.Connection$ExceptionListenerRunnable.run(Connection.java:1356)<br>
at java.lang.Thread.run(Thread.java:619)<br></i></div></div><div><br></div><div>In my GC.log I can see <span lang="en"><span>a rapid increase in heap memory:</span></span><br><br>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10pt;font-family:"Courier New"">54967.049: [GC [PSYoungGen:
171815K->3032K(2024448K)] 1716963K->1583328K(8315904K), 0.0466930 secs]
[Times: user=0.20 sys=0.09, real=0.04 secs] </span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10pt;font-family:"Courier New"">54967.097: [<b><span style="color:red">Full GC (System)</span></b> [PSYoungGen:
3032K->0K(2024448K)] [</span><b><span style="font-family:"Courier New";color:red">ParOldGen: 1580296K->1501278K</span></b><span style="font-size:10pt;font-family:"Courier New"">(6291456K)] 1583328K->1501278K(8315904K)
[PSPermGen: 230071K->229632K(239744K)], 4.5397660 secs] [Times: user=18.01
sys=2.81, real=4.53 secs] </span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10pt;font-family:"Courier New"">...</span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10pt;font-family:"Courier New"">55546.522: [GC [PSYoungGen:
1883953K->129792K(1929216K)] 6315956K->4689948K(8220672K), 0.7681860
secs] [Times: user=8.76 sys=0.61, real=0.77 secs] </span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10pt;font-family:"Courier New"">55561.317: [GC [PSYoungGen:
1890304K->124543K(1928448K)] 6450460K->4814699K(8219904K), 1.8698640
secs] [Times: user=3.30 sys=0.26, real=1.87 secs] </span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10pt;font-family:"Courier New"">...</span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10pt;font-family:"Courier New"">55754.485: [GC [PSYoungGen:
1753886K->116213K(1881920K)] 7755780K->6232689K(8173376K), 0.5959420
secs] [Times: user=4.34 sys=0.30, real=0.60 secs] </span></p>
<p class="MsoNormal" style="margin-left:0.5in"><span style="font-size:10pt;font-family:"Courier New"">55755.083: [Full GC
[PSYoungGen: 116213K->0K(1881920K)] [<b><span style="color:red">ParOldGen:
6116476K->6031245K</span></b>(6291456K)] 6232689K->6031245K(8173376K)
[PSPermGen: 229665K->222795K(231488K)], 36.6400980 secs] [Times: user=160.17
sys=8.40, real=36.63 secs] </span></p>
<br>Could be OutOfmemory a side effect related to not enough free sockets on the server or something else?<br><br><br>thanks in advance for any suggestions<br>luca<br><div><div>P.S.:my gc triggers:<br> -Xms6g -Xmx6g -XX:MaxPermSize=512m<br>
-Dsun.rmi.dgc.client.gcInterval=2100000 -Dsun.rmi.dgc.server.gcInterval=2100000<br> -XX:+UseParallelOldGC -XX:+UseParallelGC<br> -XX:MaxHeapFreeRatio=70 -XX:MinHeapFreeRatio=40 -Xverify:none -XX:+BindGCTaskThreadsToCPUs <br>
-XX:NewSize=2g -XX:MaxNewSize=2g -XX:SurvivorRatio=4<br> -Djava.awt.headless=true<br></div></div></div></div></div></div>
<br></div></div>_______________________________________________<br>
hotspot-gc-use mailing list<br>
<a href="mailto:hotspot-gc-use@openjdk.java.net" target="_blank">hotspot-gc-use@openjdk.java.net</a><br>
<a href="http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use" target="_blank">http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use</a><br>
<br></blockquote></div><br></div>
</blockquote></div></div></div><br></div></div></div></div>
</blockquote></div><br></div>