From tony.printezis at oracle.com Wed Jun 1 04:40:21 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 01 Jun 2011 07:40:21 -0400 Subject: CRR: 7045662: G1: OopsInHeapRegionClosure::set_region() should not be virtual (XS) Message-ID: <4DE62525.7080006@oracle.com> Hi, (first of a series of code review requests for changes I've been recently working on) Could I please have a couple of code reviews for this simple change? http://cr.openjdk.java.net/~tonyp/7045662/webrev.0/ The main change is to make the set_region() method non-virtual and remove two closures that were the reason for that method being virtual but that are not used any more. This change shows a very small (~1%) but measurable improvement in GC times on some platforms. I also piggy-backed a couple of extra minor fixes on this change: removal of a third unused closure and fixing two small typos in the copyright text on two files (I have no idea how that C become lowercase!). Tony From tony.printezis at oracle.com Wed Jun 1 04:51:45 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 01 Jun 2011 07:51:45 -0400 Subject: CRR: 7042285 and 7045330 : fixes and improvements to the HeapRegionSeq class (S/M) Message-ID: <4DE627D1.4060007@oracle.com> Hi, Could I please have a couple of code reviews for this change: http://cr.openjdk.java.net/~tonyp/7045330.7042285/webrev.0/ The associated CRs are: 7042285: G1: native memory leak during humongous object allocation 7045330: G1: Simplify/fix the HeapRegionSeq class The first is the fix for the memory leak when the heap is expanded / retracted aggressively that I opened for code review some time ago. We are currently not reclaiming HeapRegion instances when the heap is retracted, but we are creating new ones when the heap is expanded. But, in all honesty, the HeapRegionSeq class is a bit of a mess so I decided to expand the fix, cleanup that class, and introduce a couple of small performance improvements too (all these are covered by the second CR). Here's a list of the improvements (copied from the CR): a) replace the _regions growable array with a standard C array (as we do not grow or shrink it so there's no point in using a growable array) b) avoid de-allocating / re-allocating HeapRegion instances when the heap shrinks / grows (fix for 7042285) c) introduce fast method to map address to HeapRegion via a "biased" array pointer d) embed the _hrs object in G1CollectedHeap, instead of pointing to it via an indirection e) assume that all the regions added to the HeapRegionSeq instance are in a contiguous address space f) replace int's with size_t's for region indexes (part of 6804436) g) remove unnecessary / unused methods h) rename a couple of fields to give them a more reasonable name (_alloc_search_start and _seq_bottom) i) fix iterate_from() to do the right thing and not always start from index 0 irrespective of the region passed to it j) add a verification method to check the HeapRegionSeq assumptions k) always call the wrappers for _hrs.iterate(), _hrs_length(), and _hrs.at() from G1CollectedHeap, not those methods directly. In terms of volume of changes, the HeapRegionSeq class has been rewritten heavily. The rest of the changes should be straightforward (there are changes in G1CollectedHeap that reflect the slightly different API that HeapRegionSeq now exposes). Performance-wise, I see a again modest (~2% or maybe a bit more) improvement in GC times on a couple of platforms. Tony From stefan.karlsson at oracle.com Wed Jun 1 05:01:14 2011 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 01 Jun 2011 14:01:14 +0200 Subject: CRR: 7045662: G1: OopsInHeapRegionClosure::set_region() should not be virtual (XS) In-Reply-To: <4DE62525.7080006@oracle.com> References: <4DE62525.7080006@oracle.com> Message-ID: <4DE62A0A.6020201@oracle.com> What about IntoCSOopClosure? It also declares a set_region method. But since it's not used anywhere, we should probably just remove it. Otherwise, this looks good. StefanK On 06/01/2011 01:40 PM, Tony Printezis wrote: > Hi, > > (first of a series of code review requests for changes I've been > recently working on) > > Could I please have a couple of code reviews for this simple change? > > http://cr.openjdk.java.net/~tonyp/7045662/webrev.0/ > > The main change is to make the set_region() method non-virtual and > remove two closures that were the reason for that method being virtual > but that are not used any more. This change shows a very small (~1%) > but measurable improvement in GC times on some platforms. > > I also piggy-backed a couple of extra minor fixes on this change: > removal of a third unused closure and fixing two small typos in the > copyright text on two files (I have no idea how that C become > lowercase!). > > Tony > From stefan.karlsson at oracle.com Wed Jun 1 05:01:43 2011 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 01 Jun 2011 14:01:43 +0200 Subject: CRR: 7045662: G1: OopsInHeapRegionClosure::set_region() should not be virtual (XS) In-Reply-To: <4DE62525.7080006@oracle.com> References: <4DE62525.7080006@oracle.com> Message-ID: <4DE62A27.8070208@oracle.com> What about IntoCSOopClosure? It also declares a set_region method. But since it's not used anywhere, we should probably just remove it. Otherwise, this looks good. StefanK On 06/01/2011 01:40 PM, Tony Printezis wrote: > Hi, > > (first of a series of code review requests for changes I've been > recently working on) > > Could I please have a couple of code reviews for this simple change? > > http://cr.openjdk.java.net/~tonyp/7045662/webrev.0/ > > The main change is to make the set_region() method non-virtual and > remove two closures that were the reason for that method being virtual > but that are not used any more. This change shows a very small (~1%) > but measurable improvement in GC times on some platforms. > > I also piggy-backed a couple of extra minor fixes on this change: > removal of a third unused closure and fixing two small typos in the > copyright text on two files (I have no idea how that C become > lowercase!). > > Tony > From tony.printezis at oracle.com Wed Jun 1 05:04:45 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 01 Jun 2011 08:04:45 -0400 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) Message-ID: <4DE62ADD.1040601@oracle.com> Hi, Could I please have a couple of code reviews for this simple change: http://cr.openjdk.java.net/~tonyp/6804436/webrev.0/ Some of the int -> size_t changes were done as part of the HeapRegionSeq cleanup (see the separate code review request for 7042285 and 7045330 that I just sent out). This change does the rest (specifically: the changes related to the HeapRegion class). Note: I know that the webrev index says that some files have more than one change applied to them. This is incorrect and a side-effect of generating the webrev from a workspace with multiple patches stacked up. The diffs are actually correct (I checked). Tony From tony.printezis at oracle.com Wed Jun 1 05:19:48 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 01 Jun 2011 08:19:48 -0400 Subject: CRR: 7046558 G1: concurrent marking optimizations (S) Message-ID: <4DE62E64.1040902@oracle.com> Hi, Could I please have a couple of code reviews for the following change: http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.all/ It includes a few optimizations in the concurrent marking code. The fact that webrev shows that almost 400 lines of code have been changed is actually a bit misleading given that the changes are actually relatively simple. Here's a breakdown to make your life a bit easier: Change 1: Specialize the G1CMOopClosure to avoid the virtual call to do_oop() every time we visit an oop during marking (the closure used to be called CMOopClosure, I prefixed its name with G1 since it's been moved out of the concurrentMark.cpp file). Note: I didn't change much code here, just moved things around. http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptSpecial/ Change 2 (builds on Change 1): Move methods CM::deal_with_reference() and CM::push() to the .inline.hpp file, without changing them, to make sure they get inlined, as they are called from the fast path. You can either check line by line that they are unchanged. Or you can take my word for it. :-) http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptInline/ Change 3 (builds on Change 2): A few minor performance improvements here and there (plus a couple of cosmetic changes to introduce some missing curly brackets). http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptMisc/ I kept track of the separate patches to be able to evaluate them separately so I thought I'd present the changes separately to make the code review process a bit easier. I will of course push them as a single changeset. I was not quite sure how to evaluate the performance impact of the changes given that marking runs concurrently and not always in a deterministic fashion (from a performance point-of-view!!!). So I did the evaluation using collect and analyze. When looking at time-based profiles, I see that the above changes decrease the time we spend in the marking code (i.e., how long we spend in the CMConcurrentMarkingTask::work(int i) method) by between 7%-10%. The improvement is similar when looking at cycle counts. And I also see a nice decrease in L2 cache misses. Each separate Change does improve on the previous one, but the Change with the biggest impact was 1. Tony From stefan.karlsson at oracle.com Wed Jun 1 06:03:21 2011 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 01 Jun 2011 15:03:21 +0200 Subject: CRR: 7042285 and 7045330 : fixes and improvements to the HeapRegionSeq class (S/M) In-Reply-To: <4DE627D1.4060007@oracle.com> References: <4DE627D1.4060007@oracle.com> Message-ID: <4DE63899.7090504@oracle.com> Looks good. Some nits: heapRegion.hpp - Shouldn't G1_INVALID_HRS_INDEX be defined in heapRegionSeq.hpp? heapRegionSeq.hpp - Could you move the class comment to above the class declaration? - type on line 147: early of doHeapRegion -> early if doHeapRegion StefanK On 06/01/2011 01:51 PM, Tony Printezis wrote: > Hi, > > Could I please have a couple of code reviews for this change: > > http://cr.openjdk.java.net/~tonyp/7045330.7042285/webrev.0/ > > The associated CRs are: > > 7042285: G1: native memory leak during humongous object allocation > 7045330: G1: Simplify/fix the HeapRegionSeq class > > The first is the fix for the memory leak when the heap is expanded / > retracted aggressively that I opened for code review some time ago. We > are currently not reclaiming HeapRegion instances when the heap is > retracted, but we are creating new ones when the heap is expanded. > But, in all honesty, the HeapRegionSeq class is a bit of a mess so I > decided to expand the fix, cleanup that class, and introduce a couple > of small performance improvements too (all these are covered by the > second CR). Here's a list of the improvements (copied from the CR): > > a) replace the _regions growable array with a standard C array (as we > do not grow or shrink it so there's no point in using a growable array) > b) avoid de-allocating / re-allocating HeapRegion instances when the > heap shrinks / grows (fix for 7042285) > c) introduce fast method to map address to HeapRegion via a "biased" > array pointer > d) embed the _hrs object in G1CollectedHeap, instead of pointing to it > via an indirection > e) assume that all the regions added to the HeapRegionSeq instance are > in a contiguous address space > f) replace int's with size_t's for region indexes (part of 6804436) > g) remove unnecessary / unused methods > h) rename a couple of fields to give them a more reasonable name > (_alloc_search_start and _seq_bottom) > i) fix iterate_from() to do the right thing and not always start from > index 0 irrespective of the region passed to it > j) add a verification method to check the HeapRegionSeq assumptions > k) always call the wrappers for _hrs.iterate(), _hrs_length(), and > _hrs.at() from G1CollectedHeap, not those methods directly. > > In terms of volume of changes, the HeapRegionSeq class has been > rewritten heavily. The rest of the changes should be straightforward > (there are changes in G1CollectedHeap that reflect the slightly > different API that HeapRegionSeq now exposes). > > Performance-wise, I see a again modest (~2% or maybe a bit more) > improvement in GC times on a couple of platforms. > > Tony > From stefan.karlsson at oracle.com Wed Jun 1 06:11:19 2011 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 01 Jun 2011 15:11:19 +0200 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DE62ADD.1040601@oracle.com> References: <4DE62ADD.1040601@oracle.com> Message-ID: <4DE63A77.2030104@oracle.com> Looks good. StefanK On 06/01/2011 02:04 PM, Tony Printezis wrote: > Hi, > > Could I please have a couple of code reviews for this simple change: > > http://cr.openjdk.java.net/~tonyp/6804436/webrev.0/ > > Some of the int -> size_t changes were done as part of the > HeapRegionSeq cleanup (see the separate code review request for > 7042285 and 7045330 that I just sent out). This change does the rest > (specifically: the changes related to the HeapRegion class). > > Note: I know that the webrev index says that some files have more than > one change applied to them. This is incorrect and a side-effect of > generating the webrev from a workspace with multiple patches stacked > up. The diffs are actually correct (I checked). > > Tony > From stefan.karlsson at oracle.com Wed Jun 1 06:28:46 2011 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 01 Jun 2011 15:28:46 +0200 Subject: CRR: 7046558 G1: concurrent marking optimizations (S) In-Reply-To: <4DE62E64.1040902@oracle.com> References: <4DE62E64.1040902@oracle.com> Message-ID: <4DE63E8E.60305@oracle.com> Note that you have introduced a circular dependency between the files: concurrentMark.inline.hpp and g1OopClosures.inline.hpp. Otherwise, this looks good. StefanK On 06/01/2011 02:19 PM, Tony Printezis wrote: > Hi, > > Could I please have a couple of code reviews for the following change: > > http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.all/ > > It includes a few optimizations in the concurrent marking code. The > fact that webrev shows that almost 400 lines of code have been changed > is actually a bit misleading given that the changes are actually > relatively simple. Here's a breakdown to make your life a bit easier: > > Change 1: Specialize the G1CMOopClosure to avoid the virtual call to > do_oop() every time we visit an oop during marking (the closure used > to be called CMOopClosure, I prefixed its name with G1 since it's been > moved out of the concurrentMark.cpp file). Note: I didn't change much > code here, just moved things around. > > http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptSpecial/ > > Change 2 (builds on Change 1): Move methods CM::deal_with_reference() > and CM::push() to the .inline.hpp file, without changing them, to make > sure they get inlined, as they are called from the fast path. You can > either check line by line that they are unchanged. Or you can take my > word for it. :-) > > http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptInline/ > > Change 3 (builds on Change 2): A few minor performance improvements > here and there (plus a couple of cosmetic changes to introduce some > missing curly brackets). > > http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptMisc/ > > I kept track of the separate patches to be able to evaluate them > separately so I thought I'd present the changes separately to make the > code review process a bit easier. I will of course push them as a > single changeset. > > I was not quite sure how to evaluate the performance impact of the > changes given that marking runs concurrently and not always in a > deterministic fashion (from a performance point-of-view!!!). So I did > the evaluation using collect and analyze. When looking at time-based > profiles, I see that the above changes decrease the time we spend in > the marking code (i.e., how long we spend in the > CMConcurrentMarkingTask::work(int i) method) by between 7%-10%. The > improvement is similar when looking at cycle counts. And I also see a > nice decrease in L2 cache misses. Each separate Change does improve on > the previous one, but the Change with the biggest impact was 1. > > Tony > From jon.masamitsu at oracle.com Wed Jun 1 10:24:04 2011 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 01 Jun 2011 10:24:04 -0700 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DDFE61F.2090307@fastmail.fm> References: <4DDFE61F.2090307@fastmail.fm> Message-ID: <4DE675B4.3060009@oracle.com> Raman, The gc.log looks like it has the young collections filtered out. Is that right? If so, please upload the complete log. Jon On 05/27/11 10:57, Raman Gupta wrote: > I am getting a heap OOM for no apparent reason. Normally permanent generation size or a large finalization queue would be culprits for this sort of OOM, but AFAICT GC logs and heap dump show that neither is the case here. There is also no native code being executed. > > Linux RHEL 5.6 > > java version "1.6.0_24" > Java(TM) SE Runtime Environment (build 1.6.0_24-b07) > Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode) > > (I have also tried it with 1.6.0_25 64-bit and 1.6.0_25 32-bit with the same OOM result for both, but the logs below are from 1.6.0_24 64-bit). > > Startup parameters: > > -server > -Xms256m > -Xmx256m > -XX:+HeapDumpOnOutOfMemoryError > -XX:MaxPermSize=64m > -verbose:gc > -XX:-UseGCOverheadLimit > -XX:+DisableExplicitGC > -XX:+UseParallelGC > -XX:+UseFastAccessorMethods > -XX:AdaptiveSizePolicyOutputInterval=1 > -XX:+PrintGCDateStamps > -XX:+PrintGCTimeStamps > -XX:+PrintGCDetails > -XX:+PrintGCApplicationConcurrentTime > -XX:+PrintGCApplicationStoppedTime > > The complete GC log is available here: > > http://dl.dropbox.com/u/3430279/gc.log > > but here is a short snippet from that log before and after the OOM: > > 2011-05-27T11:35:18.214-0400: 147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs] [Times: user=0.29 sys=0.03, real=0.81 secs] > 2011-05-27T11:35:19.022-0400: 148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs] [Times: user=0.32 sys=0.02, real=0.37 secs] > Total time for which application threads were stopped: 1.2760680 seconds > java.lang.OutOfMemoryError: Java heap space > Dumping heap to java_pid18706.hprof ... > Application time: 0.9442610 seconds > Total time for which application threads were stopped: 2.4584870 seconds > Heap dump file created [83874513 bytes in 3.330 secs] > 2011-05-27T11:35:30.942-0400: 160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00, real=0.28 secs] > > Note that: > > 1) I can reproduce this easily. > > 2) This seems to happen only once shortly after load is applied to the system shortly after startup -- after that everything seems fine (though I haven't yet verified this with a longer test). > > 3) There is *always* a Full GC showing a young generation 0K->0K collection before this happens. > > 4) There seems to be plenty of space in the tenured generation as well as in permanent at OOM time. > > 5) The heap dump in pid18706.hprof shows only 70 MB of live objects in the heap. > > 6) The heap dump in pid18706.hprof shows no objects in the finalization queue. > > 7) I note the OutOfMemoryError in the log does not have any stack trace associated with it as is normally present. > > 8) When running the JVM with hprof sites profiling turned on (-agentlib:hprof=heap=sites) no OOM occurs (though again I did not try a very long test). > > I'm not sure what else to try or where else to look. Any suggestions? > > Cheers, > Raman Gupta From kirk at kodewerk.com Wed Jun 1 10:35:21 2011 From: kirk at kodewerk.com (Charles K Pepperdine) Date: Wed, 1 Jun 2011 19:35:21 +0200 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DE675B4.3060009@oracle.com> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> Message-ID: <6B0D53EF-B469-48C7-AC39-28EEE1015903@kodewerk.com> +1, can't see anything in these logs that indicates you're heading to a OOME due to Java heap. But, incomplete log... Regards, Kirk On Jun 1, 2011, at 7:24 PM, Jon Masamitsu wrote: > Raman, > > The gc.log looks like it has the young collections > filtered out. Is that right? If so, please upload > the complete log. > > Jon > > On 05/27/11 10:57, Raman Gupta wrote: >> I am getting a heap OOM for no apparent reason. Normally permanent generation size or a large finalization queue would be culprits for this sort of OOM, but AFAICT GC logs and heap dump show that neither is the case here. There is also no native code being executed. >> >> Linux RHEL 5.6 >> >> java version "1.6.0_24" >> Java(TM) SE Runtime Environment (build 1.6.0_24-b07) >> Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode) >> >> (I have also tried it with 1.6.0_25 64-bit and 1.6.0_25 32-bit with the same OOM result for both, but the logs below are from 1.6.0_24 64-bit). >> >> Startup parameters: >> >> -server >> -Xms256m >> -Xmx256m >> -XX:+HeapDumpOnOutOfMemoryError >> -XX:MaxPermSize=64m >> -verbose:gc >> -XX:-UseGCOverheadLimit >> -XX:+DisableExplicitGC >> -XX:+UseParallelGC >> -XX:+UseFastAccessorMethods >> -XX:AdaptiveSizePolicyOutputInterval=1 >> -XX:+PrintGCDateStamps >> -XX:+PrintGCTimeStamps >> -XX:+PrintGCDetails >> -XX:+PrintGCApplicationConcurrentTime >> -XX:+PrintGCApplicationStoppedTime >> >> The complete GC log is available here: >> >> http://dl.dropbox.com/u/3430279/gc.log >> >> but here is a short snippet from that log before and after the OOM: >> >> 2011-05-27T11:35:18.214-0400: 147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs] [Times: user=0.29 sys=0.03, real=0.81 secs] >> 2011-05-27T11:35:19.022-0400: 148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs] [Times: user=0.32 sys=0.02, real=0.37 secs] >> Total time for which application threads were stopped: 1.2760680 seconds >> java.lang.OutOfMemoryError: Java heap space >> Dumping heap to java_pid18706.hprof ... >> Application time: 0.9442610 seconds >> Total time for which application threads were stopped: 2.4584870 seconds >> Heap dump file created [83874513 bytes in 3.330 secs] >> 2011-05-27T11:35:30.942-0400: 160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00, real=0.28 secs] >> >> Note that: >> >> 1) I can reproduce this easily. >> >> 2) This seems to happen only once shortly after load is applied to the system shortly after startup -- after that everything seems fine (though I haven't yet verified this with a longer test). >> >> 3) There is *always* a Full GC showing a young generation 0K->0K collection before this happens. >> >> 4) There seems to be plenty of space in the tenured generation as well as in permanent at OOM time. >> >> 5) The heap dump in pid18706.hprof shows only 70 MB of live objects in the heap. >> >> 6) The heap dump in pid18706.hprof shows no objects in the finalization queue. >> >> 7) I note the OutOfMemoryError in the log does not have any stack trace associated with it as is normally present. >> >> 8) When running the JVM with hprof sites profiling turned on (-agentlib:hprof=heap=sites) no OOM occurs (though again I did not try a very long test). >> >> I'm not sure what else to try or where else to look. Any suggestions? >> >> Cheers, >> Raman Gupta From tony.printezis at oracle.com Wed Jun 1 15:01:22 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 01 Jun 2011 18:01:22 -0400 Subject: CRR: 7045662: G1: OopsInHeapRegionClosure::set_region() should not be virtual (XS) In-Reply-To: <4DE62A0A.6020201@oracle.com> References: <4DE62525.7080006@oracle.com> <4DE62A0A.6020201@oracle.com> Message-ID: <4DE6B6B2.8010905@oracle.com> Stefan and John, First, thanks for taking a look at this so promptly! Yes, good catch. I had missed IntoCSOopClosure. It's gone too! Along with VerifyRSCleanCardOopClosure which is not used either (this is yet another problem with closures: you remove the code but it's very easy to leave the closures behind....). Anyway, here's the updated webrev: http://cr.openjdk.java.net/~tonyp/7045662/webrev.1/ And I'm all set with this one! Tony On 6/1/2011 8:01 AM, Stefan Karlsson wrote: > What about IntoCSOopClosure? It also declares a set_region method. But > since it's not used anywhere, we should probably just remove it. > > Otherwise, this looks good. > > StefanK > > On 06/01/2011 01:40 PM, Tony Printezis wrote: >> Hi, >> >> (first of a series of code review requests for changes I've been >> recently working on) >> >> Could I please have a couple of code reviews for this simple change? >> >> http://cr.openjdk.java.net/~tonyp/7045662/webrev.0/ >> >> The main change is to make the set_region() method non-virtual and >> remove two closures that were the reason for that method being >> virtual but that are not used any more. This change shows a very >> small (~1%) but measurable improvement in GC times on some platforms. >> >> I also piggy-backed a couple of extra minor fixes on this change: >> removal of a third unused closure and fixing two small typos in the >> copyright text on two files (I have no idea how that C become >> lowercase!). >> >> Tony >> > From tony.printezis at oracle.com Wed Jun 1 15:15:04 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Thu, 02 Jun 2011 00:15:04 +0200 Subject: CRR: 7042285 and 7045330 : fixes and improvements to the HeapRegionSeq class (S/M) In-Reply-To: <4DE63899.7090504@oracle.com> References: <4DE627D1.4060007@oracle.com> <4DE63899.7090504@oracle.com> Message-ID: <4DE6B9E8.40409@oracle.com> Stefan, Thanks for the code review. Please see answers inline: On 06/01/2011 03:03 PM, Stefan Karlsson wrote: > Looks good. > > Some nits: > > heapRegion.hpp > - Shouldn't G1_INVALID_HRS_INDEX be defined in heapRegionSeq.hpp? You probably saw in the follow-up change that I use G1_INVALID_HRS_INDEX to initialize the _hrs_index field of the HeapRegion class. So I thought it was pointless to put it in one file and then move it to another one. BTW, I'm not 100% happy with the G1_INVALID_HRS_INDEX name. Would G1_HRS_NULL_INDEX, or something similar, be better? > heapRegionSeq.hpp > - Could you move the class comment to above the class declaration? Done. > - type on line 147: early of doHeapRegion -> early if doHeapRegion Good catch. Fixed. Updated webrev here: http://cr.openjdk.java.net/~tonyp/7045330.7042285/webrev.1/ Tony > StefanK > > On 06/01/2011 01:51 PM, Tony Printezis wrote: >> Hi, >> >> Could I please have a couple of code reviews for this change: >> >> http://cr.openjdk.java.net/~tonyp/7045330.7042285/webrev.0/ >> >> The associated CRs are: >> >> 7042285: G1: native memory leak during humongous object allocation >> 7045330: G1: Simplify/fix the HeapRegionSeq class >> >> The first is the fix for the memory leak when the heap is expanded / >> retracted aggressively that I opened for code review some time ago. >> We are currently not reclaiming HeapRegion instances when the heap is >> retracted, but we are creating new ones when the heap is expanded. >> But, in all honesty, the HeapRegionSeq class is a bit of a mess so I >> decided to expand the fix, cleanup that class, and introduce a couple >> of small performance improvements too (all these are covered by the >> second CR). Here's a list of the improvements (copied from the CR): >> >> a) replace the _regions growable array with a standard C array (as we >> do not grow or shrink it so there's no point in using a growable array) >> b) avoid de-allocating / re-allocating HeapRegion instances when the >> heap shrinks / grows (fix for 7042285) >> c) introduce fast method to map address to HeapRegion via a "biased" >> array pointer >> d) embed the _hrs object in G1CollectedHeap, instead of pointing to >> it via an indirection >> e) assume that all the regions added to the HeapRegionSeq instance >> are in a contiguous address space >> f) replace int's with size_t's for region indexes (part of 6804436) >> g) remove unnecessary / unused methods >> h) rename a couple of fields to give them a more reasonable name >> (_alloc_search_start and _seq_bottom) >> i) fix iterate_from() to do the right thing and not always start from >> index 0 irrespective of the region passed to it >> j) add a verification method to check the HeapRegionSeq assumptions >> k) always call the wrappers for _hrs.iterate(), _hrs_length(), and >> _hrs.at() from G1CollectedHeap, not those methods directly. >> >> In terms of volume of changes, the HeapRegionSeq class has been >> rewritten heavily. The rest of the changes should be straightforward >> (there are changes in G1CollectedHeap that reflect the slightly >> different API that HeapRegionSeq now exposes). >> >> Performance-wise, I see a again modest (~2% or maybe a bit more) >> improvement in GC times on a couple of platforms. >> >> Tony >> > From tony.printezis at oracle.com Wed Jun 1 15:18:25 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Thu, 02 Jun 2011 00:18:25 +0200 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DE63A77.2030104@oracle.com> References: <4DE62ADD.1040601@oracle.com> <4DE63A77.2030104@oracle.com> Message-ID: <4DE6BAB1.7090405@oracle.com> Stefan and John, Thanks! All set with this one too. Tony On 06/01/2011 03:11 PM, Stefan Karlsson wrote: > Looks good. > > StefanK > > On 06/01/2011 02:04 PM, Tony Printezis wrote: >> Hi, >> >> Could I please have a couple of code reviews for this simple change: >> >> http://cr.openjdk.java.net/~tonyp/6804436/webrev.0/ >> >> Some of the int -> size_t changes were done as part of the >> HeapRegionSeq cleanup (see the separate code review request for >> 7042285 and 7045330 that I just sent out). This change does the rest >> (specifically: the changes related to the HeapRegion class). >> >> Note: I know that the webrev index says that some files have more >> than one change applied to them. This is incorrect and a side-effect >> of generating the webrev from a workspace with multiple patches >> stacked up. The diffs are actually correct (I checked). >> >> Tony >> > From rocketraman at fastmail.fm Wed Jun 1 19:15:08 2011 From: rocketraman at fastmail.fm (Raman Gupta) Date: Wed, 01 Jun 2011 22:15:08 -0400 Subject: Heap OOM for no apparent reason? In-Reply-To: <6B0D53EF-B469-48C7-AC39-28EEE1015903@kodewerk.com> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <6B0D53EF-B469-48C7-AC39-28EEE1015903@kodewerk.com> Message-ID: <4DE6F22C.40601@fastmail.fm> My bad -- I filtered out the young GCs with an errant grep command... this log should be ok: http://dl.dropbox.com/u/3430279/gc.log This is from a different test run than the one before, but demonstrates the same "problem". Cheers, Raman On 06/01/2011 01:35 PM, Charles K Pepperdine wrote: > +1, can't see anything in these logs that indicates you're heading to a OOME due to Java heap. But, incomplete log... > > Regards, > Kirk > > On Jun 1, 2011, at 7:24 PM, Jon Masamitsu wrote: > >> Raman, >> >> The gc.log looks like it has the young collections >> filtered out. Is that right? If so, please upload >> the complete log. >> >> Jon >> >> On 05/27/11 10:57, Raman Gupta wrote: >>> I am getting a heap OOM for no apparent reason. Normally permanent generation size or a large finalization queue would be culprits for this sort of OOM, but AFAICT GC logs and heap dump show that neither is the case here. There is also no native code being executed. >>> >>> Linux RHEL 5.6 >>> >>> java version "1.6.0_24" >>> Java(TM) SE Runtime Environment (build 1.6.0_24-b07) >>> Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode) >>> >>> (I have also tried it with 1.6.0_25 64-bit and 1.6.0_25 32-bit with the same OOM result for both, but the logs below are from 1.6.0_24 64-bit). >>> >>> Startup parameters: >>> >>> -server >>> -Xms256m >>> -Xmx256m >>> -XX:+HeapDumpOnOutOfMemoryError >>> -XX:MaxPermSize=64m >>> -verbose:gc >>> -XX:-UseGCOverheadLimit >>> -XX:+DisableExplicitGC >>> -XX:+UseParallelGC >>> -XX:+UseFastAccessorMethods >>> -XX:AdaptiveSizePolicyOutputInterval=1 >>> -XX:+PrintGCDateStamps >>> -XX:+PrintGCTimeStamps >>> -XX:+PrintGCDetails >>> -XX:+PrintGCApplicationConcurrentTime >>> -XX:+PrintGCApplicationStoppedTime >>> >>> The complete GC log is available here: >>> >>> http://dl.dropbox.com/u/3430279/gc.log >>> >>> but here is a short snippet from that log before and after the OOM: >>> >>> 2011-05-27T11:35:18.214-0400: 147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs] [Times: user=0.29 sys=0.03, real=0.81 secs] >>> 2011-05-27T11:35:19.022-0400: 148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs] [Times: user=0.32 sys=0.02, real=0.37 secs] >>> Total time for which application threads were stopped: 1.2760680 seconds >>> java.lang.OutOfMemoryError: Java heap space >>> Dumping heap to java_pid18706.hprof ... >>> Application time: 0.9442610 seconds >>> Total time for which application threads were stopped: 2.4584870 seconds >>> Heap dump file created [83874513 bytes in 3.330 secs] >>> 2011-05-27T11:35:30.942-0400: 160.552: [Full GC [PSYoungGen: 2624K->0K(44800K)] [PSOldGen: 86916K->76124K(174784K)] 89540K->76124K(219584K) [PSPermGen: 26902K->26902K(58880K)], 0.2801510 secs] [Times: user=0.28 sys=0.00, real=0.28 secs] >>> >>> Note that: >>> >>> 1) I can reproduce this easily. >>> >>> 2) This seems to happen only once shortly after load is applied to the system shortly after startup -- after that everything seems fine (though I haven't yet verified this with a longer test). >>> >>> 3) There is *always* a Full GC showing a young generation 0K->0K collection before this happens. >>> >>> 4) There seems to be plenty of space in the tenured generation as well as in permanent at OOM time. >>> >>> 5) The heap dump in pid18706.hprof shows only 70 MB of live objects in the heap. >>> >>> 6) The heap dump in pid18706.hprof shows no objects in the finalization queue. >>> >>> 7) I note the OutOfMemoryError in the log does not have any stack trace associated with it as is normally present. >>> >>> 8) When running the JVM with hprof sites profiling turned on (-agentlib:hprof=heap=sites) no OOM occurs (though again I did not try a very long test). >>> >>> I'm not sure what else to try or where else to look. Any suggestions? >>> >>> Cheers, >>> Raman Gupta > From tony.printezis at oracle.com Thu Jun 2 01:25:02 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Thu, 02 Jun 2011 04:25:02 -0400 Subject: CRR: 7046558 G1: concurrent marking optimizations (S) In-Reply-To: <4DE63E8E.60305@oracle.com> References: <4DE62E64.1040902@oracle.com> <4DE63E8E.60305@oracle.com> Message-ID: <4DE748DE.5060504@oracle.com> Stefan, Thanks for the code review! Good catch about the circular dependency. I re-organized things a bit: I moved scan_object() from the .inline.hpp to the .cpp file (it's only called from there so it should be inlined anyway) so now I don't need to include g1OopClosures.inline.hpp in concurrentMark.inline.hpp. Here's the updated webrev: All included: http://cr.openjdk.java.net/~tonyp/7046558/webrev.1/webrev.all/ Change 1: http://cr.openjdk.java.net/~tonyp/7046558/webrev.1/webrev.G1CMOptSpecial/ Change 2: http://cr.openjdk.java.net/~tonyp/7046558/webrev.1/webrev.G1CMOptInline/ Change 3: http://cr.openjdk.java.net/~tonyp/7046558/webrev.1/webrev.G1CMOptMisc/ Tony On 6/1/2011 9:28 AM, Stefan Karlsson wrote: > Note that you have introduced a circular dependency between the files: > concurrentMark.inline.hpp and g1OopClosures.inline.hpp. > > Otherwise, this looks good. > > StefanK > > On 06/01/2011 02:19 PM, Tony Printezis wrote: >> Hi, >> >> Could I please have a couple of code reviews for the following change: >> >> http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.all/ >> >> It includes a few optimizations in the concurrent marking code. The >> fact that webrev shows that almost 400 lines of code have been >> changed is actually a bit misleading given that the changes are >> actually relatively simple. Here's a breakdown to make your life a >> bit easier: >> >> Change 1: Specialize the G1CMOopClosure to avoid the virtual call to >> do_oop() every time we visit an oop during marking (the closure used >> to be called CMOopClosure, I prefixed its name with G1 since it's >> been moved out of the concurrentMark.cpp file). Note: I didn't change >> much code here, just moved things around. >> >> http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptSpecial/ >> >> >> Change 2 (builds on Change 1): Move methods CM::deal_with_reference() >> and CM::push() to the .inline.hpp file, without changing them, to >> make sure they get inlined, as they are called from the fast path. >> You can either check line by line that they are unchanged. Or you can >> take my word for it. :-) >> >> http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptInline/ >> >> Change 3 (builds on Change 2): A few minor performance improvements >> here and there (plus a couple of cosmetic changes to introduce some >> missing curly brackets). >> >> http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptMisc/ >> >> I kept track of the separate patches to be able to evaluate them >> separately so I thought I'd present the changes separately to make >> the code review process a bit easier. I will of course push them as a >> single changeset. >> >> I was not quite sure how to evaluate the performance impact of the >> changes given that marking runs concurrently and not always in a >> deterministic fashion (from a performance point-of-view!!!). So I did >> the evaluation using collect and analyze. When looking at time-based >> profiles, I see that the above changes decrease the time we spend in >> the marking code (i.e., how long we spend in the >> CMConcurrentMarkingTask::work(int i) method) by between 7%-10%. The >> improvement is similar when looking at cycle counts. And I also see a >> nice decrease in L2 cache misses. Each separate Change does improve >> on the previous one, but the Change with the biggest impact was 1. >> >> Tony >> > From kirk at kodewerk.com Thu Jun 2 06:20:11 2011 From: kirk at kodewerk.com (Charles K Pepperdine) Date: Thu, 2 Jun 2011 15:20:11 +0200 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DE675B4.3060009@oracle.com> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> Message-ID: <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> 2011-05-27T11:35:18.214-0400: 147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs] [Times: user=0.29 sys=0.03, real=0.81 secs] UseAdaptiveSizePolicy actions to meet *** throughput goal *** GC overhead (%) Young generation: 2.65 (attempted to grow) Tenured generation: 0.54 (attempted to grow) Tenuring threshold: (attempted to decrease to balance GC costs) = 1 2011-05-27T11:35:19.021-0400: 148.631: [GC [PSYoungGen: 0K->0K(57856K)] 62518K->62518K(232640K), 0.0009670 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] UseAdaptiveSizePolicy actions to meet *** throughput goal *** GC overhead (%) Young generation: 2.01 (attempted to grow) Tenured generation: 0.54 (no change) Tenuring threshold: (attempted to decrease to balance GC costs) = 1 2011-05-27T11:35:19.022-0400: 148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs] [Times: user=0.32 sys=0.02, real=0.37 secs] First Full GC looks normal as PSOldGen is (174784-169093)K = 5691K from being full. Nothing happening in Perm. The second is where things start to get weird. I don't see why that GC was called. Stranger still, it's ~800ms *after* the full gc and yet no application thread allocated any memory out of young gen. for some reason that "failed" young gen collection triggers an immediate Full GC. Bug in the collector? Did you check the bug database? Regards, Kirk From rocketraman at fastmail.fm Thu Jun 2 08:35:29 2011 From: rocketraman at fastmail.fm (Raman Gupta) Date: Thu, 02 Jun 2011 11:35:29 -0400 Subject: Heap OOM for no apparent reason? In-Reply-To: <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> Message-ID: <4DE7ADC1.1010205@fastmail.fm> I did check the database but didn't find anything relevant. My search terms may not be optimal, though I did scan through all the results returned by "java.lang.OutOfMemoryError: Java heap space" as well as "0K->0K". I also suspected a bug in the collector and so I tried the same test with the G1 collector, with the same OOM result. I didn't save the log from the G1 test, but I can quite easily redo the test with any set of JVM parameters that may be helpful in debugging -- the OOM seems to be easily and consistently reproducible with this application. Cheers, Raman On 06/02/2011 09:20 AM, Charles K Pepperdine wrote: > 2011-05-27T11:35:18.214-0400: 147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs] [Times: user=0.29 sys=0.03, real=0.81 secs] > UseAdaptiveSizePolicy actions to meet *** throughput goal *** > GC overhead (%) > Young generation: 2.65 (attempted to grow) > Tenured generation: 0.54 (attempted to grow) > Tenuring threshold: (attempted to decrease to balance GC costs) = 1 > 2011-05-27T11:35:19.021-0400: 148.631: [GC [PSYoungGen: 0K->0K(57856K)] 62518K->62518K(232640K), 0.0009670 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] > UseAdaptiveSizePolicy actions to meet *** throughput goal *** > GC overhead (%) > Young generation: 2.01 (attempted to grow) > Tenured generation: 0.54 (no change) > Tenuring threshold: (attempted to decrease to balance GC costs) = 1 > 2011-05-27T11:35:19.022-0400: 148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs] [Times: user=0.32 sys=0.02, real=0.37 secs] > > First Full GC looks normal as PSOldGen is (174784-169093)K = 5691K from being full. Nothing happening in Perm. > The second is where things start to get weird. I don't see why that GC was called. Stranger still, it's ~800ms *after* the full gc and yet no application thread allocated any memory out of young gen. > for some reason that "failed" young gen collection triggers an immediate Full GC. > > Bug in the collector? Did you check the bug database? > > Regards, > Kirk > From kirk at kodewerk.com Thu Jun 2 09:01:17 2011 From: kirk at kodewerk.com (Charles K Pepperdine) Date: Thu, 2 Jun 2011 18:01:17 +0200 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DE7ADC1.1010205@fastmail.fm> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> Message-ID: <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> are you trying to create a humungous object or array? Accidentally? On Jun 2, 2011, at 5:35 PM, Raman Gupta wrote: > I did check the database but didn't find anything relevant. My search > terms may not be optimal, though I did scan through all the results > returned by "java.lang.OutOfMemoryError: Java heap space" as well as > "0K->0K". > > I also suspected a bug in the collector and so I tried the same test > with the G1 collector, with the same OOM result. I didn't save the log > from the G1 test, but I can quite easily redo the test with any set of > JVM parameters that may be helpful in debugging -- the OOM seems to be > easily and consistently reproducible with this application. > > Cheers, > Raman > > On 06/02/2011 09:20 AM, Charles K Pepperdine wrote: >> 2011-05-27T11:35:18.214-0400: 147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs] [Times: user=0.29 sys=0.03, real=0.81 secs] >> UseAdaptiveSizePolicy actions to meet *** throughput goal *** >> GC overhead (%) >> Young generation: 2.65 (attempted to grow) >> Tenured generation: 0.54 (attempted to grow) >> Tenuring threshold: (attempted to decrease to balance GC costs) = 1 >> 2011-05-27T11:35:19.021-0400: 148.631: [GC [PSYoungGen: 0K->0K(57856K)] 62518K->62518K(232640K), 0.0009670 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] >> UseAdaptiveSizePolicy actions to meet *** throughput goal *** >> GC overhead (%) >> Young generation: 2.01 (attempted to grow) >> Tenured generation: 0.54 (no change) >> Tenuring threshold: (attempted to decrease to balance GC costs) = 1 >> 2011-05-27T11:35:19.022-0400: 148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs] [Times: user=0.32 sys=0.02, real=0.37 secs] >> >> First Full GC looks normal as PSOldGen is (174784-169093)K = 5691K from being full. Nothing happening in Perm. >> The second is where things start to get weird. I don't see why that GC was called. Stranger still, it's ~800ms *after* the full gc and yet no application thread allocated any memory out of young gen. >> for some reason that "failed" young gen collection triggers an immediate Full GC. >> >> Bug in the collector? Did you check the bug database? >> >> Regards, >> Kirk >> From poonam.bajaj at oracle.com Wed Jun 1 22:53:37 2011 From: poonam.bajaj at oracle.com (Poonam Bajaj) Date: Thu, 02 Jun 2011 11:23:37 +0530 Subject: CRR: 7045662: G1: OopsInHeapRegionClosure::set_region() should not be virtual (XS) In-Reply-To: <4DE6B6B2.8010905@oracle.com> References: <4DE62525.7080006@oracle.com> <4DE62A0A.6020201@oracle.com> <4DE6B6B2.8010905@oracle.com> Message-ID: <4DE72561.8080908@oracle.com> Looks good to me! Thanks, Poonam On 6/2/2011 3:31 AM, Tony Printezis wrote: > Stefan and John, > > First, thanks for taking a look at this so promptly! > > Yes, good catch. I had missed IntoCSOopClosure. It's gone too! Along > with VerifyRSCleanCardOopClosure which is not used either (this is yet > another problem with closures: you remove the code but it's very easy > to leave the closures behind....). > > Anyway, here's the updated webrev: > > http://cr.openjdk.java.net/~tonyp/7045662/webrev.1/ > > And I'm all set with this one! > > Tony > > On 6/1/2011 8:01 AM, Stefan Karlsson wrote: >> What about IntoCSOopClosure? It also declares a set_region method. >> But since it's not used anywhere, we should probably just remove it. >> >> Otherwise, this looks good. >> >> StefanK >> >> On 06/01/2011 01:40 PM, Tony Printezis wrote: >>> Hi, >>> >>> (first of a series of code review requests for changes I've been >>> recently working on) >>> >>> Could I please have a couple of code reviews for this simple change? >>> >>> http://cr.openjdk.java.net/~tonyp/7045662/webrev.0/ >>> >>> The main change is to make the set_region() method non-virtual and >>> remove two closures that were the reason for that method being >>> virtual but that are not used any more. This change shows a very >>> small (~1%) but measurable improvement in GC times on some platforms. >>> >>> I also piggy-backed a couple of extra minor fixes on this change: >>> removal of a third unused closure and fixing two small typos in the >>> copyright text on two files (I have no idea how that C become >>> lowercase!). >>> >>> Tony >>> >> From rocketraman at fastmail.fm Thu Jun 2 09:56:03 2011 From: rocketraman at fastmail.fm (Raman Gupta) Date: Thu, 02 Jun 2011 12:56:03 -0400 Subject: Heap OOM for no apparent reason? In-Reply-To: <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> Message-ID: <4DE7C0A3.90703@fastmail.fm> I do tend to think that somewhere a large object or array is being created. In particular, Infinispan is one library we are using that may be allocating large chunks of memory -- indeed, replacing Infinispan with a local cache does seem to "fix" the problem. However, more information from the JVM would really be useful in isolating the offending code in Infinispan. Ideally, a) any large allocations should show up as part of the heap dump if the allocation succeeded but then some other subsequent code caused the OOM, or b) if the allocation itself failed, the OOM exception should include a stack trace that would allow me to isolate the allocation point (as it does normally, but for some reason in this case doesn't). In this case the heap dump shows plenty of room in heap, and there is no stack trace at the OOM, so I don't really have any way to isolate the offending allocation point. In which situations does the OOM exception get printed without an associated stack trace? Cheers, Raman On 06/02/2011 12:01 PM, Charles K Pepperdine wrote: > are you trying to create a humungous object or array? Accidentally? > > On Jun 2, 2011, at 5:35 PM, Raman Gupta wrote: > >> I did check the database but didn't find anything relevant. My search >> terms may not be optimal, though I did scan through all the results >> returned by "java.lang.OutOfMemoryError: Java heap space" as well as >> "0K->0K". >> >> I also suspected a bug in the collector and so I tried the same test >> with the G1 collector, with the same OOM result. I didn't save the log >> from the G1 test, but I can quite easily redo the test with any set of >> JVM parameters that may be helpful in debugging -- the OOM seems to be >> easily and consistently reproducible with this application. >> >> Cheers, >> Raman >> >> On 06/02/2011 09:20 AM, Charles K Pepperdine wrote: >>> 2011-05-27T11:35:18.214-0400: 147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs] [Times: user=0.29 sys=0.03, real=0.81 secs] >>> UseAdaptiveSizePolicy actions to meet *** throughput goal *** >>> GC overhead (%) >>> Young generation: 2.65 (attempted to grow) >>> Tenured generation: 0.54 (attempted to grow) >>> Tenuring threshold: (attempted to decrease to balance GC costs) = 1 >>> 2011-05-27T11:35:19.021-0400: 148.631: [GC [PSYoungGen: 0K->0K(57856K)] 62518K->62518K(232640K), 0.0009670 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] >>> UseAdaptiveSizePolicy actions to meet *** throughput goal *** >>> GC overhead (%) >>> Young generation: 2.01 (attempted to grow) >>> Tenured generation: 0.54 (no change) >>> Tenuring threshold: (attempted to decrease to balance GC costs) = 1 >>> 2011-05-27T11:35:19.022-0400: 148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs] [Times: user=0.32 sys=0.02, real=0.37 secs] >>> >>> First Full GC looks normal as PSOldGen is (174784-169093)K = 5691K from being full. Nothing happening in Perm. >>> The second is where things start to get weird. I don't see why that GC was called. Stranger still, it's ~800ms *after* the full gc and yet no application thread allocated any memory out of young gen. >>> for some reason that "failed" young gen collection triggers an immediate Full GC. >>> >>> Bug in the collector? Did you check the bug database? >>> >>> Regards, >>> Kirk >>> > From y.s.ramakrishna at oracle.com Thu Jun 2 10:46:47 2011 From: y.s.ramakrishna at oracle.com (Y. S. Ramakrishna) Date: Thu, 02 Jun 2011 10:46:47 -0700 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DE7C0A3.90703@fastmail.fm> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> <4DE7C0A3.90703@fastmail.fm> Message-ID: <4DE7CC87.8070106@oracle.com> If your code is not catching the OOM exception you'd expect to see the stack retrace when the program dies. If it catches the exception and carries on, you'd want it to print the exception detail. I don't know of cases where the exception would just disappear. In your case the report to stdout/stderr(?)that an OOM occurred and that the heap is being dumped comes from inside the JVM because you have asked for -XX:+HeapDumpOnOutOfMemoryError. After this point, your allocating thread would have gotten an OOME which it probably caught and swallowed, and hence the silence wrt the stack retrace you would normally see. You will want to look at your Infinispan code to see how it deals with the inability to allocate said large objects. Recall that object size is limited by the size of and available space in the largest area (Eden or Old) in your Java heap. As Kirk noted, the full gc was to attempt allocation of an object that didn't fit into the available space in Eden or in Old (so from that you can estimate the size of the request). Note also that the JDK libraries will resize hashtables under you and that can also cause large allocation requests (but i don't know how they handle OOM's resulting from such allocations). -- ramki On 06/02/11 09:56, Raman Gupta wrote: > I do tend to think that somewhere a large object or array is being > created. In particular, Infinispan is one library we are using that > may be allocating large chunks of memory -- indeed, replacing > Infinispan with a local cache does seem to "fix" the problem. > > However, more information from the JVM would really be useful in > isolating the offending code in Infinispan. Ideally, > > a) any large allocations should show up as part of the heap dump if > the allocation succeeded but then some other subsequent code caused > the OOM, or > > b) if the allocation itself failed, the OOM exception should include a > stack trace that would allow me to isolate the allocation point (as > it does normally, but for some reason in this case doesn't). > > In this case the heap dump shows plenty of room in heap, and there is > no stack trace at the OOM, so I don't really have any way to isolate > the offending allocation point. In which situations does the OOM > exception get printed without an associated stack trace? > > Cheers, > Raman > > > On 06/02/2011 12:01 PM, Charles K Pepperdine wrote: >> are you trying to create a humungous object or array? Accidentally? >> >> On Jun 2, 2011, at 5:35 PM, Raman Gupta wrote: >> >>> I did check the database but didn't find anything relevant. My search >>> terms may not be optimal, though I did scan through all the results >>> returned by "java.lang.OutOfMemoryError: Java heap space" as well as >>> "0K->0K". >>> >>> I also suspected a bug in the collector and so I tried the same test >>> with the G1 collector, with the same OOM result. I didn't save the log >>> from the G1 test, but I can quite easily redo the test with any set of >>> JVM parameters that may be helpful in debugging -- the OOM seems to be >>> easily and consistently reproducible with this application. >>> >>> Cheers, >>> Raman >>> >>> On 06/02/2011 09:20 AM, Charles K Pepperdine wrote: >>>> 2011-05-27T11:35:18.214-0400: 147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs] [Times: user=0.29 sys=0.03, real=0.81 secs] >>>> UseAdaptiveSizePolicy actions to meet *** throughput goal *** >>>> GC overhead (%) >>>> Young generation: 2.65 (attempted to grow) >>>> Tenured generation: 0.54 (attempted to grow) >>>> Tenuring threshold: (attempted to decrease to balance GC costs) = 1 >>>> 2011-05-27T11:35:19.021-0400: 148.631: [GC [PSYoungGen: 0K->0K(57856K)] 62518K->62518K(232640K), 0.0009670 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] >>>> UseAdaptiveSizePolicy actions to meet *** throughput goal *** >>>> GC overhead (%) >>>> Young generation: 2.01 (attempted to grow) >>>> Tenured generation: 0.54 (no change) >>>> Tenuring threshold: (attempted to decrease to balance GC costs) = 1 >>>> 2011-05-27T11:35:19.022-0400: 148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs] [Times: user=0.32 sys=0.02, real=0.37 secs] >>>> >>>> First Full GC looks normal as PSOldGen is (174784-169093)K = 5691K from being full. Nothing happening in Perm. >>>> The second is where things start to get weird. I don't see why that GC was called. Stranger still, it's ~800ms *after* the full gc and yet no application thread allocated any memory out of young gen. >>>> for some reason that "failed" young gen collection triggers an immediate Full GC. >>>> >>>> Bug in the collector? Did you check the bug database? >>>> >>>> Regards, >>>> Kirk >>>> From kirk at kodewerk.com Thu Jun 2 10:54:17 2011 From: kirk at kodewerk.com (Charles K Pepperdine) Date: Thu, 2 Jun 2011 19:54:17 +0200 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DE7C0A3.90703@fastmail.fm> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> <4DE7C0A3.90703@fastmail.fm> Message-ID: Well, GC is some what orthogonal to what you're application is up to except for this special case. I've cc'ed Manik in on this one maybe he's had someone run into it before. Regards, Kirk On Jun 2, 2011, at 6:56 PM, Raman Gupta wrote: > I do tend to think that somewhere a large object or array is being > created. In particular, Infinispan is one library we are using that > may be allocating large chunks of memory -- indeed, replacing > Infinispan with a local cache does seem to "fix" the problem. > > However, more information from the JVM would really be useful in > isolating the offending code in Infinispan. Ideally, > > a) any large allocations should show up as part of the heap dump if > the allocation succeeded but then some other subsequent code caused > the OOM, or > > b) if the allocation itself failed, the OOM exception should include a > stack trace that would allow me to isolate the allocation point (as > it does normally, but for some reason in this case doesn't). > > In this case the heap dump shows plenty of room in heap, and there is > no stack trace at the OOM, so I don't really have any way to isolate > the offending allocation point. In which situations does the OOM > exception get printed without an associated stack trace? > > Cheers, > Raman > > > On 06/02/2011 12:01 PM, Charles K Pepperdine wrote: >> are you trying to create a humungous object or array? Accidentally? >> >> On Jun 2, 2011, at 5:35 PM, Raman Gupta wrote: >> >>> I did check the database but didn't find anything relevant. My search >>> terms may not be optimal, though I did scan through all the results >>> returned by "java.lang.OutOfMemoryError: Java heap space" as well as >>> "0K->0K". >>> >>> I also suspected a bug in the collector and so I tried the same test >>> with the G1 collector, with the same OOM result. I didn't save the log >>> from the G1 test, but I can quite easily redo the test with any set of >>> JVM parameters that may be helpful in debugging -- the OOM seems to be >>> easily and consistently reproducible with this application. >>> >>> Cheers, >>> Raman >>> >>> On 06/02/2011 09:20 AM, Charles K Pepperdine wrote: >>>> 2011-05-27T11:35:18.214-0400: 147.823: [Full GC [PSYoungGen: 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], 0.8074580 secs] [Times: user=0.29 sys=0.03, real=0.81 secs] >>>> UseAdaptiveSizePolicy actions to meet *** throughput goal *** >>>> GC overhead (%) >>>> Young generation: 2.65 (attempted to grow) >>>> Tenured generation: 0.54 (attempted to grow) >>>> Tenuring threshold: (attempted to decrease to balance GC costs) = 1 >>>> 2011-05-27T11:35:19.021-0400: 148.631: [GC [PSYoungGen: 0K->0K(57856K)] 62518K->62518K(232640K), 0.0009670 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] >>>> UseAdaptiveSizePolicy actions to meet *** throughput goal *** >>>> GC overhead (%) >>>> Young generation: 2.01 (attempted to grow) >>>> Tenured generation: 0.54 (no change) >>>> Tenuring threshold: (attempted to decrease to balance GC costs) = 1 >>>> 2011-05-27T11:35:19.022-0400: 148.632: [Full GC [PSYoungGen: 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], 0.3614330 secs] [Times: user=0.32 sys=0.02, real=0.37 secs] >>>> >>>> First Full GC looks normal as PSOldGen is (174784-169093)K = 5691K from being full. Nothing happening in Perm. >>>> The second is where things start to get weird. I don't see why that GC was called. Stranger still, it's ~800ms *after* the full gc and yet no application thread allocated any memory out of young gen. >>>> for some reason that "failed" young gen collection triggers an immediate Full GC. >>>> >>>> Bug in the collector? Did you check the bug database? >>>> >>>> Regards, >>>> Kirk >>>> >> From rocketraman at fastmail.fm Thu Jun 2 18:15:24 2011 From: rocketraman at fastmail.fm (Raman Gupta) Date: Thu, 02 Jun 2011 21:15:24 -0400 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DE7CC87.8070106@oracle.com> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> <4DE7C0A3.90703@fastmail.fm> <4DE7CC87.8070106@oracle.com> Message-ID: <4DE835AC.8060404@fastmail.fm> It would be *really* handy if there were a switch like: -XX:+StackTraceOnOutOfMemoryError to force the stack trace to be shown. Obviously looking at every line of code of every library my application uses, including core JDK libraries, for code paths where large amounts of heap may be allocated and the associated OOME is caught and swallowed, is pretty much impossible. I think my next step is to increase the max heap size to a large value which hopefully allows the large allocation to occur without failure, and then periodically take heap dumps to isolate it. Thanks, Raman On 06/02/2011 01:46 PM, Y. S. Ramakrishna wrote: > If your code is not catching the OOM exception you'd > expect to see the stack retrace when the program dies. > If it catches the exception and carries on, you'd want > it to print the exception detail. I don't know of > cases where the exception would just disappear. > > In your case the report to stdout/stderr(?)that an OOM occurred and that > the heap is being dumped comes from inside the JVM > because you have asked for -XX:+HeapDumpOnOutOfMemoryError. > After this point, your allocating thread would have gotten > an OOME which it probably caught and swallowed, and hence > the silence wrt the stack retrace you would normally see. You > will want to look at your Infinispan code to see how > it deals with the inability to allocate said large objects. > > Recall that object size is limited by the size of and > available space in the largest area (Eden or Old) in your > Java heap. As Kirk noted, the full gc was to attempt allocation > of an object that didn't fit into the available space in > Eden or in Old (so from that you can estimate the size of > the request). > > Note also that the JDK libraries will resize hashtables under > you and that can also cause large allocation requests > (but i don't know how they handle OOM's resulting from such > allocations). > > -- ramki > > On 06/02/11 09:56, Raman Gupta wrote: >> I do tend to think that somewhere a large object or array is being >> created. In particular, Infinispan is one library we are using that >> may be allocating large chunks of memory -- indeed, replacing >> Infinispan with a local cache does seem to "fix" the problem. >> >> However, more information from the JVM would really be useful in >> isolating the offending code in Infinispan. Ideally, >> >> a) any large allocations should show up as part of the heap dump if >> the allocation succeeded but then some other subsequent code caused >> the OOM, or >> >> b) if the allocation itself failed, the OOM exception should include a >> stack trace that would allow me to isolate the allocation point (as >> it does normally, but for some reason in this case doesn't). >> >> In this case the heap dump shows plenty of room in heap, and there is >> no stack trace at the OOM, so I don't really have any way to isolate >> the offending allocation point. In which situations does the OOM >> exception get printed without an associated stack trace? >> >> Cheers, >> Raman >> >> >> On 06/02/2011 12:01 PM, Charles K Pepperdine wrote: >>> are you trying to create a humungous object or array? Accidentally? >>> >>> On Jun 2, 2011, at 5:35 PM, Raman Gupta wrote: >>> >>>> I did check the database but didn't find anything relevant. My search >>>> terms may not be optimal, though I did scan through all the results >>>> returned by "java.lang.OutOfMemoryError: Java heap space" as well as >>>> "0K->0K". >>>> >>>> I also suspected a bug in the collector and so I tried the same test >>>> with the G1 collector, with the same OOM result. I didn't save the >>>> log >>>> from the G1 test, but I can quite easily redo the test with any >>>> set of >>>> JVM parameters that may be helpful in debugging -- the OOM seems >>>> to be >>>> easily and consistently reproducible with this application. >>>> >>>> Cheers, >>>> Raman >>>> >>>> On 06/02/2011 09:20 AM, Charles K Pepperdine wrote: >>>>> 2011-05-27T11:35:18.214-0400: 147.823: [Full GC [PSYoungGen: >>>>> 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] >>>>> 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], >>>>> 0.8074580 secs] [Times: user=0.29 sys=0.03, real=0.81 secs] >>>>> UseAdaptiveSizePolicy actions to meet *** throughput goal *** >>>>> GC overhead (%) >>>>> Young generation: 2.65 (attempted to grow) >>>>> Tenured generation: 0.54 (attempted to grow) >>>>> Tenuring threshold: (attempted to decrease to balance GC >>>>> costs) = 1 >>>>> 2011-05-27T11:35:19.021-0400: 148.631: [GC [PSYoungGen: >>>>> 0K->0K(57856K)] 62518K->62518K(232640K), 0.0009670 secs] [Times: >>>>> user=0.00 sys=0.00, real=0.00 secs] UseAdaptiveSizePolicy >>>>> actions to meet *** throughput goal *** >>>>> GC overhead (%) >>>>> Young generation: 2.01 (attempted to grow) >>>>> Tenured generation: 0.54 (no change) >>>>> Tenuring threshold: (attempted to decrease to balance GC >>>>> costs) = 1 >>>>> 2011-05-27T11:35:19.022-0400: 148.632: [Full GC [PSYoungGen: >>>>> 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] >>>>> 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], >>>>> 0.3614330 secs] [Times: user=0.32 sys=0.02, real=0.37 secs] >>>>> First Full GC looks normal as PSOldGen is (174784-169093)K = >>>>> 5691K from being full. Nothing happening in Perm. >>>>> The second is where things start to get weird. I don't see why >>>>> that GC was called. Stranger still, it's ~800ms *after* the full >>>>> gc and yet no application thread allocated any memory out of >>>>> young gen. >>>>> for some reason that "failed" young gen collection triggers an >>>>> immediate Full GC. >>>>> >>>>> Bug in the collector? Did you check the bug database? >>>>> >>>>> Regards, >>>>> Kirk >>>>> From y.s.ramakrishna at oracle.com Thu Jun 2 22:40:49 2011 From: y.s.ramakrishna at oracle.com (Y. Srinivas Ramakrishna) Date: Thu, 02 Jun 2011 22:40:49 -0700 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DE835AC.8060404@fastmail.fm> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> <4DE7C0A3.90703@fastmail.fm> <4DE7CC87.8070106@oracle.com> <4DE835AC.8060404@fastmail.fm> Message-ID: <4DE873E1.3070703@oracle.com> You "just" have to find all the places where OOME is caught. Hopefully there aren't too many of those? -- ramki On 6/2/2011 6:15 PM, Raman Gupta wrote: > It would be *really* handy if there were a switch like: > > -XX:+StackTraceOnOutOfMemoryError > > to force the stack trace to be shown. Obviously looking at every line > of code of every library my application uses, including core JDK > libraries, for code paths where large amounts of heap may be allocated > and the associated OOME is caught and swallowed, is pretty much > impossible. > > I think my next step is to increase the max heap size to a large value > which hopefully allows the large allocation to occur without failure, > and then periodically take heap dumps to isolate it. > > Thanks, > Raman > > On 06/02/2011 01:46 PM, Y. S. Ramakrishna wrote: >> If your code is not catching the OOM exception you'd >> expect to see the stack retrace when the program dies. >> If it catches the exception and carries on, you'd want >> it to print the exception detail. I don't know of >> cases where the exception would just disappear. >> >> In your case the report to stdout/stderr(?)that an OOM occurred and that >> the heap is being dumped comes from inside the JVM >> because you have asked for -XX:+HeapDumpOnOutOfMemoryError. >> After this point, your allocating thread would have gotten >> an OOME which it probably caught and swallowed, and hence >> the silence wrt the stack retrace you would normally see. You >> will want to look at your Infinispan code to see how >> it deals with the inability to allocate said large objects. >> >> Recall that object size is limited by the size of and >> available space in the largest area (Eden or Old) in your >> Java heap. As Kirk noted, the full gc was to attempt allocation >> of an object that didn't fit into the available space in >> Eden or in Old (so from that you can estimate the size of >> the request). >> >> Note also that the JDK libraries will resize hashtables under >> you and that can also cause large allocation requests >> (but i don't know how they handle OOM's resulting from such >> allocations). >> >> -- ramki >> >> On 06/02/11 09:56, Raman Gupta wrote: >>> I do tend to think that somewhere a large object or array is being >>> created. In particular, Infinispan is one library we are using that >>> may be allocating large chunks of memory -- indeed, replacing >>> Infinispan with a local cache does seem to "fix" the problem. >>> >>> However, more information from the JVM would really be useful in >>> isolating the offending code in Infinispan. Ideally, >>> >>> a) any large allocations should show up as part of the heap dump if >>> the allocation succeeded but then some other subsequent code caused >>> the OOM, or >>> >>> b) if the allocation itself failed, the OOM exception should include a >>> stack trace that would allow me to isolate the allocation point (as >>> it does normally, but for some reason in this case doesn't). >>> >>> In this case the heap dump shows plenty of room in heap, and there is >>> no stack trace at the OOM, so I don't really have any way to isolate >>> the offending allocation point. In which situations does the OOM >>> exception get printed without an associated stack trace? >>> >>> Cheers, >>> Raman >>> >>> >>> On 06/02/2011 12:01 PM, Charles K Pepperdine wrote: >>>> are you trying to create a humungous object or array? Accidentally? >>>> >>>> On Jun 2, 2011, at 5:35 PM, Raman Gupta wrote: >>>> >>>>> I did check the database but didn't find anything relevant. My search >>>>> terms may not be optimal, though I did scan through all the results >>>>> returned by "java.lang.OutOfMemoryError: Java heap space" as well as >>>>> "0K->0K". >>>>> >>>>> I also suspected a bug in the collector and so I tried the same test >>>>> with the G1 collector, with the same OOM result. I didn't save the >>>>> log >>>>> from the G1 test, but I can quite easily redo the test with any >>>>> set of >>>>> JVM parameters that may be helpful in debugging -- the OOM seems >>>>> to be >>>>> easily and consistently reproducible with this application. >>>>> >>>>> Cheers, >>>>> Raman >>>>> >>>>> On 06/02/2011 09:20 AM, Charles K Pepperdine wrote: >>>>>> 2011-05-27T11:35:18.214-0400: 147.823: [Full GC [PSYoungGen: >>>>>> 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] >>>>>> 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], >>>>>> 0.8074580 secs] [Times: user=0.29 sys=0.03, real=0.81 secs] >>>>>> UseAdaptiveSizePolicy actions to meet *** throughput goal *** >>>>>> GC overhead (%) >>>>>> Young generation: 2.65 (attempted to grow) >>>>>> Tenured generation: 0.54 (attempted to grow) >>>>>> Tenuring threshold: (attempted to decrease to balance GC >>>>>> costs) = 1 >>>>>> 2011-05-27T11:35:19.021-0400: 148.631: [GC [PSYoungGen: >>>>>> 0K->0K(57856K)] 62518K->62518K(232640K), 0.0009670 secs] [Times: >>>>>> user=0.00 sys=0.00, real=0.00 secs] UseAdaptiveSizePolicy >>>>>> actions to meet *** throughput goal *** >>>>>> GC overhead (%) >>>>>> Young generation: 2.01 (attempted to grow) >>>>>> Tenured generation: 0.54 (no change) >>>>>> Tenuring threshold: (attempted to decrease to balance GC >>>>>> costs) = 1 >>>>>> 2011-05-27T11:35:19.022-0400: 148.632: [Full GC [PSYoungGen: >>>>>> 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] >>>>>> 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], >>>>>> 0.3614330 secs] [Times: user=0.32 sys=0.02, real=0.37 secs] >>>>>> First Full GC looks normal as PSOldGen is (174784-169093)K = >>>>>> 5691K from being full. Nothing happening in Perm. >>>>>> The second is where things start to get weird. I don't see why >>>>>> that GC was called. Stranger still, it's ~800ms *after* the full >>>>>> gc and yet no application thread allocated any memory out of >>>>>> young gen. >>>>>> for some reason that "failed" young gen collection triggers an >>>>>> immediate Full GC. >>>>>> >>>>>> Bug in the collector? Did you check the bug database? >>>>>> >>>>>> Regards, >>>>>> Kirk >>>>>> From y.s.ramakrishna at oracle.com Thu Jun 2 22:54:45 2011 From: y.s.ramakrishna at oracle.com (Y. Srinivas Ramakrishna) Date: Thu, 02 Jun 2011 22:54:45 -0700 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DE835AC.8060404@fastmail.fm> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> <4DE7C0A3.90703@fastmail.fm> <4DE7CC87.8070106@oracle.com> <4DE835AC.8060404@fastmail.fm> Message-ID: <4DE87725.20706@oracle.com> Sorry, sent previous email without addressing all of the issues. On 6/2/2011 6:15 PM, Raman Gupta wrote: > It would be *really* handy if there were a switch like: > > -XX:+StackTraceOnOutOfMemoryError Yes that would be handy, and probably not too difficult. But I wonder also if something like OnOutOfMemoryError or like would already get you enough info to get close to the problem ... (although may be because it's executed in a separate shell, by the time the command executes the process has likely gone well past the point when the problem occurred). > > to force the stack trace to be shown. Obviously looking at every line > of code of every library my application uses, including core JDK > libraries, for code paths where large amounts of heap may be allocated > and the associated OOME is caught and swallowed, is pretty much > impossible. Try to look for places where OOME (or supertype?) is caught. I am hoping there aren't too many of those... > > I think my next step is to increase the max heap size to a large value > which hopefully allows the large allocation to occur without failure, > and then periodically take heap dumps to isolate it. Yes that seems reasonable, or may be use an allocation profiler with the larger heap and find it that way... -- ramki > > Thanks, > Raman > > On 06/02/2011 01:46 PM, Y. S. Ramakrishna wrote: >> If your code is not catching the OOM exception you'd >> expect to see the stack retrace when the program dies. >> If it catches the exception and carries on, you'd want >> it to print the exception detail. I don't know of >> cases where the exception would just disappear. >> >> In your case the report to stdout/stderr(?)that an OOM occurred and that >> the heap is being dumped comes from inside the JVM >> because you have asked for -XX:+HeapDumpOnOutOfMemoryError. >> After this point, your allocating thread would have gotten >> an OOME which it probably caught and swallowed, and hence >> the silence wrt the stack retrace you would normally see. You >> will want to look at your Infinispan code to see how >> it deals with the inability to allocate said large objects. >> >> Recall that object size is limited by the size of and >> available space in the largest area (Eden or Old) in your >> Java heap. As Kirk noted, the full gc was to attempt allocation >> of an object that didn't fit into the available space in >> Eden or in Old (so from that you can estimate the size of >> the request). >> >> Note also that the JDK libraries will resize hashtables under >> you and that can also cause large allocation requests >> (but i don't know how they handle OOM's resulting from such >> allocations). >> >> -- ramki >> >> On 06/02/11 09:56, Raman Gupta wrote: >>> I do tend to think that somewhere a large object or array is being >>> created. In particular, Infinispan is one library we are using that >>> may be allocating large chunks of memory -- indeed, replacing >>> Infinispan with a local cache does seem to "fix" the problem. >>> >>> However, more information from the JVM would really be useful in >>> isolating the offending code in Infinispan. Ideally, >>> >>> a) any large allocations should show up as part of the heap dump if >>> the allocation succeeded but then some other subsequent code caused >>> the OOM, or >>> >>> b) if the allocation itself failed, the OOM exception should include a >>> stack trace that would allow me to isolate the allocation point (as >>> it does normally, but for some reason in this case doesn't). >>> >>> In this case the heap dump shows plenty of room in heap, and there is >>> no stack trace at the OOM, so I don't really have any way to isolate >>> the offending allocation point. In which situations does the OOM >>> exception get printed without an associated stack trace? >>> >>> Cheers, >>> Raman >>> >>> >>> On 06/02/2011 12:01 PM, Charles K Pepperdine wrote: >>>> are you trying to create a humungous object or array? Accidentally? >>>> >>>> On Jun 2, 2011, at 5:35 PM, Raman Gupta wrote: >>>> >>>>> I did check the database but didn't find anything relevant. My search >>>>> terms may not be optimal, though I did scan through all the results >>>>> returned by "java.lang.OutOfMemoryError: Java heap space" as well as >>>>> "0K->0K". >>>>> >>>>> I also suspected a bug in the collector and so I tried the same test >>>>> with the G1 collector, with the same OOM result. I didn't save the >>>>> log >>>>> from the G1 test, but I can quite easily redo the test with any >>>>> set of >>>>> JVM parameters that may be helpful in debugging -- the OOM seems >>>>> to be >>>>> easily and consistently reproducible with this application. >>>>> >>>>> Cheers, >>>>> Raman >>>>> >>>>> On 06/02/2011 09:20 AM, Charles K Pepperdine wrote: >>>>>> 2011-05-27T11:35:18.214-0400: 147.823: [Full GC [PSYoungGen: >>>>>> 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] >>>>>> 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], >>>>>> 0.8074580 secs] [Times: user=0.29 sys=0.03, real=0.81 secs] >>>>>> UseAdaptiveSizePolicy actions to meet *** throughput goal *** >>>>>> GC overhead (%) >>>>>> Young generation: 2.65 (attempted to grow) >>>>>> Tenured generation: 0.54 (attempted to grow) >>>>>> Tenuring threshold: (attempted to decrease to balance GC >>>>>> costs) = 1 >>>>>> 2011-05-27T11:35:19.021-0400: 148.631: [GC [PSYoungGen: >>>>>> 0K->0K(57856K)] 62518K->62518K(232640K), 0.0009670 secs] [Times: >>>>>> user=0.00 sys=0.00, real=0.00 secs] UseAdaptiveSizePolicy >>>>>> actions to meet *** throughput goal *** >>>>>> GC overhead (%) >>>>>> Young generation: 2.01 (attempted to grow) >>>>>> Tenured generation: 0.54 (no change) >>>>>> Tenuring threshold: (attempted to decrease to balance GC >>>>>> costs) = 1 >>>>>> 2011-05-27T11:35:19.022-0400: 148.632: [Full GC [PSYoungGen: >>>>>> 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] >>>>>> 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], >>>>>> 0.3614330 secs] [Times: user=0.32 sys=0.02, real=0.37 secs] >>>>>> First Full GC looks normal as PSOldGen is (174784-169093)K = >>>>>> 5691K from being full. Nothing happening in Perm. >>>>>> The second is where things start to get weird. I don't see why >>>>>> that GC was called. Stranger still, it's ~800ms *after* the full >>>>>> gc and yet no application thread allocated any memory out of >>>>>> young gen. >>>>>> for some reason that "failed" young gen collection triggers an >>>>>> immediate Full GC. >>>>>> >>>>>> Bug in the collector? Did you check the bug database? >>>>>> >>>>>> Regards, >>>>>> Kirk >>>>>> From kirk at kodewerk.com Thu Jun 2 23:35:32 2011 From: kirk at kodewerk.com (Charles K Pepperdine) Date: Fri, 3 Jun 2011 08:35:32 +0200 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DE835AC.8060404@fastmail.fm> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> <4DE7C0A3.90703@fastmail.fm> <4DE7CC87.8070106@oracle.com> <4DE835AC.8060404@fastmail.fm> Message-ID: you should get a stack trace, something must be eating it. Regards, Kirk On Jun 3, 2011, at 3:15 AM, Raman Gupta wrote: > It would be *really* handy if there were a switch like: > > -XX:+StackTraceOnOutOfMemoryError > > to force the stack trace to be shown. Obviously looking at every line > of code of every library my application uses, including core JDK > libraries, for code paths where large amounts of heap may be allocated > and the associated OOME is caught and swallowed, is pretty much > impossible. > > I think my next step is to increase the max heap size to a large value > which hopefully allows the large allocation to occur without failure, > and then periodically take heap dumps to isolate it. > > Thanks, > Raman > > On 06/02/2011 01:46 PM, Y. S. Ramakrishna wrote: >> If your code is not catching the OOM exception you'd >> expect to see the stack retrace when the program dies. >> If it catches the exception and carries on, you'd want >> it to print the exception detail. I don't know of >> cases where the exception would just disappear. >> >> In your case the report to stdout/stderr(?)that an OOM occurred and that >> the heap is being dumped comes from inside the JVM >> because you have asked for -XX:+HeapDumpOnOutOfMemoryError. >> After this point, your allocating thread would have gotten >> an OOME which it probably caught and swallowed, and hence >> the silence wrt the stack retrace you would normally see. You >> will want to look at your Infinispan code to see how >> it deals with the inability to allocate said large objects. >> >> Recall that object size is limited by the size of and >> available space in the largest area (Eden or Old) in your >> Java heap. As Kirk noted, the full gc was to attempt allocation >> of an object that didn't fit into the available space in >> Eden or in Old (so from that you can estimate the size of >> the request). >> >> Note also that the JDK libraries will resize hashtables under >> you and that can also cause large allocation requests >> (but i don't know how they handle OOM's resulting from such >> allocations). >> >> -- ramki >> >> On 06/02/11 09:56, Raman Gupta wrote: >>> I do tend to think that somewhere a large object or array is being >>> created. In particular, Infinispan is one library we are using that >>> may be allocating large chunks of memory -- indeed, replacing >>> Infinispan with a local cache does seem to "fix" the problem. >>> >>> However, more information from the JVM would really be useful in >>> isolating the offending code in Infinispan. Ideally, >>> >>> a) any large allocations should show up as part of the heap dump if >>> the allocation succeeded but then some other subsequent code caused >>> the OOM, or >>> >>> b) if the allocation itself failed, the OOM exception should include a >>> stack trace that would allow me to isolate the allocation point (as >>> it does normally, but for some reason in this case doesn't). >>> >>> In this case the heap dump shows plenty of room in heap, and there is >>> no stack trace at the OOM, so I don't really have any way to isolate >>> the offending allocation point. In which situations does the OOM >>> exception get printed without an associated stack trace? >>> >>> Cheers, >>> Raman >>> >>> >>> On 06/02/2011 12:01 PM, Charles K Pepperdine wrote: >>>> are you trying to create a humungous object or array? Accidentally? >>>> >>>> On Jun 2, 2011, at 5:35 PM, Raman Gupta wrote: >>>> >>>>> I did check the database but didn't find anything relevant. My search >>>>> terms may not be optimal, though I did scan through all the results >>>>> returned by "java.lang.OutOfMemoryError: Java heap space" as well as >>>>> "0K->0K". >>>>> >>>>> I also suspected a bug in the collector and so I tried the same test >>>>> with the G1 collector, with the same OOM result. I didn't save the >>>>> log >>>>> from the G1 test, but I can quite easily redo the test with any >>>>> set of >>>>> JVM parameters that may be helpful in debugging -- the OOM seems >>>>> to be >>>>> easily and consistently reproducible with this application. >>>>> >>>>> Cheers, >>>>> Raman >>>>> >>>>> On 06/02/2011 09:20 AM, Charles K Pepperdine wrote: >>>>>> 2011-05-27T11:35:18.214-0400: 147.823: [Full GC [PSYoungGen: >>>>>> 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] >>>>>> 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], >>>>>> 0.8074580 secs] [Times: user=0.29 sys=0.03, real=0.81 secs] >>>>>> UseAdaptiveSizePolicy actions to meet *** throughput goal *** >>>>>> GC overhead (%) >>>>>> Young generation: 2.65 (attempted to grow) >>>>>> Tenured generation: 0.54 (attempted to grow) >>>>>> Tenuring threshold: (attempted to decrease to balance GC >>>>>> costs) = 1 >>>>>> 2011-05-27T11:35:19.021-0400: 148.631: [GC [PSYoungGen: >>>>>> 0K->0K(57856K)] 62518K->62518K(232640K), 0.0009670 secs] [Times: >>>>>> user=0.00 sys=0.00, real=0.00 secs] UseAdaptiveSizePolicy >>>>>> actions to meet *** throughput goal *** >>>>>> GC overhead (%) >>>>>> Young generation: 2.01 (attempted to grow) >>>>>> Tenured generation: 0.54 (no change) >>>>>> Tenuring threshold: (attempted to decrease to balance GC >>>>>> costs) = 1 >>>>>> 2011-05-27T11:35:19.022-0400: 148.632: [Full GC [PSYoungGen: >>>>>> 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] >>>>>> 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], >>>>>> 0.3614330 secs] [Times: user=0.32 sys=0.02, real=0.37 secs] >>>>>> First Full GC looks normal as PSOldGen is (174784-169093)K = >>>>>> 5691K from being full. Nothing happening in Perm. >>>>>> The second is where things start to get weird. I don't see why >>>>>> that GC was called. Stranger still, it's ~800ms *after* the full >>>>>> gc and yet no application thread allocated any memory out of >>>>>> young gen. >>>>>> for some reason that "failed" young gen collection triggers an >>>>>> immediate Full GC. >>>>>> >>>>>> Bug in the collector? Did you check the bug database? >>>>>> >>>>>> Regards, >>>>>> Kirk >>>>>> From John.Coomes at oracle.com Fri Jun 3 00:28:43 2011 From: John.Coomes at oracle.com (John Coomes) Date: Fri, 3 Jun 2011 00:28:43 -0700 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DE87725.20706@oracle.com> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> <4DE7C0A3.90703@fastmail.fm> <4DE7CC87.8070106@oracle.com> <4DE835AC.8060404@fastmail.fm> <4DE87725.20706@oracle.com> Message-ID: <19944.36139.44254.445391@oracle.com> Y. Srinivas Ramakrishna (y.s.ramakrishna at oracle.com) wrote: > Sorry, sent previous email without addressing all of the issues. > > On 6/2/2011 6:15 PM, Raman Gupta wrote: > > It would be *really* handy if there were a switch like: > > > > -XX:+StackTraceOnOutOfMemoryError > > Yes that would be handy, and probably not too difficult. > But I wonder also if something like OnOutOfMemoryError or > like would already get you enough info to get close to > the problem ... (although may be because it's executed in > a separate shell, by the time the command executes the > process has likely gone well past the point when the problem > occurred). No need to worry, the OnOutOfMemoryError commands are run while the JVM is at a safepoint. This worked for me: java -XX:OnOutOfMemoryError='jstack %p' ... -John > > to force the stack trace to be shown. Obviously looking at every line > > of code of every library my application uses, including core JDK > > libraries, for code paths where large amounts of heap may be allocated > > and the associated OOME is caught and swallowed, is pretty much > > impossible. > > Try to look for places where OOME (or supertype?) is caught. I am > hoping there aren't too many of those... > > > > > I think my next step is to increase the max heap size to a large value > > which hopefully allows the large allocation to occur without failure, > > and then periodically take heap dumps to isolate it. > > Yes that seems reasonable, or may be use an allocation profiler > with the larger heap and find it that way... > > -- ramki > > > > > Thanks, > > Raman > > > > On 06/02/2011 01:46 PM, Y. S. Ramakrishna wrote: > >> If your code is not catching the OOM exception you'd > >> expect to see the stack retrace when the program dies. > >> If it catches the exception and carries on, you'd want > >> it to print the exception detail. I don't know of > >> cases where the exception would just disappear. > >> > >> In your case the report to stdout/stderr(?)that an OOM occurred and that > >> the heap is being dumped comes from inside the JVM > >> because you have asked for -XX:+HeapDumpOnOutOfMemoryError. > >> After this point, your allocating thread would have gotten > >> an OOME which it probably caught and swallowed, and hence > >> the silence wrt the stack retrace you would normally see. You > >> will want to look at your Infinispan code to see how > >> it deals with the inability to allocate said large objects. > >> > >> Recall that object size is limited by the size of and > >> available space in the largest area (Eden or Old) in your > >> Java heap. As Kirk noted, the full gc was to attempt allocation > >> of an object that didn't fit into the available space in > >> Eden or in Old (so from that you can estimate the size of > >> the request). > >> > >> Note also that the JDK libraries will resize hashtables under > >> you and that can also cause large allocation requests > >> (but i don't know how they handle OOM's resulting from such > >> allocations). > >> > >> -- ramki > >> > >> On 06/02/11 09:56, Raman Gupta wrote: > >>> I do tend to think that somewhere a large object or array is being > >>> created. In particular, Infinispan is one library we are using that > >>> may be allocating large chunks of memory -- indeed, replacing > >>> Infinispan with a local cache does seem to "fix" the problem. > >>> > >>> However, more information from the JVM would really be useful in > >>> isolating the offending code in Infinispan. Ideally, > >>> > >>> a) any large allocations should show up as part of the heap dump if > >>> the allocation succeeded but then some other subsequent code caused > >>> the OOM, or > >>> > >>> b) if the allocation itself failed, the OOM exception should include a > >>> stack trace that would allow me to isolate the allocation point (as > >>> it does normally, but for some reason in this case doesn't). > >>> > >>> In this case the heap dump shows plenty of room in heap, and there is > >>> no stack trace at the OOM, so I don't really have any way to isolate > >>> the offending allocation point. In which situations does the OOM > >>> exception get printed without an associated stack trace? > >>> > >>> Cheers, > >>> Raman > >>> > >>> > >>> On 06/02/2011 12:01 PM, Charles K Pepperdine wrote: > >>>> are you trying to create a humungous object or array? Accidentally? > >>>> > >>>> On Jun 2, 2011, at 5:35 PM, Raman Gupta wrote: > >>>> > >>>>> I did check the database but didn't find anything relevant. My search > >>>>> terms may not be optimal, though I did scan through all the results > >>>>> returned by "java.lang.OutOfMemoryError: Java heap space" as well as > >>>>> "0K->0K". > >>>>> > >>>>> I also suspected a bug in the collector and so I tried the same test > >>>>> with the G1 collector, with the same OOM result. I didn't save the > >>>>> log > >>>>> from the G1 test, but I can quite easily redo the test with any > >>>>> set of > >>>>> JVM parameters that may be helpful in debugging -- the OOM seems > >>>>> to be > >>>>> easily and consistently reproducible with this application. > >>>>> > >>>>> Cheers, > >>>>> Raman > >>>>> > >>>>> On 06/02/2011 09:20 AM, Charles K Pepperdine wrote: > >>>>>> 2011-05-27T11:35:18.214-0400: 147.823: [Full GC [PSYoungGen: > >>>>>> 7662K->0K(72896K)] [PSOldGen: 169093K->62518K(174784K)] > >>>>>> 176755K->62518K(247680K) [PSPermGen: 27342K->27342K(55232K)], > >>>>>> 0.8074580 secs] [Times: user=0.29 sys=0.03, real=0.81 secs] > >>>>>> UseAdaptiveSizePolicy actions to meet *** throughput goal *** > >>>>>> GC overhead (%) > >>>>>> Young generation: 2.65 (attempted to grow) > >>>>>> Tenured generation: 0.54 (attempted to grow) > >>>>>> Tenuring threshold: (attempted to decrease to balance GC > >>>>>> costs) = 1 > >>>>>> 2011-05-27T11:35:19.021-0400: 148.631: [GC [PSYoungGen: > >>>>>> 0K->0K(57856K)] 62518K->62518K(232640K), 0.0009670 secs] [Times: > >>>>>> user=0.00 sys=0.00, real=0.00 secs] UseAdaptiveSizePolicy > >>>>>> actions to meet *** throughput goal *** > >>>>>> GC overhead (%) > >>>>>> Young generation: 2.01 (attempted to grow) > >>>>>> Tenured generation: 0.54 (no change) > >>>>>> Tenuring threshold: (attempted to decrease to balance GC > >>>>>> costs) = 1 > >>>>>> 2011-05-27T11:35:19.022-0400: 148.632: [Full GC [PSYoungGen: > >>>>>> 0K->0K(57856K)] [PSOldGen: 62518K->52158K(174784K)] > >>>>>> 62518K->52158K(232640K) [PSPermGen: 27342K->26866K(61056K)], > >>>>>> 0.3614330 secs] [Times: user=0.32 sys=0.02, real=0.37 secs] > >>>>>> First Full GC looks normal as PSOldGen is (174784-169093)K = > >>>>>> 5691K from being full. Nothing happening in Perm. > >>>>>> The second is where things start to get weird. I don't see why > >>>>>> that GC was called. Stranger still, it's ~800ms *after* the full > >>>>>> gc and yet no application thread allocated any memory out of > >>>>>> young gen. > >>>>>> for some reason that "failed" young gen collection triggers an > >>>>>> immediate Full GC. > >>>>>> > >>>>>> Bug in the collector? Did you check the bug database? > >>>>>> > >>>>>> Regards, > >>>>>> Kirk > >>>>>> > From rocketraman at fastmail.fm Fri Jun 3 07:52:35 2011 From: rocketraman at fastmail.fm (Raman Gupta) Date: Fri, 03 Jun 2011 10:52:35 -0400 Subject: Heap OOM for no apparent reason? In-Reply-To: <19944.36139.44254.445391@oracle.com> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> <4DE7C0A3.90703@fastmail.fm> <4DE7CC87.8070106@oracle.com> <4DE835AC.8060404@fastmail.fm> <4DE87725.20706@oracle.com> <19944.36139.44254.445391@oracle.com> Message-ID: <4DE8F533.9090506@fastmail.fm> On 06/03/2011 03:28 AM, John Coomes wrote: > Y. Srinivas Ramakrishna (y.s.ramakrishna at oracle.com) wrote: >> Sorry, sent previous email without addressing all of the issues. >> >> On 6/2/2011 6:15 PM, Raman Gupta wrote: >>> It would be *really* handy if there were a switch like: >>> >>> -XX:+StackTraceOnOutOfMemoryError >> >> Yes that would be handy, and probably not too difficult. >> But I wonder also if something like OnOutOfMemoryError or >> like would already get you enough info to get close to >> the problem ... (although may be because it's executed in >> a separate shell, by the time the command executes the >> process has likely gone well past the point when the problem >> occurred). > > No need to worry, the OnOutOfMemoryError commands are run while the > JVM is at a safepoint. This worked for me: > > java -XX:OnOutOfMemoryError='jstack %p' ... > > -John Excellent -- will try this later today. I did a quick search for places where OOME is caught and swallowed and found a few places within the JDK (such as direct ByteBuffer allocation), as well as a couple places in other libraries such as commons-pool and jgroups, the latter of which is used by Infinispan (though in some cases, but not all, those are logged before being swallowed). In short though, I still don't definitively know where the problem allocation is. So running jstack via OnOutOfMemoryError sounds like it is just the ticket. Cheers, Raman From y.s.ramakrishna at oracle.com Fri Jun 3 08:26:51 2011 From: y.s.ramakrishna at oracle.com (Y. Srinivas Ramakrishna) Date: Fri, 03 Jun 2011 08:26:51 -0700 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DE8F533.9090506@fastmail.fm> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> <4DE7C0A3.90703@fastmail.fm> <4DE7CC87.8070106@oracle.com> <4DE835AC.8060404@fastmail.fm> <4DE87725.20706@oracle.com> <19944.36139.44254.445391@oracle.com> <4DE8F533.9090506@fastmail.fm> Message-ID: <4DE8FD3B.3020606@oracle.com> On 6/3/2011 7:52 AM, Raman Gupta wrote: > On 06/03/2011 03:28 AM, John Coomes wrote: >> Y. Srinivas Ramakrishna (y.s.ramakrishna at oracle.com) wrote: >>> Sorry, sent previous email without addressing all of the issues. >>> >>> On 6/2/2011 6:15 PM, Raman Gupta wrote: >>>> It would be *really* handy if there were a switch like: >>>> >>>> -XX:+StackTraceOnOutOfMemoryError >>> >>> Yes that would be handy, and probably not too difficult. >>> But I wonder also if something like OnOutOfMemoryError or >>> like would already get you enough info to get close to >>> the problem ... (although may be because it's executed in >>> a separate shell, by the time the command executes the >>> process has likely gone well past the point when the problem >>> occurred). >> >> No need to worry, the OnOutOfMemoryError commands are run while the >> JVM is at a safepoint. This worked for me: >> >> java -XX:OnOutOfMemoryError='jstack %p' ... Really, are you sure? I'd assumed you spawn off a separate (i.e. asynchronous) shell process rather than waiting for it to complete while you waited in the safepoint (i.e. synchronous). It could still be that one is "lucky" and the shell happens to complete before the safepoint is exited? Anyway, a good idea to check the code to see if there is a synchronicity guarantee or one relies on plain luck to sometimes get something useful (which itself is not bad, but good to know when it is good fortune vs actual design :-) -- ramki >> >> -John > > Excellent -- will try this later today. > > I did a quick search for places where OOME is caught and swallowed and > found a few places within the JDK (such as direct ByteBuffer > allocation), as well as a couple places in other libraries such as > commons-pool and jgroups, the latter of which is used by Infinispan > (though in some cases, but not all, those are logged before being > swallowed). In short though, I still don't definitively know where the > problem allocation is. So running jstack via OnOutOfMemoryError sounds > like it is just the ticket. > > Cheers, > Raman From y.s.ramakrishna at oracle.com Fri Jun 3 08:32:40 2011 From: y.s.ramakrishna at oracle.com (Y. Srinivas Ramakrishna) Date: Fri, 03 Jun 2011 08:32:40 -0700 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DE8FD3B.3020606@oracle.com> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> <4DE7C0A3.90703@fastmail.fm> <4DE7CC87.8070106@oracle.com> <4DE835AC.8060404@fastmail.fm> <4DE87725.20706@oracle.com> <19944.36139.44254.445391@oracle.com> <4DE8F533.9090506@fastmail.fm> <4DE8FD3B.3020606@oracle.com> Message-ID: <4DE8FE98.6010901@oracle.com> By the way, it would seem that a "safepoint synchronous" OnOutOfMemoryError execution would restrict what you could do, just in case that caused a deadlock because the target (self) might need to be at a non-safepoint to react to that command.... Is there such a documented restriction on what commands can be run within OnOutOfMemeoryError (or even a flat caveat emptor)? -- ramki On 6/3/2011 8:26 AM, Y. Srinivas Ramakrishna wrote: > On 6/3/2011 7:52 AM, Raman Gupta wrote: >> On 06/03/2011 03:28 AM, John Coomes wrote: >>> Y. Srinivas Ramakrishna (y.s.ramakrishna at oracle.com) wrote: >>>> Sorry, sent previous email without addressing all of the issues. >>>> >>>> On 6/2/2011 6:15 PM, Raman Gupta wrote: >>>>> It would be *really* handy if there were a switch like: >>>>> >>>>> -XX:+StackTraceOnOutOfMemoryError >>>> >>>> Yes that would be handy, and probably not too difficult. >>>> But I wonder also if something like OnOutOfMemoryError or >>>> like would already get you enough info to get close to >>>> the problem ... (although may be because it's executed in >>>> a separate shell, by the time the command executes the >>>> process has likely gone well past the point when the problem >>>> occurred). >>> >>> No need to worry, the OnOutOfMemoryError commands are run while the >>> JVM is at a safepoint. This worked for me: >>> >>> java -XX:OnOutOfMemoryError='jstack %p' ... > > Really, are you sure? I'd assumed you spawn off a separate (i.e. asynchronous) > shell process rather than waiting for it to complete while you waited > in the safepoint (i.e. synchronous). It could still be that one is > "lucky" and the shell happens to complete before the safepoint is > exited? Anyway, a good idea to check the code to see if there is > a synchronicity guarantee or one relies on plain luck to sometimes > get something useful (which itself is not bad, but good to know > when it is good fortune vs actual design :-) > > -- ramki > >>> >>> -John >> >> Excellent -- will try this later today. >> >> I did a quick search for places where OOME is caught and swallowed and >> found a few places within the JDK (such as direct ByteBuffer >> allocation), as well as a couple places in other libraries such as >> commons-pool and jgroups, the latter of which is used by Infinispan >> (though in some cases, but not all, those are logged before being >> swallowed). In short though, I still don't definitively know where the >> problem allocation is. So running jstack via OnOutOfMemoryError sounds >> like it is just the ticket. >> >> Cheers, >> Raman > From y.s.ramakrishna at oracle.com Fri Jun 3 08:50:13 2011 From: y.s.ramakrishna at oracle.com (Y. Srinivas Ramakrishna) Date: Fri, 03 Jun 2011 08:50:13 -0700 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DE8FE98.6010901@oracle.com> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> <4DE7C0A3.90703@fastmail.fm> <4DE7CC87.8070106@oracle.com> <4DE835AC.8060404@fastmail.fm> <4DE87725.20706@oracle.com> <19944.36139.44254.445391@oracle.com> <4DE8F533.9090506@fastmail.fm> <4DE8FD3B.3020606@oracle.com> <4DE8FE98.6010901@oracle.com> Message-ID: <4DE902B5.3030401@oracle.com> John was right. I looked at the code and the process waits for the child to complete and return: the command is "safepoint-synchronous". May be all the j* commands are such that they can produce their results without needing mutator threads to run, because i also could not find any documented restrictions on the use of OnOutOfMemoryError. so, yes, looks like OnOutOfMemoryError will do what you want here, and my worries were baseless. thanks John! -- ramki On 6/3/2011 8:32 AM, Y. Srinivas Ramakrishna wrote: > By the way, it would seem that a "safepoint synchronous" > OnOutOfMemoryError execution would restrict what you could do, > just in case that caused a deadlock because the target (self) > might need to be at a non-safepoint to react to that command.... > > Is there such a documented restriction on what commands can > be run within OnOutOfMemeoryError (or even a flat caveat emptor)? > > -- ramki > > On 6/3/2011 8:26 AM, Y. Srinivas Ramakrishna wrote: >> On 6/3/2011 7:52 AM, Raman Gupta wrote: >>> On 06/03/2011 03:28 AM, John Coomes wrote: >>>> Y. Srinivas Ramakrishna (y.s.ramakrishna at oracle.com) wrote: >>>>> Sorry, sent previous email without addressing all of the issues. >>>>> >>>>> On 6/2/2011 6:15 PM, Raman Gupta wrote: >>>>>> It would be *really* handy if there were a switch like: >>>>>> >>>>>> -XX:+StackTraceOnOutOfMemoryError >>>>> >>>>> Yes that would be handy, and probably not too difficult. >>>>> But I wonder also if something like OnOutOfMemoryError or >>>>> like would already get you enough info to get close to >>>>> the problem ... (although may be because it's executed in >>>>> a separate shell, by the time the command executes the >>>>> process has likely gone well past the point when the problem >>>>> occurred). >>>> >>>> No need to worry, the OnOutOfMemoryError commands are run while the >>>> JVM is at a safepoint. This worked for me: >>>> >>>> java -XX:OnOutOfMemoryError='jstack %p' ... >> >> Really, are you sure? I'd assumed you spawn off a separate (i.e. asynchronous) >> shell process rather than waiting for it to complete while you waited >> in the safepoint (i.e. synchronous). It could still be that one is >> "lucky" and the shell happens to complete before the safepoint is >> exited? Anyway, a good idea to check the code to see if there is >> a synchronicity guarantee or one relies on plain luck to sometimes >> get something useful (which itself is not bad, but good to know >> when it is good fortune vs actual design :-) >> >> -- ramki >> >>>> >>>> -John >>> >>> Excellent -- will try this later today. >>> >>> I did a quick search for places where OOME is caught and swallowed and >>> found a few places within the JDK (such as direct ByteBuffer >>> allocation), as well as a couple places in other libraries such as >>> commons-pool and jgroups, the latter of which is used by Infinispan >>> (though in some cases, but not all, those are logged before being >>> swallowed). In short though, I still don't definitively know where the >>> problem allocation is. So running jstack via OnOutOfMemoryError sounds >>> like it is just the ticket. >>> >>> Cheers, >>> Raman >> > From jon.masamitsu at oracle.com Fri Jun 3 16:46:08 2011 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Fri, 03 Jun 2011 16:46:08 -0700 Subject: help with long par new times In-Reply-To: <593141.16056.qm@web111104.mail.gq1.yahoo.com> References: <593141.16056.qm@web111104.mail.gq1.yahoo.com> Message-ID: <4DE97240.1000701@oracle.com> Your system time is rather high on this GC. Check the other long GC's to see if there is similar high system times. 8706.691: [GC 8706.691: [ParNew: 921599K->95151K(921600K), 18.2093573 secs] 2598 628K->1818996K(4091904K), 18.2099612 secs] [Times: user=5.11 sys=33.09, real=18. 21 secs] Something other than GC is likely contributing to these long pauses. On 6/3/2011 3:17 PM, Erwin wrote: >> Hello, >> >> We performed load testing in our environment today and >> there were 3 high par new times. There were no failure in >> promotions and no full gc, the par new just took long on 7 >> occasions. The long times were: 4.1402018, 19.4847804, >> 18.2093573, 11.7687385, 3.8394909, 3.1041563 and 2.1444040. >> The load test was ran for about 3 hours. >> Any help would be appreciated. I'm attaching the >> native_stdout.log. I'm also showing the jvm args below. >> M5000 with 16GB physical memory >> Min heap - 4096 >> Mam heap - 6016 >> -server >> -Xmn1000m >> -XX:PermSize=512m >> -XX:+UseConcMarkSweepGC >> -XX:+HeapDumpOnOutOfMemoryError >> -DUseSunHttpHandler=true >> -Djavax.xml.soap.MessageFactory=weblogic.xml.saaj.MessageFactoryImpl >> -Doracle.jdbc.V8Compatible=true >> -XX:CMSInitiatingOccupancyFraction=70 >> -XX:+CMSClassUnloadingEnabled >> -verbose:gc >> -XX:+PrintGCTimeStamps >> -XX:+PrintGCDetails >> -XX:-TraceClassUnloading >> -XX:+PrintHeapAtGC >> -XX:MaxPermSize=694m >> -XX:+DisableExplicitGC >> -XX:+CMSParallelRemarkEnabled >> >> Any help would be appreciated. >> Thanks, >> Erwin >> >> >> _______________________________________________ >> hotspot-gc-use mailing list >> hotspot-gc-use at openjdk.java.net >> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110603/47eb889c/attachment.html -------------- next part -------------- _______________________________________________ hotspot-gc-use mailing list hotspot-gc-use at openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use From rocketraman at fastmail.fm Mon Jun 6 09:15:07 2011 From: rocketraman at fastmail.fm (Raman Gupta) Date: Mon, 06 Jun 2011 12:15:07 -0400 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DE902B5.3030401@oracle.com> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> <4DE7C0A3.90703@fastmail.fm> <4DE7CC87.8070106@oracle.com> <4DE835AC.8060404@fastmail.fm> <4DE87725.20706@oracle.com> <19944.36139.44254.445391@oracle.com> <4DE8F533.9090506@fastmail.fm> <4DE8FD3B.3020606@oracle.com> <4DE8FE98.6010901@oracle.com> <4DE902B5.3030401@oracle.com> Message-ID: <4DECFD0B.2000704@fastmail.fm> To close this off from the perspective of the gc-dev list... using jstack in combination with the OnOutOfMemoryError parameter was indeed extremely useful in determining the allocation point causing the OOME. The allocation point was an Object deserialization triggered by Infinispan's unmarshalling code, and the OOME was swallowed by org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle() by catching Throwable -- which only logs the Throwable if log.trace is enabled (yuck). In addition, at the point of the allocation failure the unmarshaller is trying to allocate a very large array of chars which *should* be no longer than 6, so basically it looks like there is likely a bug somewhere in the application, in JGroups, or in Infinispan. Thanks a lot to all of you for your help, as usual. Cheers, Raman Gupta On 06/03/2011 11:50 AM, Y. Srinivas Ramakrishna wrote: > John was right. I looked at the code and the process > waits for the child to complete and return: the > command is "safepoint-synchronous". May be all > the j* commands are such that they can produce their > results without needing mutator threads to run, because > i also could not find any documented restrictions on the use of > OnOutOfMemoryError. > > so, yes, looks like OnOutOfMemoryError will do what you want here, > and my worries were baseless. > > thanks John! > -- ramki > > On 6/3/2011 8:32 AM, Y. Srinivas Ramakrishna wrote: >> By the way, it would seem that a "safepoint synchronous" >> OnOutOfMemoryError execution would restrict what you could do, >> just in case that caused a deadlock because the target (self) >> might need to be at a non-safepoint to react to that command.... >> >> Is there such a documented restriction on what commands can >> be run within OnOutOfMemeoryError (or even a flat caveat emptor)? >> >> -- ramki >> >> On 6/3/2011 8:26 AM, Y. Srinivas Ramakrishna wrote: >>> On 6/3/2011 7:52 AM, Raman Gupta wrote: >>>> On 06/03/2011 03:28 AM, John Coomes wrote: >>>>> Y. Srinivas Ramakrishna (y.s.ramakrishna at oracle.com) wrote: >>>>>> Sorry, sent previous email without addressing all of the issues. >>>>>> >>>>>> On 6/2/2011 6:15 PM, Raman Gupta wrote: >>>>>>> It would be *really* handy if there were a switch like: >>>>>>> >>>>>>> -XX:+StackTraceOnOutOfMemoryError >>>>>> >>>>>> Yes that would be handy, and probably not too difficult. >>>>>> But I wonder also if something like OnOutOfMemoryError or >>>>>> like would already get you enough info to get close to >>>>>> the problem ... (although may be because it's executed in >>>>>> a separate shell, by the time the command executes the >>>>>> process has likely gone well past the point when the problem >>>>>> occurred). >>>>> >>>>> No need to worry, the OnOutOfMemoryError commands are run while the >>>>> JVM is at a safepoint. This worked for me: >>>>> >>>>> java -XX:OnOutOfMemoryError='jstack %p' ... >>> >>> Really, are you sure? I'd assumed you spawn off a separate (i.e. >>> asynchronous) >>> shell process rather than waiting for it to complete while you waited >>> in the safepoint (i.e. synchronous). It could still be that one is >>> "lucky" and the shell happens to complete before the safepoint is >>> exited? Anyway, a good idea to check the code to see if there is >>> a synchronicity guarantee or one relies on plain luck to sometimes >>> get something useful (which itself is not bad, but good to know >>> when it is good fortune vs actual design :-) >>> >>> -- ramki >>> >>>>> >>>>> -John >>>> >>>> Excellent -- will try this later today. >>>> >>>> I did a quick search for places where OOME is caught and swallowed >>>> and >>>> found a few places within the JDK (such as direct ByteBuffer >>>> allocation), as well as a couple places in other libraries such as >>>> commons-pool and jgroups, the latter of which is used by Infinispan >>>> (though in some cases, but not all, those are logged before being >>>> swallowed). In short though, I still don't definitively know where >>>> the >>>> problem allocation is. So running jstack via OnOutOfMemoryError >>>> sounds >>>> like it is just the ticket. >>>> >>>> Cheers, >>>> Raman >>> >> > From manik at surtani.org Mon Jun 6 09:39:37 2011 From: manik at surtani.org (Manik Surtani) Date: Mon, 6 Jun 2011 17:39:37 +0100 Subject: Heap OOM for no apparent reason? In-Reply-To: <4DECFD0B.2000704@fastmail.fm> References: <4DDFE61F.2090307@fastmail.fm> <4DE675B4.3060009@oracle.com> <8652216D-B88F-4AC2-8831-B3CE9E9C4EB2@kodewerk.com> <4DE7ADC1.1010205@fastmail.fm> <063F9D69-68A5-45B1-B741-0A62DA76E400@kodewerk.com> <4DE7C0A3.90703@fastmail.fm> <4DE7CC87.8070106@oracle.com> <4DE835AC.8060404@fastmail.fm> <4DE87725.20706@oracle.com> <19944.36139.44254.445391@oracle.com> <4DE8F533.9090506@fastmail.fm> <4DE8FD3B.3020606@oracle.com> <4DE8FE98.6010901@oracle.com> <4DE902B5.3030401@oracle.com> <4DECFD0B.2000704@fastmail.fm> Message-ID: <7BB304B1B21548CEAE86F5B6D96823C5@surtani.org> On Monday, 6 June 2011 at 17:15, Raman Gupta wrote: > To close this off from the perspective of the gc-dev list... using > jstack in combination with the OnOutOfMemoryError parameter was indeed > extremely useful in determining the allocation point causing the OOME. > > The allocation point was an Object deserialization triggered by > Infinispan's unmarshalling code, and the OOME was swallowed by > org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle() > by catching Throwable -- which only logs the Throwable if log.trace is > enabled (yuck). Yes, pretty ugly - thanks for bringing this up. https://issues.jboss.org/browse/ISPN-1155 > In addition, at the point of the allocation failure the unmarshaller > is trying to allocate a very large array of chars which *should* be no > longer than 6, so basically it looks like there is likely a bug > somewhere in the application, in JGroups, or in Infinispan. How large an array are we talking about? Could be related to https://issues.jboss.org/browse/ISPN-1102 which is fixed in 5.0.0.CR4. Cheers Manik From tony.printezis at oracle.com Tue Jun 7 11:10:12 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Tue, 07 Jun 2011 14:10:12 -0400 Subject: CRR: 7032531: G1: enhance GC logging to include more accurate eden / survivor size transitions (S/XS) Message-ID: <4DEE6984.9020904@oracle.com> Hi all, Could I have a couple of reviews for this change? http://cr.openjdk.java.net/~tonyp/7032531/webrev.0/ This enhances the G1 +PrintGCDetails output to include how much space there is in the eden and survivor regions before / after a GC. Here's an example of the output: [Eden: 18M->0B Survivors: 1024K->2048K Heap: 27M(105M)->12M(125M)] It shows that the eden had 18MB before the GC and it's empty after it (as expected!), the survivors were 1MB before the GC and grew to 2MB after the GC, the used space in the heap fell from 27MB to 12MB, and finally the heap size grew from 105MB to 125MB. Note that, currently, the eden / survivor sizes that are reported are not very accurate. They are currently calculated as region_num * region_size. It'd be better if they were calculated as the used byte total in the eden and survivor regions. But this would require a fair amount of extra changes to do (I'd rather avoid iterating over them to calculate that). We'll do that change when we revamp the region lists that keep track of the eden / survivor regions (which is currently work-in-progress). Tony From y.s.ramakrishna at oracle.com Tue Jun 7 12:31:59 2011 From: y.s.ramakrishna at oracle.com (Y. S. Ramakrishna) Date: Tue, 07 Jun 2011 12:31:59 -0700 Subject: CRR: 7032531: G1: enhance GC logging to include more accurate eden / survivor size transitions (S/XS) In-Reply-To: <4DEE6984.9020904@oracle.com> References: <4DEE6984.9020904@oracle.com> Message-ID: <4DEE7CAF.9000600@oracle.com> Hi Tony -- high level question: Does it make sense in G1 to report the "capacity" of Eden/Survivor (or Eden+Survivor) after a scavenge, or does that not make sense because perhaps it's dynamically decided based on other factors and so all that really matters is the capacity of the whole heap (reported as 125 MB in your snippet below). thanks. -- ramki On 06/07/11 11:10, Tony Printezis wrote: > Hi all, > > Could I have a couple of reviews for this change? > > http://cr.openjdk.java.net/~tonyp/7032531/webrev.0/ > > This enhances the G1 +PrintGCDetails output to include how much space > there is in the eden and survivor regions before / after a GC. Here's an > example of the output: > > [Eden: 18M->0B Survivors: 1024K->2048K Heap: 27M(105M)->12M(125M)] > > It shows that the eden had 18MB before the GC and it's empty after it > (as expected!), the survivors were 1MB before the GC and grew to 2MB > after the GC, the used space in the heap fell from 27MB to 12MB, and > finally the heap size grew from 105MB to 125MB. > > Note that, currently, the eden / survivor sizes that are reported are > not very accurate. They are currently calculated as region_num * > region_size. It'd be better if they were calculated as the used byte > total in the eden and survivor regions. But this would require a fair > amount of extra changes to do (I'd rather avoid iterating over them to > calculate that). We'll do that change when we revamp the region lists > that keep track of the eden / survivor regions (which is currently > work-in-progress). > > Tony > From tony.printezis at oracle.com Tue Jun 7 13:24:30 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Tue, 07 Jun 2011 16:24:30 -0400 Subject: CRR: 7032531: G1: enhance GC logging to include more accurate eden / survivor size transitions (S/XS) In-Reply-To: <4DEE7CAF.9000600@oracle.com> References: <4DEE6984.9020904@oracle.com> <4DEE7CAF.9000600@oracle.com> Message-ID: <4DEE88FE.8090208@oracle.com> Ramki, You're bringing a very good point, as always! Both the max eden and survivor sizes can potentially change at the end of each GC. I was going to send those resizing decision to the "ergonomic decisions" output (for which I recently opened a CR - 7050392). Including them here like the max heap size is a reasonably suggestion. Note, however, that the max heap size is actually a slightly different concept to the max eden / survivor sizes. The max heap size indicates that we have actually reserved that much space. The max eden / survivor sizes are instead a goal and we'll reserve that space on demand (and in some cases we might not be able to: imagine if suddenly the heap fills up with a humongous object). So I decided not to show those in a similar way as the max heap size. Maybe I'm being too pedantic here? Tony On 06/07/2011 03:31 PM, Y. S. Ramakrishna wrote: > Hi Tony -- high level question: Does it make sense in G1 > to report the "capacity" of Eden/Survivor (or Eden+Survivor) > after a scavenge, or does that not make sense because > perhaps it's dynamically decided based on other factors > and so all that really matters is the capacity of the whole > heap (reported as 125 MB in your snippet below). > > thanks. > -- ramki > > On 06/07/11 11:10, Tony Printezis wrote: >> Hi all, >> >> Could I have a couple of reviews for this change? >> >> http://cr.openjdk.java.net/~tonyp/7032531/webrev.0/ >> >> This enhances the G1 +PrintGCDetails output to include how much space >> there is in the eden and survivor regions before / after a GC. Here's >> an example of the output: >> >> [Eden: 18M->0B Survivors: 1024K->2048K Heap: 27M(105M)->12M(125M)] >> >> It shows that the eden had 18MB before the GC and it's empty after it >> (as expected!), the survivors were 1MB before the GC and grew to 2MB >> after the GC, the used space in the heap fell from 27MB to 12MB, and >> finally the heap size grew from 105MB to 125MB. >> >> Note that, currently, the eden / survivor sizes that are reported are >> not very accurate. They are currently calculated as region_num * >> region_size. It'd be better if they were calculated as the used byte >> total in the eden and survivor regions. But this would require a fair >> amount of extra changes to do (I'd rather avoid iterating over them >> to calculate that). We'll do that change when we revamp the region >> lists that keep track of the eden / survivor regions (which is >> currently work-in-progress). >> >> Tony >> From tony.printezis at oracle.com Tue Jun 7 13:35:48 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Tue, 07 Jun 2011 16:35:48 -0400 Subject: CRR: 7046558 G1: concurrent marking optimizations (S) In-Reply-To: <4DE748DE.5060504@oracle.com> References: <4DE62E64.1040902@oracle.com> <4DE63E8E.60305@oracle.com> <4DE748DE.5060504@oracle.com> Message-ID: <4DEE8BA4.7070903@oracle.com> Hi all, All set thanks to Stefan and a second code review from John Cuthbertson. In case anyone wants to have a look, here's the updated webrev(s): Complete: http://cr.openjdk.java.net/~tonyp/7046558/webrev.2/webrev.all/ Change 1 (closure specialization): http://cr.openjdk.java.net/~tonyp/7046558/webrev.2/webrev.0.G1CMOptSpecial/ Change 2 (method inlining): http://cr.openjdk.java.net/~tonyp/7046558/webrev.2/webrev.1.G1CMOptInline/ Change 3 (misc optimizations): http://cr.openjdk.java.net/~tonyp/7046558/webrev.2/webrev.2.G1CMOptMisc/ The only one that's changed compared to the previous version is the last change. I introduced and use the is_in_g1_heap() method in ConcurrentMark to avoid calling into the G1 heap. I also added a comment near the start of the claim_region() method which addresses an issue that John Cuthbertson brought up. Tony On 06/02/2011 04:25 AM, Tony Printezis wrote: > Stefan, > > Thanks for the code review! Good catch about the circular dependency. > I re-organized things a bit: I moved scan_object() from the > .inline.hpp to the .cpp file (it's only called from there so it should > be inlined anyway) so now I don't need to include > g1OopClosures.inline.hpp in concurrentMark.inline.hpp. > > Here's the updated webrev: > > All included: > http://cr.openjdk.java.net/~tonyp/7046558/webrev.1/webrev.all/ > > Change 1: > http://cr.openjdk.java.net/~tonyp/7046558/webrev.1/webrev.G1CMOptSpecial/ > > Change 2: > http://cr.openjdk.java.net/~tonyp/7046558/webrev.1/webrev.G1CMOptInline/ > > Change 3: > http://cr.openjdk.java.net/~tonyp/7046558/webrev.1/webrev.G1CMOptMisc/ > > Tony > > On 6/1/2011 9:28 AM, Stefan Karlsson wrote: >> Note that you have introduced a circular dependency between the >> files: concurrentMark.inline.hpp and g1OopClosures.inline.hpp. >> >> Otherwise, this looks good. >> >> StefanK >> >> On 06/01/2011 02:19 PM, Tony Printezis wrote: >>> Hi, >>> >>> Could I please have a couple of code reviews for the following change: >>> >>> http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.all/ >>> >>> It includes a few optimizations in the concurrent marking code. The >>> fact that webrev shows that almost 400 lines of code have been >>> changed is actually a bit misleading given that the changes are >>> actually relatively simple. Here's a breakdown to make your life a >>> bit easier: >>> >>> Change 1: Specialize the G1CMOopClosure to avoid the virtual call to >>> do_oop() every time we visit an oop during marking (the closure used >>> to be called CMOopClosure, I prefixed its name with G1 since it's >>> been moved out of the concurrentMark.cpp file). Note: I didn't >>> change much code here, just moved things around. >>> >>> http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptSpecial/ >>> >>> >>> Change 2 (builds on Change 1): Move methods >>> CM::deal_with_reference() and CM::push() to the .inline.hpp file, >>> without changing them, to make sure they get inlined, as they are >>> called from the fast path. You can either check line by line that >>> they are unchanged. Or you can take my word for it. :-) >>> >>> http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptInline/ >>> >>> >>> Change 3 (builds on Change 2): A few minor performance improvements >>> here and there (plus a couple of cosmetic changes to introduce some >>> missing curly brackets). >>> >>> http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptMisc/ >>> >>> I kept track of the separate patches to be able to evaluate them >>> separately so I thought I'd present the changes separately to make >>> the code review process a bit easier. I will of course push them as >>> a single changeset. >>> >>> I was not quite sure how to evaluate the performance impact of the >>> changes given that marking runs concurrently and not always in a >>> deterministic fashion (from a performance point-of-view!!!). So I >>> did the evaluation using collect and analyze. When looking at >>> time-based profiles, I see that the above changes decrease the time >>> we spend in the marking code (i.e., how long we spend in the >>> CMConcurrentMarkingTask::work(int i) method) by between 7%-10%. The >>> improvement is similar when looking at cycle counts. And I also see >>> a nice decrease in L2 cache misses. Each separate Change does >>> improve on the previous one, but the Change with the biggest impact >>> was 1. >>> >>> Tony >>> >> From y.s.ramakrishna at oracle.com Tue Jun 7 14:06:17 2011 From: y.s.ramakrishna at oracle.com (Y. S. Ramakrishna) Date: Tue, 07 Jun 2011 14:06:17 -0700 Subject: CRR: 7032531: G1: enhance GC logging to include more accurate eden / survivor size transitions (S/XS) In-Reply-To: <4DEE88FE.8090208@oracle.com> References: <4DEE6984.9020904@oracle.com> <4DEE7CAF.9000600@oracle.com> <4DEE88FE.8090208@oracle.com> Message-ID: <4DEE92C9.7060703@oracle.com> On 06/07/11 13:24, Tony Printezis wrote: > Ramki, > > You're bringing a very good point, as always! > > Both the max eden and survivor sizes can potentially change at the end > of each GC. I was going to send those resizing decision to the > "ergonomic decisions" output (for which I recently opened a CR - > 7050392). Including them here like the max heap size is a reasonably > suggestion. Note, however, that the max heap size is actually a slightly > different concept to the max eden / survivor sizes. The max heap size > indicates that we have actually reserved that much space. The max eden / > survivor sizes are instead a goal and we'll reserve that space on demand > (and in some cases we might not be able to: imagine if suddenly the heap > fills up with a humongous object). So I decided not to show those in a > similar way as the max heap size. OK, i see. > > Maybe I'm being too pedantic here? Probably not; it's best to give less info (as you've decided to) than give more and have the users confused as to what happened with that space... may be :-) FWIW, i think it's fine to provide that extra info in the ergo decisions output, as you've chosen to, -- ramki > > Tony > > On 06/07/2011 03:31 PM, Y. S. Ramakrishna wrote: >> Hi Tony -- high level question: Does it make sense in G1 >> to report the "capacity" of Eden/Survivor (or Eden+Survivor) >> after a scavenge, or does that not make sense because >> perhaps it's dynamically decided based on other factors >> and so all that really matters is the capacity of the whole >> heap (reported as 125 MB in your snippet below). >> >> thanks. >> -- ramki >> >> On 06/07/11 11:10, Tony Printezis wrote: >>> Hi all, >>> >>> Could I have a couple of reviews for this change? >>> >>> http://cr.openjdk.java.net/~tonyp/7032531/webrev.0/ >>> >>> This enhances the G1 +PrintGCDetails output to include how much space >>> there is in the eden and survivor regions before / after a GC. Here's >>> an example of the output: >>> >>> [Eden: 18M->0B Survivors: 1024K->2048K Heap: 27M(105M)->12M(125M)] >>> >>> It shows that the eden had 18MB before the GC and it's empty after it >>> (as expected!), the survivors were 1MB before the GC and grew to 2MB >>> after the GC, the used space in the heap fell from 27MB to 12MB, and >>> finally the heap size grew from 105MB to 125MB. >>> >>> Note that, currently, the eden / survivor sizes that are reported are >>> not very accurate. They are currently calculated as region_num * >>> region_size. It'd be better if they were calculated as the used byte >>> total in the eden and survivor regions. But this would require a fair >>> amount of extra changes to do (I'd rather avoid iterating over them >>> to calculate that). We'll do that change when we revamp the region >>> lists that keep track of the eden / survivor regions (which is >>> currently work-in-progress). >>> >>> Tony >>> From john.coomes at oracle.com Tue Jun 7 20:16:10 2011 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Wed, 08 Jun 2011 03:16:10 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 15 new changesets Message-ID: <20110608031716.E51A347C8C@hg.openjdk.java.net> Changeset: 789d04408ca3 Author: kvn Date: 2011-05-21 11:44 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/789d04408ca3 7045693: java/util/EnumSet/EnumSetBash.java still failing intermittently Summary: New limit for unrolled loop should be set only for zero trip guard and loop iteration test. Reviewed-by: never ! src/share/vm/opto/loopTransform.cpp Changeset: b55f5bd7ec66 Author: kvn Date: 2011-05-21 13:59 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/b55f5bd7ec66 7045506: assert(!can_reshape || !new_phi) failed: for igvn new phi should be hooked Summary: Replace the assert in PhiNode::Ideal with check to avoid transformation of new phi. Reviewed-by: never ! src/share/vm/opto/cfgnode.cpp Changeset: 7523488edce5 Author: kvn Date: 2011-05-24 12:54 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/7523488edce5 7047300: VM crashes with assert(_base == InstPtr) failed: Not an object pointer Summary: The code incorrectly used is_instptr() instead of is_oopptr() to get const_oop. Reviewed-by: never ! src/share/vm/opto/output.cpp Changeset: ccf072cdba91 Author: iveresov Date: 2011-05-24 15:30 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/ccf072cdba91 7046893: LP64 problem with double_quadword in c1_LIRAssembler_x86.cpp Summary: Fixed invalid casts in address computation Reviewed-by: kvn, never Contributed-by: thomas.salter at unisys.com ! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp Changeset: 28a9fe9534ea Author: kvn Date: 2011-05-24 20:24 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/28a9fe9534ea 7048030: is_scavengable changes causing compiler to embed more constants Summary: ciObject::can_be_constant() and should_be_constant() should use is_perm() instead of !is_scavengable() Reviewed-by: never, jrose ! src/share/vm/ci/ciObject.cpp ! src/share/vm/ci/ciObject.hpp Changeset: 7db2b9499c36 Author: never Date: 2011-05-25 16:04 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/7db2b9499c36 7046732: JSR 292 assert(result == cpce->f1()) failed: expected result for assembly code Reviewed-by: kvn, iveresov, jrose ! src/share/vm/interpreter/interpreterRuntime.cpp Changeset: c7c81f18c834 Author: kvn Date: 2011-05-25 21:17 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/c7c81f18c834 7048332: Cadd_cmpLTMask doesn't handle 64-bit tmp register properly Summary: Use ins_encode %{ %} form to encode cadd_cmpLTMask() instruction and remove unused code. Reviewed-by: never ! src/cpu/x86/vm/x86_64.ad + test/compiler/7048332/Test7048332.java Changeset: 28263a73ebfb Author: iveresov Date: 2011-05-26 13:15 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/28263a73ebfb 7047491: C1: registers saved incorrectly when calling checkcast_arraycopy stub Summary: Save and restore the argument registers around the call to checkcast_arraycopy Reviewed-by: never, roland ! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp Changeset: 5ac411b3b8fc Author: never Date: 2011-05-26 14:44 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/5ac411b3b8fc 7047961: JSR 292 MethodHandleWalk swap args doesn't handle T_LONG and T_DOUBLE properly Reviewed-by: kvn, jrose ! src/share/vm/prims/methodHandleWalk.cpp ! src/share/vm/prims/methodHandleWalk.hpp ! src/share/vm/prims/methodHandles.cpp Changeset: c76c13577460 Author: never Date: 2011-05-26 16:39 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/c76c13577460 Merge Changeset: b2cb497dec28 Author: kvn Date: 2011-05-27 12:47 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/b2cb497dec28 7047069: Array can dynamically change size when assigned to an object field Summary: Fix initialization of a newly-allocated array with arraycopy Reviewed-by: never ! src/share/vm/opto/library_call.cpp + test/compiler/7047069/Test7047069.java Changeset: 33e2b8f1d466 Author: kvn Date: 2011-05-31 10:05 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/33e2b8f1d466 6956668: misbehavior of XOR operator (^) with int Summary: optimize cmp_ne(xor(X,1),0) to cmp_eq(X,0) only for boolean values X. Reviewed-by: never ! src/share/vm/opto/subnode.cpp + test/compiler/6956668/Test6956668.java Changeset: 60b8287df30e Author: jrose Date: 2011-06-01 23:25 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/60b8287df30e 7049415: Failure of resolution of sym.reference to the c.s.s. should be wrapped in BootstrapMethodError Summary: Delegate invokedynamic linkage errors to MethodHandleNatives.raiseException. Reviewed-by: never ! src/share/vm/classfile/systemDictionary.hpp ! src/share/vm/classfile/vmSymbols.hpp ! src/share/vm/interpreter/linkResolver.cpp ! src/share/vm/prims/methodHandles.cpp ! src/share/vm/prims/methodHandles.hpp Changeset: a93146d0e4be Author: jrose Date: 2011-06-01 23:25 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/a93146d0e4be 7049410: JSR 292 old method name MethodHandle.invokeGeneric should not be accepted by the JVM Summary: change the default setting of the flag AllowInvokeGeneric to false Reviewed-by: never ! src/share/vm/runtime/globals.hpp Changeset: f153114134c8 Author: jcoomes Date: 2011-06-07 13:17 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/f153114134c8 Merge From bengt.rutisson at oracle.com Wed Jun 8 01:10:39 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 08 Jun 2011 10:10:39 +0200 Subject: CRR: 7042285 and 7045330 : fixes and improvements to the HeapRegionSeq class (S/M) In-Reply-To: <4DE6B9E8.40409@oracle.com> References: <4DE627D1.4060007@oracle.com> <4DE63899.7090504@oracle.com> <4DE6B9E8.40409@oracle.com> Message-ID: <4DEF2E7F.7080807@oracle.com> Tony, I finally got around to doing this review. Overall it looks good, thanks for cleaning this up. A couple of comments: - reuse_if_possible(HeapWord* bottom) * bottom is only used in an assert - hardly worth passing it as a parameter. * I thnink the name is difficult to understand. I would prefer to wrap this code in a method called get_heap_region() or something like that: HeapRegion* hr = _hrs.reuse_if_possible(base); if (hr == NULL) { // Create a new HeapRegion. MemRegion mr(base, high); bool is_zeroed = !_g1_max_committed.contains(base); hr = new HeapRegion(_bot_shared, mr, is_zeroed); // Add it to the HeapRegionSeq. _hrs.insert(hr); } else { assert(hr->bottom() == base, "post-condition"); } - _regions_biased * If this is an optimization that is important I think we should use it for addr_to_region() as well. I think this could be done by keeping a biased_length and not just a length. * if this optimization is not important I think we should avoid complicating the code with it. - G1_INVALID_HRS_INDEX * I agree with Stefan that it is a bit strange that this is defined in heapregion.hpp. I'll look at the following reviews and get back to you on this. * I would also prefer to have it as a "static const size_t G1_INVALID_HRS_INDEX = (size_t) -1" rather than a #define. It is more C++ like and that way it is possible to see the value in a debugger. Bengt On 2011-06-02 00:15, Tony Printezis wrote: > Stefan, > > Thanks for the code review. Please see answers inline: > > On 06/01/2011 03:03 PM, Stefan Karlsson wrote: >> Looks good. >> >> Some nits: >> >> heapRegion.hpp >> - Shouldn't G1_INVALID_HRS_INDEX be defined in heapRegionSeq.hpp? > > You probably saw in the follow-up change that I use > G1_INVALID_HRS_INDEX to initialize the _hrs_index field of the > HeapRegion class. So I thought it was pointless to put it in one file > and then move it to another one. BTW, I'm not 100% happy with the > G1_INVALID_HRS_INDEX name. Would G1_HRS_NULL_INDEX, or something > similar, be better? > >> heapRegionSeq.hpp >> - Could you move the class comment to above the class declaration? > > Done. > >> - type on line 147: early of doHeapRegion -> early if doHeapRegion > > Good catch. Fixed. > > Updated webrev here: > > http://cr.openjdk.java.net/~tonyp/7045330.7042285/webrev.1/ > > Tony > >> StefanK >> >> On 06/01/2011 01:51 PM, Tony Printezis wrote: >>> Hi, >>> >>> Could I please have a couple of code reviews for this change: >>> >>> http://cr.openjdk.java.net/~tonyp/7045330.7042285/webrev.0/ >>> >>> The associated CRs are: >>> >>> 7042285: G1: native memory leak during humongous object allocation >>> 7045330: G1: Simplify/fix the HeapRegionSeq class >>> >>> The first is the fix for the memory leak when the heap is expanded / >>> retracted aggressively that I opened for code review some time ago. >>> We are currently not reclaiming HeapRegion instances when the heap >>> is retracted, but we are creating new ones when the heap is >>> expanded. But, in all honesty, the HeapRegionSeq class is a bit of a >>> mess so I decided to expand the fix, cleanup that class, and >>> introduce a couple of small performance improvements too (all these >>> are covered by the second CR). Here's a list of the improvements >>> (copied from the CR): >>> >>> a) replace the _regions growable array with a standard C array (as >>> we do not grow or shrink it so there's no point in using a growable >>> array) >>> b) avoid de-allocating / re-allocating HeapRegion instances when the >>> heap shrinks / grows (fix for 7042285) >>> c) introduce fast method to map address to HeapRegion via a "biased" >>> array pointer >>> d) embed the _hrs object in G1CollectedHeap, instead of pointing to >>> it via an indirection >>> e) assume that all the regions added to the HeapRegionSeq instance >>> are in a contiguous address space >>> f) replace int's with size_t's for region indexes (part of 6804436) >>> g) remove unnecessary / unused methods >>> h) rename a couple of fields to give them a more reasonable name >>> (_alloc_search_start and _seq_bottom) >>> i) fix iterate_from() to do the right thing and not always start >>> from index 0 irrespective of the region passed to it >>> j) add a verification method to check the HeapRegionSeq assumptions >>> k) always call the wrappers for _hrs.iterate(), _hrs_length(), and >>> _hrs.at() from G1CollectedHeap, not those methods directly. >>> >>> In terms of volume of changes, the HeapRegionSeq class has been >>> rewritten heavily. The rest of the changes should be straightforward >>> (there are changes in G1CollectedHeap that reflect the slightly >>> different API that HeapRegionSeq now exposes). >>> >>> Performance-wise, I see a again modest (~2% or maybe a bit more) >>> improvement in GC times on a couple of platforms. >>> >>> Tony >>> >> From bengt.rutisson at oracle.com Wed Jun 8 01:10:47 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 08 Jun 2011 10:10:47 +0200 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DE6BAB1.7090405@oracle.com> References: <4DE62ADD.1040601@oracle.com> <4DE63A77.2030104@oracle.com> <4DE6BAB1.7090405@oracle.com> Message-ID: <4DEF2E87.80508@oracle.com> Tony, I had a look at this as well since it is connected to the previous review: "CRR: 7042285 and 7045330 : fixes and improvements to the HeapRegionSeq class (S/M) ". It looks good. Two things, though: * I think it is very closely connected to the previous review. So, close that I in fact think they should be reviewed and pushed together. I don't really see why they are separate changes. * I still think it is a bit strange that G1_INVALID_HRS_INDEX is defined in heapRegion.hpp. I would prefer to keep it in heapRegionSeq. The only use in heapRegion is in the constructor. This constructor is only called from two places: - G1CollectedHeap::expand() Here we actually know the index we want and could pass that to the constructor. No need to first set it to an invalid value. I think this would be even cleaner with the get_heap_region() method that I suggested in the previous review. - G1CollectedHeap::initialize() Here we know about heapRegionSeq and could ask it for the invalid value and pass that to the constructor. Bengt On 2011-06-02 00:18, Tony Printezis wrote: > Stefan and John, > > Thanks! All set with this one too. > > Tony > > On 06/01/2011 03:11 PM, Stefan Karlsson wrote: >> Looks good. >> >> StefanK >> >> On 06/01/2011 02:04 PM, Tony Printezis wrote: >>> Hi, >>> >>> Could I please have a couple of code reviews for this simple change: >>> >>> http://cr.openjdk.java.net/~tonyp/6804436/webrev.0/ >>> >>> Some of the int -> size_t changes were done as part of the >>> HeapRegionSeq cleanup (see the separate code review request for >>> 7042285 and 7045330 that I just sent out). This change does the rest >>> (specifically: the changes related to the HeapRegion class). >>> >>> Note: I know that the webrev index says that some files have more >>> than one change applied to them. This is incorrect and a side-effect >>> of generating the webrev from a workspace with multiple patches >>> stacked up. The diffs are actually correct (I checked). >>> >>> Tony >>> >> From bengt.rutisson at oracle.com Wed Jun 8 01:39:29 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 08 Jun 2011 10:39:29 +0200 Subject: CRR: 7032531: G1: enhance GC logging to include more accurate eden / survivor size transitions (S/XS) In-Reply-To: <4DEE6984.9020904@oracle.com> References: <4DEE6984.9020904@oracle.com> Message-ID: <4DEF3541.8010304@oracle.com> Tony, Looks good! I hope you get this in before I pick up the work on the young space sizing. I will be useful to have this logging for that. Thanks, Bengt On 2011-06-07 20:10, Tony Printezis wrote: > Hi all, > > Could I have a couple of reviews for this change? > > http://cr.openjdk.java.net/~tonyp/7032531/webrev.0/ > > This enhances the G1 +PrintGCDetails output to include how much space > there is in the eden and survivor regions before / after a GC. Here's > an example of the output: > > [Eden: 18M->0B Survivors: 1024K->2048K Heap: 27M(105M)->12M(125M)] > > It shows that the eden had 18MB before the GC and it's empty after it > (as expected!), the survivors were 1MB before the GC and grew to 2MB > after the GC, the used space in the heap fell from 27MB to 12MB, and > finally the heap size grew from 105MB to 125MB. > > Note that, currently, the eden / survivor sizes that are reported are > not very accurate. They are currently calculated as region_num * > region_size. It'd be better if they were calculated as the used byte > total in the eden and survivor regions. But this would require a fair > amount of extra changes to do (I'd rather avoid iterating over them to > calculate that). We'll do that change when we revamp the region lists > that keep track of the eden / survivor regions (which is currently > work-in-progress). > > Tony > From poonam.bajaj at oracle.com Wed Jun 8 02:14:47 2011 From: poonam.bajaj at oracle.com (Poonam Bajaj) Date: Wed, 08 Jun 2011 14:44:47 +0530 Subject: CRR: 7032531: G1: enhance GC logging to include more accurate eden / survivor size transitions (S/XS) In-Reply-To: <4DEE6984.9020904@oracle.com> References: <4DEE6984.9020904@oracle.com> Message-ID: <4DEF3D87.6000100@oracle.com> Hi Tony, On 6/7/2011 11:40 PM, Tony Printezis wrote: > Hi all, > > Could I have a couple of reviews for this change? > > http://cr.openjdk.java.net/~tonyp/7032531/webrev.0/ > > This enhances the G1 +PrintGCDetails output to include how much space > there is in the eden and survivor regions before / after a GC. Here's > an example of the output: > > [Eden: 18M->0B Survivors: 1024K->2048K Heap: 27M(105M)->12M(125M)] > > It shows that the eden had 18MB before the GC and it's empty after it > (as expected!), the survivors were 1MB before the GC and grew to 2MB > after the GC, the used space in the heap fell from 27MB to 12MB, and > finally the heap size grew from 105MB to 125MB. > The heap size ( grew from 105K to 125K) here means the total size of all the regions in G1 heap (including young regions) ? Thanks, Poonam > Note that, currently, the eden / survivor sizes that are reported are > not very accurate. They are currently calculated as region_num * > region_size. It'd be better if they were calculated as the used byte > total in the eden and survivor regions. But this would require a fair > amount of extra changes to do (I'd rather avoid iterating over them to > calculate that). We'll do that change when we revamp the region lists > that keep track of the eden / survivor regions (which is currently > work-in-progress). > > Tony > -- Best regards, Poonam Sun, an Oracle company Sun, an Oracle Company Poonam Bajaj | Staff Engineer Phone: +66937451 | Mobile: +9844511366 JVM Sustaining Engineering | Bangalore Green Oracle Oracle is committed to developing practices and products that help protect the environment -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110608/325e59f6/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: sun-oracle-logo.gif Type: image/gif Size: 2088 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110608/325e59f6/attachment.gif -------------- next part -------------- A non-text attachment was scrubbed... Name: green-for-email-sig_0.gif Type: image/gif Size: 356 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110608/325e59f6/attachment-0001.gif From tony.printezis at oracle.com Wed Jun 8 07:01:11 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 08 Jun 2011 10:01:11 -0400 Subject: CRR: 7032531: G1: enhance GC logging to include more accurate eden / survivor size transitions (S/XS) In-Reply-To: <4DEF3541.8010304@oracle.com> References: <4DEE6984.9020904@oracle.com> <4DEF3541.8010304@oracle.com> Message-ID: <4DEF80A7.7050000@oracle.com> Thanks Bengt, I'll try to push asap! On 06/08/2011 04:39 AM, Bengt Rutisson wrote: > > Tony, > > Looks good! I hope you get this in before I pick up the work on the > young space sizing. I will be useful to have this logging for that. > > Thanks, > Bengt > > > On 2011-06-07 20:10, Tony Printezis wrote: >> Hi all, >> >> Could I have a couple of reviews for this change? >> >> http://cr.openjdk.java.net/~tonyp/7032531/webrev.0/ >> >> This enhances the G1 +PrintGCDetails output to include how much space >> there is in the eden and survivor regions before / after a GC. Here's >> an example of the output: >> >> [Eden: 18M->0B Survivors: 1024K->2048K Heap: 27M(105M)->12M(125M)] >> >> It shows that the eden had 18MB before the GC and it's empty after it >> (as expected!), the survivors were 1MB before the GC and grew to 2MB >> after the GC, the used space in the heap fell from 27MB to 12MB, and >> finally the heap size grew from 105MB to 125MB. >> >> Note that, currently, the eden / survivor sizes that are reported are >> not very accurate. They are currently calculated as region_num * >> region_size. It'd be better if they were calculated as the used byte >> total in the eden and survivor regions. But this would require a fair >> amount of extra changes to do (I'd rather avoid iterating over them >> to calculate that). We'll do that change when we revamp the region >> lists that keep track of the eden / survivor regions (which is >> currently work-in-progress). >> >> Tony >> > From tony.printezis at oracle.com Wed Jun 8 07:01:51 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 08 Jun 2011 10:01:51 -0400 Subject: CRR: 7032531: G1: enhance GC logging to include more accurate eden / survivor size transitions (S/XS) In-Reply-To: <4DEF3D87.6000100@oracle.com> References: <4DEE6984.9020904@oracle.com> <4DEF3D87.6000100@oracle.com> Message-ID: <4DEF80CF.3000808@oracle.com> Poonam, Yes, the whole heap can grow at the end of a young GC in G1. Tony On 06/08/2011 05:14 AM, Poonam Bajaj wrote: > Hi Tony, > > On 6/7/2011 11:40 PM, Tony Printezis wrote: >> Hi all, >> >> Could I have a couple of reviews for this change? >> >> http://cr.openjdk.java.net/~tonyp/7032531/webrev.0/ >> >> This enhances the G1 +PrintGCDetails output to include how much space >> there is in the eden and survivor regions before / after a GC. Here's >> an example of the output: >> >> [Eden: 18M->0B Survivors: 1024K->2048K Heap: 27M(105M)->12M(125M)] >> >> It shows that the eden had 18MB before the GC and it's empty after it >> (as expected!), the survivors were 1MB before the GC and grew to 2MB >> after the GC, the used space in the heap fell from 27MB to 12MB, and >> finally the heap size grew from 105MB to 125MB. >> > The heap size ( grew from 105K to 125K) here means the total size of > all the regions in G1 heap (including young regions) ? > > Thanks, > Poonam > >> Note that, currently, the eden / survivor sizes that are reported are >> not very accurate. They are currently calculated as region_num * >> region_size. It'd be better if they were calculated as the used byte >> total in the eden and survivor regions. But this would require a fair >> amount of extra changes to do (I'd rather avoid iterating over them >> to calculate that). We'll do that change when we revamp the region >> lists that keep track of the eden / survivor regions (which is >> currently work-in-progress). >> >> Tony >> > > -- > Best regards, Poonam > > Sun, an Oracle company > Sun, an Oracle Company > Poonam Bajaj | Staff Engineer > Phone: +66937451 | Mobile: +9844511366 > JVM Sustaining Engineering > | Bangalore > Green Oracle Oracle is committed to > developing practices and products that help protect the environment > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110608/64f542a3/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 2088 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110608/64f542a3/attachment-0002.gif -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 356 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110608/64f542a3/attachment-0003.gif From tony.printezis at oracle.com Wed Jun 8 07:29:23 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 08 Jun 2011 10:29:23 -0400 Subject: CRR: 7046558 G1: concurrent marking optimizations (S) In-Reply-To: <4DEE8BA4.7070903@oracle.com> References: <4DE62E64.1040902@oracle.com> <4DE63E8E.60305@oracle.com> <4DE748DE.5060504@oracle.com> <4DEE8BA4.7070903@oracle.com> Message-ID: <4DEF8743.2080002@oracle.com> Hi all, While testing these changes last night I hit a rare failure (the _heap_end field would sometimes become inconsistent with what's in G1CollectedHeap and is_in_g1_heap() would return the wrong result; thankfully the assert I added caught that). So, I'm going to back out of the introduction of the is_in_g1_heap() method and revert back to using G1CH::is_in_g1_reserved(). We can look at maybe re-introducing it in the future. Here are the latest (and hopefully final!) webrevs: Change 1: http://cr.openjdk.java.net/~tonyp/7046558/webrev.3/webrev.0.G1CMOptSpecial/ Change 2: http://cr.openjdk.java.net/~tonyp/7046558/webrev.3/webrev.1.G1CMOptInline/ Change 3: http://cr.openjdk.java.net/~tonyp/7046558/webrev.3/webrev.2.G1CMOptMisc/ Complete: http://cr.openjdk.java.net/~tonyp/7046558/webrev.3/webrev.all/ Only Change 3 is affected by this. Testing ran successfully for 10+ hours. Let me know if you're planning to look at it, otherwise I'm going to push it at the first available opportunity. Tony On 06/07/2011 04:35 PM, Tony Printezis wrote: > Hi all, > > All set thanks to Stefan and a second code review from John > Cuthbertson. In case anyone wants to have a look, here's the updated > webrev(s): > > Complete: > http://cr.openjdk.java.net/~tonyp/7046558/webrev.2/webrev.all/ > > Change 1 (closure specialization): > http://cr.openjdk.java.net/~tonyp/7046558/webrev.2/webrev.0.G1CMOptSpecial/ > > > Change 2 (method inlining): > http://cr.openjdk.java.net/~tonyp/7046558/webrev.2/webrev.1.G1CMOptInline/ > > > Change 3 (misc optimizations): > http://cr.openjdk.java.net/~tonyp/7046558/webrev.2/webrev.2.G1CMOptMisc/ > > The only one that's changed compared to the previous version is the > last change. I introduced and use the is_in_g1_heap() method in > ConcurrentMark to avoid calling into the G1 heap. I also added a > comment near the start of the claim_region() method which addresses an > issue that John Cuthbertson brought up. > > Tony > > On 06/02/2011 04:25 AM, Tony Printezis wrote: >> Stefan, >> >> Thanks for the code review! Good catch about the circular dependency. >> I re-organized things a bit: I moved scan_object() from the >> .inline.hpp to the .cpp file (it's only called from there so it >> should be inlined anyway) so now I don't need to include >> g1OopClosures.inline.hpp in concurrentMark.inline.hpp. >> >> Here's the updated webrev: >> >> All included: >> http://cr.openjdk.java.net/~tonyp/7046558/webrev.1/webrev.all/ >> >> Change 1: >> http://cr.openjdk.java.net/~tonyp/7046558/webrev.1/webrev.G1CMOptSpecial/ >> >> >> Change 2: >> http://cr.openjdk.java.net/~tonyp/7046558/webrev.1/webrev.G1CMOptInline/ >> >> Change 3: >> http://cr.openjdk.java.net/~tonyp/7046558/webrev.1/webrev.G1CMOptMisc/ >> >> Tony >> >> On 6/1/2011 9:28 AM, Stefan Karlsson wrote: >>> Note that you have introduced a circular dependency between the >>> files: concurrentMark.inline.hpp and g1OopClosures.inline.hpp. >>> >>> Otherwise, this looks good. >>> >>> StefanK >>> >>> On 06/01/2011 02:19 PM, Tony Printezis wrote: >>>> Hi, >>>> >>>> Could I please have a couple of code reviews for the following change: >>>> >>>> http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.all/ >>>> >>>> It includes a few optimizations in the concurrent marking code. The >>>> fact that webrev shows that almost 400 lines of code have been >>>> changed is actually a bit misleading given that the changes are >>>> actually relatively simple. Here's a breakdown to make your life a >>>> bit easier: >>>> >>>> Change 1: Specialize the G1CMOopClosure to avoid the virtual call >>>> to do_oop() every time we visit an oop during marking (the closure >>>> used to be called CMOopClosure, I prefixed its name with G1 since >>>> it's been moved out of the concurrentMark.cpp file). Note: I didn't >>>> change much code here, just moved things around. >>>> >>>> http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptSpecial/ >>>> >>>> >>>> Change 2 (builds on Change 1): Move methods >>>> CM::deal_with_reference() and CM::push() to the .inline.hpp file, >>>> without changing them, to make sure they get inlined, as they are >>>> called from the fast path. You can either check line by line that >>>> they are unchanged. Or you can take my word for it. :-) >>>> >>>> http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptInline/ >>>> >>>> >>>> Change 3 (builds on Change 2): A few minor performance improvements >>>> here and there (plus a couple of cosmetic changes to introduce some >>>> missing curly brackets). >>>> >>>> http://cr.openjdk.java.net/~tonyp/7046558/webrev.0/webrev.G1CMOptMisc/ >>>> >>>> I kept track of the separate patches to be able to evaluate them >>>> separately so I thought I'd present the changes separately to make >>>> the code review process a bit easier. I will of course push them as >>>> a single changeset. >>>> >>>> I was not quite sure how to evaluate the performance impact of the >>>> changes given that marking runs concurrently and not always in a >>>> deterministic fashion (from a performance point-of-view!!!). So I >>>> did the evaluation using collect and analyze. When looking at >>>> time-based profiles, I see that the above changes decrease the time >>>> we spend in the marking code (i.e., how long we spend in the >>>> CMConcurrentMarkingTask::work(int i) method) by between 7%-10%. The >>>> improvement is similar when looking at cycle counts. And I also see >>>> a nice decrease in L2 cache misses. Each separate Change does >>>> improve on the previous one, but the Change with the biggest impact >>>> was 1. >>>> >>>> Tony >>>> >>> From tony.printezis at oracle.com Wed Jun 8 07:40:04 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 08 Jun 2011 10:40:04 -0400 Subject: CRR: 7042285 and 7045330 : fixes and improvements to the HeapRegionSeq class (S/M) In-Reply-To: <4DEF2E7F.7080807@oracle.com> References: <4DE627D1.4060007@oracle.com> <4DE63899.7090504@oracle.com> <4DE6B9E8.40409@oracle.com> <4DEF2E7F.7080807@oracle.com> Message-ID: <4DEF89C4.7070104@oracle.com> Hi Bengt, Thanks for the code review. Please see inline. On 06/08/2011 04:10 AM, Bengt Rutisson wrote: > > Tony, > > I finally got around to doing this review. Overall it looks good, > thanks for cleaning this up. Yeah, and sorry for the slightly lengthy change. I think the new version is much more understandable. > A couple of comments: > > - reuse_if_possible(HeapWord* bottom) > * bottom is only used in an assert - hardly worth passing it as a > parameter. I did a few different versions of that code which is how that parameter ended up there but ultimately not being "really" used. I have to say that, normally, passing a parameter for an assert in a non-critical part of the code would actually be acceptable (IMHO). Having said that, in this case the caller can do the assert after getting back the result so, you're right, I'll remove it. > * I thnink the name is difficult to understand. I would prefer to > wrap this code in a method called get_heap_region() or something like > that: > > HeapRegion* hr = _hrs.reuse_if_possible(base); > if (hr == NULL) { > // Create a new HeapRegion. > MemRegion mr(base, high); > bool is_zeroed = !_g1_max_committed.contains(base); > hr = new HeapRegion(_bot_shared, mr, is_zeroed); > > // Add it to the HeapRegionSeq. > _hrs.insert(hr); > } else { > assert(hr->bottom() == base, "post-condition"); > } Yeah, you're right the name was not the best. How the following? HeapRegion* hr = _hrs.expand_to_next_region(); if (hr == NULL) { // Create a new HeapRegion. MemRegion mr(base, high); bool is_zeroed = !_g1_max_committed.contains(base); hr = new HeapRegion(_bot_shared, mr, is_zeroed); _hrs.expand_to_next_region(hr); } else { assert(hr->bottom() == base, "post-condition"); } expand_to_next_region() is more descriptive. Also note that renamed insert(HR *hr) as expand_to_next_region(HR* hr) as the two are in a way similar (and opposite to shrink). > - _regions_biased > * If this is an optimization that is important It was one of the things that was showing up during the profiling we did a few weeks ago. > I think we should use it for addr_to_region() as well. I think this > could be done by keeping a biased_length and not just a length. I was going to write a lengthy paragraph about the differences in assumptions between addr_to_region() and addr_to_region_unsafe(). But while looking at it I think we can do a bit better than the current version given that, if addr <= heap_bottom, it can only be NULL. So, let me rework this code a bit and let me know what you think of the new version. > * if this optimization is not important I think we should avoid > complicating the code with it. I agree, but it looked as if it gave a small but measurable improvement. > - G1_INVALID_HRS_INDEX > * I agree with Stefan that it is a bit strange that this is defined > in heapregion.hpp. I'll look at the following reviews and get back to > you on this. OK. BTW, would you prefer G1_NULL_HRS_INDEX instead of INVALID? > * I would also prefer to have it as a "static const size_t > G1_INVALID_HRS_INDEX = (size_t) -1" rather than a #define. It is more > C++ like and that way it is possible to see the value in a debugger. In HotSpot we generally use macros for constants. Maybe, I like the fact that they are guaranteed to be treated as constants by the C++ compilers. :-) More on the next e-mail... Tony > Bengt > > On 2011-06-02 00:15, Tony Printezis wrote: >> Stefan, >> >> Thanks for the code review. Please see answers inline: >> >> On 06/01/2011 03:03 PM, Stefan Karlsson wrote: >>> Looks good. >>> >>> Some nits: >>> >>> heapRegion.hpp >>> - Shouldn't G1_INVALID_HRS_INDEX be defined in heapRegionSeq.hpp? >> >> You probably saw in the follow-up change that I use >> G1_INVALID_HRS_INDEX to initialize the _hrs_index field of the >> HeapRegion class. So I thought it was pointless to put it in one file >> and then move it to another one. BTW, I'm not 100% happy with the >> G1_INVALID_HRS_INDEX name. Would G1_HRS_NULL_INDEX, or something >> similar, be better? >> >>> heapRegionSeq.hpp >>> - Could you move the class comment to above the class declaration? >> >> Done. >> >>> - type on line 147: early of doHeapRegion -> early if doHeapRegion >> >> Good catch. Fixed. >> >> Updated webrev here: >> >> http://cr.openjdk.java.net/~tonyp/7045330.7042285/webrev.1/ >> >> Tony >> >>> StefanK >>> >>> On 06/01/2011 01:51 PM, Tony Printezis wrote: >>>> Hi, >>>> >>>> Could I please have a couple of code reviews for this change: >>>> >>>> http://cr.openjdk.java.net/~tonyp/7045330.7042285/webrev.0/ >>>> >>>> The associated CRs are: >>>> >>>> 7042285: G1: native memory leak during humongous object allocation >>>> 7045330: G1: Simplify/fix the HeapRegionSeq class >>>> >>>> The first is the fix for the memory leak when the heap is expanded >>>> / retracted aggressively that I opened for code review some time >>>> ago. We are currently not reclaiming HeapRegion instances when the >>>> heap is retracted, but we are creating new ones when the heap is >>>> expanded. But, in all honesty, the HeapRegionSeq class is a bit of >>>> a mess so I decided to expand the fix, cleanup that class, and >>>> introduce a couple of small performance improvements too (all these >>>> are covered by the second CR). Here's a list of the improvements >>>> (copied from the CR): >>>> >>>> a) replace the _regions growable array with a standard C array (as >>>> we do not grow or shrink it so there's no point in using a growable >>>> array) >>>> b) avoid de-allocating / re-allocating HeapRegion instances when >>>> the heap shrinks / grows (fix for 7042285) >>>> c) introduce fast method to map address to HeapRegion via a >>>> "biased" array pointer >>>> d) embed the _hrs object in G1CollectedHeap, instead of pointing to >>>> it via an indirection >>>> e) assume that all the regions added to the HeapRegionSeq instance >>>> are in a contiguous address space >>>> f) replace int's with size_t's for region indexes (part of 6804436) >>>> g) remove unnecessary / unused methods >>>> h) rename a couple of fields to give them a more reasonable name >>>> (_alloc_search_start and _seq_bottom) >>>> i) fix iterate_from() to do the right thing and not always start >>>> from index 0 irrespective of the region passed to it >>>> j) add a verification method to check the HeapRegionSeq assumptions >>>> k) always call the wrappers for _hrs.iterate(), _hrs_length(), and >>>> _hrs.at() from G1CollectedHeap, not those methods directly. >>>> >>>> In terms of volume of changes, the HeapRegionSeq class has been >>>> rewritten heavily. The rest of the changes should be >>>> straightforward (there are changes in G1CollectedHeap that reflect >>>> the slightly different API that HeapRegionSeq now exposes). >>>> >>>> Performance-wise, I see a again modest (~2% or maybe a bit more) >>>> improvement in GC times on a couple of platforms. >>>> >>>> Tony >>>> >>> > From tony.printezis at oracle.com Wed Jun 8 07:43:20 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 08 Jun 2011 10:43:20 -0400 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DEF2E87.80508@oracle.com> References: <4DE62ADD.1040601@oracle.com> <4DE63A77.2030104@oracle.com> <4DE6BAB1.7090405@oracle.com> <4DEF2E87.80508@oracle.com> Message-ID: <4DEF8A88.5020704@oracle.com> On 06/08/2011 04:10 AM, Bengt Rutisson wrote: > > Tony, > > I had a look at this as well since it is connected to the previous > review: "CRR: 7042285 and 7045330 : fixes and improvements to the > HeapRegionSeq class (S/M) ". > > It looks good. > > Two things, though: > > * I think it is very closely connected to the previous review. So, > close that I in fact think they should be reviewed and pushed > together. I don't really see why they are separate changes. Yeah, you're right. I did a bunch of the int -> size_t changes as part of the first changeset (as I didn't think there was much point in rewriting half the HeapRegionSeq class using the wrong type and fixing it later). And, when I went it to do the rest it turned out there was not that many additional changes. When I push I'll fold both changesets into one and tag it with all three CRs. > * I still think it is a bit strange that G1_INVALID_HRS_INDEX is > defined in heapRegion.hpp. I would prefer to keep it in heapRegionSeq. > The only use in heapRegion is in the constructor. This constructor is > only called from two places: > - G1CollectedHeap::expand() > Here we actually know the index we want and could pass that to > the constructor. No need to first set it to an invalid value. I think > this would be even cleaner with the get_heap_region() method that I > suggested in the previous review. > - G1CollectedHeap::initialize() > Here we know about heapRegionSeq and could ask it for the > invalid value and pass that to the constructor. Yeah, I was not crazy about having another "invalid" index. But maybe it's worth it. OK, I'll change that too and I'll post the new set of changes in a bit. Tony > Bengt > > On 2011-06-02 00:18, Tony Printezis wrote: >> Stefan and John, >> >> Thanks! All set with this one too. >> >> Tony >> >> On 06/01/2011 03:11 PM, Stefan Karlsson wrote: >>> Looks good. >>> >>> StefanK >>> >>> On 06/01/2011 02:04 PM, Tony Printezis wrote: >>>> Hi, >>>> >>>> Could I please have a couple of code reviews for this simple change: >>>> >>>> http://cr.openjdk.java.net/~tonyp/6804436/webrev.0/ >>>> >>>> Some of the int -> size_t changes were done as part of the >>>> HeapRegionSeq cleanup (see the separate code review request for >>>> 7042285 and 7045330 that I just sent out). This change does the >>>> rest (specifically: the changes related to the HeapRegion class). >>>> >>>> Note: I know that the webrev index says that some files have more >>>> than one change applied to them. This is incorrect and a >>>> side-effect of generating the webrev from a workspace with multiple >>>> patches stacked up. The diffs are actually correct (I checked). >>>> >>>> Tony >>>> >>> > From tony.printezis at oracle.com Wed Jun 8 09:57:31 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 08 Jun 2011 12:57:31 -0400 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DEF8A88.5020704@oracle.com> References: <4DE62ADD.1040601@oracle.com> <4DE63A77.2030104@oracle.com> <4DE6BAB1.7090405@oracle.com> <4DEF2E87.80508@oracle.com> <4DEF8A88.5020704@oracle.com> Message-ID: <4DEFA9FB.8060705@oracle.com> Bengt, Here's the new webrev that includes all the changes (I'll push them as a single changeset): http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.0/webrev.all/ And here are my latest the changes wrt to the version you reviewed: http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.0/webrev.2.G1HRSLatest/ Let me know if you think I haven't addressed your concerns. Tony On 06/08/2011 10:43 AM, Tony Printezis wrote: > > > On 06/08/2011 04:10 AM, Bengt Rutisson wrote: >> >> Tony, >> >> I had a look at this as well since it is connected to the previous >> review: "CRR: 7042285 and 7045330 : fixes and improvements to the >> HeapRegionSeq class (S/M) ". >> >> It looks good. >> >> Two things, though: >> >> * I think it is very closely connected to the previous review. So, >> close that I in fact think they should be reviewed and pushed >> together. I don't really see why they are separate changes. > > Yeah, you're right. I did a bunch of the int -> size_t changes as part > of the first changeset (as I didn't think there was much point in > rewriting half the HeapRegionSeq class using the wrong type and fixing > it later). And, when I went it to do the rest it turned out there was > not that many additional changes. When I push I'll fold both > changesets into one and tag it with all three CRs. > >> * I still think it is a bit strange that G1_INVALID_HRS_INDEX is >> defined in heapRegion.hpp. I would prefer to keep it in >> heapRegionSeq. The only use in heapRegion is in the constructor. This >> constructor is only called from two places: >> - G1CollectedHeap::expand() >> Here we actually know the index we want and could pass that to >> the constructor. No need to first set it to an invalid value. I think >> this would be even cleaner with the get_heap_region() method that I >> suggested in the previous review. >> - G1CollectedHeap::initialize() >> Here we know about heapRegionSeq and could ask it for the >> invalid value and pass that to the constructor. > > Yeah, I was not crazy about having another "invalid" index. But maybe > it's worth it. OK, I'll change that too and I'll post the new set of > changes in a bit. > > Tony > >> Bengt >> >> On 2011-06-02 00:18, Tony Printezis wrote: >>> Stefan and John, >>> >>> Thanks! All set with this one too. >>> >>> Tony >>> >>> On 06/01/2011 03:11 PM, Stefan Karlsson wrote: >>>> Looks good. >>>> >>>> StefanK >>>> >>>> On 06/01/2011 02:04 PM, Tony Printezis wrote: >>>>> Hi, >>>>> >>>>> Could I please have a couple of code reviews for this simple change: >>>>> >>>>> http://cr.openjdk.java.net/~tonyp/6804436/webrev.0/ >>>>> >>>>> Some of the int -> size_t changes were done as part of the >>>>> HeapRegionSeq cleanup (see the separate code review request for >>>>> 7042285 and 7045330 that I just sent out). This change does the >>>>> rest (specifically: the changes related to the HeapRegion class). >>>>> >>>>> Note: I know that the webrev index says that some files have more >>>>> than one change applied to them. This is incorrect and a >>>>> side-effect of generating the webrev from a workspace with >>>>> multiple patches stacked up. The diffs are actually correct (I >>>>> checked). >>>>> >>>>> Tony >>>>> >>>> >> From y.s.ramakrishna at oracle.com Wed Jun 8 11:45:19 2011 From: y.s.ramakrishna at oracle.com (y.s.ramakrishna at oracle.com) Date: Wed, 08 Jun 2011 18:45:19 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 3 new changesets Message-ID: <20110608184524.EB57347CF1@hg.openjdk.java.net> Changeset: 1e3493ac2d11 Author: ysr Date: 2011-05-27 10:23 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/1e3493ac2d11 7048342: CMS: eob == _limit || fc->isFree() failed: Only a free chunk should allow us to cross over the limit Summary: The freeness bit was being cleared in debug code when it shouldn't have been. Also removed unused FreeChunk methods linkAfterNonNull and clearPrev. Reviewed-by: brutisso ! src/share/vm/gc_implementation/concurrentMarkSweep/compactibleFreeListSpace.cpp ! src/share/vm/gc_implementation/concurrentMarkSweep/freeChunk.hpp Changeset: 5c0a3c1858b1 Author: ysr Date: 2011-06-02 10:23 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/5c0a3c1858b1 7048782: CMS: assert(last_chunk_index_to_check<= last_chunk_index) failed: parCardTableModRefBS.cpp:359 Summary: The LNC array is sized before the start of a scavenge, while the heap may expand during a scavenge. With CMS, the last block of an arbitrary suffice of the LNC array may expand due to coalition with the expansion delta. We now take care not to attempt access past the end of the LNC array. LNC array code will be cleaned up and suitably encapsulated as part of the forthcoming performance RFE 7043675. Reviewed-by: brutisso ! src/share/vm/gc_implementation/parNew/parCardTableModRefBS.cpp Changeset: e66f38dd58a9 Author: ysr Date: 2011-06-08 08:39 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/e66f38dd58a9 Merge From bengt.rutisson at oracle.com Wed Jun 8 14:00:32 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 08 Jun 2011 23:00:32 +0200 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DEFA9FB.8060705@oracle.com> References: <4DE62ADD.1040601@oracle.com> <4DE63A77.2030104@oracle.com> <4DE6BAB1.7090405@oracle.com> <4DEF2E87.80508@oracle.com> <4DEF8A88.5020704@oracle.com> <4DEFA9FB.8060705@oracle.com> Message-ID: <4DEFE2F0.6010204@oracle.com> Tony, Thanks for addressing these comments so quickly! I am fine with this version. However, I realize that I was unclear with what I meant in my last email when I talked about reuse_if_possible(), now called expand_to_next_region(). What I meant was that I thought it would be a good idea to move all of this code into heapRegionSeq: 1632 HeapRegion* hr = _hrs.expand_to_next_region(); 1633 if (hr == NULL) { 1634 // We'll have to create a new region. 1635 MemRegion mr(base, high); 1636 bool is_zeroed = !_g1_max_committed.contains(base); 1637 size_t hrs_index = _hrs.next_hrs_index(); 1638 hr = new HeapRegion(hrs_index, _bot_shared, mr, is_zeroed); 1639 _hrs.expand_to_next_region(hr); 1640 } else { 1641 // We'll re-use the existing region. 1642 assert(hr->bottom() == base, "post-condition"); 1643 } I would wrap that in a method called something like get_next_heap_region(). I will not persist on this. And I do like the name expand_to_next() much better than the old name. I don't fully understand why you changed the name of HeapRegionSeq::insert(HeapRegion* hr) to HeapRegionSeq::expand_to_next_region(HeapRegion* hr) . I think "insert" was a better name. But maybe "append" would be more appropriate. If the code above would be moved into heapRegionSeq we would not need to expose the insert/append/:expand_to_next_region(HeapRegion* hr) method. Sorry for being difficult. But, as I said, I am fine with pushing it the way it is. Also, just for my understanding, regarding "const" vs. "#define" you said "Maybe, I like the fact that they are guaranteed to be treated as constants by the C++ compilers". What do you mean by that? Bengt On 2011-06-08 18:57, Tony Printezis wrote: > Bengt, > > Here's the new webrev that includes all the changes (I'll push them as > a single changeset): > > http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.0/webrev.all/ > > > And here are my latest the changes wrt to the version you reviewed: > > http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.0/webrev.2.G1HRSLatest/ > > > Let me know if you think I haven't addressed your concerns. > > Tony > > On 06/08/2011 10:43 AM, Tony Printezis wrote: >> >> >> On 06/08/2011 04:10 AM, Bengt Rutisson wrote: >>> >>> Tony, >>> >>> I had a look at this as well since it is connected to the previous >>> review: "CRR: 7042285 and 7045330 : fixes and improvements to the >>> HeapRegionSeq class (S/M) ". >>> >>> It looks good. >>> >>> Two things, though: >>> >>> * I think it is very closely connected to the previous review. So, >>> close that I in fact think they should be reviewed and pushed >>> together. I don't really see why they are separate changes. >> >> Yeah, you're right. I did a bunch of the int -> size_t changes as >> part of the first changeset (as I didn't think there was much point >> in rewriting half the HeapRegionSeq class using the wrong type and >> fixing it later). And, when I went it to do the rest it turned out >> there was not that many additional changes. When I push I'll fold >> both changesets into one and tag it with all three CRs. >> >>> * I still think it is a bit strange that G1_INVALID_HRS_INDEX is >>> defined in heapRegion.hpp. I would prefer to keep it in >>> heapRegionSeq. The only use in heapRegion is in the constructor. >>> This constructor is only called from two places: >>> - G1CollectedHeap::expand() >>> Here we actually know the index we want and could pass that to >>> the constructor. No need to first set it to an invalid value. I >>> think this would be even cleaner with the get_heap_region() method >>> that I suggested in the previous review. >>> - G1CollectedHeap::initialize() >>> Here we know about heapRegionSeq and could ask it for the >>> invalid value and pass that to the constructor. >> >> Yeah, I was not crazy about having another "invalid" index. But maybe >> it's worth it. OK, I'll change that too and I'll post the new set of >> changes in a bit. >> >> Tony >> >>> Bengt >>> >>> On 2011-06-02 00:18, Tony Printezis wrote: >>>> Stefan and John, >>>> >>>> Thanks! All set with this one too. >>>> >>>> Tony >>>> >>>> On 06/01/2011 03:11 PM, Stefan Karlsson wrote: >>>>> Looks good. >>>>> >>>>> StefanK >>>>> >>>>> On 06/01/2011 02:04 PM, Tony Printezis wrote: >>>>>> Hi, >>>>>> >>>>>> Could I please have a couple of code reviews for this simple change: >>>>>> >>>>>> http://cr.openjdk.java.net/~tonyp/6804436/webrev.0/ >>>>>> >>>>>> Some of the int -> size_t changes were done as part of the >>>>>> HeapRegionSeq cleanup (see the separate code review request for >>>>>> 7042285 and 7045330 that I just sent out). This change does the >>>>>> rest (specifically: the changes related to the HeapRegion class). >>>>>> >>>>>> Note: I know that the webrev index says that some files have more >>>>>> than one change applied to them. This is incorrect and a >>>>>> side-effect of generating the webrev from a workspace with >>>>>> multiple patches stacked up. The diffs are actually correct (I >>>>>> checked). >>>>>> >>>>>> Tony >>>>>> >>>>> >>> From tony.printezis at oracle.com Wed Jun 8 14:41:28 2011 From: tony.printezis at oracle.com (tony.printezis at oracle.com) Date: Wed, 08 Jun 2011 21:41:28 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7032531: G1: enhance GC logging to include more accurate eden / survivor size transitions Message-ID: <20110608214130.08F5647D05@hg.openjdk.java.net> Changeset: 053d84a76d3d Author: tonyp Date: 2011-06-08 15:31 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/053d84a76d3d 7032531: G1: enhance GC logging to include more accurate eden / survivor size transitions Summary: This changeset extends the logging information generated by +PrintGCDetails to also print out separate size transitions for the eden, survivors, and old regions. Reviewed-by: ysr, brutisso ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp ! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp ! src/share/vm/gc_implementation/g1/g1CollectorPolicy.hpp From tony.printezis at oracle.com Wed Jun 8 18:46:38 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 08 Jun 2011 21:46:38 -0400 Subject: CRR (S/M): 6994322: Remove the is_tlab and is_noref / is_large_noref parameters from the CollectedHeap interface Message-ID: <4DF025FE.1040607@oracle.com> Hi all, This is the one you've all been waiting for. :-) I'd like a couple of code reviews for this change: http://cr.openjdk.java.net/~tonyp/6994322/webrev.0/ It removes two parameters from the mem_allocate() method and updates its implementations and uses. The two parameters are the following. a) is_tlab : indicates whether the allocation is for a TLAB or not. However, given that all TLAB allocations go through the allocate_new_tlab() method, all calls to mem_allocate() have is_tlab == false. (Note: in G1 we have had an assert that checks that !is_tlab for a few months now without any failures.) So, having the is_tlab parameter seems pointless. b) is_noref (also called: is_large_noref) : indicates that the allocation is for a large scalar object (and in particular: an array). The large_typearray_limit() method specifies how large such an object should be and, based on this value, the allocation goes through either array_allocate() or large_typearray_allocate(), the only difference between the two is the value of the is_noref parameter that's passed to the allocate_routines. However, all mem_allocate() implementations just ignore that parameter (we think it's some baggage we're still carrying from the Train GC!!!). The changes for a) were trivial for G1 and the framework GCs but a bit more involved for PS given that the is_tlab parameter is propagated through a few methods and a VM operation. But note that none of those are actually called from allocate_new_tlab(), so it's safe to assume that they are only called for non-TLAB allocations. I did go over that code very carefully to convince myself that this is the case and I don't see any flaws. I would appreciate if someone else also does that just in case... The changes for b) were trivial overall, given that the is_noref parameter was not actually used by any of the GCs. So I just had to remove the large_typearray_limit() and large_typearray_allocate()methods. I tested this workspace with our GCs and I haven't seen any issues. And please let me know if there are any concerns about this. Tony From tony.printezis at oracle.com Wed Jun 8 19:09:31 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 08 Jun 2011 22:09:31 -0400 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DEFE2F0.6010204@oracle.com> References: <4DE62ADD.1040601@oracle.com> <4DE63A77.2030104@oracle.com> <4DE6BAB1.7090405@oracle.com> <4DEF2E87.80508@oracle.com> <4DEF8A88.5020704@oracle.com> <4DEFA9FB.8060705@oracle.com> <4DEFE2F0.6010204@oracle.com> Message-ID: <4DF02B5B.2090408@oracle.com> Bengt, On 06/08/2011 05:00 PM, Bengt Rutisson wrote: > > Tony, > > Thanks for addressing these comments so quickly! I am fine with this > version. However, I realize that I was unclear with what I meant in my > last email when I talked about reuse_if_possible(), now called > expand_to_next_region(). > > What I meant was that I thought it would be a good idea to move all of > this code into heapRegionSeq: Ah, I see. Apologies I missed that point. > 1632 HeapRegion* hr = _hrs.expand_to_next_region(); > 1633 if (hr == NULL) { > 1634 // We'll have to create a new region. > 1635 MemRegion mr(base, high); > 1636 bool is_zeroed = !_g1_max_committed.contains(base); > 1637 size_t hrs_index = _hrs.next_hrs_index(); > 1638 hr = new HeapRegion(hrs_index, _bot_shared, mr, is_zeroed); > 1639 _hrs.expand_to_next_region(hr); > 1640 } else { > 1641 // We'll re-use the existing region. > 1642 assert(hr->bottom() == base, "post-condition"); > 1643 } > > I would wrap that in a method called something like > get_next_heap_region(). > > I will not persist on this. This is a good suggestion, Bengt, but I'll need to expose more things from G1CH to the HeapRegionSeq to do that (i.e., the fields needed in the HeapRegion constructor). And I just didn't want to re-arrange things further. So, I think I'll leave it as is, i.e., G1CH creating the regions and passing them on to HeapRegionSeq. > And I do like the name expand_to_next() much better than the old name. Good! > I don't fully understand why you changed the name of > HeapRegionSeq::insert(HeapRegion* hr) to > HeapRegionSeq::expand_to_next_region(HeapRegion* hr) . I think > "insert" was a better name. But maybe "append" would be more appropriate. Well, both expand_to_next_region() and insert() essentially do the same thing: they expand the sequence by one region, either by re-using an existing one or by adding a new one. So, I thought having a similar name was appropriate. > If the code above would be moved into heapRegionSeq we would not need > to expose the insert/append/:expand_to_next_region(HeapRegion* hr) > method. > > Sorry for being difficult. But, as I said, I am fine with pushing it > the way it is. :-) > Also, just for my understanding, regarding "const" vs. "#define" you > said "Maybe, I like the fact that they are guaranteed to be treated as > constants by the C++ compilers". What do you mean by that? In case the compiler decides not to the treat the const field as a constant for whatever reason... Tony > Bengt > > > On 2011-06-08 18:57, Tony Printezis wrote: >> Bengt, >> >> Here's the new webrev that includes all the changes (I'll push them >> as a single changeset): >> >> http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.0/webrev.all/ >> >> >> And here are my latest the changes wrt to the version you reviewed: >> >> http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.0/webrev.2.G1HRSLatest/ >> >> >> Let me know if you think I haven't addressed your concerns. >> >> Tony >> >> On 06/08/2011 10:43 AM, Tony Printezis wrote: >>> >>> >>> On 06/08/2011 04:10 AM, Bengt Rutisson wrote: >>>> >>>> Tony, >>>> >>>> I had a look at this as well since it is connected to the previous >>>> review: "CRR: 7042285 and 7045330 : fixes and improvements to the >>>> HeapRegionSeq class (S/M) ". >>>> >>>> It looks good. >>>> >>>> Two things, though: >>>> >>>> * I think it is very closely connected to the previous review. So, >>>> close that I in fact think they should be reviewed and pushed >>>> together. I don't really see why they are separate changes. >>> >>> Yeah, you're right. I did a bunch of the int -> size_t changes as >>> part of the first changeset (as I didn't think there was much point >>> in rewriting half the HeapRegionSeq class using the wrong type and >>> fixing it later). And, when I went it to do the rest it turned out >>> there was not that many additional changes. When I push I'll fold >>> both changesets into one and tag it with all three CRs. >>> >>>> * I still think it is a bit strange that G1_INVALID_HRS_INDEX is >>>> defined in heapRegion.hpp. I would prefer to keep it in >>>> heapRegionSeq. The only use in heapRegion is in the constructor. >>>> This constructor is only called from two places: >>>> - G1CollectedHeap::expand() >>>> Here we actually know the index we want and could pass that >>>> to the constructor. No need to first set it to an invalid value. I >>>> think this would be even cleaner with the get_heap_region() method >>>> that I suggested in the previous review. >>>> - G1CollectedHeap::initialize() >>>> Here we know about heapRegionSeq and could ask it for the >>>> invalid value and pass that to the constructor. >>> >>> Yeah, I was not crazy about having another "invalid" index. But >>> maybe it's worth it. OK, I'll change that too and I'll post the new >>> set of changes in a bit. >>> >>> Tony >>> >>>> Bengt >>>> >>>> On 2011-06-02 00:18, Tony Printezis wrote: >>>>> Stefan and John, >>>>> >>>>> Thanks! All set with this one too. >>>>> >>>>> Tony >>>>> >>>>> On 06/01/2011 03:11 PM, Stefan Karlsson wrote: >>>>>> Looks good. >>>>>> >>>>>> StefanK >>>>>> >>>>>> On 06/01/2011 02:04 PM, Tony Printezis wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Could I please have a couple of code reviews for this simple >>>>>>> change: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~tonyp/6804436/webrev.0/ >>>>>>> >>>>>>> Some of the int -> size_t changes were done as part of the >>>>>>> HeapRegionSeq cleanup (see the separate code review request for >>>>>>> 7042285 and 7045330 that I just sent out). This change does the >>>>>>> rest (specifically: the changes related to the HeapRegion class). >>>>>>> >>>>>>> Note: I know that the webrev index says that some files have >>>>>>> more than one change applied to them. This is incorrect and a >>>>>>> side-effect of generating the webrev from a workspace with >>>>>>> multiple patches stacked up. The diffs are actually correct (I >>>>>>> checked). >>>>>>> >>>>>>> Tony >>>>>>> >>>>>> >>>> > From tony.printezis at oracle.com Wed Jun 8 23:32:32 2011 From: tony.printezis at oracle.com (tony.printezis at oracle.com) Date: Thu, 09 Jun 2011 06:32:32 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7045662: G1: OopsInHeapRegionClosure::set_region() should not be virtual Message-ID: <20110609063237.4647747D7C@hg.openjdk.java.net> Changeset: ae5b2f1dcf12 Author: tonyp Date: 2011-06-08 21:48 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/ae5b2f1dcf12 7045662: G1: OopsInHeapRegionClosure::set_region() should not be virtual Summary: make the method non-virtual, remove five unused closures, and fix a couple of copyright typos. Reviewed-by: stefank, johnc, poonam ! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp ! src/share/vm/gc_implementation/g1/g1OopClosures.hpp ! src/share/vm/gc_implementation/g1/g1OopClosures.inline.hpp ! src/share/vm/gc_implementation/g1/g1RemSet.cpp ! src/share/vm/gc_implementation/g1/g1_specialized_oop_closures.hpp ! src/share/vm/gc_implementation/g1/heapRegionSet.hpp ! src/share/vm/gc_implementation/g1/heapRegionSets.hpp From bengt.rutisson at oracle.com Thu Jun 9 00:00:54 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 09 Jun 2011 09:00:54 +0200 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DF02B5B.2090408@oracle.com> References: <4DE62ADD.1040601@oracle.com> <4DE63A77.2030104@oracle.com> <4DE6BAB1.7090405@oracle.com> <4DEF2E87.80508@oracle.com> <4DEF8A88.5020704@oracle.com> <4DEFA9FB.8060705@oracle.com> <4DEFE2F0.6010204@oracle.com> <4DF02B5B.2090408@oracle.com> Message-ID: <4DF06FA6.2000708@oracle.com> Tony, On 2011-06-09 04:09, Tony Printezis wrote: >> Thanks for addressing these comments so quickly! I am fine with this >> version. However, I realize that I was unclear with what I meant in >> my last email when I talked about reuse_if_possible(), now called >> expand_to_next_region(). >> >> What I meant was that I thought it would be a good idea to move all >> of this code into heapRegionSeq: > > > Ah, I see. Apologies I missed that point. No problem. I re-read my email and realized that I was being pretty unclear. > >> 1632 HeapRegion* hr = _hrs.expand_to_next_region(); >> 1633 if (hr == NULL) { >> 1634 // We'll have to create a new region. >> 1635 MemRegion mr(base, high); >> 1636 bool is_zeroed = !_g1_max_committed.contains(base); >> 1637 size_t hrs_index = _hrs.next_hrs_index(); >> 1638 hr = new HeapRegion(hrs_index, _bot_shared, mr, is_zeroed); >> 1639 _hrs.expand_to_next_region(hr); >> 1640 } else { >> 1641 // We'll re-use the existing region. >> 1642 assert(hr->bottom() == base, "post-condition"); >> 1643 } >> >> I would wrap that in a method called something like >> get_next_heap_region(). >> >> I will not persist on this. > > This is a good suggestion, Bengt, but I'll need to expose more things > from G1CH to the HeapRegionSeq to do that (i.e., the fields needed in > the HeapRegion constructor). And I just didn't want to re-arrange > things further. So, I think I'll leave it as is, i.e., G1CH creating > the regions and passing them on to HeapRegionSeq. That's fine. Let's leave it as it is. >> And I do like the name expand_to_next() much better than the old name. > > Good! > >> I don't fully understand why you changed the name of >> HeapRegionSeq::insert(HeapRegion* hr) to >> HeapRegionSeq::expand_to_next_region(HeapRegion* hr) . I think >> "insert" was a better name. But maybe "append" would be more >> appropriate. > > Well, both expand_to_next_region() and insert() essentially do the > same thing: they expand the sequence by one region, either by re-using > an existing one or by adding a new one. So, I thought having a similar > name was appropriate. Ok. I see your point. >> Also, just for my understanding, regarding "const" vs. "#define" you >> said "Maybe, I like the fact that they are guaranteed to be treated >> as constants by the C++ compilers". What do you mean by that? > > In case the compiler decides not to the treat the const field as a > constant for whatever reason... Thanks for elaborating a bit on this. I realize that this is a topic for a wider discussion than your review request, so we should maybe take this in another thread. Let me just say that I think there are several advantages to using "const" instead of "#define". As I mentioned it will be possible to see the values in debuggers, the compiler can do type checking and they respect scope. Also, for all I know const in C++ is a compile time constant expression, so I don't know why a compiler would ever treat a const field as a non-constant. It is different in C of course. Anyway, I think your review looks good. Ship it! Bengt > > Tony > >> Bengt >> >> >> On 2011-06-08 18:57, Tony Printezis wrote: >>> Bengt, >>> >>> Here's the new webrev that includes all the changes (I'll push them >>> as a single changeset): >>> >>> http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.0/webrev.all/ >>> >>> >>> And here are my latest the changes wrt to the version you reviewed: >>> >>> http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.0/webrev.2.G1HRSLatest/ >>> >>> >>> Let me know if you think I haven't addressed your concerns. >>> >>> Tony >>> >>> On 06/08/2011 10:43 AM, Tony Printezis wrote: >>>> >>>> >>>> On 06/08/2011 04:10 AM, Bengt Rutisson wrote: >>>>> >>>>> Tony, >>>>> >>>>> I had a look at this as well since it is connected to the previous >>>>> review: "CRR: 7042285 and 7045330 : fixes and improvements to the >>>>> HeapRegionSeq class (S/M) ". >>>>> >>>>> It looks good. >>>>> >>>>> Two things, though: >>>>> >>>>> * I think it is very closely connected to the previous review. So, >>>>> close that I in fact think they should be reviewed and pushed >>>>> together. I don't really see why they are separate changes. >>>> >>>> Yeah, you're right. I did a bunch of the int -> size_t changes as >>>> part of the first changeset (as I didn't think there was much point >>>> in rewriting half the HeapRegionSeq class using the wrong type and >>>> fixing it later). And, when I went it to do the rest it turned out >>>> there was not that many additional changes. When I push I'll fold >>>> both changesets into one and tag it with all three CRs. >>>> >>>>> * I still think it is a bit strange that G1_INVALID_HRS_INDEX is >>>>> defined in heapRegion.hpp. I would prefer to keep it in >>>>> heapRegionSeq. The only use in heapRegion is in the constructor. >>>>> This constructor is only called from two places: >>>>> - G1CollectedHeap::expand() >>>>> Here we actually know the index we want and could pass that >>>>> to the constructor. No need to first set it to an invalid value. I >>>>> think this would be even cleaner with the get_heap_region() >>>>> method that I suggested in the previous review. >>>>> - G1CollectedHeap::initialize() >>>>> Here we know about heapRegionSeq and could ask it for the >>>>> invalid value and pass that to the constructor. >>>> >>>> Yeah, I was not crazy about having another "invalid" index. But >>>> maybe it's worth it. OK, I'll change that too and I'll post the new >>>> set of changes in a bit. >>>> >>>> Tony >>>> >>>>> Bengt >>>>> >>>>> On 2011-06-02 00:18, Tony Printezis wrote: >>>>>> Stefan and John, >>>>>> >>>>>> Thanks! All set with this one too. >>>>>> >>>>>> Tony >>>>>> >>>>>> On 06/01/2011 03:11 PM, Stefan Karlsson wrote: >>>>>>> Looks good. >>>>>>> >>>>>>> StefanK >>>>>>> >>>>>>> On 06/01/2011 02:04 PM, Tony Printezis wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Could I please have a couple of code reviews for this simple >>>>>>>> change: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~tonyp/6804436/webrev.0/ >>>>>>>> >>>>>>>> Some of the int -> size_t changes were done as part of the >>>>>>>> HeapRegionSeq cleanup (see the separate code review request for >>>>>>>> 7042285 and 7045330 that I just sent out). This change does the >>>>>>>> rest (specifically: the changes related to the HeapRegion class). >>>>>>>> >>>>>>>> Note: I know that the webrev index says that some files have >>>>>>>> more than one change applied to them. This is incorrect and a >>>>>>>> side-effect of generating the webrev from a workspace with >>>>>>>> multiple patches stacked up. The diffs are actually correct (I >>>>>>>> checked). >>>>>>>> >>>>>>>> Tony >>>>>>>> >>>>>>> >>>>> >> From tony.printezis at oracle.com Thu Jun 9 04:11:41 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Thu, 09 Jun 2011 07:11:41 -0400 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DF06FA6.2000708@oracle.com> References: <4DE62ADD.1040601@oracle.com> <4DE63A77.2030104@oracle.com> <4DE6BAB1.7090405@oracle.com> <4DEF2E87.80508@oracle.com> <4DEF8A88.5020704@oracle.com> <4DEFA9FB.8060705@oracle.com> <4DEFE2F0.6010204@oracle.com> <4DF02B5B.2090408@oracle.com> <4DF06FA6.2000708@oracle.com> Message-ID: <4DF0AA6D.9080901@oracle.com> Bengt, On 06/09/2011 03:00 AM, Bengt Rutisson wrote: >>> >>> I will not persist on this. >> >> This is a good suggestion, Bengt, but I'll need to expose more things >> from G1CH to the HeapRegionSeq to do that (i.e., the fields needed in >> the HeapRegion constructor). And I just didn't want to re-arrange >> things further. So, I think I'll leave it as is, i.e., G1CH creating >> the regions and passing them on to HeapRegionSeq. > > > That's fine. Let's leave it as it is. Actually (and I should have thought of that earlier!) I think I can do this by creating a factory method for HeapRegion instances in G1CH and calling it from HRS. So HRS will only have a single expand() method (opposite to shrink()) and will deal with the rest internally. Let me have a stab at it and I'll get back to you. Tony From bengt.rutisson at oracle.com Thu Jun 9 04:51:21 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 09 Jun 2011 13:51:21 +0200 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DF0AA6D.9080901@oracle.com> References: <4DE62ADD.1040601@oracle.com> <4DE63A77.2030104@oracle.com> <4DE6BAB1.7090405@oracle.com> <4DEF2E87.80508@oracle.com> <4DEF8A88.5020704@oracle.com> <4DEFA9FB.8060705@oracle.com> <4DEFE2F0.6010204@oracle.com> <4DF02B5B.2090408@oracle.com> <4DF06FA6.2000708@oracle.com> <4DF0AA6D.9080901@oracle.com> Message-ID: <4DF0B3B9.4040808@oracle.com> Tony, >>> >>> This is a good suggestion, Bengt, but I'll need to expose more >>> things from G1CH to the HeapRegionSeq to do that (i.e., the fields >>> needed in the HeapRegion constructor). And I just didn't want to >>> re-arrange things further. So, I think I'll leave it as is, i.e., >>> G1CH creating the regions and passing them on to HeapRegionSeq. >> >> >> That's fine. Let's leave it as it is. > > Actually (and I should have thought of that earlier!) I think I can do > this by creating a factory method for HeapRegion instances in G1CH and > calling it from HRS. So HRS will only have a single expand() method > (opposite to shrink()) and will deal with the rest internally. Let me > have a stab at it and I'll get back to you. Sounds like a good plan. Looking forward to seeing it. Bengt From stefan.karlsson at oracle.com Thu Jun 9 06:35:48 2011 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 09 Jun 2011 15:35:48 +0200 Subject: CRR (S/M): 6994322: Remove the is_tlab and is_noref / is_large_noref parameters from the CollectedHeap interface In-Reply-To: <4DF025FE.1040607@oracle.com> References: <4DF025FE.1040607@oracle.com> Message-ID: <4DF0CC34.7080003@oracle.com> Hi Tony, Looks good. Thanks for cleaning this up. One nit. The following comment shouldn't refer to the removed is_tlab parameter anymore: g1CollectedHeap.hpp 437 // * All non-TLAB allocation requests should go to mem_allocate() 438 // and mem_allocate() should never be called with is_tlab == true. StefanK On 06/09/2011 03:46 AM, Tony Printezis wrote: > Hi all, > > This is the one you've all been waiting for. :-) I'd like a couple of > code reviews for this change: > > http://cr.openjdk.java.net/~tonyp/6994322/webrev.0/ > > It removes two parameters from the mem_allocate() method and updates > its implementations and uses. The two parameters are the following. > > a) is_tlab : indicates whether the allocation is for a TLAB or not. > However, given that all TLAB allocations go through the > allocate_new_tlab() method, all calls to mem_allocate() have is_tlab > == false. (Note: in G1 we have had an assert that checks that !is_tlab > for a few months now without any failures.) So, having the is_tlab > parameter seems pointless. > > b) is_noref (also called: is_large_noref) : indicates that the > allocation is for a large scalar object (and in particular: an array). > The large_typearray_limit() method specifies how large such an object > should be and, based on this value, the allocation goes through either > array_allocate() or large_typearray_allocate(), the only difference > between the two is the value of the is_noref parameter that's passed > to the allocate_routines. However, all mem_allocate() implementations > just ignore that parameter (we think it's some baggage we're still > carrying from the Train GC!!!). > > The changes for a) were trivial for G1 and the framework GCs but a bit > more involved for PS given that the is_tlab parameter is propagated > through a few methods and a VM operation. But note that none of those > are actually called from allocate_new_tlab(), so it's safe to assume > that they are only called for non-TLAB allocations. I did go over that > code very carefully to convince myself that this is the case and I > don't see any flaws. I would appreciate if someone else also does that > just in case... > > The changes for b) were trivial overall, given that the is_noref > parameter was not actually used by any of the GCs. So I just had to > remove the large_typearray_limit() and large_typearray_allocate()methods. > > I tested this workspace with our GCs and I haven't seen any issues. > And please let me know if there are any concerns about this. > > Tony > From tony.printezis at oracle.com Thu Jun 9 11:10:05 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Thu, 09 Jun 2011 14:10:05 -0400 Subject: CRR (S/M): 6994322: Remove the is_tlab and is_noref / is_large_noref parameters from the CollectedHeap interface In-Reply-To: <4DF0CC34.7080003@oracle.com> References: <4DF025FE.1040607@oracle.com> <4DF0CC34.7080003@oracle.com> Message-ID: <4DF10C7D.5040803@oracle.com> Thanks Stefan for the prompt code review! And thanks for catching the issue below. Here's the updated webrev: http://cr.openjdk.java.net/~tonyp/6994322/webrev.1/ Tony On 06/09/2011 09:35 AM, Stefan Karlsson wrote: > Hi Tony, > > Looks good. Thanks for cleaning this up. > > One nit. The following comment shouldn't refer to the removed is_tlab > parameter anymore: > > g1CollectedHeap.hpp > 437 // * All non-TLAB allocation requests should go to mem_allocate() > 438 // and mem_allocate() should never be called with is_tlab == > true. > > StefanK > > On 06/09/2011 03:46 AM, Tony Printezis wrote: >> Hi all, >> >> This is the one you've all been waiting for. :-) I'd like a couple of >> code reviews for this change: >> >> http://cr.openjdk.java.net/~tonyp/6994322/webrev.0/ >> >> It removes two parameters from the mem_allocate() method and updates >> its implementations and uses. The two parameters are the following. >> >> a) is_tlab : indicates whether the allocation is for a TLAB or not. >> However, given that all TLAB allocations go through the >> allocate_new_tlab() method, all calls to mem_allocate() have is_tlab >> == false. (Note: in G1 we have had an assert that checks that >> !is_tlab for a few months now without any failures.) So, having the >> is_tlab parameter seems pointless. >> >> b) is_noref (also called: is_large_noref) : indicates that the >> allocation is for a large scalar object (and in particular: an >> array). The large_typearray_limit() method specifies how large such >> an object should be and, based on this value, the allocation goes >> through either array_allocate() or large_typearray_allocate(), the >> only difference between the two is the value of the is_noref >> parameter that's passed to the allocate_routines. However, all >> mem_allocate() implementations just ignore that parameter (we think >> it's some baggage we're still carrying from the Train GC!!!). >> >> The changes for a) were trivial for G1 and the framework GCs but a >> bit more involved for PS given that the is_tlab parameter is >> propagated through a few methods and a VM operation. But note that >> none of those are actually called from allocate_new_tlab(), so it's >> safe to assume that they are only called for non-TLAB allocations. I >> did go over that code very carefully to convince myself that this is >> the case and I don't see any flaws. I would appreciate if someone >> else also does that just in case... >> >> The changes for b) were trivial overall, given that the is_noref >> parameter was not actually used by any of the GCs. So I just had to >> remove the large_typearray_limit() and >> large_typearray_allocate()methods. >> >> I tested this workspace with our GCs and I haven't seen any issues. >> And please let me know if there are any concerns about this. >> >> Tony >> > From tony.printezis at oracle.com Thu Jun 9 15:46:46 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Thu, 09 Jun 2011 18:46:46 -0400 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DF0AA6D.9080901@oracle.com> References: <4DE62ADD.1040601@oracle.com> <4DE63A77.2030104@oracle.com> <4DE6BAB1.7090405@oracle.com> <4DEF2E87.80508@oracle.com> <4DEF8A88.5020704@oracle.com> <4DEFA9FB.8060705@oracle.com> <4DEFE2F0.6010204@oracle.com> <4DF02B5B.2090408@oracle.com> <4DF06FA6.2000708@oracle.com> <4DF0AA6D.9080901@oracle.com> Message-ID: <4DF14D56.70706@oracle.com> Bengt, Here's the latest version: complete webrev: http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.1/webrev.all/ diffs from the previous one: http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.1/webrev.1.G1HRSFactory/ I created a factory method for HeapRegions, G1CH::new_heap_region(), combined the two expand_to_next_region() methods of HeapRegionSeq class into one (HRS::expand_by()), and updated G1CH::expand() accordingly. I did notice however that the HeapRegion constructor could fail to allocate space, if we're out of swap. Even though it's a marginal condition, JohnC2 recently put some effort to make the expansion code robust wrt to expansion failing. So, I made some additional changes to deal with the case where we cannot allocate more HeapRegions (the committed space will be shrunk accordingly). An additional minor change is that I renamed the HTS fields _end and _bottom to _heap_end and _heap_bottom to make what they represent a bit clearer. Better? Tony On 06/09/2011 07:11 AM, Tony Printezis wrote: > Bengt, > > On 06/09/2011 03:00 AM, Bengt Rutisson wrote: >>>> >>>> I will not persist on this. >>> >>> This is a good suggestion, Bengt, but I'll need to expose more >>> things from G1CH to the HeapRegionSeq to do that (i.e., the fields >>> needed in the HeapRegion constructor). And I just didn't want to >>> re-arrange things further. So, I think I'll leave it as is, i.e., >>> G1CH creating the regions and passing them on to HeapRegionSeq. >> >> >> That's fine. Let's leave it as it is. > > Actually (and I should have thought of that earlier!) I think I can do > this by creating a factory method for HeapRegion instances in G1CH and > calling it from HRS. So HRS will only have a single expand() method > (opposite to shrink()) and will deal with the rest internally. Let me > have a stab at it and I'll get back to you. > > Tony From john.coomes at oracle.com Thu Jun 9 20:41:20 2011 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 10 Jun 2011 03:41:20 +0000 Subject: hg: jdk7/hotspot-gc: 3 new changesets Message-ID: <20110610034120.5DD2547E22@hg.openjdk.java.net> Changeset: 93d2590fd849 Author: jeff Date: 2011-05-27 14:57 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/rev/93d2590fd849 7045697: JDK7 THIRD PARTY README update Reviewed-by: lana ! THIRD_PARTY_README Changeset: 55e9ebf03218 Author: lana Date: 2011-06-02 13:37 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/rev/55e9ebf03218 Merge Changeset: 2d38c2a79c14 Author: schien Date: 2011-06-07 14:00 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/rev/2d38c2a79c14 Added tag jdk7-b145 for changeset 55e9ebf03218 ! .hgtags From john.coomes at oracle.com Thu Jun 9 20:41:30 2011 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 10 Jun 2011 03:41:30 +0000 Subject: hg: jdk7/hotspot-gc/corba: 5 new changesets Message-ID: <20110610034137.4359E47E25@hg.openjdk.java.net> Changeset: 93e77c49b3bb Author: miroslawzn Date: 2011-05-26 13:05 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/corba/rev/93e77c49b3bb 7046882: Regression : JDK 7b123 : Enum exchanged as parameters using CORBA call results in Exception Reviewed-by: raginip ! src/share/classes/com/sun/corba/se/impl/io/ObjectStreamClass.java Changeset: 643f267ca234 Author: jeff Date: 2011-05-27 14:58 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/corba/rev/643f267ca234 7045697: JDK7 THIRD PARTY README update Reviewed-by: lana ! THIRD_PARTY_README Changeset: 7839048ec99e Author: jeff Date: 2011-05-27 15:27 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/corba/rev/7839048ec99e Merge Changeset: 77ec0541aa2a Author: lana Date: 2011-06-02 13:37 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/corba/rev/77ec0541aa2a Merge Changeset: 770227a4087e Author: schien Date: 2011-06-07 14:00 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/corba/rev/770227a4087e Added tag jdk7-b145 for changeset 77ec0541aa2a ! .hgtags From john.coomes at oracle.com Thu Jun 9 20:41:46 2011 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 10 Jun 2011 03:41:46 +0000 Subject: hg: jdk7/hotspot-gc/jaxp: 5 new changesets Message-ID: <20110610034147.21DBC47E28@hg.openjdk.java.net> Changeset: bdf77cbd9958 Author: ohair Date: 2011-05-19 08:38 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxp/rev/bdf77cbd9958 7044493: Incorrectly formated GPL headers in JDK7 JAXP source drop Reviewed-by: joehw ! jaxp.properties Changeset: 775dd77e4730 Author: lana Date: 2011-05-20 21:00 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxp/rev/775dd77e4730 Merge Changeset: a70a042c8600 Author: jeff Date: 2011-05-27 15:01 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxp/rev/a70a042c8600 7045697: JDK7 THIRD PARTY README update Reviewed-by: lana ! THIRD_PARTY_README Changeset: 10ca7570f47f Author: lana Date: 2011-06-02 13:37 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxp/rev/10ca7570f47f Merge Changeset: bcd31fa1e3c6 Author: schien Date: 2011-06-07 14:01 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxp/rev/bcd31fa1e3c6 Added tag jdk7-b145 for changeset 10ca7570f47f ! .hgtags From john.coomes at oracle.com Thu Jun 9 20:41:56 2011 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 10 Jun 2011 03:41:56 +0000 Subject: hg: jdk7/hotspot-gc/jaxws: 4 new changesets Message-ID: <20110610034156.BFDE547E2C@hg.openjdk.java.net> Changeset: c902e39c384e Author: jeff Date: 2011-05-27 15:01 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/c902e39c384e 7045697: JDK7 THIRD PARTY README update Reviewed-by: lana ! THIRD_PARTY_README Changeset: bcca8afc019f Author: ohair Date: 2011-06-01 10:36 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/bcca8afc019f 7049699: Problem with xml/jax-ws Reviewed-by: ramap ! jaxws.properties Changeset: 42bfba80beb7 Author: lana Date: 2011-06-02 13:37 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/42bfba80beb7 Merge Changeset: 6ec12c60ad13 Author: schien Date: 2011-06-07 14:01 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/6ec12c60ad13 Added tag jdk7-b145 for changeset 42bfba80beb7 ! .hgtags From john.coomes at oracle.com Thu Jun 9 20:42:51 2011 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 10 Jun 2011 03:42:51 +0000 Subject: hg: jdk7/hotspot-gc/jdk: 42 new changesets Message-ID: <20110610035038.BC07B47E3E@hg.openjdk.java.net> Changeset: f09930d526ba Author: jrose Date: 2011-05-26 17:37 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/f09930d526ba 7032323: code changes for JSR 292 EG adjustments to API, through Public Review Summary: API code changes and javadoc changes following JSR 292 Public Review comments, through PFD Reviewed-by: never ! src/share/classes/java/lang/BootstrapMethodError.java ! src/share/classes/java/lang/ClassValue.java ! src/share/classes/java/lang/invoke/AdapterMethodHandle.java ! src/share/classes/java/lang/invoke/BoundMethodHandle.java ! src/share/classes/java/lang/invoke/CallSite.java ! src/share/classes/java/lang/invoke/ConstantCallSite.java ! src/share/classes/java/lang/invoke/Invokers.java ! src/share/classes/java/lang/invoke/MemberName.java ! src/share/classes/java/lang/invoke/MethodHandle.java ! src/share/classes/java/lang/invoke/MethodHandleImpl.java ! src/share/classes/java/lang/invoke/MethodHandleNatives.java + src/share/classes/java/lang/invoke/MethodHandleProxies.java ! src/share/classes/java/lang/invoke/MethodHandles.java ! src/share/classes/java/lang/invoke/MethodType.java ! src/share/classes/java/lang/invoke/MutableCallSite.java ! src/share/classes/java/lang/invoke/SwitchPoint.java ! src/share/classes/java/lang/invoke/package-info.java ! src/share/classes/sun/invoke/util/ValueConversions.java ! src/share/classes/sun/invoke/util/VerifyAccess.java ! src/share/classes/sun/invoke/util/Wrapper.java ! test/java/lang/invoke/6998541/Test6998541.java ! test/java/lang/invoke/InvokeDynamicPrintArgs.java ! test/java/lang/invoke/InvokeGenericTest.java ! test/java/lang/invoke/JavaDocExamplesTest.java ! test/java/lang/invoke/MethodHandlesTest.java ! test/java/lang/invoke/indify/Indify.java Changeset: 81f957f86ba5 Author: jcoomes Date: 2011-05-27 19:03 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/81f957f86ba5 Merge Changeset: 8318d03e1766 Author: jrose Date: 2011-06-01 23:56 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/8318d03e1766 7049415: Failure of resolution of sym.reference to the c.s.s. should be wrapped in BootstrapMethodError Summary: Wrap invokedynamic linkage errors in BootstrapMethodError, as needed. Reviewed-by: never ! src/share/classes/java/lang/invoke/MethodHandleNatives.java Changeset: 0b8b6eace473 Author: jrose Date: 2011-06-01 23:56 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/0b8b6eace473 7050328: (jsr-292) findConstructor throws ExceptionInInitializerError if run under SecurityManager Summary: Wrap system property and reflection accesses under doPrivileged. Ensure constant pool linkage bypasses the SM as specified. Reviewed-by: kvn, never ! src/share/classes/java/lang/invoke/MethodHandleImpl.java ! src/share/classes/java/lang/invoke/MethodHandleNatives.java ! src/share/classes/java/lang/invoke/MethodHandleStatics.java ! src/share/classes/java/lang/invoke/MethodHandles.java ! src/share/classes/sun/invoke/util/ValueConversions.java ! test/java/lang/invoke/InvokeDynamicPrintArgs.java Changeset: 34481a4012c3 Author: jrose Date: 2011-06-01 23:56 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/34481a4012c3 7049122: java/lang/invoke/RicochetTest.java with MAX_ARITY=255 in -Xcomp mode overflows code cache Summary: reduce the scope of the unit test (mark high water mark of testing with @ignore tags) Reviewed-by: never ! test/java/lang/invoke/RicochetTest.java Changeset: 802994506203 Author: jrose Date: 2011-06-03 11:20 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/802994506203 7051206: JSR 292 method name SwitchPoint.isValid is misleading to unwary users; should be hasBeenInvalidated Reviewed-by: kvn, never, ysr ! src/share/classes/java/lang/invoke/SwitchPoint.java ! test/java/lang/invoke/JavaDocExamplesTest.java Changeset: bc97b962330e Author: mfang Date: 2011-05-26 20:32 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/bc97b962330e 7045184: GTK L&F doesn't have hotkeys in jdk7 b141, while b139 has. Reviewed-by: yhuang, ogino ! src/share/classes/com/sun/java/swing/plaf/gtk/resources/gtk.properties Changeset: 6943c4d9caa3 Author: mfang Date: 2011-05-31 13:58 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/6943c4d9caa3 Merge Changeset: 7c5bc5a807ee Author: dholmes Date: 2011-05-27 19:04 -0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/7c5bc5a807ee 7024120: Verify reduced JRE contents for java 7 Summary: stripped all symbols from libs and executables to reduce JRE size. Restored missing classes needed to pass JCK in headless mode Reviewed-by: bobv, ohair ! make/common/Defs-embedded.gmk ! make/common/Release-embedded.gmk Changeset: f4895b3fe1be Author: dholmes Date: 2011-05-31 17:28 -0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/f4895b3fe1be Merge Changeset: eab27338b3d9 Author: schien Date: 2011-06-01 11:16 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/eab27338b3d9 Merge Changeset: 8d91855a1f4e Author: prr Date: 2011-05-27 13:25 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/8d91855a1f4e 7046587: Outlines in OTF/CFF fonts are misclassified as quadratic curves Reviewed-by: igor ! src/share/classes/sun/font/FileFontStrike.java ! src/share/classes/sun/font/FontScaler.java ! src/share/classes/sun/font/FreetypeFontScaler.java ! src/share/classes/sun/font/NullFontScaler.java Changeset: 0b0b92707cf5 Author: bae Date: 2011-05-30 12:05 +0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/0b0b92707cf5 7032904: XRender: Java2Demo : Infinite loop in Java_sun_java2d_loops_MaskBlit_MaskBlit on OEL 5.6 x64 Reviewed-by: prr ! src/solaris/classes/sun/awt/X11GraphicsEnvironment.java ! src/solaris/native/sun/java2d/x11/XRBackendNative.c Changeset: 290daca0a693 Author: prr Date: 2011-05-30 22:00 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/290daca0a693 7049874: OpenJDK Build breakage fix: freetypescaler.c needs to match updated signature Reviewed-by: lana, igor ! src/share/native/sun/font/freetypeScaler.c Changeset: b351af09bfa3 Author: lana Date: 2011-06-02 13:35 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/b351af09bfa3 Merge Changeset: d2081a1f417f Author: bagiras Date: 2011-05-27 11:45 +0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/d2081a1f417f 7045174: Most of the tests in awt area fails with jdk 7b142 on windows with -Xcheck:jni Reviewed-by: art, denis ! src/windows/native/sun/windows/awt_Object.cpp ! src/windows/native/sun/windows/awt_Toolkit.cpp Changeset: 000a845b1e38 Author: denis Date: 2011-05-28 12:56 +0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/000a845b1e38 7046325: Broken links in java.awt.Toolkit's javadoc Reviewed-by: dav, yan ! src/share/classes/java/awt/Toolkit.java Changeset: eab94f59b6dc Author: dcherepanov Date: 2011-05-30 13:25 +0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/eab94f59b6dc 7045354: Korean IME's Hanja candidate window is not displayed on IMFDemo Reviewed-by: art, ant ! src/windows/native/sun/windows/awt_Component.cpp Changeset: f05164caa490 Author: serb Date: 2011-05-30 17:16 +0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/f05164caa490 7045193: interactive JCK tests java_awt/interactive/FileDialogTests fail Reviewed-by: dcherepanov, dav, art, denis ! src/solaris/classes/sun/awt/X11/GtkFileDialogPeer.java Changeset: b955226868af Author: lana Date: 2011-06-02 13:36 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/b955226868af Merge Changeset: 1fbaf2b688a6 Author: rupashka Date: 2011-05-24 11:37 +0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/1fbaf2b688a6 7045593: Possible Regression : JTextfield cursor placement behavior algorithm has changed. Reviewed-by: peterz ! src/share/classes/javax/swing/text/Utilities.java + test/javax/swing/text/Utilities/bug7045593.java Changeset: 442237d3ec2c Author: rupashka Date: 2011-05-28 11:55 +0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/442237d3ec2c 7048204: NPE from NimbusLookAndFeel.addDefault Reviewed-by: peterz ! src/share/classes/javax/swing/plaf/nimbus/NimbusLookAndFeel.java + test/javax/swing/plaf/nimbus/Test7048204.java Changeset: 386516fdf40b Author: lana Date: 2011-06-02 13:37 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/386516fdf40b Merge Changeset: 0a80650409e1 Author: mullan Date: 2011-05-24 14:15 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/0a80650409e1 7044443: Permissions resolved incorrectly for jar protocol (Patch from bugs.openjdk.java.net) Reviewed-by: alanb, chegar Contributed-by: dbhole at redhat.com ! src/share/classes/sun/security/provider/PolicyFile.java + test/java/security/Policy/GetPermissions/JarURL.java Changeset: ace2dfdecda1 Author: mullan Date: 2011-05-24 14:32 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/ace2dfdecda1 Merge Changeset: 70942be348af Author: jeff Date: 2011-05-27 15:02 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/70942be348af 7045697: JDK7 THIRD PARTY README update Reviewed-by: lana ! THIRD_PARTY_README Changeset: b49a0af85821 Author: vinnie Date: 2011-05-30 16:37 +0100 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/b49a0af85821 7049173: Replace the software license for ECC native code Reviewed-by: alanb ! src/share/native/sun/security/ec/impl/ec.c ! src/share/native/sun/security/ec/impl/ec.h ! src/share/native/sun/security/ec/impl/ec2.h ! src/share/native/sun/security/ec/impl/ec2_163.c ! src/share/native/sun/security/ec/impl/ec2_193.c ! src/share/native/sun/security/ec/impl/ec2_233.c ! src/share/native/sun/security/ec/impl/ec2_aff.c ! src/share/native/sun/security/ec/impl/ec2_mont.c ! src/share/native/sun/security/ec/impl/ec_naf.c ! src/share/native/sun/security/ec/impl/ecc_impl.h ! src/share/native/sun/security/ec/impl/ecdecode.c ! src/share/native/sun/security/ec/impl/ecl-curve.h ! src/share/native/sun/security/ec/impl/ecl-exp.h ! src/share/native/sun/security/ec/impl/ecl-priv.h ! src/share/native/sun/security/ec/impl/ecl.c ! src/share/native/sun/security/ec/impl/ecl.h ! src/share/native/sun/security/ec/impl/ecl_curve.c ! src/share/native/sun/security/ec/impl/ecl_gf.c ! src/share/native/sun/security/ec/impl/ecl_mult.c ! src/share/native/sun/security/ec/impl/ecp.h ! src/share/native/sun/security/ec/impl/ecp_192.c ! src/share/native/sun/security/ec/impl/ecp_224.c ! src/share/native/sun/security/ec/impl/ecp_256.c ! src/share/native/sun/security/ec/impl/ecp_384.c ! src/share/native/sun/security/ec/impl/ecp_521.c ! src/share/native/sun/security/ec/impl/ecp_aff.c ! src/share/native/sun/security/ec/impl/ecp_jac.c ! src/share/native/sun/security/ec/impl/ecp_jm.c ! src/share/native/sun/security/ec/impl/ecp_mont.c ! src/share/native/sun/security/ec/impl/logtab.h ! src/share/native/sun/security/ec/impl/mp_gf2m-priv.h ! src/share/native/sun/security/ec/impl/mp_gf2m.c ! src/share/native/sun/security/ec/impl/mp_gf2m.h ! src/share/native/sun/security/ec/impl/mpi-config.h ! src/share/native/sun/security/ec/impl/mpi-priv.h ! src/share/native/sun/security/ec/impl/mpi.c ! src/share/native/sun/security/ec/impl/mpi.h ! src/share/native/sun/security/ec/impl/mplogic.c ! src/share/native/sun/security/ec/impl/mplogic.h ! src/share/native/sun/security/ec/impl/mpmontg.c ! src/share/native/sun/security/ec/impl/mpprime.h ! src/share/native/sun/security/ec/impl/oid.c ! src/share/native/sun/security/ec/impl/secitem.c ! src/share/native/sun/security/ec/impl/secoidt.h Changeset: 4ed7c877a463 Author: michaelm Date: 2011-05-30 23:36 +0100 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/4ed7c877a463 7042550: Reintegrate 6569621 Reviewed-by: chegar, alanb ! src/share/classes/java/net/InetAddress.java ! src/share/classes/java/net/Socket.java ! src/share/classes/java/net/SocketPermission.java + src/share/classes/sun/net/RegisteredDomain.java ! src/share/classes/sun/net/www/URLConnection.java ! src/share/classes/sun/net/www/http/HttpClient.java Changeset: c79a089ae13b Author: wetmore Date: 2011-05-31 12:45 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/c79a089ae13b 7042097: JDK 7's Unlimited Cryptographic Policy bundle text files must be updated. Reviewed-by: valeriep ! make/javax/crypto/Makefile Changeset: a00f48c96345 Author: lancea Date: 2011-06-02 12:02 -0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/a00f48c96345 7049107: Cannot call initCause() on BatchUpdateException Reviewed-by: darcy ! src/share/classes/java/sql/BatchUpdateException.java Changeset: 39de8937c1d8 Author: lana Date: 2011-06-02 13:38 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/39de8937c1d8 Merge Changeset: e8e6cdff5995 Author: trims Date: 2011-06-03 20:13 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/e8e6cdff5995 Merge Changeset: 8f19b165347b Author: bae Date: 2011-06-04 23:08 +0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/8f19b165347b 7042594: 3 testis api/java_awt/Color/ICC_ProfileRGB/index.html fail against RI b138 OEL6x64. Reviewed-by: prr ! src/share/classes/java/awt/color/ICC_Profile.java ! src/share/native/sun/java2d/cmm/lcms/LCMS.c ! src/share/native/sun/java2d/cmm/lcms/cmsio0.c ! test/sun/java2d/cmm/ProfileOp/ReadWriteProfileTest.java + test/sun/java2d/cmm/ProfileOp/SetDataTest.java Changeset: bbb4cef2208b Author: lana Date: 2011-06-04 17:30 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/bbb4cef2208b Merge Changeset: 03a828e368ca Author: rupashka Date: 2011-06-04 01:13 +0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/03a828e368ca 6977587: GTK L&F: jnlp: java.io.IOException thrown when choosing more than 1 file in the dialog Reviewed-by: alexp ! src/share/classes/com/sun/java/swing/plaf/gtk/GTKFileChooserUI.java Changeset: 6c9c55ae313b Author: lana Date: 2011-06-03 22:14 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/6c9c55ae313b Merge Changeset: e81d259442ed Author: lana Date: 2011-06-04 17:32 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/e81d259442ed Merge Changeset: 53d759eb580c Author: alanb Date: 2011-06-04 11:18 +0100 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/53d759eb580c 7050358: (fs spec) Path.toUri doesn't allow custom providers to use opaque URIs Reviewed-by: sherman ! src/share/classes/java/nio/file/Path.java Changeset: 49aef5a5416e Author: mullan Date: 2011-06-04 06:45 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/49aef5a5416e 7050329: test/java/security/Policy/GetPermissions/JarURL.java fails on Windows Reviewed-by: alanb ! test/java/security/Policy/GetPermissions/JarURL.java + test/java/security/Policy/GetPermissions/JarURL.policy Changeset: 1f39ca0b9598 Author: mullan Date: 2011-06-04 06:52 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/1f39ca0b9598 Merge Changeset: 1e04b38b3824 Author: lana Date: 2011-06-04 17:33 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/1e04b38b3824 Merge Changeset: 7a341c412ea9 Author: schien Date: 2011-06-07 14:01 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/7a341c412ea9 Added tag jdk7-b145 for changeset 1e04b38b3824 ! .hgtags From bengt.rutisson at oracle.com Thu Jun 9 23:30:21 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 10 Jun 2011 08:30:21 +0200 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DF14D56.70706@oracle.com> References: <4DE62ADD.1040601@oracle.com> <4DE63A77.2030104@oracle.com> <4DE6BAB1.7090405@oracle.com> <4DEF2E87.80508@oracle.com> <4DEF8A88.5020704@oracle.com> <4DEFA9FB.8060705@oracle.com> <4DEFE2F0.6010204@oracle.com> <4DF02B5B.2090408@oracle.com> <4DF06FA6.2000708@oracle.com> <4DF0AA6D.9080901@oracle.com> <4DF14D56.70706@oracle.com> Message-ID: <4DF1B9FD.3070703@oracle.com> Tony, Thanks for iterating this so many times. I think it looks great now. Thumbs up! Just one nit: In heapRegionSeq.cpp you added the include "gc_implementation/g1/heapRegionSets.hpp". I think that the one you want is "gc_implementation/g1/heapRegionSeq.hpp". Bengt On 2011-06-10 00:46, Tony Printezis wrote: > Bengt, > > Here's the latest version: > > complete webrev: > http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.1/webrev.all/ > > > diffs from the previous one: > http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.1/webrev.1.G1HRSFactory/ > > > I created a factory method for HeapRegions, G1CH::new_heap_region(), > combined the two expand_to_next_region() methods of HeapRegionSeq > class into one (HRS::expand_by()), and updated G1CH::expand() > accordingly. > > I did notice however that the HeapRegion constructor could fail to > allocate space, if we're out of swap. Even though it's a marginal > condition, JohnC2 recently put some effort to make the expansion code > robust wrt to expansion failing. So, I made some additional changes to > deal with the case where we cannot allocate more HeapRegions (the > committed space will be shrunk accordingly). > > An additional minor change is that I renamed the HTS fields _end and > _bottom to _heap_end and _heap_bottom to make what they represent a > bit clearer. > > Better? > > Tony > > > On 06/09/2011 07:11 AM, Tony Printezis wrote: >> Bengt, >> >> On 06/09/2011 03:00 AM, Bengt Rutisson wrote: >>>>> >>>>> I will not persist on this. >>>> >>>> This is a good suggestion, Bengt, but I'll need to expose more >>>> things from G1CH to the HeapRegionSeq to do that (i.e., the fields >>>> needed in the HeapRegion constructor). And I just didn't want to >>>> re-arrange things further. So, I think I'll leave it as is, i.e., >>>> G1CH creating the regions and passing them on to HeapRegionSeq. >>> >>> >>> That's fine. Let's leave it as it is. >> >> Actually (and I should have thought of that earlier!) I think I can >> do this by creating a factory method for HeapRegion instances in G1CH >> and calling it from HRS. So HRS will only have a single expand() >> method (opposite to shrink()) and will deal with the rest internally. >> Let me have a stab at it and I'll get back to you. >> >> Tony From tony.printezis at oracle.com Fri Jun 10 05:07:06 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Fri, 10 Jun 2011 08:07:06 -0400 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DF1B9FD.3070703@oracle.com> References: <4DE62ADD.1040601@oracle.com> <4DE63A77.2030104@oracle.com> <4DE6BAB1.7090405@oracle.com> <4DEF2E87.80508@oracle.com> <4DEF8A88.5020704@oracle.com> <4DEFA9FB.8060705@oracle.com> <4DEFE2F0.6010204@oracle.com> <4DF02B5B.2090408@oracle.com> <4DF06FA6.2000708@oracle.com> <4DF0AA6D.9080901@oracle.com> <4DF14D56.70706@oracle.com> <4DF1B9FD.3070703@oracle.com> Message-ID: <4DF208EA.803@oracle.com> On 06/10/2011 02:30 AM, Bengt Rutisson wrote: > > Tony, > > Thanks for iterating this so many times. I think it looks great now. > Thumbs up! Thanks for looking at all the iterations! I ran tests on it last night and everything is OK. I'll schedule it for a push today. > Just one nit: > > In heapRegionSeq.cpp you added the include > "gc_implementation/g1/heapRegionSets.hpp". I think that the one you > want is "gc_implementation/g1/heapRegionSeq.hpp". Actually, ...Sets.hpp is correct as it's the one that FreeRegionList declaration (needed by expand_by()). Tony > Bengt > > On 2011-06-10 00:46, Tony Printezis wrote: >> Bengt, >> >> Here's the latest version: >> >> complete webrev: >> http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.1/webrev.all/ >> >> >> diffs from the previous one: >> http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.1/webrev.1.G1HRSFactory/ >> >> >> I created a factory method for HeapRegions, G1CH::new_heap_region(), >> combined the two expand_to_next_region() methods of HeapRegionSeq >> class into one (HRS::expand_by()), and updated G1CH::expand() >> accordingly. >> >> I did notice however that the HeapRegion constructor could fail to >> allocate space, if we're out of swap. Even though it's a marginal >> condition, JohnC2 recently put some effort to make the expansion code >> robust wrt to expansion failing. So, I made some additional changes >> to deal with the case where we cannot allocate more HeapRegions (the >> committed space will be shrunk accordingly). >> >> An additional minor change is that I renamed the HTS fields _end and >> _bottom to _heap_end and _heap_bottom to make what they represent a >> bit clearer. >> >> Better? >> >> Tony >> >> >> On 06/09/2011 07:11 AM, Tony Printezis wrote: >>> Bengt, >>> >>> On 06/09/2011 03:00 AM, Bengt Rutisson wrote: >>>>>> >>>>>> I will not persist on this. >>>>> >>>>> This is a good suggestion, Bengt, but I'll need to expose more >>>>> things from G1CH to the HeapRegionSeq to do that (i.e., the fields >>>>> needed in the HeapRegion constructor). And I just didn't want to >>>>> re-arrange things further. So, I think I'll leave it as is, i.e., >>>>> G1CH creating the regions and passing them on to HeapRegionSeq. >>>> >>>> >>>> That's fine. Let's leave it as it is. >>> >>> Actually (and I should have thought of that earlier!) I think I can >>> do this by creating a factory method for HeapRegion instances in >>> G1CH and calling it from HRS. So HRS will only have a single >>> expand() method (opposite to shrink()) and will deal with the rest >>> internally. Let me have a stab at it and I'll get back to you. >>> >>> Tony > From bengt.rutisson at oracle.com Fri Jun 10 06:23:15 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Fri, 10 Jun 2011 15:23:15 +0200 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DF208EA.803@oracle.com> References: <4DE62ADD.1040601@oracle.com> <4DE63A77.2030104@oracle.com> <4DE6BAB1.7090405@oracle.com> <4DEF2E87.80508@oracle.com> <4DEF8A88.5020704@oracle.com> <4DEFA9FB.8060705@oracle.com> <4DEFE2F0.6010204@oracle.com> <4DF02B5B.2090408@oracle.com> <4DF06FA6.2000708@oracle.com> <4DF0AA6D.9080901@oracle.com> <4DF14D56.70706@oracle.com> <4DF1B9FD.3070703@oracle.com> <4DF208EA.803@oracle.com> Message-ID: <4DF21AC3.6060001@oracle.com> Tony, >> In heapRegionSeq.cpp you added the include >> "gc_implementation/g1/heapRegionSets.hpp". I think that the one you >> want is "gc_implementation/g1/heapRegionSeq.hpp". > > Actually, ...Sets.hpp is correct as it's the one that FreeRegionList > declaration (needed by expand_by()). I see. I tried chaning ...Sets.hpp to ..Seq.hpp and it compiled, so I assumed that this was what you needed. But you are of course right about FreeRegionList. The ...Sets.hpp must be included through one of the other header files since I got it to compile. We should of course include it explicitly as you did. On the other hand I think we should also explicitly include the "own" header file. So, it would be good to add ...Seq.hpp as well. Thanks, Bengt > > Tony > >> Bengt >> >> On 2011-06-10 00:46, Tony Printezis wrote: >>> Bengt, >>> >>> Here's the latest version: >>> >>> complete webrev: >>> http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.1/webrev.all/ >>> >>> >>> diffs from the previous one: >>> http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.1/webrev.1.G1HRSFactory/ >>> >>> >>> I created a factory method for HeapRegions, G1CH::new_heap_region(), >>> combined the two expand_to_next_region() methods of HeapRegionSeq >>> class into one (HRS::expand_by()), and updated G1CH::expand() >>> accordingly. >>> >>> I did notice however that the HeapRegion constructor could fail to >>> allocate space, if we're out of swap. Even though it's a marginal >>> condition, JohnC2 recently put some effort to make the expansion >>> code robust wrt to expansion failing. So, I made some additional >>> changes to deal with the case where we cannot allocate more >>> HeapRegions (the committed space will be shrunk accordingly). >>> >>> An additional minor change is that I renamed the HTS fields _end and >>> _bottom to _heap_end and _heap_bottom to make what they represent a >>> bit clearer. >>> >>> Better? >>> >>> Tony >>> >>> >>> On 06/09/2011 07:11 AM, Tony Printezis wrote: >>>> Bengt, >>>> >>>> On 06/09/2011 03:00 AM, Bengt Rutisson wrote: >>>>>>> >>>>>>> I will not persist on this. >>>>>> >>>>>> This is a good suggestion, Bengt, but I'll need to expose more >>>>>> things from G1CH to the HeapRegionSeq to do that (i.e., the >>>>>> fields needed in the HeapRegion constructor). And I just didn't >>>>>> want to re-arrange things further. So, I think I'll leave it as >>>>>> is, i.e., G1CH creating the regions and passing them on to >>>>>> HeapRegionSeq. >>>>> >>>>> >>>>> That's fine. Let's leave it as it is. >>>> >>>> Actually (and I should have thought of that earlier!) I think I can >>>> do this by creating a factory method for HeapRegion instances in >>>> G1CH and calling it from HRS. So HRS will only have a single >>>> expand() method (opposite to shrink()) and will deal with the rest >>>> internally. Let me have a stab at it and I'll get back to you. >>>> >>>> Tony >> From tony.printezis at oracle.com Fri Jun 10 06:30:33 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Fri, 10 Jun 2011 09:30:33 -0400 Subject: CRR: 6804436: G1: heap region indices should be size_t (S) In-Reply-To: <4DF21AC3.6060001@oracle.com> References: <4DE62ADD.1040601@oracle.com> <4DE63A77.2030104@oracle.com> <4DE6BAB1.7090405@oracle.com> <4DEF2E87.80508@oracle.com> <4DEF8A88.5020704@oracle.com> <4DEFA9FB.8060705@oracle.com> <4DEFE2F0.6010204@oracle.com> <4DF02B5B.2090408@oracle.com> <4DF06FA6.2000708@oracle.com> <4DF0AA6D.9080901@oracle.com> <4DF14D56.70706@oracle.com> <4DF1B9FD.3070703@oracle.com> <4DF208EA.803@oracle.com> <4DF21AC3.6060001@oracle.com> Message-ID: <4DF21C79.1010600@oracle.com> Bengt, On 06/10/2011 09:23 AM, Bengt Rutisson wrote: > > Tony, > >>> In heapRegionSeq.cpp you added the include >>> "gc_implementation/g1/heapRegionSets.hpp". I think that the one you >>> want is "gc_implementation/g1/heapRegionSeq.hpp". >> >> Actually, ...Sets.hpp is correct as it's the one that FreeRegionList >> declaration (needed by expand_by()). Apologies for the bad grammar! It of course meant to say: "...that has the FreeRegionList..." > I see. I tried chaning ...Sets.hpp to ..Seq.hpp and it compiled, so I > assumed that this was what you needed. But you are of course right > about FreeRegionList. The ...Sets.hpp must be included through one of > the other header files since I got it to compile. We should of course > include it explicitly as you did. > > On the other hand I think we should also explicitly include the "own" > header file. So, it would be good to add ...Seq.hpp as well. The heapRegionSeq header file is being included from g1CollectedHeap.hpp. But, sure, I'll add it explicitly (in fact I think I'll need the .inline.hpp version). Tony > Thanks, > Bengt > >> >> Tony >> >>> Bengt >>> >>> On 2011-06-10 00:46, Tony Printezis wrote: >>>> Bengt, >>>> >>>> Here's the latest version: >>>> >>>> complete webrev: >>>> http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.1/webrev.all/ >>>> >>>> >>>> diffs from the previous one: >>>> http://cr.openjdk.java.net/~tonyp/7045330.6804436.7042285/webrev.1/webrev.1.G1HRSFactory/ >>>> >>>> >>>> I created a factory method for HeapRegions, >>>> G1CH::new_heap_region(), combined the two expand_to_next_region() >>>> methods of HeapRegionSeq class into one (HRS::expand_by()), and >>>> updated G1CH::expand() accordingly. >>>> >>>> I did notice however that the HeapRegion constructor could fail to >>>> allocate space, if we're out of swap. Even though it's a marginal >>>> condition, JohnC2 recently put some effort to make the expansion >>>> code robust wrt to expansion failing. So, I made some additional >>>> changes to deal with the case where we cannot allocate more >>>> HeapRegions (the committed space will be shrunk accordingly). >>>> >>>> An additional minor change is that I renamed the HTS fields _end >>>> and _bottom to _heap_end and _heap_bottom to make what they >>>> represent a bit clearer. >>>> >>>> Better? >>>> >>>> Tony >>>> >>>> >>>> On 06/09/2011 07:11 AM, Tony Printezis wrote: >>>>> Bengt, >>>>> >>>>> On 06/09/2011 03:00 AM, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> I will not persist on this. >>>>>>> >>>>>>> This is a good suggestion, Bengt, but I'll need to expose more >>>>>>> things from G1CH to the HeapRegionSeq to do that (i.e., the >>>>>>> fields needed in the HeapRegion constructor). And I just didn't >>>>>>> want to re-arrange things further. So, I think I'll leave it as >>>>>>> is, i.e., G1CH creating the regions and passing them on to >>>>>>> HeapRegionSeq. >>>>>> >>>>>> >>>>>> That's fine. Let's leave it as it is. >>>>> >>>>> Actually (and I should have thought of that earlier!) I think I >>>>> can do this by creating a factory method for HeapRegion instances >>>>> in G1CH and calling it from HRS. So HRS will only have a single >>>>> expand() method (opposite to shrink()) and will deal with the rest >>>>> internally. Let me have a stab at it and I'll get back to you. >>>>> >>>>> Tony >>> > From y.s.ramakrishna at oracle.com Fri Jun 10 10:39:53 2011 From: y.s.ramakrishna at oracle.com (Y. S. Ramakrishna) Date: Fri, 10 Jun 2011 10:39:53 -0700 Subject: request for review (XS): 7051430: CMS: ongoing CMS cycle should terminate abruptly to allow prompt JVM termination at exit Message-ID: <4DF256E9.8010506@oracle.com> 7051430: CMS: ongoing CMS cycle should terminate abruptly to allow prompt JVM termination at exit webrev: http://cr.openjdk.java.net/~ysr/7051430/webrev.0/ We don't really need to explicitly stop the CMS threads because the JVM is taken down at a terminal safepoint during which these threads are (terminally) inactive. thanks for your review! -- ramki From tony.printezis at oracle.com Fri Jun 10 11:27:31 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Fri, 10 Jun 2011 14:27:31 -0400 Subject: CRR (M/L - re-opened): 7039627: G1: avoid BOT updates for survivor allocations and dirty survivor regions incrementally In-Reply-To: <4DB6FAB9.9000705@oracle.com> References: <4DB6FAB9.9000705@oracle.com> Message-ID: <4DF26213.307@oracle.com> Hi, I'd like to re-open this for code review. I brought the workspace up-to-date. Here's the webrev: http://cr.openjdk.java.net/~tonyp/7039627/webrev.0/ Still need a couple of code review pretty please. :-) Tony Tony Printezis wrote: > Hi, > > This is the long-awaited :-) GC alloc region refactoring for G1 that > I've been working on for a while now (in the background). > > A lot of that allocation code during GC is very similar to the code > that manages the mutator allocation regions so we might as well share > it. We recently introduced the G1AllocRegion abstraction for mutator > alloc regions. Now we're going to re-use it for GC alloc regions too > (and remove a lot of replicated code in the process). > > The webrev is here: > > http://cr.openjdk.java.net/~tonyp/7039627/webrev.0/ > > (don't let the number of lines changed intimidate you; over 60% of > them correspond to code that was removed) > > Quick summary of the improvements: > > - Removed most of the code that manages the GC alloc regions and > replaced it with subclasses of G1AllocRegion (one for survivor > regions, the other for old regions). We now keep the two GC alloc > regions separate (before they could point to the same physical region) > as we have to handle them differently (do/don't do BOT updates, retire > them differently, etc.) and we don't want to have to add checks > everywhere. > - No BOT updates for survivor regions (the same way we do not need > them for mutator allocation regions). > - The cards of survivor regions are now dirtied incrementally (the > same way it's done for mutator allocation regions). > - We do not link the GC alloc regions into a list any more in order to > do any post-GC cleanup on them at the end of the GC. Instead, any > cleanup that needs to be done it's done as each region is retired. So > we save the extra post-GC step. > - Apart from not linking the GC alloc regions, I also removed the > "is_gc_alloc" flag as we do not need to check it any more (and this > saves us having to reset the flag at the end of the GC, which helped > in eliminating the post-GC cleanup step). > - The new code also fixes a subtle bug. In the old code, when a GC > thread allocated a new region is allowed other threads to allocate out > of it before attempting its allocation (the allocation that > essentially caused the new region to be allocated). But, that > allocation is not guaranteed to succeeded (given that other threads > might have meanwhile filled up the region) and this was not handled > correctly in the code. This would cause an unnecessary evacuation > failure. The new code fixes this bug as this case is handled correctly > in the G1AllocRegion class (the thread that allocates the region will > first satisfy its own allocation request before allowing anybody else > to allocate out of the new region). > > I'd like a couple of reviews please. :-) > > Tony > > From tony.printezis at oracle.com Fri Jun 10 12:28:54 2011 From: tony.printezis at oracle.com (tony.printezis at oracle.com) Date: Fri, 10 Jun 2011 19:28:54 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7045330: G1: Simplify/fix the HeapRegionSeq class; ... Message-ID: <20110610192858.B657847EBB@hg.openjdk.java.net> Changeset: c3f1170908be Author: tonyp Date: 2011-06-10 13:16 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/c3f1170908be 7045330: G1: Simplify/fix the HeapRegionSeq class 7042285: G1: native memory leak during humongous object allocation 6804436: G1: heap region indices should be size_t Summary: A series of fixes and improvements to the HeapRegionSeq class: a) replace the _regions growable array with a standard C array, b) avoid de-allocating / re-allocating HeapRegion instances when the heap shrinks / grows (fix for 7042285), c) introduce fast method to map address to HeapRegion via a "biased" array pointer, d) embed the _hrs object in G1CollectedHeap, instead of pointing to it via an indirection, e) assume that all the regions added to the HeapRegionSeq instance are contiguous, f) replace int's with size_t's for indexes (and expand that to HeapRegion as part of 6804436), g) remove unnecessary / unused methods, h) rename a couple of fields (_alloc_search_start and _seq_bottom), i) fix iterate_from() not to always start from index 0 irrespective of the region passed to it, j) add a verification method to check the HeapRegionSeq assumptions, k) always call the wrappers for _hrs.iterate(), _hrs_length(), and _hrs.at() from G1CollectedHeap, not those methods di rectly, and l) unify the code that expands the sequence (by either re-using or creating a new HeapRegion) and make it robust wrt to a HeapRegion allocation failing. Reviewed-by: stefank, johnc, brutisso ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.inline.hpp ! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp ! src/share/vm/gc_implementation/g1/heapRegion.cpp ! src/share/vm/gc_implementation/g1/heapRegion.hpp ! src/share/vm/gc_implementation/g1/heapRegionRemSet.cpp ! src/share/vm/gc_implementation/g1/heapRegionSeq.cpp ! src/share/vm/gc_implementation/g1/heapRegionSeq.hpp ! src/share/vm/gc_implementation/g1/heapRegionSeq.inline.hpp ! src/share/vm/gc_implementation/g1/sparsePRT.cpp From john.coomes at oracle.com Fri Jun 10 17:16:40 2011 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Sat, 11 Jun 2011 00:16:40 +0000 Subject: hg: jdk7/hotspot-gc/langtools: 5 new changesets Message-ID: <20110611001658.E9D6D47EF9@hg.openjdk.java.net> Changeset: 6bb526ccf5ff Author: mcimadamore Date: 2011-05-23 11:55 +0100 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/6bb526ccf5ff 7046348: Regression: javac complains of missing classfile for a seemingly unrelated interface Summary: Types.implementation forces unnecessary symbol completion on superinterfaces of a given type Reviewed-by: jjg ! src/share/classes/com/sun/tools/javac/code/Symbol.java ! src/share/classes/com/sun/tools/javac/code/Types.java ! src/share/classes/com/sun/tools/javac/comp/Check.java + test/tools/javac/scope/7046348/EagerInterfaceCompletionTest.java Changeset: 6211df69f7e0 Author: jeff Date: 2011-05-27 15:02 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/6211df69f7e0 7045697: JDK7 THIRD PARTY README update Reviewed-by: lana ! THIRD_PARTY_README Changeset: 6762754eb7c0 Author: jjg Date: 2011-06-01 11:25 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/6762754eb7c0 7042623: Regression: javac silently crash when attributing non-existent annotation Reviewed-by: mcimadamore ! src/share/classes/com/sun/tools/javac/comp/Check.java + test/tools/javac/T7042623.java + test/tools/javac/T7042623.out Changeset: c455e2ae5c93 Author: lana Date: 2011-06-02 13:38 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/c455e2ae5c93 Merge Changeset: 9ff91d0e7154 Author: schien Date: 2011-06-07 14:01 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/9ff91d0e7154 Added tag jdk7-b145 for changeset c455e2ae5c93 ! .hgtags From yumin.qi at sun.com Sat Jun 11 01:28:58 2011 From: yumin.qi at sun.com (yumin.qi at sun.com) Date: Sat, 11 Jun 2011 08:28:58 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 2 new changesets Message-ID: <20110611082906.D6DDB47F19@hg.openjdk.java.net> Changeset: 2a241e764894 Author: minqi Date: 2011-06-10 15:08 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/2a241e764894 6941923: RFE: Handling large log files produced by long running Java Applications Summary: supply optinal flags to realize gc log rotation Reviewed-by: ysr, jwilhelm ! src/share/vm/gc_implementation/shared/concurrentGCThread.cpp ! src/share/vm/gc_implementation/shared/concurrentGCThread.hpp ! src/share/vm/runtime/arguments.cpp ! src/share/vm/runtime/atomic.cpp ! src/share/vm/runtime/atomic.hpp ! src/share/vm/runtime/globals.hpp ! src/share/vm/runtime/safepoint.cpp ! src/share/vm/utilities/ostream.cpp ! src/share/vm/utilities/ostream.hpp + test/gc/6941923/test6941923.sh Changeset: 42df21744b50 Author: minqi Date: 2011-06-10 15:44 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/42df21744b50 Merge From rainer.jung at kippdata.de Sat Jun 11 02:01:33 2011 From: rainer.jung at kippdata.de (Rainer Jung) Date: Sat, 11 Jun 2011 11:01:33 +0200 Subject: Request for review: 6941923: RFE: Handling large logfiles produced by long running Java Applications In-Reply-To: <4DD6870E.6060600@oracle.com> References: <4DB9D9BB.6010905@oracle.com> <4DBAAE1E.20904@oracle.com> <4DBAF21D.8010502@oracle.com> <4DBEBA1D.5090106@oracle.com> <4DBECA3F.8070307@oracle.com> <4DBECED4.3050207@oracle.com> <4DBF449E.6070406@oracle.com> <4DBFC525.7070809@oracle.com> <4DC026F7.8090405@oracle.com> <4DC23FCB.3050408@oracle.com> <4DC2C46D.9000001@oracle.com> <4DC2D1C6.5050707@oracle.com> <4DC47B39.3090709@oracle.com><4DD16577.6090904@oracle.com> <4DD6671C.9080802@oracle.com> <4DD6870E.6060600@oracle.com> Message-ID: <4DF32EED.2060304@kippdata.de> When I looked at the published change Changeset: 2a241e764894 Author: minqi Date: 2011-06-10 15:08 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/2a241e764894 6941923: RFE: Handling large log files produced by long running Java Applications Summary: supply optinal flags to realize gc log rotation Reviewed-by: ysr, jwilhelm I noticed a flaw in a log message. File src/share/vm/runtime/arguments.cpp contains the new block: + jio_fprintf(defaultStream::output_stream(), + "To enable GC log rotation, use -Xloggc: -XX:+UseGCLogRotaion -XX:NumberOfGCLogFiles= -XX:GCLogFileSize=\n" + "where num_of_file > 0 and num_of_size > 0\n" + "GC log rotation is turned off\n"); Note that it says "-XX:+UseGCLogRotaion" instead of "-XX:+UseGCLogFileRotation". The name of the switch contains a typo *and* is wrong. Sorry to not have seen this earlier (did not review the webrev). Regards, Rainer From yumin.qi at oracle.com Mon Jun 13 08:46:13 2011 From: yumin.qi at oracle.com (yumin.qi at oracle.com) Date: Mon, 13 Jun 2011 08:46:13 -0700 Subject: Request for review: 6941923: RFE: Handling large logfiles produced by long running Java Applications In-Reply-To: <4DF32EED.2060304@kippdata.de> References: <4DB9D9BB.6010905@oracle.com> <4DBAAE1E.20904@oracle.com> <4DBAF21D.8010502@oracle.com> <4DBEBA1D.5090106@oracle.com> <4DBECA3F.8070307@oracle.com> <4DBECED4.3050207@oracle.com> <4DBF449E.6070406@oracle.com> <4DBFC525.7070809@oracle.com> <4DC026F7.8090405@oracle.com> <4DC23FCB.3050408@oracle.com> <4DC2C46D.9000001@oracle.com> <4DC2D1C6.5050707@oracle.com> <4DC47B39.3090709@oracle.com><4DD16577.6090904@oracle.com> <4DD6671C.9080802@oracle.com> <4DD6870E.6060600@oracle.com> <4DF32EED.2060304@kippdata.de> Message-ID: <4DF630C5.5050505@oracle.com> Rainer, Thanks for your catch! Yes, there is a typo in the output message. It will be corrected in next putback in some GC CR. Codereviewers and me did not notice the message output in this printout. Thanks again. Yumin On 6/11/2011 2:01 AM, Rainer Jung wrote: > When I looked at the published change > > Changeset: 2a241e764894 > Author: minqi > Date: 2011-06-10 15:08 -0700 > URL: > http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/2a241e764894 > > 6941923: RFE: Handling large log files produced by long running Java > Applications > Summary: supply optinal flags to realize gc log rotation > Reviewed-by: ysr, jwilhelm > > > I noticed a flaw in a log message. > > File src/share/vm/runtime/arguments.cpp contains the new block: > > + jio_fprintf(defaultStream::output_stream(), > + "To enable GC log rotation, use -Xloggc: > -XX:+UseGCLogRotaion -XX:NumberOfGCLogFiles= > -XX:GCLogFileSize=\n" > + "where num_of_file> 0 and num_of_size> 0\n" > + "GC log rotation is turned off\n"); > > Note that it says "-XX:+UseGCLogRotaion" instead of > "-XX:+UseGCLogFileRotation". The name of the switch contains a typo > *and* is wrong. > > Sorry to not have seen this earlier (did not review the webrev). > > Regards, > > Rainer From john.cuthbertson at oracle.com Mon Jun 13 10:16:41 2011 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Mon, 13 Jun 2011 10:16:41 -0700 Subject: RFR (XS): 7045751 G1: +ExplicitGCInvokesConcurrent causes excessive single region evacuation pauses Message-ID: <4DF645F9.6000700@oracle.com> Hi Everyone, Can I have a couple of volunteers look of the small fix for this CR? The webrev can be found at: http://cr.openjdk.java.net/~johnc/7045751/webrev.0/ The issue here was that if we had multiple threads executing a System.gc() (with +ExplicitGCInvokesConcurrent) we would have a thundering herd trying to start an initial mark evacuation pause. One thread would win, do the pause, and start the marking cycle, and block until the cycle complete. The next thread in would see that a marking cycle was in progress and do a regular evacuation pause. If none of the mutators (that were not trying to do a System.gc() call) did not manage to run, the collection set for this second pause would comprise of only the survivors from the first pause. The second thread would then block until the marking cycle was complete. And so on for the remaining threads requesting a System.gc(). The solution is to skip the evacuation pause if we fail to force it to be an initial mark pause and just block (as normal) until the existing cycle completes. Tested with: GC test suite with and without ExplicitGCInvokesConcurrent. Thanks, JohnC From tony.printezis at oracle.com Mon Jun 13 10:16:51 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Mon, 13 Jun 2011 13:16:51 -0400 Subject: CRR (S): 7046182: G1: remove unnecessary iterations over the collection set Message-ID: <4DF64603.3060702@oracle.com> Hi all, Could I have a couple of code reviews for this change? http://syros.us.oracle.com/local_ws/hotspot-g1-cset-iter/webrev/ Basically, we (unnecessarily) iterate over the collection set a couple of times during each GC to reset two fields on the RSet data structure. It's better to make sure that those fields are reset when the region is freed (so it's "ready to go" when it's re-allocated) to avoid those two iterations. I added some extra code to assert that these two fields are in the correct state before each GC and I also added an extra verification step on the master free list to also check that condition on all the regions on it (since the two fields in question should be reset before the region is added to the master free list). I'm piggy-backing the removal of the G1_REM_SET_LOGGING code on this (it's kinda related as it also iterates over the collection set - so, we remove more collection set iterations!). We haven't used it for a while and it's helpfulness is marginal. Thanks, Tony From tony.printezis at oracle.com Mon Jun 13 10:22:19 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Mon, 13 Jun 2011 13:22:19 -0400 Subject: CRR (S): 7046182: G1: remove unnecessary iterations over the collection set In-Reply-To: <4DF64603.3060702@oracle.com> References: <4DF64603.3060702@oracle.com> Message-ID: <4DF6474B.3010601@oracle.com> ..and of course the webrev url is wrong. This is the right one: http://cr.openjdk.java.net/~tonyp/7046182/webrev.0/ Apologies! Tony Tony Printezis wrote: > Hi all, > > Could I have a couple of code reviews for this change? > > http://syros.us.oracle.com/local_ws/hotspot-g1-cset-iter/webrev/ > > Basically, we (unnecessarily) iterate over the collection set a couple > of times during each GC to reset two fields on the RSet data > structure. It's better to make sure that those fields are reset when > the region is freed (so it's "ready to go" when it's re-allocated) to > avoid those two iterations. > > I added some extra code to assert that these two fields are in the > correct state before each GC and I also added an extra verification > step on the master free list to also check that condition on all the > regions on it (since the two fields in question should be reset before > the region is added to the master free list). > > I'm piggy-backing the removal of the G1_REM_SET_LOGGING code on this > (it's kinda related as it also iterates over the collection set - so, > we remove more collection set iterations!). We haven't used it for a > while and it's helpfulness is marginal. > > Thanks, > > Tony > From john.cuthbertson at oracle.com Mon Jun 13 12:00:00 2011 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Mon, 13 Jun 2011 12:00:00 -0700 Subject: RFR (XS): 7045751 G1: +ExplicitGCInvokesConcurrent causes excessive single region evacuation pauses In-Reply-To: <4DF653A4.6090104@oracle.com> References: <4DF645F9.6000700@oracle.com> <4DF653A4.6090104@oracle.com> Message-ID: <4DF65E30.8090009@oracle.com> Hi Ramki, Using code similar to CMS, the requesting thread is blocked until the # of full collections completed is incremented and the FullGCCount_lock is notified. This takes place at the very end of the marking cycle - after cleanup and and after clearing the next mark bitmap. JohnC On 06/13/11 11:15, Y. Srinivas Ramakrishna wrote: > Looks good. One question: we return when the marking cycle is > complete, or when the marking cycle _and_ the succeeding > concurrent clean-up is also complete? > > - ramki > > On 6/13/2011 10:16 AM, John Cuthbertson wrote: >> Hi Everyone, >> >> Can I have a couple of volunteers look of the small fix for this CR? >> The webrev can be found at: >> http://cr.openjdk.java.net/~johnc/7045751/webrev.0/ >> >> The issue here was that if we had multiple threads executing a >> System.gc() (with >> +ExplicitGCInvokesConcurrent) we would have a thundering herd trying >> to start an initial mark >> evacuation pause. One thread would win, do the pause, and start the >> marking cycle, and block until >> the cycle complete. The next thread in would see that a marking cycle >> was in progress and do a >> regular evacuation pause. If none of the mutators (that were not >> trying to do a System.gc() call) >> did not manage to run, the collection set for this second pause would >> comprise of only the survivors >> from the first pause. The second thread would then block until the >> marking cycle was complete. And >> so on for the remaining threads requesting a System.gc(). >> >> The solution is to skip the evacuation pause if we fail to force it >> to be an initial mark pause and >> just block (as normal) until the existing cycle completes. >> >> Tested with: GC test suite with and without ExplicitGCInvokesConcurrent. >> >> Thanks, >> >> JohnC > From y.s.ramakrishna at oracle.com Mon Jun 13 12:10:59 2011 From: y.s.ramakrishna at oracle.com (Y. Srinivas Ramakrishna) Date: Mon, 13 Jun 2011 12:10:59 -0700 Subject: RFR (XS): 7045751 G1: +ExplicitGCInvokesConcurrent causes excessive single region evacuation pauses In-Reply-To: <4DF65E30.8090009@oracle.com> References: <4DF645F9.6000700@oracle.com> <4DF653A4.6090104@oracle.com> <4DF65E30.8090009@oracle.com> Message-ID: <4DF660C3.4030801@oracle.com> Very good; thanks! Reviewed. -- ramki On 6/13/2011 12:00 PM, John Cuthbertson wrote: > Hi Ramki, > > Using code similar to CMS, the requesting thread is blocked until the # of full collections > completed is incremented and the FullGCCount_lock is notified. This takes place at the very end of > the marking cycle - after cleanup and and after clearing the next mark bitmap. > > JohnC > > On 06/13/11 11:15, Y. Srinivas Ramakrishna wrote: >> Looks good. One question: we return when the marking cycle is >> complete, or when the marking cycle _and_ the succeeding >> concurrent clean-up is also complete? >> >> - ramki >> >> On 6/13/2011 10:16 AM, John Cuthbertson wrote: >>> Hi Everyone, >>> >>> Can I have a couple of volunteers look of the small fix for this CR? The webrev can be found at: >>> http://cr.openjdk.java.net/~johnc/7045751/webrev.0/ >>> >>> The issue here was that if we had multiple threads executing a System.gc() (with >>> +ExplicitGCInvokesConcurrent) we would have a thundering herd trying to start an initial mark >>> evacuation pause. One thread would win, do the pause, and start the marking cycle, and block until >>> the cycle complete. The next thread in would see that a marking cycle was in progress and do a >>> regular evacuation pause. If none of the mutators (that were not trying to do a System.gc() call) >>> did not manage to run, the collection set for this second pause would comprise of only the survivors >>> from the first pause. The second thread would then block until the marking cycle was complete. And >>> so on for the remaining threads requesting a System.gc(). >>> >>> The solution is to skip the evacuation pause if we fail to force it to be an initial mark pause and >>> just block (as normal) until the existing cycle completes. >>> >>> Tested with: GC test suite with and without ExplicitGCInvokesConcurrent. >>> >>> Thanks, >>> >>> JohnC >> > From tony.printezis at oracle.com Mon Jun 13 12:10:40 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Mon, 13 Jun 2011 15:10:40 -0400 Subject: RFR (XS): 7045751 G1: +ExplicitGCInvokesConcurrent causes excessive single region evacuation pauses In-Reply-To: <4DF65E30.8090009@oracle.com> References: <4DF645F9.6000700@oracle.com> <4DF653A4.6090104@oracle.com> <4DF65E30.8090009@oracle.com> Message-ID: <4DF660B0.5090603@oracle.com> John, This is correct. Tony John Cuthbertson wrote: > Hi Ramki, > > Using code similar to CMS, the requesting thread is blocked until the > # of full collections completed is incremented and the > FullGCCount_lock is notified. This takes place at the very end of the > marking cycle - after cleanup and and after clearing the next mark > bitmap. > > JohnC > > On 06/13/11 11:15, Y. Srinivas Ramakrishna wrote: >> Looks good. One question: we return when the marking cycle is >> complete, or when the marking cycle _and_ the succeeding >> concurrent clean-up is also complete? >> >> - ramki >> >> On 6/13/2011 10:16 AM, John Cuthbertson wrote: >>> Hi Everyone, >>> >>> Can I have a couple of volunteers look of the small fix for this CR? >>> The webrev can be found at: >>> http://cr.openjdk.java.net/~johnc/7045751/webrev.0/ >>> >>> The issue here was that if we had multiple threads executing a >>> System.gc() (with >>> +ExplicitGCInvokesConcurrent) we would have a thundering herd trying >>> to start an initial mark >>> evacuation pause. One thread would win, do the pause, and start the >>> marking cycle, and block until >>> the cycle complete. The next thread in would see that a marking >>> cycle was in progress and do a >>> regular evacuation pause. If none of the mutators (that were not >>> trying to do a System.gc() call) >>> did not manage to run, the collection set for this second pause >>> would comprise of only the survivors >>> from the first pause. The second thread would then block until the >>> marking cycle was complete. And >>> so on for the remaining threads requesting a System.gc(). >>> >>> The solution is to skip the evacuation pause if we fail to force it >>> to be an initial mark pause and >>> just block (as normal) until the existing cycle completes. >>> >>> Tested with: GC test suite with and without >>> ExplicitGCInvokesConcurrent. >>> >>> Thanks, >>> >>> JohnC >> > From y.s.ramakrishna at ORACLE.COM Mon Jun 13 11:15:00 2011 From: y.s.ramakrishna at ORACLE.COM (Y. Srinivas Ramakrishna) Date: Mon, 13 Jun 2011 11:15:00 -0700 Subject: RFR (XS): 7045751 G1: +ExplicitGCInvokesConcurrent causes excessive single region evacuation pauses In-Reply-To: <4DF645F9.6000700@oracle.com> References: <4DF645F9.6000700@oracle.com> Message-ID: <4DF653A4.6090104@oracle.com> Looks good. One question: we return when the marking cycle is complete, or when the marking cycle _and_ the succeeding concurrent clean-up is also complete? - ramki On 6/13/2011 10:16 AM, John Cuthbertson wrote: > Hi Everyone, > > Can I have a couple of volunteers look of the small fix for this CR? The webrev can be found at: > http://cr.openjdk.java.net/~johnc/7045751/webrev.0/ > > The issue here was that if we had multiple threads executing a System.gc() (with > +ExplicitGCInvokesConcurrent) we would have a thundering herd trying to start an initial mark > evacuation pause. One thread would win, do the pause, and start the marking cycle, and block until > the cycle complete. The next thread in would see that a marking cycle was in progress and do a > regular evacuation pause. If none of the mutators (that were not trying to do a System.gc() call) > did not manage to run, the collection set for this second pause would comprise of only the survivors > from the first pause. The second thread would then block until the marking cycle was complete. And > so on for the remaining threads requesting a System.gc(). > > The solution is to skip the evacuation pause if we fail to force it to be an initial mark pause and > just block (as normal) until the existing cycle completes. > > Tested with: GC test suite with and without ExplicitGCInvokesConcurrent. > > Thanks, > > JohnC From y.s.ramakrishna at oracle.com Mon Jun 13 12:27:09 2011 From: y.s.ramakrishna at oracle.com (y.s.ramakrishna at oracle.com) Date: Mon, 13 Jun 2011 19:27:09 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7051430: CMS: ongoing CMS cycle should terminate abruptly to allow prompt JVM termination at exit Message-ID: <20110613192715.BEEFB47FDE@hg.openjdk.java.net> Changeset: ef2d1b8f2dd4 Author: ysr Date: 2011-06-13 09:58 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/ef2d1b8f2dd4 7051430: CMS: ongoing CMS cycle should terminate abruptly to allow prompt JVM termination at exit Summary: It turns out that there is no need to explicitly stop CMS since the JVM is taken down at a terminal safepoint during which CMS threads are (terminally) inactive. This will need to be revised if and when we evolve in the future to a point where we allow JVM reincarnation in the same process, but those changes will be much more sweeping than just terminating CMS threads. The unused ::stop() methods will be removed in a separate CR. Also include in this CR is the fix for a small typo in the spelling of UseGCLogFileRotation in a message in arguments.cpp, brought to our attention by Rainer Jung and reviewed by minqi. Reviewed-by: johnc, jwilhelm ! src/share/vm/runtime/arguments.cpp ! src/share/vm/runtime/java.cpp ! src/share/vm/runtime/thread.cpp From bengt.rutisson at oracle.com Mon Jun 13 13:45:06 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 13 Jun 2011 22:45:06 +0200 Subject: Review request: 6918185 Remove unused code for lost card-marking optimization in BacktraceBuilder (S) Message-ID: <4DF676D2.1020400@oracle.com> Hi Runtime and GC teams, Could I have a couple of reviews for this simple change to remove some dead code? I am sending this request to both the runtime and GC teams, since it is a change in runtime code but I will be pushing it through hotspot-gc. I discussed this with Ramki and Coleen. This is not an optimization that will be implemented any time soon and Coleen was intending to close the CR as WNF at some point. However, there is some dead code left from the orginal fix, so I re-purposed the CR to remove this dead code. Webrev: http://cr.openjdk.java.net/~brutisso/CR6918185-b/webrev.0/ CR: 6918185 Remove unused code for lost card-marking optimization in BacktraceBuilder http://monaco.sfbay.sun.com/detail.jsf?cr=6918185 Thanks, Bengt From David.Holmes at oracle.com Mon Jun 13 15:51:05 2011 From: David.Holmes at oracle.com (David Holmes) Date: Tue, 14 Jun 2011 08:51:05 +1000 Subject: Review request: 6918185 Remove unused code for lost card-marking optimization in BacktraceBuilder (S) In-Reply-To: <4DF676D2.1020400@oracle.com> References: <4DF676D2.1020400@oracle.com> Message-ID: <4DF69459.90801@oracle.com> Hi Bengt, Looks okay to me. David Bengt Rutisson said the following on 06/14/11 06:45: > > Hi Runtime and GC teams, > > Could I have a couple of reviews for this simple change to remove some > dead code? > > I am sending this request to both the runtime and GC teams, since it is > a change in runtime code but I will be pushing it through hotspot-gc. > > I discussed this with Ramki and Coleen. This is not an optimization that > will be implemented any time soon and Coleen was intending to close the > CR as WNF at some point. However, there is some dead code left from the > orginal fix, so I re-purposed the CR to remove this dead code. > > Webrev: > http://cr.openjdk.java.net/~brutisso/CR6918185-b/webrev.0/ > > CR: > 6918185 Remove unused code for lost card-marking optimization in > BacktraceBuilder > http://monaco.sfbay.sun.com/detail.jsf?cr=6918185 > > Thanks, > Bengt > From y.s.ramakrishna at oracle.com Mon Jun 13 16:04:40 2011 From: y.s.ramakrishna at oracle.com (Y. Srinivas Ramakrishna) Date: Mon, 13 Jun 2011 16:04:40 -0700 Subject: Review request: 6918185 Remove unused code for lost card-marking optimization in BacktraceBuilder (S) In-Reply-To: <4DF676D2.1020400@oracle.com> References: <4DF676D2.1020400@oracle.com> Message-ID: <4DF69788.4090306@oracle.com> Looks good. -- ramki On 6/13/2011 1:45 PM, Bengt Rutisson wrote: > > Hi Runtime and GC teams, > > Could I have a couple of reviews for this simple change to remove some dead code? > > I am sending this request to both the runtime and GC teams, since it is a change in runtime code but > I will be pushing it through hotspot-gc. > > I discussed this with Ramki and Coleen. This is not an optimization that will be implemented any > time soon and Coleen was intending to close the CR as WNF at some point. However, there is some dead > code left from the orginal fix, so I re-purposed the CR to remove this dead code. > > Webrev: > http://cr.openjdk.java.net/~brutisso/CR6918185-b/webrev.0/ > > CR: > 6918185 Remove unused code for lost card-marking optimization in BacktraceBuilder > http://monaco.sfbay.sun.com/detail.jsf?cr=6918185 > > Thanks, > Bengt > From bengt.rutisson at oracle.com Tue Jun 14 00:21:12 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 14 Jun 2011 09:21:12 +0200 Subject: Review request: 6918185 Remove unused code for lost card-marking optimization in BacktraceBuilder (S) In-Reply-To: <4DF6A295.1090600@oracle.com> References: <4DF676D2.1020400@oracle.com> <4DF6A295.1090600@oracle.com> Message-ID: <4DF70BE8.2080706@oracle.com> David, Ramki and Coleen, Thanks for your reviews. I'm all set to push this now. Bengt On 2011-06-14 01:51, Coleen Phillimore wrote: > Looks good. Thanks for doing this cleanup! > Coleen > > On 6/13/2011 4:45 PM, Bengt Rutisson wrote: >> >> Hi Runtime and GC teams, >> >> Could I have a couple of reviews for this simple change to remove >> some dead code? >> >> I am sending this request to both the runtime and GC teams, since it >> is a change in runtime code but I will be pushing it through hotspot-gc. >> >> I discussed this with Ramki and Coleen. This is not an optimization >> that will be implemented any time soon and Coleen was intending to >> close the CR as WNF at some point. However, there is some dead code >> left from the orginal fix, so I re-purposed the CR to remove this >> dead code. >> >> Webrev: >> http://cr.openjdk.java.net/~brutisso/CR6918185-b/webrev.0/ >> >> CR: >> 6918185 Remove unused code for lost card-marking optimization in >> BacktraceBuilder >> http://monaco.sfbay.sun.com/detail.jsf?cr=6918185 >> >> Thanks, >> Bengt >> From bengt.rutisson at oracle.com Tue Jun 14 01:23:02 2011 From: bengt.rutisson at oracle.com (bengt.rutisson at oracle.com) Date: Tue, 14 Jun 2011 08:23:02 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 6918185: Remove unused code for lost card-marking optimization in BacktraceBuilder Message-ID: <20110614082306.8B06F48000@hg.openjdk.java.net> Changeset: 74cd10898bea Author: brutisso Date: 2011-06-13 13:48 +0200 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/74cd10898bea 6918185: Remove unused code for lost card-marking optimization in BacktraceBuilder Summary: Removed dead code Reviewed-by: ysr, coleenp, dholmes ! src/share/vm/classfile/javaClasses.cpp From tony.printezis at oracle.com Tue Jun 14 10:38:02 2011 From: tony.printezis at oracle.com (tony.printezis at oracle.com) Date: Tue, 14 Jun 2011 17:38:02 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7046558: G1: concurrent marking optimizations Message-ID: <20110614173807.041DD47015@hg.openjdk.java.net> Changeset: 842b840e67db Author: tonyp Date: 2011-06-14 10:33 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/842b840e67db 7046558: G1: concurrent marking optimizations Summary: Some optimizations to improve the concurrent marking phase: specialize the main oop closure, make sure a few methods in the fast path are properly inlined, a few more bits and pieces, and some cosmetic fixes. Reviewed-by: stefank, johnc ! src/share/vm/gc_implementation/g1/concurrentMark.cpp ! src/share/vm/gc_implementation/g1/concurrentMark.hpp + src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp ! src/share/vm/gc_implementation/g1/g1OopClosures.hpp ! src/share/vm/gc_implementation/g1/g1OopClosures.inline.hpp ! src/share/vm/gc_implementation/g1/g1_specialized_oop_closures.hpp ! src/share/vm/utilities/bitMap.hpp From john.cuthbertson at oracle.com Tue Jun 14 13:15:17 2011 From: john.cuthbertson at oracle.com (john.cuthbertson at oracle.com) Date: Tue, 14 Jun 2011 20:15:17 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7004681: G1: Extend marking verification to Full GCs Message-ID: <20110614201521.DB6414701C@hg.openjdk.java.net> Changeset: 6747fd0512e0 Author: johnc Date: 2011-06-14 11:01 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/6747fd0512e0 7004681: G1: Extend marking verification to Full GCs Summary: Perform a heap verification after the first phase of G1's full GC using objects' mark words to determine liveness. The third parameter of the heap verification routines, which was used in G1 to determine which marking bitmap to use in liveness calculations, has been changed from a boolean to an enum with values defined for using the mark word, and the 'prev' and 'next' bitmaps. Reviewed-by: tonyp, ysr ! src/share/vm/gc_implementation/g1/concurrentMark.cpp ! src/share/vm/gc_implementation/g1/concurrentMark.hpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp ! src/share/vm/gc_implementation/g1/g1MarkSweep.cpp ! src/share/vm/gc_implementation/g1/heapRegion.cpp ! src/share/vm/gc_implementation/g1/heapRegion.hpp ! src/share/vm/gc_implementation/parallelScavenge/parallelScavengeHeap.cpp ! src/share/vm/gc_implementation/parallelScavenge/parallelScavengeHeap.hpp ! src/share/vm/gc_interface/collectedHeap.hpp ! src/share/vm/memory/genCollectedHeap.cpp ! src/share/vm/memory/genCollectedHeap.hpp ! src/share/vm/memory/universe.cpp ! src/share/vm/memory/universe.hpp From tony.printezis at oracle.com Tue Jun 14 15:35:43 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Tue, 14 Jun 2011 18:35:43 -0400 Subject: CRR (M/L - re-opened): 7039627: G1: avoid BOT updates for survivor allocations and dirty survivor regions incrementally In-Reply-To: <4DF26213.307@oracle.com> References: <4DB6FAB9.9000705@oracle.com> <4DF26213.307@oracle.com> Message-ID: <4DF7E23F.6050802@oracle.com> Hi, Latest workspace after merging with the last few changesets that we've pushed recently: http://cr.openjdk.java.net/~tonyp/7039627/webrev.1/ (if you started looking at the first version you don't need to switch to this one, the contents should be mostly the same) Tony Tony Printezis wrote: > Hi, > > I'd like to re-open this for code review. I brought the workspace > up-to-date. Here's the webrev: > > http://cr.openjdk.java.net/~tonyp/7039627/webrev.0/ > > Still need a couple of code review pretty please. :-) > > Tony > > Tony Printezis wrote: >> Hi, >> >> This is the long-awaited :-) GC alloc region refactoring for G1 that >> I've been working on for a while now (in the background). >> >> A lot of that allocation code during GC is very similar to the code >> that manages the mutator allocation regions so we might as well share >> it. We recently introduced the G1AllocRegion abstraction for mutator >> alloc regions. Now we're going to re-use it for GC alloc regions too >> (and remove a lot of replicated code in the process). >> >> The webrev is here: >> >> http://cr.openjdk.java.net/~tonyp/7039627/webrev.0/ >> >> (don't let the number of lines changed intimidate you; over 60% of >> them correspond to code that was removed) >> >> Quick summary of the improvements: >> >> - Removed most of the code that manages the GC alloc regions and >> replaced it with subclasses of G1AllocRegion (one for survivor >> regions, the other for old regions). We now keep the two GC alloc >> regions separate (before they could point to the same physical >> region) as we have to handle them differently (do/don't do BOT >> updates, retire them differently, etc.) and we don't want to have to >> add checks everywhere. >> - No BOT updates for survivor regions (the same way we do not need >> them for mutator allocation regions). >> - The cards of survivor regions are now dirtied incrementally (the >> same way it's done for mutator allocation regions). >> - We do not link the GC alloc regions into a list any more in order >> to do any post-GC cleanup on them at the end of the GC. Instead, any >> cleanup that needs to be done it's done as each region is retired. So >> we save the extra post-GC step. >> - Apart from not linking the GC alloc regions, I also removed the >> "is_gc_alloc" flag as we do not need to check it any more (and this >> saves us having to reset the flag at the end of the GC, which helped >> in eliminating the post-GC cleanup step). >> - The new code also fixes a subtle bug. In the old code, when a GC >> thread allocated a new region is allowed other threads to allocate >> out of it before attempting its allocation (the allocation that >> essentially caused the new region to be allocated). But, that >> allocation is not guaranteed to succeeded (given that other threads >> might have meanwhile filled up the region) and this was not handled >> correctly in the code. This would cause an unnecessary evacuation >> failure. The new code fixes this bug as this case is handled >> correctly in the G1AllocRegion class (the thread that allocates the >> region will first satisfy its own allocation request before allowing >> anybody else to allocate out of the new region). >> >> I'd like a couple of reviews please. :-) >> >> Tony >> >> > From coleen.phillimore at oracle.com Mon Jun 13 16:51:49 2011 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 13 Jun 2011 19:51:49 -0400 Subject: Review request: 6918185 Remove unused code for lost card-marking optimization in BacktraceBuilder (S) In-Reply-To: <4DF676D2.1020400@oracle.com> References: <4DF676D2.1020400@oracle.com> Message-ID: <4DF6A295.1090600@oracle.com> Looks good. Thanks for doing this cleanup! Coleen On 6/13/2011 4:45 PM, Bengt Rutisson wrote: > > Hi Runtime and GC teams, > > Could I have a couple of reviews for this simple change to remove some > dead code? > > I am sending this request to both the runtime and GC teams, since it > is a change in runtime code but I will be pushing it through hotspot-gc. > > I discussed this with Ramki and Coleen. This is not an optimization > that will be implemented any time soon and Coleen was intending to > close the CR as WNF at some point. However, there is some dead code > left from the orginal fix, so I re-purposed the CR to remove this dead > code. > > Webrev: > http://cr.openjdk.java.net/~brutisso/CR6918185-b/webrev.0/ > > CR: > 6918185 Remove unused code for lost card-marking optimization in > BacktraceBuilder > http://monaco.sfbay.sun.com/detail.jsf?cr=6918185 > > Thanks, > Bengt > From alexey.ragozin at gmail.com Wed Jun 15 01:03:26 2011 From: alexey.ragozin at gmail.com (Alexey Ragozin) Date: Wed, 15 Jun 2011 12:03:26 +0400 Subject: Optimizing card table scanning in CMS collector Message-ID: Hi, Recently I was analyzing CMS GC pause times on JVM with 32Gb of heap (using Oracle Coherence node as sample application). It seems like young collection pause time is totally dominated by time required to scan card table (I suppose size of table should be 64Mb in this case). I believe time to scan card table could be cut significantly at price of slightly more complex write-barrier. By introducing super-cards collector can avoid scanning whole ranges of card table. I would like to implement POC to prove reduction of young collection pause (also it should probably reduce CMS remark pause time). I need an advice to locate right places for modification in code base (I?m not familiar with it). I thing I can ignore JIT for sake of POC (running JVM in interpreter mode). So I need to modify write barrier used in interpreter and card table scanning procedure. Thank you for advice. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110615/1fbd03af/attachment.html From bengt.rutisson at oracle.com Wed Jun 15 02:15:59 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 15 Jun 2011 11:15:59 +0200 Subject: CRR (S): 7046182: G1: remove unnecessary iterations over the collection set In-Reply-To: <4DF6474B.3010601@oracle.com> References: <4DF64603.3060702@oracle.com> <4DF6474B.3010601@oracle.com> Message-ID: <4DF8784F.3030004@oracle.com> Tony, Looks good. Bengt On 2011-06-13 19:22, Tony Printezis wrote: > ..and of course the webrev url is wrong. This is the right one: > > http://cr.openjdk.java.net/~tonyp/7046182/webrev.0/ > > Apologies! > > Tony > > Tony Printezis wrote: >> Hi all, >> >> Could I have a couple of code reviews for this change? >> >> http://syros.us.oracle.com/local_ws/hotspot-g1-cset-iter/webrev/ >> >> Basically, we (unnecessarily) iterate over the collection set a >> couple of times during each GC to reset two fields on the RSet data >> structure. It's better to make sure that those fields are reset when >> the region is freed (so it's "ready to go" when it's re-allocated) to >> avoid those two iterations. >> >> I added some extra code to assert that these two fields are in the >> correct state before each GC and I also added an extra verification >> step on the master free list to also check that condition on all the >> regions on it (since the two fields in question should be reset >> before the region is added to the master free list). >> >> I'm piggy-backing the removal of the G1_REM_SET_LOGGING code on this >> (it's kinda related as it also iterates over the collection set - so, >> we remove more collection set iterations!). We haven't used it for a >> while and it's helpfulness is marginal. >> >> Thanks, >> >> Tony >> From john.cuthbertson at oracle.com Wed Jun 15 12:27:53 2011 From: john.cuthbertson at oracle.com (john.cuthbertson at oracle.com) Date: Wed, 15 Jun 2011 19:27:53 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7045751: G1: +ExplicitGCInvokesConcurrent causes excessive single region evacuation pauses Message-ID: <20110615192757.C1D4B47061@hg.openjdk.java.net> Changeset: 5130fa1b24f1 Author: johnc Date: 2011-06-15 10:18 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/5130fa1b24f1 7045751: G1: +ExplicitGCInvokesConcurrent causes excessive single region evacuation pauses Summary: When ExplicitGCInvokesConcurrent is enabled, do not perform an evacuation pause if a marking cycle is already in progress and block the requesting thread until the marking cycle completes. Reviewed-by: tonyp, ysr ! src/share/vm/gc_implementation/g1/vm_operations_g1.cpp From y.s.ramakrishna at oracle.com Wed Jun 15 13:53:34 2011 From: y.s.ramakrishna at oracle.com (Y. Srinivas Ramakrishna) Date: Wed, 15 Jun 2011 13:53:34 -0700 Subject: request for review (XS): 6916968 CMS: freeList.cpp:304 assert(_allocation_stats.prevSweep() + ..., "Conservation Principle") Message-ID: <4DF91BCE.7000705@oracle.com> 6916968 CMS: freeList.cpp:304 assert(_allocation_stats.prevSweep() + ..., "Conservation Principle") The conservation principle had been incorrectly formulated, with at least one production factor missing. This also necessitated a change in the demand computation which was similarly missing a production factor and thus slightly underestimating demand. Since "coalBirths" are rare in practice (except during sweeps which this does not count) this change has minimal effect on demand estimates, and also explains why it was violated so rarely. The correction also allowed us to reenable an assert that we had disabled before, but i left in a defensive floor on the demand computation in the event that our counting is still imprecise because of inadvertently missing certain birth or death events. webrev: http://cr.openjdk.java.net/~ysr/6916968/webrev.0/ testing: refworkload (will be run to verify the expectation of no appreciable change in demand estimates following this correction), jprt, the gcl001 gclocker test which had recently started failing somewhat regularly in nightly testing), the largeObject tests that had failed the assert in the distant past thanks for your reviews. -- ramki From tony.printezis at oracle.com Thu Jun 16 13:10:52 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Thu, 16 Jun 2011 16:10:52 -0400 Subject: Feedback requested: HotSpot GC logging improvements In-Reply-To: References: <4BE3194E.902@oracle.com> <4BE43C97.6000805@oracle.com> <9694A6C3D68A4249BD9E1A875B6BA81E105CCD27@bos0ex01.corp.attivio.com> <4BE96ACD.2070805@oracle.com> Message-ID: <4DFA634C.7020805@oracle.com> David, We haven't had a chance to work on the %d format.... Tony On 06/16/2011 04:09 PM, David Ely wrote: > I'm curious if there was any update on this issue. Is there someplace > I can track whether this is in JDK 7 or not? Thanks. > > David > > On Tue, May 11, 2010 at 9:33 AM, Tony Printezis > > wrote: > > Martin, > > Hi, thanks for the feedback. > > Martin Serrano wrote: > > I would prefer to have a consistent suffix (like .log), in the > filename. Perhaps you could support just the %d format for the > counter in the generated log name. > > > Well, if you allow parameters in the log file name, like > 'foo.%d.%n.log' > then folks can give their one suffix. I don't think we want to start > adding one... > > Tony > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110616/52b7cc0a/attachment.html -------------- next part -------------- _______________________________________________ hotspot-gc-use mailing list hotspot-gc-use at openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use From y.s.ramakrishna at oracle.com Thu Jun 16 15:20:19 2011 From: y.s.ramakrishna at oracle.com (Y. S. Ramakrishna) Date: Thu, 16 Jun 2011 15:20:19 -0700 Subject: G1GC: many consecutive full gcs (was Re: hotspot-gc-use Digest, Vol 40, Issue 4) In-Reply-To: References: Message-ID: <4DFA81A3.1050802@oracle.com> Stefan and Alex -- (1) what's the version of the JVM used? (2) could you try PermSize == MaxPermSize and see if it made any difference? (3) do you have lots of WeakReference or Finalizer objects in your system? (4) is what you have below your complete GC log (i.e. no gaps between first and last line?) (5) what do you see when using CMS (in case you tried it)? (6) do you have a simple test case that you can share with us that exhibits this behaviour? thanks! -- ramki On 06/16/11 14:02, Alex Aisinzon wrote: > Stefan > > I have had similar experience with G1 and reported it here. > I tried the option to try to limit GC pauses (-XX:MaxGCPauseMillis with > 1s and then 2s). It did not improve things. > I am happy to try anything else to help improve G1. > > Alex A > > -----Original Message----- > From: hotspot-gc-use-bounces at openjdk.java.net > [mailto:hotspot-gc-use-bounces at openjdk.java.net] On Behalf Of > hotspot-gc-use-request at openjdk.java.net > Sent: Thursday, June 16, 2011 12:00 PM > To: hotspot-gc-use at openjdk.java.net > Subject: hotspot-gc-use Digest, Vol 40, Issue 4 > > Send hotspot-gc-use mailing list submissions to > hotspot-gc-use at openjdk.java.net > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use > or, via email, send a message with subject or body 'help' to > hotspot-gc-use-request at openjdk.java.net > > You can reach the person managing the list at > hotspot-gc-use-owner at openjdk.java.net > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of hotspot-gc-use digest..." > > > Today's Topics: > > 1. G1GC: many consecutive full gcs (Stefan Wachter) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 16 Jun 2011 09:23:05 +0200 > From: Stefan Wachter > Subject: G1GC: many consecutive full gcs > To: hotspot-gc-use at openjdk.java.net > Message-ID: <4DF9AF59.90708 at gmx.de> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Hi all, > > the G1GC runs nicely for some time but then it starts to have bursts of > consecutive full gcs (see log below). Can anyone explain that behaviour? > > The JVM is stared with these options: > > -Xmx10000M > -Xms1024M > -XX:MaxPermSize=1024m > -XX:+UseG1GC > -XX:MaxGCPauseMillis=2000 > -XX:GCPauseIntervalMillis=10000 > > --Stefan > > > 31314.569: [GC pause (young) 3286M->3234M(8937M), 0.0485530 secs] > 31374.932: [GC pause (young) 3279M->3236M(8937M), 0.0486560 secs] > 31837.688: [GC pause (young) 4614M->3252M(8937M), 0.0739370 secs] > 33528.784: [GC pause (young) 5997M->3330M(8937M), 0.1543940 secs] > 35372.029: [GC pause (young) 6038M->3405M(8937M), 0.2364930 secs] > 36508.343: [Full GC 8525M->2903M(9678M), 8.9121160 secs] > 38204.291: [Full GC 8647M->2295M(10000M), 7.4105940 secs] > 39038.017: [Full GC 8790M->2092M(6975M), 6.7242110 secs] > 39047.736: [Full GC 8996M->2089M(9998M), 6.0741630 secs] > 39095.551: [Full GC 8936M->2090M(6969M), 5.5573390 secs] > 39104.326: [Full GC 9175M->2092M(9997M), 5.3413860 secs] > 39112.760: [Full GC 9392M->2094M(9978M), 5.4884150 secs] > 41965.304: [Full GC 9549M->2112M(7043M), 6.4774550 secs] > 43830.656: [GC pause (young) 4615M->2309M(7043M), 0.1727720 secs] > 46024.398: [GC pause (young) 4638M->2497M(7043M), 0.3039900 secs] > 46068.009: [Full GC 8832M->2237M(10000M), 6.6717340 secs] > 46077.691: [Full GC 9322M->2187M(9987M), 5.8603570 secs] > 46086.731: [Full GC 9487M->2189M(9996M), 5.8521360 secs] > 46124.713: [Full GC 8792M->2185M(9895M), 5.7782260 secs] > 46133.366: [Full GC 9297M->2188M(9999M), 5.7263360 secs] > 46142.238: [Full GC 9509M->2189M(9985M), 5.5704850 secs] > 47454.530: [Full GC 9676M->2295M(7653M), 7.3169840 secs] > 48163.422: [GC pause (young) 4937M->2431M(7653M), 0.2379570 secs] > 48887.388: [GC pause (young) 5007M->2497M(7653M), 0.1999790 secs] > > > > ------------------------------ > > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use > > > End of hotspot-gc-use Digest, Vol 40, Issue 4 > ********************************************* > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use _______________________________________________ hotspot-gc-use mailing list hotspot-gc-use at openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use From tony.printezis at oracle.com Thu Jun 16 18:17:11 2011 From: tony.printezis at oracle.com (tony.printezis at oracle.com) Date: Fri, 17 Jun 2011 01:17:11 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 6994322: Remove the is_tlab and is_noref / is_large_noref parameters from the CollectedHeap Message-ID: <20110617011715.35A4C470AC@hg.openjdk.java.net> Changeset: c9ca3f51cf41 Author: tonyp Date: 2011-06-16 15:51 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/c9ca3f51cf41 6994322: Remove the is_tlab and is_noref / is_large_noref parameters from the CollectedHeap Summary: Remove two unused parameters from the mem_allocate() method and update its uses accordingly. Reviewed-by: stefank, johnc ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp ! src/share/vm/gc_implementation/parallelScavenge/parallelScavengeHeap.cpp ! src/share/vm/gc_implementation/parallelScavenge/parallelScavengeHeap.hpp ! src/share/vm/gc_implementation/parallelScavenge/psOldGen.cpp ! src/share/vm/gc_implementation/parallelScavenge/psOldGen.hpp ! src/share/vm/gc_implementation/parallelScavenge/psPermGen.cpp ! src/share/vm/gc_implementation/parallelScavenge/psYoungGen.hpp ! src/share/vm/gc_implementation/parallelScavenge/vmPSOperations.cpp ! src/share/vm/gc_implementation/parallelScavenge/vmPSOperations.hpp ! src/share/vm/gc_interface/collectedHeap.hpp ! src/share/vm/gc_interface/collectedHeap.inline.hpp ! src/share/vm/memory/collectorPolicy.cpp ! src/share/vm/memory/collectorPolicy.hpp ! src/share/vm/memory/genCollectedHeap.cpp ! src/share/vm/memory/genCollectedHeap.hpp ! src/share/vm/oops/typeArrayKlass.cpp From y.s.ramakrishna at oracle.com Fri Jun 17 08:10:51 2011 From: y.s.ramakrishna at oracle.com (Y. Srinivas Ramakrishna) Date: Fri, 17 Jun 2011 08:10:51 -0700 Subject: G1GC: many consecutive full gcs (was Re: hotspot-gc-use Digest, Vol 40, Issue 4) In-Reply-To: <4DFAFB95.7070405@gmx.de> References: <4DFA81A3.1050802@oracle.com> <4DFAFB95.7070405@gmx.de> Message-ID: <4DFB6E7B.8000200@oracle.com> Thanks for the responses and the extra information, Stefan and Alex. Stefan, please try PermSize == MaxPermSize at the next opportunity, although it's likely not to have any effect based on a preliminary reading of the info you provided below. As regards class histogram, please note that this would induce a pause as long as the full gc's you see (or may be a bit shorter), so use it with care when you obtain that information. Meanwhile, we'll look over the information you have provided and get back to you once we have made some headway. Thanks for bringing this to our attention and for the data so far. -- ramki On 6/17/2011 12:00 AM, Stefan Wachter wrote: > Hi Ramki, > > thanks for your interest. See my answers below. > > Cheers, > > Stefan > > > On 06/17/2011 12:20 AM, Y. S. Ramakrishna wrote: >> Stefan and Alex -- >> >> (1) what's the version of the JVM used? > > Java(TM) SE Runtime Environment (build 1.6.0_24-b07) > Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode) > >> >> (2) could you try PermSize == MaxPermSize and see if it made any >> difference? > > PermSize was not set until now. I try this setting and can report results the next time our cluster > nodes are switched. > >> >> (3) do you have lots of WeakReference or Finalizer objects in your system? > > I have no information about this. I guess a heap dump would provide that information. I will ask our > admins if it is possible to take a heap dump. > >> >> (4) is what you have below your complete GC log (i.e. no gaps between first and last line?) > > Yes, that is the original output of the GC. No lines were removed. >> >> (5) what do you see when using CMS (in case you tried it)? > > CMS seems to behave better (see attached gc.log). The settings used with CMS are: > > -Xmx10000M > -Xms1024M > -XX:MaxPermSize=1024m > -XX:+PrintGCDateStamps > -XX:+UseConcMarkSweepGC > >> >> (6) do you have a simple test case that you can share with >> us that exhibits this behaviour? > > Unfortunately I have no simple test case. The output is from our life system running a tomcat with > various wars deployed. > >> >> thanks! >> -- ramki >> >> On 06/16/11 14:02, Alex Aisinzon wrote: >>> Stefan >>> >>> I have had similar experience with G1 and reported it here. >>> I tried the option to try to limit GC pauses (-XX:MaxGCPauseMillis with >>> 1s and then 2s). It did not improve things. >>> I am happy to try anything else to help improve G1. >>> >>> Alex A >>> >>> -----Original Message----- >>> From: hotspot-gc-use-bounces at openjdk.java.net >>> [mailto:hotspot-gc-use-bounces at openjdk.java.net] On Behalf Of >>> hotspot-gc-use-request at openjdk.java.net >>> Sent: Thursday, June 16, 2011 12:00 PM >>> To: hotspot-gc-use at openjdk.java.net >>> Subject: hotspot-gc-use Digest, Vol 40, Issue 4 >>> >>> Send hotspot-gc-use mailing list submissions to >>> hotspot-gc-use at openjdk.java.net >>> >>> To subscribe or unsubscribe via the World Wide Web, visit >>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use >>> or, via email, send a message with subject or body 'help' to >>> hotspot-gc-use-request at openjdk.java.net >>> >>> You can reach the person managing the list at >>> hotspot-gc-use-owner at openjdk.java.net >>> >>> When replying, please edit your Subject line so it is more specific >>> than "Re: Contents of hotspot-gc-use digest..." >>> >>> >>> Today's Topics: >>> >>> 1. G1GC: many consecutive full gcs (Stefan Wachter) >>> >>> >>> ---------------------------------------------------------------------- >>> >>> Message: 1 >>> Date: Thu, 16 Jun 2011 09:23:05 +0200 >>> From: Stefan Wachter >>> Subject: G1GC: many consecutive full gcs >>> To: hotspot-gc-use at openjdk.java.net >>> Message-ID: <4DF9AF59.90708 at gmx.de> >>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >>> >>> Hi all, >>> >>> the G1GC runs nicely for some time but then it starts to have bursts of consecutive full gcs (see >>> log below). Can anyone explain that behaviour? >>> >>> The JVM is stared with these options: >>> >>> -Xmx10000M >>> -Xms1024M >>> -XX:MaxPermSize=1024m >>> -XX:+UseG1GC >>> -XX:MaxGCPauseMillis=2000 >>> -XX:GCPauseIntervalMillis=10000 >>> >>> --Stefan >>> >>> >>> 31314.569: [GC pause (young) 3286M->3234M(8937M), 0.0485530 secs] >>> 31374.932: [GC pause (young) 3279M->3236M(8937M), 0.0486560 secs] >>> 31837.688: [GC pause (young) 4614M->3252M(8937M), 0.0739370 secs] >>> 33528.784: [GC pause (young) 5997M->3330M(8937M), 0.1543940 secs] >>> 35372.029: [GC pause (young) 6038M->3405M(8937M), 0.2364930 secs] >>> 36508.343: [Full GC 8525M->2903M(9678M), 8.9121160 secs] >>> 38204.291: [Full GC 8647M->2295M(10000M), 7.4105940 secs] >>> 39038.017: [Full GC 8790M->2092M(6975M), 6.7242110 secs] >>> 39047.736: [Full GC 8996M->2089M(9998M), 6.0741630 secs] >>> 39095.551: [Full GC 8936M->2090M(6969M), 5.5573390 secs] >>> 39104.326: [Full GC 9175M->2092M(9997M), 5.3413860 secs] >>> 39112.760: [Full GC 9392M->2094M(9978M), 5.4884150 secs] >>> 41965.304: [Full GC 9549M->2112M(7043M), 6.4774550 secs] >>> 43830.656: [GC pause (young) 4615M->2309M(7043M), 0.1727720 secs] >>> 46024.398: [GC pause (young) 4638M->2497M(7043M), 0.3039900 secs] >>> 46068.009: [Full GC 8832M->2237M(10000M), 6.6717340 secs] >>> 46077.691: [Full GC 9322M->2187M(9987M), 5.8603570 secs] >>> 46086.731: [Full GC 9487M->2189M(9996M), 5.8521360 secs] >>> 46124.713: [Full GC 8792M->2185M(9895M), 5.7782260 secs] >>> 46133.366: [Full GC 9297M->2188M(9999M), 5.7263360 secs] >>> 46142.238: [Full GC 9509M->2189M(9985M), 5.5704850 secs] >>> 47454.530: [Full GC 9676M->2295M(7653M), 7.3169840 secs] >>> 48163.422: [GC pause (young) 4937M->2431M(7653M), 0.2379570 secs] >>> 48887.388: [GC pause (young) 5007M->2497M(7653M), 0.1999790 secs] >>> >>> >>> >>> ------------------------------ >>> >>> _______________________________________________ >>> hotspot-gc-use mailing list >>> hotspot-gc-use at openjdk.java.net >>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use >>> >>> >>> End of hotspot-gc-use Digest, Vol 40, Issue 4 >>> ********************************************* >>> _______________________________________________ >>> hotspot-gc-use mailing list >>> hotspot-gc-use at openjdk.java.net >>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use >> > _______________________________________________ hotspot-gc-use mailing list hotspot-gc-use at openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use From y.s.ramakrishna at oracle.com Mon Jun 20 13:05:05 2011 From: y.s.ramakrishna at oracle.com (y.s.ramakrishna at oracle.com) Date: Mon, 20 Jun 2011 20:05:05 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 6916968: CMS: freeList.cpp:304 assert(_allocation_stats.prevSweep() + ..., "Conservation Principle") Message-ID: <20110620200509.7B01D47185@hg.openjdk.java.net> Changeset: f75137faa7fe Author: ysr Date: 2011-06-20 09:42 -0700 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/f75137faa7fe 6916968: CMS: freeList.cpp:304 assert(_allocation_stats.prevSweep() + ..., "Conservation Principle") Summary: Fix assert and adjust demand volume computation by adding missing factor. Reviewed-by: jmasa, tonyp ! src/share/vm/gc_implementation/concurrentMarkSweep/freeList.cpp ! src/share/vm/gc_implementation/shared/allocationStats.hpp From tony.printezis at oracle.com Mon Jun 20 14:22:22 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Mon, 20 Jun 2011 17:22:22 -0400 Subject: CRR (S / cosmetic): 7055073: G1: code cleanup in the concurrentMark.* files Message-ID: <4DFFBA0E.50708@oracle.com> Hi all, When I recently worked on the concurrentMark.* files I was a bit annoyed that the style in those files doesn't quite match what's done in the rest of G1. In particular, we have a lot of instances of: if (condition) statement (i.e. without curly brackets) which I found particularly irritating. I did a cleanup pass to try to bring the file a bit more in order with the rest. Here's the webrev: http://cr.openjdk.java.net/~tonyp/7055073/webrev.0/ I'll only need one code review for this. Thanks. Tony From tony.printezis at oracle.com Mon Jun 20 14:25:08 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Mon, 20 Jun 2011 17:25:08 -0400 Subject: CRR (S): 7046182: G1: remove unnecessary iterations over the collection set In-Reply-To: <4DF64603.3060702@oracle.com> References: <4DF64603.3060702@oracle.com> Message-ID: <4DFFBAB4.3090208@oracle.com> Hi, I have one code review for this (from Bengt). Any chance of getting a second one to get it off my plate? Thanks. Tony On 06/13/2011 01:16 PM, Tony Printezis wrote: > Hi all, > > Could I have a couple of code reviews for this change? > > http://syros.us.oracle.com/local_ws/hotspot-g1-cset-iter/webrev/ > > Basically, we (unnecessarily) iterate over the collection set a couple > of times during each GC to reset two fields on the RSet data > structure. It's better to make sure that those fields are reset when > the region is freed (so it's "ready to go" when it's re-allocated) to > avoid those two iterations. > > I added some extra code to assert that these two fields are in the > correct state before each GC and I also added an extra verification > step on the master free list to also check that condition on all the > regions on it (since the two fields in question should be reset before > the region is added to the master free list). > > I'm piggy-backing the removal of the G1_REM_SET_LOGGING code on this > (it's kinda related as it also iterates over the collection set - so, > we remove more collection set iterations!). We haven't used it for a > while and it's helpfulness is marginal. > > Thanks, > > Tony > > From tony.printezis at oracle.com Mon Jun 20 14:39:22 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Mon, 20 Jun 2011 17:39:22 -0400 Subject: CRR (S / cosmetic): 7055073: G1: code cleanup in the concurrentMark.* files In-Reply-To: <4DFFBA0E.50708@oracle.com> References: <4DFFBA0E.50708@oracle.com> Message-ID: <4DFFBE0A.8020803@oracle.com> (I already got a comment on this, so I'd better clarify...) The only time I skip the curly brackets is if the if-body has a single return, break, or continue statement: if (foo) return true; I feel the code highlighting makes the keyword stand-out. If folks hate it I can put curly brackets on those too. Tony On 06/20/2011 05:22 PM, Tony Printezis wrote: > Hi all, > > When I recently worked on the concurrentMark.* files I was a bit > annoyed that the style in those files doesn't quite match what's done > in the rest of G1. In particular, we have a lot of instances of: > > if (condition) > statement > > (i.e. without curly brackets) which I found particularly irritating. I > did a cleanup pass to try to bring the file a bit more in order with > the rest. Here's the webrev: > > http://cr.openjdk.java.net/~tonyp/7055073/webrev.0/ > > I'll only need one code review for this. Thanks. > > Tony > > From john.cuthbertson at oracle.com Mon Jun 20 14:45:46 2011 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Mon, 20 Jun 2011 14:45:46 -0700 Subject: CRR (S): 7046182: G1: remove unnecessary iterations over the collection set In-Reply-To: <4DFFBAB4.3090208@oracle.com> References: <4DF64603.3060702@oracle.com> <4DFFBAB4.3090208@oracle.com> Message-ID: <4DFFBF8A.307@oracle.com> Hi Tony, I'm looking at it now - should be done very shortly. JohnC On 06/20/11 14:25, Tony Printezis wrote: > Hi, > > I have one code review for this (from Bengt). Any chance of getting a > second one to get it off my plate? Thanks. > > Tony > > On 06/13/2011 01:16 PM, Tony Printezis wrote: >> Hi all, >> >> Could I have a couple of code reviews for this change? >> >> http://syros.us.oracle.com/local_ws/hotspot-g1-cset-iter/webrev/ >> >> Basically, we (unnecessarily) iterate over the collection set a >> couple of times during each GC to reset two fields on the RSet data >> structure. It's better to make sure that those fields are reset when >> the region is freed (so it's "ready to go" when it's re-allocated) to >> avoid those two iterations. >> >> I added some extra code to assert that these two fields are in the >> correct state before each GC and I also added an extra verification >> step on the master free list to also check that condition on all the >> regions on it (since the two fields in question should be reset >> before the region is added to the master free list). >> >> I'm piggy-backing the removal of the G1_REM_SET_LOGGING code on this >> (it's kinda related as it also iterates over the collection set - so, >> we remove more collection set iterations!). We haven't used it for a >> while and it's helpfulness is marginal. >> >> Thanks, >> >> Tony >> >> From tony.printezis at oracle.com Mon Jun 20 16:09:46 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Mon, 20 Jun 2011 19:09:46 -0400 Subject: CRR (S / cosmetic): 7055073: G1: code cleanup in the concurrentMark.* files In-Reply-To: <4DFFBA0E.50708@oracle.com> References: <4DFFBA0E.50708@oracle.com> Message-ID: <4DFFD33A.9020806@oracle.com> All set thanks to John Cuthbertson (thanks!). Tony On 06/20/2011 05:22 PM, Tony Printezis wrote: > Hi all, > > When I recently worked on the concurrentMark.* files I was a bit > annoyed that the style in those files doesn't quite match what's done > in the rest of G1. In particular, we have a lot of instances of: > > if (condition) > statement > > (i.e. without curly brackets) which I found particularly irritating. I > did a cleanup pass to try to bring the file a bit more in order with > the rest. Here's the webrev: > > http://cr.openjdk.java.net/~tonyp/7055073/webrev.0/ > > I'll only need one code review for this. Thanks. > > Tony > > From y.s.ramakrishna at oracle.com Mon Jun 20 19:04:00 2011 From: y.s.ramakrishna at oracle.com (Y. Srinivas Ramakrishna) Date: Mon, 20 Jun 2011 19:04:00 -0700 Subject: CRR (S / cosmetic): 7055073: G1: code cleanup in the concurrentMark.* files In-Reply-To: <4DFFBA0E.50708@oracle.com> References: <4DFFBA0E.50708@oracle.com> Message-ID: <4DFFFC10.3080004@oracle.com> Looks fine. (Cosmetic all right, but definitely not small :-) I am glad you did this as a separate CR!) -- ramki On 6/20/2011 2:22 PM, Tony Printezis wrote: > Hi all, > > When I recently worked on the concurrentMark.* files I was a bit annoyed that the style in those > files doesn't quite match what's done in the rest of G1. In particular, we have a lot of instances of: > > if (condition) > statement > > (i.e. without curly brackets) which I found particularly irritating. I did a cleanup pass to try to > bring the file a bit more in order with the rest. Here's the webrev: > > http://cr.openjdk.java.net/~tonyp/7055073/webrev.0/ > > I'll only need one code review for this. Thanks. > > Tony > > From tony.printezis at oracle.com Tue Jun 21 01:01:49 2011 From: tony.printezis at oracle.com (tony.printezis at oracle.com) Date: Tue, 21 Jun 2011 08:01:49 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7055073: G1: code cleanup in the concurrentMark.* files Message-ID: <20110621080151.8016F471B7@hg.openjdk.java.net> Changeset: 23d434c6290d Author: tonyp Date: 2011-06-20 22:03 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/23d434c6290d 7055073: G1: code cleanup in the concurrentMark.* files Summary: Only cosmetic changes to make the concurrentMark.* more consistent, code-style-wise, with the rest of the codebase. Reviewed-by: johnc, ysr ! src/share/vm/gc_implementation/g1/concurrentMark.cpp ! src/share/vm/gc_implementation/g1/concurrentMark.hpp ! src/share/vm/gc_implementation/g1/concurrentMark.inline.hpp From bengt.rutisson at oracle.com Tue Jun 21 05:50:33 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 21 Jun 2011 14:50:33 +0200 Subject: Request for review (M): 7016112 CMS: crash during promotion testing Message-ID: <4E009399.4060000@oracle.com> Hi Runtime and GC, Sending this review request to both groups. I fixed a GC bug, but the changes are in runtime code. The bug that I fixed is this one: 7016112 CMS: crash during promotion testing http://monaco.sfbay.sun.com/detail.jsf?cr=7016112 And here is the webrev: http://cr.openjdk.java.net/~brutisso/7016112/webrev.01/ The investigation to find the root of the crashes reported in 7016112 was quite lengthy. It is not that easy to read the CR and figure out what is going on, so here is some background: When we load classes we store references to the methods of a class in an object array. When we have loaded all methods we sort the object array to allow binary search within the array. To do this sort we use stdlib::qsort(), which is the standard library quicksort implementation. If we are using CMS we might be doing concurrent marking while we are sorting the object array. The object array can be found by the concurrent marking code and it may start iterating over the array while we are sorting it. The problem is that on Windows the stdlib::qsort() is not implemented to do atomic updates of elements in the array that it sorts. Instead it does a byte-by-byte swap when it needs to swap two values. That is an easy way to implement different element widths, but it means that at some point in time one element may contain a few bytes from the element above or below. If this happens at the same time as the marking code is trying to read that element, we will be marking some random address and not the method that was supposed to be marked. On Solaris and Linux the stdlib::qsort() implementations try to swap as wide data types as possible so this issue should not occur there. On the other hand we have no guarantees that this will always be how stdlib::qsort() is implemented. After some discussions about different ways of solving this we came to the conclusion that the simplest way is to implement our own quicksort that operates on the correct pointer width (oop or narrowOop). So, this is what I have done to fix this bug. Also, it is likely that this problem will go away when the perm gen removal project is finished. Right now it looks like we will not be tracing and marking methods at all after that change. * Questions * - Should we keep the bubble sort that is done before calling quicksort in methodOopDesc::sort_methods() ? - Should we keep the idempotent option or should we try to always use idempotent sorting (see performance test below)? - What is the best way to handle unit tests? I added a flag called ExecuteInternalVMTests to run unit tests. This is in line with the existing ErrorHandlerTest flag. My thought is that we can use this same flag for other unit tests than just the quicksort tests. Would be good if we could get these tests executed by JPRT as well. I simply run these with "java -XX:+ExecuteInternalVMTests -version". * Testing * Did the obvious testing: Ran JPRT with the changes in the webrev and ran the failing nsk test from the bug (nsk/sysdict/vm/stress/jck12a/sysdictj12a008) repeatedly for sevaral days without failing. I created some unit tests for the quicksort implementation and they all pass. I also made a build that sorts both with my own quicksort and with stdlib::qsort and then compares that the arrays have the same sort order. Ran this through JPRT and it seems to work on all platforms. That run also included the unit tests. If anybody wants to see how this testing was done, there is a separate webrev for that build. The interesting code is in methodOop.cpp: http://cr.openjdk.java.net/~brutisso/7016112/webrev-verify-sorting/ * Performance * I am a bit unsure how to get any relevant performance data for this change. What I have done so far is to create a class that has 65535 methods (which is the maximum - the length of the method array is u2) and I have measured how long it takes to sort this method array. The methods have random names but every hundredth method has 4 overloaded version. This makes sure that there are some duplicates in the array. For now I have run this on my Windows x64 work station with 4 cpus and on a Solaris Sparc machine with 2 cpus (uname says: SunOS sthsparc24 5.10 Generic_139555-08 sun4us sparc FJSV,GPUZC-M Solaris). I am attaching graphs for the results. The Y-axis has time in nano seconds. Judging by this my quicksort is a bit faster on Windows and a bit slower on Solaris. But the interesting thing is that the idempotent version is faster than the default behavior on Windows and on par with the default on Solaris. I assume that this is due to the fact that some stores can be avoided if we don't do swap of duplicates. This means that (at least on Windows) swap is more expensive than compare. If this is true we should probably remove the special treatment of idempotent. I could run this on more machines, but I am not sure how relevant this type of data is. Most method arrays will be much shorter and compared to reading the class from a file the sort will be in the noise. Long email...I hope I covered most of the issues here. Let me know if you have any questions. Thanks, Bengt -------------- next part -------------- A non-text attachment was scrubbed... Name: win_x64_quicksort-perf.png Type: image/png Size: 32741 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110621/6d14848a/attachment-0002.png -------------- next part -------------- A non-text attachment was scrubbed... Name: sparc_quicksort-perf.png Type: image/png Size: 24748 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110621/6d14848a/attachment-0003.png From tony.printezis at oracle.com Tue Jun 21 12:34:36 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Tue, 21 Jun 2011 15:34:36 -0400 Subject: CRR (S/M): 7049999: G1: Make the G1PrintHeapRegions output consistent and complete Message-ID: <4E00F24C.6070001@oracle.com> Hi, Could I get a couple of folks to look at this? http://cr.openjdk.java.net/~tonyp/7049999/webrev.0/ +G1PrintHeapRegions is a very helpful debugging feature that we have used a lot in the past. It'll be very helpful in the long run to ensure that it generates more complete output to save us having to deduce some of it. What was missing in the past was: * output for humongous region allocations * output per region post compaction (as compaction changes the shape of the heap) * output when regions are reclaimed during cleanup Further enhancements include: * commit / uncommit events * trimmed the output to keep the generated log files smaller while not losing any information * introduced the G1HRPrinter class and all output goes through it to ensure consistency I attached an example of the new output. Tony -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: out.compress Url: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110621/dcd5d396/attachment.ksh From bengt.rutisson at oracle.com Tue Jun 21 14:32:27 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 21 Jun 2011 23:32:27 +0200 Subject: Request for review (M): 7016112 CMS: crash during promotion testing In-Reply-To: <4E009399.4060000@oracle.com> References: <4E009399.4060000@oracle.com> Message-ID: <4E010DEB.3030104@oracle.com> Hi again, For completeness. Here is the graph for sorting maximum length arrays on Linux x64 (run on my laptop). These runs show that my implementation takes twice as long as the stdlib version. I am not happy about that, but I don't know how much effort it is worth to optimize for this case. Bengt On 2011-06-21 14:50, Bengt Rutisson wrote: > > Hi Runtime and GC, > > Sending this review request to both groups. I fixed a GC bug, but the > changes are in runtime code. > > The bug that I fixed is this one: > 7016112 CMS: crash during promotion testing > http://monaco.sfbay.sun.com/detail.jsf?cr=7016112 > > And here is the webrev: > http://cr.openjdk.java.net/~brutisso/7016112/webrev.01/ > > The investigation to find the root of the crashes reported in 7016112 > was quite lengthy. It is not that easy to read the CR and figure out > what is going on, so here is some background: > > When we load classes we store references to the methods of a class in > an object array. When we have loaded all methods we sort the object > array to allow binary search within the array. To do this sort we use > stdlib::qsort(), which is the standard library quicksort implementation. > > If we are using CMS we might be doing concurrent marking while we are > sorting the object array. The object array can be found by the > concurrent marking code and it may start iterating over the array > while we are sorting it. The problem is that on Windows the > stdlib::qsort() is not implemented to do atomic updates of elements in > the array that it sorts. Instead it does a byte-by-byte swap when it > needs to swap two values. That is an easy way to implement different > element widths, but it means that at some point in time one element > may contain a few bytes from the element above or below. If this > happens at the same time as the marking code is trying to read that > element, we will be marking some random address and not the method > that was supposed to be marked. > > On Solaris and Linux the stdlib::qsort() implementations try to swap > as wide data types as possible so this issue should not occur there. > On the other hand we have no guarantees that this will always be how > stdlib::qsort() is implemented. > > After some discussions about different ways of solving this we came to > the conclusion that the simplest way is to implement our own quicksort > that operates on the correct pointer width (oop or narrowOop). > > So, this is what I have done to fix this bug. > > Also, it is likely that this problem will go away when the perm gen > removal project is finished. Right now it looks like we will not be > tracing and marking methods at all after that change. > > * Questions * > > - Should we keep the bubble sort that is done before calling quicksort > in methodOopDesc::sort_methods() ? > > - Should we keep the idempotent option or should we try to always use > idempotent sorting (see performance test below)? > > - What is the best way to handle unit tests? I added a flag called > ExecuteInternalVMTests to run unit tests. This is in line with the > existing ErrorHandlerTest flag. My thought is that we can use this > same flag for other unit tests than just the quicksort tests. Would be > good if we could get these tests executed by JPRT as well. I simply > run these with "java -XX:+ExecuteInternalVMTests -version". > > > * Testing * > > Did the obvious testing: Ran JPRT with the changes in the webrev and > ran the failing nsk test from the bug > (nsk/sysdict/vm/stress/jck12a/sysdictj12a008) repeatedly for sevaral > days without failing. > > I created some unit tests for the quicksort implementation and they > all pass. > > I also made a build that sorts both with my own quicksort and with > stdlib::qsort and then compares that the arrays have the same sort > order. Ran this through JPRT and it seems to work on all platforms. > That run also included the unit tests. If anybody wants to see how > this testing was done, there is a separate webrev for that build. The > interesting code is in methodOop.cpp: > > http://cr.openjdk.java.net/~brutisso/7016112/webrev-verify-sorting/ > > * Performance * > > I am a bit unsure how to get any relevant performance data for this > change. What I have done so far is to create a class that has 65535 > methods (which is the maximum - the length of the method array is u2) > and I have measured how long it takes to sort this method array. The > methods have random names but every hundredth method has 4 overloaded > version. This makes sure that there are some duplicates in the array. > > For now I have run this on my Windows x64 work station with 4 cpus and > on a Solaris Sparc machine with 2 cpus (uname says: SunOS sthsparc24 > 5.10 Generic_139555-08 sun4us sparc FJSV,GPUZC-M Solaris). > > I am attaching graphs for the results. The Y-axis has time in nano > seconds. Judging by this my quicksort is a bit faster on Windows and a > bit slower on Solaris. But the interesting thing is that the > idempotent version is faster than the default behavior on Windows and > on par with the default on Solaris. I assume that this is due to the > fact that some stores can be avoided if we don't do swap of > duplicates. This means that (at least on Windows) swap is more > expensive than compare. If this is true we should probably remove the > special treatment of idempotent. > > I could run this on more machines, but I am not sure how relevant this > type of data is. Most method arrays will be much shorter and compared > to reading the class from a file the sort will be in the noise. > > Long email...I hope I covered most of the issues here. Let me know if > you have any questions. > > Thanks, > Bengt -------------- next part -------------- A non-text attachment was scrubbed... Name: linux_x64_quicksort-perf.png Type: image/png Size: 21477 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110621/51404556/attachment-0001.png From tony.printezis at oracle.com Tue Jun 21 15:38:13 2011 From: tony.printezis at oracle.com (tony.printezis at oracle.com) Date: Tue, 21 Jun 2011 22:38:13 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7046182: G1: remove unnecessary iterations over the collection set Message-ID: <20110621223817.3D74B471ED@hg.openjdk.java.net> Changeset: e8b0b0392037 Author: tonyp Date: 2011-06-21 15:23 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/e8b0b0392037 7046182: G1: remove unnecessary iterations over the collection set Summary: Remove two unnecessary iterations over the collection set which are supposed to prepare the RSet's of the CSet regions for parallel iterations (we'll make sure this is done incrementally). I'll piggyback on this CR the removal of the G1_REM_SET_LOGGING code. Reviewed-by: brutisso, johnc ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp ! src/share/vm/gc_implementation/g1/g1RemSet.cpp ! src/share/vm/gc_implementation/g1/g1RemSet.hpp ! src/share/vm/gc_implementation/g1/g1RemSet.inline.hpp ! src/share/vm/gc_implementation/g1/heapRegionRemSet.cpp ! src/share/vm/gc_implementation/g1/heapRegionRemSet.hpp ! src/share/vm/gc_implementation/g1/heapRegionSets.cpp ! src/share/vm/gc_implementation/g1/heapRegionSets.hpp From tom.rodriguez at oracle.com Wed Jun 22 01:05:32 2011 From: tom.rodriguez at oracle.com (Tom Rodriguez) Date: Wed, 22 Jun 2011 01:05:32 -0700 Subject: Request for review (M): 7016112 CMS: crash during promotion testing In-Reply-To: <4E009399.4060000@oracle.com> References: <4E009399.4060000@oracle.com> Message-ID: Overall this looks fine to me. quicksort should really be QuickSort since its a class name and normally we'd make it inherit from AllStatic to indicate that's just a collection of functions. method names should be lower case with underscores though that isn't universally followed. I assume we need to keep the idempotent stuff for the original reasons stated in that comment. We could leave the bubble sort in place or just do some measurement to decide if it's really worth it. It would be nice to get rid of it though. tom On Jun 21, 2011, at 5:50 AM, Bengt Rutisson wrote: > > Hi Runtime and GC, > > Sending this review request to both groups. I fixed a GC bug, but the changes are in runtime code. > > The bug that I fixed is this one: > 7016112 CMS: crash during promotion testing > http://monaco.sfbay.sun.com/detail.jsf?cr=7016112 > > And here is the webrev: > http://cr.openjdk.java.net/~brutisso/7016112/webrev.01/ > > The investigation to find the root of the crashes reported in 7016112 was quite lengthy. It is not that easy to read the CR and figure out what is going on, so here is some background: > > When we load classes we store references to the methods of a class in an object array. When we have loaded all methods we sort the object array to allow binary search within the array. To do this sort we use stdlib::qsort(), which is the standard library quicksort implementation. > > If we are using CMS we might be doing concurrent marking while we are sorting the object array. The object array can be found by the concurrent marking code and it may start iterating over the array while we are sorting it. The problem is that on Windows the stdlib::qsort() is not implemented to do atomic updates of elements in the array that it sorts. Instead it does a byte-by-byte swap when it needs to swap two values. That is an easy way to implement different element widths, but it means that at some point in time one element may contain a few bytes from the element above or below. If this happens at the same time as the marking code is trying to read that element, we will be marking some random address and not the method that was supposed to be marked. > > On Solaris and Linux the stdlib::qsort() implementations try to swap as wide data types as possible so this issue should not occur there. On the other hand we have no guarantees that this will always be how stdlib::qsort() is implemented. > > After some discussions about different ways of solving this we came to the conclusion that the simplest way is to implement our own quicksort that operates on the correct pointer width (oop or narrowOop). > > So, this is what I have done to fix this bug. > > Also, it is likely that this problem will go away when the perm gen removal project is finished. Right now it looks like we will not be tracing and marking methods at all after that change. > > * Questions * > > - Should we keep the bubble sort that is done before calling quicksort in methodOopDesc::sort_methods() ? > > - Should we keep the idempotent option or should we try to always use idempotent sorting (see performance test below)? > > - What is the best way to handle unit tests? I added a flag called ExecuteInternalVMTests to run unit tests. This is in line with the existing ErrorHandlerTest flag. My thought is that we can use this same flag for other unit tests than just the quicksort tests. Would be good if we could get these tests executed by JPRT as well. I simply run these with "java -XX:+ExecuteInternalVMTests -version". > > > * Testing * > > Did the obvious testing: Ran JPRT with the changes in the webrev and ran the failing nsk test from the bug (nsk/sysdict/vm/stress/jck12a/sysdictj12a008) repeatedly for sevaral days without failing. > > I created some unit tests for the quicksort implementation and they all pass. > > I also made a build that sorts both with my own quicksort and with stdlib::qsort and then compares that the arrays have the same sort order. Ran this through JPRT and it seems to work on all platforms. That run also included the unit tests. If anybody wants to see how this testing was done, there is a separate webrev for that build. The interesting code is in methodOop.cpp: > > http://cr.openjdk.java.net/~brutisso/7016112/webrev-verify-sorting/ > > * Performance * > > I am a bit unsure how to get any relevant performance data for this change. What I have done so far is to create a class that has 65535 methods (which is the maximum - the length of the method array is u2) and I have measured how long it takes to sort this method array. The methods have random names but every hundredth method has 4 overloaded version. This makes sure that there are some duplicates in the array. > > For now I have run this on my Windows x64 work station with 4 cpus and on a Solaris Sparc machine with 2 cpus (uname says: SunOS sthsparc24 5.10 Generic_139555-08 sun4us sparc FJSV,GPUZC-M Solaris). > > I am attaching graphs for the results. The Y-axis has time in nano seconds. Judging by this my quicksort is a bit faster on Windows and a bit slower on Solaris. But the interesting thing is that the idempotent version is faster than the default behavior on Windows and on par with the default on Solaris. I assume that this is due to the fact that some stores can be avoided if we don't do swap of duplicates. This means that (at least on Windows) swap is more expensive than compare. If this is true we should probably remove the special treatment of idempotent. > > I could run this on more machines, but I am not sure how relevant this type of data is. Most method arrays will be much shorter and compared to reading the class from a file the sort will be in the noise. > > Long email...I hope I covered most of the issues here. Let me know if you have any questions. > > Thanks, > Bengt > From bengt.rutisson at oracle.com Wed Jun 22 05:17:44 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 22 Jun 2011 14:17:44 +0200 Subject: Request for review (M): 7016112 CMS: crash during promotion testing In-Reply-To: References: <4E009399.4060000@oracle.com> Message-ID: <4E01DD68.4060007@oracle.com> Hi Tom, Thanks for the review! > Overall this looks fine to me. quicksort should really be QuickSort since its a class name and normally we'd make it inherit from AllStatic to indicate that's just a collection of functions. method names should be lower case with underscores though that isn't universally followed. Good points. I am still getting used to the coding standard. I made the changes you proposed. Here is a new webrev: http://cr.openjdk.java.net/~brutisso/7016112/webrev.02/ FYI: I changed the name of the files quicksort.cpp/hpp to quickSort.cpp/hpp. This is not trivial on Windows, since it does not care about case in file names. I got mercurial, Visual Studio and the file system to accept my change. But for some reason webrev still thinks the files are named with all lower case. I'll make sure they are correctly named when I push. > I assume we need to keep the idempotent stuff for the original reasons stated in that comment. We could leave the bubble sort in place or just do some measurement to decide if it's really worth it. It would be nice to get rid of it though. Just to be clear; What I proposed was not to remove the idempotent sorting but to remove the parameter for it and always sort in an idempotent way. I have not seen any performance issues with always sorting with idempotent=true. On the other hand I am still a bit uncertain about performance so it might be best to keep it. Bengt > tom > > On Jun 21, 2011, at 5:50 AM, Bengt Rutisson wrote: > >> Hi Runtime and GC, >> >> Sending this review request to both groups. I fixed a GC bug, but the changes are in runtime code. >> >> The bug that I fixed is this one: >> 7016112 CMS: crash during promotion testing >> http://monaco.sfbay.sun.com/detail.jsf?cr=7016112 >> >> And here is the webrev: >> http://cr.openjdk.java.net/~brutisso/7016112/webrev.01/ >> >> The investigation to find the root of the crashes reported in 7016112 was quite lengthy. It is not that easy to read the CR and figure out what is going on, so here is some background: >> >> When we load classes we store references to the methods of a class in an object array. When we have loaded all methods we sort the object array to allow binary search within the array. To do this sort we use stdlib::qsort(), which is the standard library quicksort implementation. >> >> If we are using CMS we might be doing concurrent marking while we are sorting the object array. The object array can be found by the concurrent marking code and it may start iterating over the array while we are sorting it. The problem is that on Windows the stdlib::qsort() is not implemented to do atomic updates of elements in the array that it sorts. Instead it does a byte-by-byte swap when it needs to swap two values. That is an easy way to implement different element widths, but it means that at some point in time one element may contain a few bytes from the element above or below. If this happens at the same time as the marking code is trying to read that element, we will be marking some random address and not the method that was supposed to be marked. >> >> On Solaris and Linux the stdlib::qsort() implementations try to swap as wide data types as possible so this issue should not occur there. On the other hand we have no guarantees that this will always be how stdlib::qsort() is implemented. >> >> After some discussions about different ways of solving this we came to the conclusion that the simplest way is to implement our own quicksort that operates on the correct pointer width (oop or narrowOop). >> >> So, this is what I have done to fix this bug. >> >> Also, it is likely that this problem will go away when the perm gen removal project is finished. Right now it looks like we will not be tracing and marking methods at all after that change. >> >> * Questions * >> >> - Should we keep the bubble sort that is done before calling quicksort in methodOopDesc::sort_methods() ? >> >> - Should we keep the idempotent option or should we try to always use idempotent sorting (see performance test below)? >> >> - What is the best way to handle unit tests? I added a flag called ExecuteInternalVMTests to run unit tests. This is in line with the existing ErrorHandlerTest flag. My thought is that we can use this same flag for other unit tests than just the quicksort tests. Would be good if we could get these tests executed by JPRT as well. I simply run these with "java -XX:+ExecuteInternalVMTests -version". >> >> >> * Testing * >> >> Did the obvious testing: Ran JPRT with the changes in the webrev and ran the failing nsk test from the bug (nsk/sysdict/vm/stress/jck12a/sysdictj12a008) repeatedly for sevaral days without failing. >> >> I created some unit tests for the quicksort implementation and they all pass. >> >> I also made a build that sorts both with my own quicksort and with stdlib::qsort and then compares that the arrays have the same sort order. Ran this through JPRT and it seems to work on all platforms. That run also included the unit tests. If anybody wants to see how this testing was done, there is a separate webrev for that build. The interesting code is in methodOop.cpp: >> >> http://cr.openjdk.java.net/~brutisso/7016112/webrev-verify-sorting/ >> >> * Performance * >> >> I am a bit unsure how to get any relevant performance data for this change. What I have done so far is to create a class that has 65535 methods (which is the maximum - the length of the method array is u2) and I have measured how long it takes to sort this method array. The methods have random names but every hundredth method has 4 overloaded version. This makes sure that there are some duplicates in the array. >> >> For now I have run this on my Windows x64 work station with 4 cpus and on a Solaris Sparc machine with 2 cpus (uname says: SunOS sthsparc24 5.10 Generic_139555-08 sun4us sparc FJSV,GPUZC-M Solaris). >> >> I am attaching graphs for the results. The Y-axis has time in nano seconds. Judging by this my quicksort is a bit faster on Windows and a bit slower on Solaris. But the interesting thing is that the idempotent version is faster than the default behavior on Windows and on par with the default on Solaris. I assume that this is due to the fact that some stores can be avoided if we don't do swap of duplicates. This means that (at least on Windows) swap is more expensive than compare. If this is true we should probably remove the special treatment of idempotent. >> >> I could run this on more machines, but I am not sure how relevant this type of data is. Most method arrays will be much shorter and compared to reading the class from a file the sort will be in the noise. >> >> Long email...I hope I covered most of the issues here. Let me know if you have any questions. >> >> Thanks, >> Bengt >> From poonam.bajaj at oracle.com Wed Jun 22 07:01:28 2011 From: poonam.bajaj at oracle.com (Poonam Bajaj) Date: Wed, 22 Jun 2011 19:31:28 +0530 Subject: CRR (S/M): 7049999: G1: Make the G1PrintHeapRegions output consistent and complete In-Reply-To: <4E00F24C.6070001@oracle.com> References: <4E00F24C.6070001@oracle.com> Message-ID: <4E01F5B8.9000400@oracle.com> Hi Tony, The new output looks completely different from the earlier output. e.g. here's output generated with jdk7-b141: added region to incremental cset (LHS) 0:[0x02c00000, 0x02d00000], top 0x02c53040, young YES new alloc region 1:[0x02d00000,0x02e00000], top 0x02d00000 added region to incremental cset (LHS) 1:[0x02d00000, 0x02e00000], top 0x02e00000, young YES new alloc region 2:[0x02e00000,0x02f00000], top 0x02e00000 new alloc region 1:[0x02d00000,0x02e00000], top 0x02d00000 added region to incremental cset (LHS) 1:[0x02d00000, 0x02e00000], top 0x02e00000, young YES new alloc region 2:[0x02e00000, 0x02f00000], top 0x02e01000 ..... new alloc region 19:[0x03f00000,0x04000000], top 0x03f00000 added region to incremental cset (RHS) 19:[0x03f00000, 0x04000000], top 0x03f04000, young YES new alloc region 17:[0x03d00000,0x03e00000], top 0x03d00000 It prints the index of the region, and whether it is added to eden or survivor (based on LHS or RHS). The index of the region is missing from the new output. I think it would be good to have it; makes it easier to read the logs. In the new output, I see events like COMMIT, ALLOC, UNCOMMIT, RETIRE. Could you please explain these ? G1HR COMMIT [0x72700000,0x72800000] G1HR ALLOC(Eden) 0x6e800000 Will gc in between benchmarks G1HR RETIRE 0x6e800000 0x6e87bd98 G1HR #StartFullGC 1 G1HR UNCOMMIT [0x72700000,0x72800000] There are some numbers printed between the log entries. What are these numbers ? G1HR ALLOC(ContinuesH) 0x6f100000 0x6f102010 3153920 <----- 931067 <----- G1HR COMMIT [0x6f200000,0x6f300000] For the ALLOC and CSET entries, only one address is printed, looks like it is the lower bound of the region. It would be good to have both the bounds printed (along with the index). G1HR #StartGC 2 G1HR CSET 0x6e900000 G1HR COMMIT [0x6f800000,0x6f900000] G1HR ALLOC(Old) 0x6f800000 G1HR RETIRE 0x6f800000 0x6f826838 Thanks, Poonam On 6/22/2011 1:04 AM, Tony Printezis wrote: > Hi, > > Could I get a couple of folks to look at this? > > http://cr.openjdk.java.net/~tonyp/7049999/webrev.0/ > > +G1PrintHeapRegions is a very helpful debugging feature that we have > used a lot in the past. It'll be very helpful in the long run to > ensure that it generates more complete output to save us having to > deduce some of it. What was missing in the past was: > > * output for humongous region allocations > * output per region post compaction (as compaction changes the shape > of the heap) > * output when regions are reclaimed during cleanup > > Further enhancements include: > > * commit / uncommit events > * trimmed the output to keep the generated log files smaller while not > losing any information > * introduced the G1HRPrinter class and all output goes through it to > ensure consistency > > I attached an example of the new output. > > Tony > -- Best regards, Poonam Sun, an Oracle company Sun, an Oracle Company Poonam Bajaj | Staff Engineer Phone: +66937451 | Mobile: +9844511366 JVM Sustaining Engineering | Bangalore Green Oracle Oracle is committed to developing practices and products that help protect the environment -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110622/e83babe1/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: sun-oracle-logo.gif Type: image/gif Size: 2088 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110622/e83babe1/attachment-0002.gif -------------- next part -------------- A non-text attachment was scrubbed... Name: green-for-email-sig_0.gif Type: image/gif Size: 356 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110622/e83babe1/attachment-0003.gif From tony.printezis at oracle.com Wed Jun 22 08:18:52 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 22 Jun 2011 11:18:52 -0400 Subject: CRR (S/M): 7049999: G1: Make the G1PrintHeapRegions output consistent and complete In-Reply-To: <4E01F5B8.9000400@oracle.com> References: <4E00F24C.6070001@oracle.com> <4E01F5B8.9000400@oracle.com> Message-ID: <4E0207DC.8000009@oracle.com> Poonam, Poonam Bajaj wrote: > Hi Tony, > > The new output looks completely different from the earlier output. Indeed. The new output is more concise, more complete, and easier to parse (and avoids some duplication). > e.g. here's output generated with jdk7-b141: > > added region to incremental cset (LHS) 0:[0x02c00000, 0x02d00000], > top 0x02c53040, young YES > new alloc region 1:[0x02d00000,0x02e00000], top 0x02d00000 [snip] > It prints the index of the region, We can deduce that from the new output (from the COMMIT events which are done in order), so it's unnecessary to print it. > and whether it is added to eden or survivor (based on LHS > or RHS). Region allocation events have a region type in the new output too, like ALLOC(Eden) > The index of the region is missing from the new output. I think it > would be good > to have it; makes it easier to read the logs. See above. > In the new output, I see events like COMMIT, ALLOC, UNCOMMIT, RETIRE. > Could you please explain these ? > G1HR COMMIT [0x72700000,0x72800000] > The heap is initialized or expanded and the region is freshly committed (this was missing from the previous output). > G1HR ALLOC(Eden) 0x6e800000 > The region just started being used for allocation. In this case it's an Eden region and allocated to by a mutator thread. ALLOC events are also generated for regions we allocate to during GC (for Survivor and Old, depending on the type of region), as well as for humongous regions. > Will gc in between benchmarks > G1HR RETIRE 0x6e800000 0x6e87bd98 > The stop using the region for allocation so the retire even has the region's top (in this case: 0x6e87bd98). Note that most regions are full when retired and we ommit those events (we can easily deduce those) to reduce the output volume. > G1HR #StartFullGC 1 > G1HR UNCOMMIT [0x72700000,0x72800000] > Opposite to COMMIT. The heap got shrunk at the end of a Full GC. > There are some numbers printed between the log entries. What are > these numbers ? > G1HR ALLOC(ContinuesH) 0x6f100000 0x6f102010 > 3153920 <----- > 931067 <----- > G1HR COMMIT [0x6f200000,0x6f300000] > Output from the benchmark (_201_compress). > For the ALLOC and CSET entries, only one address is printed, looks > like it is the lower > bound of the region. Indeed. > It would be good to have both the bounds printed (along with the index). This was done to reduce the output volume. Both bounds are only printed when a region is committed / uncommitted. Tony > G1HR #StartGC 2 > G1HR CSET 0x6e900000 > G1HR COMMIT [0x6f800000,0x6f900000] > G1HR ALLOC(Old) 0x6f800000 > G1HR RETIRE 0x6f800000 0x6f826838 > > > Thanks, > Poonam > > > > On 6/22/2011 1:04 AM, Tony Printezis wrote: >> Hi, >> >> Could I get a couple of folks to look at this? >> >> http://cr.openjdk.java.net/~tonyp/7049999/webrev.0/ >> >> +G1PrintHeapRegions is a very helpful debugging feature that we have >> used a lot in the past. It'll be very helpful in the long run to >> ensure that it generates more complete output to save us having to >> deduce some of it. What was missing in the past was: >> >> * output for humongous region allocations >> * output per region post compaction (as compaction changes the shape >> of the heap) >> * output when regions are reclaimed during cleanup >> >> Further enhancements include: >> >> * commit / uncommit events >> * trimmed the output to keep the generated log files smaller while >> not losing any information >> * introduced the G1HRPrinter class and all output goes through it to >> ensure consistency >> >> I attached an example of the new output. >> >> Tony >> > > -- > Best regards, Poonam > > Sun, an Oracle company > Sun, an Oracle Company > Poonam Bajaj | Staff Engineer > Phone: +66937451 | Mobile: +9844511366 > JVM Sustaining Engineering > | Bangalore > Green Oracle Oracle is committed to > developing practices and products that help protect the environment > From poonam.bajaj at oracle.com Wed Jun 22 09:47:50 2011 From: poonam.bajaj at oracle.com (Poonam Bajaj) Date: Wed, 22 Jun 2011 22:17:50 +0530 Subject: CRR (S/M): 7049999: G1: Make the G1PrintHeapRegions output consistent and complete In-Reply-To: <4E0207DC.8000009@oracle.com> References: <4E00F24C.6070001@oracle.com> <4E01F5B8.9000400@oracle.com> <4E0207DC.8000009@oracle.com> Message-ID: <4E021CB6.8020201@oracle.com> Tony, Thanks for the detailed explanation. The overall output looks good. >> Will gc in between benchmarks >> G1HR RETIRE 0x6e800000 0x6e87bd98 > > The stop using the region for allocation so the retire even has the > region's top (in this case: 0x6e87bd98). Note that most regions are > full when retired and we ommit those events (we can easily deduce > those) to reduce the output volume. One small question - How do we find the full retired regions ? Do we print a region when it is full ? Thanks, Poonam On 6/22/2011 8:48 PM, Tony Printezis wrote: > Poonam, > > Poonam Bajaj wrote: >> Hi Tony, >> >> The new output looks completely different from the earlier output. > > Indeed. The new output is more concise, more complete, and easier to > parse (and avoids some duplication). > >> e.g. here's output generated with jdk7-b141: >> >> added region to incremental cset (LHS) 0:[0x02c00000, 0x02d00000], >> top 0x02c53040, young YES >> new alloc region 1:[0x02d00000,0x02e00000], top 0x02d00000 > > [snip] > >> It prints the index of the region, > > We can deduce that from the new output (from the COMMIT events which > are done in order), so it's unnecessary to print it. > >> and whether it is added to eden or survivor (based on LHS >> or RHS). > > Region allocation events have a region type in the new output too, > like ALLOC(Eden) > >> The index of the region is missing from the new output. I think it >> would be good >> to have it; makes it easier to read the logs. > > See above. > >> In the new output, I see events like COMMIT, ALLOC, UNCOMMIT, RETIRE. >> Could you please explain these ? >> G1HR COMMIT [0x72700000,0x72800000] >> > > The heap is initialized or expanded and the region is freshly > committed (this was missing from the previous output). > >> G1HR ALLOC(Eden) 0x6e800000 >> > > The region just started being used for allocation. In this case it's > an Eden region and allocated to by a mutator thread. ALLOC events are > also generated for regions we allocate to during GC (for Survivor and > Old, depending on the type of region), as well as for humongous regions. > >> Will gc in between benchmarks >> G1HR RETIRE 0x6e800000 0x6e87bd98 >> > > The stop using the region for allocation so the retire even has the > region's top (in this case: 0x6e87bd98). Note that most regions are > full when retired and we ommit those events (we can easily deduce > those) to reduce the output volume. > >> G1HR #StartFullGC 1 >> G1HR UNCOMMIT [0x72700000,0x72800000] >> > > Opposite to COMMIT. The heap got shrunk at the end of a Full GC. > >> There are some numbers printed between the log entries. What are >> these numbers ? >> G1HR ALLOC(ContinuesH) 0x6f100000 0x6f102010 >> 3153920 <----- >> 931067 <----- >> G1HR COMMIT [0x6f200000,0x6f300000] >> > > Output from the benchmark (_201_compress). > >> For the ALLOC and CSET entries, only one address is printed, looks >> like it is the lower >> bound of the region. > > Indeed. > >> It would be good to have both the bounds printed (along with the index). > > This was done to reduce the output volume. Both bounds are only > printed when a region is committed / uncommitted. > > Tony > >> G1HR #StartGC 2 >> G1HR CSET 0x6e900000 >> G1HR COMMIT [0x6f800000,0x6f900000] >> G1HR ALLOC(Old) 0x6f800000 >> G1HR RETIRE 0x6f800000 0x6f826838 >> >> >> Thanks, >> Poonam >> >> >> >> On 6/22/2011 1:04 AM, Tony Printezis wrote: >>> Hi, >>> >>> Could I get a couple of folks to look at this? >>> >>> http://cr.openjdk.java.net/~tonyp/7049999/webrev.0/ >>> >>> +G1PrintHeapRegions is a very helpful debugging feature that we have >>> used a lot in the past. It'll be very helpful in the long run to >>> ensure that it generates more complete output to save us having to >>> deduce some of it. What was missing in the past was: >>> >>> * output for humongous region allocations >>> * output per region post compaction (as compaction changes the shape >>> of the heap) >>> * output when regions are reclaimed during cleanup >>> >>> Further enhancements include: >>> >>> * commit / uncommit events >>> * trimmed the output to keep the generated log files smaller while >>> not losing any information >>> * introduced the G1HRPrinter class and all output goes through it to >>> ensure consistency >>> >>> I attached an example of the new output. >>> >>> Tony >>> >> >> -- >> Best regards, Poonam >> >> Sun, an Oracle company >> Sun, an Oracle Company >> Poonam Bajaj | Staff Engineer >> Phone: +66937451 | Mobile: +9844511366 >> JVM Sustaining Engineering >> | Bangalore >> Green Oracle Oracle is committed >> to developing practices and products that help protect the environment >> -- Best regards, Poonam Sun, an Oracle company Sun, an Oracle Company Poonam Bajaj | Staff Engineer Phone: +66937451 | Mobile: +9844511366 JVM Sustaining Engineering | Bangalore Green Oracle Oracle is committed to developing practices and products that help protect the environment -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110622/fefc34eb/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: sun-oracle-logo.gif Type: image/gif Size: 2088 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110622/fefc34eb/attachment.gif -------------- next part -------------- A non-text attachment was scrubbed... Name: green-for-email-sig_0.gif Type: image/gif Size: 356 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110622/fefc34eb/attachment-0001.gif From tony.printezis at oracle.com Wed Jun 22 09:58:54 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Wed, 22 Jun 2011 12:58:54 -0400 Subject: CRR (S/M): 7049999: G1: Make the G1PrintHeapRegions output consistent and complete In-Reply-To: <4E021CB6.8020201@oracle.com> References: <4E00F24C.6070001@oracle.com> <4E01F5B8.9000400@oracle.com> <4E0207DC.8000009@oracle.com> <4E021CB6.8020201@oracle.com> Message-ID: <4E021F4E.9090706@oracle.com> Poonam Bajaj wrote: >>> Will gc in between benchmarks >>> G1HR RETIRE 0x6e800000 0x6e87bd98 >> >> The stop using the region for allocation so the retire even has the >> region's top (in this case: 0x6e87bd98). Note that most regions are >> full when retired and we ommit those events (we can easily deduce >> those) to reduce the output volume. > One small question - How do we find the full retired regions ? Do we > print a region when it is full ? A region is retired when another region of the same type is allocated or we reach the start or end of a GC (depending on the region). So for Eden regions: 1. ALLOC(Eden) Foo 2. ALLOC(Eden) Bar 3. StartGC At point 2 you know that Foo has just been retired and it was full. At point 3 you know that Bar was retired and it was full. If they were not full when they were retired, we will have seen a RETIRE event: 1. ALLOC(Eden) Foo 2. RETIRE Foo top 3. ALLOC(Eden) Bar 3. StartGC Tony > Thanks, > Poonam > > > On 6/22/2011 8:48 PM, Tony Printezis wrote: >> Poonam, >> >> Poonam Bajaj wrote: >>> Hi Tony, >>> >>> The new output looks completely different from the earlier output. >> >> Indeed. The new output is more concise, more complete, and easier to >> parse (and avoids some duplication). >> >>> e.g. here's output generated with jdk7-b141: >>> >>> added region to incremental cset (LHS) 0:[0x02c00000, 0x02d00000], >>> top 0x02c53040, young YES >>> new alloc region 1:[0x02d00000,0x02e00000], top 0x02d00000 >> >> [snip] >> >>> It prints the index of the region, >> >> We can deduce that from the new output (from the COMMIT events which >> are done in order), so it's unnecessary to print it. >> >>> and whether it is added to eden or survivor (based on LHS >>> or RHS). >> >> Region allocation events have a region type in the new output too, >> like ALLOC(Eden) >> >>> The index of the region is missing from the new output. I think it >>> would be good >>> to have it; makes it easier to read the logs. >> >> See above. >> >>> In the new output, I see events like COMMIT, ALLOC, UNCOMMIT, RETIRE. >>> Could you please explain these ? >>> G1HR COMMIT [0x72700000,0x72800000] >>> >> >> The heap is initialized or expanded and the region is freshly >> committed (this was missing from the previous output). >> >>> G1HR ALLOC(Eden) 0x6e800000 >>> >> >> The region just started being used for allocation. In this case it's >> an Eden region and allocated to by a mutator thread. ALLOC events are >> also generated for regions we allocate to during GC (for Survivor and >> Old, depending on the type of region), as well as for humongous regions. >> >>> Will gc in between benchmarks >>> G1HR RETIRE 0x6e800000 0x6e87bd98 >>> >> >> The stop using the region for allocation so the retire even has the >> region's top (in this case: 0x6e87bd98). Note that most regions are >> full when retired and we ommit those events (we can easily deduce >> those) to reduce the output volume. >> >>> G1HR #StartFullGC 1 >>> G1HR UNCOMMIT [0x72700000,0x72800000] >>> >> >> Opposite to COMMIT. The heap got shrunk at the end of a Full GC. >> >>> There are some numbers printed between the log entries. What are >>> these numbers ? >>> G1HR ALLOC(ContinuesH) 0x6f100000 0x6f102010 >>> 3153920 >>> <----- >>> 931067 <----- >>> G1HR COMMIT [0x6f200000,0x6f300000] >>> >> >> Output from the benchmark (_201_compress). >> >>> For the ALLOC and CSET entries, only one address is printed, looks >>> like it is the lower >>> bound of the region. >> >> Indeed. >> >>> It would be good to have both the bounds printed (along with the >>> index). >> >> This was done to reduce the output volume. Both bounds are only >> printed when a region is committed / uncommitted. >> >> Tony >> >>> G1HR #StartGC 2 >>> G1HR CSET 0x6e900000 >>> G1HR COMMIT [0x6f800000,0x6f900000] >>> G1HR ALLOC(Old) 0x6f800000 >>> G1HR RETIRE 0x6f800000 0x6f826838 >>> >>> >>> Thanks, >>> Poonam >>> >>> >>> >>> On 6/22/2011 1:04 AM, Tony Printezis wrote: >>>> Hi, >>>> >>>> Could I get a couple of folks to look at this? >>>> >>>> http://cr.openjdk.java.net/~tonyp/7049999/webrev.0/ >>>> >>>> +G1PrintHeapRegions is a very helpful debugging feature that we >>>> have used a lot in the past. It'll be very helpful in the long run >>>> to ensure that it generates more complete output to save us having >>>> to deduce some of it. What was missing in the past was: >>>> >>>> * output for humongous region allocations >>>> * output per region post compaction (as compaction changes the >>>> shape of the heap) >>>> * output when regions are reclaimed during cleanup >>>> >>>> Further enhancements include: >>>> >>>> * commit / uncommit events >>>> * trimmed the output to keep the generated log files smaller while >>>> not losing any information >>>> * introduced the G1HRPrinter class and all output goes through it >>>> to ensure consistency >>>> >>>> I attached an example of the new output. >>>> >>>> Tony >>>> >>> >>> -- >>> Best regards, Poonam >>> >>> Sun, an Oracle company >>> Sun, an Oracle Company >>> Poonam Bajaj | Staff Engineer >>> Phone: +66937451 | Mobile: +9844511366 >>> >>> JVM Sustaining Engineering >>> | Bangalore >>> Green Oracle Oracle is committed >>> to developing practices and products that help protect the environment >>> > > -- > Best regards, Poonam > > Sun, an Oracle company > Sun, an Oracle Company > Poonam Bajaj | Staff Engineer > Phone: +66937451 | Mobile: +9844511366 > JVM Sustaining Engineering > | Bangalore > Green Oracle Oracle is committed to > developing practices and products that help protect the environment > From tom.rodriguez at oracle.com Wed Jun 22 14:55:18 2011 From: tom.rodriguez at oracle.com (Tom Rodriguez) Date: Wed, 22 Jun 2011 14:55:18 -0700 Subject: Request for review (M): 7016112 CMS: crash during promotion testing In-Reply-To: <4E01DD68.4060007@oracle.com> References: <4E009399.4060000@oracle.com> <4E01DD68.4060007@oracle.com> Message-ID: On Jun 22, 2011, at 5:17 AM, Bengt Rutisson wrote: > > Hi Tom, > > Thanks for the review! > >> Overall this looks fine to me. quicksort should really be QuickSort since its a class name and normally we'd make it inherit from AllStatic to indicate that's just a collection of functions. method names should be lower case with underscores though that isn't universally followed. > > Good points. I am still getting used to the coding standard. I made the changes you proposed. Here is a new webrev: > > http://cr.openjdk.java.net/~brutisso/7016112/webrev.02/ > > FYI: I changed the name of the files quicksort.cpp/hpp to quickSort.cpp/hpp. This is not trivial on Windows, since it does not care about case in file names. I got mercurial, Visual Studio and the file system to accept my change. But for some reason webrev still thinks the files are named with all lower case. I'll make sure they are correctly named when I push. Ok. > >> I assume we need to keep the idempotent stuff for the original reasons stated in that comment. We could leave the bubble sort in place or just do some measurement to decide if it's really worth it. It would be nice to get rid of it though. > > Just to be clear; What I proposed was not to remove the idempotent sorting but to remove the parameter for it and always sort in an idempotent way. I have not seen any performance issues with always sorting with idempotent=true. On the other hand I am still a bit uncertain about performance so it might be best to keep it. Sorry I misunderstood. I would probably keep it as is. You could use an algorithm which is stable in first place instead but that opens a whole other can of worms. tom > > Bengt > >> tom >> >> On Jun 21, 2011, at 5:50 AM, Bengt Rutisson wrote: >> >>> Hi Runtime and GC, >>> >>> Sending this review request to both groups. I fixed a GC bug, but the changes are in runtime code. >>> >>> The bug that I fixed is this one: >>> 7016112 CMS: crash during promotion testing >>> http://monaco.sfbay.sun.com/detail.jsf?cr=7016112 >>> >>> And here is the webrev: >>> http://cr.openjdk.java.net/~brutisso/7016112/webrev.01/ >>> >>> The investigation to find the root of the crashes reported in 7016112 was quite lengthy. It is not that easy to read the CR and figure out what is going on, so here is some background: >>> >>> When we load classes we store references to the methods of a class in an object array. When we have loaded all methods we sort the object array to allow binary search within the array. To do this sort we use stdlib::qsort(), which is the standard library quicksort implementation. >>> >>> If we are using CMS we might be doing concurrent marking while we are sorting the object array. The object array can be found by the concurrent marking code and it may start iterating over the array while we are sorting it. The problem is that on Windows the stdlib::qsort() is not implemented to do atomic updates of elements in the array that it sorts. Instead it does a byte-by-byte swap when it needs to swap two values. That is an easy way to implement different element widths, but it means that at some point in time one element may contain a few bytes from the element above or below. If this happens at the same time as the marking code is trying to read that element, we will be marking some random address and not the method that was supposed to be marked. >>> >>> On Solaris and Linux the stdlib::qsort() implementations try to swap as wide data types as possible so this issue should not occur there. On the other hand we have no guarantees that this will always be how stdlib::qsort() is implemented. >>> >>> After some discussions about different ways of solving this we came to the conclusion that the simplest way is to implement our own quicksort that operates on the correct pointer width (oop or narrowOop). >>> >>> So, this is what I have done to fix this bug. >>> >>> Also, it is likely that this problem will go away when the perm gen removal project is finished. Right now it looks like we will not be tracing and marking methods at all after that change. >>> >>> * Questions * >>> >>> - Should we keep the bubble sort that is done before calling quicksort in methodOopDesc::sort_methods() ? >>> >>> - Should we keep the idempotent option or should we try to always use idempotent sorting (see performance test below)? >>> >>> - What is the best way to handle unit tests? I added a flag called ExecuteInternalVMTests to run unit tests. This is in line with the existing ErrorHandlerTest flag. My thought is that we can use this same flag for other unit tests than just the quicksort tests. Would be good if we could get these tests executed by JPRT as well. I simply run these with "java -XX:+ExecuteInternalVMTests -version". >>> >>> >>> * Testing * >>> >>> Did the obvious testing: Ran JPRT with the changes in the webrev and ran the failing nsk test from the bug (nsk/sysdict/vm/stress/jck12a/sysdictj12a008) repeatedly for sevaral days without failing. >>> >>> I created some unit tests for the quicksort implementation and they all pass. >>> >>> I also made a build that sorts both with my own quicksort and with stdlib::qsort and then compares that the arrays have the same sort order. Ran this through JPRT and it seems to work on all platforms. That run also included the unit tests. If anybody wants to see how this testing was done, there is a separate webrev for that build. The interesting code is in methodOop.cpp: >>> >>> http://cr.openjdk.java.net/~brutisso/7016112/webrev-verify-sorting/ >>> >>> * Performance * >>> >>> I am a bit unsure how to get any relevant performance data for this change. What I have done so far is to create a class that has 65535 methods (which is the maximum - the length of the method array is u2) and I have measured how long it takes to sort this method array. The methods have random names but every hundredth method has 4 overloaded version. This makes sure that there are some duplicates in the array. >>> >>> For now I have run this on my Windows x64 work station with 4 cpus and on a Solaris Sparc machine with 2 cpus (uname says: SunOS sthsparc24 5.10 Generic_139555-08 sun4us sparc FJSV,GPUZC-M Solaris). >>> >>> I am attaching graphs for the results. The Y-axis has time in nano seconds. Judging by this my quicksort is a bit faster on Windows and a bit slower on Solaris. But the interesting thing is that the idempotent version is faster than the default behavior on Windows and on par with the default on Solaris. I assume that this is due to the fact that some stores can be avoided if we don't do swap of duplicates. This means that (at least on Windows) swap is more expensive than compare. If this is true we should probably remove the special treatment of idempotent. >>> >>> I could run this on more machines, but I am not sure how relevant this type of data is. Most method arrays will be much shorter and compared to reading the class from a file the sort will be in the noise. >>> >>> Long email...I hope I covered most of the issues here. Let me know if you have any questions. >>> >>> Thanks, >>> Bengt >>> > From bengt.rutisson at oracle.com Wed Jun 22 22:49:22 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 23 Jun 2011 07:49:22 +0200 Subject: Request for review (M): 7016112 CMS: crash during promotion testing In-Reply-To: <4E029B13.7030308@oracle.com> References: <4E009399.4060000@oracle.com> <4E010DEB.3030104@oracle.com> <4E029B13.7030308@oracle.com> Message-ID: <4E02D3E2.2010608@oracle.com> Coleen, Thanks for the review! On 2011-06-23 03:46, Coleen Phillimore wrote: > http://cr.openjdk.java.net/~brutisso/7016112/webrev.01/src/share/vm/utilities/quicksort.cpp.html > > 46 bool aIsOdd = a % 2 == 1; > 47 bool bIsOdd = b % 2 == 1; > Can you add () around the bit operations? Since the order precedence > of these operators is always surprising (at least to me). Yes, I'll do that. > I like the way you set the pattern for the internal testing > framework. Thanks for getting this started. Thanks, it was really useful to have the unit tests when I made this fix. > When we move these methodOops out of permgen, we might want to go back > to stdlib::quicksort. It would be less code for us to maintain. > Otherwise, this looks pretty straightforward and I like this fix. I agree that we should get back to stdlib::quicksort when it is safe to use it again. Thanks, Bengt > > Thanks, > Coleen > > > On 6/21/2011 5:32 PM, Bengt Rutisson wrote: >> >> Hi again, >> >> For completeness. Here is the graph for sorting maximum length arrays >> on Linux x64 (run on my laptop). These runs show that my >> implementation takes twice as long as the stdlib version. I am not >> happy about that, but I don't know how much effort it is worth to >> optimize for this case. >> >> Bengt >> >> >> On 2011-06-21 14:50, Bengt Rutisson wrote: >>> >>> Hi Runtime and GC, >>> >>> Sending this review request to both groups. I fixed a GC bug, but >>> the changes are in runtime code. >>> >>> The bug that I fixed is this one: >>> 7016112 CMS: crash during promotion testing >>> http://monaco.sfbay.sun.com/detail.jsf?cr=7016112 >>> >>> And here is the webrev: >>> http://cr.openjdk.java.net/~brutisso/7016112/webrev.01/ >>> >>> The investigation to find the root of the crashes reported in >>> 7016112 was quite lengthy. It is not that easy to read the CR and >>> figure out what is going on, so here is some background: >>> >>> When we load classes we store references to the methods of a class >>> in an object array. When we have loaded all methods we sort the >>> object array to allow binary search within the array. To do this >>> sort we use stdlib::qsort(), which is the standard library quicksort >>> implementation. >>> >>> If we are using CMS we might be doing concurrent marking while we >>> are sorting the object array. The object array can be found by the >>> concurrent marking code and it may start iterating over the array >>> while we are sorting it. The problem is that on Windows the >>> stdlib::qsort() is not implemented to do atomic updates of elements >>> in the array that it sorts. Instead it does a byte-by-byte swap when >>> it needs to swap two values. That is an easy way to implement >>> different element widths, but it means that at some point in time >>> one element may contain a few bytes from the element above or below. >>> If this happens at the same time as the marking code is trying to >>> read that element, we will be marking some random address and not >>> the method that was supposed to be marked. >>> >>> On Solaris and Linux the stdlib::qsort() implementations try to swap >>> as wide data types as possible so this issue should not occur there. >>> On the other hand we have no guarantees that this will always be how >>> stdlib::qsort() is implemented. >>> >>> After some discussions about different ways of solving this we came >>> to the conclusion that the simplest way is to implement our own >>> quicksort that operates on the correct pointer width (oop or >>> narrowOop). >>> >>> So, this is what I have done to fix this bug. >>> >>> Also, it is likely that this problem will go away when the perm gen >>> removal project is finished. Right now it looks like we will not be >>> tracing and marking methods at all after that change. >>> >>> * Questions * >>> >>> - Should we keep the bubble sort that is done before calling >>> quicksort in methodOopDesc::sort_methods() ? >>> >>> - Should we keep the idempotent option or should we try to always >>> use idempotent sorting (see performance test below)? >>> >>> - What is the best way to handle unit tests? I added a flag called >>> ExecuteInternalVMTests to run unit tests. This is in line with the >>> existing ErrorHandlerTest flag. My thought is that we can use this >>> same flag for other unit tests than just the quicksort tests. Would >>> be good if we could get these tests executed by JPRT as well. I >>> simply run these with "java -XX:+ExecuteInternalVMTests -version". >>> >>> >>> * Testing * >>> >>> Did the obvious testing: Ran JPRT with the changes in the webrev and >>> ran the failing nsk test from the bug >>> (nsk/sysdict/vm/stress/jck12a/sysdictj12a008) repeatedly for sevaral >>> days without failing. >>> >>> I created some unit tests for the quicksort implementation and they >>> all pass. >>> >>> I also made a build that sorts both with my own quicksort and with >>> stdlib::qsort and then compares that the arrays have the same sort >>> order. Ran this through JPRT and it seems to work on all platforms. >>> That run also included the unit tests. If anybody wants to see how >>> this testing was done, there is a separate webrev for that build. >>> The interesting code is in methodOop.cpp: >>> >>> http://cr.openjdk.java.net/~brutisso/7016112/webrev-verify-sorting/ >>> >>> * Performance * >>> >>> I am a bit unsure how to get any relevant performance data for this >>> change. What I have done so far is to create a class that has 65535 >>> methods (which is the maximum - the length of the method array is >>> u2) and I have measured how long it takes to sort this method array. >>> The methods have random names but every hundredth method has 4 >>> overloaded version. This makes sure that there are some duplicates >>> in the array. >>> >>> For now I have run this on my Windows x64 work station with 4 cpus >>> and on a Solaris Sparc machine with 2 cpus (uname says: SunOS >>> sthsparc24 5.10 Generic_139555-08 sun4us sparc FJSV,GPUZC-M Solaris). >>> >>> I am attaching graphs for the results. The Y-axis has time in nano >>> seconds. Judging by this my quicksort is a bit faster on Windows and >>> a bit slower on Solaris. But the interesting thing is that the >>> idempotent version is faster than the default behavior on Windows >>> and on par with the default on Solaris. I assume that this is due to >>> the fact that some stores can be avoided if we don't do swap of >>> duplicates. This means that (at least on Windows) swap is more >>> expensive than compare. If this is true we should probably remove >>> the special treatment of idempotent. >>> >>> I could run this on more machines, but I am not sure how relevant >>> this type of data is. Most method arrays will be much shorter and >>> compared to reading the class from a file the sort will be in the >>> noise. >>> >>> Long email...I hope I covered most of the issues here. Let me know >>> if you have any questions. >>> >>> Thanks, >>> Bengt >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110623/3d18fd0c/attachment-0001.html From tony.printezis at oracle.com Thu Jun 23 09:06:39 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Thu, 23 Jun 2011 12:06:39 -0400 Subject: CRR (S/M): 7049999: G1: Make the G1PrintHeapRegions output consistent and complete In-Reply-To: <4E00F24C.6070001@oracle.com> References: <4E00F24C.6070001@oracle.com> Message-ID: <4E03648F.8090201@oracle.com> Hi all, Is anyone looking at this? Any chance of getting it reviewed soon-sh so that I can push it by tomorrow? Tony On 06/21/2011 03:34 PM, Tony Printezis wrote: > Hi, > > Could I get a couple of folks to look at this? > > http://cr.openjdk.java.net/~tonyp/7049999/webrev.0/ > > +G1PrintHeapRegions is a very helpful debugging feature that we have > used a lot in the past. It'll be very helpful in the long run to > ensure that it generates more complete output to save us having to > deduce some of it. What was missing in the past was: > > * output for humongous region allocations > * output per region post compaction (as compaction changes the shape > of the heap) > * output when regions are reclaimed during cleanup > > Further enhancements include: > > * commit / uncommit events > * trimmed the output to keep the generated log files smaller while not > losing any information > * introduced the G1HRPrinter class and all output goes through it to > ensure consistency > > I attached an example of the new output. > > Tony > From john.cuthbertson at oracle.com Thu Jun 23 14:35:36 2011 From: john.cuthbertson at oracle.com (John Cuthbertson) Date: Thu, 23 Jun 2011 14:35:36 -0700 Subject: RFR(M/L): 6484982: G1: process references during evacuation pauses Message-ID: <4E03B1A8.7020703@oracle.com> Hi Everyone, I would like to get a couple of volunteers to review the code changes for this CR - the webrev can be found at http://cr.openjdk.java.net/~johnc/6484982/webrev.0/ Summary: G1 now contains 2 instances of the reference processor class - one for concurrent marking and the other for STW GCs (both full and incremental evacuation pauses). For evacuation pauses, during object scanning and RSet scanning I embed the STW reference processor into the OopClosures used to scan objects. This causes reference objects to be 'discovered' by the reference processor. Towards the end of the evacuation pause (just prior to retiring the the GC alloc regions) I have added the code to process these discovered reference objects, preserving (and copying) referent objects (and their reachable graphs) as appropriate. The code that does this makes extensive use of the existing copying oop closures and the G1ParScanThreadState structure (to handle to-space allocation). The code changes also include a couple of fixes that were exposed by the reference processing: * In satbQueue.cpp, the routine SATBMarkQueueSet::par_iterate_closure_all_threads() was claiming all JavaThreads (giving them one parity value) but skipping the VMThread. In a subsequent call to Thread::possibly_parallel_oops_do, the Java threads were successfully claimed but the VMThread was not. This could cause the VMThread's handle area to be skipped during the root scanning. * There were a couple of assignments to the discovered field of Reference objects that were not guarded by _discovery_needs_barrier resulting in the G1 C++ write-barrier to dirty the card spanning the Reference object's discovered field. This was causing the card table verification (during card table clearing) to fail. * There were also a couple of assignments of NULL to the next field of Reference objects causing the same symptom. Testing: The GC test suite (32/64 bit) (+UseG1GC, +UseG1GC +ExplicitGCInvokesConcurrent, +UseG1GC InitiatingHeapOccupancyPercent=5, +UseG1GC +ParallelRefProcEnabled), KitchenSink (48 hour runs with +UseG1GC, +UseG1GC +ExplicitGCInvokesConcurrent), OpenDS (+UseG1GC, +UseG1GC +ParallelRefProcEnabled), nsk GC and compiler tests, and jprt. Testing was conducted with the _is_alive_non_header field in the STW ref procssor both cleared and set (when cleared, more reference objects are 'discovered'). Thanks, JohnC From tony.printezis at oracle.com Thu Jun 23 19:31:57 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Thu, 23 Jun 2011 22:31:57 -0400 Subject: CRR (S/M): 7049999: G1: Make the G1PrintHeapRegions output consistent and complete In-Reply-To: <4E03648F.8090201@oracle.com> References: <4E00F24C.6070001@oracle.com> <4E03648F.8090201@oracle.com> Message-ID: <4E03F71D.3090606@oracle.com> Hi all, All set. Many thanks to John Cuthbertson and Jon Masa for the super prompt code reviews on this! Tony On 06/23/2011 12:06 PM, Tony Printezis wrote: > Hi all, > > Is anyone looking at this? Any chance of getting it reviewed soon-sh > so that I can push it by tomorrow? > > Tony > > On 06/21/2011 03:34 PM, Tony Printezis wrote: >> Hi, >> >> Could I get a couple of folks to look at this? >> >> http://cr.openjdk.java.net/~tonyp/7049999/webrev.0/ >> >> +G1PrintHeapRegions is a very helpful debugging feature that we have >> used a lot in the past. It'll be very helpful in the long run to >> ensure that it generates more complete output to save us having to >> deduce some of it. What was missing in the past was: >> >> * output for humongous region allocations >> * output per region post compaction (as compaction changes the shape >> of the heap) >> * output when regions are reclaimed during cleanup >> >> Further enhancements include: >> >> * commit / uncommit events >> * trimmed the output to keep the generated log files smaller while >> not losing any information >> * introduced the G1HRPrinter class and all output goes through it to >> ensure consistency >> >> I attached an example of the new output. >> >> Tony >> From john.coomes at oracle.com Thu Jun 23 20:40:50 2011 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 24 Jun 2011 03:40:50 +0000 Subject: hg: jdk7/hotspot-gc: Added tag jdk7-b146 for changeset 2d38c2a79c14 Message-ID: <20110624034050.BD987472AB@hg.openjdk.java.net> Changeset: 3e0b96f8f6a0 Author: schien Date: 2011-06-20 16:24 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/rev/3e0b96f8f6a0 Added tag jdk7-b146 for changeset 2d38c2a79c14 ! .hgtags From john.coomes at oracle.com Thu Jun 23 20:40:59 2011 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 24 Jun 2011 03:40:59 +0000 Subject: hg: jdk7/hotspot-gc/corba: Added tag jdk7-b146 for changeset 770227a4087e Message-ID: <20110624034103.807D4472AC@hg.openjdk.java.net> Changeset: 36f0efbc66ef Author: schien Date: 2011-06-20 16:25 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/corba/rev/36f0efbc66ef Added tag jdk7-b146 for changeset 770227a4087e ! .hgtags From john.coomes at oracle.com Thu Jun 23 20:41:12 2011 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 24 Jun 2011 03:41:12 +0000 Subject: hg: jdk7/hotspot-gc/jaxp: Added tag jdk7-b146 for changeset bcd31fa1e3c6 Message-ID: <20110624034112.89389472AD@hg.openjdk.java.net> Changeset: 9a4d09f33f01 Author: schien Date: 2011-06-20 16:25 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxp/rev/9a4d09f33f01 Added tag jdk7-b146 for changeset bcd31fa1e3c6 ! .hgtags From john.coomes at oracle.com Thu Jun 23 20:41:21 2011 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 24 Jun 2011 03:41:21 +0000 Subject: hg: jdk7/hotspot-gc/jaxws: 11 new changesets Message-ID: <20110624034121.E9252472AE@hg.openjdk.java.net> Changeset: 581dab3f0773 Author: asaha Date: 2011-04-21 16:15 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/581dab3f0773 Merge Changeset: 26610bb80151 Author: asaha Date: 2011-05-04 12:00 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/26610bb80151 Merge Changeset: c6ff860428c7 Author: asaha Date: 2011-05-05 22:28 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/c6ff860428c7 Merge Changeset: f4e1caef46d0 Author: asaha Date: 2011-05-24 11:11 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/f4e1caef46d0 Merge Changeset: 9896cee00786 Author: asaha Date: 2011-05-26 17:25 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/9896cee00786 Merge Changeset: d1febdcb0351 Author: asaha Date: 2011-05-26 21:36 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/d1febdcb0351 Merge Changeset: 239c80c331da Author: asaha Date: 2011-06-06 10:19 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/239c80c331da Merge Changeset: 09412171ca4b Author: asaha Date: 2011-06-03 07:54 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/09412171ca4b Merge Changeset: 9d8fd0982fb8 Author: asaha Date: 2011-06-06 10:54 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/9d8fd0982fb8 Merge Changeset: 05469dd4c366 Author: lana Date: 2011-06-15 16:04 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/05469dd4c366 Merge Changeset: faa394edbfe3 Author: schien Date: 2011-06-20 16:25 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/faa394edbfe3 Added tag jdk7-b146 for changeset 05469dd4c366 ! .hgtags From john.coomes at oracle.com Thu Jun 23 20:44:21 2011 From: john.coomes at oracle.com (john.coomes at oracle.com) Date: Fri, 24 Jun 2011 03:44:21 +0000 Subject: hg: jdk7/hotspot-gc/jdk: 44 new changesets Message-ID: <20110624035242.31293472B4@hg.openjdk.java.net> Changeset: cf0632d2db2c Author: jrose Date: 2011-06-14 22:47 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/cf0632d2db2c 7052202: JSR 292: Crash in sun.invoke.util.ValueConversions.fillArray Summary: Fix corner cases involving MethodHandles.permuteArguments with long or double argument lists. Reviewed-by: twisti, never ! src/share/classes/java/lang/invoke/AdapterMethodHandle.java ! src/share/classes/java/lang/invoke/MethodHandleImpl.java ! src/share/classes/java/lang/invoke/MethodHandleNatives.java ! src/share/classes/java/lang/invoke/SwitchPoint.java + test/java/lang/invoke/PermuteArgsTest.java Changeset: ae731399e525 Author: dav Date: 2011-06-07 22:58 +0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/ae731399e525 7048568: Crash in Java_sun_awt_Win32GraphicsEnvironment_isVistaOS Reviewed-by: dcherepanov, art, amenkov ! src/windows/native/sun/windows/awt_Win32GraphicsDevice.cpp Changeset: f08fcae94813 Author: lana Date: 2011-06-10 11:43 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/f08fcae94813 Merge Changeset: 6e961c328276 Author: michaelm Date: 2011-06-08 10:56 +0100 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/6e961c328276 7050028: ISE "zip file closed" from JarURLConnection.getInputStream on JDK 7 when !useCaches Reviewed-by: chegar, alanb ! src/share/classes/sun/misc/URLClassPath.java + test/java/net/URLClassLoader/B7050028.java Changeset: b6ced5ad7a62 Author: dwanvik Date: 2011-06-10 17:44 +0200 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/b6ced5ad7a62 7046557: Changes to the Java DB README files in JDK7 Summary: Update /README.html with correct mention of Java DB, add JDK specific README files to /db and /demo/db. Reviewed-by: ohair ! make/common/Release.gmk Changeset: 646ab254ff80 Author: lana Date: 2011-06-10 11:44 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/646ab254ff80 Merge Changeset: aca0dc2b921c Author: weijun Date: 2011-02-09 11:50 +0800 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/aca0dc2b921c 6618658: Deserialization allows creation of mutable SignedObject Reviewed-by: hawtin, mullan ! src/share/classes/java/security/SignedObject.java + test/java/security/SignedObject/Correctness.java Changeset: df445f522425 Author: bae Date: 2011-02-17 12:21 +0300 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/df445f522425 7013519: [parfait] Integer overflows in 2D code Reviewed-by: prr, valeriep ! src/share/native/sun/awt/image/jpeg/imageioJPEG.c ! src/share/native/sun/font/layout/SunLayoutEngine.cpp Changeset: ccb2fcfb6d6b Author: chegar Date: 2011-02-18 13:31 +0000 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/ccb2fcfb6d6b 7013969: NetworkInterface.toString can reveal bindings Reviewed-by: alanb, michaelm, hawtin ! src/share/classes/java/net/NetworkInterface.java Changeset: 026adaac71f1 Author: dcherepanov Date: 2011-02-25 15:54 +0300 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/026adaac71f1 7012520: Heap overflow vulnerability in FileDialog.show() Reviewed-by: art, anthony ! src/windows/native/sun/windows/awt_FileDialog.cpp Changeset: d489f00d6c65 Author: flar Date: 2011-03-02 05:35 -0800 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/d489f00d6c65 7016495: Crash in Java 2D transforming an image with scale close to zero Reviewed-by: prr, bae ! src/share/classes/sun/java2d/pipe/DrawImage.java ! src/share/native/sun/java2d/loops/TransformHelper.c + test/java/awt/image/BufferedImage/TinyScale.java Changeset: fe27fe44ac51 Author: ksrini Date: 2011-03-03 14:16 -0800 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/fe27fe44ac51 7016985: (launcher) implement safe secure dll loading Reviewed-by: mchung ! src/windows/bin/java_md.c Changeset: 0efa64f13302 Author: chegar Date: 2011-04-05 14:49 +0100 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/0efa64f13302 7033865: JDK: Add private API for secure/restrictive loading of system dlls Reviewed-by: alanb ! src/share/native/common/jdk_util.h + src/solaris/native/common/jdk_util_md.h ! src/windows/native/common/jdk_util_md.c + src/windows/native/common/jdk_util_md.h Changeset: 67992a58bfba Author: ksrini Date: 2011-04-05 16:19 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/67992a58bfba 7032593: DLL_LOADING: Upgrade solution to 7016985 to reflect JDK7 solution Reviewed-by: mchung, asaha ! src/windows/bin/java_md.c Changeset: 7181441faf72 Author: dcherepanov Date: 2011-04-08 16:44 +0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/7181441faf72 7003962: AWT: securely load DLLs and launch executables using fully qualified path Reviewed-by: art, bae, alanb ! src/windows/native/sun/java2d/d3d/D3DPipelineManager.cpp ! src/windows/native/sun/java2d/opengl/OGLFuncs_md.h ! src/windows/native/sun/windows/DllUtil.cpp ! src/windows/native/sun/windows/DllUtil.h ! src/windows/native/sun/windows/ShellFolder2.cpp ! src/windows/native/sun/windows/ThemeReader.cpp ! src/windows/native/sun/windows/awt_Mlib.cpp ! src/windows/native/sun/windows/awt_TextArea.cpp ! src/windows/native/sun/windows/awt_TrayIcon.cpp ! src/windows/native/sun/windows/awt_Win32GraphicsEnv.cpp ! src/windows/native/sun/windows/stdhdrs.h Changeset: 05a3923f516f Author: dcherepanov Date: 2011-04-08 17:58 +0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/05a3923f516f 7035077: Minor addition to the changes for 7003962 Reviewed-by: chegar ! src/windows/native/sun/windows/DllUtil.cpp Changeset: afcc1530e68b Author: asaha Date: 2011-04-08 10:27 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/afcc1530e68b Merge - make/java/dyn/Makefile - src/share/classes/java/dyn/CallSite.java - src/share/classes/java/dyn/ClassValue.java - src/share/classes/java/dyn/ConstantCallSite.java - src/share/classes/java/dyn/InvokeDynamic.java - src/share/classes/java/dyn/InvokeDynamicBootstrapError.java - src/share/classes/java/dyn/Linkage.java - src/share/classes/java/dyn/MethodHandle.java - src/share/classes/java/dyn/MethodHandles.java - src/share/classes/java/dyn/MethodType.java - src/share/classes/java/dyn/MethodTypeForm.java - src/share/classes/java/dyn/MutableCallSite.java - src/share/classes/java/dyn/SwitchPoint.java - src/share/classes/java/dyn/VolatileCallSite.java - src/share/classes/java/dyn/WrongMethodTypeException.java - src/share/classes/java/dyn/package-info.java - src/share/classes/sun/dyn/Access.java - src/share/classes/sun/dyn/AdapterMethodHandle.java - src/share/classes/sun/dyn/BoundMethodHandle.java - src/share/classes/sun/dyn/CallSiteImpl.java - src/share/classes/sun/dyn/DirectMethodHandle.java - src/share/classes/sun/dyn/FilterGeneric.java - src/share/classes/sun/dyn/FilterOneArgument.java - src/share/classes/sun/dyn/FromGeneric.java - src/share/classes/sun/dyn/InvokeGeneric.java - src/share/classes/sun/dyn/Invokers.java - src/share/classes/sun/dyn/MemberName.java - src/share/classes/sun/dyn/MethodHandleImpl.java - src/share/classes/sun/dyn/MethodHandleNatives.java - src/share/classes/sun/dyn/MethodTypeImpl.java - src/share/classes/sun/dyn/SpreadGeneric.java - src/share/classes/sun/dyn/ToGeneric.java - src/share/classes/sun/dyn/WrapperInstance.java - src/share/classes/sun/dyn/anon/AnonymousClassLoader.java - src/share/classes/sun/dyn/anon/ConstantPoolParser.java - src/share/classes/sun/dyn/anon/ConstantPoolPatch.java - src/share/classes/sun/dyn/anon/ConstantPoolVisitor.java - src/share/classes/sun/dyn/anon/InvalidConstantPoolFormatException.java - src/share/classes/sun/dyn/empty/Empty.java - src/share/classes/sun/dyn/package-info.java - src/share/classes/sun/dyn/util/BytecodeDescriptor.java - src/share/classes/sun/dyn/util/BytecodeName.java - src/share/classes/sun/dyn/util/ValueConversions.java - src/share/classes/sun/dyn/util/VerifyAccess.java - src/share/classes/sun/dyn/util/VerifyType.java - src/share/classes/sun/dyn/util/Wrapper.java - src/share/classes/sun/dyn/util/package-info.java ! src/share/native/sun/awt/image/jpeg/imageioJPEG.c - test/java/dyn/ClassValueTest.java - test/java/dyn/InvokeDynamicPrintArgs.java - test/java/dyn/InvokeGenericTest.java - test/java/dyn/JavaDocExamplesTest.java - test/java/dyn/MethodHandlesTest.java - test/java/dyn/MethodTypeTest.java - test/java/dyn/indify/Indify.java Changeset: 557bd9b5d92f Author: asaha Date: 2011-04-08 10:31 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/557bd9b5d92f Merge Changeset: e142148d8b54 Author: asaha Date: 2011-04-12 14:23 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/e142148d8b54 Merge - src/share/classes/sun/security/ssl/DefaultSSLContextImpl.java Changeset: 76e0e562b617 Author: dcherepanov Date: 2011-04-15 17:06 +0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/76e0e562b617 7036952: build warning after the changes for 7003962 Reviewed-by: art, bae ! src/windows/native/sun/java2d/opengl/OGLFuncs_md.h Changeset: f8eddc85cc02 Author: zgu Date: 2011-04-15 09:53 -0400 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/f8eddc85cc02 7003964: SERV: securely load DLLs and launch executables using fully qualified path Summary: Linked in Windows libraries that are available on jdk7 supported platforms, and used GetModuleHandle instead of LoadLibrary for already loaded Dlls. Reviewed-by: dcubed, alanb ! make/com/sun/tools/attach/Makefile ! src/windows/classes/sun/tools/attach/WindowsAttachProvider.java ! src/windows/native/sun/tools/attach/WindowsAttachProvider.c ! src/windows/native/sun/tools/attach/WindowsVirtualMachine.c ! src/windows/native/sun/tracing/dtrace/jvm_symbols_md.c ! src/windows/npt/npt_md.h Changeset: 0865aa0ad9b2 Author: zgu Date: 2011-04-19 10:26 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/0865aa0ad9b2 Merge Changeset: 6f8a4d334fb2 Author: asaha Date: 2011-04-20 09:31 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/6f8a4d334fb2 Merge ! make/com/sun/tools/attach/Makefile ! src/share/classes/java/net/NetworkInterface.java ! src/share/native/sun/awt/image/jpeg/imageioJPEG.c ! src/windows/bin/java_md.c ! src/windows/native/sun/windows/awt_TrayIcon.cpp ! src/windows/native/sun/windows/awt_Win32GraphicsEnv.cpp Changeset: f3645b5d6e62 Author: asaha Date: 2011-04-20 21:24 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/f3645b5d6e62 Merge Changeset: b626f78c57e1 Author: asaha Date: 2011-04-21 08:38 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/b626f78c57e1 Merge Changeset: cec45f3353be Author: asaha Date: 2011-04-21 08:37 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/cec45f3353be Merge Changeset: 6133c9ee3a01 Author: asaha Date: 2011-04-21 08:39 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/6133c9ee3a01 Merge Changeset: dd06e8d3da91 Author: asaha Date: 2011-04-21 15:43 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/dd06e8d3da91 Merge Changeset: b2295905901a Author: asaha Date: 2011-04-21 16:42 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/b2295905901a Merge - src/share/classes/sun/security/ssl/DefaultSSLContextImpl.java Changeset: 3fedf261fb4f Author: valeriep Date: 2011-04-26 15:59 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/3fedf261fb4f 7003952: SEC: securely load DLLs and launch executables using fully qualified path Summary: Enforce full path when specifying library locations. Reviewed-by: wetmore, ohair ! make/sun/security/pkcs11/Makefile ! src/share/classes/sun/security/pkcs11/Config.java ! src/share/classes/sun/security/pkcs11/Secmod.java ! src/share/native/sun/security/pkcs11/j2secmod.c + test/sun/security/pkcs11/Provider/Absolute.cfg + test/sun/security/pkcs11/Provider/Absolute.java Changeset: 94ea3b8288f1 Author: alexp Date: 2011-05-04 11:35 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/94ea3b8288f1 7020198: ImageIcon creates Component with null acc Reviewed-by: rupashka, hawtin ! src/share/classes/javax/swing/ImageIcon.java Changeset: e6fdfb249e31 Author: asaha Date: 2011-05-04 16:39 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/e6fdfb249e31 Merge - src/share/native/sun/font/layout/Features.h ! src/windows/native/sun/windows/awt_FileDialog.cpp - test/javax/swing/text/GlyphView/6539700/bug6539700.java Changeset: 49244980d692 Author: asaha Date: 2011-05-05 22:29 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/49244980d692 Merge - src/share/classes/sun/security/util/SignatureFileManifest.java Changeset: 647b031200f0 Author: asaha Date: 2011-05-06 14:33 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/647b031200f0 Merge Changeset: 92b5197e9ff5 Author: asaha Date: 2011-05-26 21:37 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/92b5197e9ff5 Merge ! src/windows/native/sun/java2d/d3d/D3DPipelineManager.cpp ! src/windows/native/sun/windows/awt_TrayIcon.cpp Changeset: cca9ea306c6e Author: asaha Date: 2011-05-26 21:51 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/cca9ea306c6e Merge Changeset: dab3e66ebda7 Author: lana Date: 2011-06-06 19:04 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/dab3e66ebda7 Merge Changeset: 9f17be5136d1 Author: wetmore Date: 2011-06-09 14:24 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/9f17be5136d1 7052537: java/security/Security/NotInstalledProviders.java is causing -samevm tests to fail. Reviewed-by: valeriep, asaha, alanb ! test/java/security/Security/NoInstalledProviders.java Changeset: 4961be00d3b5 Author: lana Date: 2011-06-15 16:10 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/4961be00d3b5 Merge Changeset: a65fa0f6717e Author: trims Date: 2011-06-17 16:25 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/a65fa0f6717e Merge Changeset: c46f97579fe6 Author: alanb Date: 2011-06-17 16:47 +0100 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/c46f97579fe6 7055508: (aio) EXCEPTION_ACCESS_VIOLATION in AsynchronousSocketChannel.connect on Windows 7 Reviewed-by: chegar ! src/windows/native/sun/nio/ch/WindowsAsynchronousSocketChannelImpl.c Changeset: c102e1221afa Author: lana Date: 2011-06-17 10:27 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/c102e1221afa Merge Changeset: 539e576793a8 Author: lana Date: 2011-06-18 10:12 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/539e576793a8 Merge Changeset: 7b4f4230fecf Author: schien Date: 2011-06-20 16:25 -0700 URL: http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/7b4f4230fecf Added tag jdk7-b146 for changeset 539e576793a8 ! .hgtags From tony.printezis at oracle.com Fri Jun 24 15:42:32 2011 From: tony.printezis at oracle.com (tony.printezis at oracle.com) Date: Fri, 24 Jun 2011 22:42:32 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7049999: G1: Make the G1PrintHeapRegions output consistent and complete Message-ID: <20110624224236.E6843472E7@hg.openjdk.java.net> Changeset: 5f6f2615433a Author: tonyp Date: 2011-06-24 12:38 -0400 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/5f6f2615433a 7049999: G1: Make the G1PrintHeapRegions output consistent and complete Summary: Extend and make more consistent the output from the G1PrintHeapRegions flag. Reviewed-by: johnc, jmasa ! src/share/vm/gc_implementation/g1/concurrentMark.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp ! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp ! src/share/vm/gc_implementation/g1/g1CollectorPolicy.cpp + src/share/vm/gc_implementation/g1/g1HRPrinter.cpp + src/share/vm/gc_implementation/g1/g1HRPrinter.hpp From bengt.rutisson at oracle.com Mon Jun 27 00:42:18 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 27 Jun 2011 09:42:18 +0200 Subject: Nominating Mikael Gerdin to be an OpenJDK committer and a reviewer Message-ID: <4E08345A.6000901@oracle.com> Hi all, With the new OpenJDK bylaws almost in place we need to follow the process that they outline to get OpenJDK user names even for the people working at Oracle. Mikael Gerdin does not have an OpenJDK user name yet. He helped me a lot with the CMS bug that I have been working on during the spring and he also reviewed and had good comments on my quicksort implementation. I would very much like to list him as a reviewer when I push this change. To grant Mikael reviewer status in OpenJDK we need to first make him a committer (lazy consensus - no vetoes) and then vote him in as a reviewer (at least three votes in favor and no vetoes). Since this is blocking my push, could I please have some votes for making Mikael Gerdin a committer and a reviewer? Relevant part of the bylaws: http://openjdk.java.net/groups/gb/bylaws/draft-openjdk-bylaws-10#_7 Copying Mark on this thread since this is the first time this is done and I would like to know that we are on the right track. Thanks, Bengt From jesper.wilhelmsson at oracle.com Mon Jun 27 01:50:19 2011 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Mon, 27 Jun 2011 10:50:19 +0200 Subject: Nominating Mikael Gerdin to be an OpenJDK committer and a reviewer In-Reply-To: <4E08345A.6000901@oracle.com> References: <4E08345A.6000901@oracle.com> Message-ID: <4E08444B.9060304@oracle.com> I vote in favor to grant Mikael the holy title as committer and reviewer. And let me state for the record that I think this process is ridiculous when applied to Oracle employed JVM developers. If we don't trust someone to review our code then maybe we shouldn't hire them in the first place. /Jesper On 06/27/2011 09:42 AM, Bengt Rutisson wrote: > > Hi all, > > With the new OpenJDK bylaws almost in place we need to follow the process that > they outline to get OpenJDK user names even for the people working at Oracle. > Mikael Gerdin does not have an OpenJDK user name yet. He helped me a lot with > the CMS bug that I have been working on during the spring and he also reviewed > and had good comments on my quicksort implementation. I would very much like > to list him as a reviewer when I push this change. > > To grant Mikael reviewer status in OpenJDK we need to first make him a > committer (lazy consensus - no vetoes) and then vote him in as a reviewer (at > least three votes in favor and no vetoes). > > Since this is blocking my push, could I please have some votes for making > Mikael Gerdin a committer and a reviewer? > > Relevant part of the bylaws: > http://openjdk.java.net/groups/gb/bylaws/draft-openjdk-bylaws-10#_7 > > Copying Mark on this thread since this is the first time this is done and I > would like to know that we are on the right track. > > Thanks, > Bengt From tony.printezis at oracle.com Mon Jun 27 05:11:20 2011 From: tony.printezis at oracle.com (Tony Printezis) Date: Mon, 27 Jun 2011 08:11:20 -0400 Subject: Nominating Mikael Gerdin to be an OpenJDK committer and a reviewer In-Reply-To: <4E08345A.6000901@oracle.com> References: <4E08345A.6000901@oracle.com> Message-ID: <4E087368.3010901@oracle.com> Totally in favor. Tony On 6/27/2011 3:42 AM, Bengt Rutisson wrote: > > Hi all, > > With the new OpenJDK bylaws almost in place we need to follow the > process that they outline to get OpenJDK user names even for the > people working at Oracle. Mikael Gerdin does not have an OpenJDK user > name yet. He helped me a lot with the CMS bug that I have been working > on during the spring and he also reviewed and had good comments on my > quicksort implementation. I would very much like to list him as a > reviewer when I push this change. > > To grant Mikael reviewer status in OpenJDK we need to first make him a > committer (lazy consensus - no vetoes) and then vote him in as a > reviewer (at least three votes in favor and no vetoes). > > Since this is blocking my push, could I please have some votes for > making Mikael Gerdin a committer and a reviewer? > > Relevant part of the bylaws: > http://openjdk.java.net/groups/gb/bylaws/draft-openjdk-bylaws-10#_7 > > Copying Mark on this thread since this is the first time this is done > and I would like to know that we are on the right track. > > Thanks, > Bengt From james.melvin at oracle.com Mon Jun 27 06:42:31 2011 From: james.melvin at oracle.com (James Melvin) Date: Mon, 27 Jun 2011 09:42:31 -0400 Subject: Nominating Mikael Gerdin to be an OpenJDK committer and a reviewer In-Reply-To: <4E08345A.6000901@oracle.com> References: <4E08345A.6000901@oracle.com> Message-ID: <4E0888C7.3090905@oracle.com> Fully supportive! - Jim On 6/27/11 3:42 AM, Bengt Rutisson wrote: > > Hi all, > > With the new OpenJDK bylaws almost in place we need to follow the > process that they outline to get OpenJDK user names even for the people > working at Oracle. Mikael Gerdin does not have an OpenJDK user name yet. > He helped me a lot with the CMS bug that I have been working on during > the spring and he also reviewed and had good comments on my quicksort > implementation. I would very much like to list him as a reviewer when I > push this change. > > To grant Mikael reviewer status in OpenJDK we need to first make him a > committer (lazy consensus - no vetoes) and then vote him in as a > reviewer (at least three votes in favor and no vetoes). > > Since this is blocking my push, could I please have some votes for > making Mikael Gerdin a committer and a reviewer? > > Relevant part of the bylaws: > http://openjdk.java.net/groups/gb/bylaws/draft-openjdk-bylaws-10#_7 > > Copying Mark on this thread since this is the first time this is done > and I would like to know that we are on the right track. > > Thanks, > Bengt From y.s.ramakrishna at oracle.com Mon Jun 27 10:46:16 2011 From: y.s.ramakrishna at oracle.com (Y. Srinivas Ramakrishna) Date: Mon, 27 Jun 2011 10:46:16 -0700 Subject: G1GC: many consecutive full gcs (was Re: hotspot-gc-use Digest, Vol 40, Issue 4) In-Reply-To: <4E08890A.9030601@gmx.de> References: <4DFA81A3.1050802@oracle.com> <4DFAFB95.7070405@gmx.de> <4DFB6E7B.8000200@oracle.com> <4E08890A.9030601@gmx.de> Message-ID: <4E08C1E8.9000609@oracle.com> No smoking guns yet. I should have asked for this earlier, sorry i forgot... Could you share data from +PrintHeapAtGC +PrintHeapAtGCExtended ? -- ramki On 6/27/2011 6:43 AM, Stefan Wachter wrote: > Hi Ramki, > > I took a heap dump and checked for WeakReferences and Finalizer objects. > > The heap dump contained 36,071,040 instances with 12,405 instances being WeakReference instances. I > am not sure if this is a lot or not. The "Basic info" section that is displayed in NetBeans reports > "Number of objects pending for finalization: 0". Corresponds this to the number of Finalizer objects > you asked for? > > The complete heap summary is: > > Total bytes: 2,472,537,236 > Total classes: 15,953 > Total instances: 36,071,040 > Classloaders: 648 > GC roots: 3,138 > Number of objects pending for finalization: 0 > > > Cheers, > > Stefan > > On 06/17/2011 05:10 PM, Y. Srinivas Ramakrishna wrote: >> Thanks for the responses and the extra information, Stefan and Alex. >> >> Stefan, please try PermSize == MaxPermSize at the next opportunity, >> although it's likely not to have any effect based on a preliminary >> reading of the info you provided below. >> >> As regards class histogram, please note that this would induce a >> pause as long as the full gc's you see (or may be a bit shorter), >> so use it with care when you obtain that information. >> >> Meanwhile, we'll look over the information you have provided and >> get back to you once we have made some headway. Thanks for bringing >> this to our attention and for the data so far. >> >> -- ramki >> >> On 6/17/2011 12:00 AM, Stefan Wachter wrote: >>> Hi Ramki, >>> >>> thanks for your interest. See my answers below. >>> >>> Cheers, >>> >>> Stefan >>> >>> >>> On 06/17/2011 12:20 AM, Y. S. Ramakrishna wrote: >>>> Stefan and Alex -- >>>> >>>> (1) what's the version of the JVM used? >>> >>> Java(TM) SE Runtime Environment (build 1.6.0_24-b07) >>> Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode) >>> >>>> >>>> (2) could you try PermSize == MaxPermSize and see if it made any >>>> difference? >>> >>> PermSize was not set until now. I try this setting and can report results the next time our cluster >>> nodes are switched. >>> >>>> >>>> (3) do you have lots of WeakReference or Finalizer objects in your system? >>> >>> I have no information about this. I guess a heap dump would provide that information. I will ask our >>> admins if it is possible to take a heap dump. >>> >>>> >>>> (4) is what you have below your complete GC log (i.e. no gaps between first and last line?) >>> >>> Yes, that is the original output of the GC. No lines were removed. >>>> >>>> (5) what do you see when using CMS (in case you tried it)? >>> >>> CMS seems to behave better (see attached gc.log). The settings used with CMS are: >>> >>> -Xmx10000M >>> -Xms1024M >>> -XX:MaxPermSize=1024m >>> -XX:+PrintGCDateStamps >>> -XX:+UseConcMarkSweepGC >>> >>>> >>>> (6) do you have a simple test case that you can share with >>>> us that exhibits this behaviour? >>> >>> Unfortunately I have no simple test case. The output is from our life system running a tomcat with >>> various wars deployed. >>> >>>> >>>> thanks! >>>> -- ramki >>>> >>>> On 06/16/11 14:02, Alex Aisinzon wrote: >>>>> Stefan >>>>> >>>>> I have had similar experience with G1 and reported it here. >>>>> I tried the option to try to limit GC pauses (-XX:MaxGCPauseMillis with >>>>> 1s and then 2s). It did not improve things. >>>>> I am happy to try anything else to help improve G1. >>>>> >>>>> Alex A >>>>> >>>>> -----Original Message----- >>>>> From: hotspot-gc-use-bounces at openjdk.java.net >>>>> [mailto:hotspot-gc-use-bounces at openjdk.java.net] On Behalf Of >>>>> hotspot-gc-use-request at openjdk.java.net >>>>> Sent: Thursday, June 16, 2011 12:00 PM >>>>> To: hotspot-gc-use at openjdk.java.net >>>>> Subject: hotspot-gc-use Digest, Vol 40, Issue 4 >>>>> >>>>> Send hotspot-gc-use mailing list submissions to >>>>> hotspot-gc-use at openjdk.java.net >>>>> >>>>> To subscribe or unsubscribe via the World Wide Web, visit >>>>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use >>>>> or, via email, send a message with subject or body 'help' to >>>>> hotspot-gc-use-request at openjdk.java.net >>>>> >>>>> You can reach the person managing the list at >>>>> hotspot-gc-use-owner at openjdk.java.net >>>>> >>>>> When replying, please edit your Subject line so it is more specific >>>>> than "Re: Contents of hotspot-gc-use digest..." >>>>> >>>>> >>>>> Today's Topics: >>>>> >>>>> 1. G1GC: many consecutive full gcs (Stefan Wachter) >>>>> >>>>> >>>>> ---------------------------------------------------------------------- >>>>> >>>>> Message: 1 >>>>> Date: Thu, 16 Jun 2011 09:23:05 +0200 >>>>> From: Stefan Wachter >>>>> Subject: G1GC: many consecutive full gcs >>>>> To: hotspot-gc-use at openjdk.java.net >>>>> Message-ID: <4DF9AF59.90708 at gmx.de> >>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >>>>> >>>>> Hi all, >>>>> >>>>> the G1GC runs nicely for some time but then it starts to have bursts of consecutive full gcs (see >>>>> log below). Can anyone explain that behaviour? >>>>> >>>>> The JVM is stared with these options: >>>>> >>>>> -Xmx10000M >>>>> -Xms1024M >>>>> -XX:MaxPermSize=1024m >>>>> -XX:+UseG1GC >>>>> -XX:MaxGCPauseMillis=2000 >>>>> -XX:GCPauseIntervalMillis=10000 >>>>> >>>>> --Stefan >>>>> >>>>> >>>>> 31314.569: [GC pause (young) 3286M->3234M(8937M), 0.0485530 secs] >>>>> 31374.932: [GC pause (young) 3279M->3236M(8937M), 0.0486560 secs] >>>>> 31837.688: [GC pause (young) 4614M->3252M(8937M), 0.0739370 secs] >>>>> 33528.784: [GC pause (young) 5997M->3330M(8937M), 0.1543940 secs] >>>>> 35372.029: [GC pause (young) 6038M->3405M(8937M), 0.2364930 secs] >>>>> 36508.343: [Full GC 8525M->2903M(9678M), 8.9121160 secs] >>>>> 38204.291: [Full GC 8647M->2295M(10000M), 7.4105940 secs] >>>>> 39038.017: [Full GC 8790M->2092M(6975M), 6.7242110 secs] >>>>> 39047.736: [Full GC 8996M->2089M(9998M), 6.0741630 secs] >>>>> 39095.551: [Full GC 8936M->2090M(6969M), 5.5573390 secs] >>>>> 39104.326: [Full GC 9175M->2092M(9997M), 5.3413860 secs] >>>>> 39112.760: [Full GC 9392M->2094M(9978M), 5.4884150 secs] >>>>> 41965.304: [Full GC 9549M->2112M(7043M), 6.4774550 secs] >>>>> 43830.656: [GC pause (young) 4615M->2309M(7043M), 0.1727720 secs] >>>>> 46024.398: [GC pause (young) 4638M->2497M(7043M), 0.3039900 secs] >>>>> 46068.009: [Full GC 8832M->2237M(10000M), 6.6717340 secs] >>>>> 46077.691: [Full GC 9322M->2187M(9987M), 5.8603570 secs] >>>>> 46086.731: [Full GC 9487M->2189M(9996M), 5.8521360 secs] >>>>> 46124.713: [Full GC 8792M->2185M(9895M), 5.7782260 secs] >>>>> 46133.366: [Full GC 9297M->2188M(9999M), 5.7263360 secs] >>>>> 46142.238: [Full GC 9509M->2189M(9985M), 5.5704850 secs] >>>>> 47454.530: [Full GC 9676M->2295M(7653M), 7.3169840 secs] >>>>> 48163.422: [GC pause (young) 4937M->2431M(7653M), 0.2379570 secs] >>>>> 48887.388: [GC pause (young) 5007M->2497M(7653M), 0.1999790 secs] >>>>> >>>>> >>>>> >>>>> ------------------------------ >>>>> >>>>> _______________________________________________ >>>>> hotspot-gc-use mailing list >>>>> hotspot-gc-use at openjdk.java.net >>>>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use >>>>> >>>>> >>>>> End of hotspot-gc-use Digest, Vol 40, Issue 4 >>>>> ********************************************* >>>>> _______________________________________________ >>>>> hotspot-gc-use mailing list >>>>> hotspot-gc-use at openjdk.java.net >>>>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use >>>> >>> >> >> > _______________________________________________ hotspot-gc-use mailing list hotspot-gc-use at openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use From y.s.ramakrishna at oracle.com Mon Jun 27 10:51:12 2011 From: y.s.ramakrishna at oracle.com (Y. Srinivas Ramakrishna) Date: Mon, 27 Jun 2011 10:51:12 -0700 Subject: Nominating Mikael Gerdin to be an OpenJDK committer and a reviewer In-Reply-To: <4E08345A.6000901@oracle.com> References: <4E08345A.6000901@oracle.com> Message-ID: <4E08C310.8020507@oracle.com> Vote: yes I support him in the role of both a reviewer and a committer. -- ramki On 6/27/2011 12:42 AM, Bengt Rutisson wrote: > > Hi all, > > With the new OpenJDK bylaws almost in place we need to follow the process that they outline to get > OpenJDK user names even for the people working at Oracle. Mikael Gerdin does not have an OpenJDK > user name yet. He helped me a lot with the CMS bug that I have been working on during the spring and > he also reviewed and had good comments on my quicksort implementation. I would very much like to > list him as a reviewer when I push this change. > > To grant Mikael reviewer status in OpenJDK we need to first make him a committer (lazy consensus - > no vetoes) and then vote him in as a reviewer (at least three votes in favor and no vetoes). > > Since this is blocking my push, could I please have some votes for making Mikael Gerdin a committer > and a reviewer? > > Relevant part of the bylaws: > http://openjdk.java.net/groups/gb/bylaws/draft-openjdk-bylaws-10#_7 > > Copying Mark on this thread since this is the first time this is done and I would like to know that > we are on the right track. > > Thanks, > Bengt From bengt.rutisson at oracle.com Mon Jun 27 16:22:15 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 28 Jun 2011 01:22:15 +0200 Subject: Request for review (M): 7016112 CMS: crash during promotion testing In-Reply-To: References: <4E009399.4060000@oracle.com> <4E01DD68.4060007@oracle.com> Message-ID: <4E0910A7.1050604@oracle.com> Tom, Coleen and Ramki, Thanks for the reviews! Here is an updated webrev that I hope includes fixes to all the comments that you had: http://cr.openjdk.java.net/~brutisso/7016112/webrev.03/ The main change compared to the previous version is that I skipped the "middle layer sort" in objArrayOop. This was based on a comment from Ramki that this sort would not be safe for all collectors anyway. Better in that case to do the dirtying of cards directly in methodOop. I also looked some more at performance. It was bugging me that the Linux implementation of stdlib::qsort was so much faster than my version. I picked up some of the optimization that the Linux stdlib::qsort does and this made my implementation quite a bit faster. Not just on Linux, but also on Windows and Solaris. As always there is a trade-off between clean code and fast code. I basically tweaked quickSort.hpp in four ways: 1) Use insertion sort if the length of the array is less than 7 2) Use insertion sort if it appears that the array is almost sorted (partition did not have to do any calls to swap()) 3) Added inline to all inner methods 4) Use more samples for calculating the pivot element if the length of the array is larger than 40 Of these, 4) is the one that has the biggest impact. But they all contribute in some extent to better performance. Here is a webrev with these optimizations. It just contains the changes compared to the webrev above: http://cr.openjdk.java.net/~brutisso/7016112/webrev.03-opt/ I would like to push the optimized version, but I would like to hear your thougths. If you think it is too ugly I am ok with pusing the non-optimized version. I did some runs on Win x64, Linux x64 and Solaris x64 using my test class with 65535 methods. As you can see from the graphs that I attach, my implementation is still behind on Linux but not quite as bad as before. The Linux and Windows runs are made on the same dual-booted computer. So they have identical hardware. It is interesting to see that both my implementations (non-optimized and optimized) take about the same time on Windows and Linux, but stdlib:qsort is much faster on Linux than on Windows. The Y-axis in the graphs now has time in seconds. I also ran RefWorkLoad a bit. I will run it some more, but so far I have not seen any significant differences there. I'll probably focus my runs on Linux to see if I can see any regressions. Any thoughts on this? I would like to push this within the next few days since I will be going on vacation next week. Thanks, Bengt On 2011-06-22 23:55, Tom Rodriguez wrote: > On Jun 22, 2011, at 5:17 AM, Bengt Rutisson wrote: > >> Hi Tom, >> >> Thanks for the review! >> >>> Overall this looks fine to me. quicksort should really be QuickSort since its a class name and normally we'd make it inherit from AllStatic to indicate that's just a collection of functions. method names should be lower case with underscores though that isn't universally followed. >> Good points. I am still getting used to the coding standard. I made the changes you proposed. Here is a new webrev: >> >> http://cr.openjdk.java.net/~brutisso/7016112/webrev.02/ >> >> FYI: I changed the name of the files quicksort.cpp/hpp to quickSort.cpp/hpp. This is not trivial on Windows, since it does not care about case in file names. I got mercurial, Visual Studio and the file system to accept my change. But for some reason webrev still thinks the files are named with all lower case. I'll make sure they are correctly named when I push. > Ok. > >>> I assume we need to keep the idempotent stuff for the original reasons stated in that comment. We could leave the bubble sort in place or just do some measurement to decide if it's really worth it. It would be nice to get rid of it though. >> Just to be clear; What I proposed was not to remove the idempotent sorting but to remove the parameter for it and always sort in an idempotent way. I have not seen any performance issues with always sorting with idempotent=true. On the other hand I am still a bit uncertain about performance so it might be best to keep it. > Sorry I misunderstood. I would probably keep it as is. You could use an algorithm which is stable in first place instead but that opens a whole other can of worms. > > tom > >> Bengt >> >>> tom >>> >>> On Jun 21, 2011, at 5:50 AM, Bengt Rutisson wrote: >>> >>>> Hi Runtime and GC, >>>> >>>> Sending this review request to both groups. I fixed a GC bug, but the changes are in runtime code. >>>> >>>> The bug that I fixed is this one: >>>> 7016112 CMS: crash during promotion testing >>>> http://monaco.sfbay.sun.com/detail.jsf?cr=7016112 >>>> >>>> And here is the webrev: >>>> http://cr.openjdk.java.net/~brutisso/7016112/webrev.01/ >>>> >>>> The investigation to find the root of the crashes reported in 7016112 was quite lengthy. It is not that easy to read the CR and figure out what is going on, so here is some background: >>>> >>>> When we load classes we store references to the methods of a class in an object array. When we have loaded all methods we sort the object array to allow binary search within the array. To do this sort we use stdlib::qsort(), which is the standard library quicksort implementation. >>>> >>>> If we are using CMS we might be doing concurrent marking while we are sorting the object array. The object array can be found by the concurrent marking code and it may start iterating over the array while we are sorting it. The problem is that on Windows the stdlib::qsort() is not implemented to do atomic updates of elements in the array that it sorts. Instead it does a byte-by-byte swap when it needs to swap two values. That is an easy way to implement different element widths, but it means that at some point in time one element may contain a few bytes from the element above or below. If this happens at the same time as the marking code is trying to read that element, we will be marking some random address and not the method that was supposed to be marked. >>>> >>>> On Solaris and Linux the stdlib::qsort() implementations try to swap as wide data types as possible so this issue should not occur there. On the other hand we have no guarantees that this will always be how stdlib::qsort() is implemented. >>>> >>>> After some discussions about different ways of solving this we came to the conclusion that the simplest way is to implement our own quicksort that operates on the correct pointer width (oop or narrowOop). >>>> >>>> So, this is what I have done to fix this bug. >>>> >>>> Also, it is likely that this problem will go away when the perm gen removal project is finished. Right now it looks like we will not be tracing and marking methods at all after that change. >>>> >>>> * Questions * >>>> >>>> - Should we keep the bubble sort that is done before calling quicksort in methodOopDesc::sort_methods() ? >>>> >>>> - Should we keep the idempotent option or should we try to always use idempotent sorting (see performance test below)? >>>> >>>> - What is the best way to handle unit tests? I added a flag called ExecuteInternalVMTests to run unit tests. This is in line with the existing ErrorHandlerTest flag. My thought is that we can use this same flag for other unit tests than just the quicksort tests. Would be good if we could get these tests executed by JPRT as well. I simply run these with "java -XX:+ExecuteInternalVMTests -version". >>>> >>>> >>>> * Testing * >>>> >>>> Did the obvious testing: Ran JPRT with the changes in the webrev and ran the failing nsk test from the bug (nsk/sysdict/vm/stress/jck12a/sysdictj12a008) repeatedly for sevaral days without failing. >>>> >>>> I created some unit tests for the quicksort implementation and they all pass. >>>> >>>> I also made a build that sorts both with my own quicksort and with stdlib::qsort and then compares that the arrays have the same sort order. Ran this through JPRT and it seems to work on all platforms. That run also included the unit tests. If anybody wants to see how this testing was done, there is a separate webrev for that build. The interesting code is in methodOop.cpp: >>>> >>>> http://cr.openjdk.java.net/~brutisso/7016112/webrev-verify-sorting/ >>>> >>>> * Performance * >>>> >>>> I am a bit unsure how to get any relevant performance data for this change. What I have done so far is to create a class that has 65535 methods (which is the maximum - the length of the method array is u2) and I have measured how long it takes to sort this method array. The methods have random names but every hundredth method has 4 overloaded version. This makes sure that there are some duplicates in the array. >>>> >>>> For now I have run this on my Windows x64 work station with 4 cpus and on a Solaris Sparc machine with 2 cpus (uname says: SunOS sthsparc24 5.10 Generic_139555-08 sun4us sparc FJSV,GPUZC-M Solaris). >>>> >>>> I am attaching graphs for the results. The Y-axis has time in nano seconds. Judging by this my quicksort is a bit faster on Windows and a bit slower on Solaris. But the interesting thing is that the idempotent version is faster than the default behavior on Windows and on par with the default on Solaris. I assume that this is due to the fact that some stores can be avoided if we don't do swap of duplicates. This means that (at least on Windows) swap is more expensive than compare. If this is true we should probably remove the special treatment of idempotent. >>>> >>>> I could run this on more machines, but I am not sure how relevant this type of data is. Most method arrays will be much shorter and compared to reading the class from a file the sort will be in the noise. >>>> >>>> Long email...I hope I covered most of the issues here. Let me know if you have any questions. >>>> >>>> Thanks, >>>> Bengt >>>> -------------- next part -------------- A non-text attachment was scrubbed... Name: solarisx64.png Type: image/png Size: 16422 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110628/14479ce3/attachment-0003.png -------------- next part -------------- A non-text attachment was scrubbed... Name: winx64.png Type: image/png Size: 25386 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110628/14479ce3/attachment-0004.png -------------- next part -------------- A non-text attachment was scrubbed... Name: linuxx64.png Type: image/png Size: 16601 bytes Desc: not available Url : http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20110628/14479ce3/attachment-0005.png From mark.reinhold at oracle.com Mon Jun 27 22:05:05 2011 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Mon, 27 Jun 2011 22:05:05 -0700 Subject: Nominating Mikael Gerdin to be an OpenJDK committer and a reviewer In-Reply-To: bengt.rutisson@oracle.com; Mon, 27 Jun 2011 09:42:18 +0200; <4E08345A.6000901@oracle.com> Message-ID: <20110628050505.52E761106@eggemoggin.niobe.net> 2011/6/27 0:42 -0700, bengt.rutisson at oracle.com: > With the new OpenJDK bylaws almost in place we need to follow the process that > they outline to get OpenJDK user names even for the people working at > Oracle. Mikael Gerdin does not have an OpenJDK user name yet. He helped me a > lot with the CMS bug that I have been working on during the spring and he also > reviewed and had good comments on my quicksort implementation. I would very > much like to list him as a reviewer when I push this change. > > To grant Mikael reviewer status in OpenJDK we need to first make him a > committer (lazy consensus - no vetoes) and then vote him in as a reviewer (at > least three votes in favor and no vetoes). > > Since this is blocking my push, could I please have some votes for making > Mikael Gerdin a committer and a reviewer? > > Relevant part of the bylaws: > http://openjdk.java.net/groups/gb/bylaws/draft-openjdk-bylaws-10#_7 > > Copying Mark on this thread since this is the first time this is done and I > would like to know that we are on the right track. I appreciate the effort you're taking to try to follow the proposed Bylaws, but I'm afraid it's a bit premature. Even assuming that they're ratified tomorrow -- which looks likely -- it will take a bit of time to execute the transition outlined in Appendix B [1]. Until we've finished that we can't really even identify the proper set of Committers to vote upon any particular nomination of a new Committer. Aside from that, while I'm sure that Mikael is a very capable engineer he has no history of past code contributions in OpenJDK and thus there isn't a basis upon which existing Committers could judge his prior work when evaluating his nomination. Finally, the Reviewer role is meant to align with the definition in the OpenJDK Developers' Guide [2]: Reviewers should be aware that they take full responsibility for the appropriateness and correctness of any changes in their area of expertise. If something goes wrong (e.g., the build breaks) and the change's author is unavailable, they may be asked to deal with the problem. A Reviewer, in other words, needs to be an experienced Committer who is ready, willing, and able to deal with any failure arising from the change under review, including but not limited to dealing with a broken build, repairing merge errors, producing follow-on fixes, and so forth. I've heard from others that Mikael provided valuable assistance to you in producing this fix, and I have no reason to doubt that. He should by all means be given due credit for his work, but trying to vote him into the Committer and Reviewer roles is not the right approach. Unfortunately we do not, at the moment, have a great alternative in place and ready to go. If you're anxious to push your changeset very soon then I suggest acknowledging Mikael in the "Summary" line of your changeset comment, something like this: Summary: Also reviewed by mikael.gerdin at oracle.com A better solution might be to modify the jcheck extension to accept new lines of the form: Also-reviewed-by: mikael.gerdin at oracle.com This is different from the existing "Reviewed-by" line in that it accepts an e-mail address rather than an OpenJDK username, and it doesn't carry the implication that the named individual has the formal Reviewer role. If you can wait a few days we can probably extend jcheck to do this, but we'd have to discuss such a proposal more widely a bit first. Thanks again for trying to follow the proposed Bylaws here, and sorry for all the confusion. I hope this guidance helps. We're already planning to write up a "how-to" document to help people understand and operate the processes defined in the Bylaws; hopefully that will make things more efficient for everyone once it's available. - Mark [1] http://openjdk.java.net/groups/gb/bylaws/draft-openjdk-bylaws-10#_B [2] http://openjdk.java.net/guide/changePlanning.html From bengt.rutisson at oracle.com Tue Jun 28 02:57:18 2011 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 28 Jun 2011 11:57:18 +0200 Subject: Nominating Mikael Gerdin to be an OpenJDK committer and a reviewer In-Reply-To: <20110628050505.52E761106@eggemoggin.niobe.net> References: <20110628050505.52E761106@eggemoggin.niobe.net> Message-ID: <4E09A57E.4000903@oracle.com> Mark, Thanks for taking the time to explain the details. I realize that we will not be able to create an OpenJDK user name for Mikael Gerdin any time soon, so I'll stop pushing for it. Some comments inline. [...] > Aside from that, while I'm sure that Mikael is a very capable engineer he > has no history of past code contributions in OpenJDK and thus there isn't > a basis upon which existing Committers could judge his prior work when > evaluating his nomination. As I said, I will not push more for making an exception in this particular case. But in general I think that code contributions to the OpenJDK code base is just one way of judging someones capabilities. There are many other factors that can build trust among developers. For example, contributions and reviews to other projects such as JRockit, building tools or test frameworks around the OpenJDK base and just communicating around development issues - helping colleagues out. So, I definitely think that there will be cases where someone can be trusted to get reviewer status without having made very many contributions. It is important that there is a fast path to become a reviewer if we should be able to hire (and keep) experienced developers. They will not settle for being baby-sitted for too long. [...] > Unfortunately we do not, at the moment, have a great alternative in place > and ready to go. If you're anxious to push your changeset very soon then > I suggest acknowledging Mikael in the "Summary" line of your changeset > comment, something like this: > > Summary: Also reviewed by mikael.gerdin at oracle.com > > A better solution might be to modify the jcheck extension to accept new > lines of the form: > > Also-reviewed-by: mikael.gerdin at oracle.com > > This is different from the existing "Reviewed-by" line in that it accepts > an e-mail address rather than an OpenJDK username, and it doesn't carry > the implication that the named individual has the formal Reviewer role. Thanks for these suggestions. I like the "Also reviewd-by:" field, but it is not urgent for me to get that fixed in jcheck. I will use the "Summary: Also reviewed by" version when I push. Just one comment on the "Also reviewd-by:" field. It would be good if it would accept both email addresses and OpenJDK user names. Since there will be OpenJDK users that are committers but not reviewers. They will have their name and that feels like a better identifier. > If you can wait a few days we can probably extend jcheck to do this, but > we'd have to discuss such a proposal more widely a bit first. > > Thanks again for trying to follow the proposed Bylaws here, and sorry for > all the confusion. I hope this guidance helps. We're already planning > to write up a "how-to" document to help people understand and operate the > processes defined in the Bylaws; hopefully that will make things more > efficient for everyone once it's available. Looking forward to the "how-to" document. That will be great to have. Thanks, Bengt > - Mark > > > [1] http://openjdk.java.net/groups/gb/bylaws/draft-openjdk-bylaws-10#_B > [2] http://openjdk.java.net/guide/changePlanning.html From bengt.rutisson at oracle.com Thu Jun 30 04:25:41 2011 From: bengt.rutisson at oracle.com (bengt.rutisson at oracle.com) Date: Thu, 30 Jun 2011 11:25:41 +0000 Subject: hg: hsx/hotspot-gc/hotspot: 7016112: CMS: crash during promotion testing Message-ID: <20110630112543.4E2DF47098@hg.openjdk.java.net> Changeset: 04760e41b01e Author: brutisso Date: 2011-06-28 14:23 +0200 URL: http://hg.openjdk.java.net/hsx/hotspot-gc/hotspot/rev/04760e41b01e 7016112: CMS: crash during promotion testing Summary: Also reviewed by mikael.gerdin at oracle.com; stdlib:qsort() does byte-by-byte swapping on Windows. This leads to pointer shearing. Fix is to implement a quicksort that does full pointer updates. Reviewed-by: never, coleenp, ysr ! src/share/vm/oops/methodOop.cpp ! src/share/vm/prims/jni.cpp ! src/share/vm/runtime/globals.hpp + src/share/vm/utilities/quickSort.cpp + src/share/vm/utilities/quickSort.hpp