From denka.b at gmail.com Tue May 8 21:33:34 2007 From: denka.b at gmail.com (Denis Baranov) Date: Tue, 8 May 2007 21:33:34 -0700 Subject: Advance notice of an imminent Full GC Message-ID: Hi! I was told at the JavaOne booth that it may be possible to expose HotSpot's inner workings to the degree that when HotSpot makes the decision to collect or not, those values it bases its decision upon can be exposed via JMX and some application may be notified when certain threshold is crossed. I think this would be helpful (in absence of RealTime JVM) to be able to notify load balancer in load-balanced configuration to stop sending requests to this instance, force GC or wait for it to occur naturally, and then notify load balancer again that the instance is ready to process requests. Otherwise we observe situation when requests pile up at the "gate" and cause an abrupt spike in load after Full GC. Thanks! From Jon.Masamitsu at Sun.COM Wed May 9 07:39:01 2007 From: Jon.Masamitsu at Sun.COM (Jon Masamitsu) Date: Wed, 09 May 2007 07:39:01 -0700 Subject: Advance notice of an imminent Full GC In-Reply-To: References: Message-ID: <4641DD05.3010103@sun.com> Denis Baranov wrote: >Hi! > >I was told at the JavaOne booth that it may be possible to expose >HotSpot's inner workings to the degree that when HotSpot makes the >decision to collect or not, those values it bases its decision upon >can be exposed via JMX and some application may be notified when >certain threshold is crossed. I think this would be helpful (in >absence of RealTime JVM) to be able to notify load balancer in >load-balanced configuration to stop sending requests to this instance, >force GC or wait for it to occur naturally, and then notify load >balancer again that the instance is ready to process requests. >Otherwise we observe situation when requests pile up at the "gate" and >cause an abrupt spike in load after Full GC. > >Thanks! > > Take a look at http://java.sun.com/j2se/1.5.0/docs/guide/management/mxbeans.html#low_memory There is a section that talks about detecting low memory which might be of interest. That page has links that might also be useful. This is the jdk5 documentation. The jdk6 monitoring guide will be more current but I don't think it is linked to other interesting stuff. http://java.sun.com/javase/6/docs/technotes/guides/management/MonitoringGuide.pdf From Mandy.Chung at Sun.COM Wed May 9 11:42:58 2007 From: Mandy.Chung at Sun.COM (Mandy Chung) Date: Wed, 09 May 2007 11:42:58 -0700 Subject: Advance notice of an imminent Full GC In-Reply-To: <4641DD05.3010103@sun.com> References: <4641DD05.3010103@sun.com> Message-ID: <46421632.80209@sun.com> This article also talks about the low memory detection: http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html#LowMemoryDetection Mandy Jon Masamitsu wrote: > Denis Baranov wrote: > > >> Hi! >> >> I was told at the JavaOne booth that it may be possible to expose >> HotSpot's inner workings to the degree that when HotSpot makes the >> decision to collect or not, those values it bases its decision upon >> can be exposed via JMX and some application may be notified when >> certain threshold is crossed. I think this would be helpful (in >> absence of RealTime JVM) to be able to notify load balancer in >> load-balanced configuration to stop sending requests to this instance, >> force GC or wait for it to occur naturally, and then notify load >> balancer again that the instance is ready to process requests. >> Otherwise we observe situation when requests pile up at the "gate" and >> cause an abrupt spike in load after Full GC. >> >> Thanks! >> >> >> > Take a look at > > http://java.sun.com/j2se/1.5.0/docs/guide/management/mxbeans.html#low_memory > > There is a section that talks about detecting low memory which > might be of interest. That page has links that might also be > useful. This is the jdk5 documentation. > > The jdk6 monitoring guide will be more current but I don't think it > is linked to other interesting stuff. > > http://java.sun.com/javase/6/docs/technotes/guides/management/MonitoringGuide.pdf > > > From Tim.Bell at Sun.COM Thu May 10 18:01:50 2007 From: Tim.Bell at Sun.COM (Tim Bell) Date: Thu, 10 May 2007 18:01:50 -0700 Subject: [PATCH] Adding limitation while checking for cardTable Message-ID: <4643C07E.8060302@sun.com> Forwarded message Please update your address books. Stop using hotspot-gc-dev at openjdk.dev.java.net ^^^^ and use hotspot-gc-dev at openjdk.java.net instead (note the missing .dev). -------- Original Message -------- From: Neo Jia Subject: Re: [PATCH] Adding limitation while checking for cardTable On 5/10/07, Y Srinivas Ramakrishna wrote: > Hi Neo -- > > Thanks for the heads-up. This seems worth fixing, at least > for hygiene, How about the following instead, where we avoid the > addr_for() translation until we are sure it will be safe, > provided that the mr argument to the method is backed by > card table entries. This avoids the need for the > extra compare-and-branch inside the loop in your patch. > Ramki, Thanks. Your fix makes more sense than mine. Neo > void do_MemRegion(MemRegion mr) { > HeapWord* end_of_non_clean = mr.end(); > HeapWord* start_of_non_clean = end_of_non_clean; > jbyte* entry = _ct->byte_for(mr.last()); > jbyte* first_entry = _ct->byte_for(mr.start()); > while (entry >= first_entry) { > jbyte entry_val = *entry; > HeapWord* cur = _ct->addr_for(entry); > if (!clear_card(entry)) { > if (start_of_non_clean < end_of_non_clean) { > MemRegion mr2(start_of_non_clean, end_of_non_clean); > _dirty_card_closure->do_MemRegion(mr2); > } > end_of_non_clean = cur; > start_of_non_clean = end_of_non_clean; > } > start_of_non_clean = cur; > entry--; > } > if (start_of_non_clean < end_of_non_clean) { > MemRegion mr2(start_of_non_clean, end_of_non_clean); > _dirty_card_closure->do_MemRegion(mr2); > } > } > > -- Ramki Ramakrishna > > Neo Jia wrote On 05/09/07 03:31 PM,: > > > hi, > > > > Acturally, the existing code works well although it misses the > > limitation checking, because the current heap layout will make sure > > the memory region checking failed before it goes beyond the limitation > > of byte map. > > > > The reason is that the nursery space is on the lower address, which > > will make its committed card table also on the lower address of inside > > the byte maps. While walking through the card table of the mature > > space and keeping reduce the entry, we will finally hit the boundary > > of its byte map before doing the check of memory region, which will > > fire the assert of the addr_for, if it is on the lower address. > > > > So, just adding more safety for this function. > > > > Thanks, > > Neo > > Index: memory/cardTableRS.cpp > > =================================================================== > > --- memory/cardTableRS.cpp (revision 86) > > +++ memory/cardTableRS.cpp (working copy) > > @@ -163,6 +163,9 @@ > > start_of_non_clean = end_of_non_clean; > > } > > entry--; > > + // Adding a limit checking for safety. > > + if (entry < _ct->base_map()) > > + break; > > start_of_non_clean = cur; > > cur = _ct->addr_for(entry); > > } > > Index: memory/cardTableRS.hpp > > =================================================================== > > --- memory/cardTableRS.hpp (revision 86) > > +++ memory/cardTableRS.hpp (working copy) > > @@ -101,6 +101,8 @@ > > > > CardTableModRefBS* ct_bs() { return &_ct_bs; } > > > > + jbyte * base_map() { return &(_ct_bs._byte_map[0]); } > > + > > // Override. > > void prepare_for_younger_refs_iterate(bool parallel); > > > > -- > > ---------------------------------------------------------------------------- > Y. Srinivas Ramakrishna HotSpot JVM > Sun Microsystems, Inc., USCA 14-102 JWS / Software Group > 4140 Network Circle 408 276 7250 (x17250) > Santa Clara, CA 95054, U.S.A. Y dot S dot Ramakrishna at Sun dot COM > ---------------------------------------------------------------------------- > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > NOTICE: This email message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any unauthorized > review, use, disclosure or distribution is prohibited. If you are not the > intended recipient, please contact the sender by reply email and destroy > all copies of the original message. > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > -- I would remember that if researchers were not ambitious probably today we haven't the technology we are using! -------------- next part -------------- An embedded message was scrubbed... From: Neo Jia Subject: Re: [PATCH] Adding limitation while checking for cardTable Date: Thu, 10 May 2007 15:39:18 -0500 Size: 6662 Url: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20070510/96bc6234/attachment-0001.mht From Tim.Bell at Sun.COM Thu May 10 18:03:10 2007 From: Tim.Bell at Sun.COM (Tim Bell) Date: Thu, 10 May 2007 18:03:10 -0700 Subject: [PATCH] Adding limitation while checking for cardTable Message-ID: <4643C0CE.5010100@sun.com> Forwarded message Please update your address books. Stop using hotspot-gc-dev at openjdk.dev.java.net ^^^^ and use hotspot-gc-dev at openjdk.java.net instead (note the missing .dev). -------- Original Message -------- Hi Neo -- Thanks for the heads-up. This seems worth fixing, at least for hygiene, How about the following instead, where we avoid the addr_for() translation until we are sure it will be safe, provided that the mr argument to the method is backed by card table entries. This avoids the need for the extra compare-and-branch inside the loop in your patch. void do_MemRegion(MemRegion mr) { HeapWord* end_of_non_clean = mr.end(); HeapWord* start_of_non_clean = end_of_non_clean; jbyte* entry = _ct->byte_for(mr.last()); jbyte* first_entry = _ct->byte_for(mr.start()); while (entry >= first_entry) { jbyte entry_val = *entry; HeapWord* cur = _ct->addr_for(entry); if (!clear_card(entry)) { if (start_of_non_clean < end_of_non_clean) { MemRegion mr2(start_of_non_clean, end_of_non_clean); _dirty_card_closure->do_MemRegion(mr2); } end_of_non_clean = cur; start_of_non_clean = end_of_non_clean; } start_of_non_clean = cur; entry--; } if (start_of_non_clean < end_of_non_clean) { MemRegion mr2(start_of_non_clean, end_of_non_clean); _dirty_card_closure->do_MemRegion(mr2); } } -- Ramki Ramakrishna Neo Jia wrote On 05/09/07 03:31 PM,: > hi, > > Acturally, the existing code works well although it misses the > limitation checking, because the current heap layout will make sure > the memory region checking failed before it goes beyond the limitation > of byte map. > > The reason is that the nursery space is on the lower address, which > will make its committed card table also on the lower address of inside > the byte maps. While walking through the card table of the mature > space and keeping reduce the entry, we will finally hit the boundary > of its byte map before doing the check of memory region, which will > fire the assert of the addr_for, if it is on the lower address. > > So, just adding more safety for this function. > > Thanks, > Neo > Index: memory/cardTableRS.cpp > =================================================================== > --- memory/cardTableRS.cpp (revision 86) > +++ memory/cardTableRS.cpp (working copy) > @@ -163,6 +163,9 @@ > start_of_non_clean = end_of_non_clean; > } > entry--; > + // Adding a limit checking for safety. > + if (entry < _ct->base_map()) > + break; > start_of_non_clean = cur; > cur = _ct->addr_for(entry); > } > Index: memory/cardTableRS.hpp > =================================================================== > --- memory/cardTableRS.hpp (revision 86) > +++ memory/cardTableRS.hpp (working copy) > @@ -101,6 +101,8 @@ > > CardTableModRefBS* ct_bs() { return &_ct_bs; } > > + jbyte * base_map() { return &(_ct_bs._byte_map[0]); } > + > // Override. > void prepare_for_younger_refs_iterate(bool parallel); > -- ---------------------------------------------------------------------------- Y. Srinivas Ramakrishna HotSpot JVM Sun Microsystems, Inc., USCA 14-102 JWS / Software Group 4140 Network Circle 408 276 7250 (x17250) Santa Clara, CA 95054, U.S.A. Y dot S dot Ramakrishna at Sun dot COM ---------------------------------------------------------------------------- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -------------- next part -------------- An embedded message was scrubbed... From: Y Srinivas Ramakrishna Subject: Re: [PATCH] Adding limitation while checking for cardTable Date: Thu, 10 May 2007 13:21:02 -0700 Size: 5964 Url: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20070510/eb785d37/attachment-0001.mht From neojia at gmail.com Thu May 10 21:27:05 2007 From: neojia at gmail.com (Neo Jia) Date: Thu, 10 May 2007 23:27:05 -0500 Subject: Questions about different default value of NewRatio Message-ID: <5d649bdb0705102127j82691e2v78071f7ae04ef770@mail.gmail.com> hi, I just found that different compiler is using different ratio on JDK7, which make sense. But for the same compiler, the default value will be changed across different platforms. Is there any implementation concern for this? For example, why using two different value on 64 bit and 32 bit for C2? I assume that the NewRatio will only impact the efficiency of the minor GC. 1 31 hotspot/src/cpu/amd64/vm/c2_globals_amd64.hpp <> define_pd_global(intx, NewRatio, 2); 2 28 hotspot/src/cpu/i486/vm/c1_globals_i486.hpp <> define_pd_global(intx, NewRatio, 12 ); 3 43 hotspot/src/cpu/i486/vm/c2_globals_i486.hpp <> define_pd_global(intx, NewRatio, 8); 4 29 hotspot/src/cpu/ia64/vm/c2_globals_ia64.hpp <> define_pd_global(intx, NewRatio, 2); Thanks, Neo -- I would remember that if researchers were not ambitious probably today we haven't the technology we are using! From Jon.Masamitsu at Sun.COM Fri May 11 08:26:41 2007 From: Jon.Masamitsu at Sun.COM (Jon Masamitsu) Date: Fri, 11 May 2007 08:26:41 -0700 Subject: Questions about different default value of NewRatio In-Reply-To: <5d649bdb0705102127j82691e2v78071f7ae04ef770@mail.gmail.com> References: <5d649bdb0705102127j82691e2v78071f7ae04ef770@mail.gmail.com> Message-ID: <46448B31.8040801@sun.com> Neo Jia wrote: >hi, > >I just found that different compiler is using different ratio on JDK7, >which make sense. But for the same compiler, the default value will be >changed across different platforms. Is there any implementation >concern for this? > >For example, why using two different value on 64 bit and 32 bit for >C2? I assume that the NewRatio will only impact the efficiency of the >minor GC. > > 1 31 hotspot/src/cpu/amd64/vm/c2_globals_amd64.hpp <> > define_pd_global(intx, NewRatio, 2); > 2 28 hotspot/src/cpu/i486/vm/c1_globals_i486.hpp <> > define_pd_global(intx, NewRatio, 12 ); > 3 43 hotspot/src/cpu/i486/vm/c2_globals_i486.hpp <> > define_pd_global(intx, NewRatio, 8); > 4 29 hotspot/src/cpu/ia64/vm/c2_globals_ia64.hpp <> > define_pd_global(intx, NewRatio, 2); > >Thanks, >Neo > > These different NewRatio's reflect several beliefs. A) Applications run on a 64 bit machine will use large heaps and are applications where throughput is more important. B) Applications run on IA32 (i486) are smaller client type applications and pause time is more important. C) C1 is used more often for smaller client applications. D) C2 is used more often for larger server applictions. Belief A) leads us to use NewRatio of 2 on amd64 and ia64. Believe B) leads us to use smaller values on i486. Beliefs C) and D) lead us to make the value for C1 on i486 a bit smaller (12) and for C2 on i486 a bit larger (8). The biggest difficulty I've had with this is that on 32bit sparc the NewRatio is 2 and on 32bit i486 the value is 8 for C2. I think this reflects the belief that customers buy sparc platforms to run large applications. But currently when moving between the two 32bit platforms (sparc and i486) there is a significant difference in GC behavior because of the different NewRatio values. From neojia at gmail.com Sat May 12 23:21:53 2007 From: neojia at gmail.com (Neo Jia) Date: Sun, 13 May 2007 01:21:53 -0500 Subject: Current heap layout of generational mark-sweep GC Message-ID: <5d649bdb0705122321w5ef63508me8f04b988dfc71e5@mail.gmail.com> hi, I am wondering that for the generational mark-sweep GC, if the block offset shared array is the last allocated object on the CHeap. The following is my understanding of the current heap layout: 1. The nursery space is on the lowest address 2. The mature space is following the nursery address. 3. The permanent space is following the nursery address at a higher address. 4. The card table is allocated from the end of the permanent space, the regions inside it will be mapped to these previous three spaces sequentially, from lower address to higher address. 5. The block offset shared array used by the mature space will be allocated at the end of the card table area, which is used by mature space. And the virtual space size of this area will be set according to the mature space and word bits. So, is there any space allocated following? Thanks, Neo -- I would remember that if researchers were not ambitious probably today we haven't the technology we are using! From Jon.Masamitsu at Sun.COM Mon May 14 07:19:24 2007 From: Jon.Masamitsu at Sun.COM (Jon Masamitsu) Date: Mon, 14 May 2007 07:19:24 -0700 Subject: Current heap layout of generational mark-sweep GC In-Reply-To: <5d649bdb0705122321w5ef63508me8f04b988dfc71e5@mail.gmail.com> References: <5d649bdb0705122321w5ef63508me8f04b988dfc71e5@mail.gmail.com> Message-ID: <46486FEC.4090202@Sun.COM> Neo, Are you asking specifically about CHeap usage by the garbage collector (i.e., you're not interested in the compiler's use CHeap space)? Jon Neo Jia wrote: > hi, > > I am wondering that for the generational mark-sweep GC, if the block > offset shared array is the last allocated object on the CHeap. > > The following is my understanding of the current heap layout: > > 1. The nursery space is on the lowest address > 2. The mature space is following the nursery address. > 3. The permanent space is following the nursery address at a higher > address. > 4. The card table is allocated from the end of the permanent space, > the regions inside it will be mapped to these previous three spaces > sequentially, from lower address to higher address. > 5. The block offset shared array used by the mature space will be > allocated at the end of the card table area, which is used by mature > space. And the virtual space size of this area will be set according > to the mature space and word bits. > > So, is there any space allocated following? > > Thanks, > Neo From neojia at gmail.com Mon May 14 08:55:22 2007 From: neojia at gmail.com (Neo Jia) Date: Mon, 14 May 2007 10:55:22 -0500 Subject: Current heap layout of generational mark-sweep GC In-Reply-To: <46486FEC.4090202@Sun.COM> References: <5d649bdb0705122321w5ef63508me8f04b988dfc71e5@mail.gmail.com> <46486FEC.4090202@Sun.COM> Message-ID: <5d649bdb0705140855t3774a170oac4b2b48d7835ac3@mail.gmail.com> On 5/14/07, Jon Masamitsu wrote: > Neo, > > Are you asking specifically about CHeap usage by the > garbage collector (i.e., you're not interested in the > compiler's use CHeap space)? Jon, Although my question is about the CHeap usage by GC, I am also interested in knowing the CHeap space usage of compiler. Thanks, Neo > > Jon > > > Neo Jia wrote: > > > hi, > > > > I am wondering that for the generational mark-sweep GC, if the block > > offset shared array is the last allocated object on the CHeap. > > > > The following is my understanding of the current heap layout: > > > > 1. The nursery space is on the lowest address > > 2. The mature space is following the nursery address. > > 3. The permanent space is following the nursery address at a higher > > address. > > 4. The card table is allocated from the end of the permanent space, > > the regions inside it will be mapped to these previous three spaces > > sequentially, from lower address to higher address. > > 5. The block offset shared array used by the mature space will be > > allocated at the end of the card table area, which is used by mature > > space. And the virtual space size of this area will be set according > > to the mature space and word bits. > > > > So, is there any space allocated following? > > > > Thanks, > > Neo > > -- I would remember that if researchers were not ambitious probably today we haven't the technology we are using! From neojia at gmail.com Mon May 14 09:00:38 2007 From: neojia at gmail.com (Neo Jia) Date: Mon, 14 May 2007 11:00:38 -0500 Subject: Eden space is not empty after a Full GC? Message-ID: <5d649bdb0705140900x2540737al34c4c90dba73b48d@mail.gmail.com> hi, I just found that for the generational mark-sweep garbage collector, after a full GC, the eden space may not be empty. Is there any implementation concern for this? Can I force the full GC to clean the eden space? Thanks, Neo -- I would remember that if researchers were not ambitious probably today we haven't the technology we are using! From Y.S.Ramakrishna at Sun.COM Mon May 14 09:23:26 2007 From: Y.S.Ramakrishna at Sun.COM (Y Srinivas Ramakrishna) Date: Mon, 14 May 2007 09:23:26 -0700 Subject: Eden space is not empty after a Full GC? In-Reply-To: <5d649bdb0705140900x2540737al34c4c90dba73b48d@mail.gmail.com> References: <5d649bdb0705140900x2540737al34c4c90dba73b48d@mail.gmail.com> Message-ID: <46488CFE.1090400@Sun.COM> Neo Jia wrote On 05/14/07 09:00 AM,: > hi, > > I just found that for the generational mark-sweep garbage collector, > after a full GC, the eden space may not be empty. That would usually be the case because of bin-packing issues with the sliding compaction that might prevent a large object in Eden from being "slid" into the available space in old gen or in the survivor space. It'll usually be the case then that your are running with memory quite tight (or with a huge Eden and very large objects). Are there other situations where you see a non-empty Eden following a mark-compact full collection? > > Is there any implementation concern for this? Can I force the full GC > to clean the eden space? A full gc would do a mark-compact collection. -- ramki. > > Thanks, > Neo -- ---------------------------------------------------------------------------- Y. Srinivas Ramakrishna HotSpot JVM Sun Microsystems, Inc., USCA 22-123 CSG / Sun Software 4220 Network Circle 408 276 7250 (x17250) Santa Clara, CA 95054, U.S.A. Y dot S dot Ramakrishna at Sun dot COM ---------------------------------------------------------------------------- ----------------- Sun Proprietary / Confidential --------------------------- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From neojia at gmail.com Mon May 14 09:37:55 2007 From: neojia at gmail.com (Neo Jia) Date: Mon, 14 May 2007 11:37:55 -0500 Subject: Eden space is not empty after a Full GC? In-Reply-To: <46488CFE.1090400@Sun.COM> References: <5d649bdb0705140900x2540737al34c4c90dba73b48d@mail.gmail.com> <46488CFE.1090400@Sun.COM> Message-ID: <5d649bdb0705140937k4e152204odb9a84949cda093b@mail.gmail.com> On 5/14/07, Y Srinivas Ramakrishna wrote: > > > Neo Jia wrote On 05/14/07 09:00 AM,: > > > hi, > > > > I just found that for the generational mark-sweep garbage collector, > > after a full GC, the eden space may not be empty. > > That would usually be the case because of bin-packing > issues with the sliding compaction that might prevent a large > object in Eden from being "slid" into the available space > in old gen or in the survivor space. It'll usually be the case > then that your are running with memory quite tight (or > with a huge Eden and very large objects). Could you point me to the code where the full GC found it cannot slide this huge chunk? Thanks, Neo > > Are there other situations where you see a non-empty Eden > following a mark-compact full collection? > > > > > Is there any implementation concern for this? Can I force the full GC > > to clean the eden space? > > > A full gc would do a mark-compact collection. > > -- ramki. > > > > > Thanks, > > Neo > > > -- > > ---------------------------------------------------------------------------- > Y. Srinivas Ramakrishna HotSpot JVM > Sun Microsystems, Inc., USCA 22-123 CSG / Sun Software > 4220 Network Circle 408 276 7250 (x17250) > Santa Clara, CA 95054, U.S.A. Y dot S dot Ramakrishna at Sun dot COM > ---------------------------------------------------------------------------- > ----------------- Sun Proprietary / Confidential --------------------------- > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > NOTICE: This email message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any unauthorized > review, use, disclosure or distribution is prohibited. If you are not the > intended recipient, please contact the sender by reply email and destroy > all copies of the original message. > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > -- I would remember that if researchers were not ambitious probably today we haven't the technology we are using! From Jon.Masamitsu at Sun.COM Mon May 14 09:51:31 2007 From: Jon.Masamitsu at Sun.COM (Jon Masamitsu) Date: Mon, 14 May 2007 09:51:31 -0700 Subject: Eden space is not empty after a Full GC? In-Reply-To: <5d649bdb0705140937k4e152204odb9a84949cda093b@mail.gmail.com> References: <5d649bdb0705140900x2540737al34c4c90dba73b48d@mail.gmail.com> <46488CFE.1090400@Sun.COM> <5d649bdb0705140937k4e152204odb9a84949cda093b@mail.gmail.com> Message-ID: <46489393.4030405@Sun.COM> A perhaps more general case (i.e., no necessarily involving large objects) is one where all the live data does not fit into the old generation. A full collection will attempt to compact the young generation into the old generation. If the old generation becomes full during the compaction, the remaining live data in the young generation will be compacted into the young generation. Neo Jia wrote: > On 5/14/07, Y Srinivas Ramakrishna wrote: > >> >> >> Neo Jia wrote On 05/14/07 09:00 AM,: >> >> > hi, >> > >> > I just found that for the generational mark-sweep garbage collector, >> > after a full GC, the eden space may not be empty. >> >> That would usually be the case because of bin-packing >> issues with the sliding compaction that might prevent a large >> object in Eden from being "slid" into the available space >> in old gen or in the survivor space. It'll usually be the case >> then that your are running with memory quite tight (or >> with a huge Eden and very large objects). > > > Could you point me to the code where the full GC found it cannot slide > this huge chunk? > > Thanks, > Neo > >> >> Are there other situations where you see a non-empty Eden >> following a mark-compact full collection? >> >> > >> > Is there any implementation concern for this? Can I force the full GC >> > to clean the eden space? >> >> >> A full gc would do a mark-compact collection. >> >> -- ramki. >> >> > >> > Thanks, >> > Neo >> >> >> -- >> >> ---------------------------------------------------------------------------- >> >> Y. Srinivas Ramakrishna HotSpot JVM >> Sun Microsystems, Inc., USCA 22-123 CSG / Sun Software >> 4220 Network Circle 408 276 7250 (x17250) >> Santa Clara, CA 95054, U.S.A. Y dot S dot Ramakrishna at Sun >> dot COM >> ---------------------------------------------------------------------------- >> >> ----------------- Sun Proprietary / Confidential >> --------------------------- >> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> >> NOTICE: This email message is for the sole use of the intended >> recipient(s) >> and may contain confidential and privileged information. Any >> unauthorized >> review, use, disclosure or distribution is prohibited. If you are not >> the >> intended recipient, please contact the sender by reply email and destroy >> all copies of the original message. >> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> >> > > From neojia at gmail.com Mon May 14 10:02:43 2007 From: neojia at gmail.com (Neo Jia) Date: Mon, 14 May 2007 12:02:43 -0500 Subject: Eden space is not empty after a Full GC? In-Reply-To: <46489393.4030405@Sun.COM> References: <5d649bdb0705140900x2540737al34c4c90dba73b48d@mail.gmail.com> <46488CFE.1090400@Sun.COM> <5d649bdb0705140937k4e152204odb9a84949cda093b@mail.gmail.com> <46489393.4030405@Sun.COM> Message-ID: <5d649bdb0705141002o6a0f82a3l23c658a7bd38da22@mail.gmail.com> On 5/14/07, Jon Masamitsu wrote: > A perhaps more general case (i.e., no necessarily involving large objects) > is one where all the live data does not fit into the old generation. A full > collection will attempt to compact the young generation into the old > generation. If the old generation becomes full during the compaction, > the remaining live data in the young generation will be compacted > into the young generation. Jon, Thanks. I assume that which objects will be left in the nursery space is pretty random not based on the ages. Will it improve the performance if we intend to leave those younger objects in nursery? Or, just enlarging the mature space to fit in those young objects? But the reserved space is almost fully committed. Thanks, Neo > > Neo Jia wrote: > > > On 5/14/07, Y Srinivas Ramakrishna wrote: > > > >> > >> > >> Neo Jia wrote On 05/14/07 09:00 AM,: > >> > >> > hi, > >> > > >> > I just found that for the generational mark-sweep garbage collector, > >> > after a full GC, the eden space may not be empty. > >> > >> That would usually be the case because of bin-packing > >> issues with the sliding compaction that might prevent a large > >> object in Eden from being "slid" into the available space > >> in old gen or in the survivor space. It'll usually be the case > >> then that your are running with memory quite tight (or > >> with a huge Eden and very large objects). > > > > > > Could you point me to the code where the full GC found it cannot slide > > this huge chunk? > > > > Thanks, > > Neo > > > >> > >> Are there other situations where you see a non-empty Eden > >> following a mark-compact full collection? > >> > >> > > >> > Is there any implementation concern for this? Can I force the full GC > >> > to clean the eden space? > >> > >> > >> A full gc would do a mark-compact collection. > >> > >> -- ramki. > >> > >> > > >> > Thanks, > >> > Neo > >> > >> > >> -- > >> > >> ---------------------------------------------------------------------------- > >> > >> Y. Srinivas Ramakrishna HotSpot JVM > >> Sun Microsystems, Inc., USCA 22-123 CSG / Sun Software > >> 4220 Network Circle 408 276 7250 (x17250) > >> Santa Clara, CA 95054, U.S.A. Y dot S dot Ramakrishna at Sun > >> dot COM > >> ---------------------------------------------------------------------------- > >> > >> ----------------- Sun Proprietary / Confidential > >> --------------------------- > >> > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > >> > >> > >> NOTICE: This email message is for the sole use of the intended > >> recipient(s) > >> and may contain confidential and privileged information. Any > >> unauthorized > >> review, use, disclosure or distribution is prohibited. If you are not > >> the > >> intended recipient, please contact the sender by reply email and destroy > >> all copies of the original message. > >> > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > >> > >> > >> > > > > > -- I would remember that if researchers were not ambitious probably today we haven't the technology we are using! From Jon.Masamitsu at Sun.COM Mon May 14 10:10:18 2007 From: Jon.Masamitsu at Sun.COM (Jon Masamitsu) Date: Mon, 14 May 2007 10:10:18 -0700 Subject: Current heap layout of generational mark-sweep GC In-Reply-To: <5d649bdb0705140855t3774a170oac4b2b48d7835ac3@mail.gmail.com> References: <5d649bdb0705122321w5ef63508me8f04b988dfc71e5@mail.gmail.com> <46486FEC.4090202@Sun.COM> <5d649bdb0705140855t3774a170oac4b2b48d7835ac3@mail.gmail.com> Message-ID: <464897FA.5020401@Sun.COM> Neo, If by "generational mark-sweep GC" you mean the UseSerialGC or UseParNewGC, yes the storage you listed are the major pieces that we allocate in the CHeap. There are many smaller data structures that are allocated out of the CHeap. For example, performance counters are CHeap allocated. Or just the storage for various subclasses of Generation that the GC uses. I believe the largest piece of the CHeap that the compilers use is the CodeCache and it is allocated before the Java heap is allocated. Jon Neo Jia wrote: > On 5/14/07, Jon Masamitsu wrote: > >> Neo, >> >> Are you asking specifically about CHeap usage by the >> garbage collector (i.e., you're not interested in the >> compiler's use CHeap space)? > > > Jon, > > Although my question is about the CHeap usage by GC, I am also interested > in knowing the CHeap space usage of compiler. > > Thanks, > Neo > >> >> Jon >> >> >> Neo Jia wrote: >> >> > hi, >> > >> > I am wondering that for the generational mark-sweep GC, if the block >> > offset shared array is the last allocated object on the CHeap. >> > >> > The following is my understanding of the current heap layout: >> > >> > 1. The nursery space is on the lowest address >> > 2. The mature space is following the nursery address. >> > 3. The permanent space is following the nursery address at a higher >> > address. >> > 4. The card table is allocated from the end of the permanent space, >> > the regions inside it will be mapped to these previous three spaces >> > sequentially, from lower address to higher address. >> > 5. The block offset shared array used by the mature space will be >> > allocated at the end of the card table area, which is used by mature >> > space. And the virtual space size of this area will be set according >> > to the mature space and word bits. >> > >> > So, is there any space allocated following? >> > >> > Thanks, >> > Neo >> >> > > From Y.S.Ramakrishna at Sun.COM Mon May 14 10:20:47 2007 From: Y.S.Ramakrishna at Sun.COM (Y Srinivas Ramakrishna) Date: Mon, 14 May 2007 10:20:47 -0700 Subject: Eden space is not empty after a Full GC? In-Reply-To: <46489393.4030405@Sun.COM> References: <5d649bdb0705140900x2540737al34c4c90dba73b48d@mail.gmail.com> <46488CFE.1090400@Sun.COM> <5d649bdb0705140937k4e152204odb9a84949cda093b@mail.gmail.com> <46489393.4030405@Sun.COM> Message-ID: <46489A6F.6000109@Sun.COM> [What Jon said. Of course I was ignoring (but should not have) the case where the live data size exceeds the size of old gen. As Jon indicated that's usually the most common case. (I was assuming, but should not have, that you were concerned about situations where all the free space in old gen was not used, yet there were objects left behind in Eden following compaction.)] >> >> Could you point me to the code where the full GC found it cannot slide >> this huge chunk? > Look at the macro SCAN_AND_FORWARD() in space.hpp for the serial mark-compact code where the forwarding addresses are computed in the framework collectors. Or for similar code in the PS collector, see PSMarkSweepDecorator::precompact(). The computation of the forwarding addresses in the case of parallel compact is sufficiently different that I'll let Jon or John point you to the relevant code. -- ramki. >> >> Thanks, >> Neo >> >>> >>> Are there other situations where you see a non-empty Eden >>> following a mark-compact full collection? >>> >>> > >>> > Is there any implementation concern for this? Can I force the full GC >>> > to clean the eden space? >>> >>> >>> A full gc would do a mark-compact collection. >>> >>> -- ramki. >>> >>> > >>> > Thanks, >>> > Neo >>> >>> >>> -- >>> >>> ---------------------------------------------------------------------------- >>> >>> Y. Srinivas Ramakrishna HotSpot JVM >>> Sun Microsystems, Inc., USCA 22-123 CSG / Sun Software >>> 4220 Network Circle 408 276 7250 (x17250) >>> Santa Clara, CA 95054, U.S.A. Y dot S dot Ramakrishna at Sun >>> dot COM >>> ---------------------------------------------------------------------------- >>> >>> ----------------- Sun Proprietary / Confidential >>> --------------------------- >>> >>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >>> >>> >>> NOTICE: This email message is for the sole use of the intended >>> recipient(s) >>> and may contain confidential and privileged information. Any >>> unauthorized >>> review, use, disclosure or distribution is prohibited. If you are >>> not the >>> intended recipient, please contact the sender by reply email and >>> destroy >>> all copies of the original message. >>> >>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >>> >>> >>> >> >> -- ---------------------------------------------------------------------------- Y. Srinivas Ramakrishna HotSpot JVM Sun Microsystems, Inc., USCA 22-123 CSG / Sun Software 4220 Network Circle 408 276 7250 (x17250) Santa Clara, CA 95054, U.S.A. Y dot S dot Ramakrishna at Sun dot COM ---------------------------------------------------------------------------- ----------------- Sun Proprietary / Confidential --------------------------- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From Jon.Masamitsu at Sun.COM Mon May 14 10:25:53 2007 From: Jon.Masamitsu at Sun.COM (Jon Masamitsu) Date: Mon, 14 May 2007 10:25:53 -0700 Subject: Eden space is not empty after a Full GC? In-Reply-To: <5d649bdb0705141002o6a0f82a3l23c658a7bd38da22@mail.gmail.com> References: <5d649bdb0705140900x2540737al34c4c90dba73b48d@mail.gmail.com> <46488CFE.1090400@Sun.COM> <5d649bdb0705140937k4e152204odb9a84949cda093b@mail.gmail.com> <46489393.4030405@Sun.COM> <5d649bdb0705141002o6a0f82a3l23c658a7bd38da22@mail.gmail.com> Message-ID: <46489BA1.8050005@Sun.COM> Yes, it's almost random in terms of which objects get left in the young generation. It is a sliding compaction so objects at the higher end of the generation are more likely to be left behind. Leaving objects in the young generation probably means that the following collection will also have to be a full collection. It's generally better to have some room in the old (mature) generation so that minor collections can be done so I would say it would be better to have additional space in the old generations. Neo Jia wrote: > On 5/14/07, Jon Masamitsu wrote: > >> A perhaps more general case (i.e., no necessarily involving large >> objects) >> is one where all the live data does not fit into the old generation. >> A full >> collection will attempt to compact the young generation into the old >> generation. If the old generation becomes full during the compaction, >> the remaining live data in the young generation will be compacted >> into the young generation. > > > Jon, > > Thanks. I assume that which objects will be left in the nursery space > is pretty random not based on the ages. Will it improve the > performance if we intend to leave those younger objects in nursery? > Or, just enlarging the mature space to fit in those young objects? But > the > reserved space is almost fully committed. > > Thanks, > Neo > >> >> Neo Jia wrote: >> >> > On 5/14/07, Y Srinivas Ramakrishna wrote: >> > >> >> >> >> >> >> Neo Jia wrote On 05/14/07 09:00 AM,: >> >> >> >> > hi, >> >> > >> >> > I just found that for the generational mark-sweep garbage >> collector, >> >> > after a full GC, the eden space may not be empty. >> >> >> >> That would usually be the case because of bin-packing >> >> issues with the sliding compaction that might prevent a large >> >> object in Eden from being "slid" into the available space >> >> in old gen or in the survivor space. It'll usually be the case >> >> then that your are running with memory quite tight (or >> >> with a huge Eden and very large objects). >> > >> > >> > Could you point me to the code where the full GC found it cannot slide >> > this huge chunk? >> > >> > Thanks, >> > Neo >> > >> >> >> >> Are there other situations where you see a non-empty Eden >> >> following a mark-compact full collection? >> >> >> >> > >> >> > Is there any implementation concern for this? Can I force the >> full GC >> >> > to clean the eden space? >> >> >> >> >> >> A full gc would do a mark-compact collection. >> >> >> >> -- ramki. >> >> >> >> > >> >> > Thanks, >> >> > Neo >> >> >> >> >> >> -- >> >> >> >> >> ---------------------------------------------------------------------------- >> >> >> >> >> Y. Srinivas Ramakrishna HotSpot JVM >> >> Sun Microsystems, Inc., USCA 22-123 CSG / Sun Software >> >> 4220 Network Circle 408 276 7250 (x17250) >> >> Santa Clara, CA 95054, U.S.A. Y dot S dot Ramakrishna at Sun >> >> dot COM >> >> >> ---------------------------------------------------------------------------- >> >> >> >> >> ----------------- Sun Proprietary / Confidential >> >> --------------------------- >> >> >> >> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> >> >> >> >> >> NOTICE: This email message is for the sole use of the intended >> >> recipient(s) >> >> and may contain confidential and privileged information. Any >> >> unauthorized >> >> review, use, disclosure or distribution is prohibited. If you are not >> >> the >> >> intended recipient, please contact the sender by reply email and >> destroy >> >> all copies of the original message. >> >> >> >> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> >> >> >> >> >> >> > >> > >> > > From tony.printezis at sun.com Mon May 14 12:27:43 2007 From: tony.printezis at sun.com (Tony Printezis) Date: Mon, 14 May 2007 15:27:43 -0400 (EDT) Subject: Advance notice of an imminent Full GC In-Reply-To: References: Message-ID: On Tue, 8 May 2007, Denis Baranov wrote: Denis, (apologies for the late e-mail) So, it was me who suggested this. To give a bit of background: it's probably not possible to notify a user through JMX that "be careful, we are about to do a full GC". When we decide to do a full GC, on many occasions, we are already in a stop-the-world pause and we typically carry on and do a full GC. So, any Java-level code would have no chance in actually getting the notification before the full GC completes. So, I think the best alternative is to just publish a probability of how likely a full GC is. Then it's up to the user to decide how conservative they want to be when handling this probability. I _think_ such a probability will be quite easy to calculate with the statistics we maintain in our GCs (but, maybe, we do not make enough of them public in order for the user to be able to calculate it?). Thoughts / suggestions would be appreciated. Tony > Hi! > > I was told at the JavaOne booth that it may be possible to expose > HotSpot's inner workings to the degree that when HotSpot makes the > decision to collect or not, those values it bases its decision upon > can be exposed via JMX and some application may be notified when > certain threshold is crossed. I think this would be helpful (in > absence of RealTime JVM) to be able to notify load balancer in > load-balanced configuration to stop sending requests to this instance, > force GC or wait for it to occur naturally, and then notify load > balancer again that the instance is ready to process requests. > Otherwise we observe situation when requests pile up at the "gate" and > cause an abrupt spike in load after Full GC. > > Thanks! > --------------------------------------------------------------------------- | Tony Printezis, Staff Engineer, Java SE | Sun Microsystems Inc. | | | MS BUR02-311 | | e-mail: tony.printezis at sun.com | 1 Network Drive | | office: +1 781 442 0998 (x20998) | Burlington, MA01803-0902, USA | --------------------------------------------------------------------------- From denka.b at gmail.com Mon May 14 13:41:22 2007 From: denka.b at gmail.com (Denis Baranov) Date: Mon, 14 May 2007 13:41:22 -0700 Subject: Advance notice of an imminent Full GC In-Reply-To: References: Message-ID: That sort of probability would be fine. Much easier than monitoring individual memory pools for variety of collectors, for sure. Thanks, Denis. On 5/14/07, Tony Printezis wrote: > On Tue, 8 May 2007, Denis Baranov wrote: > > Denis, > > (apologies for the late e-mail) > > So, it was me who suggested this. To give a bit of background: it's > probably not possible to notify a user through JMX that "be careful, we > are about to do a full GC". When we decide to do a full GC, on many > occasions, we are already in a stop-the-world pause and we typically carry > on and do a full GC. So, any Java-level code would have no chance in > actually getting the notification before the full GC completes. > > So, I think the best alternative is to just publish a probability of how > likely a full GC is. Then it's up to the user to decide how conservative > they want to be when handling this probability. I _think_ such a > probability will be quite easy to calculate with the statistics we > maintain in our GCs (but, maybe, we do not make enough of them public in > order for the user to be able to calculate it?). > > Thoughts / suggestions would be appreciated. > > Tony > > > Hi! > > > > I was told at the JavaOne booth that it may be possible to expose > > HotSpot's inner workings to the degree that when HotSpot makes the > > decision to collect or not, those values it bases its decision upon > > can be exposed via JMX and some application may be notified when > > certain threshold is crossed. I think this would be helpful (in > > absence of RealTime JVM) to be able to notify load balancer in > > load-balanced configuration to stop sending requests to this instance, > > force GC or wait for it to occur naturally, and then notify load > > balancer again that the instance is ready to process requests. > > Otherwise we observe situation when requests pile up at the "gate" and > > cause an abrupt spike in load after Full GC. > > > > Thanks! > > > > --------------------------------------------------------------------------- > | Tony Printezis, Staff Engineer, Java SE | Sun Microsystems Inc. | > | | MS BUR02-311 | > | e-mail: tony.printezis at sun.com | 1 Network Drive | > | office: +1 781 442 0998 (x20998) | Burlington, MA01803-0902, USA | > --------------------------------------------------------------------------- > > From neojia at gmail.com Mon May 14 15:40:32 2007 From: neojia at gmail.com (Neo Jia) Date: Mon, 14 May 2007 17:40:32 -0500 Subject: Current heap layout of generational mark-sweep GC In-Reply-To: <464897FA.5020401@Sun.COM> References: <5d649bdb0705122321w5ef63508me8f04b988dfc71e5@mail.gmail.com> <46486FEC.4090202@Sun.COM> <5d649bdb0705140855t3774a170oac4b2b48d7835ac3@mail.gmail.com> <464897FA.5020401@Sun.COM> Message-ID: <5d649bdb0705141540te9717d4t38e5c2c082f8ae2c@mail.gmail.com> On 5/14/07, Jon Masamitsu wrote: > Neo, > > If by "generational mark-sweep GC" you mean the UseSerialGC > or UseParNewGC, I mean the UseSerialGC. yes the storage you listed are the major > pieces that we allocate in the CHeap. There are many smaller > data structures that are allocated out of the CHeap. For example, > performance counters are CHeap allocated. Or just the storage > for various subclasses of Generation that the GC uses. > > I believe the largest piece of the CHeap that the compilers > use is the CodeCache and it is allocated before the Java > heap is allocated. The current "PrintMallocFree" seems broken already. Do you still use this to check the CHeap usage? Thanks, Neo > > Jon > > Neo Jia wrote: > > > On 5/14/07, Jon Masamitsu wrote: > > > >> Neo, > >> > >> Are you asking specifically about CHeap usage by the > >> garbage collector (i.e., you're not interested in the > >> compiler's use CHeap space)? > > > > > > Jon, > > > > Although my question is about the CHeap usage by GC, I am also interested > > in knowing the CHeap space usage of compiler. > > > > Thanks, > > Neo > > > >> > >> Jon > >> > >> > >> Neo Jia wrote: > >> > >> > hi, > >> > > >> > I am wondering that for the generational mark-sweep GC, if the block > >> > offset shared array is the last allocated object on the CHeap. > >> > > >> > The following is my understanding of the current heap layout: > >> > > >> > 1. The nursery space is on the lowest address > >> > 2. The mature space is following the nursery address. > >> > 3. The permanent space is following the nursery address at a higher > >> > address. > >> > 4. The card table is allocated from the end of the permanent space, > >> > the regions inside it will be mapped to these previous three spaces > >> > sequentially, from lower address to higher address. > >> > 5. The block offset shared array used by the mature space will be > >> > allocated at the end of the card table area, which is used by mature > >> > space. And the virtual space size of this area will be set according > >> > to the mature space and word bits. > >> > > >> > So, is there any space allocated following? > >> > > >> > Thanks, > >> > Neo > >> > >> > > > > > -- I would remember that if researchers were not ambitious probably today we haven't the technology we are using! From tony.printezis at sun.com Mon May 14 11:01:47 2007 From: tony.printezis at sun.com (Tony Printezis) Date: Mon, 14 May 2007 14:01:47 -0400 Subject: Is there any GC log reader? In-Reply-To: <065b01c7911b$af7b8200$8014a8c0@XPWork> References: <065b01c7911b$af7b8200$8014a8c0@XPWork> Message-ID: <4648A40B.4080101@sun.com> An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20070514/5371dbfa/attachment.html From ted at tedneward.com Mon May 14 20:41:16 2007 From: ted at tedneward.com (Ted Neward) Date: Mon, 14 May 2007 20:41:16 -0700 Subject: Current heap layout of generational mark-sweep GC In-Reply-To: <5d649bdb0705122321w5ef63508me8f04b988dfc71e5@mail.gmail.com> Message-ID: <000701c796a3$064a8090$8014a8c0@XPWork> I don't have an answer for you, Neo, but as someone outside of Sun, I wanted to express my thanks for all this "digging in" you're doing to the code--you're very quickly becoming someone whose emails I pay attention to when they come through the list, because your questions are always non-trivial and interesting (such the NewRatio email from last week). Your insights are making my job easier getting up to speed on the code. :-) Thanks! Ted Neward Java, .NET, XML Services Consulting, Teaching, Speaking, Writing http://www.tedneward.com > -----Original Message----- > From: hotspot-gc-dev-bounces at openjdk.java.net [mailto:hotspot-gc-dev- > bounces at openjdk.java.net] On Behalf Of Neo Jia > Sent: Saturday, May 12, 2007 11:22 PM > To: hotspot-gc-dev at openjdk.java.net > Subject: Current heap layout of generational mark-sweep GC > > hi, > > I am wondering that for the generational mark-sweep GC, if the block > offset shared array is the last allocated object on the CHeap. > > The following is my understanding of the current heap layout: > > 1. The nursery space is on the lowest address > 2. The mature space is following the nursery address. > 3. The permanent space is following the nursery address at a higher > address. > 4. The card table is allocated from the end of the permanent space, > the regions inside it will be mapped to these previous three spaces > sequentially, from lower address to higher address. > 5. The block offset shared array used by the mature space will be > allocated at the end of the card table area, which is used by mature > space. And the virtual space size of this area will be set according > to the mature space and word bits. > > So, is there any space allocated following? > > Thanks, > Neo > -- > I would remember that if researchers were not ambitious > probably today we haven't the technology we are using! > > No virus found in this incoming message. > Checked by AVG Free Edition. > Version: 7.5.467 / Virus Database: 269.6.8/800 - Release Date: 5/11/2007 > 7:34 PM > No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.467 / Virus Database: 269.7.0/803 - Release Date: 5/13/2007 12:17 PM From neojia at gmail.com Mon May 14 22:17:49 2007 From: neojia at gmail.com (Neo Jia) Date: Tue, 15 May 2007 00:17:49 -0500 Subject: Eden space is not empty after a Full GC? In-Reply-To: <46489BA1.8050005@Sun.COM> References: <5d649bdb0705140900x2540737al34c4c90dba73b48d@mail.gmail.com> <46488CFE.1090400@Sun.COM> <5d649bdb0705140937k4e152204odb9a84949cda093b@mail.gmail.com> <46489393.4030405@Sun.COM> <5d649bdb0705141002o6a0f82a3l23c658a7bd38da22@mail.gmail.com> <46489BA1.8050005@Sun.COM> Message-ID: <5d649bdb0705142217u297a9e00x13f59f84beee81a0@mail.gmail.com> On 5/14/07, Jon Masamitsu wrote: > Yes, it's almost random in terms of which objects get > left in the young generation. It is a sliding compaction > so objects at the higher end of the generation are > more likely to be left behind. > > Leaving objects in the young generation probably > means that the following collection will also have to > be a full collection. It's generally better to have some > room in the old (mature) generation so that minor > collections can be done so I would say it would be > better to have additional space in the old generations. Jon, But if all the reserved space for mature space are committed, is there any way to "borrow" memory from other regions? Thanks, Neo > > Neo Jia wrote: > > > On 5/14/07, Jon Masamitsu wrote: > > > >> A perhaps more general case (i.e., no necessarily involving large > >> objects) > >> is one where all the live data does not fit into the old generation. > >> A full > >> collection will attempt to compact the young generation into the old > >> generation. If the old generation becomes full during the compaction, > >> the remaining live data in the young generation will be compacted > >> into the young generation. > > > > > > Jon, > > > > Thanks. I assume that which objects will be left in the nursery space > > is pretty random not based on the ages. Will it improve the > > performance if we intend to leave those younger objects in nursery? > > Or, just enlarging the mature space to fit in those young objects? But > > the > > reserved space is almost fully committed. > > > > Thanks, > > Neo > > > >> > >> Neo Jia wrote: > >> > >> > On 5/14/07, Y Srinivas Ramakrishna wrote: > >> > > >> >> > >> >> > >> >> Neo Jia wrote On 05/14/07 09:00 AM,: > >> >> > >> >> > hi, > >> >> > > >> >> > I just found that for the generational mark-sweep garbage > >> collector, > >> >> > after a full GC, the eden space may not be empty. > >> >> > >> >> That would usually be the case because of bin-packing > >> >> issues with the sliding compaction that might prevent a large > >> >> object in Eden from being "slid" into the available space > >> >> in old gen or in the survivor space. It'll usually be the case > >> >> then that your are running with memory quite tight (or > >> >> with a huge Eden and very large objects). > >> > > >> > > >> > Could you point me to the code where the full GC found it cannot slide > >> > this huge chunk? > >> > > >> > Thanks, > >> > Neo > >> > > >> >> > >> >> Are there other situations where you see a non-empty Eden > >> >> following a mark-compact full collection? > >> >> > >> >> > > >> >> > Is there any implementation concern for this? Can I force the > >> full GC > >> >> > to clean the eden space? > >> >> > >> >> > >> >> A full gc would do a mark-compact collection. > >> >> > >> >> -- ramki. > >> >> > >> >> > > >> >> > Thanks, > >> >> > Neo > >> >> > >> >> > >> >> -- > >> >> > >> >> > >> ---------------------------------------------------------------------------- > >> > >> >> > >> >> Y. Srinivas Ramakrishna HotSpot JVM > >> >> Sun Microsystems, Inc., USCA 22-123 CSG / Sun Software > >> >> 4220 Network Circle 408 276 7250 (x17250) > >> >> Santa Clara, CA 95054, U.S.A. Y dot S dot Ramakrishna at Sun > >> >> dot COM > >> >> > >> ---------------------------------------------------------------------------- > >> > >> >> > >> >> ----------------- Sun Proprietary / Confidential > >> >> --------------------------- > >> >> > >> >> > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > >> > >> >> > >> >> > >> >> NOTICE: This email message is for the sole use of the intended > >> >> recipient(s) > >> >> and may contain confidential and privileged information. Any > >> >> unauthorized > >> >> review, use, disclosure or distribution is prohibited. If you are not > >> >> the > >> >> intended recipient, please contact the sender by reply email and > >> destroy > >> >> all copies of the original message. > >> >> > >> >> > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > >> > >> >> > >> >> > >> >> > >> > > >> > > >> > > > > > -- I would remember that if researchers were not ambitious probably today we haven't the technology we are using! From Jon.Masamitsu at Sun.COM Tue May 15 06:33:26 2007 From: Jon.Masamitsu at Sun.COM (Jon Masamitsu) Date: Tue, 15 May 2007 06:33:26 -0700 Subject: Current heap layout of generational mark-sweep GC In-Reply-To: <5d649bdb0705141540te9717d4t38e5c2c082f8ae2c@mail.gmail.com> References: <5d649bdb0705122321w5ef63508me8f04b988dfc71e5@mail.gmail.com> <46486FEC.4090202@Sun.COM> <5d649bdb0705140855t3774a170oac4b2b48d7835ac3@mail.gmail.com> <464897FA.5020401@Sun.COM> <5d649bdb0705141540te9717d4t38e5c2c082f8ae2c@mail.gmail.com> Message-ID: <4649B6A6.5000503@Sun.COM> Regarding "PrintMallocFree" it is broken in the sense that not all malloc()/free() calls are wrapped inside the corresponding AllocateHeap()/FreeHeap(). I've used it maybe once. I don't know what to say about why it's gone stale. Neo Jia wrote: > On 5/14/07, Jon Masamitsu wrote: > >> Neo, >> >> If by "generational mark-sweep GC" you mean the UseSerialGC >> or UseParNewGC, > > > I mean the UseSerialGC. > > yes the storage you listed are the major > >> pieces that we allocate in the CHeap. There are many smaller >> data structures that are allocated out of the CHeap. For example, >> performance counters are CHeap allocated. Or just the storage >> for various subclasses of Generation that the GC uses. >> >> I believe the largest piece of the CHeap that the compilers >> use is the CodeCache and it is allocated before the Java >> heap is allocated. > > > The current "PrintMallocFree" seems broken already. Do you still use > this to check the > CHeap usage? > > Thanks, > Neo > >> >> Jon >> >> Neo Jia wrote: >> >> > On 5/14/07, Jon Masamitsu wrote: >> > >> >> Neo, >> >> >> >> Are you asking specifically about CHeap usage by the >> >> garbage collector (i.e., you're not interested in the >> >> compiler's use CHeap space)? >> > >> > >> > Jon, >> > >> > Although my question is about the CHeap usage by GC, I am also >> interested >> > in knowing the CHeap space usage of compiler. >> > >> > Thanks, >> > Neo >> > >> >> >> >> Jon >> >> >> >> >> >> Neo Jia wrote: >> >> >> >> > hi, >> >> > >> >> > I am wondering that for the generational mark-sweep GC, if the >> block >> >> > offset shared array is the last allocated object on the CHeap. >> >> > >> >> > The following is my understanding of the current heap layout: >> >> > >> >> > 1. The nursery space is on the lowest address >> >> > 2. The mature space is following the nursery address. >> >> > 3. The permanent space is following the nursery address at a higher >> >> > address. >> >> > 4. The card table is allocated from the end of the permanent space, >> >> > the regions inside it will be mapped to these previous three spaces >> >> > sequentially, from lower address to higher address. >> >> > 5. The block offset shared array used by the mature space will be >> >> > allocated at the end of the card table area, which is used by >> mature >> >> > space. And the virtual space size of this area will be set >> according >> >> > to the mature space and word bits. >> >> > >> >> > So, is there any space allocated following? >> >> > >> >> > Thanks, >> >> > Neo >> >> >> >> >> > >> > >> > > From Jon.Masamitsu at Sun.COM Tue May 15 07:04:54 2007 From: Jon.Masamitsu at Sun.COM (Jon Masamitsu) Date: Tue, 15 May 2007 07:04:54 -0700 Subject: Eden space is not empty after a Full GC? In-Reply-To: <5d649bdb0705142217u297a9e00x13f59f84beee81a0@mail.gmail.com> References: <5d649bdb0705140900x2540737al34c4c90dba73b48d@mail.gmail.com> <46488CFE.1090400@Sun.COM> <5d649bdb0705140937k4e152204odb9a84949cda093b@mail.gmail.com> <46489393.4030405@Sun.COM> <5d649bdb0705141002o6a0f82a3l23c658a7bd38da22@mail.gmail.com> <46489BA1.8050005@Sun.COM> <5d649bdb0705142217u297a9e00x13f59f84beee81a0@mail.gmail.com> Message-ID: <4649BE06.9050708@Sun.COM> Response below. Neo Jia wrote: > On 5/14/07, Jon Masamitsu wrote: > >> Yes, it's almost random in terms of which objects get >> left in the young generation. It is a sliding compaction >> so objects at the higher end of the generation are >> more likely to be left behind. >> >> Leaving objects in the young generation probably >> means that the following collection will also have to >> be a full collection. It's generally better to have some >> room in the old (mature) generation so that minor >> collections can be done so I would say it would be >> better to have additional space in the old generations. > > > Jon, > > But if all the reserved space for mature space are committed, is there > any way to "borrow" memory from other regions? > > With the UseSerialGC there is not currently a way to get the space from one generation to another. I meant to say that a user who is in this situation could resize the generations to move more space into the mature (old) generation. The UseParallelGC collector has an option UseAdaptiveGCBoundary that does allow the boundary between the young generation and the old generation to move (allowing the borrowing). That feature is incomplete in that we have not developed a good policy to decide when the boundary should be moved and by how much. The simple policy currently implemented works well sometimes but other times it causes very poor performance. From neojia at gmail.com Tue May 15 09:50:10 2007 From: neojia at gmail.com (Neo Jia) Date: Tue, 15 May 2007 11:50:10 -0500 Subject: Eden space is not empty after a Full GC? In-Reply-To: <4649BE06.9050708@Sun.COM> References: <5d649bdb0705140900x2540737al34c4c90dba73b48d@mail.gmail.com> <46488CFE.1090400@Sun.COM> <5d649bdb0705140937k4e152204odb9a84949cda093b@mail.gmail.com> <46489393.4030405@Sun.COM> <5d649bdb0705141002o6a0f82a3l23c658a7bd38da22@mail.gmail.com> <46489BA1.8050005@Sun.COM> <5d649bdb0705142217u297a9e00x13f59f84beee81a0@mail.gmail.com> <4649BE06.9050708@Sun.COM> Message-ID: <5d649bdb0705150950j408fee93lce9b687f2a952599@mail.gmail.com> On 5/15/07, Jon Masamitsu wrote: > > Response below. > > Neo Jia wrote: > > > On 5/14/07, Jon Masamitsu wrote: > > > >> Yes, it's almost random in terms of which objects get > >> left in the young generation. It is a sliding compaction > >> so objects at the higher end of the generation are > >> more likely to be left behind. > >> > >> Leaving objects in the young generation probably > >> means that the following collection will also have to > >> be a full collection. It's generally better to have some > >> room in the old (mature) generation so that minor > >> collections can be done so I would say it would be > >> better to have additional space in the old generations. > > > > > > Jon, > > > > But if all the reserved space for mature space are committed, is there > > any way to "borrow" memory from other regions? > > > > > > With the UseSerialGC there is not currently a way to get the space from > one generation to another. I meant to say that a user who is in this > situation > could resize the generations to move more space into the mature (old) > generation. The UseParallelGC collector has an option UseAdaptiveGCBoundary > that does allow the boundary between the young generation and the old > generation to move (allowing the borrowing). Jon, I assume for the ParallelGC the mature space is still using generational mark-compact, right? And from the code, I knew that the nursery space is still on the lower address of the whole GC-heap, which means you can reduce the younger generation by moving the boundary upward and reduce the mature space downward. For the younger space should be easy because we can easily make the to space and from space empty to allow this kind of adjustment. But for the mature space, the compaction point (by default) will always be the lower address, so how can you adjust the boundary when there are objects after compaction? > That feature is incomplete in > that we have not developed a good policy to decide when the boundary should > be moved and by how much. The simple policy currently implemented > works well sometimes but other times it causes very poor performance. What is the current policy? What kind of applications/benchmarks can yield the best or worst performance? Thanks, Neo > > -- I would remember that if researchers were not ambitious probably today we haven't the technology we are using! From Jon.Masamitsu at Sun.COM Tue May 15 11:07:13 2007 From: Jon.Masamitsu at Sun.COM (Jon Masamitsu) Date: Tue, 15 May 2007 11:07:13 -0700 Subject: Eden space is not empty after a Full GC? In-Reply-To: <5d649bdb0705150950j408fee93lce9b687f2a952599@mail.gmail.com> References: <5d649bdb0705140900x2540737al34c4c90dba73b48d@mail.gmail.com> <46488CFE.1090400@Sun.COM> <5d649bdb0705140937k4e152204odb9a84949cda093b@mail.gmail.com> <46489393.4030405@Sun.COM> <5d649bdb0705141002o6a0f82a3l23c658a7bd38da22@mail.gmail.com> <46489BA1.8050005@Sun.COM> <5d649bdb0705142217u297a9e00x13f59f84beee81a0@mail.gmail.com> <4649BE06.9050708@Sun.COM> <5d649bdb0705150950j408fee93lce9b687f2a952599@mail.gmail.com> Message-ID: <4649F6D1.9020002@Sun.COM> Neo Jia wrote: > On 5/15/07, Jon Masamitsu wrote: > >> >> Response below. >> >> Neo Jia wrote: >> >> > On 5/14/07, Jon Masamitsu wrote: >> > >> >> Yes, it's almost random in terms of which objects get >> >> left in the young generation. It is a sliding compaction >> >> so objects at the higher end of the generation are >> >> more likely to be left behind. >> >> >> >> Leaving objects in the young generation probably >> >> means that the following collection will also have to >> >> be a full collection. It's generally better to have some >> >> room in the old (mature) generation so that minor >> >> collections can be done so I would say it would be >> >> better to have additional space in the old generations. >> > >> > >> > Jon, >> > >> > But if all the reserved space for mature space are committed, is there >> > any way to "borrow" memory from other regions? >> > >> > >> >> With the UseSerialGC there is not currently a way to get the space from >> one generation to another. I meant to say that a user who is in this >> situation >> could resize the generations to move more space into the mature (old) >> generation. The UseParallelGC collector has an option >> UseAdaptiveGCBoundary >> that does allow the boundary between the young generation and the old >> generation to move (allowing the borrowing). > > > Jon, > > I assume for the ParallelGC the mature space is still using > generational mark-compact, right? And from the code, I knew that the > nursery space is still on the lower address of the whole GC-heap, > which means you can reduce the younger generation by moving the > boundary upward and reduce the mature space downward. For the younger > space should be easy because we can easily make the to space and from > space empty to allow this kind of adjustment. But for the mature > space, the compaction point (by default) will always be the lower > address, so how can you adjust the boundary when there are objects > after compaction? Yes, ParallelGC does use the mark-compact code but ParallelGC was changed in jdk5 to have the young generation at the high end of the address space, then the mature (old) generation and then the permanent generation. Also when UseAdaptiveGCBoundary is used, the young generation grows down and the mature generation grows up. From-space and to-space are still above eden. Since eden is typically empty after a collection and it is next to the uncommitted part of the mature generation, we can move the boundary between the young generation and the old generation. There is also code to deal with the case where eden is not empty after a collection. > >> That feature is incomplete in >> that we have not developed a good policy to decide when the boundary >> should >> be moved and by how much. The simple policy currently implemented >> works well sometimes but other times it causes very poor performance. > > > What is the current policy? What kind of applications/benchmarks can > yield the best or worst performance? The current policy just moves the boundary after each collection and gives a larger generation to the collection that is taking more time (i.e., if the VM is spending more time doing minor collections than full collections, then the boundary is moved to give more space to the young generation and visa versa). When the heap starts to get full of live data, the boundary is moving constantly (minor collections push more space into the young generations and full collections push more space back into the mature generation). It does not work well for applications that make sudden changes in behavior. In particular the specjbb2000 and specjbb2005 benchmarks run with increasing work loads. The boundary moving adjusts for the last work load and at some point cannot adjust optimally for the next larger work load. I have not investigated it but I think it is because the mature generation grows larger than is optimum and then for unknown reasons cannot be shrunk back. I mentioned that we can move the boundary even when eden is empty. We do that by expanding the mature generation so that it includes all the live data in the young generation. That might leave just too small a young generation and there is currently not way to push space back into the young generation when the mature generation is full. I cannot think why it would be an advantage to push live objects from the old generation into the young generation unless we've moved too many short lived objects into the mature generation. Anyway, that's where the policy currently stands. > > Thanks, > Neo > >> >> > > From neojia at gmail.com Tue May 15 12:04:55 2007 From: neojia at gmail.com (Neo Jia) Date: Tue, 15 May 2007 14:04:55 -0500 Subject: Eden space is not empty after a Full GC? In-Reply-To: <4649F6D1.9020002@Sun.COM> References: <5d649bdb0705140900x2540737al34c4c90dba73b48d@mail.gmail.com> <46488CFE.1090400@Sun.COM> <5d649bdb0705140937k4e152204odb9a84949cda093b@mail.gmail.com> <46489393.4030405@Sun.COM> <5d649bdb0705141002o6a0f82a3l23c658a7bd38da22@mail.gmail.com> <46489BA1.8050005@Sun.COM> <5d649bdb0705142217u297a9e00x13f59f84beee81a0@mail.gmail.com> <4649BE06.9050708@Sun.COM> <5d649bdb0705150950j408fee93lce9b687f2a952599@mail.gmail.com> <4649F6D1.9020002@Sun.COM> Message-ID: <5d649bdb0705151204u38b4faf6xd56eb922dbf94b42@mail.gmail.com> On 5/15/07, Jon Masamitsu wrote: > > > Neo Jia wrote: > > > On 5/15/07, Jon Masamitsu wrote: > > > >> > >> Response below. > >> > >> Neo Jia wrote: > >> > >> > On 5/14/07, Jon Masamitsu wrote: > >> > > >> >> Yes, it's almost random in terms of which objects get > >> >> left in the young generation. It is a sliding compaction > >> >> so objects at the higher end of the generation are > >> >> more likely to be left behind. > >> >> > >> >> Leaving objects in the young generation probably > >> >> means that the following collection will also have to > >> >> be a full collection. It's generally better to have some > >> >> room in the old (mature) generation so that minor > >> >> collections can be done so I would say it would be > >> >> better to have additional space in the old generations. > >> > > >> > > >> > Jon, > >> > > >> > But if all the reserved space for mature space are committed, is there > >> > any way to "borrow" memory from other regions? > >> > > >> > > >> > >> With the UseSerialGC there is not currently a way to get the space from > >> one generation to another. I meant to say that a user who is in this > >> situation > >> could resize the generations to move more space into the mature (old) > >> generation. The UseParallelGC collector has an option > >> UseAdaptiveGCBoundary > >> that does allow the boundary between the young generation and the old > >> generation to move (allowing the borrowing). > > > > > > Jon, > > > > I assume for the ParallelGC the mature space is still using > > generational mark-compact, right? And from the code, I knew that the > > nursery space is still on the lower address of the whole GC-heap, > > which means you can reduce the younger generation by moving the > > boundary upward and reduce the mature space downward. For the younger > > space should be easy because we can easily make the to space and from > > space empty to allow this kind of adjustment. But for the mature > > space, the compaction point (by default) will always be the lower > > address, so how can you adjust the boundary when there are objects > > after compaction? > > > Yes, ParallelGC does use the mark-compact code but ParallelGC was > changed in jdk5 to have the young generation at the high end of the > address space, then the mature (old) generation and then the > permanent generation. Jon, So the permanent generation is on the lower address to make the "ages" of each generation consistent with its address, right? This will reduce the overhead of checking inter-generational pointers Also when UseAdaptiveGCBoundary is > used, the young generation grows down and the mature generation > grows up. From-space and to-space are still above eden. Since > eden is typically empty after a collection and it is next to the > uncommitted part of the mature generation, we can move the > boundary between the young generation and the old generation. > There is also code to deal with the case where eden is not empty > after a collection. > > > > >> That feature is incomplete in > >> that we have not developed a good policy to decide when the boundary > >> should > >> be moved and by how much. The simple policy currently implemented > >> works well sometimes but other times it causes very poor performance. > > > > > > What is the current policy? What kind of applications/benchmarks can > > yield the best or worst performance? > > > The current policy just moves the boundary after each collection and > gives a larger generation to the collection that is taking more time > (i.e., if the VM is spending more time doing minor collections than > full collections, then the boundary is moved to give more space to > the young generation and visa versa). When the heap starts to get full > of live data, the boundary is moving constantly (minor collections push > more space into the young generations and full collections push more > space back into the mature generation). It does not work well for > applications > that make sudden changes in behavior. In particular the specjbb2000 > and specjbb2005 benchmarks run with increasing work loads. The > boundary moving adjusts for the last work load and at some point > cannot adjust optimally for the next larger work load. It seems that your current adjustment is based on the GC time of each generation. How can this connect to the "increasing work loads"? Because I assume that the increasing work load should connect with the incoming allocation rate (bytes). I have not > investigated it but I think it is because the mature generation grows > larger than is optimum and then for unknown reasons cannot be > shrunk back. I mentioned that we can move the boundary even when > eden is empty. I assume here you mean the eden space is non-empty, right? We do that by expanding the mature generation so > that it includes all the live data in the young generation. That > might leave just too small a young generation and there is currently > not way to push space back into the young generation when the > mature generation is full. I cannot think why it would be an advantage > to push live objects from the old generation into the young generation > unless we've moved too many short lived objects into the mature > generation. Anyway, that's where the policy currently stands. Thank you for your detailed explanation about the current adjustment policy! Thanks, Neo > > > > > > > > Thanks, > > Neo > > > >> > >> > > > > > -- I would remember that if researchers were not ambitious probably today we haven't the technology we are using! From Jon.Masamitsu at Sun.COM Tue May 15 14:16:42 2007 From: Jon.Masamitsu at Sun.COM (Jon Masamitsu) Date: Tue, 15 May 2007 14:16:42 -0700 Subject: Eden space is not empty after a Full GC? In-Reply-To: <5d649bdb0705151204u38b4faf6xd56eb922dbf94b42@mail.gmail.com> References: <5d649bdb0705140900x2540737al34c4c90dba73b48d@mail.gmail.com> <46488CFE.1090400@Sun.COM> <5d649bdb0705140937k4e152204odb9a84949cda093b@mail.gmail.com> <46489393.4030405@Sun.COM> <5d649bdb0705141002o6a0f82a3l23c658a7bd38da22@mail.gmail.com> <46489BA1.8050005@Sun.COM> <5d649bdb0705142217u297a9e00x13f59f84beee81a0@mail.gmail.com> <4649BE06.9050708@Sun.COM> <5d649bdb0705150950j408fee93lce9b687f2a952599@mail.gmail.com> <4649F6D1.9020002@Sun.COM> <5d649bdb0705151204u38b4faf6xd56eb922dbf94b42@mail.gmail.com> Message-ID: <464A233A.4080007@Sun.COM> Neo Jia wrote: > On 5/15/07, Jon Masamitsu wrote: > >> ... > > > Jon, > > So the permanent generation is on the lower address to make the "ages" > of each generation consistent with its address, right? This will > reduce the overhead of checking inter-generational pointers Exactly. We wanted a single test to see if a reference pointed into the young generation (as opposed to a test to see if it pointed in from above and another test to see if it pointed in from below). > > Also when UseAdaptiveGCBoundary is > >> used, the young generation grows down and the mature generation >> grows up. From-space and to-space are still above eden. Since >> eden is typically empty after a collection and it is next to the >> uncommitted part of the mature generation, we can move the >> boundary between the young generation and the old generation. >> There is also code to deal with the case where eden is not empty >> after a collection. >> >> > >> >> That feature is incomplete in >> >> that we have not developed a good policy to decide when the boundary >> >> should >> >> be moved and by how much. The simple policy currently implemented >> >> works well sometimes but other times it causes very poor performance. >> > >> > >> > What is the current policy? What kind of applications/benchmarks can >> > yield the best or worst performance? >> >> >> The current policy just moves the boundary after each collection and >> gives a larger generation to the collection that is taking more time >> (i.e., if the VM is spending more time doing minor collections than >> full collections, then the boundary is moved to give more space to >> the young generation and visa versa). When the heap starts to get full >> of live data, the boundary is moving constantly (minor collections push >> more space into the young generations and full collections push more >> space back into the mature generation). It does not work well for >> applications >> that make sudden changes in behavior. In particular the specjbb2000 >> and specjbb2005 benchmarks run with increasing work loads. The >> boundary moving adjusts for the last work load and at some point >> cannot adjust optimally for the next larger work load. > > > It seems that your current adjustment is based on the GC time of each > generation. How > can this connect to the "increasing work loads"? Because I assume that > the increasing work > load should connect with the incoming allocation rate (bytes). I should have said fraction of time spent in GC. So increased work load means increased allocation rate which means less time between GC's. > > I have not > >> investigated it but I think it is because the mature generation grows >> larger than is optimum and then for unknown reasons cannot be >> shrunk back. I mentioned that we can move the boundary even when >> eden is empty. > > > I assume here you mean the eden space is non-empty, right? Yes, non-empty. From neojia at gmail.com Wed May 16 00:53:52 2007 From: neojia at gmail.com (Neo Jia) Date: Wed, 16 May 2007 02:53:52 -0500 Subject: Current heap layout of generational mark-sweep GC In-Reply-To: <4649B6A6.5000503@Sun.COM> References: <5d649bdb0705122321w5ef63508me8f04b988dfc71e5@mail.gmail.com> <46486FEC.4090202@Sun.COM> <5d649bdb0705140855t3774a170oac4b2b48d7835ac3@mail.gmail.com> <464897FA.5020401@Sun.COM> <5d649bdb0705141540te9717d4t38e5c2c082f8ae2c@mail.gmail.com> <4649B6A6.5000503@Sun.COM> Message-ID: <5d649bdb0705160053l636bfc54i8326d90f80400b39@mail.gmail.com> On 5/15/07, Jon Masamitsu wrote: > Regarding "PrintMallocFree" it is broken in the sense that not > all malloc()/free() calls are wrapped inside the corresponding > AllocateHeap()/FreeHeap(). I've used it maybe once. I don't > know what to say about why it's gone stale. Jon, Not only not catching all the malloc/free but also crashing the strlen function on JDK7. This is what I found several days before. So, is GC people in charge of detecting memory leak? Because after running valgrind with hotspot, it reports lots memory lost. Thanks, Neo > > Neo Jia wrote: > > > On 5/14/07, Jon Masamitsu wrote: > > > >> Neo, > >> > >> If by "generational mark-sweep GC" you mean the UseSerialGC > >> or UseParNewGC, > > > > > > I mean the UseSerialGC. > > > > yes the storage you listed are the major > > > >> pieces that we allocate in the CHeap. There are many smaller > >> data structures that are allocated out of the CHeap. For example, > >> performance counters are CHeap allocated. Or just the storage > >> for various subclasses of Generation that the GC uses. > >> > >> I believe the largest piece of the CHeap that the compilers > >> use is the CodeCache and it is allocated before the Java > >> heap is allocated. > > > > > > The current "PrintMallocFree" seems broken already. Do you still use > > this to check the > > CHeap usage? > > > > Thanks, > > Neo > > > >> > >> Jon > >> > >> Neo Jia wrote: > >> > >> > On 5/14/07, Jon Masamitsu wrote: > >> > > >> >> Neo, > >> >> > >> >> Are you asking specifically about CHeap usage by the > >> >> garbage collector (i.e., you're not interested in the > >> >> compiler's use CHeap space)? > >> > > >> > > >> > Jon, > >> > > >> > Although my question is about the CHeap usage by GC, I am also > >> interested > >> > in knowing the CHeap space usage of compiler. > >> > > >> > Thanks, > >> > Neo > >> > > >> >> > >> >> Jon > >> >> > >> >> > >> >> Neo Jia wrote: > >> >> > >> >> > hi, > >> >> > > >> >> > I am wondering that for the generational mark-sweep GC, if the > >> block > >> >> > offset shared array is the last allocated object on the CHeap. > >> >> > > >> >> > The following is my understanding of the current heap layout: > >> >> > > >> >> > 1. The nursery space is on the lowest address > >> >> > 2. The mature space is following the nursery address. > >> >> > 3. The permanent space is following the nursery address at a higher > >> >> > address. > >> >> > 4. The card table is allocated from the end of the permanent space, > >> >> > the regions inside it will be mapped to these previous three spaces > >> >> > sequentially, from lower address to higher address. > >> >> > 5. The block offset shared array used by the mature space will be > >> >> > allocated at the end of the card table area, which is used by > >> mature > >> >> > space. And the virtual space size of this area will be set > >> according > >> >> > to the mature space and word bits. > >> >> > > >> >> > So, is there any space allocated following? > >> >> > > >> >> > Thanks, > >> >> > Neo > >> >> > >> >> > >> > > >> > > >> > > > > > -- I would remember that if researchers were not ambitious probably today we haven't the technology we are using! From Jon.Masamitsu at Sun.COM Wed May 16 06:43:33 2007 From: Jon.Masamitsu at Sun.COM (Jon Masamitsu) Date: Wed, 16 May 2007 06:43:33 -0700 Subject: Current heap layout of generational mark-sweep GC In-Reply-To: <5d649bdb0705160053l636bfc54i8326d90f80400b39@mail.gmail.com> References: <5d649bdb0705122321w5ef63508me8f04b988dfc71e5@mail.gmail.com> <46486FEC.4090202@Sun.COM> <5d649bdb0705140855t3774a170oac4b2b48d7835ac3@mail.gmail.com> <464897FA.5020401@Sun.COM> <5d649bdb0705141540te9717d4t38e5c2c082f8ae2c@mail.gmail.com> <4649B6A6.5000503@Sun.COM> <5d649bdb0705160053l636bfc54i8326d90f80400b39@mail.gmail.com> Message-ID: <464B0A85.1070502@Sun.COM> Neo Jia wrote: > On 5/15/07, Jon Masamitsu wrote: > >> Regarding "PrintMallocFree" it is broken in the sense that not >> all malloc()/free() calls are wrapped inside the corresponding >> AllocateHeap()/FreeHeap(). I've used it maybe once. I don't >> know what to say about why it's gone stale. > > > Jon, > > Not only not catching all the malloc/free but also crashing the strlen > function on JDK7. This is what I found several days before. Well that's bad. Does running just anything with "PrintMallocFree" cause this crash? > > So, is GC people in charge of detecting memory leak? Because after > running valgrind with hotspot, it reports lots memory lost. No, each group (GC, runtime, compilers, and serviceability) manages its own usage of the CHeap with regard to leaks. > > Thanks, > Neo > > >> >> Neo Jia wrote: >> >> > On 5/14/07, Jon Masamitsu wrote: >> > >> >> Neo, >> >> >> >> If by "generational mark-sweep GC" you mean the UseSerialGC >> >> or UseParNewGC, >> > >> > >> > I mean the UseSerialGC. >> > >> > yes the storage you listed are the major >> > >> >> pieces that we allocate in the CHeap. There are many smaller >> >> data structures that are allocated out of the CHeap. For example, >> >> performance counters are CHeap allocated. Or just the storage >> >> for various subclasses of Generation that the GC uses. >> >> >> >> I believe the largest piece of the CHeap that the compilers >> >> use is the CodeCache and it is allocated before the Java >> >> heap is allocated. >> > >> > >> > The current "PrintMallocFree" seems broken already. Do you still use >> > this to check the >> > CHeap usage? >> > >> > Thanks, >> > Neo >> > >> >> >> >> Jon >> >> >> >> Neo Jia wrote: >> >> >> >> > On 5/14/07, Jon Masamitsu wrote: >> >> > >> >> >> Neo, >> >> >> >> >> >> Are you asking specifically about CHeap usage by the >> >> >> garbage collector (i.e., you're not interested in the >> >> >> compiler's use CHeap space)? >> >> > >> >> > >> >> > Jon, >> >> > >> >> > Although my question is about the CHeap usage by GC, I am also >> >> interested >> >> > in knowing the CHeap space usage of compiler. >> >> > >> >> > Thanks, >> >> > Neo >> >> > >> >> >> >> >> >> Jon >> >> >> >> >> >> >> >> >> Neo Jia wrote: >> >> >> >> >> >> > hi, >> >> >> > >> >> >> > I am wondering that for the generational mark-sweep GC, if the >> >> block >> >> >> > offset shared array is the last allocated object on the CHeap. >> >> >> > >> >> >> > The following is my understanding of the current heap layout: >> >> >> > >> >> >> > 1. The nursery space is on the lowest address >> >> >> > 2. The mature space is following the nursery address. >> >> >> > 3. The permanent space is following the nursery address at a >> higher >> >> >> > address. >> >> >> > 4. The card table is allocated from the end of the permanent >> space, >> >> >> > the regions inside it will be mapped to these previous three >> spaces >> >> >> > sequentially, from lower address to higher address. >> >> >> > 5. The block offset shared array used by the mature space >> will be >> >> >> > allocated at the end of the card table area, which is used by >> >> mature >> >> >> > space. And the virtual space size of this area will be set >> >> according >> >> >> > to the mature space and word bits. >> >> >> > >> >> >> > So, is there any space allocated following? >> >> >> > >> >> >> > Thanks, >> >> >> > Neo >> >> >> >> >> >> >> >> > >> >> > >> >> >> > >> > >> > > From neojia at gmail.com Wed May 16 20:56:27 2007 From: neojia at gmail.com (Neo Jia) Date: Wed, 16 May 2007 22:56:27 -0500 Subject: Current heap layout of generational mark-sweep GC In-Reply-To: <464B0A85.1070502@Sun.COM> References: <5d649bdb0705122321w5ef63508me8f04b988dfc71e5@mail.gmail.com> <46486FEC.4090202@Sun.COM> <5d649bdb0705140855t3774a170oac4b2b48d7835ac3@mail.gmail.com> <464897FA.5020401@Sun.COM> <5d649bdb0705141540te9717d4t38e5c2c082f8ae2c@mail.gmail.com> <4649B6A6.5000503@Sun.COM> <5d649bdb0705160053l636bfc54i8326d90f80400b39@mail.gmail.com> <464B0A85.1070502@Sun.COM> Message-ID: <5d649bdb0705162056j26b71e5ds9d5d46304b56ae24@mail.gmail.com> On 5/16/07, Jon Masamitsu wrote: > > > Neo Jia wrote: > > > On 5/15/07, Jon Masamitsu wrote: > > > >> Regarding "PrintMallocFree" it is broken in the sense that not > >> all malloc()/free() calls are wrapped inside the corresponding > >> AllocateHeap()/FreeHeap(). I've used it maybe once. I don't > >> know what to say about why it's gone stale. > > > > > > Jon, > > > > Not only not catching all the malloc/free but also crashing the strlen > > function on JDK7. This is what I found several days before. > > > Well that's bad. Does running just anything with "PrintMallocFree" > cause this crash? Jon, I just submit a patch for this problem to the runtime mailing list. http://thread.gmane.org/gmane.comp.java.openjdk.hotspot.runtime.devel/37 Thanks, Neo > > > > > So, is GC people in charge of detecting memory leak? Because after > > running valgrind with hotspot, it reports lots memory lost. > > > No, each group (GC, runtime, compilers, and serviceability) > manages its own usage of the CHeap with regard to leaks. > > > > > Thanks, > > Neo > > > > > >> > >> Neo Jia wrote: > >> > >> > On 5/14/07, Jon Masamitsu wrote: > >> > > >> >> Neo, > >> >> > >> >> If by "generational mark-sweep GC" you mean the UseSerialGC > >> >> or UseParNewGC, > >> > > >> > > >> > I mean the UseSerialGC. > >> > > >> > yes the storage you listed are the major > >> > > >> >> pieces that we allocate in the CHeap. There are many smaller > >> >> data structures that are allocated out of the CHeap. For example, > >> >> performance counters are CHeap allocated. Or just the storage > >> >> for various subclasses of Generation that the GC uses. > >> >> > >> >> I believe the largest piece of the CHeap that the compilers > >> >> use is the CodeCache and it is allocated before the Java > >> >> heap is allocated. > >> > > >> > > >> > The current "PrintMallocFree" seems broken already. Do you still use > >> > this to check the > >> > CHeap usage? > >> > > >> > Thanks, > >> > Neo > >> > > >> >> > >> >> Jon > >> >> > >> >> Neo Jia wrote: > >> >> > >> >> > On 5/14/07, Jon Masamitsu wrote: > >> >> > > >> >> >> Neo, > >> >> >> > >> >> >> Are you asking specifically about CHeap usage by the > >> >> >> garbage collector (i.e., you're not interested in the > >> >> >> compiler's use CHeap space)? > >> >> > > >> >> > > >> >> > Jon, > >> >> > > >> >> > Although my question is about the CHeap usage by GC, I am also > >> >> interested > >> >> > in knowing the CHeap space usage of compiler. > >> >> > > >> >> > Thanks, > >> >> > Neo > >> >> > > >> >> >> > >> >> >> Jon > >> >> >> > >> >> >> > >> >> >> Neo Jia wrote: > >> >> >> > >> >> >> > hi, > >> >> >> > > >> >> >> > I am wondering that for the generational mark-sweep GC, if the > >> >> block > >> >> >> > offset shared array is the last allocated object on the CHeap. > >> >> >> > > >> >> >> > The following is my understanding of the current heap layout: > >> >> >> > > >> >> >> > 1. The nursery space is on the lowest address > >> >> >> > 2. The mature space is following the nursery address. > >> >> >> > 3. The permanent space is following the nursery address at a > >> higher > >> >> >> > address. > >> >> >> > 4. The card table is allocated from the end of the permanent > >> space, > >> >> >> > the regions inside it will be mapped to these previous three > >> spaces > >> >> >> > sequentially, from lower address to higher address. > >> >> >> > 5. The block offset shared array used by the mature space > >> will be > >> >> >> > allocated at the end of the card table area, which is used by > >> >> mature > >> >> >> > space. And the virtual space size of this area will be set > >> >> according > >> >> >> > to the mature space and word bits. > >> >> >> > > >> >> >> > So, is there any space allocated following? > >> >> >> > > >> >> >> > Thanks, > >> >> >> > Neo > >> >> >> > >> >> >> > >> >> > > >> >> > > >> >> > >> > > >> > > >> > > > > > -- I would remember that if researchers were not ambitious probably today we haven't the technology we are using! From Y.S.Ramakrishna at Sun.COM Thu May 17 15:00:12 2007 From: Y.S.Ramakrishna at Sun.COM (Y.S.Ramakrishna at Sun.COM) Date: Thu, 17 May 2007 15:00:12 -0700 Subject: [PATCH] Adding limitation while checking for cardTable In-Reply-To: <5d649bdb0705101339i2ac40268r81398f3d792cf769@mail.gmail.com> References: <5d649bdb0705091531p72eb0cbpc5b45ab74756cbd3@mail.gmail.com> <46437EAE.70409@Sun.COM> <5d649bdb0705101339i2ac40268r81398f3d792cf769@mail.gmail.com> Message-ID: <464CD06C.8000407@Sun.COM> Hi Neo -- thanks for finding this problem and initiating a fix. I will help shepherd this bug fix via: 6559052 [OpenJDK] CardTableRS::do_MemRegion() could attempt out-of-bounds translation in addr_for() -- ramki. Neo Jia wrote: > On 5/10/07, Y Srinivas Ramakrishna wrote: > >> Hi Neo -- >> >> Thanks for the heads-up. This seems worth fixing, at least >> for hygiene, How about the following instead, where we avoid the >> addr_for() translation until we are sure it will be safe, >> provided that the mr argument to the method is backed by >> card table entries. This avoids the need for the >> extra compare-and-branch inside the loop in your patch. >> > > Ramki, > > Thanks. Your fix makes more sense than mine. > > Neo > >> void do_MemRegion(MemRegion mr) { >> HeapWord* end_of_non_clean = mr.end(); >> HeapWord* start_of_non_clean = end_of_non_clean; >> jbyte* entry = _ct->byte_for(mr.last()); >> jbyte* first_entry = _ct->byte_for(mr.start()); >> while (entry >= first_entry) { >> jbyte entry_val = *entry; >> HeapWord* cur = _ct->addr_for(entry); >> if (!clear_card(entry)) { >> if (start_of_non_clean < end_of_non_clean) { >> MemRegion mr2(start_of_non_clean, end_of_non_clean); >> _dirty_card_closure->do_MemRegion(mr2); >> } >> end_of_non_clean = cur; >> start_of_non_clean = end_of_non_clean; >> } >> start_of_non_clean = cur; >> entry--; >> } >> if (start_of_non_clean < end_of_non_clean) { >> MemRegion mr2(start_of_non_clean, end_of_non_clean); >> _dirty_card_closure->do_MemRegion(mr2); >> } >> } >> >> -- Ramki Ramakrishna >> >> Neo Jia wrote On 05/09/07 03:31 PM,: >> >> > hi, >> > >> > Acturally, the existing code works well although it misses the >> > limitation checking, because the current heap layout will make sure >> > the memory region checking failed before it goes beyond the limitation >> > of byte map. >> > >> > The reason is that the nursery space is on the lower address, which >> > will make its committed card table also on the lower address of inside >> > the byte maps. While walking through the card table of the mature >> > space and keeping reduce the entry, we will finally hit the boundary >> > of its byte map before doing the check of memory region, which will >> > fire the assert of the addr_for, if it is on the lower address. >> > >> > So, just adding more safety for this function. >> > >> > Thanks, >> > Neo >> > Index: memory/cardTableRS.cpp >> > =================================================================== >> > --- memory/cardTableRS.cpp (revision 86) >> > +++ memory/cardTableRS.cpp (working copy) >> > @@ -163,6 +163,9 @@ >> > start_of_non_clean = end_of_non_clean; >> > } >> > entry--; >> > + // Adding a limit checking for safety. >> > + if (entry < _ct->base_map()) >> > + break; >> > start_of_non_clean = cur; >> > cur = _ct->addr_for(entry); >> > } >> > Index: memory/cardTableRS.hpp >> > =================================================================== >> > --- memory/cardTableRS.hpp (revision 86) >> > +++ memory/cardTableRS.hpp (working copy) >> > @@ -101,6 +101,8 @@ >> > >> > CardTableModRefBS* ct_bs() { return &_ct_bs; } >> > >> > + jbyte * base_map() { return &(_ct_bs._byte_map[0]); } >> > + >> > // Override. >> > void prepare_for_younger_refs_iterate(bool parallel); >> > >> >> -- >> >> ---------------------------------------------------------------------------- >> >> Y. Srinivas Ramakrishna HotSpot JVM >> Sun Microsystems, Inc., USCA 14-102 JWS / Software Group >> 4140 Network Circle 408 276 7250 (x17250) >> Santa Clara, CA 95054, U.S.A. Y dot S dot Ramakrishna at Sun >> dot COM >> ---------------------------------------------------------------------------- >> >> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> >> NOTICE: This email message is for the sole use of the intended >> recipient(s) >> and may contain confidential and privileged information. Any >> unauthorized >> review, use, disclosure or distribution is prohibited. If you are not >> the >> intended recipient, please contact the sender by reply email and destroy >> all copies of the original message. >> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> >> > > From Tom.Marble at Sun.COM Sun May 20 13:26:01 2007 From: Tom.Marble at Sun.COM (Tom Marble) Date: Sun, 20 May 2007 15:26:01 -0500 Subject: test of new Gmane migration Message-ID: <4650AED9.9020309@sun.com> All: Please ignore this test message whose purpose is to confirm that the Gmane gateway has been migrated from the old list to the new list. --Tom From neojia at gmail.com Sun May 20 23:11:56 2007 From: neojia at gmail.com (Neo Jia) Date: Mon, 21 May 2007 01:11:56 -0500 Subject: [PATCH] Adding limitation while checking for cardTable In-Reply-To: <464CD06C.8000407@Sun.COM> References: <5d649bdb0705091531p72eb0cbpc5b45ab74756cbd3@mail.gmail.com> <46437EAE.70409@Sun.COM> <5d649bdb0705101339i2ac40268r81398f3d792cf769@mail.gmail.com> <464CD06C.8000407@Sun.COM> Message-ID: <5d649bdb0705202311p44a3dd8bub39542bb2ec5a571@mail.gmail.com> On 5/17/07, Y.S.Ramakrishna at sun.com wrote: > Hi Neo -- thanks for finding this problem and > initiating a fix. I will help shepherd this bug fix via: > > 6559052 [OpenJDK] CardTableRS::do_MemRegion() could attempt > out-of-bounds translation in addr_for() Ramki, Just saw the bug from SDN. Thanks. Neo > > -- ramki. > > Neo Jia wrote: > > > On 5/10/07, Y Srinivas Ramakrishna wrote: > > > >> Hi Neo -- > >> > >> Thanks for the heads-up. This seems worth fixing, at least > >> for hygiene, How about the following instead, where we avoid the > >> addr_for() translation until we are sure it will be safe, > >> provided that the mr argument to the method is backed by > >> card table entries. This avoids the need for the > >> extra compare-and-branch inside the loop in your patch. > >> > > > > Ramki, > > > > Thanks. Your fix makes more sense than mine. > > > > Neo > > > >> void do_MemRegion(MemRegion mr) { > >> HeapWord* end_of_non_clean = mr.end(); > >> HeapWord* start_of_non_clean = end_of_non_clean; > >> jbyte* entry = _ct->byte_for(mr.last()); > >> jbyte* first_entry = _ct->byte_for(mr.start()); > >> while (entry >= first_entry) { > >> jbyte entry_val = *entry; > >> HeapWord* cur = _ct->addr_for(entry); > >> if (!clear_card(entry)) { > >> if (start_of_non_clean < end_of_non_clean) { > >> MemRegion mr2(start_of_non_clean, end_of_non_clean); > >> _dirty_card_closure->do_MemRegion(mr2); > >> } > >> end_of_non_clean = cur; > >> start_of_non_clean = end_of_non_clean; > >> } > >> start_of_non_clean = cur; > >> entry--; > >> } > >> if (start_of_non_clean < end_of_non_clean) { > >> MemRegion mr2(start_of_non_clean, end_of_non_clean); > >> _dirty_card_closure->do_MemRegion(mr2); > >> } > >> } > >> > >> -- Ramki Ramakrishna > >> > >> Neo Jia wrote On 05/09/07 03:31 PM,: > >> > >> > hi, > >> > > >> > Acturally, the existing code works well although it misses the > >> > limitation checking, because the current heap layout will make sure > >> > the memory region checking failed before it goes beyond the limitation > >> > of byte map. > >> > > >> > The reason is that the nursery space is on the lower address, which > >> > will make its committed card table also on the lower address of inside > >> > the byte maps. While walking through the card table of the mature > >> > space and keeping reduce the entry, we will finally hit the boundary > >> > of its byte map before doing the check of memory region, which will > >> > fire the assert of the addr_for, if it is on the lower address. > >> > > >> > So, just adding more safety for this function. > >> > > >> > Thanks, > >> > Neo > >> > Index: memory/cardTableRS.cpp > >> > =================================================================== > >> > --- memory/cardTableRS.cpp (revision 86) > >> > +++ memory/cardTableRS.cpp (working copy) > >> > @@ -163,6 +163,9 @@ > >> > start_of_non_clean = end_of_non_clean; > >> > } > >> > entry--; > >> > + // Adding a limit checking for safety. > >> > + if (entry < _ct->base_map()) > >> > + break; > >> > start_of_non_clean = cur; > >> > cur = _ct->addr_for(entry); > >> > } > >> > Index: memory/cardTableRS.hpp > >> > =================================================================== > >> > --- memory/cardTableRS.hpp (revision 86) > >> > +++ memory/cardTableRS.hpp (working copy) > >> > @@ -101,6 +101,8 @@ > >> > > >> > CardTableModRefBS* ct_bs() { return &_ct_bs; } > >> > > >> > + jbyte * base_map() { return &(_ct_bs._byte_map[0]); } > >> > + > >> > // Override. > >> > void prepare_for_younger_refs_iterate(bool parallel); > >> > > >> > >> -- > >> > >> ---------------------------------------------------------------------------- > >> > >> Y. Srinivas Ramakrishna HotSpot JVM > >> Sun Microsystems, Inc., USCA 14-102 JWS / Software Group > >> 4140 Network Circle 408 276 7250 (x17250) > >> Santa Clara, CA 95054, U.S.A. Y dot S dot Ramakrishna at Sun > >> dot COM > >> ---------------------------------------------------------------------------- > >> > >> > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > >> > >> > >> NOTICE: This email message is for the sole use of the intended > >> recipient(s) > >> and may contain confidential and privileged information. Any > >> unauthorized > >> review, use, disclosure or distribution is prohibited. If you are not > >> the > >> intended recipient, please contact the sender by reply email and destroy > >> all copies of the original message. > >> > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > >> > >> > >> > > > > > > -- I would remember that if researchers were not ambitious probably today we haven't the technology we are using!