From fweimer at bfk.de  Mon Mar  1 00:31:25 2010
From: fweimer at bfk.de (Florian Weimer)
Date: Mon, 01 Mar 2010 08:31:25 +0000
Subject: g1: remembered set scan costs very high w/ LRU cache type of
	behavior
In-Reply-To: <5a1151761002271518q40d95865o311699ef66764d32@mail.gmail.com>
	(Peter Schuller's message of "Sun\, 28 Feb 2010 00\:18\:31 +0100")
References: <5a1151761002271048n798d8e54of9981e88e2b447a8@mail.gmail.com>
	<4B896ED3.3040107@sun.com>
	<5a1151761002271219j3f241f9aj6bdd5b2a724ceebe@mail.gmail.com>
	<4B898ABB.4000409@sun.com>
	<5a1151761002271518q40d95865o311699ef66764d32@mail.gmail.com>
Message-ID: <82635gzbmq.fsf@mid.bfk.de>

* Peter Schuller:

>> The problem you're seeing may be addressed by some
>> work in progress that is being done under CR
>>
>> 6923991: G1: improve scalability of RSet scanning
>>
>> I don't think that work has been released yet.
>
> Well, if this commit constitutes the work:
>
>    http://hg.openjdk.java.net/jdk7/hotspot/hotspot/rev/0414c1049f15
>
> Then I suspect I have it, because the JDK comes from:
>
>    jdk-7-ea-bin-b84-linux-x64-18_feb_2010.bin
>
> Which is 7 days after the commit happened.

I don't think so, 0414c1049f15 does not seem to have been promoted to
the master repository yet.

-- 
Florian Weimer                <fweimer at bfk.de>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstra?e 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From john.cuthbertson at sun.com  Wed Mar  3 00:00:00 2010
From: john.cuthbertson at sun.com (john.cuthbertson at sun.com)
Date: Wed, 03 Mar 2010 08:00:00 +0000
Subject: hg: jdk7/hotspot-gc/hotspot: 17 new changesets
Message-ID: <20100303080102.3286D412E6@hg.openjdk.java.net>

Changeset: e7b1cc79bd25
Author:    kvn
Date:      2010-02-16 16:17 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/e7b1cc79bd25

6926697: "optimized" VM build failed: The type "AdapterHandlerTableIterator" is incomplete
Summary: Define AdapterHandlerTableIterator class as non product instead of debug.
Reviewed-by: never

! src/share/vm/runtime/sharedRuntime.cpp

Changeset: 106f41e88c85
Author:    never
Date:      2010-02-16 20:07 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/106f41e88c85

6877221: Endless deoptimizations in OSR nmethod
Reviewed-by: kvn

! src/share/vm/opto/parse1.cpp

Changeset: b4b440360f1e
Author:    twisti
Date:      2010-02-18 11:35 +0100
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/b4b440360f1e

6926782: CodeBuffer size too small after 6921352
Summary: After 6921352 the CodeBuffer size was too small.
Reviewed-by: kvn, never

! src/share/vm/opto/callGenerator.cpp
! src/share/vm/opto/compile.cpp
! src/share/vm/opto/compile.hpp
! src/share/vm/opto/output.cpp

Changeset: 3b687c53c266
Author:    twisti
Date:      2010-02-18 06:54 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/3b687c53c266

6927165: Zero S/390 fixes
Summary: Fixes two failures on 31-bit S/390.
Reviewed-by: twisti
Contributed-by: Gary Benson <gbenson at redhat.com>

! src/cpu/zero/vm/globals_zero.hpp
! src/os_cpu/linux_zero/vm/os_linux_zero.hpp

Changeset: 72f1840531a4
Author:    twisti
Date:      2010-02-18 10:44 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/72f1840531a4

Merge


Changeset: 877a14af58e1
Author:    never
Date:      2010-02-18 15:05 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/877a14af58e1

6663854: assert(n != __null,"Bad immediate dominator info.") in C2 with -Xcomp
Reviewed-by: kvn

! src/share/vm/opto/split_if.cpp
+ test/compiler/6663854/Test6663854.java

Changeset: 2883969d09e7
Author:    kvn
Date:      2010-02-19 10:04 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/2883969d09e7

6910664: C2: java/util/Arrays/Sorting.java fails with DeoptimizeALot flag
Summary: Matcher::float_in_double should be true only when FPU is used for floats.
Reviewed-by: never, twisti

! src/cpu/sparc/vm/sparc.ad
! src/cpu/x86/vm/x86_32.ad
! src/cpu/x86/vm/x86_64.ad
! src/share/vm/opto/matcher.hpp
! src/share/vm/opto/output.cpp

Changeset: b71f13525cc8
Author:    never
Date:      2010-02-19 13:06 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/b71f13525cc8

6927049: assert(is_Loop(),"invalid node class")
Reviewed-by: kvn

! src/share/vm/opto/loopTransform.cpp

Changeset: 8b38237bae55
Author:    kvn
Date:      2010-02-22 16:56 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/8b38237bae55

6928717: HS17 fails to build with SS11 C++
Summary: Add missing handles.inline.hpp for codeCache.cpp.
Reviewed-by: never

! src/share/vm/includeDB_core

Changeset: 855c5171834c
Author:    twisti
Date:      2010-02-23 17:46 +0100
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/855c5171834c

6928839: JSR 292 typo in x86 _adapter_check_cast
Summary: There is a small typo in methodHandles_x86.cpp.
Reviewed-by: kvn

! src/cpu/x86/vm/methodHandles_x86.cpp

Changeset: da9559b49b84
Author:    never
Date:      2010-02-25 11:38 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/da9559b49b84

6915557: assert(_gvn.type(l)->higher_equal(type),"must constrain OSR typestate") with debug build
Reviewed-by: kvn

! src/share/vm/opto/parse1.cpp

Changeset: 2432acbee618
Author:    kvn
Date:      2010-02-25 15:55 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/2432acbee618

6930035: C2 type system incorrectly handles case j.l.Object->meet(constant AryPtr)
Summary: Add missing code.
Reviewed-by: never

! src/share/vm/opto/type.cpp

Changeset: 336c6c200f5f
Author:    kvn
Date:      2010-02-25 22:58 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/336c6c200f5f

6930116: loop predication code does not handle If nodes with only one projection
Summary: Add check for iff->outcnt() < 2.
Reviewed-by: never

! src/share/vm/opto/loopTransform.cpp

Changeset: 7d236a9688c5
Author:    never
Date:      2010-03-01 12:12 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/7d236a9688c5

6930398: fix for return address locals in OSR entries uses wrong test
Reviewed-by: kvn

! src/share/vm/opto/parse1.cpp

Changeset: ab75c83d7c37
Author:    johnc
Date:      2010-03-02 13:57 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/ab75c83d7c37

Merge

! src/share/vm/includeDB_core

Changeset: 8911d8c0596f
Author:    phh
Date:      2010-02-26 16:40 -0500
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/8911d8c0596f

6923123: Hotspot refuses to start when -Xmx4m or -Xms4m is specified
Summary: Reduce NewSize from 4m to 1m.
Reviewed-by: tonyp, jmasa

! src/share/vm/runtime/globals.hpp

Changeset: c76ca382971b
Author:    johnc
Date:      2010-03-02 13:59 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/c76ca382971b

Merge

! src/share/vm/runtime/globals.hpp


From jon.masamitsu at sun.com  Wed Mar  3 10:11:52 2010
From: jon.masamitsu at sun.com (jon.masamitsu at sun.com)
Date: Wed, 03 Mar 2010 18:11:52 +0000
Subject: hg: jdk7/hotspot-gc/hotspot: 6910182: CMS: assert(_cursor[j] ==
	_survivor_plab_array[j].end(), "Ctl pt invariant")
Message-ID: <20100303181156.B57CE43C58@hg.openjdk.java.net>

Changeset: d47555d7aca8
Author:    jmasa
Date:      2010-03-03 08:10 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/d47555d7aca8

6910182: CMS: assert(_cursor[j] == _survivor_plab_array[j].end(),"Ctl pt invariant")
Summary: Calculation of the slicing of survivor spaces for MT was incorrect.
Reviewed-by: ysr

! src/share/vm/gc_implementation/concurrentMarkSweep/concurrentMarkSweepGeneration.cpp


From libic at dsrg.mff.cuni.cz  Mon Mar  8 07:38:53 2010
From: libic at dsrg.mff.cuni.cz (Peter Libic)
Date: Mon, 08 Mar 2010 16:38:53 +0100
Subject: Counting allocations and object sizes
Message-ID: <4B951A0D.4060101@dsrg.mff.cuni.cz>

Hi,

I'd like to count allocations and object sizes in the VM. And I want it with
lowest possible overhead and perturbation - what leads me to instrumentation
of the VM. After studying sources I thought all allocation should be
intercepted by at least some of the functions mentioned at the bottom of this
mail.

unfortunately, these functions are not sufficient - as tested by following
example - I run the following code with lines tagged with V1 and V2 commented
out, and in both cases Iv got exactly the same counts. That means, the
allocations of TstCls objects are somewhere else and I don't know where :).

I'm running the class like this:
java -XX:+UseParallelGC -verbose:gc -Xint DifferentAllocs

Could someone please write me where should I look for a code that allocates
the objects?

Thanks a lot!

Peter Libic

PS: I'm not quite sure if this is the correct list to ask at, I'm sorry if
this is the case.

============================================
public class DifferentAllocs {
	public static void main(String[] args) {
		Object[] arr = new Object[1024];
		System.out.println("Starting test - ALLOCATE");
		test(arr);
		System.out.println("Finished test - ALLOCATE");
		System.out.println(arr[(new Random()).nextInt(1004)]);
	}
	
	public static boolean test(Object[] arr) {
		TstCls s, t, u;	int i = 0;
		s=new TstCls(); t=new TstCls();u=t; /*u=new TstCls();*/
		arr[0] = s; arr[1] = t; arr[2] = u;	
		for (i = 3; i < 1003; i++) {
	/*V1*/		arr[i] = new TstCls();
	/*V2*/		//arr[i] = u;
		}
		return s.equals(t) || t.equals(u);
	}

}

class TstCls {
	public int val1;
	public long val2;
	public long getV() {return val1+val2;}
	@Override public String toString() {return "123425345";}
}
============================================

Intercepted functions:

hotspot/src/share/vm/gc_interface/collectedHeap.inline.hpp:
CollectedHeap::post_allocation_setup_common
CollectedHeap::post_allocation_setup_no_klass_install
CollectedHeap::post_allocation_install_obj_klass
CollectedHeap::post_allocation_setup_obj
CollectedHeap::post_allocation_setup_array
CollectedHeap::common_mem_allocate_noinit
CollectedHeap::common_mem_allocate_init
CollectedHeap::common_permanent_mem_allocate_noinit
CollectedHeap::common_permanent_mem_allocate_init
CollectedHeap::allocate_from_tlab
CollectedHeap::init_obj
CollectedHeap::obj_allocate
CollectedHeap::array_allocate
CollectedHeap::large_typearray_allocate
CollectedHeap::permanent_obj_allocate
CollectedHeap::permanent_obj_allocate_no_klass_install
CollectedHeap::permanent_array_allocate

hotspot/src/share/vm/gc_interface/collectedHeap.cpp:
CollectedHeap::allocate_from_tlab_slow

hotspot/src/share/vm/gc_implementation/parallelScavenge/parallelScavengeHeap.cpp:
ParallelScavengeHeap::mem_allocate
ParallelScavengeHeap::failed_mem_allocate
ParallelScavengeHeap::permanent_mem_allocate
ParallelScavengeHeap::failed_permanent_mem_allocate
ParallelScavengeHeap::allocate_new_tlab
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From mbien at fh-landshut.de  Sat Mar  6 16:57:55 2010
From: mbien at fh-landshut.de (Michael Bien)
Date: Sun, 07 Mar 2010 01:57:55 +0100
Subject: performance impact of JNI GetCritical*
Message-ID: <4B92FA13.3070903@fh-landshut.de>

Hello,

I have a few performance/best practice related questions regarding 
heap<->C data transfers and GC implications.

How optimized are the Get/ReleasePrimitiveArrayCritical JNI functions? 
Do they disable GC or do they only pin the array at the specific address?

If they would disable the GC until release, i suppose this would have 
pretty severe impact esp for concurrent GCs.

Could you recommend best practices?
lets say I have a short java array of length 12. Should I copy it to a 
direct NIO buffer and use the buffer as vehicle, use Get/ReleaseCritical 
or something else?


best regards,

-- 
Michael Bien
http://michael-bien.com/

_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From john.coomes at sun.com  Tue Mar  9 11:14:48 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Tue, 09 Mar 2010 19:14:48 +0000
Subject: hg: jdk7/hotspot-gc: Added tag jdk7-b85 for changeset cf26288a114b
Message-ID: <20100309191449.03E6544497@hg.openjdk.java.net>

Changeset: 3ddf90b39176
Author:    mikejwre
Date:      2010-03-04 13:50 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/rev/3ddf90b39176

Added tag jdk7-b85 for changeset cf26288a114b

! .hgtags


From john.coomes at sun.com  Tue Mar  9 11:14:58 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Tue, 09 Mar 2010 19:14:58 +0000
Subject: hg: jdk7/hotspot-gc/corba: Added tag jdk7-b85 for changeset
	c67a9df7bc0c
Message-ID: <20100309191501.1233D44498@hg.openjdk.java.net>

Changeset: 6253e28826d1
Author:    mikejwre
Date:      2010-03-04 13:50 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/corba/rev/6253e28826d1

Added tag jdk7-b85 for changeset c67a9df7bc0c

! .hgtags


From john.coomes at sun.com  Tue Mar  9 11:16:02 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Tue, 09 Mar 2010 19:16:02 +0000
Subject: hg: jdk7/hotspot-gc/jaxp: Added tag jdk7-b85 for changeset
	6c0ccabb430d
Message-ID: <20100309191602.EE5B14449A@hg.openjdk.java.net>

Changeset: 81c0f115bbe5
Author:    mikejwre
Date:      2010-03-04 13:50 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxp/rev/81c0f115bbe5

Added tag jdk7-b85 for changeset 6c0ccabb430d

! .hgtags


From john.coomes at sun.com  Tue Mar  9 11:16:08 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Tue, 09 Mar 2010 19:16:08 +0000
Subject: hg: jdk7/hotspot-gc/jaxws: Added tag jdk7-b85 for changeset
	8424512588ff
Message-ID: <20100309191608.7086F4449B@hg.openjdk.java.net>

Changeset: 512b0e924a5a
Author:    mikejwre
Date:      2010-03-04 13:50 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/512b0e924a5a

Added tag jdk7-b85 for changeset 8424512588ff

! .hgtags


From john.coomes at sun.com  Tue Mar  9 11:17:00 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Tue, 09 Mar 2010 19:17:00 +0000
Subject: hg: jdk7/hotspot-gc/jdk: 13 new changesets
Message-ID: <20100309192136.0BFF94449D@hg.openjdk.java.net>

Changeset: 2ba381560071
Author:    dcherepanov
Date:      2010-02-12 19:58 +0300
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/2ba381560071

6705345: Enable multiple file selection in AWT FileDialog
Reviewed-by: art, anthony, alexp

! src/share/classes/java/awt/FileDialog.java
! src/share/classes/sun/awt/AWTAccessor.java
! src/solaris/classes/sun/awt/X11/XFileDialogPeer.java
! src/windows/classes/sun/awt/windows/WFileDialogPeer.java
! src/windows/native/sun/windows/awt_FileDialog.cpp
! src/windows/native/sun/windows/awt_FileDialog.h
+ test/java/awt/FileDialog/MultipleMode/MultipleMode.html
+ test/java/awt/FileDialog/MultipleMode/MultipleMode.java

Changeset: d6d2de6ee2d1
Author:    lana
Date:      2010-02-19 15:13 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/d6d2de6ee2d1

Merge


Changeset: 83c34a6b1458
Author:    mchung
Date:      2010-02-08 23:02 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/83c34a6b1458

6924497: HotSpotDiagnosticsMXBean.getDiagnosticOptions throws NPE
Summary: Check if the element in the flags array is non-null to filter unsupported flags
Reviewed-by: dcubed

! src/share/classes/sun/management/Flag.java
! src/share/native/sun/management/Flag.c

Changeset: ec438f2b6886
Author:    chegar
Date:      2010-02-10 13:23 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/ec438f2b6886

6693244: Java Web Start app fails on 6u10 beta w/ AssertionError in AuthenticationInfo.requestCompleted
Reviewed-by: michaelm

! src/share/classes/sun/net/www/protocol/http/AuthenticationInfo.java
! test/ProblemList.txt
! test/java/net/Authenticator/B4769350.java

Changeset: 784e52734b8d
Author:    mchung
Date:      2010-02-10 17:51 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/784e52734b8d

6915413: Module build: building of specified jdk components instead of all
Summary: Define new SUBDIRS_* variables for specifying components for one group
Reviewed-by: ohair

! make/Makefile
! make/com/Makefile
! make/com/sun/Makefile
! make/com/sun/demo/Makefile
! make/com/sun/demo/jvmti/Makefile
! make/com/sun/inputmethods/Makefile
! make/com/sun/java/Makefile
! make/com/sun/java/browser/Makefile
! make/com/sun/jmx/Makefile
! make/com/sun/jndi/Makefile
! make/com/sun/jndi/rmi/Makefile
! make/com/sun/nio/Makefile
! make/com/sun/org/Makefile
! make/com/sun/org/apache/Makefile
! make/com/sun/security/Makefile
! make/com/sun/tools/Makefile
! make/com/sun/tracing/Makefile
! make/common/Defs.gmk
! make/common/Sanity.gmk
+ make/common/Subdirs.gmk
! make/common/shared/Sanity.gmk
! make/java/Makefile
! make/java/hpi/Makefile
! make/java/java/Makefile
! make/java/java/genlocales.gmk
! make/java/main/Makefile
! make/java/nio/FILES_java.gmk
! make/java/nio/Makefile
+ make/java/nio/mxbean/Makefile
! make/java/redist/Makefile
- make/java/text/FILES_java.gmk
! make/java/text/Makefile
+ make/java/text/base/FILES_java.gmk
+ make/java/text/base/Makefile
+ make/java/text/bidi/Makefile
! make/javax/Makefile
! make/javax/rmi/Makefile
! make/javax/sound/Makefile
! make/javax/swing/Makefile
! make/jpda/Makefile
! make/jpda/transport/Makefile
! make/mkdemo/Makefile
! make/mkdemo/applets/Makefile
! make/mkdemo/jfc/Makefile
! make/mkdemo/jni/Makefile
! make/mkdemo/jvmti/Makefile
! make/mkdemo/management/Makefile
! make/mkdemo/scripting/Makefile
! make/mksample/Makefile
! make/mksample/jmx/Makefile
! make/mksample/nio/Makefile
! make/mksample/scripting/Makefile
! make/mksample/webservices/Makefile
! make/org/Makefile
! make/org/ietf/Makefile
! make/sun/Makefile
! make/sun/cmm/Makefile
! make/sun/image/Makefile
! make/sun/management/Makefile
! make/sun/net/Makefile
! make/sun/net/spi/Makefile
! make/sun/net/spi/nameservice/Makefile
! make/sun/nio/Makefile
! make/sun/org/Makefile
! make/sun/org/mozilla/Makefile
! make/sun/rmi/Makefile
! make/sun/security/Makefile
! make/sun/tracing/Makefile
! make/tools/Makefile

Changeset: d7d8807fca86
Author:    weijun
Date:      2010-02-12 10:24 +0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/d7d8807fca86

6925639: keytool -genkeypair -help missing dname option
Reviewed-by: mullan

! src/share/classes/sun/security/tools/KeyTool.java

Changeset: 74f493fae483
Author:    mchung
Date:      2010-02-12 11:33 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/74f493fae483

6925868: Eliminate pack200's dependency on logging
Summary: Replace j.u.l.Logger with sun.util.logging.PlatformLogger
Reviewed-by: alanb, forax

! src/share/classes/com/sun/java/util/jar/pack/PackageReader.java
! src/share/classes/com/sun/java/util/jar/pack/PackageWriter.java
! src/share/classes/com/sun/java/util/jar/pack/Utils.java

Changeset: 328c5d3974fe
Author:    mchung
Date:      2010-02-15 10:18 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/328c5d3974fe

Merge


Changeset: 84792500750c
Author:    lana
Date:      2010-02-17 10:24 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/84792500750c

Merge


Changeset: e83d9c0d5e95
Author:    chegar
Date:      2010-02-22 15:27 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/e83d9c0d5e95

6912868: "java.net.useSystemProxies" behavior fails to check "use_same_proxy" in GNOME
Reviewed-by: alanb, chegar
Contributed-by: damjan.jov at gmail.com

! src/solaris/native/sun/net/spi/DefaultProxySelector.c
+ test/java/net/ProxySelector/SystemProxies.java

Changeset: c96d6cb31723
Author:    chegar
Date:      2010-02-23 17:08 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/c96d6cb31723

6365587: Proxy-Connection header sent through tunnel
Reviewed-by: michaelm

! src/share/classes/sun/net/www/protocol/http/HttpURLConnection.java

Changeset: b396584a3e64
Author:    lana
Date:      2010-02-23 10:17 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/b396584a3e64

Merge

- make/java/text/FILES_java.gmk

Changeset: 03cd9e62961f
Author:    mikejwre
Date:      2010-03-04 13:50 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/03cd9e62961f

Added tag jdk7-b85 for changeset b396584a3e64

! .hgtags


From john.coomes at sun.com  Tue Mar  9 11:23:41 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Tue, 09 Mar 2010 19:23:41 +0000
Subject: hg: jdk7/hotspot-gc/langtools: 11 new changesets
Message-ID: <20100309192414.F3A364449E@hg.openjdk.java.net>

Changeset: 7d9e3a15d2b3
Author:    jjg
Date:      2010-02-15 16:09 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/7d9e3a15d2b3

6926555: 6921979 breaks TreePosTest
Reviewed-by: darcy

! test/tools/javac/treepostests/TreePosTest.java

Changeset: af18e3956985
Author:    darcy
Date:      2010-02-15 18:20 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/af18e3956985

6634138: Source generated in last round not compiled
Reviewed-by: jjg

! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java
! test/tools/javac/T6403466.java
+ test/tools/javac/processing/6634138/Dummy.java
+ test/tools/javac/processing/6634138/ExerciseDependency.java
+ test/tools/javac/processing/6634138/T6634138.java

Changeset: fe17a9dbef03
Author:    darcy
Date:      2010-02-15 20:06 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/fe17a9dbef03

6926699: Annotation processing regression tests should typically return SourceVersion.latest
Reviewed-by: jjg

! test/tools/javac/6341866/Anno.java
! test/tools/javac/T6406771.java
! test/tools/javac/T6411379.java
! test/tools/javac/T6423583.java
! test/tools/javac/T6855236.java
! test/tools/javac/api/6421111/T6421111.java
! test/tools/javac/api/6468404/T6468404.java
! test/tools/javac/api/T6412669.java
! test/tools/javac/enum/6424358/T6424358.java
! test/tools/javac/processing/6348499/A.java
! test/tools/javac/processing/6414633/A.java
! test/tools/javac/processing/6430209/T6430209.java
! test/tools/javac/processing/6430209/b6341534.java
! test/tools/javac/processing/T6439826.java
! test/tools/javac/processing/model/element/TypeParamBounds.java
! test/tools/javac/processing/model/type/MirroredTypeEx/OverEager.java
! test/tools/javac/processing/model/type/NoTypes.java
! test/tools/javac/processing/model/util/GetTypeElemBadArg.java
! test/tools/javac/processing/model/util/OverridesSpecEx.java

Changeset: 631a273dac0f
Author:    darcy
Date:      2010-02-15 20:17 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/631a273dac0f

6926703: apt tests should run with assertions enabled
Reviewed-by: jjg

! src/share/classes/com/sun/mirror/util/SourceOrderDeclScanner.java

Changeset: 16b9b7f45933
Author:    darcy
Date:      2010-02-17 14:30 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/16b9b7f45933

6927061: Refactor apt implemenation to use code from JSR 269
Reviewed-by: jjg

! src/share/classes/com/sun/mirror/util/SourceOrderDeclScanner.java
! src/share/classes/com/sun/tools/apt/comp/Apt.java
! src/share/classes/com/sun/tools/apt/main/Main.java
! src/share/classes/com/sun/tools/javac/file/Paths.java
! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java
! src/share/classes/com/sun/tools/javadoc/DocletInvoker.java

Changeset: 67f0e05098fa
Author:    lana
Date:      2010-02-17 10:25 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/67f0e05098fa

Merge


Changeset: 0fce6b64c258
Author:    lana
Date:      2010-02-17 16:29 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/0fce6b64c258

Merge


Changeset: a3be81d385ee
Author:    jjg
Date:      2010-02-18 15:41 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/a3be81d385ee

6927797: langtools/test/tools/javac/EarlyAssert.java fails when run with assertions enabled (-ea)
Reviewed-by: darcy

! test/tools/javac/EarlyAssert.java
+ test/tools/javac/EarlyAssertWrapper.java

Changeset: f25efdb55c99
Author:    andrew
Date:      2010-02-22 21:37 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/f25efdb55c99

6928623: Behaviour of VERBOSE=true on langtools build
Summary: VERBOSE=true causes -diagnostics to be passed to ant rather than -debug
Reviewed-by: jjg

! make/Makefile

Changeset: 136bfc679462
Author:    lana
Date:      2010-02-23 10:17 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/136bfc679462

Merge


Changeset: b816baf594e3
Author:    mikejwre
Date:      2010-03-04 13:50 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/b816baf594e3

Added tag jdk7-b85 for changeset 136bfc679462

! .hgtags


From yamauchi at google.com  Wed Mar 10 15:17:25 2010
From: yamauchi at google.com (Hiroshi Yamauchi)
Date: Wed, 10 Mar 2010 15:17:25 -0800
Subject: Free ratio based heap shrinking in the parallel collector
Message-ID: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>

Hi,

I'd like to have this patch

  http://cr.openjdk.java.net/~rasbold/69XXXXX/webrev.00/

contributed, if appropriate.

It implements a simple heap shrinking logic based on the free ratio
(MaxHeapFreeRatio) in the parallel collector. The way it works is the
following: If the free ratio, (1.0 - <used> / <capacity>) * 100 >
MaxHeapFreeRatio, this logic kicks in. Otherwise, the (default) adaptive
size policy runs the show (as before). This feature is turned off by
default. There are two new flags to turn it on:

  UseFreeRatioForParallelGC - this enables it.
  UseFreeRatioOnlyInSystemGCForParallelGC - this enables it only in explicit
System.gc() calls. This is more useful in apps that call System.gc() when
it's idle and don't want shrinking to happen at all other times.

In our tests, it appears to work well in a simple test and to help a real
app reduce its footprint (RSS) when it's idle.

Thanks to Chuck Rasbold who uploaded the webrev on my behalf.

Thanks,
Hiroshi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20100310/3d945360/attachment.html 

From Jon.Masamitsu at Sun.COM  Wed Mar 10 16:04:13 2010
From: Jon.Masamitsu at Sun.COM (Jon Masamitsu)
Date: Wed, 10 Mar 2010 16:04:13 -0800
Subject: Free ratio based heap shrinking in the parallel collector
In-Reply-To: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
References: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
Message-ID: <4B98337D.3010500@Sun.COM>

Hiroshi,

Have you signed a contributor agreement?  The gentleman
who used to take care of those isn't in the group any longer
and I'll have to go hunt it down before I look at the
code.

I'm going to be out of the office doing a
presentation tomorrow so won't be able to pursue
this until Friday at the earliest.

But just in general when UseAdativeSizePolicy is
off the UseParallelGC collector doesn't do any resizing.
Can you just make this the sizing policy when
UseAdativeSizePolicy is off (instead of adding the
new flags)?

Jon


On 03/10/10 15:17, Hiroshi Yamauchi wrote:
> Hi,
> 
> I'd like to have this patch
> 
>   http://cr.openjdk.java.net/~rasbold/69XXXXX/webrev.00/
> 
> contributed, if appropriate.
> 
> It implements a simple heap shrinking logic based on the free ratio 
> (MaxHeapFreeRatio) in the parallel collector. The way it works is the 
> following: If the free ratio, (1.0 - <used> / <capacity>) * 100 > 
> MaxHeapFreeRatio, this logic kicks in. Otherwise, the (default) adaptive 
> size policy runs the show (as before). This feature is turned off by 
> default. There are two new flags to turn it on:
> 
>   UseFreeRatioForParallelGC - this enables it.
>   UseFreeRatioOnlyInSystemGCForParallelGC - this enables it only in 
> explicit System.gc() calls. This is more useful in apps that call 
> System.gc() when it's idle and don't want shrinking to happen at all 
> other times.
> 
> In our tests, it appears to work well in a simple test and to help a 
> real app reduce its footprint (RSS) when it's idle.
> 
> Thanks to Chuck Rasbold who uploaded the webrev on my behalf.
> 
> Thanks,
> Hiroshi
> 

From Jon.Masamitsu at Sun.COM  Wed Mar 10 23:10:39 2010
From: Jon.Masamitsu at Sun.COM (Jon Masamitsu)
Date: Wed, 10 Mar 2010 23:10:39 -0800
Subject: Free ratio based heap shrinking in the parallel collector
In-Reply-To: <4B98337D.3010500@Sun.COM>
References: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
	<4B98337D.3010500@Sun.COM>
Message-ID: <391801F3-61C7-4BF6-9853-B292751E6B81@sun.com>


On Mar 10, 2010, at 4:04 PM, Jon Masamitsu wrote:

> Hiroshi,
>
> Have you signed a contributor agreement?  The gentleman
> who used to take care of those isn't in the group any longer
> and I'll have to go hunt it down before I look at the
> code.

I got confirmation that you're eligible to contribute.

>
> I'm going to be out of the office doing a
> presentation tomorrow so won't be able to pursue
> this until Friday at the earliest.
>
> But just in general when UseAdativeSizePolicy is
> off the UseParallelGC collector doesn't do any resizing.
> Can you just make this the sizing policy when
> UseAdativeSizePolicy is off (instead of adding the
> new flags)?

Sorry, I misread your mail.  I can see the reason for the
new flags.   I'll take a look at the  patch in the next
couple of days.

>
> Jon
>
>
> On 03/10/10 15:17, Hiroshi Yamauchi wrote:
>> Hi,
>> I'd like to have this patch
>>  http://cr.openjdk.java.net/~rasbold/69XXXXX/webrev.00/
>> contributed, if appropriate.
>> It implements a simple heap shrinking logic based on the free ratio  
>> (MaxHeapFreeRatio) in the parallel collector. The way it works is  
>> the following: If the free ratio, (1.0 - <used> / <capacity>) * 100  
>> > MaxHeapFreeRatio, this logic kicks in. Otherwise, the (default)  
>> adaptive size policy runs the show (as before). This feature is  
>> turned off by default. There are two new flags to turn it on:
>>  UseFreeRatioForParallelGC - this enables it.
>>  UseFreeRatioOnlyInSystemGCForParallelGC - this enables it only in  
>> explicit System.gc() calls. This is more useful in apps that call  
>> System.gc() when it's idle and don't want shrinking to happen at  
>> all other times.
>> In our tests, it appears to work well in a simple test and to help  
>> a real app reduce its footprint (RSS) when it's idle.
>> Thanks to Chuck Rasbold who uploaded the webrev on my behalf.
>> Thanks,
>> Hiroshi


From John.Coomes at sun.com  Thu Mar 11 13:12:44 2010
From: John.Coomes at sun.com (John Coomes)
Date: Thu, 11 Mar 2010 13:12:44 -0800
Subject: About Heap Profiling
In-Reply-To: <f5b533001003111105r5990c754q72170394a9095ac3@mail.gmail.com>
References: <f5b533001003111105r5990c754q72170394a9095ac3@mail.gmail.com>
Message-ID: <19353.23756.364376.795664@sun.com>

ercan canlier (ercan.canlier at gmail.com) wrote:
> Hi,
> 
> I am having problem when i try to run jmap heap profile command in openjdk.
> 
> The output of java -version is like below in my system.
> 
> [ercanlier at ercanlier ~]$ java -version
> java version "1.6.0_17"
> OpenJDK Runtime Environment (IcedTea6 1.7) (fedora-34.b17.fc12-i386)
> OpenJDK Server VM (build 14.0-b16, mixed mode)
> 
> After running jps on console i try to profile heap usage of my application
> but stucked with the problem Could not find symbol
> "gHotSpotVMTypeEntryTypeNameOffset"
> ...

This is sun/oracle bug 6932270, which was recently fixed.  Not sure
when it'll make it into an update.  See the following for more info:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6932270
https://bugzilla.redhat.com/show_bug.cgi?id=541548

You can download and install the Sun JDK to get a working version
right away; go to http://java.sun.com/javase/ and click on the
downloads tab.

-John

_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From john.coomes at sun.com  Thu Mar 11 13:55:36 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Thu, 11 Mar 2010 21:55:36 +0000
Subject: hg: jdk7/hotspot-gc/hotspot: 4396719: Mark Sweep stack overflow on
	deeply nested Object arrays
Message-ID: <20100311215546.25A0E44786@hg.openjdk.java.net>

Changeset: 2a1472c30599
Author:    jcoomes
Date:      2010-03-03 14:48 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/2a1472c30599

4396719: Mark Sweep stack overflow on deeply nested Object arrays
Summary: Use an explicit stack for object arrays and process them in chunks.
Reviewed-by: iveresov, apetrusenko

! src/share/vm/gc_implementation/g1/concurrentMark.hpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp
! src/share/vm/gc_implementation/g1/g1MarkSweep.cpp
! src/share/vm/gc_implementation/includeDB_gc_parallelScavenge
! src/share/vm/gc_implementation/parallelScavenge/pcTasks.cpp
! src/share/vm/gc_implementation/parallelScavenge/psCompactionManager.cpp
! src/share/vm/gc_implementation/parallelScavenge/psCompactionManager.hpp
+ src/share/vm/gc_implementation/parallelScavenge/psCompactionManager.inline.hpp
! src/share/vm/gc_implementation/parallelScavenge/psMarkSweep.cpp
! src/share/vm/gc_implementation/parallelScavenge/psParallelCompact.cpp
! src/share/vm/gc_implementation/parallelScavenge/psParallelCompact.hpp
! src/share/vm/gc_implementation/shared/markSweep.cpp
! src/share/vm/gc_implementation/shared/markSweep.hpp
! src/share/vm/gc_implementation/shared/markSweep.inline.hpp
! src/share/vm/includeDB_core
! src/share/vm/includeDB_gc_parallel
! src/share/vm/memory/genMarkSweep.cpp
! src/share/vm/memory/genOopClosures.hpp
! src/share/vm/oops/objArrayKlass.cpp
! src/share/vm/oops/objArrayKlass.hpp
+ src/share/vm/oops/objArrayKlass.inline.hpp
! src/share/vm/runtime/arguments.cpp
! src/share/vm/runtime/globals.hpp
! src/share/vm/utilities/globalDefinitions.hpp
! src/share/vm/utilities/taskqueue.cpp
! src/share/vm/utilities/taskqueue.hpp


From yamauchi at google.com  Thu Mar 11 15:09:15 2010
From: yamauchi at google.com (Hiroshi Yamauchi)
Date: Thu, 11 Mar 2010 15:09:15 -0800
Subject: Free ratio based heap shrinking in the parallel collector
In-Reply-To: <391801F3-61C7-4BF6-9853-B292751E6B81@sun.com>
References: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
	<4B98337D.3010500@Sun.COM>
	<391801F3-61C7-4BF6-9853-B292751E6B81@sun.com>
Message-ID: <d6262f011003111509t1b1cf847ua06ebc0eecefeba1@mail.gmail.com>

On Wed, Mar 10, 2010 at 11:10 PM, Jon Masamitsu <Jon.Masamitsu at sun.com>wrote:

>
> On Mar 10, 2010, at 4:04 PM, Jon Masamitsu wrote:
>
>  Hiroshi,
>>
>> Have you signed a contributor agreement?  The gentleman
>> who used to take care of those isn't in the group any longer
>> and I'll have to go hunt it down before I look at the
>> code.
>>
>
> I got confirmation that you're eligible to contribute.


Good.


>
>
>
>> I'm going to be out of the office doing a
>> presentation tomorrow so won't be able to pursue
>> this until Friday at the earliest.
>>
>> But just in general when UseAdativeSizePolicy is
>> off the UseParallelGC collector doesn't do any resizing.
>> Can you just make this the sizing policy when
>> UseAdativeSizePolicy is off (instead of adding the
>> new flags)?
>>
>
> Sorry, I misread your mail.  I can see the reason for the
> new flags.   I'll take a look at the  patch in the next
> couple of days.


No problem.

Thanks,
Hiroshi


>
>
>
>> Jon
>>
>>
>> On 03/10/10 15:17, Hiroshi Yamauchi wrote:
>>
>>> Hi,
>>> I'd like to have this patch
>>>  http://cr.openjdk.java.net/~rasbold/69XXXXX/webrev.00/<http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/>
>>> contributed, if appropriate.
>>> It implements a simple heap shrinking logic based on the free ratio
>>> (MaxHeapFreeRatio) in the parallel collector. The way it works is the
>>> following: If the free ratio, (1.0 - <used> / <capacity>) * 100 >
>>> MaxHeapFreeRatio, this logic kicks in. Otherwise, the (default) adaptive
>>> size policy runs the show (as before). This feature is turned off by
>>> default. There are two new flags to turn it on:
>>>  UseFreeRatioForParallelGC - this enables it.
>>>  UseFreeRatioOnlyInSystemGCForParallelGC - this enables it only in
>>> explicit System.gc() calls. This is more useful in apps that call
>>> System.gc() when it's idle and don't want shrinking to happen at all other
>>> times.
>>> In our tests, it appears to work well in a simple test and to help a real
>>> app reduce its footprint (RSS) when it's idle.
>>> Thanks to Chuck Rasbold who uploaded the webrev on my behalf.
>>> Thanks,
>>> Hiroshi
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20100311/877be771/attachment.html 

From ercan.canlier at gmail.com  Thu Mar 11 11:05:05 2010
From: ercan.canlier at gmail.com (ercan canlier)
Date: Thu, 11 Mar 2010 21:05:05 +0200
Subject: About Heap Profiling
Message-ID: <f5b533001003111105r5990c754q72170394a9095ac3@mail.gmail.com>

Hi,

I am having problem when i try to run jmap heap profile command in openjdk.

The output of java -version is like below in my system.

[ercanlier at ercanlier ~]$ java -version
java version "1.6.0_17"
OpenJDK Runtime Environment (IcedTea6 1.7) (fedora-34.b17.fc12-i386)
OpenJDK Server VM (build 14.0-b16, mixed mode)

After running jps on console i try to profile heap usage of my application
but stucked with the problem Could not find symbol
"gHotSpotVMTypeEntryTypeNameOffset"


all output of jmap is:

[ercanlier at ercanlier ~]$ jmap -heap 3774
Attaching to process ID 3774, please wait...
sun.jvm.hotspot.debugger.NoSuchSymbolException: Could not find symbol
"gHotSpotVMTypeEntryTypeNameOffset" in any of the known library names
(libjvm.so, libjvm_g.so, gamma_g)
        at
sun.jvm.hotspot.HotSpotTypeDataBase.lookupInProcess(HotSpotTypeDataBase.java:390)
        at
sun.jvm.hotspot.HotSpotTypeDataBase.getLongValueFromProcess(HotSpotTypeDataBase.java:371)
        at
sun.jvm.hotspot.HotSpotTypeDataBase.readVMTypes(HotSpotTypeDataBase.java:102)
        at
sun.jvm.hotspot.HotSpotTypeDataBase.<init>(HotSpotTypeDataBase.java:85)
        at
sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(BugSpotAgent.java:568)
        at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:494)
        at
sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:332)
        at sun.jvm.hotspot.tools.Tool.start(Tool.java:163)
        at sun.jvm.hotspot.tools.HeapSummary.main(HeapSummary.java:39)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at sun.tools.jmap.JMap.runTool(JMap.java:196)
        at sun.tools.jmap.JMap.main(JMap.java:128)
Debugger attached successfully.
sun.jvm.hotspot.tools.HeapSummary requires a java VM process/core!

My os is fedora 12, the server which hosts the application is Redhat, java
versions are both the same. I am in trouble with profiling the application
because of that with details... Thanks in advance. Regards.
-- 
ERCAN CANLIER
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20100311/b348584f/attachment.html 
-------------- next part --------------
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From Ryan.Highley at sabre-holdings.com  Fri Mar 12 08:52:43 2010
From: Ryan.Highley at sabre-holdings.com (Highley, Ryan)
Date: Fri, 12 Mar 2010 10:52:43 -0600
Subject: Discouraging CMS Due To Fragmentation
Message-ID: <32DB14FBADE66B439D6A98D2B01D27EC1297CAD9@sgtulmsp02.Global.ad.sabre.com>

Hello all,

 

My company has a cache-based application using a well-known Java memory
clustering framework for handling the usual failover and load balancing
concerns ensuring cache state is maintained when losing a node or two.

 

Naturally, application garbage collector choice has been explored during
conversations with the support staff for this framework.  The
framework's support staff has insisted any application using their
framework should use the ParallelOld collector due to issues regarding
using CMS and its inherent memory fragmentation.  The latest exchange is
below with my questions first and the response following.  The
framework's name has been removed.

 

Questions:  Are there specific outstanding reported Sun JVM CMS bugs
that are the basis for the requirement to use ParallelOld?  If so, what
are they, as other applications may also be subject to the same issue?
If not, what is so fundamentally different about <product>'s memory
usage and garbage collector interactions that makes CMS a bad choice?

 

>From <product>:   This is not <product> specific but in Java in general.
Sun itself recommends staying away from CMS if possible due to possible
fragmentation issues. In other words, with CMS you are just delaying the
problem and will possibly hit a very long pauseful collection at some
point.

 

In my now several years of reading Sun GC white papers, tuning guides,
articles, blog posts, messages threads (here and elsewhere), and the
like, I have never heard any assertion from Sun discouraging CMS use
altogether simply due to the memory fragmentation inherent to CMS'
design.  Everything I have seen, and everything we have implemented
successfully tuning several other applications, has shown with CMS
memory fragmentation is a concern to be managed through proper promotion
tuning, heap sizing and shaping, and setting a reasonable CMS initiating
occupancy fraction to be fairly certain a promotion failure is at best
extremely unlikely.

 

However that nagging possibility that I've missed something along the
way still exists.

 

Can I please get a definitive response from one (or more) of the Sun
garbage collection software engineers on this list in this matter?

 

Thank you for your attention,

 

Ryan

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20100312/10954662/attachment-0001.html 
-------------- next part --------------
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From Y.S.Ramakrishna at Sun.COM  Fri Mar 12 09:08:47 2010
From: Y.S.Ramakrishna at Sun.COM (Y. Srinivas Ramakrishna)
Date: Fri, 12 Mar 2010 09:08:47 -0800
Subject: Discouraging CMS Due To Fragmentation
In-Reply-To: <32DB14FBADE66B439D6A98D2B01D27EC1297CAD9@sgtulmsp02.Global.ad.sabre.com>
References: <32DB14FBADE66B439D6A98D2B01D27EC1297CAD9@sgtulmsp02.Global.ad.sabre.com>
Message-ID: <4B9A751F.6090501@sun.com>

I agree with your stance below; that fragmentation is a concern
that needs to be managed by means of tuning promotion rates so
as to promote only medium- and long-lived objects (which usually,
but not always, have a stationary or quasi-stationary object size and
lifetime distribution). Jon Masamitsu has a blog that describes this
in some detail, and Tony Printezis and Charlie Hunt have a JavaOne
talk in which they provide tips for such tuning, and for dealing
with possible long-term fragmentation.

See also CR 6631166 which fixes some fragmentation issues, which
certain customers have found quite effective in reducing fragmentation.

There are indeed many, many customers who use CMS successfully despite
theoretical concerns regarding fragmentation.

All that having been said, there is probably a class of applications.
especially those that store medium- and long-lived string objects
with which CMS has historically performed poorly. (I do not know
whether such customers have tried CMS post-6631166 however.)
The reason is that if the size and lifetime distributions of
medium-lived objects is long/fat-tailed or flat or, as usually happens
in some cases of programs involving long-lived strings, very
non-stationary, then the coalition and splitting heuristics used
by CMS turn out to be much less effective. You might want to try the
G1 garbage collector for such applications because G1 will not suffer
from such concerns related to fragmentation.

-- ramki

Highley, Ryan wrote:
> Hello all,
> 
>  
> 
> My company has a cache-based application using a well-known Java memory
> clustering framework for handling the usual failover and load balancing
> concerns ensuring cache state is maintained when losing a node or two.
> 
>  
> 
> Naturally, application garbage collector choice has been explored during
> conversations with the support staff for this framework.  The
> framework's support staff has insisted any application using their
> framework should use the ParallelOld collector due to issues regarding
> using CMS and its inherent memory fragmentation.  The latest exchange is
> below with my questions first and the response following.  The
> framework's name has been removed.
> 
>  
> 
> Questions:  Are there specific outstanding reported Sun JVM CMS bugs
> that are the basis for the requirement to use ParallelOld?  If so, what
> are they, as other applications may also be subject to the same issue?
> If not, what is so fundamentally different about <product>'s memory
> usage and garbage collector interactions that makes CMS a bad choice?
> 
>  
> 
>>From <product>:   This is not <product> specific but in Java in general.
> Sun itself recommends staying away from CMS if possible due to possible
> fragmentation issues. In other words, with CMS you are just delaying the
> problem and will possibly hit a very long pauseful collection at some
> point.
> 
>  
> 
> In my now several years of reading Sun GC white papers, tuning guides,
> articles, blog posts, messages threads (here and elsewhere), and the
> like, I have never heard any assertion from Sun discouraging CMS use
> altogether simply due to the memory fragmentation inherent to CMS'
> design.  Everything I have seen, and everything we have implemented
> successfully tuning several other applications, has shown with CMS
> memory fragmentation is a concern to be managed through proper promotion
> tuning, heap sizing and shaping, and setting a reasonable CMS initiating
> occupancy fraction to be fairly certain a promotion failure is at best
> extremely unlikely.
> 
>  
> 
> However that nagging possibility that I've missed something along the
> way still exists.
> 
>  
> 
> Can I please get a definitive response from one (or more) of the Sun
> garbage collection software engineers on this list in this matter?
> 
>  
> 
> Thank you for your attention,
> 
>  
> 
> Ryan
> 
>  
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From Ciciora at cboe.com  Fri Mar 12 13:33:57 2010
From: Ciciora at cboe.com (Ciciora, Paul)
Date: Fri, 12 Mar 2010 15:33:57 -0600
Subject: hotspot-gc-dev Digest, Vol 33, Issue 7
In-Reply-To: <mailman.35.1268424002.9434.hotspot-gc-dev@openjdk.java.net>
References: <mailman.35.1268424002.9434.hotspot-gc-dev@openjdk.java.net>
Message-ID: <8CDFE9CE1B9A344E9AA47C27B7C58DD801ADD6C5@MSMAIL.cboent.cboe.com>

I think an important factor with CMS and fragmentation (amongst the
several others) is how long the process runs. We here have combated the
issue that crops up due to extreme short term changes in object
allocation patterns. Some times successfully, some times not. The
fragmentation death spiral is a horrible thing to have to watch.
Fortunately we ended up being able to greatly expand the heap sizes
thanks to -d64 and UseCompressedOops. 

What we have never had to deal with is fragmentation that happens over
long periods of time. Several days of running between bounces for
example. I don't know what if anything you can do about that other than
force a compacting GC when it's safe.

Paul Ciciora 

-----Original Message-----
From: hotspot-gc-dev-bounces at openjdk.java.net
[mailto:hotspot-gc-dev-bounces at openjdk.java.net] On Behalf Of
hotspot-gc-dev-request at openjdk.java.net
Sent: Friday, March 12, 2010 2:00 PM
To: hotspot-gc-dev at openjdk.java.net
Subject: hotspot-gc-dev Digest, Vol 33, Issue 7

Send hotspot-gc-dev mailing list submissions to
	hotspot-gc-dev at openjdk.java.net

To subscribe or unsubscribe via the World Wide Web, visit
	http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-dev
or, via email, send a message with subject or body 'help' to
	hotspot-gc-dev-request at openjdk.java.net

You can reach the person managing the list at
	hotspot-gc-dev-owner at openjdk.java.net

When replying, please edit your Subject line so it is more specific
than "Re: Contents of hotspot-gc-dev digest..."


Today's Topics:

   1. Re: Discouraging CMS Due To Fragmentation
      (Y. Srinivas Ramakrishna)


----------------------------------------------------------------------

Message: 1
Date: Fri, 12 Mar 2010 09:08:47 -0800
From: "Y. Srinivas Ramakrishna" <Y.S.Ramakrishna at Sun.COM>
Subject: Re: Discouraging CMS Due To Fragmentation
To: "Highley, Ryan" <Ryan.Highley at sabre-holdings.com>
Cc: hotspot-gc-use at openjdk.java.net
Message-ID: <4B9A751F.6090501 at sun.com>
Content-Type: text/plain; charset="us-ascii"

I agree with your stance below; that fragmentation is a concern
that needs to be managed by means of tuning promotion rates so
as to promote only medium- and long-lived objects (which usually,
but not always, have a stationary or quasi-stationary object size and
lifetime distribution). Jon Masamitsu has a blog that describes this
in some detail, and Tony Printezis and Charlie Hunt have a JavaOne
talk in which they provide tips for such tuning, and for dealing
with possible long-term fragmentation.

See also CR 6631166 which fixes some fragmentation issues, which
certain customers have found quite effective in reducing fragmentation.

There are indeed many, many customers who use CMS successfully despite
theoretical concerns regarding fragmentation.

All that having been said, there is probably a class of applications.
especially those that store medium- and long-lived string objects
with which CMS has historically performed poorly. (I do not know
whether such customers have tried CMS post-6631166 however.)
The reason is that if the size and lifetime distributions of
medium-lived objects is long/fat-tailed or flat or, as usually happens
in some cases of programs involving long-lived strings, very
non-stationary, then the coalition and splitting heuristics used
by CMS turn out to be much less effective. You might want to try the
G1 garbage collector for such applications because G1 will not suffer
from such concerns related to fragmentation.

-- ramki

Highley, Ryan wrote:
> Hello all,
> 
>  
> 
> My company has a cache-based application using a well-known Java
memory
> clustering framework for handling the usual failover and load
balancing
> concerns ensuring cache state is maintained when losing a node or two.
> 
>  
> 
> Naturally, application garbage collector choice has been explored
during
> conversations with the support staff for this framework.  The
> framework's support staff has insisted any application using their
> framework should use the ParallelOld collector due to issues regarding
> using CMS and its inherent memory fragmentation.  The latest exchange
is
> below with my questions first and the response following.  The
> framework's name has been removed.
> 
>  
> 
> Questions:  Are there specific outstanding reported Sun JVM CMS bugs
> that are the basis for the requirement to use ParallelOld?  If so,
what
> are they, as other applications may also be subject to the same issue?
> If not, what is so fundamentally different about <product>'s memory
> usage and garbage collector interactions that makes CMS a bad choice?
> 
>  
> 
>>From <product>:   This is not <product> specific but in Java in
general.
> Sun itself recommends staying away from CMS if possible due to
possible
> fragmentation issues. In other words, with CMS you are just delaying
the
> problem and will possibly hit a very long pauseful collection at some
> point.
> 
>  
> 
> In my now several years of reading Sun GC white papers, tuning guides,
> articles, blog posts, messages threads (here and elsewhere), and the
> like, I have never heard any assertion from Sun discouraging CMS use
> altogether simply due to the memory fragmentation inherent to CMS'
> design.  Everything I have seen, and everything we have implemented
> successfully tuning several other applications, has shown with CMS
> memory fragmentation is a concern to be managed through proper
promotion
> tuning, heap sizing and shaping, and setting a reasonable CMS
initiating
> occupancy fraction to be fairly certain a promotion failure is at best
> extremely unlikely.
> 
>  
> 
> However that nagging possibility that I've missed something along the
> way still exists.
> 
>  
> 
> Can I please get a definitive response from one (or more) of the Sun
> garbage collection software engineers on this list in this matter?
> 
>  
> 
> Thank you for your attention,
> 
>  
> 
> Ryan
> 
>  
> 
> 
> 
> 
>
------------------------------------------------------------------------
> 
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use


End of hotspot-gc-dev Digest, Vol 33, Issue 7
*********************************************

From andrey.petrusenko at sun.com  Mon Mar 15 06:00:17 2010
From: andrey.petrusenko at sun.com (andrey.petrusenko at sun.com)
Date: Mon, 15 Mar 2010 13:00:17 +0000
Subject: hg: jdk7/hotspot-gc/hotspot: 4 new changesets
Message-ID: <20100315130033.3DDA544C58@hg.openjdk.java.net>

Changeset: c8a467bf56ad
Author:    coleenp
Date:      2010-03-02 12:09 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/c8a467bf56ad

6914050: jvm assertion "guard pages must be in use" in -Xcomp mode
Summary: Move creating stack guard pages in jni attach thread before potential java call rather than after. Also cleanup stack guard pages when jni attach fails
Reviewed-by: never, dholmes

! src/share/vm/prims/jni.cpp
! src/share/vm/runtime/thread.cpp

Changeset: 4b0f2f4918ed
Author:    xlu
Date:      2010-03-10 21:42 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/4b0f2f4918ed

6933402: RFE: Improve PrintSafepointStatistics output to track cleanup time
Summary: Improve the usability of safepoint statistics data. See bug evaluation for more details.
Reviewed-by: ysr, dholmes

! src/share/vm/runtime/safepoint.cpp
! src/share/vm/runtime/safepoint.hpp

Changeset: 12d91eb0f579
Author:    acorn
Date:      2010-03-11 14:41 -0500
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/12d91eb0f579

Merge


Changeset: 94946bdf36bd
Author:    apetrusenko
Date:      2010-03-15 02:56 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/94946bdf36bd

Merge



From Jon.Masamitsu at Sun.COM  Mon Mar 15 12:03:53 2010
From: Jon.Masamitsu at Sun.COM (Jon Masamitsu)
Date: Mon, 15 Mar 2010 12:03:53 -0700
Subject: Free ratio based heap shrinking in the parallel collector
In-Reply-To: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
References: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
Message-ID: <4B9E8499.8030904@sun.com>

Hiroshi,

I'm looking at the code in psParallelCompact.cpp

2076     bool free_ratio_in_effect = false;
2077     if ((UseFreeRatioForParallelGC ||
2078          (UseFreeRatioOnlyInSystemGCForParallelGC &&
2079           gc_cause == GCCause::_java_lang_system_gc))) {
2080       ParallelScavengeHeap* heap = (ParallelScavengeHeap*)Universe::heap();
2081       free_ratio_in_effect = heap->try_to_shrink_by_free_ratio(true);
2082     }
2083 
2084     if (!free_ratio_in_effect && UseAdaptiveSizePolicy) {

If only UseFreeRatioForParallelGC is set to true, is it correct
that the heap

  a) will grow according to the original UseAdaptiveSizePolicy
policy until MaxHeapFreeRatio is exceeded (i.e., until
try_to_shrink_by_free_ratio() returns true)
  b) will then shrink until MaxHeapFreeRatio is no longer exceeded
(likely just that once)

then start with a) again?

What type of application fits with such a combination of policies?
Do you have a plots of how the heap sizing behaves (i.e., size of the
generations vs. collections/time)?

Jon

Hiroshi Yamauchi wrote On 03/10/10 15:17,:

> Hi,
>
> I'd like to have this patch
>
>   http://cr.openjdk.java.net/~rasbold/69XXXXX/webrev.00/
> <http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/>
>
> contributed, if appropriate.
>
> It implements a simple heap shrinking logic based on the free ratio
> (MaxHeapFreeRatio) in the parallel collector. The way it works is the
> following: If the free ratio, (1.0 - <used> / <capacity>) * 100 >
> MaxHeapFreeRatio, this logic kicks in. Otherwise, the (default)
> adaptive size policy runs the show (as before). This feature is turned
> off by default. There are two new flags to turn it on:
>
>   UseFreeRatioForParallelGC - this enables it.
>   UseFreeRatioOnlyInSystemGCForParallelGC - this enables it only in
> explicit System.gc() calls. This is more useful in apps that call
> System.gc() when it's idle and don't want shrinking to happen at all
> other times.
>
> In our tests, it appears to work well in a simple test and to help a
> real app reduce its footprint (RSS) when it's idle.
>
> Thanks to Chuck Rasbold who uploaded the webrev on my behalf.
>
> Thanks,
> Hiroshi
>


From yamauchi at google.com  Mon Mar 15 16:50:25 2010
From: yamauchi at google.com (Hiroshi Yamauchi)
Date: Mon, 15 Mar 2010 16:50:25 -0700
Subject: Free ratio based heap shrinking in the parallel collector
In-Reply-To: <4B9E8499.8030904@sun.com>
References: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
	<4B9E8499.8030904@sun.com>
Message-ID: <d6262f011003151650v7d58e699gd868cae490f2a6f2@mail.gmail.com>

On Mon, Mar 15, 2010 at 12:03 PM, Jon Masamitsu <Jon.Masamitsu at sun.com>wrote:

> Hiroshi,
>
> I'm looking at the code in psParallelCompact.cpp
>
> 2076     bool free_ratio_in_effect = false;
> 2077     if ((UseFreeRatioForParallelGC ||
> 2078          (UseFreeRatioOnlyInSystemGCForParallelGC &&
> 2079           gc_cause == GCCause::_java_lang_system_gc))) {
> 2080       ParallelScavengeHeap* heap =
> (ParallelScavengeHeap*)Universe::heap();
> 2081       free_ratio_in_effect = heap->try_to_shrink_by_free_ratio(true);
> 2082     }
> 2083
> 2084     if (!free_ratio_in_effect && UseAdaptiveSizePolicy) {
>
> If only UseFreeRatioForParallelGC is set to true, is it correct
> that the heap
>
>  a) will grow according to the original UseAdaptiveSizePolicy
> policy until MaxHeapFreeRatio is exceeded (i.e., until
> try_to_shrink_by_free_ratio() returns true)
>  b) will then shrink until MaxHeapFreeRatio is no longer exceeded
> (likely just that once)
>
> then start with a) again?
>

I think that mostly matches my intention. The cyclical pattern
(a->b->a->b->...) is assumed not to happen more often than necessary though.
An expectation is that the value of MaxHeapFreeRatio is high enough (70 by
default) that the b) only happens when the app is under reasonably light(er)
load (or just idle.) UseFreeRatioOnlyInSystemGCForParallelGC (instead of
UseFreeRatioForParallelGC ) is for apps that call System.gc() to give a hint
about the need to have its heap shrunk and want to avoid heap shrinking
otherwise.

An intuition is the following. To reduce the memory footprint, the free
ratio logic holds back the heap expansion if the original adaptive size
policy expands too aggressively, or overrides and shrink the heap if the
original adaptive size policy leaves too much free space, in terms of the
value of MaxHeapFreeRatio. I think it's not far from what flag
MaxHeapFreeRatio originally meant. Otherwise, the heap size depends on the
original adaptive size policy (for performance.) It's a tradeoff between
performance and footprint.


> What type of application fits with such a combination of policies?
> Do you have a plots of how the heap sizing behaves (i.e., size of the
> generations vs. collections/time)?
>

The reason to combine the two logics (size policies) isn't for particular
applications. I don't get too surprised if you say it should be a separate
size policy and leave the original adaptive size policy alone. My thinking
is that users may want the same level of performance from the parallel
collector (the adaptive size policy is enabled by default) unless a large
part of the heap (> 70% by default) is free. This is my attempt to get the
best of both. The flags are off by default since of course, not all
applications need to have their heap shrunk this way.

No, I don't have plots.

Hiroshi



> Jon
>
> Hiroshi Yamauchi wrote On 03/10/10 15:17,:
>
> > Hi,
> >
> > I'd like to have this patch
> >
> >   http://cr.openjdk.java.net/~rasbold/69XXXXX/webrev.00/<http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/>
> > <http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/>
> >
> > contributed, if appropriate.
> >
> > It implements a simple heap shrinking logic based on the free ratio
> > (MaxHeapFreeRatio) in the parallel collector. The way it works is the
> > following: If the free ratio, (1.0 - <used> / <capacity>) * 100 >
> > MaxHeapFreeRatio, this logic kicks in. Otherwise, the (default)
> > adaptive size policy runs the show (as before). This feature is turned
> > off by default. There are two new flags to turn it on:
> >
> >   UseFreeRatioForParallelGC - this enables it.
> >   UseFreeRatioOnlyInSystemGCForParallelGC - this enables it only in
> > explicit System.gc() calls. This is more useful in apps that call
> > System.gc() when it's idle and don't want shrinking to happen at all
> > other times.
> >
> > In our tests, it appears to work well in a simple test and to help a
> > real app reduce its footprint (RSS) when it's idle.
> >
> > Thanks to Chuck Rasbold who uploaded the webrev on my behalf.
> >
> > Thanks,
> > Hiroshi
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20100315/7afe13d2/attachment.html 

From Y.S.Ramakrishna at Sun.COM  Mon Mar 15 18:02:01 2010
From: Y.S.Ramakrishna at Sun.COM (Y. Srinivas Ramakrishna)
Date: Mon, 15 Mar 2010 18:02:01 -0700
Subject: hotspot-gc-dev Digest, Vol 33, Issue 7
In-Reply-To: <8CDFE9CE1B9A344E9AA47C27B7C58DD801ADD6C5@MSMAIL.cboent.cboe.com>
References: <mailman.35.1268424002.9434.hotspot-gc-dev@openjdk.java.net>
	<8CDFE9CE1B9A344E9AA47C27B7C58DD801ADD6C5@MSMAIL.cboent.cboe.com>
Message-ID: <4B9ED889.6020602@Sun.COM>

Hi Paul --

On 03/12/10 13:33, Ciciora, Paul wrote:

> ... The
> fragmentation death spiral is a horrible thing to have to watch.

I'd like to see if the death-spirals you've witnessed in the past
become less steep (or less of spirals ;-) with the fixes of 6631166;
as well as perhaps the likelihood of the long-term creep that some
folks (especially in the telecoms sector) see and which you describe below ...

> What we have never had to deal with is fragmentation that happens over
> long periods of time. Several days of running between bounces for
> example. ...

thanks.
-- ramki

From Ryan.Highley at sabre-holdings.com  Tue Mar 16 09:58:20 2010
From: Ryan.Highley at sabre-holdings.com (Highley, Ryan)
Date: Tue, 16 Mar 2010 11:58:20 -0500
Subject: Discouraging CMS Due To Fragmentation
In-Reply-To: <4B9A751F.6090501@sun.com>
References: <32DB14FBADE66B439D6A98D2B01D27EC1297CAD9@sgtulmsp02.Global.ad.sabre.com>
	<4B9A751F.6090501@sun.com>
Message-ID: <32DB14FBADE66B439D6A98D2B01D27EC12A90E37@sgtulmsp02.Global.ad.sabre.com>

Ramki,

Thank you for your prompt response and suggestions.

As was mentioned in the Dev mailing list thread, the app instances we
tend to deal with typically remain up for weeks, if not months, at a
time.  For most, the allocation patterns are fairly constant and do not
require "forcing" a compacting collector to run periodically.  However,
there are a few requiring exactly that.

We will definitely be giving G1 a try, but won't be able to deploy it to
production until it's in a GA release.  (The PTBs get nervous when the
deployment documentation describes compiling a JVM. ;) )  CR 6631166
also looks extremely promising for many of our applications.

Ryan



-----Original Message-----
From: Y.S.Ramakrishna at Sun.COM [mailto:Y.S.Ramakrishna at Sun.COM] 
Sent: Friday, March 12, 2010 11:09 AM
To: Highley, Ryan
Cc: hotspot-gc-use at openjdk.java.net
Subject: Re: Discouraging CMS Due To Fragmentation

I agree with your stance below; that fragmentation is a concern
that needs to be managed by means of tuning promotion rates so
as to promote only medium- and long-lived objects (which usually,
but not always, have a stationary or quasi-stationary object size and
lifetime distribution). Jon Masamitsu has a blog that describes this
in some detail, and Tony Printezis and Charlie Hunt have a JavaOne
talk in which they provide tips for such tuning, and for dealing
with possible long-term fragmentation.

See also CR 6631166 which fixes some fragmentation issues, which
certain customers have found quite effective in reducing fragmentation.

There are indeed many, many customers who use CMS successfully despite
theoretical concerns regarding fragmentation.

All that having been said, there is probably a class of applications.
especially those that store medium- and long-lived string objects
with which CMS has historically performed poorly. (I do not know
whether such customers have tried CMS post-6631166 however.)
The reason is that if the size and lifetime distributions of
medium-lived objects is long/fat-tailed or flat or, as usually happens
in some cases of programs involving long-lived strings, very
non-stationary, then the coalition and splitting heuristics used
by CMS turn out to be much less effective. You might want to try the
G1 garbage collector for such applications because G1 will not suffer
from such concerns related to fragmentation.

-- ramki

Highley, Ryan wrote:
> Hello all,
> 
>  
> 
> My company has a cache-based application using a well-known Java
memory
> clustering framework for handling the usual failover and load
balancing
> concerns ensuring cache state is maintained when losing a node or two.
> 
>  
> 
> Naturally, application garbage collector choice has been explored
during
> conversations with the support staff for this framework.  The
> framework's support staff has insisted any application using their
> framework should use the ParallelOld collector due to issues regarding
> using CMS and its inherent memory fragmentation.  The latest exchange
is
> below with my questions first and the response following.  The
> framework's name has been removed.
> 
>  
> 
> Questions:  Are there specific outstanding reported Sun JVM CMS bugs
> that are the basis for the requirement to use ParallelOld?  If so,
what
> are they, as other applications may also be subject to the same issue?
> If not, what is so fundamentally different about <product>'s memory
> usage and garbage collector interactions that makes CMS a bad choice?
> 
>  
> 
>>From <product>:   This is not <product> specific but in Java in
general.
> Sun itself recommends staying away from CMS if possible due to
possible
> fragmentation issues. In other words, with CMS you are just delaying
the
> problem and will possibly hit a very long pauseful collection at some
> point.
> 
>  
> 
> In my now several years of reading Sun GC white papers, tuning guides,
> articles, blog posts, messages threads (here and elsewhere), and the
> like, I have never heard any assertion from Sun discouraging CMS use
> altogether simply due to the memory fragmentation inherent to CMS'
> design.  Everything I have seen, and everything we have implemented
> successfully tuning several other applications, has shown with CMS
> memory fragmentation is a concern to be managed through proper
promotion
> tuning, heap sizing and shaping, and setting a reasonable CMS
initiating
> occupancy fraction to be fairly certain a promotion failure is at best
> extremely unlikely.
> 
>  
> 
> However that nagging possibility that I've missed something along the
> way still exists.
> 
>  
> 
> Can I please get a definitive response from one (or more) of the Sun
> garbage collection software engineers on this list in this matter?
> 
>  
> 
> Thank you for your attention,
> 
>  
> 
> Ryan
> 
>  
> 
> 
> 
> 
>
------------------------------------------------------------------------
> 
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From Jon.Masamitsu at Sun.COM  Tue Mar 16 15:38:50 2010
From: Jon.Masamitsu at Sun.COM (Jon Masamitsu)
Date: Tue, 16 Mar 2010 15:38:50 -0700
Subject: Free ratio based heap shrinking in the parallel collector
In-Reply-To: <d6262f011003151650v7d58e699gd868cae490f2a6f2@mail.gmail.com>
References: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
	<4B9E8499.8030904@sun.com>
	<d6262f011003151650v7d58e699gd868cae490f2a6f2@mail.gmail.com>
Message-ID: <4BA0087A.2040006@Sun.COM>

Hiroshi,

The goal of UseAdaptiveSizePolicy with regard to performance is
to size the generations so as to achieve the throughput goal.
GC cost is reduced by reducing the frequency of collections
and that is achieved by more free space in the generations
(more space for more allocations before the next GC).

I can see how a user might want to limit the footprint.  Without
your change I would tell the user to reduce the throughput goal.
Not a particularly friendly interface to limit footprint I'll
admit.

There is a method

PSAdaptiveSizePolicy::adjust_for_throughput()

which is called to calculate the amount that the heap
should grow in order to achieve a throughput goal.
If that method said not to increase the heap based
on the value of MaxHeapFreeRatio is that close to
what you want?  It's not the same but it would be
a way of limiting the growth of the heap (stops
growing the heap when the free space after the collection
exceeds a threshold).  I think it would fit in more
easily with the rest of UseAdativeSizePolicy.  I think
it would be easier to explain too.

Jon

On 03/15/10 16:50, Hiroshi Yamauchi wrote:
> 
> 
> On Mon, Mar 15, 2010 at 12:03 PM, Jon Masamitsu <Jon.Masamitsu at sun.com 
> <mailto:Jon.Masamitsu at sun.com>> wrote:
> 
>     Hiroshi,
> 
>     I'm looking at the code in psParallelCompact.cpp
> 
>     2076     bool free_ratio_in_effect = false;
>     2077     if ((UseFreeRatioForParallelGC ||
>     2078          (UseFreeRatioOnlyInSystemGCForParallelGC &&
>     2079           gc_cause == GCCause::_java_lang_system_gc))) {
>     2080       ParallelScavengeHeap* heap =
>     (ParallelScavengeHeap*)Universe::heap();
>     2081       free_ratio_in_effect =
>     heap->try_to_shrink_by_free_ratio(true);
>     2082     }
>     2083
>     2084     if (!free_ratio_in_effect && UseAdaptiveSizePolicy) {
> 
>     If only UseFreeRatioForParallelGC is set to true, is it correct
>     that the heap
> 
>      a) will grow according to the original UseAdaptiveSizePolicy
>     policy until MaxHeapFreeRatio is exceeded (i.e., until
>     try_to_shrink_by_free_ratio() returns true)
>      b) will then shrink until MaxHeapFreeRatio is no longer exceeded
>     (likely just that once)
> 
>     then start with a) again?
> 
> 
> I think that mostly matches my intention. The cyclical pattern 
> (a->b->a->b->...) is assumed not to happen more often than necessary 
> though. An expectation is that the value of MaxHeapFreeRatio is high 
> enough (70 by default) that the b) only happens when the app is under 
> reasonably light(er) load (or just idle.) 
> UseFreeRatioOnlyInSystemGCForParallelGC (instead of 
> UseFreeRatioForParallelGC ) is for apps that call System.gc() to give a 
> hint about the need to have its heap shrunk and want to avoid heap 
> shrinking otherwise.
> 
> An intuition is the following. To reduce the memory footprint, the free 
> ratio logic holds back the heap expansion if the original adaptive size 
> policy expands too aggressively, or overrides and shrink the heap if the 
> original adaptive size policy leaves too much free space, in terms of 
> the value of MaxHeapFreeRatio. I think it's not far from what flag 
> MaxHeapFreeRatio originally meant. Otherwise, the heap size depends on 
> the original adaptive size policy (for performance.) It's a tradeoff 
> between performance and footprint.
> 
> 
>     What type of application fits with such a combination of policies?
>     Do you have a plots of how the heap sizing behaves (i.e., size of the
>     generations vs. collections/time)?
> 
> 
> The reason to combine the two logics (size policies) isn't for 
> particular applications. I don't get too surprised if you say it should 
> be a separate size policy and leave the original adaptive size policy 
> alone. My thinking is that users may want the same level of performance 
> from the parallel collector (the adaptive size policy is enabled by 
> default) unless a large part of the heap (> 70% by default) is free. 
> This is my attempt to get the best of both. The flags are off by default 
> since of course, not all applications need to have their heap shrunk 
> this way.
> 
> No, I don't have plots.
> 
> Hiroshi
> 
> 
> 
>     Jon
> 
>     Hiroshi Yamauchi wrote On 03/10/10 15:17,:
> 
>      > Hi,
>      >
>      > I'd like to have this patch
>      >
>      >   http://cr.openjdk.java.net/~rasbold/69XXXXX/webrev.00/
>     <http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/>
>      > <http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/>
>      >
>      > contributed, if appropriate.
>      >
>      > It implements a simple heap shrinking logic based on the free ratio
>      > (MaxHeapFreeRatio) in the parallel collector. The way it works is the
>      > following: If the free ratio, (1.0 - <used> / <capacity>) * 100 >
>      > MaxHeapFreeRatio, this logic kicks in. Otherwise, the (default)
>      > adaptive size policy runs the show (as before). This feature is
>     turned
>      > off by default. There are two new flags to turn it on:
>      >
>      >   UseFreeRatioForParallelGC - this enables it.
>      >   UseFreeRatioOnlyInSystemGCForParallelGC - this enables it only in
>      > explicit System.gc() calls. This is more useful in apps that call
>      > System.gc() when it's idle and don't want shrinking to happen at all
>      > other times.
>      >
>      > In our tests, it appears to work well in a simple test and to help a
>      > real app reduce its footprint (RSS) when it's idle.
>      >
>      > Thanks to Chuck Rasbold who uploaded the webrev on my behalf.
>      >
>      > Thanks,
>      > Hiroshi
>      >
> 
> 

From Jon.Masamitsu at Sun.COM  Tue Mar 16 20:04:09 2010
From: Jon.Masamitsu at Sun.COM (Jon Masamitsu)
Date: Tue, 16 Mar 2010 20:04:09 -0700
Subject: Free ratio based heap shrinking in the parallel collector
In-Reply-To: <4BA0087A.2040006@Sun.COM>
References: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
	<4B9E8499.8030904@sun.com>
	<d6262f011003151650v7d58e699gd868cae490f2a6f2@mail.gmail.com>
	<4BA0087A.2040006@Sun.COM>
Message-ID: <4BA046A9.6080103@sun.com>

Hiroshi,

I had forgotten the other half of this which was the behavior
on System.gc().   I agree that it would be nice to have a way
for the user to collapse the heap quickly if the user knows that
the application is going to be inactive.   The code you suggested
only shrinks the heap.  I think it would be more complete if it
would also grow the heap according to MinHeapFreeRatio.
I don't know the scenario where it would be used but I
wouldn't be surprised if there is one.  It would also make it
practical to run UseParallelGC with UseAdaptiveSizePolicy
turned off (the heap resizing would then be done similarly
to the other collectors).

Jon

Jon Masamitsu wrote On 03/16/10 15:38,:

>Hiroshi,
>
>The goal of UseAdaptiveSizePolicy with regard to performance is
>to size the generations so as to achieve the throughput goal.
>GC cost is reduced by reducing the frequency of collections
>and that is achieved by more free space in the generations
>(more space for more allocations before the next GC).
>
>I can see how a user might want to limit the footprint.  Without
>your change I would tell the user to reduce the throughput goal.
>Not a particularly friendly interface to limit footprint I'll
>admit.
>
>There is a method
>
>PSAdaptiveSizePolicy::adjust_for_throughput()
>
>which is called to calculate the amount that the heap
>should grow in order to achieve a throughput goal.
>If that method said not to increase the heap based
>on the value of MaxHeapFreeRatio is that close to
>what you want?  It's not the same but it would be
>a way of limiting the growth of the heap (stops
>growing the heap when the free space after the collection
>exceeds a threshold).  I think it would fit in more
>easily with the rest of UseAdativeSizePolicy.  I think
>it would be easier to explain too.
>
>Jon
>
>On 03/15/10 16:50, Hiroshi Yamauchi wrote:
>  
>
>>On Mon, Mar 15, 2010 at 12:03 PM, Jon Masamitsu <Jon.Masamitsu at sun.com 
>><mailto:Jon.Masamitsu at sun.com>> wrote:
>>
>>    Hiroshi,
>>
>>    I'm looking at the code in psParallelCompact.cpp
>>
>>    2076     bool free_ratio_in_effect = false;
>>    2077     if ((UseFreeRatioForParallelGC ||
>>    2078          (UseFreeRatioOnlyInSystemGCForParallelGC &&
>>    2079           gc_cause == GCCause::_java_lang_system_gc))) {
>>    2080       ParallelScavengeHeap* heap =
>>    (ParallelScavengeHeap*)Universe::heap();
>>    2081       free_ratio_in_effect =
>>    heap->try_to_shrink_by_free_ratio(true);
>>    2082     }
>>    2083
>>    2084     if (!free_ratio_in_effect && UseAdaptiveSizePolicy) {
>>
>>    If only UseFreeRatioForParallelGC is set to true, is it correct
>>    that the heap
>>
>>     a) will grow according to the original UseAdaptiveSizePolicy
>>    policy until MaxHeapFreeRatio is exceeded (i.e., until
>>    try_to_shrink_by_free_ratio() returns true)
>>     b) will then shrink until MaxHeapFreeRatio is no longer exceeded
>>    (likely just that once)
>>
>>    then start with a) again?
>>
>>
>>I think that mostly matches my intention. The cyclical pattern 
>>(a->b->a->b->...) is assumed not to happen more often than necessary 
>>though. An expectation is that the value of MaxHeapFreeRatio is high 
>>enough (70 by default) that the b) only happens when the app is under 
>>reasonably light(er) load (or just idle.) 
>>UseFreeRatioOnlyInSystemGCForParallelGC (instead of 
>>UseFreeRatioForParallelGC ) is for apps that call System.gc() to give a 
>>hint about the need to have its heap shrunk and want to avoid heap 
>>shrinking otherwise.
>>
>>An intuition is the following. To reduce the memory footprint, the free 
>>ratio logic holds back the heap expansion if the original adaptive size 
>>policy expands too aggressively, or overrides and shrink the heap if the 
>>original adaptive size policy leaves too much free space, in terms of 
>>the value of MaxHeapFreeRatio. I think it's not far from what flag 
>>MaxHeapFreeRatio originally meant. Otherwise, the heap size depends on 
>>the original adaptive size policy (for performance.) It's a tradeoff 
>>between performance and footprint.
>>
>>
>>    What type of application fits with such a combination of policies?
>>    Do you have a plots of how the heap sizing behaves (i.e., size of the
>>    generations vs. collections/time)?
>>
>>
>>The reason to combine the two logics (size policies) isn't for 
>>particular applications. I don't get too surprised if you say it should 
>>be a separate size policy and leave the original adaptive size policy 
>>alone. My thinking is that users may want the same level of performance 
>>from the parallel collector (the adaptive size policy is enabled by 
>>default) unless a large part of the heap (> 70% by default) is free. 
>>This is my attempt to get the best of both. The flags are off by default 
>>since of course, not all applications need to have their heap shrunk 
>>this way.
>>
>>No, I don't have plots.
>>
>>Hiroshi
>>
>>
>>
>>    Jon
>>
>>    Hiroshi Yamauchi wrote On 03/10/10 15:17,:
>>
>>     > Hi,
>>     >
>>     > I'd like to have this patch
>>     >
>>     >   http://cr.openjdk.java.net/~rasbold/69XXXXX/webrev.00/
>>    <http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/>
>>     > <http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/>
>>     >
>>     > contributed, if appropriate.
>>     >
>>     > It implements a simple heap shrinking logic based on the free ratio
>>     > (MaxHeapFreeRatio) in the parallel collector. The way it works is the
>>     > following: If the free ratio, (1.0 - <used> / <capacity>) * 100 >
>>     > MaxHeapFreeRatio, this logic kicks in. Otherwise, the (default)
>>     > adaptive size policy runs the show (as before). This feature is
>>    turned
>>     > off by default. There are two new flags to turn it on:
>>     >
>>     >   UseFreeRatioForParallelGC - this enables it.
>>     >   UseFreeRatioOnlyInSystemGCForParallelGC - this enables it only in
>>     > explicit System.gc() calls. This is more useful in apps that call
>>     > System.gc() when it's idle and don't want shrinking to happen at all
>>     > other times.
>>     >
>>     > In our tests, it appears to work well in a simple test and to help a
>>     > real app reduce its footprint (RSS) when it's idle.
>>     >
>>     > Thanks to Chuck Rasbold who uploaded the webrev on my behalf.
>>     >
>>     > Thanks,
>>     > Hiroshi
>>     >
>>
>>
>>    
>>


From John.Coomes at sun.com  Wed Mar 17 16:42:55 2010
From: John.Coomes at sun.com (John Coomes)
Date: Wed, 17 Mar 2010 16:42:55 -0700
Subject: review request (XS) 6935839: excessive marking stack growth
Message-ID: <19361.26879.54327.926759@sun.com>

Hi all,

I'd appreciate reviews for this simple fix (6 lines in 2 files).  The
bug was introduced by my recent fix for 4396719--when processing large
object arrays, marking stacks would grow too large.  Details are in
the webrev:

	http://cr.openjdk.java.net/~jcoomes/6935839-objarray/

-John


From john.cuthbertson at sun.com  Thu Mar 18 00:31:58 2010
From: john.cuthbertson at sun.com (john.cuthbertson at sun.com)
Date: Thu, 18 Mar 2010 07:31:58 +0000
Subject: hg: jdk7/hotspot-gc/hotspot: 6755988: G1: assert(new_obj != 0 || ...
	"should be forwarded")
Message-ID: <20100318073220.98E074405A@hg.openjdk.java.net>

Changeset: 664ae0c5e0e5
Author:    johnc
Date:      2010-03-11 11:44 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/664ae0c5e0e5

6755988: G1: assert(new_obj != 0 || ... "should be forwarded")
Summary: A TLAB became large enough to be considered a humongous object allowing multiple objects to be allocated in a humongous region, which violates a basic assumption about humongous regions. The changes ensure that TLABs cannot be regarded as humongous.
Reviewed-by: iveresov, tonyp

! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.hpp


From linuxhippy at gmail.com  Thu Mar 18 10:20:20 2010
From: linuxhippy at gmail.com (Clemens Eisserer)
Date: Thu, 18 Mar 2010 18:20:20 +0100
Subject: performance impact of JNI GetCritical*
In-Reply-To: <4B92FA13.3070903@fh-landshut.de>
References: <4B92FA13.3070903@fh-landshut.de>
Message-ID: <194f62551003181020n30b122ccjc092aeecd8e32a26@mail.gmail.com>

Hi Michael,

> How optimized are the Get/ReleasePrimitiveArrayCritical JNI functions?
> Do they disable GC or do they only pin the array at the specific address?

For moving gc's they actually stop the whole GC process as long as all
arrays are released by ReleasePrimitiveArrayCritical.
So basically the cost is some thread-synchronization (to make the gc
know on all threads not to run now), plus the impact on the GC. So for
small arrays theres quite high overhead.
CMS itself is non-moving (old gen) so basically there could be some
optimizations, but no idea wether this is actually done.

> lets say I have a short java array of length 12. Should I copy it to a
> direct NIO buffer and use the buffer as vehicle, use Get/ReleaseCritical
> or something else?
For very short arrays this is quite likely beneficial (at least if
you're on JDK7, at least older JDK6 releases do get/releasecritical +
memcpy in jni code).

Just to be curious, whats your actual usecase?

- Clemens
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From yamauchi at google.com  Thu Mar 18 11:27:24 2010
From: yamauchi at google.com (Hiroshi Yamauchi)
Date: Thu, 18 Mar 2010 11:27:24 -0700
Subject: Free ratio based heap shrinking in the parallel collector
In-Reply-To: <4BA0087A.2040006@Sun.COM>
References: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
	<4B9E8499.8030904@sun.com>
	<d6262f011003151650v7d58e699gd868cae490f2a6f2@mail.gmail.com>
	<4BA0087A.2040006@Sun.COM>
Message-ID: <d6262f011003181127x104b729cw8dbdd3040d6a2dd3@mail.gmail.com>

On Tue, Mar 16, 2010 at 3:38 PM, Jon Masamitsu <Jon.Masamitsu at sun.com>wrote:

> Hiroshi,
>
> The goal of UseAdaptiveSizePolicy with regard to performance is
> to size the generations so as to achieve the throughput goal.
> GC cost is reduced by reducing the frequency of collections
> and that is achieved by more free space in the generations
> (more space for more allocations before the next GC).
>

OK.


>
> I can see how a user might want to limit the footprint.  Without
> your change I would tell the user to reduce the throughput goal.
> Not a particularly friendly interface to limit footprint I'll
> admit.
>

The motive is to make it easier for users to control the footprint. The free
ratio is very easy to understand for users.


>
> There is a method
>
> PSAdaptiveSizePolicy::adjust_for_throughput()
>
> which is called to calculate the amount that the heap
> should grow in order to achieve a throughput goal.
> If that method said not to increase the heap based
> on the value of MaxHeapFreeRatio is that close to
> what you want?  It's not the same but it would be
> a way of limiting the growth of the heap (stops
> growing the heap when the free space after the collection
> exceeds a threshold).  I think it would fit in more
> easily with the rest of UseAdativeSizePolicy.  I think
> it would be easier to explain too.
>

I think even if adjust_for_throughput tries to limit the growth that way in
each invocation of it, it's likely that the heap will eventually fully
expanded with enough amount of load (and objects). So, it could delay the
expansion under light load. But I think the key here is to be able to shrink
the heap after it is expanded. It looks to me that adjust_for_throughput
never shrinks the size. If I'm not mistaken, it's closer, but not quite
there, I think.

Hiroshi


> Jon
>
> On 03/15/10 16:50, Hiroshi Yamauchi wrote:
>
>
>>
>> On Mon, Mar 15, 2010 at 12:03 PM, Jon Masamitsu <Jon.Masamitsu at sun.com<mailto:
>> Jon.Masamitsu at sun.com>> wrote:
>>
>>    Hiroshi,
>>
>>    I'm looking at the code in psParallelCompact.cpp
>>
>>    2076     bool free_ratio_in_effect = false;
>>    2077     if ((UseFreeRatioForParallelGC ||
>>    2078          (UseFreeRatioOnlyInSystemGCForParallelGC &&
>>    2079           gc_cause == GCCause::_java_lang_system_gc))) {
>>    2080       ParallelScavengeHeap* heap =
>>    (ParallelScavengeHeap*)Universe::heap();
>>    2081       free_ratio_in_effect =
>>    heap->try_to_shrink_by_free_ratio(true);
>>    2082     }
>>    2083
>>    2084     if (!free_ratio_in_effect && UseAdaptiveSizePolicy) {
>>
>>    If only UseFreeRatioForParallelGC is set to true, is it correct
>>    that the heap
>>
>>     a) will grow according to the original UseAdaptiveSizePolicy
>>    policy until MaxHeapFreeRatio is exceeded (i.e., until
>>    try_to_shrink_by_free_ratio() returns true)
>>     b) will then shrink until MaxHeapFreeRatio is no longer exceeded
>>    (likely just that once)
>>
>>    then start with a) again?
>>
>>
>> I think that mostly matches my intention. The cyclical pattern
>> (a->b->a->b->...) is assumed not to happen more often than necessary though.
>> An expectation is that the value of MaxHeapFreeRatio is high enough (70 by
>> default) that the b) only happens when the app is under reasonably light(er)
>> load (or just idle.) UseFreeRatioOnlyInSystemGCForParallelGC (instead of
>> UseFreeRatioForParallelGC ) is for apps that call System.gc() to give a hint
>> about the need to have its heap shrunk and want to avoid heap shrinking
>> otherwise.
>>
>> An intuition is the following. To reduce the memory footprint, the free
>> ratio logic holds back the heap expansion if the original adaptive size
>> policy expands too aggressively, or overrides and shrink the heap if the
>> original adaptive size policy leaves too much free space, in terms of the
>> value of MaxHeapFreeRatio. I think it's not far from what flag
>> MaxHeapFreeRatio originally meant. Otherwise, the heap size depends on the
>> original adaptive size policy (for performance.) It's a tradeoff between
>> performance and footprint.
>>
>>
>>    What type of application fits with such a combination of policies?
>>    Do you have a plots of how the heap sizing behaves (i.e., size of the
>>    generations vs. collections/time)?
>>
>>
>> The reason to combine the two logics (size policies) isn't for particular
>> applications. I don't get too surprised if you say it should be a separate
>> size policy and leave the original adaptive size policy alone. My thinking
>> is that users may want the same level of performance from the parallel
>> collector (the adaptive size policy is enabled by default) unless a large
>> part of the heap (> 70% by default) is free. This is my attempt to get the
>> best of both. The flags are off by default since of course, not all
>> applications need to have their heap shrunk this way.
>>
>> No, I don't have plots.
>>
>> Hiroshi
>>
>>
>>
>>    Jon
>>
>>    Hiroshi Yamauchi wrote On 03/10/10 15:17,:
>>
>>     > Hi,
>>     >
>>     > I'd like to have this patch
>>     >
>>     >   http://cr.openjdk.java.net/~rasbold/69XXXXX/webrev.00/<http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/>
>>    <http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/>
>>     > <http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/>
>>     >
>>     > contributed, if appropriate.
>>     >
>>     > It implements a simple heap shrinking logic based on the free ratio
>>     > (MaxHeapFreeRatio) in the parallel collector. The way it works is
>> the
>>     > following: If the free ratio, (1.0 - <used> / <capacity>) * 100 >
>>     > MaxHeapFreeRatio, this logic kicks in. Otherwise, the (default)
>>     > adaptive size policy runs the show (as before). This feature is
>>    turned
>>     > off by default. There are two new flags to turn it on:
>>     >
>>     >   UseFreeRatioForParallelGC - this enables it.
>>     >   UseFreeRatioOnlyInSystemGCForParallelGC - this enables it only in
>>     > explicit System.gc() calls. This is more useful in apps that call
>>     > System.gc() when it's idle and don't want shrinking to happen at all
>>     > other times.
>>     >
>>     > In our tests, it appears to work well in a simple test and to help a
>>     > real app reduce its footprint (RSS) when it's idle.
>>     >
>>     > Thanks to Chuck Rasbold who uploaded the webrev on my behalf.
>>     >
>>     > Thanks,
>>     > Hiroshi
>>     >
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20100318/e7697fbe/attachment.html 

From mbien at fh-landshut.de  Thu Mar 18 11:18:39 2010
From: mbien at fh-landshut.de (Michael Bien)
Date: Thu, 18 Mar 2010 19:18:39 +0100
Subject: performance impact of JNI GetCritical*
In-Reply-To: <194f62551003181020n30b122ccjc092aeecd8e32a26@mail.gmail.com>
References: <4B92FA13.3070903@fh-landshut.de>
	<194f62551003181020n30b122ccjc092aeecd8e32a26@mail.gmail.com>
Message-ID: <4BA26E7F.1030506@fh-landshut.de>

Thanks for the answer Clemens,

comments inline...

On 03/18/2010 06:20 PM, Clemens Eisserer wrote:
> Hi Michael,
>
>    
>> How optimized are the Get/ReleasePrimitiveArrayCritical JNI functions?
>> Do they disable GC or do they only pin the array at the specific address?
>>      
> For moving gc's they actually stop the whole GC process as long as all
> arrays are released by ReleasePrimitiveArrayCritical.
> So basically the cost is some thread-synchronization (to make the gc
> know on all threads not to run now), plus the impact on the GC. So for
> small arrays theres quite high overhead.
> CMS itself is non-moving (old gen) so basically there could be some
> optimizations, but no idea wether this is actually done.
>
>    
>> lets say I have a short java array of length 12. Should I copy it to a
>> direct NIO buffer and use the buffer as vehicle, use Get/ReleaseCritical
>> or something else?
>>      
> For very short arrays this is quite likely beneficial (at least if
> you're on JDK7, at least older JDK6 releases do get/releasecritical +
> memcpy in jni code).
>
> Just to be curious, whats your actual usecase?
>    
the usecase was to optimize the generated code of JOGL 2 / JOAL / JOCL 
etc. for some corner cases.
I feared that putting load on GetCritical* could cause GC-dependent 
problems. Thats why I am already using ThreadLocal direct NIO buffers in 
the high-level API of JOCL to prevent this situation.

- michael


> - Clemens
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>
>
>    

-- 
Michael Bien
http://michael-bien.com/

_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From Y.S.Ramakrishna at Sun.COM  Thu Mar 18 11:49:44 2010
From: Y.S.Ramakrishna at Sun.COM (Y. Srinivas Ramakrishna)
Date: Thu, 18 Mar 2010 11:49:44 -0700
Subject: performance impact of JNI GetCritical*
In-Reply-To: <4BA26E7F.1030506@fh-landshut.de>
References: <4B92FA13.3070903@fh-landshut.de>
	<194f62551003181020n30b122ccjc092aeecd8e32a26@mail.gmail.com>
	<4BA26E7F.1030506@fh-landshut.de>
Message-ID: <4BA275C8.6050600@Sun.COM>

Hi Clemens. Michael --

On 03/18/10 11:18, Michael Bien wrote:
> Thanks for the answer Clemens,
> 
> comments inline...
> 
> On 03/18/2010 06:20 PM, Clemens Eisserer wrote:
>> Hi Michael,
>>
>>    
>>> How optimized are the Get/ReleasePrimitiveArrayCritical JNI functions?
>>> Do they disable GC or do they only pin the array at the specific address?
>>>      
>> For moving gc's they actually stop the whole GC process as long as all
>> arrays are released by ReleasePrimitiveArrayCritical.
>> So basically the cost is some thread-synchronization (to make the gc
>> know on all threads not to run now), plus the impact on the GC. So for
>> small arrays theres quite high overhead.
>> CMS itself is non-moving (old gen) so basically there could be some
>> optimizations, but no idea wether this is actually done.

CMS GC is indeed permitted even when in JNI critical sections, but
since most objects are allocated in Eden which uses copying
collection, the performance effect of long-lived JNI CS can
indeed be disastrous even when you are using CMS. In fact,
it might be worse in a configuration using CMS because
when Eden is full and because of a long-lived JNI CS we do
not scavenge, allocations happen from the old gen, and
as we know allocation out of CMS' free lists can be much
slower (although of course, we can theoretically withstand
much-longer-lived CS because CMS GC can concurrently collect
the garbage produced from these allocations). Bottom line:
CMS does not give you any practical performance advantage
wrt the over(ab?)use of JNI CS.

>>
>>    
>>> lets say I have a short java array of length 12. Should I copy it to a
>>> direct NIO buffer and use the buffer as vehicle, use Get/ReleaseCritical
>>> or something else?
>>>      
>> For very short arrays this is quite likely beneficial (at least if
>> you're on JDK7, at least older JDK6 releases do get/releasecritical +
>> memcpy in jni code).

Right; this is a fact that I learnt much to my delight recently.
I requested that this improvement be backported to JDK 6, but
I do not know if/when that might happen.

-- ramki

>>
>> Just to be curious, whats your actual usecase?
>>    
> the usecase was to optimize the generated code of JOGL 2 / JOAL / JOCL 
> etc. for some corner cases.
> I feared that putting load on GetCritical* could cause GC-dependent 
> problems. Thats why I am already using ThreadLocal direct NIO buffers in 
> the high-level API of JOCL to prevent this situation.
> 
> - michael
> 
> 
>> - Clemens
>> _______________________________________________
>> hotspot-gc-use mailing list
>> hotspot-gc-use at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>
>>
>>    
> 

_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From yamauchi at google.com  Thu Mar 18 12:01:59 2010
From: yamauchi at google.com (Hiroshi Yamauchi)
Date: Thu, 18 Mar 2010 12:01:59 -0700
Subject: Free ratio based heap shrinking in the parallel collector
In-Reply-To: <4BA046A9.6080103@sun.com>
References: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
	<4B9E8499.8030904@sun.com>
	<d6262f011003151650v7d58e699gd868cae490f2a6f2@mail.gmail.com>
	<4BA0087A.2040006@Sun.COM> <4BA046A9.6080103@sun.com>
Message-ID: <d6262f011003181201h502819d8i22e57684f564a986@mail.gmail.com>

Jon,

Thanks for your comments.

On Tue, Mar 16, 2010 at 8:04 PM, Jon Masamitsu <Jon.Masamitsu at sun.com>wrote:

> I had forgotten the other half of this which was the behavior
> on System.gc().   I agree that it would be nice to have a way
> for the user to collapse the heap quickly if the user knows that
> the application is going to be inactive.   The code you suggested
> only shrinks the heap.  I think it would be more complete if it
> would also grow the heap according to MinHeapFreeRatio.
>

In practice, the expansion side isn't a problem in my experience. i.e., it
will expand eventually. Also, in the way the patch is written currently, the
expansion is taken care of by PSAdaptiveSizePolicy which I think does a good
job. That said, I agree that it'd be more complete if the MinHeapFreeRatio
part is implemented from the point of view of seeing MinHeapFreeRatio and
MaxHeapFreeRatio as a inseparable pair.


> I don't know the scenario where it would be used but I
> wouldn't be surprised if there is one.  It would also make it
> practical to run UseParallelGC with UseAdaptiveSizePolicy
> turned off (the heap resizing would then be done similarly
> to the other collectors).
>

One scenario that I have for system.gc() as a hint for shrinkage is where
you have a daemon-like (background) process, running on your
desktop/workstation, that performs occasional batch type jobs. After each
job is done, it is highly likely that the process will be idle for a while
until a next job comes. It isn't the best if the idle background process is
occupying a large amount of (resident) memory. Another would be, if there is
some server that does a batch type of job (so, latency isn't a priority)
that could be idle for an extended period of time and there are other
processes on the same machine that could benefit from more free memory.

Do you mean that if one runs with -XX:+UseParallelGC and
-XX:-UseAdaptiveSizePolicy, the free ratio flags are honored by the parallel
collector or the heap shrinks in practice some other ways?

Thanks,
Hiroshi


> Jon
>
> Jon Masamitsu wrote On 03/16/10 15:38,:
>
> >Hiroshi,
> >
> >The goal of UseAdaptiveSizePolicy with regard to performance is
> >to size the generations so as to achieve the throughput goal.
> >GC cost is reduced by reducing the frequency of collections
> >and that is achieved by more free space in the generations
> >(more space for more allocations before the next GC).
> >
> >I can see how a user might want to limit the footprint.  Without
> >your change I would tell the user to reduce the throughput goal.
> >Not a particularly friendly interface to limit footprint I'll
> >admit.
> >
> >There is a method
> >
> >PSAdaptiveSizePolicy::adjust_for_throughput()
> >
> >which is called to calculate the amount that the heap
> >should grow in order to achieve a throughput goal.
> >If that method said not to increase the heap based
> >on the value of MaxHeapFreeRatio is that close to
> >what you want?  It's not the same but it would be
> >a way of limiting the growth of the heap (stops
> >growing the heap when the free space after the collection
> >exceeds a threshold).  I think it would fit in more
> >easily with the rest of UseAdativeSizePolicy.  I think
> >it would be easier to explain too.
> >
> >Jon
> >
> >On 03/15/10 16:50, Hiroshi Yamauchi wrote:
> >
> >
> >>On Mon, Mar 15, 2010 at 12:03 PM, Jon Masamitsu <Jon.Masamitsu at sun.com
> >><mailto:Jon.Masamitsu at sun.com>> wrote:
> >>
> >>    Hiroshi,
> >>
> >>    I'm looking at the code in psParallelCompact.cpp
> >>
> >>    2076     bool free_ratio_in_effect = false;
> >>    2077     if ((UseFreeRatioForParallelGC ||
> >>    2078          (UseFreeRatioOnlyInSystemGCForParallelGC &&
> >>    2079           gc_cause == GCCause::_java_lang_system_gc))) {
> >>    2080       ParallelScavengeHeap* heap =
> >>    (ParallelScavengeHeap*)Universe::heap();
> >>    2081       free_ratio_in_effect =
> >>    heap->try_to_shrink_by_free_ratio(true);
> >>    2082     }
> >>    2083
> >>    2084     if (!free_ratio_in_effect && UseAdaptiveSizePolicy) {
> >>
> >>    If only UseFreeRatioForParallelGC is set to true, is it correct
> >>    that the heap
> >>
> >>     a) will grow according to the original UseAdaptiveSizePolicy
> >>    policy until MaxHeapFreeRatio is exceeded (i.e., until
> >>    try_to_shrink_by_free_ratio() returns true)
> >>     b) will then shrink until MaxHeapFreeRatio is no longer exceeded
> >>    (likely just that once)
> >>
> >>    then start with a) again?
> >>
> >>
> >>I think that mostly matches my intention. The cyclical pattern
> >>(a->b->a->b->...) is assumed not to happen more often than necessary
> >>though. An expectation is that the value of MaxHeapFreeRatio is high
> >>enough (70 by default) that the b) only happens when the app is under
> >>reasonably light(er) load (or just idle.)
> >>UseFreeRatioOnlyInSystemGCForParallelGC (instead of
> >>UseFreeRatioForParallelGC ) is for apps that call System.gc() to give a
> >>hint about the need to have its heap shrunk and want to avoid heap
> >>shrinking otherwise.
> >>
> >>An intuition is the following. To reduce the memory footprint, the free
> >>ratio logic holds back the heap expansion if the original adaptive size
> >>policy expands too aggressively, or overrides and shrink the heap if the
> >>original adaptive size policy leaves too much free space, in terms of
> >>the value of MaxHeapFreeRatio. I think it's not far from what flag
> >>MaxHeapFreeRatio originally meant. Otherwise, the heap size depends on
> >>the original adaptive size policy (for performance.) It's a tradeoff
> >>between performance and footprint.
> >>
> >>
> >>    What type of application fits with such a combination of policies?
> >>    Do you have a plots of how the heap sizing behaves (i.e., size of the
> >>    generations vs. collections/time)?
> >>
> >>
> >>The reason to combine the two logics (size policies) isn't for
> >>particular applications. I don't get too surprised if you say it should
> >>be a separate size policy and leave the original adaptive size policy
> >>alone. My thinking is that users may want the same level of performance
> >>from the parallel collector (the adaptive size policy is enabled by
> >>default) unless a large part of the heap (> 70% by default) is free.
> >>This is my attempt to get the best of both. The flags are off by default
> >>since of course, not all applications need to have their heap shrunk
> >>this way.
> >>
> >>No, I don't have plots.
> >>
> >>Hiroshi
> >>
> >>
> >>
> >>    Jon
> >>
> >>    Hiroshi Yamauchi wrote On 03/10/10 15:17,:
> >>
> >>     > Hi,
> >>     >
> >>     > I'd like to have this patch
> >>     >
> >>     >   http://cr.openjdk.java.net/~rasbold/69XXXXX/webrev.00/<http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/>
> >>    <http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/>
> >>     > <http://cr.openjdk.java.net/%7Erasbold/69XXXXX/webrev.00/>
> >>     >
> >>     > contributed, if appropriate.
> >>     >
> >>     > It implements a simple heap shrinking logic based on the free
> ratio
> >>     > (MaxHeapFreeRatio) in the parallel collector. The way it works is
> the
> >>     > following: If the free ratio, (1.0 - <used> / <capacity>) * 100 >
> >>     > MaxHeapFreeRatio, this logic kicks in. Otherwise, the (default)
> >>     > adaptive size policy runs the show (as before). This feature is
> >>    turned
> >>     > off by default. There are two new flags to turn it on:
> >>     >
> >>     >   UseFreeRatioForParallelGC - this enables it.
> >>     >   UseFreeRatioOnlyInSystemGCForParallelGC - this enables it only
> in
> >>     > explicit System.gc() calls. This is more useful in apps that call
> >>     > System.gc() when it's idle and don't want shrinking to happen at
> all
> >>     > other times.
> >>     >
> >>     > In our tests, it appears to work well in a simple test and to help
> a
> >>     > real app reduce its footprint (RSS) when it's idle.
> >>     >
> >>     > Thanks to Chuck Rasbold who uploaded the webrev on my behalf.
> >>     >
> >>     > Thanks,
> >>     > Hiroshi
> >>     >
> >>
> >>
> >>
> >>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20100318/8fb8aa44/attachment.html 

From Y.S.Ramakrishna at Sun.COM  Thu Mar 18 12:18:58 2010
From: Y.S.Ramakrishna at Sun.COM (Y. Srinivas Ramakrishna)
Date: Thu, 18 Mar 2010 12:18:58 -0700
Subject: performance impact of JNI GetCritical*
In-Reply-To: <4BA275C8.6050600@Sun.COM>
References: <4B92FA13.3070903@fh-landshut.de>
	<194f62551003181020n30b122ccjc092aeecd8e32a26@mail.gmail.com>
	<4BA26E7F.1030506@fh-landshut.de> <4BA275C8.6050600@Sun.COM>
Message-ID: <4BA27CA2.60707@Sun.COM>

On 03/18/10 11:49, Y. Srinivas Ramakrishna wrote:

>>>> lets say I have a short java array of length 12. Should I copy it to a
>>>> direct NIO buffer and use the buffer as vehicle, use 
>>>> Get/ReleaseCritical
>>>> or something else?
>>>>      
>>> For very short arrays this is quite likely beneficial (at least if
>>> you're on JDK7, at least older JDK6 releases do get/releasecritical +
>>> memcpy in jni code).
> 
> Right; this is a fact that I learnt much to my delight recently.
> I requested that this improvement be backported to JDK 6, but
> I do not know if/when that might happen.

Let me clarify/correct my comment a bit.

What I meant to say was that NIO performance/behaviour improved a lot
in JDK 7 because it stopped using JNI CS, using unsafe instead.
(This also had the pleasant side-effect of working around a particularly
egregious CMS bug related to JNI CS recently discussed on this alias,
and still unfortunately not fixed, which is what initially brought this
change to my attention.)

As regards Michael's original question on whether NIO or JNI CS
is better suited for short arrays, at the risk of stating the obvious,
I think the answer is that how much better NIO is (in JDK 7) would likely
depend on the frequency of copying and size of these arrays: If JNI CS
are very short-lived and infrequent and the arrays are as small as 12 bytes,
the difference would likely be negligible. As frequency of use (and degree
of concurrent use) and/or array size increases, NIO would likely become
increasingly better than JNI CS (in JDK 7 at least). At least that's my hunch.

-- ramki
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From andrey.petrusenko at sun.com  Thu Mar 18 12:58:53 2010
From: andrey.petrusenko at sun.com (andrey.petrusenko at sun.com)
Date: Thu, 18 Mar 2010 19:58:53 +0000
Subject: hg: jdk7/hotspot-gc/hotspot: 6921710: G1: assert(new_finger >=
	_finger && new_finger < _region_limit, "invariant")
Message-ID: <20100318195909.9DA954412F@hg.openjdk.java.net>

Changeset: 3f0549ed0c98
Author:    apetrusenko
Date:      2010-03-18 01:48 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/3f0549ed0c98

6921710: G1: assert(new_finger >= _finger && new_finger < _region_limit,"invariant")
Summary: If CM task was aborted while scanning the last object of the specified region and the size of that object is equal to bitmap's granularity then the next offset would be equal or over the region limit which is exactly what the assertion states.
Reviewed-by: ysr, tonyp, jmasa

! src/share/vm/gc_implementation/g1/concurrentMark.cpp


From linuxhippy at gmail.com  Thu Mar 18 14:02:29 2010
From: linuxhippy at gmail.com (Clemens Eisserer)
Date: Thu, 18 Mar 2010 22:02:29 +0100
Subject: performance impact of JNI GetCritical*
In-Reply-To: <4BA27CA2.60707@Sun.COM>
References: <4B92FA13.3070903@fh-landshut.de>
	<194f62551003181020n30b122ccjc092aeecd8e32a26@mail.gmail.com>
	<4BA26E7F.1030506@fh-landshut.de> <4BA275C8.6050600@Sun.COM>
	<4BA27CA2.60707@Sun.COM>
Message-ID: <194f62551003181402pda11201xfb6522814574b48c@mail.gmail.com>

> If JNI CS
> are very short-lived and infrequent and the arrays are as small as 12 bytes,
> the difference would likely be negligible. As frequency of use (and degree
> of concurrent use) and/or array size increases, NIO would likely become
> increasingly better than JNI CS (in JDK 7 at least). At least that's my
> hunch.

I fear in the case of JOGL the problem is frequent JNI-CS with small arrays.
So basically you do very little work per JNI-CS, thus the
CS-enter/leave overhead is higher than with larger arrays. Best would
probably to use NIO-Buffers directly, which would remove
copy-overhead.

- Clemens

From mbien at fh-landshut.de  Thu Mar 18 14:26:16 2010
From: mbien at fh-landshut.de (Michael Bien)
Date: Thu, 18 Mar 2010 22:26:16 +0100
Subject: performance impact of JNI GetCritical*
In-Reply-To: <194f62551003181402pda11201xfb6522814574b48c@mail.gmail.com>
References: <4B92FA13.3070903@fh-landshut.de><194f62551003181020n30b122ccjc092aeecd8e32a26@mail.gmail.com><4BA26E7F.1030506@fh-landshut.de>
	<4BA275C8.6050600@Sun.COM><4BA27CA2.60707@Sun.COM>
	<194f62551003181402pda11201xfb6522814574b48c@mail.gmail.com>
Message-ID: <4BA29A78.5000103@fh-landshut.de>

On 03/18/2010 10:02 PM, Clemens Eisserer wrote:
>> If JNI CS
>> are very short-lived and infrequent and the arrays are as small as 12 bytes,
>> the difference would likely be negligible. As frequency of use (and degree
>> of concurrent use) and/or array size increases, NIO would likely become
>> increasingly better than JNI CS (in JDK 7 at least). At least that's my
>> hunch.
>>      
> I fear in the case of JOGL the problem is frequent JNI-CS with small arrays.
> So basically you do very little work per JNI-CS, thus the
> CS-enter/leave overhead is higher than with larger arrays. Best would
> probably to use NIO-Buffers directly, which would remove
> copy-overhead.
>
> - Clemens
>    
sure thats what we doing - we force nio everywhere possible. But in some 
situations its to verbose for the host application.
e.g. JOCL is not all about big payloads. There are a lot of flags/small 
structs etc involved.

take a look at this method:
http://michael-bien.com/hudson/job/jocl/label=u64JogAmp/javadoc/com/mbien/opencl/CL.html#clEnqueueNDRangeKernel%28long,%20long,%20int,%20com.sun.gluegen.runtime.PointerBuffer,%20com.sun.gluegen.runtime.PointerBuffer,%20com.sun.gluegen.runtime.PointerBuffer,%20int,%20com.sun.gluegen.runtime.PointerBuffer,%20com.sun.gluegen.runtime.PointerBuffer%29

...lots of small buffers. (You find this kind of methods usually in 
inner loops)
The high-level api uses internal ThreadLocal buffers for those 
situations (and it works pretty well).

but thanks to everyone for answering - good discussion, I'll take a look 
at the NIO improvements.

-- 
Michael Bien
http://michael-bien.com/


From john.coomes at sun.com  Thu Mar 18 18:50:43 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Fri, 19 Mar 2010 01:50:43 +0000
Subject: hg: jdk7/hotspot-gc/hotspot: 6935839: excessive marking stack growth
	during full gcs
Message-ID: <20100319015057.4AB2B44187@hg.openjdk.java.net>

Changeset: c385bf94cfb8
Author:    jcoomes
Date:      2010-03-18 13:31 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/c385bf94cfb8

6935839: excessive marking stack growth during full gcs
Summary: process one item at a time from the objarray stack/queue
Reviewed-by: apetrusenko, tonyp

! src/share/vm/gc_implementation/parallelScavenge/psCompactionManager.cpp
! src/share/vm/gc_implementation/shared/markSweep.cpp


From john.coomes at sun.com  Fri Mar 19 02:41:55 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Fri, 19 Mar 2010 09:41:55 +0000
Subject: hg: jdk7/hotspot-gc: 4 new changesets
Message-ID: <20100319094155.EA96F44218@hg.openjdk.java.net>

Changeset: 4d7419e4b759
Author:    ohair
Date:      2010-03-06 15:00 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/rev/4d7419e4b759

6928700: Configure top repo for JPRT testing
Reviewed-by: alanb, jjg

! make/jprt.properties
+ test/Makefile

Changeset: f3664d6879ab
Author:    ohair
Date:      2010-03-06 15:01 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/rev/f3664d6879ab

Merge


Changeset: 433a60a9c0bf
Author:    lana
Date:      2010-03-09 15:28 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/rev/433a60a9c0bf

Merge


Changeset: 6b1069f53fbc
Author:    mikejwre
Date:      2010-03-18 13:52 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/rev/6b1069f53fbc

Added tag jdk7-b86 for changeset 433a60a9c0bf

! .hgtags


From john.coomes at sun.com  Fri Mar 19 02:42:08 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Fri, 19 Mar 2010 09:42:08 +0000
Subject: hg: jdk7/hotspot-gc/corba: Added tag jdk7-b86 for changeset
	6253e28826d1
Message-ID: <20100319094211.D0C6A44219@hg.openjdk.java.net>

Changeset: 09a41111a401
Author:    mikejwre
Date:      2010-03-18 13:52 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/corba/rev/09a41111a401

Added tag jdk7-b86 for changeset 6253e28826d1

! .hgtags


From john.coomes at sun.com  Fri Mar 19 02:45:55 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Fri, 19 Mar 2010 09:45:55 +0000
Subject: hg: jdk7/hotspot-gc/jaxp: Added tag jdk7-b86 for changeset
	81c0f115bbe5
Message-ID: <20100319094600.5D2F94421B@hg.openjdk.java.net>

Changeset: 8b493f1aa136
Author:    mikejwre
Date:      2010-03-18 13:52 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxp/rev/8b493f1aa136

Added tag jdk7-b86 for changeset 81c0f115bbe5

! .hgtags


From john.coomes at sun.com  Fri Mar 19 02:46:11 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Fri, 19 Mar 2010 09:46:11 +0000
Subject: hg: jdk7/hotspot-gc/jaxws: Added tag jdk7-b86 for changeset
	512b0e924a5a
Message-ID: <20100319094611.9DB2C4421C@hg.openjdk.java.net>

Changeset: 3febd6fab2ac
Author:    mikejwre
Date:      2010-03-18 13:52 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/3febd6fab2ac

Added tag jdk7-b86 for changeset 512b0e924a5a

! .hgtags


From john.coomes at sun.com  Fri Mar 19 02:47:40 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Fri, 19 Mar 2010 09:47:40 +0000
Subject: hg: jdk7/hotspot-gc/jdk: 40 new changesets
Message-ID: <20100319101429.6AA2144223@hg.openjdk.java.net>

Changeset: 840601ac5ab7
Author:    rkennke
Date:      2010-03-03 15:50 +0100
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/840601ac5ab7

6892485: Deadlock in SunGraphicsEnvironment / FontManager
Summary: Synchronize on correct monitor in SunFontManager.
Reviewed-by: igor, prr

! src/share/classes/sun/font/SunFontManager.java

Changeset: 1d7db2d5c4c5
Author:    minqi
Date:      2010-03-08 11:35 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/1d7db2d5c4c5

6918065: Crash in Java2D blit loop (IntArgbToIntArgbPreSrcOverMaskBlit) in 64bit mode
Reviewed-by: igor, bae

! src/share/classes/java/awt/AlphaComposite.java
+ test/java/awt/AlphaComposite/TestAlphaCompositeForNaN.java

Changeset: 494f5e4f24da
Author:    lana
Date:      2010-03-09 15:26 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/494f5e4f24da

Merge


Changeset: e64331144648
Author:    rupashka
Date:      2010-02-10 15:15 +0300
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/e64331144648

6848475: JSlider does not display the correct value of its BoundedRangeModel
Reviewed-by: peterz

! src/share/classes/javax/swing/plaf/basic/BasicSliderUI.java
+ test/javax/swing/JSlider/6848475/bug6848475.java

Changeset: f81c8041ccf4
Author:    peytoia
Date:      2010-02-11 15:58 +0900
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/f81c8041ccf4

6909002: Remove indicim.jar and thaiim.jar from JRE and move to samples if needed
Reviewed-by: okutsu

! make/com/sun/Makefile

Changeset: e2b58a45a426
Author:    peytoia
Date:      2010-02-12 14:38 +0900
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/e2b58a45a426

6921289: (tz) Support tzdata2010b
Reviewed-by: okutsu

! make/sun/javazic/tzdata/VERSION
! make/sun/javazic/tzdata/antarctica
! make/sun/javazic/tzdata/asia
! make/sun/javazic/tzdata/australasia
! make/sun/javazic/tzdata/europe
! make/sun/javazic/tzdata/northamerica
! make/sun/javazic/tzdata/zone.tab
! src/share/classes/sun/util/resources/TimeZoneNames.java
! src/share/classes/sun/util/resources/TimeZoneNames_de.java
! src/share/classes/sun/util/resources/TimeZoneNames_es.java
! src/share/classes/sun/util/resources/TimeZoneNames_fr.java
! src/share/classes/sun/util/resources/TimeZoneNames_it.java
! src/share/classes/sun/util/resources/TimeZoneNames_ja.java
! src/share/classes/sun/util/resources/TimeZoneNames_ko.java
! src/share/classes/sun/util/resources/TimeZoneNames_sv.java
! src/share/classes/sun/util/resources/TimeZoneNames_zh_CN.java
! src/share/classes/sun/util/resources/TimeZoneNames_zh_TW.java

Changeset: e8340332745e
Author:    malenkov
Date:      2010-02-18 17:46 +0300
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/e8340332745e

4498236: RFE: Provide a toString method for PropertyChangeEvent and other classes
Reviewed-by: peterz

! src/share/classes/java/beans/BeanDescriptor.java
! src/share/classes/java/beans/EventSetDescriptor.java
! src/share/classes/java/beans/FeatureDescriptor.java
! src/share/classes/java/beans/IndexedPropertyChangeEvent.java
! src/share/classes/java/beans/IndexedPropertyDescriptor.java
! src/share/classes/java/beans/MethodDescriptor.java
! src/share/classes/java/beans/PropertyChangeEvent.java
! src/share/classes/java/beans/PropertyDescriptor.java
+ test/java/beans/Introspector/Test4498236.java

Changeset: 5c03237838e1
Author:    rupashka
Date:      2010-02-27 14:26 +0300
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/5c03237838e1

6913758: Specification for SynthViewportUI.paintBorder(...) should mention that this method is never called
Reviewed-by: peterz

! src/share/classes/javax/swing/plaf/synth/SynthViewportUI.java

Changeset: 96205ed1b196
Author:    rupashka
Date:      2010-02-27 14:47 +0300
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/96205ed1b196

6918447: SynthToolBarUI.setBorderToXXXX() methods don't correspond inherited spec. They do nothing.
Reviewed-by: peterz

! src/share/classes/javax/swing/plaf/synth/SynthToolBarUI.java

Changeset: 621e921a14cd
Author:    rupashka
Date:      2010-02-27 15:09 +0300
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/621e921a14cd

6918861: SynthSliderUI.uninstallDefaults() is not called when UI is uninstalled
Reviewed-by: malenkov

! src/share/classes/javax/swing/plaf/basic/BasicSliderUI.java
! src/share/classes/javax/swing/plaf/synth/SynthSliderUI.java
+ test/javax/swing/JSlider/6918861/bug6918861.java

Changeset: 28741de0bb4a
Author:    rupashka
Date:      2010-02-27 16:03 +0300
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/28741de0bb4a

6923305: SynthSliderUI paints the slider track when the slider's "paintTrack" property is set to false
Reviewed-by: alexp

! src/share/classes/javax/swing/plaf/synth/SynthSliderUI.java
+ test/javax/swing/JSlider/6923305/bug6923305.java

Changeset: 2bf137beb9bd
Author:    rupashka
Date:      2010-02-27 16:14 +0300
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/2bf137beb9bd

6929298: The SynthSliderUI#calculateTickRect method should be removed
Reviewed-by: peterz

! src/share/classes/javax/swing/plaf/synth/SynthSliderUI.java

Changeset: d6b3a07c8752
Author:    rupashka
Date:      2010-03-03 17:57 +0300
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/d6b3a07c8752

6924059: SynthScrollBarUI.configureScrollBarColors() should have spec different from the overridden method
Reviewed-by: peterz

! src/share/classes/javax/swing/plaf/synth/SynthScrollBarUI.java
+ test/javax/swing/JScrollBar/6924059/bug6924059.java

Changeset: 30c520bd148f
Author:    rupashka
Date:      2010-03-03 20:08 +0300
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/30c520bd148f

6913768: With default SynthLookAndFeel instance installed new JTable creation leads to throwing NPE
Reviewed-by: peterz

! src/share/classes/javax/swing/JTable.java
! src/share/classes/javax/swing/plaf/synth/SynthTableUI.java
+ test/javax/swing/JTable/6913768/bug6913768.java

Changeset: f13fc955be62
Author:    rupashka
Date:      2010-03-03 20:53 +0300
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/f13fc955be62

6917744: JScrollPane Page Up/Down keys do not handle correctly html tables with different cells contents
Reviewed-by: peterz, alexp

! src/share/classes/javax/swing/text/DefaultEditorKit.java
+ test/javax/swing/JEditorPane/6917744/bug6917744.java
+ test/javax/swing/JEditorPane/6917744/test.html

Changeset: 0622086d82ac
Author:    malenkov
Date:      2010-03-04 21:17 +0300
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/0622086d82ac

6921644: XMLEncoder generates invalid XML
Reviewed-by: peterz

! src/share/classes/java/beans/XMLEncoder.java
+ test/java/beans/XMLEncoder/Test5023550.java
+ test/java/beans/XMLEncoder/Test5023557.java
+ test/java/beans/XMLEncoder/Test6921644.java

Changeset: 79a509ac8f35
Author:    lana
Date:      2010-03-01 18:30 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/79a509ac8f35

Merge

! make/com/sun/Makefile
- make/java/text/FILES_java.gmk

Changeset: 90248595ec35
Author:    lana
Date:      2010-03-04 13:07 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/90248595ec35

Merge


Changeset: 2fe4e72288ce
Author:    lana
Date:      2010-03-09 15:28 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/2fe4e72288ce

Merge


Changeset: 38fbb2353a6a
Author:    alanb
Date:      2010-02-23 17:56 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/38fbb2353a6a

6925977: (file) test/java/nio/file/Path/CheckPermissions.java fails if test.src on read-only file system
Reviewed-by: chegar

! test/java/nio/file/Path/CheckPermissions.java

Changeset: 00abf8c232be
Author:    alanb
Date:      2010-02-23 17:58 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/00abf8c232be

6925932: (file) Path.endsWith can throw ArrayIndexOutOfBoundsException (unx)
Reviewed-by: chegar

! src/solaris/classes/sun/nio/fs/UnixPath.java
! test/java/nio/file/Path/PathOps.java

Changeset: be5db597f3be
Author:    alanb
Date:      2010-02-23 18:19 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/be5db597f3be

6928960: make modules fails to build class analyzer
Reviewed-by: mchung

! make/modules/tools/Makefile

Changeset: e94b296b53b4
Author:    alanb
Date:      2010-02-23 18:21 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/e94b296b53b4

6926800: TEST_BUG: java/nio/file/Files/walk_file_tree.sh fails with newer versions of find(1)
Reviewed-by: forax

! test/java/nio/file/Files/PrintFileTree.java
! test/java/nio/file/Files/walk_file_tree.sh

Changeset: e842e99b514a
Author:    darcy
Date:      2010-02-24 10:48 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/e842e99b514a

6929382: Various core classes in util and elsewhere are missing @param <T> tags
Reviewed-by: dholmes, martin

! src/share/classes/java/lang/Iterable.java
! src/share/classes/java/util/Collection.java
! src/share/classes/java/util/Iterator.java
! src/share/classes/java/util/List.java

Changeset: 9929203a8b98
Author:    xuelei
Date:      2010-02-25 13:32 +0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/9929203a8b98

6916202: More cases of invalid ldap filters accepted and processed
Reviewed-by: vinnie, weijun

! src/share/classes/com/sun/jndi/ldap/Filter.java
+ test/com/sun/jndi/ldap/InvalidLdapFilters.java

Changeset: 77beb60b39c6
Author:    alanb
Date:      2010-02-27 18:18 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/77beb60b39c6

6929532: (file) WatchService should avoid queuing new modify events when lots of files are changing
Reviewed-by: alanb
Contributed-by: sebastian.sickelmann at gmx.de

! src/share/classes/sun/nio/fs/AbstractWatchKey.java
+ test/java/nio/file/WatchService/LotsOfEvents.java
- test/java/nio/file/WatchService/OverflowEventIsLoner.java

Changeset: b77e94f5a601
Author:    alanb
Date:      2010-02-27 19:15 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/b77e94f5a601

6929259: Remove double spaces from Dual-pivot quicksort
Reviewed-by: alanb
Contributed-by: vladimir.yaroslavskiy at sun.com

! src/share/classes/java/util/DualPivotQuicksort.java

Changeset: 529d2da0aee2
Author:    alanb
Date:      2010-02-27 19:26 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/529d2da0aee2

6815768: File.getxxxSpace() methods fail for very large file systems under 32bit Java
Reviewed-by: ohair

! src/solaris/native/java/io/UnixFileSystem_md.c

Changeset: f7a6eae6e1eb
Author:    alanb
Date:      2010-02-27 19:29 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/f7a6eae6e1eb

6921374: java.lang.String::hashCode() should check for count == 0 to avoid repeated stores hash = 0
Reviewed-by: darcy, ohair

! src/share/classes/java/lang/String.java

Changeset: 78d91c4223cb
Author:    vinnie
Date:      2010-03-01 17:54 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/78d91c4223cb

6921001: api/java_security/IdentityScope/IdentityScopeTests.html#getSystemScope fails starting from b78 JDK7
Reviewed-by: mullan

! src/share/classes/java/security/IdentityScope.java
! src/share/lib/security/java.security
+ test/java/security/IdentityScope/NoDefaultSystemScope.java

Changeset: 893034df4ec2
Author:    vinnie
Date:      2010-03-01 18:00 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/893034df4ec2

Merge

- test/java/nio/file/WatchService/OverflowEventIsLoner.java

Changeset: cddb43b12d28
Author:    alanb
Date:      2010-03-03 16:09 +0000
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/cddb43b12d28

6931216: TEST_BUG: test/java/nio/file/WatchService/LotsOfEvents.java failed with NPE
Reviewed-by: chegar

! test/java/nio/file/WatchService/LotsOfEvents.java

Changeset: 507159d8d143
Author:    ohair
Date:      2010-03-03 11:29 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/507159d8d143

6931763: sanity checks broken with latest cygwin, newer egrep -i option problems
Reviewed-by: jjg

! make/common/shared/Sanity.gmk

Changeset: 61c298558549
Author:    weijun
Date:      2010-03-04 10:37 +0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/61c298558549

6844909: support allow_weak_crypto in krb5.conf
Reviewed-by: valeriep

! src/share/classes/sun/security/krb5/internal/crypto/EType.java
+ test/sun/security/krb5/etype/WeakCrypto.java
+ test/sun/security/krb5/etype/weakcrypto.conf

Changeset: 0f383673ce31
Author:    weijun
Date:      2010-03-04 10:38 +0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/0f383673ce31

6923681: Jarsigner crashes during timestamping
Reviewed-by: vinnie

! src/share/classes/sun/security/tools/TimestampedSigner.java

Changeset: 5e15b70e6d27
Author:    weijun
Date:      2010-03-04 10:38 +0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/5e15b70e6d27

6880321: sun.security.provider.JavaKeyStore abuse of OOM Exception handling
Reviewed-by: xuelei

! src/share/classes/sun/security/provider/JavaKeyStore.java

Changeset: c2d29e5695c2
Author:    lana
Date:      2010-03-04 13:40 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/c2d29e5695c2

Merge


Changeset: 58b44ac0b10d
Author:    ohair
Date:      2010-03-06 14:59 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/58b44ac0b10d

6915983: testing problems, adjusting list of tests, needs some investigation
Reviewed-by: alanb

! test/Makefile
! test/ProblemList.txt

Changeset: eae6e9ab2606
Author:    lana
Date:      2010-03-09 15:29 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/eae6e9ab2606

Merge

- test/java/nio/file/WatchService/OverflowEventIsLoner.java

Changeset: 2cafbbe9825e
Author:    mikejwre
Date:      2010-03-18 13:53 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/2cafbbe9825e

Added tag jdk7-b86 for changeset eae6e9ab2606

! .hgtags


From john.coomes at sun.com  Fri Mar 19 03:32:39 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Fri, 19 Mar 2010 10:32:39 +0000
Subject: hg: jdk7/hotspot-gc/langtools: 19 new changesets
Message-ID: <20100319103349.8A1FA44228@hg.openjdk.java.net>

Changeset: 6eca0895a644
Author:    jjg
Date:      2010-02-23 18:43 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/6eca0895a644

6511613: javac unexpectedly doesn't fail in some cases if an annotation processor specified
Reviewed-by: darcy

! src/share/classes/com/sun/tools/javac/comp/Attr.java
! src/share/classes/com/sun/tools/javac/main/JavaCompiler.java
! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java
! src/share/classes/com/sun/tools/javac/util/Log.java
+ test/tools/javac/processing/6511613/DummyProcessor.java
+ test/tools/javac/processing/6511613/clss41701.java

Changeset: 87eb6edd4f21
Author:    jjg
Date:      2010-02-25 09:42 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/87eb6edd4f21

4880220: Add a warning when accessing a static method via an reference
Reviewed-by: darcy

! make/build.properties
! src/share/classes/com/sun/tools/javac/code/Lint.java
! src/share/classes/com/sun/tools/javac/comp/Attr.java
! src/share/classes/com/sun/tools/javac/comp/Check.java
! src/share/classes/com/sun/tools/javac/resources/compiler.properties
+ test/tools/javac/4880220/T4880220.empty.out
+ test/tools/javac/4880220/T4880220.error.out
+ test/tools/javac/4880220/T4880220.java
+ test/tools/javac/4880220/T4880220.warn.out

Changeset: 85242c273d31
Author:    darcy
Date:      2010-02-25 11:04 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/85242c273d31

6929645: Address various findbugs warnings in langtools
Reviewed-by: jjg

! src/share/classes/com/sun/tools/apt/comp/Apt.java
! src/share/classes/com/sun/tools/apt/mirror/declaration/AnnotationProxyMaker.java
! src/share/classes/com/sun/tools/apt/mirror/declaration/DeclarationImpl.java
! src/share/classes/com/sun/tools/javac/model/AnnotationProxyMaker.java
! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java

Changeset: dbcba45123cd
Author:    jjg
Date:      2010-02-25 12:26 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/dbcba45123cd

6929544: langtools source code uses statics qualified by instance variables
Reviewed-by: darcy

! make/build.properties
! src/share/classes/com/sun/tools/apt/main/CommandLine.java
! src/share/classes/com/sun/tools/apt/mirror/type/TypeMirrorImpl.java
! src/share/classes/com/sun/tools/doclets/standard/Standard.java
! src/share/classes/com/sun/tools/javac/Launcher.java
! src/share/classes/com/sun/tools/javac/api/JavacTool.java
! src/share/classes/com/sun/tools/javac/code/Types.java
! src/share/classes/com/sun/tools/javac/jvm/Gen.java
! src/share/classes/com/sun/tools/javac/jvm/Items.java
! src/share/classes/com/sun/tools/javac/main/CommandLine.java
! src/share/classes/com/sun/tools/javac/main/JavaCompiler.java

Changeset: af75fd6155de
Author:    jjg
Date:      2010-02-25 13:32 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/af75fd6155de

6893943: exit code from javah with no args is 0
Reviewed-by: darcy

! src/share/classes/com/sun/tools/javah/JavahTask.java
+ test/tools/javah/T6893943.java

Changeset: b030706da5b4
Author:    jjg
Date:      2010-02-26 08:42 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/b030706da5b4

6881645: Unchecked method call on a method declared inside anonymous inner causes javac to crash
Reviewed-by: mcimadamore

! src/share/classes/com/sun/tools/javac/code/Symbol.java
+ test/tools/javac/T6881645.java

Changeset: 72833a8a6086
Author:    jjg
Date:      2010-02-26 15:26 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/72833a8a6086

6930076: "null" can incorrectly appear in error message compiler.err.error.reading.file
Reviewed-by: darcy

! src/share/classes/com/sun/tools/javac/file/JavacFileManager.java
! src/share/classes/com/sun/tools/javac/file/Paths.java
! src/share/classes/com/sun/tools/javac/main/JavaCompiler.java

Changeset: 7b69c7083a97
Author:    jjg
Date:      2010-02-26 15:30 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/7b69c7083a97

6930032: fix findbugs errors in com.sun.tools.javac.comp
Reviewed-by: darcy

! src/share/classes/com/sun/tools/javac/comp/Enter.java
! src/share/classes/com/sun/tools/javac/comp/TransTypes.java

Changeset: 7c23bbbe0dbd
Author:    darcy
Date:      2010-03-02 14:06 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/7c23bbbe0dbd

6931130: Remove unused AnnotationCollector code from JavacProcessingEnvironment
Reviewed-by: jjg

! src/share/classes/com/sun/tools/javac/processing/JavacProcessingEnvironment.java

Changeset: 6e1e2738c530
Author:    jjg
Date:      2010-03-02 16:40 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/6e1e2738c530

6931482: minor findbugs fixes
Reviewed-by: darcy

! src/share/classes/com/sun/tools/classfile/ConstantPool.java
! src/share/classes/com/sun/tools/javadoc/DocEnv.java
! src/share/classes/com/sun/tools/javadoc/SeeTagImpl.java

Changeset: 235135d61974
Author:    jjg
Date:      2010-03-02 16:43 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/235135d61974

6931127: strange test class files
Reviewed-by: darcy

! test/tools/javac/annotations/neg/Constant.java
! test/tools/javac/generics/Casting.java
! test/tools/javac/generics/Casting3.java
! test/tools/javac/generics/Casting4.java
! test/tools/javac/generics/InnerInterface1.java
! test/tools/javac/generics/InnerInterface2.java
! test/tools/javac/generics/Multibound1.java
! test/tools/javac/generics/MultipleInheritance.java
! test/tools/javac/generics/NameOrder.java
! test/tools/javac/generics/PermuteBound.java
! test/tools/javac/generics/PrimitiveVariant.java

Changeset: fc7132746501
Author:    darcy
Date:      2010-03-03 16:05 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/fc7132746501

6449781: TypeElement.getQualifiedName for anonymous classes returns null instead of an empty name
Reviewed-by: jjg

! src/share/classes/com/sun/tools/javac/jvm/ClassReader.java
+ test/tools/javac/processing/model/element/TestAnonClassNames.java
+ test/tools/javac/processing/model/element/TestAnonSourceNames.java

Changeset: 7f5db2e8b423
Author:    jjg
Date:      2010-03-03 17:22 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/7f5db2e8b423

6931927: position issues with synthesized anonymous class
Reviewed-by: darcy

! src/share/classes/com/sun/tools/javac/parser/JavacParser.java
+ test/tools/javac/tree/TestAnnotatedAnonClass.java
+ test/tools/javac/tree/TreePosTest.java
- test/tools/javac/treepostests/TreePosTest.java

Changeset: 117c95448ab9
Author:    jjg
Date:      2010-03-03 19:34 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/117c95448ab9

6931126: jtreg tests not Windows friendly
Reviewed-by: darcy

! test/tools/javac/ThrowsIntersection_1.java
! test/tools/javac/ThrowsIntersection_2.java
! test/tools/javac/ThrowsIntersection_3.java
! test/tools/javac/ThrowsIntersection_4.java
! test/tools/javac/generics/NameOrder.java

Changeset: c55733ceed61
Author:    lana
Date:      2010-03-04 13:40 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/c55733ceed61

Merge


Changeset: a23282f17d0b
Author:    jjg
Date:      2010-03-05 16:12 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/a23282f17d0b

6930108: IllegalArgumentException in AbstractDiagnosticFormatter for tools/javac/api/TestJavacTaskScanner.jav
Reviewed-by: darcy

! src/share/classes/com/sun/tools/javac/util/BasicDiagnosticFormatter.java
! test/tools/javac/api/TestJavacTaskScanner.java
+ test/tools/javac/api/TestResolveError.java

Changeset: a4f3b97c8028
Author:    jjg
Date:      2010-03-05 16:13 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/a4f3b97c8028

Merge


Changeset: ef07347428f2
Author:    lana
Date:      2010-03-09 15:29 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/ef07347428f2

Merge

- test/tools/javac/treepostests/TreePosTest.java

Changeset: 409db93d19c0
Author:    mikejwre
Date:      2010-03-18 13:53 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/409db93d19c0

Added tag jdk7-b86 for changeset ef07347428f2

! .hgtags


From Jon.Masamitsu at Sun.COM  Fri Mar 19 10:04:08 2010
From: Jon.Masamitsu at Sun.COM (Jon Masamitsu)
Date: Fri, 19 Mar 2010 10:04:08 -0700
Subject: Free ratio based heap shrinking in the parallel collector
In-Reply-To: <d6262f011003181127x104b729cw8dbdd3040d6a2dd3@mail.gmail.com>
References: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
	<4B9E8499.8030904@sun.com>
	<d6262f011003151650v7d58e699gd868cae490f2a6f2@mail.gmail.com>
	<4BA0087A.2040006@Sun.COM>
	<d6262f011003181127x104b729cw8dbdd3040d6a2dd3@mail.gmail.com>
Message-ID: <4BA3AE88.3090002@sun.com>

Hiroshi Yamauchi wrote On 03/18/10 11:27,:

>
>
>
>  
>
>
>     There is a method
>
>     PSAdaptiveSizePolicy::adjust_for_throughput()
>
>     which is called to calculate the amount that the heap
>     should grow in order to achieve a throughput goal.
>     If that method said not to increase the heap based
>     on the value of MaxHeapFreeRatio is that close to
>     what you want?  It's not the same but it would be
>     a way of limiting the growth of the heap (stops
>     growing the heap when the free space after the collection
>     exceeds a threshold).  I think it would fit in more
>     easily with the rest of UseAdativeSizePolicy.  I think
>     it would be easier to explain too.
>
>
> I think even if adjust_for_throughput tries to limit the growth that
> way in each invocation of it, it's likely that the heap will
> eventually fully expanded with enough amount of load (and objects).
> So, it could delay the expansion under light load. But I think the key
> here is to be able to shrink the heap after it is expanded. It looks
> to me that adjust_for_throughput never shrinks the size. If I'm not
> mistaken, it's closer, but not quite there, I think.
>
You're right that adjust_for_throughput() will not shrink the heap.
Another part of the policy is that the heap is shrunk if the
throughput goal and the pause time goal are being
met.   It's a slower process than just enforcing MaxHeapFreeRatio
and requires a number of GC's (that is, the heap would be
shrunk down a percentage at each GC).    We would
need to change the UseAdaptiveSizePolicy to do something like


if ( average-pause > pause-goal)
          shrink heap
else if (average-throughput < throughput-goal) &&
            ! (UseFreeRatioForParallelGC && heap too empty))  <<<< new
         grow heap
else
         shrink heap


Or maybe something like


if ( average-pause > pause-goal)
          shrink heap
else if (average-throughput < throughput-goal)
          if  (UseFreeRatioForParallelGC && heap too empty)    <<<< new
                 shrink heap (policy-to-be-determined)         <<<<
          else                                                 <<<<
                 grow heap
else if (average-throughput < throughput-goal)
         grow heap to reduce gc overhead
else
         shrink heap


From Jon.Masamitsu at Sun.COM  Fri Mar 19 20:43:08 2010
From: Jon.Masamitsu at Sun.COM (Jon Masamitsu)
Date: Fri, 19 Mar 2010 20:43:08 -0700
Subject: Using CMS, any chance of forewarning a serial full GC is	imminent?
In-Reply-To: <7c962aed1003192031j31dc4c6blb82f2dd42d954795@mail.gmail.com>
References: <7c962aed1003192021t2d53858fx796cba7414a90fe7@mail.gmail.com>
	<7c962aed1003192031j31dc4c6blb82f2dd42d954795@mail.gmail.com>
Message-ID: <22DC392D-5732-4D3B-8D0A-E7BFD7787995@sun.com>

System.gc() will cause CMS to do a full collection.  Does that
help?  As long as you have not turned off explicit GC's nor
set flags so that CMS does System.gc() concurrently, you can
compact the tenured (CMS) gen that way.

On Mar 19, 2010, at 8:31 PM, Stack wrote:

> Or, running CMS, is there a way to trigger full serial GC?  If we
> could make it run explicitly at preordained times, this would be an
> improvement over it happening at maximum-embarrassment time.
>
> Thanks,
> St.Ack
>
>
> On Fri, Mar 19, 2010 at 8:21 PM, Stack <stack at duboce.net> wrote:
>> Our app, a distributed database, usually does fine running CMS but
>> when we trip a full serial GC, its disruptive (speaking
>> euphemistically).
>>
>> Monitoring the running application, watching the logs or patching  
>> into
>> the JVM-TI, is there any indicator that you know of that would give  
>> us
>> forewarning of an imminent full serial GC?  If we had this, we could
>> take evasive action.
>>
>> Thanks,
>> St.Ack
>>
>> P.S. G1 is what we really need but going by the response up on this
>> and by how easy our application crashes recent releases, it looks  
>> like
>> its going to be a good while before it'll work for our case (We're an
>> open source database so talking to our sun/oracle vendor is not an
>> option).
>>
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From MASLOWSC at cboe.com  Sat Mar 20 12:11:38 2010
From: MASLOWSC at cboe.com (Maslowski, Chuck)
Date: Sat, 20 Mar 2010 14:11:38 -0500
Subject: Using CMS, any chance of forewarning a serial full GC is imminent?
In-Reply-To: <mailman.25.1269111603.28638.hotspot-gc-dev@openjdk.java.net>
References: <mailman.25.1269111603.28638.hotspot-gc-dev@openjdk.java.net>
Message-ID: <4BD174404CF4A34C98322DC926CF862B14D2854D@MSMAIL.cboent.cboe.com>

Ramki gave us this D script awhile back. In our case we wanted to kill
the process should this occur.

hotspot$target:::gc-begin
{
  self->partial = !arg0;
}

hotspot$target:::mem-pool-gc-begin
/ self->partial && copyinstr(arg0) == "ConcurrentMarkSweep" &&
copyinstr(arg2) == "CMS Old Gen" /
{
  system("kill %d", pid);
}

From Y.S.Ramakrishna at Sun.COM  Sat Mar 20 19:18:31 2010
From: Y.S.Ramakrishna at Sun.COM (Y. Srinivas Ramakrishna)
Date: Sat, 20 Mar 2010 19:18:31 -0700
Subject: Using CMS, any chance of forewarning a serial full GC is imminent?
In-Reply-To: <7c962aed1003192021t2d53858fx796cba7414a90fe7@mail.gmail.com>
References: <7c962aed1003192021t2d53858fx796cba7414a90fe7@mail.gmail.com>
Message-ID: <4BA581F7.1080904@sun.com>

Hello St.Ack --

Stack wrote:
> Our app, a distributed database, usually does fine running CMS but
> when we trip a full serial GC, its disruptive (speaking
> euphemistically).
> 
> Monitoring the running application, watching the logs or patching into
> the JVM-TI, is there any indicator that you know of that would give us
> forewarning of an imminent full serial GC?  If we had this, we could
> take evasive action.

Probably the most important input factors are the statistics internally maintained
by the CMS collector regarding the population spread of free blocks,
their expected historical demand and their recent demand, some
combination of which should, at least theoretically, yield a suitable
statistical predictor of the imminency of promotion failure.
Unfortunately such a predictor has not been synthesized by us yet (and
would probably be a challenge if we are to avoid false positives; and
we would almost certainly need help from an expert statistician to
synthesize such a predictor).
If properly designed (and that's a big if, such a predictor could
be exported via a suitable GC MBean and could be polled to
automate the triggering of evasive action (i.e. move load to
another node and trigger a GC on this node; it would be an
interesting distributed coordination problem to avoid situations
where a large majority of nodes decide at roughly the same time
that a full GC is imminent and try to push their load to a
neighbour that is ill-prepared to handle it).
May be you can tell us whether -- when promotion failure
occurs in your current distributed system, whether it is an isolated
incident on one node or if it has a more catastrophic quality where
multiple nodes succumb to the problem at about the same time, causing
a promotion failure contagion, as it were, to spread rapidly through
the entire system.

> 
> Thanks,
> St.Ack
> 
> P.S. G1 is what we really need but going by the response up on this
> and by how easy our application crashes recent releases, it looks like
> its going to be a good while before it'll work for our case (We're an
> open source database so talking to our sun/oracle vendor is not an
> option).

Do give G1 a try in 6u20 though. The reliability has been much improved
since 6u18.

Relatedly, and out of curiosity, focusing on a single node/jvm of yr system,
what is the rough frequency of promotion failure that you see?

-- ramki

> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From peter.schuller at infidyne.com  Sun Mar 21 06:12:59 2010
From: peter.schuller at infidyne.com (Peter Schuller)
Date: Sun, 21 Mar 2010 14:12:59 +0100
Subject: G1 heap growth very aggressive in spite of low heap usage
Message-ID: <5a1151761003210612j5cd8c3afh57d94b75b5c24c55@mail.gmail.com>

Hello,

what is the intended behavior of G1 with respect to heap size? I was
playing around with a fairly simple test[1] designed for something
else, when I realized that G1 was expanding my heap in ways that I
find unexpected[2]. Reading g1CollectedHeap.cpp there seems to be
attempts to adher to MaxHeapFreeRatio and MinHeapFreeRatio. However,
even when I run with (on purpose) extreme options of 10/25
minimum/maximum, heap expansion happens as in [2].

My expected behavior would for the heep size to remain within some
reasonable percentage of actual memory usage (assuming I only specify
max heap size and leave minimum heap size untouched), and it is a
desirable behavior in part because concurrent marking only seems to
trigger as a function of heap usage relative to the heap size. In many
situations it is desirable to have the JVM heap size be a reasonable
indicator of actual memory requirements/use. If I am only using a
fraction of heap size (5% in this case), the delay in concurrent
marking will mean that my application (unless it *only* generates
ephemeral garbage) will grow unreasonably untill concurrent marking is
triggered. After it completes the heap use would go down very very
significantly. The end result is that (1) I actually need a lot more
memory than I otherwise would, and (2) it is difficult to monitor the
real memory requirements of the application.

I do understand that maximum throughput tend to be achieved by
postponing concurrent marking to maximize yield, but if that was my
aim, I would specify -Xms == -Xmx. With a small -Xms and a large -Xmx,
I would want the heap to expand only as "necessary" (though I
understand that is difficult to define), resulting in a much more
reasonable trigger threshold for concurrent marking.

The test is a fairly simple web server that i use 'ab' to throw
traffic at. It spawns a number of threads per requests, each of which
tries to produce a non-trivial stack depth, and then sleeps for a
random period. The intent was to use this to try to see what kind of
overhead was incurred for root set scanning with many threads (though
few active) during young generation pauses.

I then added some generation of permanent data that I could trigger by
submitting HTTP requests. The sudden growth seen in [2], when it
spikes up from 100-200 MB to 3-4 GB, is when I submit some request to
trigger generation of permanent data. The actual amount is very
limited however, and as you can see it very quickly spikes the memory
use far beyond any reasonable 'min free' percentage might be designed
to.

The options I used for the run from which [2] comes from are:

   -XX:+PrintGC
   -XX:+UnlockExperimentalVMOptions
   -XX:+UnlockDiagnosticVMOptions
   -XX:+UseG1GC
   -XX:MaxGCPauseMillis=25
   -XX:GCPauseIntervalMillis=50
   -XX:+G1ParallelRSetUpdatingEnabled
   -XX:+G1ParallelRSetScanningEnabled
   -Xmx4G
   -XX:G1ConfidencePercent=100
   -XX:MinHeapFreeRatio=10
   -XX:MaxHeapFreeRatio=25

This was run on a recent checkout of the bsdport of openjdk7:

   changeset:   192:9f250d0d1b40
   tag:         tip
   parent:      187:417d1a0aa480
   parent:      191:6b1069f53fbc
   user:        Greg Lewis <glewis at eyesbeyond.com>
   date:        Sat Mar 20 11:11:42 2010 -0700
   summary:     Merge from main OpenJDK repository

I can provide an executable jar file if someone wants to test it but
is not set up to build clojure/leiningen projects.

[1] http://github.com/scode/httpgctest

[2]:

Here is an excerpt. Each pause is 0.25-5 seconds in between depending
on whether I am generating permanent data at the time.

[GC pause (young) 45M->35M(110M), 0.0482680 secs]
[GC pause (young) 49M->42M(110M), 0.0428210 secs]
[GC pause (young) 53M->46M(110M), 0.0238750 secs]
[GC pause (young) 59M->51M(110M), 0.0312480 secs]
[GC pause (young) 63M->58M(110M), 0.0404710 secs]
[GC pause (young) 74M->62M(110M), 0.0262080 secs]
[GC pause (young) 75M->69M(220M), 0.0509200 secs]
[GC pause (young) 85M->77M(440M), 0.0478070 secs]
[GC pause (young) 95M->86M(880M), 0.0560680 secs]
[GC pause (young) 102M->96M(1524M), 0.0618900 secs]
[GC pause (young) 112M->102M(2039M), 0.0404470 secs]
[GC pause (young) 116M->111M(2451M), 0.0546120 secs]
[GC pause (young) 126M->120M(2780M), 0.0441630 secs]
[GC pause (young) 137M->121M(3044M), 0.0201850 secs]
[GC pause (young) 136M->122M(3255M), 0.0113520 secs]
[GC pause (young) 134M->122M(3424M), 0.0111860 secs]
[GC pause (young) 133M->122M(3559M), 0.0089910 secs]
[GC pause (young) 133M->122M(3667M), 0.0079770 secs]
[GC pause (young) 140M->122M(3753M), 0.0074160 secs]
[GC pause (young) 144M->136M(3753M), 0.0738920 secs]
[GC pause (young) 146M->141M(3753M), 0.0363720 secs]
[GC pause (young) 154M->147M(3753M), 0.0256830 secs]
[GC pause (young) 158M->148M(3753M), 0.0116360 secs]
[GC pause (young) 157M->149M(3753M), 0.0172780 secs]
[GC pause (young) 157M->154M(3753M), 0.0273040 secs]
[GC pause (young) 168M->163M(3753M), 0.0526970 secs]
[GC pause (young) 179M->172M(3822M), 0.0547310 secs]
[GC pause (young) 190M->180M(3877M), 0.0337350 secs]
[GC pause (young) 195M->189M(3921M), 0.0530290 secs]
[GC pause (young) 206M->199M(3956M), 0.0598740 secs]
[GC pause (young) 218M->206M(3984M), 0.0320840 secs]
[GC pause (young) 222M->207M(4007M), 0.0140790 secs]
[GC pause (young) 221M->207M(4025M), 0.0084930 secs]
[GC pause (young) 218M->207M(4040M), 0.0066170 secs]
[GC pause (young) 219M->207M(4052M), 0.0069570 secs]
[GC pause (young) 223M->207M(4061M), 0.0067710 secs]
[GC pause (young) 227M->207M(4061M), 0.0071300 secs]
[GC pause (young) 231M->207M(4061M), 0.0070520 secs]


-- 
/ Peter Schuller
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From andrey.petrusenko at sun.com  Mon Mar 22 07:11:54 2010
From: andrey.petrusenko at sun.com (andrey.petrusenko at sun.com)
Date: Mon, 22 Mar 2010 14:11:54 +0000
Subject: hg: jdk7/hotspot-gc/hotspot: 45 new changesets
Message-ID: <20100322141422.E32204467D@hg.openjdk.java.net>

Changeset: 51db1e4b379d
Author:    twisti
Date:      2010-03-08 04:46 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/51db1e4b379d

6932536: JSR 292 modified JDK MethodHandlesTest fails on x86_64
Summary: A modified MethodHandlesTest revealed two bugs on x86_64.
Reviewed-by: never, jrose

! src/cpu/x86/vm/methodHandles_x86.cpp

Changeset: 7de45b5044c3
Author:    never
Date:      2010-03-09 11:02 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/7de45b5044c3

6932270: Allow Java's ELF symtab reader to use separate debuginfo files
Reviewed-by: never
Contributed-by: Andrew Haley <aph at redhat.com>

! agent/src/os/linux/libproc_impl.c
! agent/src/os/linux/symtab.c
! agent/src/os/linux/symtab.h
+ make/linux/makefiles/build_vm_def.sh
! make/linux/makefiles/mapfile-vers-debug
! make/linux/makefiles/mapfile-vers-product
! make/linux/makefiles/vm.make

Changeset: 3cf667df43ef
Author:    twisti
Date:      2010-03-09 20:16 +0100
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/3cf667df43ef

6919934: JSR 292 needs to support x86 C1
Summary: This implements JSR 292 support for C1 x86.
Reviewed-by: never, jrose, kvn

! src/cpu/sparc/vm/c1_CodeStubs_sparc.cpp
! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp
! src/cpu/sparc/vm/c1_MacroAssembler_sparc.cpp
! src/cpu/sparc/vm/c1_Runtime1_sparc.cpp
! src/cpu/sparc/vm/interp_masm_sparc.cpp
! src/cpu/sparc/vm/interp_masm_sparc.hpp
! src/cpu/sparc/vm/stubGenerator_sparc.cpp
! src/cpu/sparc/vm/templateInterpreter_sparc.cpp
! src/cpu/x86/vm/c1_CodeStubs_x86.cpp
! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp
! src/cpu/x86/vm/c1_MacroAssembler_x86.cpp
! src/cpu/x86/vm/c1_Runtime1_x86.cpp
! src/cpu/x86/vm/stubGenerator_x86_32.cpp
! src/cpu/x86/vm/stubGenerator_x86_64.cpp
! src/cpu/x86/vm/templateInterpreter_x86_32.cpp
! src/cpu/x86/vm/templateInterpreter_x86_64.cpp
! src/share/vm/c1/c1_Canonicalizer.cpp
! src/share/vm/c1/c1_CodeStubs.hpp
! src/share/vm/c1/c1_GraphBuilder.cpp
! src/share/vm/c1/c1_IR.cpp
! src/share/vm/c1/c1_IR.hpp
! src/share/vm/c1/c1_Instruction.cpp
! src/share/vm/c1/c1_Instruction.hpp
! src/share/vm/c1/c1_LIR.cpp
! src/share/vm/c1/c1_LIR.hpp
! src/share/vm/c1/c1_LIRAssembler.cpp
! src/share/vm/c1/c1_LIRAssembler.hpp
! src/share/vm/c1/c1_LIRGenerator.cpp
! src/share/vm/c1/c1_MacroAssembler.hpp
! src/share/vm/ci/ciCPCache.cpp
! src/share/vm/ci/ciCPCache.hpp
! src/share/vm/includeDB_compiler1
! src/share/vm/includeDB_core
! src/share/vm/opto/runtime.cpp
! src/share/vm/runtime/sharedRuntime.cpp
! src/share/vm/runtime/sharedRuntime.hpp
! src/share/vm/runtime/vframeArray.cpp

Changeset: d8e270c4f609
Author:    twisti
Date:      2010-03-09 23:57 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/d8e270c4f609

Merge


Changeset: c466efa608d5
Author:    roland
Date:      2010-03-05 13:58 +0100
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/c466efa608d5

6932496: c1: deoptimization of jsr subroutine fails on sparcv9
Summary: store jsr ret bci as intptr constant in c1 debug info
Reviewed-by: never

! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp
! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp
! src/share/vm/c1/c1_LIR.cpp
! src/share/vm/c1/c1_LIR.hpp
! src/share/vm/c1/c1_LinearScan.cpp
+ test/compiler/6932496/Test6932496.java

Changeset: da06d1795d84
Author:    twisti
Date:      2010-03-11 05:09 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/da06d1795d84

6934089: Zero 32-bit/64kb page fix
Summary: The fix for 6927165 increased the number of shadow pages for 32-bit platforms and this causes a problem on systems with 64kb pages.
Reviewed-by: twisti
Contributed-by: Gary Benson <gbenson at redhat.com>

! src/os_cpu/linux_zero/vm/globals_linux_zero.hpp

Changeset: 9eba43136cb5
Author:    twisti
Date:      2010-03-16 11:52 +0100
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/9eba43136cb5

6934494: JSR 292 MethodHandles adapters should be generated into their own CodeBlob
Summary: Passing a null pointer to an InvokeDynamic function call should lead to a NullPointerException.
Reviewed-by: kvn, never

! src/cpu/sparc/vm/stubRoutines_sparc.hpp
! src/cpu/x86/vm/stubGenerator_x86_32.cpp
! src/cpu/x86/vm/stubGenerator_x86_64.cpp
! src/cpu/x86/vm/stubRoutines_x86_32.hpp
! src/cpu/x86/vm/stubRoutines_x86_64.hpp
! src/share/vm/code/codeBlob.cpp
! src/share/vm/code/codeBlob.hpp
! src/share/vm/includeDB_core
! src/share/vm/prims/methodHandles.cpp
! src/share/vm/prims/methodHandles.hpp
! src/share/vm/runtime/init.cpp
! src/share/vm/runtime/sharedRuntime.cpp
! src/share/vm/runtime/sharedRuntime.hpp

Changeset: 428a9c451986
Author:    kvn
Date:      2010-03-16 15:35 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/428a9c451986

6935466: new CodeCache flushing code is not guarded by the flag
Summary: Add missing guard.
Reviewed-by: never

! src/share/vm/compiler/compileBroker.cpp

Changeset: fc2c71045ada
Author:    twisti
Date:      2010-03-17 10:22 +0100
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/fc2c71045ada

6934966: JSR 292 add C1 logic for saved SP over MethodHandle calls
Summary: The logic for x86 C1 to save the SP over MH calls is pretty straight forward but SPARC handles that differently.
Reviewed-by: never, jrose

! src/cpu/sparc/vm/c1_FrameMap_sparc.hpp
! src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp
! src/cpu/x86/vm/c1_FrameMap_x86.hpp
! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp
! src/share/vm/c1/c1_LIR.cpp
! src/share/vm/c1/c1_LIRAssembler.cpp
! src/share/vm/c1/c1_LIRAssembler.hpp

Changeset: 2484f4d6a54e
Author:    kvn
Date:      2010-03-17 10:47 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/2484f4d6a54e

6935535: String.indexOf() returns incorrect result on x86 with SSE4.2
Summary: Added missing counter decrement when substring search restarted.
Reviewed-by: twisti

! src/cpu/x86/vm/assembler_x86.cpp
+ test/compiler/6935535/Test.java

Changeset: c047da02984c
Author:    never
Date:      2010-03-17 16:40 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/c047da02984c

6930043: C2: SIGSEGV in javasoft.sqe.tests.lang.arr017.arr01702.arr01702.loop_forw(II)I
Reviewed-by: kvn

! src/share/vm/opto/loopTransform.cpp
! src/share/vm/opto/loopnode.hpp
+ test/compiler/6930043/Test6930043.java

Changeset: 76c1d7d13ec5
Author:    twisti
Date:      2010-03-18 09:56 +0100
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/76c1d7d13ec5

6932091: JSR 292 x86 code cleanup
Summary: Some code cleanups found during the JSR 292 SPARC port.
Reviewed-by: kvn, never

! src/cpu/x86/vm/methodHandles_x86.cpp
! src/cpu/x86/vm/templateTable_x86_32.cpp
! src/cpu/x86/vm/templateTable_x86_64.cpp
! src/share/vm/c1/c1_LIRGenerator.cpp
! src/share/vm/prims/methodHandles.hpp
! src/share/vm/runtime/arguments.cpp

Changeset: 97fe2cc98b1d
Author:    twisti
Date:      2010-03-18 06:36 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/97fe2cc98b1d

Merge


Changeset: 26ecc6fa29e6
Author:    mikejwre
Date:      2010-02-04 11:19 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/26ecc6fa29e6

Added tag jdk7-b82 for changeset 1999f5b12482

! .hgtags

Changeset: 1e3c5d0474d4
Author:    trims
Date:      2010-02-05 16:21 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/1e3c5d0474d4

Merge


Changeset: 39e0a32bc49b
Author:    trims
Date:      2010-02-11 19:52 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/39e0a32bc49b

Added tag hs17-b01 for changeset a94714c55065

! .hgtags

Changeset: bd1260aafd87
Author:    trims
Date:      2010-02-11 19:52 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/bd1260aafd87

Added tag hs17-b02 for changeset faf94d94786b

! .hgtags

Changeset: d9c445aa7bb1
Author:    trims
Date:      2010-02-11 19:52 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/d9c445aa7bb1

Added tag hs17-b03 for changeset f4b900403d6e

! .hgtags

Changeset: 3940517a1f13
Author:    trims
Date:      2010-02-11 19:52 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/3940517a1f13

Added tag hs17-b04 for changeset d8dd291a362a

! .hgtags

Changeset: 4458e32d9125
Author:    trims
Date:      2010-02-11 19:52 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/4458e32d9125

Added tag hs17-b05 for changeset 9174bb32e934

! .hgtags

Changeset: 36a78dac746f
Author:    trims
Date:      2010-02-11 19:52 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/36a78dac746f

Added tag hs17-b06 for changeset a5a6adfca6ec

! .hgtags

Changeset: bfa6d67a7a29
Author:    trims
Date:      2010-02-11 19:53 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/bfa6d67a7a29

Added tag hs17-b07 for changeset 3003ddd1d433

! .hgtags

Changeset: 73047d0b13cf
Author:    trims
Date:      2010-02-11 19:53 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/73047d0b13cf

Added tag hs17-b08 for changeset 1f9b07674480

! .hgtags

Changeset: 12076a98a540
Author:    trims
Date:      2010-02-11 19:53 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/12076a98a540

Added tag hs17-b09 for changeset ff3232b68fbb

! .hgtags

Changeset: 704a172a0918
Author:    trims
Date:      2010-02-11 20:11 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/704a172a0918

Added tag hs16-b01 for changeset 981375ca07b7

! .hgtags

Changeset: e114a6374471
Author:    trims
Date:      2010-02-11 20:11 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/e114a6374471

Added tag hs16-b02 for changeset f4cbf78110c7

! .hgtags

Changeset: 3469eafe9bf4
Author:    trims
Date:      2010-02-11 20:11 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/3469eafe9bf4

Added tag hs16-b03 for changeset 07c1c01e0315

! .hgtags

Changeset: 26dba59fc9ec
Author:    trims
Date:      2010-02-11 20:11 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/26dba59fc9ec

Added tag hs16-b04 for changeset 08f86fa55a31

! .hgtags

Changeset: 8b0989046c93
Author:    trims
Date:      2010-02-11 20:11 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/8b0989046c93

Added tag hs16-b05 for changeset 32c83fb84370

! .hgtags

Changeset: 5fe06b3f6753
Author:    trims
Date:      2010-02-11 20:11 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/5fe06b3f6753

Added tag hs16-b06 for changeset ba313800759b

! .hgtags

Changeset: 36ae83035b8e
Author:    trims
Date:      2010-02-11 20:11 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/36ae83035b8e

Added tag hs16-b07 for changeset 3c0f72981560

! .hgtags

Changeset: 89ef87b378cd
Author:    trims
Date:      2010-02-11 20:11 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/89ef87b378cd

Added tag hs16-b08 for changeset ac59d4e6dae5

! .hgtags

Changeset: cd89ef31a9c8
Author:    trims
Date:      2010-02-11 20:36 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/cd89ef31a9c8

Added tag hs15-b01 for changeset 3f844a28c5f4

! .hgtags

Changeset: 2099657b92a1
Author:    trims
Date:      2010-02-11 20:36 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/2099657b92a1

Added tag hs15-b02 for changeset 1605bb4eb5a7

! .hgtags

Changeset: 9dcad51c5c70
Author:    trims
Date:      2010-02-11 20:37 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/9dcad51c5c70

Added tag hs15-b03 for changeset 2581d90c6c9b

! .hgtags

Changeset: 07118aaebf50
Author:    trims
Date:      2010-02-11 20:37 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/07118aaebf50

Added tag hs15-b04 for changeset 9ab385cb0c42

! .hgtags

Changeset: 3f370a32906e
Author:    trims
Date:      2010-02-11 20:37 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/3f370a32906e

Added tag hs15-b05 for changeset fafab5d5349c

! .hgtags

Changeset: ffc8d176b84b
Author:    mikejwre
Date:      2010-02-12 13:25 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/ffc8d176b84b

Added tag jdk7-b83 for changeset 3f370a32906e

! .hgtags

Changeset: 125eb6a9fccf
Author:    mikejwre
Date:      2010-02-18 13:31 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/125eb6a9fccf

Added tag jdk7-b84 for changeset ffc8d176b84b

! .hgtags

Changeset: 1f341bb67b5b
Author:    trims
Date:      2010-02-18 22:15 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/1f341bb67b5b

Merge


Changeset: 6c9796468b91
Author:    trims
Date:      2010-02-18 22:16 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/6c9796468b91

6927886: Bump the HS17 build number to 10
Summary: Update the HS17 build number to 10
Reviewed-by: jcoomes

! make/hotspot_version

Changeset: 418bc80ce139
Author:    mikejwre
Date:      2010-03-04 13:50 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/418bc80ce139

Added tag jdk7-b85 for changeset 6c9796468b91

! .hgtags

Changeset: bf823ef06b4f
Author:    trims
Date:      2010-03-08 15:50 -0800
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/bf823ef06b4f

Added tag hs17-b10 for changeset 418bc80ce139

! .hgtags

Changeset: 6c94fe3c8df3
Author:    trims
Date:      2010-03-18 16:06 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/6c94fe3c8df3

Merge


Changeset: cc98cc548f51
Author:    apetrusenko
Date:      2010-03-22 02:40 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/cc98cc548f51

Merge

! src/share/vm/includeDB_core
! src/share/vm/runtime/arguments.cpp


From tony.printezis at sun.com  Mon Mar 22 08:35:37 2010
From: tony.printezis at sun.com (Tony Printezis)
Date: Mon, 22 Mar 2010 11:35:37 -0400
Subject: G1 heap growth very aggressive in spite of low heap usage
In-Reply-To: <5a1151761003210612j5cd8c3afh57d94b75b5c24c55@mail.gmail.com>
References: <5a1151761003210612j5cd8c3afh57d94b75b5c24c55@mail.gmail.com>
Message-ID: <4BA78E49.30906@sun.com>

Peter,

Yes, right now G1 expands the heap a bit too aggressively. It checks the 
overall GC overhead and, if it's higher than the goal, it'd expand the 
heap hoping that it will reach the required GC overhead (typically, the 
larger the heap, the lower the GC overhead). I'd guess that any micro 
benchmark that tries to stress the GC (i.e., do a lot of allocations, 
not much else) would cause G1 to expand the heap aggressively, given 
that such benchmarks typically do mostly GC and not much else (can't 
tell whether this is the case for your test as you don't have 
-XX:+PrintGCTimeStamps enabled to see how close together the GCs are).

Setting the max heap size with -Xmx will control how much G1 will expand 
the heap.

Tony

Peter Schuller wrote:
> Hello,
>
> what is the intended behavior of G1 with respect to heap size? I was
> playing around with a fairly simple test[1] designed for something
> else, when I realized that G1 was expanding my heap in ways that I
> find unexpected[2]. Reading g1CollectedHeap.cpp there seems to be
> attempts to adher to MaxHeapFreeRatio and MinHeapFreeRatio. However,
> even when I run with (on purpose) extreme options of 10/25
> minimum/maximum, heap expansion happens as in [2].
>
> My expected behavior would for the heep size to remain within some
> reasonable percentage of actual memory usage (assuming I only specify
> max heap size and leave minimum heap size untouched), and it is a
> desirable behavior in part because concurrent marking only seems to
> trigger as a function of heap usage relative to the heap size. In many
> situations it is desirable to have the JVM heap size be a reasonable
> indicator of actual memory requirements/use. If I am only using a
> fraction of heap size (5% in this case), the delay in concurrent
> marking will mean that my application (unless it *only* generates
> ephemeral garbage) will grow unreasonably untill concurrent marking is
> triggered. After it completes the heap use would go down very very
> significantly. The end result is that (1) I actually need a lot more
> memory than I otherwise would, and (2) it is difficult to monitor the
> real memory requirements of the application.
>
> I do understand that maximum throughput tend to be achieved by
> postponing concurrent marking to maximize yield, but if that was my
> aim, I would specify -Xms == -Xmx. With a small -Xms and a large -Xmx,
> I would want the heap to expand only as "necessary" (though I
> understand that is difficult to define), resulting in a much more
> reasonable trigger threshold for concurrent marking.
>
> The test is a fairly simple web server that i use 'ab' to throw
> traffic at. It spawns a number of threads per requests, each of which
> tries to produce a non-trivial stack depth, and then sleeps for a
> random period. The intent was to use this to try to see what kind of
> overhead was incurred for root set scanning with many threads (though
> few active) during young generation pauses.
>
> I then added some generation of permanent data that I could trigger by
> submitting HTTP requests. The sudden growth seen in [2], when it
> spikes up from 100-200 MB to 3-4 GB, is when I submit some request to
> trigger generation of permanent data. The actual amount is very
> limited however, and as you can see it very quickly spikes the memory
> use far beyond any reasonable 'min free' percentage might be designed
> to.
>
> The options I used for the run from which [2] comes from are:
>
>    -XX:+PrintGC
>    -XX:+UnlockExperimentalVMOptions
>    -XX:+UnlockDiagnosticVMOptions
>    -XX:+UseG1GC
>    -XX:MaxGCPauseMillis=25
>    -XX:GCPauseIntervalMillis=50
>    -XX:+G1ParallelRSetUpdatingEnabled
>    -XX:+G1ParallelRSetScanningEnabled
>    -Xmx4G
>    -XX:G1ConfidencePercent=100
>    -XX:MinHeapFreeRatio=10
>    -XX:MaxHeapFreeRatio=25
>
> This was run on a recent checkout of the bsdport of openjdk7:
>
>    changeset:   192:9f250d0d1b40
>    tag:         tip
>    parent:      187:417d1a0aa480
>    parent:      191:6b1069f53fbc
>    user:        Greg Lewis <glewis at eyesbeyond.com>
>    date:        Sat Mar 20 11:11:42 2010 -0700
>    summary:     Merge from main OpenJDK repository
>
> I can provide an executable jar file if someone wants to test it but
> is not set up to build clojure/leiningen projects.
>
> [1] http://github.com/scode/httpgctest
>
> [2]:
>
> Here is an excerpt. Each pause is 0.25-5 seconds in between depending
> on whether I am generating permanent data at the time.
>
> [GC pause (young) 45M->35M(110M), 0.0482680 secs]
> [GC pause (young) 49M->42M(110M), 0.0428210 secs]
> [GC pause (young) 53M->46M(110M), 0.0238750 secs]
> [GC pause (young) 59M->51M(110M), 0.0312480 secs]
> [GC pause (young) 63M->58M(110M), 0.0404710 secs]
> [GC pause (young) 74M->62M(110M), 0.0262080 secs]
> [GC pause (young) 75M->69M(220M), 0.0509200 secs]
> [GC pause (young) 85M->77M(440M), 0.0478070 secs]
> [GC pause (young) 95M->86M(880M), 0.0560680 secs]
> [GC pause (young) 102M->96M(1524M), 0.0618900 secs]
> [GC pause (young) 112M->102M(2039M), 0.0404470 secs]
> [GC pause (young) 116M->111M(2451M), 0.0546120 secs]
> [GC pause (young) 126M->120M(2780M), 0.0441630 secs]
> [GC pause (young) 137M->121M(3044M), 0.0201850 secs]
> [GC pause (young) 136M->122M(3255M), 0.0113520 secs]
> [GC pause (young) 134M->122M(3424M), 0.0111860 secs]
> [GC pause (young) 133M->122M(3559M), 0.0089910 secs]
> [GC pause (young) 133M->122M(3667M), 0.0079770 secs]
> [GC pause (young) 140M->122M(3753M), 0.0074160 secs]
> [GC pause (young) 144M->136M(3753M), 0.0738920 secs]
> [GC pause (young) 146M->141M(3753M), 0.0363720 secs]
> [GC pause (young) 154M->147M(3753M), 0.0256830 secs]
> [GC pause (young) 158M->148M(3753M), 0.0116360 secs]
> [GC pause (young) 157M->149M(3753M), 0.0172780 secs]
> [GC pause (young) 157M->154M(3753M), 0.0273040 secs]
> [GC pause (young) 168M->163M(3753M), 0.0526970 secs]
> [GC pause (young) 179M->172M(3822M), 0.0547310 secs]
> [GC pause (young) 190M->180M(3877M), 0.0337350 secs]
> [GC pause (young) 195M->189M(3921M), 0.0530290 secs]
> [GC pause (young) 206M->199M(3956M), 0.0598740 secs]
> [GC pause (young) 218M->206M(3984M), 0.0320840 secs]
> [GC pause (young) 222M->207M(4007M), 0.0140790 secs]
> [GC pause (young) 221M->207M(4025M), 0.0084930 secs]
> [GC pause (young) 218M->207M(4040M), 0.0066170 secs]
> [GC pause (young) 219M->207M(4052M), 0.0069570 secs]
> [GC pause (young) 223M->207M(4061M), 0.0067710 secs]
> [GC pause (young) 227M->207M(4061M), 0.0071300 secs]
> [GC pause (young) 231M->207M(4061M), 0.0070520 secs]
>
>
>   
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From peter.schuller at infidyne.com  Mon Mar 22 12:27:02 2010
From: peter.schuller at infidyne.com (Peter Schuller)
Date: Mon, 22 Mar 2010 20:27:02 +0100
Subject: G1 heap growth very aggressive in spite of low heap usage
In-Reply-To: <4BA78E49.30906@sun.com>
References: <5a1151761003210612j5cd8c3afh57d94b75b5c24c55@mail.gmail.com>
	<4BA78E49.30906@sun.com>
Message-ID: <5a1151761003221227h452c9dbaqc39fdf4c7e95ff8c@mail.gmail.com>

Hello,

> Yes, right now G1 expands the heap a bit too aggressively. It checks the
> overall GC overhead and, if it's higher than the goal, it'd expand the heap
> hoping that it will reach the required GC overhead (typically, the larger
> the heap, the lower the GC overhead).

Interesting. Looking at G1CollectorPolicy::expansion_amount(), this
seems to be determined by average pause time ratio.

Based on my understanding of G1, it seems to me that this would be
fundamentally prone to "false positives". What can happen, and seems
to happen in this case, is that purely young GC:s end up triggering
heap growth. This is in spite of the young generation / region count
being determined by the desired pause time (normally) rather than any
lack of heap space.

I don't know what G1 does if there is too little free heap for the
preferred amount of young generation regions, but I would expect that
the primary effect of increasing heap size, in terms of GC overhead,
would be that of:

(1) Decreasing the frequency of concurrent marks and the overhead
associated with it.
(2) Increasing the pay-off of partial GC:s after such a mark, due to
(presumably) a larger average free ratio in selected regions.

But under normal circumstances, when not actually running out of heap
space, would this policy every be expected to be effective in any
significant percentage of cases?

While I can understand that spending too much time on GC means you
want to cut down on GC overhead *in general*, there seems to me to be
a very weak relation between the cost of non-young evacuations
(directly and indirectly through marking) and the cost of the young
generation collections.

If there is an excessive cost to young generation collections due to
excessive promotion into old generations, would not that rather be an
indication that the pause time goal and desired GC overhead are simply
incompatible given the workload of the application?

If so, a way out might be to accept the added overhead (but perhaps
provide diagnostic feedback).

Increasing the young generation size in an attempt to increase
efficiency at the detriment of collection pause time could be an
option, but it would only work if the application exhibits behavior
consistent with the generational hypothesis, so is not a very safe
bet. Probably in an ideal world this would be a knob (which to
prefer).

Ideally, maybe heap expansion would primarily be triggered by high
cost of non-young collections. However I also understand that it is
difficult to impossible to measure the overhead of concurrent marking,
even if the cost of non-young region evacuations could be measured.

Another observation is that it may be advantageous to only expand the
heap when an allocation rate fails (as a result of
G1CollectedHeap::expand_and_allocate() perhaps?), or at least not
until the lack of heap space is in some way affecting the cost of
young generation collections (or e.g. at the start of concurrent
marking where we expect to start incurring costs associated with the
heap size being too small).

Thoughts?

> I'd guess that any micro benchmark
> that tries to stress the GC (i.e., do a lot of allocations, not much else)
> would cause G1 to expand the heap aggressively, given that such benchmarks
> typically do mostly GC and not much else (can't tell whether this is the
> case for your test as you don't have -XX:+PrintGCTimeStamps enabled to see
> how close together the GCs are).

Yes, but this also highlights, I think, an issue with the ratio of
time spent since it specifically does not take into account the
allocation rate of the mutator. For non-young evacuation and
concurrent marking cost that would likely not matter since a larger
heap would always lead to better throughput (and greater GC
efficiency), but because the logic is applied also based on the cost
of young generation collections the effects on heap size are probably
likely to be very non-expected for any application that has temporary
bursts of allocation (which I think is very much realistic even in
production code).

On time stamps: I can provide a sample runt with PrintGCTimeStamps,
but the short answer is that they happened relatively frequently
(several times per second) and the growth exhibited was not preceeded
by a concurrent mark or non-young evacuations.

> Setting the max heap size with -Xmx will control how much G1 will expand the
> heap.

Understood. However for general-purpose use I am very interested in
seeing the JVM self-regulate it's memory use in such a way that a
non-developer can look at the memory use of a JVM (or for that matter
the heap free/total:s) and draw some kind of reasonable ballpark
conclusion on memory demands.

Currently one can fairly easily trigger extreme heap growth to the
point of multiple orders of magnitude. And since concurrent marking
and non-young evacuations won't happen until the heap size is
significantly exhausted, that effectively means that your program may
end up seemingly "needing" orders of magnitude more memory than what
it actually does.

-- 
/ Peter Schuller
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From peter.schuller at infidyne.com  Mon Mar 22 13:53:52 2010
From: peter.schuller at infidyne.com (Peter Schuller)
Date: Mon, 22 Mar 2010 21:53:52 +0100
Subject: G1 heap growth very aggressive in spite of low heap usage
In-Reply-To: <5a1151761003221227h452c9dbaqc39fdf4c7e95ff8c@mail.gmail.com>
References: <5a1151761003210612j5cd8c3afh57d94b75b5c24c55@mail.gmail.com>
	<4BA78E49.30906@sun.com>
	<5a1151761003221227h452c9dbaqc39fdf4c7e95ff8c@mail.gmail.com>
Message-ID: <5a1151761003221353y5d312b7br40afbe6454274d42@mail.gmail.com>

>> Yes, right now G1 expands the heap a bit too aggressively. It checks the
>> overall GC overhead and, if it's higher than the goal, it'd expand the heap
>> hoping that it will reach the required GC overhead (typically, the larger
>> the heap, the lower the GC overhead).
>
> Interesting. Looking at G1CollectorPolicy::expansion_amount(), this
> seems to be determined by average pause time ratio.

For the record, changing G1GCPercent to 100 (from the default of 10)
did indeed eliminate this behavior.

(Should this option be enabled in non-development builds so that it
can be tweaked on an out-of-the-box JVM?)

-- 
/ Peter Schuller
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From tony.printezis at sun.com  Mon Mar 22 15:02:01 2010
From: tony.printezis at sun.com (Tony Printezis)
Date: Mon, 22 Mar 2010 18:02:01 -0400
Subject: G1 heap growth very aggressive in spite of low heap usage
In-Reply-To: <5a1151761003221227h452c9dbaqc39fdf4c7e95ff8c@mail.gmail.com>
References: <5a1151761003210612j5cd8c3afh57d94b75b5c24c55@mail.gmail.com>
	<4BA78E49.30906@sun.com>
	<5a1151761003221227h452c9dbaqc39fdf4c7e95ff8c@mail.gmail.com>
Message-ID: <4BA7E8D9.3020200@sun.com>

Peter,

Peter Schuller wrote:
> I don't know what G1 does if there is too little free heap for the
> preferred amount of young generation regions, but I would expect that
> the primary effect of increasing heap size, in terms of GC overhead,
> would be that of:
>
> (1) Decreasing the frequency of concurrent marks and the overhead
> associated with it.
> (2) Increasing the pay-off of partial GC:s after such a mark, due to
> (presumably) a larger average free ratio in selected regions.
>   
There's another advantage of increasing heap size. The larger the heap 
size is, the larger the young gen can grow (assuming the prediction 
heuristics determine that the pause times will not go over the desired 
goal) and, as a result, the less frequent collections will be. This will 
also decrease the overall GC overhead.
> But under normal circumstances, when not actually running out of heap
> space, would this policy every be expected to be effective in any
> significant percentage of cases?
>   
Yes. See above. But, I agree that it's a bit too aggressive as it is 
right now.

Tony
> While I can understand that spending too much time on GC means you
> want to cut down on GC overhead *in general*, there seems to me to be
> a very weak relation between the cost of non-young evacuations
> (directly and indirectly through marking) and the cost of the young
> generation collections.
>
> If there is an excessive cost to young generation collections due to
> excessive promotion into old generations, would not that rather be an
> indication that the pause time goal and desired GC overhead are simply
> incompatible given the workload of the application?
>
> If so, a way out might be to accept the added overhead (but perhaps
> provide diagnostic feedback).
>
> Increasing the young generation size in an attempt to increase
> efficiency at the detriment of collection pause time could be an
> option, but it would only work if the application exhibits behavior
> consistent with the generational hypothesis, so is not a very safe
> bet. Probably in an ideal world this would be a knob (which to
> prefer).
>
> Ideally, maybe heap expansion would primarily be triggered by high
> cost of non-young collections. However I also understand that it is
> difficult to impossible to measure the overhead of concurrent marking,
> even if the cost of non-young region evacuations could be measured.
>
> Another observation is that it may be advantageous to only expand the
> heap when an allocation rate fails (as a result of
> G1CollectedHeap::expand_and_allocate() perhaps?), or at least not
> until the lack of heap space is in some way affecting the cost of
> young generation collections (or e.g. at the start of concurrent
> marking where we expect to start incurring costs associated with the
> heap size being too small).
>
> Thoughts?
>
>   
>> I'd guess that any micro benchmark
>> that tries to stress the GC (i.e., do a lot of allocations, not much else)
>> would cause G1 to expand the heap aggressively, given that such benchmarks
>> typically do mostly GC and not much else (can't tell whether this is the
>> case for your test as you don't have -XX:+PrintGCTimeStamps enabled to see
>> how close together the GCs are).
>>     
>
> Yes, but this also highlights, I think, an issue with the ratio of
> time spent since it specifically does not take into account the
> allocation rate of the mutator. For non-young evacuation and
> concurrent marking cost that would likely not matter since a larger
> heap would always lead to better throughput (and greater GC
> efficiency), but because the logic is applied also based on the cost
> of young generation collections the effects on heap size are probably
> likely to be very non-expected for any application that has temporary
> bursts of allocation (which I think is very much realistic even in
> production code).
>
> On time stamps: I can provide a sample runt with PrintGCTimeStamps,
> but the short answer is that they happened relatively frequently
> (several times per second) and the growth exhibited was not preceeded
> by a concurrent mark or non-young evacuations.
>
>   
>> Setting the max heap size with -Xmx will control how much G1 will expand the
>> heap.
>>     
>
> Understood. However for general-purpose use I am very interested in
> seeing the JVM self-regulate it's memory use in such a way that a
> non-developer can look at the memory use of a JVM (or for that matter
> the heap free/total:s) and draw some kind of reasonable ballpark
> conclusion on memory demands.
>
> Currently one can fairly easily trigger extreme heap growth to the
> point of multiple orders of magnitude. And since concurrent marking
> and non-young evacuations won't happen until the heap size is
> significantly exhausted, that effectively means that your program may
> end up seemingly "needing" orders of magnitude more memory than what
> it actually does.
>
>   
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From tony.printezis at sun.com  Mon Mar 22 15:15:34 2010
From: tony.printezis at sun.com (Tony Printezis)
Date: Mon, 22 Mar 2010 18:15:34 -0400
Subject: G1 heap growth very aggressive in spite of low heap usage
In-Reply-To: <5a1151761003221353y5d312b7br40afbe6454274d42@mail.gmail.com>
References: <5a1151761003210612j5cd8c3afh57d94b75b5c24c55@mail.gmail.com>
	<4BA78E49.30906@sun.com>
	<5a1151761003221227h452c9dbaqc39fdf4c7e95ff8c@mail.gmail.com>
	<5a1151761003221353y5d312b7br40afbe6454274d42@mail.gmail.com>
Message-ID: <4BA7EC06.9090300@sun.com>

Peter,

We have recently made some changes to G1 to re-use existing cmd line 
parameters where appropriate instead of introducing new ones. In this 
case I think we should actually observe GCTimeRatio instead of making 
G1GCPercent a product flag. I opened "6937160: G1: should observe 
GCTimeRatio" to track this.

Tony

Peter Schuller wrote:
>>> Yes, right now G1 expands the heap a bit too aggressively. It checks the
>>> overall GC overhead and, if it's higher than the goal, it'd expand the heap
>>> hoping that it will reach the required GC overhead (typically, the larger
>>> the heap, the lower the GC overhead).
>>>       
>> Interesting. Looking at G1CollectorPolicy::expansion_amount(), this
>> seems to be determined by average pause time ratio.
>>     
>
> For the record, changing G1GCPercent to 100 (from the default of 10)
> did indeed eliminate this behavior.
>
> (Should this option be enabled in non-development builds so that it
> can be tweaked on an out-of-the-box JVM?)
>
>   
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From antonios.printezis at sun.com  Tue Mar 23 08:19:18 2010
From: antonios.printezis at sun.com (antonios.printezis at sun.com)
Date: Tue, 23 Mar 2010 15:19:18 +0000
Subject: hg: jdk7/hotspot-gc/hotspot: 6935821: G1: threads created during
	marking do not active their SATB queues
Message-ID: <20100323151939.E39DF447E1@hg.openjdk.java.net>

Changeset: d4197f8d516a
Author:    tonyp
Date:      2010-03-18 12:14 -0400
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/hotspot/rev/d4197f8d516a

6935821: G1: threads created during marking do not active their SATB queues
Summary: Newly-created threads always had the active field of their SATB queue initialized to false, even if they were created during marking. As a result, updates from threads created during a marking cycle were never enqueued and never processed. The fix includes remaining a method from active() to is_active() for readability and naming consistency.
Reviewed-by: ysr, johnc

! src/share/vm/gc_implementation/g1/concurrentMark.cpp
! src/share/vm/gc_implementation/g1/g1SATBCardTableModRefBS.cpp
! src/share/vm/gc_implementation/g1/ptrQueue.cpp
! src/share/vm/gc_implementation/g1/ptrQueue.hpp
! src/share/vm/gc_implementation/g1/satbQueue.cpp
! src/share/vm/gc_implementation/g1/satbQueue.hpp


From linuxhippy at gmail.com  Tue Mar 23 08:21:30 2010
From: linuxhippy at gmail.com (Clemens Eisserer)
Date: Tue, 23 Mar 2010 16:21:30 +0100
Subject: Heap Size in 32 bit Redhat
In-Reply-To: <f5b533001003230257y4ba9eb20he5dc3474ad4b9b5a@mail.gmail.com>
References: <f5b533001003230257y4ba9eb20he5dc3474ad4b9b5a@mail.gmail.com>
Message-ID: <194f62551003230821h1cd4b6f5mb73f89ae70bbebbc@mail.gmail.com>

Hi,

As far as I know there's still the 3GB/1GB user/kernel-space split
even when using PAE.
PAE doesn't expand the memory available per-process, but enables the
OS to use up to 64GB with each process still limited to its 32-bit
adress range (with some percentage reserved for the kernel).

I would suggest using 64-bit JVM with CompressedOops, usually that
performs very good :)

- Clemens
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From guanxiaohua at gmail.com  Tue Mar 23 10:57:57 2010
From: guanxiaohua at gmail.com (Tony Guan)
Date: Tue, 23 Mar 2010 12:57:57 -0500
Subject: Avoid collecting young gen while doing the full GC
Message-ID: <2fcb552b1003231057q600ec43bla3f7e86b9623ffb@mail.gmail.com>

Hi all,

In my current project, one crazy idea is to leave the young generation
untouched(not collected) during a Full Gc. Is it feasible if I want to
collect the tenured gen only in a full GC?

To do this, the young gen should be included as part of the root set,
and the reference processor should only care about the old gen. Is
there anything else I should take care of?

Thanks!

Tony

From Y.S.Ramakrishna at Sun.COM  Tue Mar 23 12:27:12 2010
From: Y.S.Ramakrishna at Sun.COM (Y. Srinivas Ramakrishna)
Date: Tue, 23 Mar 2010 12:27:12 -0700
Subject: Avoid collecting young gen while doing the full GC
In-Reply-To: <2fcb552b1003231057q600ec43bla3f7e86b9623ffb@mail.gmail.com>
References: <2fcb552b1003231057q600ec43bla3f7e86b9623ffb@mail.gmail.com>
Message-ID: <4BA91610.8010009@Sun.COM>

Hi Tony --

On 03/23/10 10:57, Tony Guan wrote:
> Hi all,
> 
> In my current project, one crazy idea is to leave the young generation
> untouched(not collected) during a Full Gc. Is it feasible if I want to
> collect the tenured gen only in a full GC?
> 
> To do this, the young gen should be included as part of the root set,
> and the reference processor should only care about the old gen. Is
> there anything else I should take care of?

Yes, i believe that should work. (In fact, the CMS collector, which collects
only the old gen and perm gen, does pretty much that. There are some disadvantages
to it though, for example, dead objects in the young gen possibly keeping
objects in old and perm gen artificially alive, but such cases are probably
rare.)

Just out of curiosity, when and wny would you want to not collect the young gen?

-- ramki

> 
> Thanks!
> 
> Tony


From tony.printezis at sun.com  Tue Mar 23 12:44:41 2010
From: tony.printezis at sun.com (Tony Printezis)
Date: Tue, 23 Mar 2010 15:44:41 -0400
Subject: Avoid collecting young gen while doing the full GC
In-Reply-To: <4BA91610.8010009@Sun.COM>
References: <2fcb552b1003231057q600ec43bla3f7e86b9623ffb@mail.gmail.com>
	<4BA91610.8010009@Sun.COM>
Message-ID: <4BA91A29.2080004@sun.com>

Tony,

You also have to make sure that the card table (the data structure that 
keeps track of old-to-young refs) is left consistent at the end of the 
GCs. So, as you move old objects that point to the young gen, you might 
have to update the card table accordingly. An easy way to do that (good 
to at least make sure that the rest of the implementation is working 
correctly) is to dirty the cards of the entire old gen. This will make 
the next young GCs really slow, but the young collector should deal with 
it correctly and clean up the card table as necessary.

I'm also curious why you want to do this. My wild guess would be that 
you're maybe using a huge young gen, relatively to the old gen, and as a 
result it's better to let the young collector deal with it rather than 
compacting it (which would be much slower).

Tony

Y. Srinivas Ramakrishna wrote:
> Hi Tony --
>
> On 03/23/10 10:57, Tony Guan wrote:
>> Hi all,
>>
>> In my current project, one crazy idea is to leave the young generation
>> untouched(not collected) during a Full Gc. Is it feasible if I want to
>> collect the tenured gen only in a full GC?
>>
>> To do this, the young gen should be included as part of the root set,
>> and the reference processor should only care about the old gen. Is
>> there anything else I should take care of?
>
> Yes, i believe that should work. (In fact, the CMS collector, which 
> collects
> only the old gen and perm gen, does pretty much that. There are some 
> disadvantages
> to it though, for example, dead objects in the young gen possibly keeping
> objects in old and perm gen artificially alive, but such cases are 
> probably
> rare.)
>
> Just out of curiosity, when and wny would you want to not collect the 
> young gen?
>
> -- ramki
>
>>
>> Thanks!
>>
>> Tony
>

From guanxiaohua at gmail.com  Tue Mar 23 12:45:59 2010
From: guanxiaohua at gmail.com (Tony Guan)
Date: Tue, 23 Mar 2010 14:45:59 -0500
Subject: Avoid collecting young gen while doing the full GC
In-Reply-To: <4BA91610.8010009@Sun.COM>
References: <2fcb552b1003231057q600ec43bla3f7e86b9623ffb@mail.gmail.com> 
	<4BA91610.8010009@Sun.COM>
Message-ID: <2fcb552b1003231245l72919799lfd1acbbdd64530f3@mail.gmail.com>

Hi Ramki,

Thanks!

In my study, I want to reclaim some clustered objects all at once
instead of relying on regular GCs. To achieve this goal, I need to
reserve  in the heap some space which is free of garbage collections.
Does this idea sound familiar to you or just another crazy one?

Actually, I am stuck somewhere implementing it. I know what to do
generally, but down to the code, I am really lost.

I tried not to expand the span of the ref processor, so refs discovery
is over just the old generation in the method
OneContigSpaceCardGeneration::collect(), but this doesn't do the
trick, a full GC will still collect everything. So I just keep staring
at the code currently.

Do you have any hint for me?

Thanks!

Tony



On Tue, Mar 23, 2010 at 2:27 PM, Y. Srinivas Ramakrishna
<Y.S.Ramakrishna at sun.com> wrote:
> Hi Tony --
>
> On 03/23/10 10:57, Tony Guan wrote:
>>
>> Hi all,
>>
>> In my current project, one crazy idea is to leave the young generation
>> untouched(not collected) during a Full Gc. Is it feasible if I want to
>> collect the tenured gen only in a full GC?
>>
>> To do this, the young gen should be included as part of the root set,
>> and the reference processor should only care about the old gen. Is
>> there anything else I should take care of?
>
> Yes, i believe that should work. (In fact, the CMS collector, which collects
> only the old gen and perm gen, does pretty much that. There are some
> disadvantages
> to it though, for example, dead objects in the young gen possibly keeping
> objects in old and perm gen artificially alive, but such cases are probably
> rare.)
>
> Just out of curiosity, when and wny would you want to not collect the young
> gen?
>
> -- ramki
>
>>
>> Thanks!
>>
>> Tony
>
>

From guanxiaohua at gmail.com  Tue Mar 23 12:59:13 2010
From: guanxiaohua at gmail.com (Tony Guan)
Date: Tue, 23 Mar 2010 14:59:13 -0500
Subject: Avoid collecting young gen while doing the full GC
In-Reply-To: <4BA91A29.2080004@sun.com>
References: <2fcb552b1003231057q600ec43bla3f7e86b9623ffb@mail.gmail.com> 
	<4BA91610.8010009@Sun.COM> <4BA91A29.2080004@sun.com>
Message-ID: <2fcb552b1003231259j1f51e9e0q380810eeea1e0d4b@mail.gmail.com>

Hi Big Tony,

Thank you for the reply.  As explained by my previous email to Ramki,
I need to reserve one space to store some special objects for a
once-for-all wipeout. After I disabled the space extension in the
reference processor, I am really waiting for some exceptions from the
assertions(either from the card marking or null pointer exception),
but to disappoint me, the full GC just finish without any problem, and
the young gen is still collected during a full GC.

This is a rare case that a developer will expect some exceptions from
his program.

And many thanks to Ramki, I will take a look at the CMS collector for a clue.

Thanks!

Tony



On Tue, Mar 23, 2010 at 2:44 PM, Tony Printezis <tony.printezis at sun.com> wrote:
> Tony,
>
> You also have to make sure that the card table (the data structure that
> keeps track of old-to-young refs) is left consistent at the end of the GCs.
> So, as you move old objects that point to the young gen, you might have to
> update the card table accordingly. An easy way to do that (good to at least
> make sure that the rest of the implementation is working correctly) is to
> dirty the cards of the entire old gen. This will make the next young GCs
> really slow, but the young collector should deal with it correctly and clean
> up the card table as necessary.
>
> I'm also curious why you want to do this. My wild guess would be that you're
> maybe using a huge young gen, relatively to the old gen, and as a result
> it's better to let the young collector deal with it rather than compacting
> it (which would be much slower).
>
> Tony
>
> Y. Srinivas Ramakrishna wrote:
>>
>> Hi Tony --
>>
>> On 03/23/10 10:57, Tony Guan wrote:
>>>
>>> Hi all,
>>>
>>> In my current project, one crazy idea is to leave the young generation
>>> untouched(not collected) during a Full Gc. Is it feasible if I want to
>>> collect the tenured gen only in a full GC?
>>>
>>> To do this, the young gen should be included as part of the root set,
>>> and the reference processor should only care about the old gen. Is
>>> there anything else I should take care of?
>>
>> Yes, i believe that should work. (In fact, the CMS collector, which
>> collects
>> only the old gen and perm gen, does pretty much that. There are some
>> disadvantages
>> to it though, for example, dead objects in the young gen possibly keeping
>> objects in old and perm gen artificially alive, but such cases are
>> probably
>> rare.)
>>
>> Just out of curiosity, when and wny would you want to not collect the
>> young gen?
>>
>> -- ramki
>>
>>>
>>> Thanks!
>>>
>>> Tony
>>
>

From Y.S.Ramakrishna at Sun.COM  Tue Mar 23 18:06:46 2010
From: Y.S.Ramakrishna at Sun.COM (Y. Srinivas Ramakrishna)
Date: Tue, 23 Mar 2010 18:06:46 -0700
Subject: Avoid collecting young gen while doing the full GC
In-Reply-To: <4BA91A29.2080004@sun.com>
References: <2fcb552b1003231057q600ec43bla3f7e86b9623ffb@mail.gmail.com>
	<4BA91610.8010009@Sun.COM> <4BA91A29.2080004@sun.com>
Message-ID: <4BA965A6.10004@sun.com>

Good point. You'd think that the current compacting collectors would
do this (either in the pointer update phase or as you suggest the short-cut
of conditionally dirtying the entire old and perm gens) if the compacted
data did not all fit in the old gen and some of it overflowed into
young. I haven't looked at the code in a while, and I wonder which
(if either, yikes!) is actually done. I'll take a look when i get
some free time unless someone else gets to it first ;-)

-- ramki

Tony Printezis wrote:
> Tony,
> 
> You also have to make sure that the card table (the data structure that 
> keeps track of old-to-young refs) is left consistent at the end of the 
> GCs. So, as you move old objects that point to the young gen, you might 
> have to update the card table accordingly. An easy way to do that (good 
> to at least make sure that the rest of the implementation is working 
> correctly) is to dirty the cards of the entire old gen. This will make 
> the next young GCs really slow, but the young collector should deal with 
> it correctly and clean up the card table as necessary.
> 
> I'm also curious why you want to do this. My wild guess would be that 
> you're maybe using a huge young gen, relatively to the old gen, and as a 
> result it's better to let the young collector deal with it rather than 
> compacting it (which would be much slower).
> 
> Tony
> 
> Y. Srinivas Ramakrishna wrote:
>> Hi Tony --
>>
>> On 03/23/10 10:57, Tony Guan wrote:
>>> Hi all,
>>>
>>> In my current project, one crazy idea is to leave the young generation
>>> untouched(not collected) during a Full Gc. Is it feasible if I want to
>>> collect the tenured gen only in a full GC?
>>>
>>> To do this, the young gen should be included as part of the root set,
>>> and the reference processor should only care about the old gen. Is
>>> there anything else I should take care of?
>>
>> Yes, i believe that should work. (In fact, the CMS collector, which 
>> collects
>> only the old gen and perm gen, does pretty much that. There are some 
>> disadvantages
>> to it though, for example, dead objects in the young gen possibly keeping
>> objects in old and perm gen artificially alive, but such cases are 
>> probably
>> rare.)
>>
>> Just out of curiosity, when and wny would you want to not collect the 
>> young gen?
>>
>> -- ramki
>>
>>>
>>> Thanks!
>>>
>>> Tony
>>


From Y.S.Ramakrishna at Sun.COM  Tue Mar 23 18:12:02 2010
From: Y.S.Ramakrishna at Sun.COM (Y. Srinivas Ramakrishna)
Date: Tue, 23 Mar 2010 18:12:02 -0700
Subject: Avoid collecting young gen while doing the full GC
In-Reply-To: <4BA965A6.10004@sun.com>
References: <2fcb552b1003231057q600ec43bla3f7e86b9623ffb@mail.gmail.com>
	<4BA91610.8010009@Sun.COM> <4BA91A29.2080004@sun.com>
	<4BA965A6.10004@sun.com>
Message-ID: <4BA966E2.4020109@sun.com>

I just checked, and ...

Y. Srinivas Ramakrishna wrote:
> Good point. You'd think that the current compacting collectors would
> do this (either in the pointer update phase or as you suggest the short-cut
> of conditionally dirtying the entire old and perm gens) if the compacted
> data did not all fit in the old gen and some of it overflowed into
> young. I haven't looked at the code in a while, and I wonder which

... (Serial) mark-compact (at least) does the latter. (Probably makes sense in most
conventional scenarios where overflow is rare.) John if you are reading this
perhaps you can tell us what parallel compact does? (Or i'll eventually get around to
looking at it :-)

-- ramki

> (if either, yikes!) is actually done. I'll take a look when i get
> some free time unless someone else gets to it first ;-)
> 
> -- ramki
> 
> Tony Printezis wrote:
>> Tony,
>>
>> You also have to make sure that the card table (the data structure 
>> that keeps track of old-to-young refs) is left consistent at the end 
>> of the GCs. So, as you move old objects that point to the young gen, 
>> you might have to update the card table accordingly. An easy way to do 
>> that (good to at least make sure that the rest of the implementation 
>> is working correctly) is to dirty the cards of the entire old gen. 
>> This will make the next young GCs really slow, but the young collector 
>> should deal with it correctly and clean up the card table as necessary.
>>
>> I'm also curious why you want to do this. My wild guess would be that 
>> you're maybe using a huge young gen, relatively to the old gen, and as 
>> a result it's better to let the young collector deal with it rather 
>> than compacting it (which would be much slower).
>>
>> Tony
>>
>> Y. Srinivas Ramakrishna wrote:
>>> Hi Tony --
>>>
>>> On 03/23/10 10:57, Tony Guan wrote:
>>>> Hi all,
>>>>
>>>> In my current project, one crazy idea is to leave the young generation
>>>> untouched(not collected) during a Full Gc. Is it feasible if I want to
>>>> collect the tenured gen only in a full GC?
>>>>
>>>> To do this, the young gen should be included as part of the root set,
>>>> and the reference processor should only care about the old gen. Is
>>>> there anything else I should take care of?
>>>
>>> Yes, i believe that should work. (In fact, the CMS collector, which 
>>> collects
>>> only the old gen and perm gen, does pretty much that. There are some 
>>> disadvantages
>>> to it though, for example, dead objects in the young gen possibly 
>>> keeping
>>> objects in old and perm gen artificially alive, but such cases are 
>>> probably
>>> rare.)
>>>
>>> Just out of curiosity, when and wny would you want to not collect the 
>>> young gen?
>>>
>>> -- ramki
>>>
>>>>
>>>> Thanks!
>>>>
>>>> Tony
>>>
> 


From Y.S.Ramakrishna at Sun.COM  Tue Mar 23 18:16:46 2010
From: Y.S.Ramakrishna at Sun.COM (Y. Srinivas Ramakrishna)
Date: Tue, 23 Mar 2010 18:16:46 -0700
Subject: Avoid collecting young gen while doing the full GC
In-Reply-To: <4BA966E2.4020109@sun.com>
References: <2fcb552b1003231057q600ec43bla3f7e86b9623ffb@mail.gmail.com>
	<4BA91610.8010009@Sun.COM> <4BA91A29.2080004@sun.com>
	<4BA965A6.10004@sun.com> <4BA966E2.4020109@sun.com>
Message-ID: <4BA967FE.404@sun.com>

Parallel Compact does the same.

Y. Srinivas Ramakrishna wrote:
> I just checked, and ...
> 
> Y. Srinivas Ramakrishna wrote:
>> Good point. You'd think that the current compacting collectors would
>> do this (either in the pointer update phase or as you suggest the 
>> short-cut
>> of conditionally dirtying the entire old and perm gens) if the compacted
>> data did not all fit in the old gen and some of it overflowed into
>> young. I haven't looked at the code in a while, and I wonder which
> 
> ... (Serial) mark-compact (at least) does the latter. (Probably makes 
> sense in most
> conventional scenarios where overflow is rare.) John if you are reading 
> this
> perhaps you can tell us what parallel compact does? (Or i'll eventually 
> get around to
> looking at it :-)
> 
> -- ramki
> 
>> (if either, yikes!) is actually done. I'll take a look when i get
>> some free time unless someone else gets to it first ;-)
>>
>> -- ramki
>>
>> Tony Printezis wrote:
>>> Tony,
>>>
>>> You also have to make sure that the card table (the data structure 
>>> that keeps track of old-to-young refs) is left consistent at the end 
>>> of the GCs. So, as you move old objects that point to the young gen, 
>>> you might have to update the card table accordingly. An easy way to 
>>> do that (good to at least make sure that the rest of the 
>>> implementation is working correctly) is to dirty the cards of the 
>>> entire old gen. This will make the next young GCs really slow, but 
>>> the young collector should deal with it correctly and clean up the 
>>> card table as necessary.
>>>
>>> I'm also curious why you want to do this. My wild guess would be that 
>>> you're maybe using a huge young gen, relatively to the old gen, and 
>>> as a result it's better to let the young collector deal with it 
>>> rather than compacting it (which would be much slower).
>>>
>>> Tony
>>>
>>> Y. Srinivas Ramakrishna wrote:
>>>> Hi Tony --
>>>>
>>>> On 03/23/10 10:57, Tony Guan wrote:
>>>>> Hi all,
>>>>>
>>>>> In my current project, one crazy idea is to leave the young generation
>>>>> untouched(not collected) during a Full Gc. Is it feasible if I want to
>>>>> collect the tenured gen only in a full GC?
>>>>>
>>>>> To do this, the young gen should be included as part of the root set,
>>>>> and the reference processor should only care about the old gen. Is
>>>>> there anything else I should take care of?
>>>>
>>>> Yes, i believe that should work. (In fact, the CMS collector, which 
>>>> collects
>>>> only the old gen and perm gen, does pretty much that. There are some 
>>>> disadvantages
>>>> to it though, for example, dead objects in the young gen possibly 
>>>> keeping
>>>> objects in old and perm gen artificially alive, but such cases are 
>>>> probably
>>>> rare.)
>>>>
>>>> Just out of curiosity, when and wny would you want to not collect 
>>>> the young gen?
>>>>
>>>> -- ramki
>>>>
>>>>>
>>>>> Thanks!
>>>>>
>>>>> Tony
>>>>
>>
> 


From Y.S.Ramakrishna at Sun.COM  Wed Mar 24 10:13:04 2010
From: Y.S.Ramakrishna at Sun.COM (Y. Srinivas Ramakrishna)
Date: Wed, 24 Mar 2010 10:13:04 -0700
Subject: Strange long pauses between 2 Young GCs
In-Reply-To: <op.u92wjoz4a5r2ku@nb07-spt1>
References: <op.u92wjoz4a5r2ku@nb07-spt1>
Message-ID: <4BAA4820.7050805@sun.com>

David Tavoularis wrote:
> Hi,
> 
> In my application, I noticed very strange pauses (15s+19s+25s+30s+59s+28s = 
> 2min56) between 2 Young GCs (on Java6u13 64-bits / Solaris 10) with 

Those are not pauses of the application. Rather those are the durations for
which the application runs (is not paused). The pauses are the short ones
in between those. They are likely related to bias lock revocations or
deoptimization or something. If you use -XX:+PrintSafepointStatistics (especially
with a latest hs18 jvm) you would know what those very short stoppages
in between are for.

> ParallelGC+ParallelOldGC collectors.
> Has anyone encountered this kind of pauses and has an explanation (see extract 
> of GC logs at the end of the mail) ?
> 
> My issue is that some JMS messages have been sent to DeadMessageQueue because 
> the process was not "responding".

Hmm, that seems strange. What does mpstat/prstat indicate during those times?
You might want to run this on Sun Studio Collector (perf anakyzer) and
see if it shows up the issues? If this is on a production system, and
this occurs at some time that lasts a while, you may be able to collect the
performance data during that period of time.

But first a simple mpstat/prstat data may be useful before a SS collector experiment.

-- ramki

> 
> I do not think that my server was swapping (30GB free RAM) and usually, a Full 
> GC takes between 2s and 10s (WallTime="real"), between 3s and 60s (CpuTime="user")
> 
> Thanks in advance
> 
> -- 
> David
> 
> JVM options :
>    -server
>    -Xms13000m
>    -Xmx13000m
>    -XX:+UseParallelGC
>    -XX:+AggressiveHeap
>    -XX:GCHeapFreeLimit=5
>    -XX:+PrintGCDetails
>    -XX:+PrintGCTimeStamps
>    -XX:+PrintHeapAtGC
>    -XX:+PrintTenuringDistribution
>    -XX:+PrintGCApplicationStoppedTime
>    -XX:+PrintGCApplicationConcurrentTime
>    -XX:+PrintGCDateStamps
>    -Xloggc:/path/to/gc/logs
>    -XX:+HeapDumpOnOutOfMemoryError
>    -XX:HeapDumpPath=/path/to/heap/dumps
>    -XX:NewSize=2048m
>    -XX:MaxNewSize=2048m
>    -XX:+UseParallelOldGC
>    -XX:MaxPermSize=256m
> 
> 2010-03-24T*05:54:23*.467+0100: 47058.915: [GC
> Desired survivor size 181665792 bytes, new threshold 1 (max 15)
> [PSYoungGen: 1792448K->16384K(1763776K)] 2472857K->702761K(12978624K), 0.1969989 
> secs] [Times: user=0.29 sys=0.07, real=0.20 secs]
> Heap after GC invocations=3606 (full 0):
> PSYoungGen total 1763776K, used 16384K [0xfffffffeec800000, 0xffffffff6c800000, 
> 0xffffffff6c800000)
> eden space 1747392K, 0% used 
> [0xfffffffeec800000,0xfffffffeec800000,0xffffffff57270000)
> from space 16384K, 100% used 
> [0xffffffff57270000,0xffffffff58270000,0xffffffff58270000)
> to space 177408K, 0% used [0xffffffff61ac0000,0xffffffff61ac0000,0xffffffff6c800000)
> ParOldGen total 11214848K, used 686377K [0xfffffffc40000000, 0xfffffffeec800000, 
> 0xfffffffeec800000)
> object space 11214848K, 6% used 
> [0xfffffffc40000000,0xfffffffc69e4a480,0xfffffffeec800000)
> PSPermGen total 73728K, used 56019K [0xfffffffc30000000, 0xfffffffc34800000, 
> 0xfffffffc40000000)
> object space 73728K, 75% used 
> [0xfffffffc30000000,0xfffffffc336b4c30,0xfffffffc34800000)
> }
> Total time for which application threads were stopped: 0.2054411 seconds
> Application time: 2.1629642 seconds
> Total time for which application threads were stopped: 0.0089378 seconds
> Application time: 0.0000853 seconds
> Total time for which application threads were stopped: 0.0012767 seconds
> Application time: 0.0001092 seconds
> Total time for which application threads were stopped: 0.0010313 seconds
> Application time: 0.0000721 seconds
> Total time for which application threads were stopped: 0.0010318 seconds
> Application time: 0.3016250 seconds
> Total time for which application threads were stopped: 0.0087502 seconds
> *Application time: 15.0009372 seconds*
> Total time for which application threads were stopped: 0.0074670 seconds
> *Application time: 18.9151668 seconds*
> Total time for which application threads were stopped: 0.0230399 seconds
> Application time: 0.0001326 seconds
> Total time for which application threads were stopped: 0.0012976 seconds
> Application time: 0.0000646 seconds
> Total time for which application threads were stopped: 0.0010412 seconds
> *Application time: 25.4543868 seconds*
> Total time for which application threads were stopped: 0.0087742 seconds
> Application time: 0.0001073 seconds
> Total time for which application threads were stopped: 0.0013600 seconds
> Application time: 0.0001158 seconds
> Total time for which application threads were stopped: 0.0049206 seconds
> Application time: 0.0025729 seconds
> Total time for which application threads were stopped: 0.0012530 seconds
> Application time: 0.0001156 seconds
> Total time for which application threads were stopped: 0.0009663 seconds
> Application time: 0.0007467 seconds
> Total time for which application threads were stopped: 0.0010364 seconds
> Application time: 0.0001326 seconds
> Total time for which application threads were stopped: 0.0009763 seconds
> *Application time: 29.9838922 seconds*
> Total time for which application threads were stopped: 0.0075727 seconds
> Application time: 4.4557342 seconds
> Total time for which application threads were stopped: 0.0187998 seconds
> Application time: 0.0000837 seconds
> Total time for which application threads were stopped: 0.0012345 seconds
> Application time: 0.0000634 seconds
> Total time for which application threads were stopped: 0.0010140 seconds
> *Application time: 59.1763410 seconds*
> Total time for which application threads were stopped: 0.0198018 seconds
> Application time: 0.0001009 seconds
> Total time for which application threads were stopped: 0.0011658 seconds
> Application time: 0.0000614 seconds
> Total time for which application threads were stopped: 0.0010269 seconds
> *Application time: 27.9028562 seconds*
> Total time for which application threads were stopped: 0.0080420 seconds
> Application time: 0.0001698 seconds
> Total time for which application threads were stopped: 0.0012317 seconds
> Application time: 0.1080374 seconds
> Total time for which application threads were stopped: 0.0073232 seconds
> Application time: 0.1215694 seconds
> Total time for which application threads were stopped: 0.0078813 seconds
> Application time: 0.0000801 seconds
> Total time for which application threads were stopped: 0.0014642 seconds
> Application time: 0.0001074 seconds
> Total time for which application threads were stopped: 0.0012944 seconds
> Application time: 0.0000676 seconds
> Total time for which application threads were stopped: 0.0012189 seconds
> Application time: 0.0192653 seconds
> Total time for which application threads were stopped: 0.0021554 seconds
> Application time: 0.0001444 seconds
> Total time for which application threads were stopped: 0.0013011 seconds
> Application time: 0.2620606 seconds
> Total time for which application threads were stopped: 0.0034052 seconds
> Application time: 0.8994294 seconds
> {Heap before GC invocations=3607 (full 0):
> PSYoungGen total 1763776K, used 1763776K [0xfffffffeec800000, 
> 0xffffffff6c800000, 0xffffffff6c800000)
> eden space 1747392K, 100% used 
> [0xfffffffeec800000,0xffffffff57270000,0xffffffff57270000)
> from space 16384K, 100% used 
> [0xffffffff57270000,0xffffffff58270000,0xffffffff58270000)
> to space 177408K, 0% used [0xffffffff61ac0000,0xffffffff61ac0000,0xffffffff6c800000)
> ParOldGen total 11214848K, used 686377K [0xfffffffc40000000, 0xfffffffeec800000, 
> 0xfffffffeec800000)
> object space 11214848K, 6% used 
> [0xfffffffc40000000,0xfffffffc69e4a480,0xfffffffeec800000)
> PSPermGen total 73728K, used 56020K [0xfffffffc30000000, 0xfffffffc34800000, 
> 0xfffffffc40000000)
> object space 73728K, 75% used 
> [0xfffffffc30000000,0xfffffffc336b5330,0xfffffffc34800000)
> 2010-03-24T*05:57:29*.620+0100: 47245.068: [GC
> 
> 
> $/usr/jdk/instances/jdk1.6.0/jre/bin/sparcv9/java -version
> java version "1.6.0_13"
> Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02, mixed mode)
> 
> $ uname -a
> SunOS XXX 5.10 Generic_120011-14 sun4u sparc SUNW,Sun-Fire
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From yamauchi at google.com  Wed Mar 24 17:01:30 2010
From: yamauchi at google.com (Hiroshi Yamauchi)
Date: Wed, 24 Mar 2010 17:01:30 -0700
Subject: Free ratio based heap shrinking in the parallel collector
In-Reply-To: <4BA3AE88.3090002@sun.com>
References: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
	<4B9E8499.8030904@sun.com>
	<d6262f011003151650v7d58e699gd868cae490f2a6f2@mail.gmail.com>
	<4BA0087A.2040006@Sun.COM>
	<d6262f011003181127x104b729cw8dbdd3040d6a2dd3@mail.gmail.com>
	<4BA3AE88.3090002@sun.com>
Message-ID: <d6262f011003241701p6ad9b425y12c5c1236ead9ee7@mail.gmail.com>

Hi Jon,

You're right that adjust_for_throughput() will not shrink the heap.
> Another part of the policy is that the heap is shrunk if the
> throughput goal and the pause time goal are being
> met.   It's a slower process than just enforcing MaxHeapFreeRatio
> and requires a number of GC's (that is, the heap would be
> shrunk down a percentage at each GC).


GC usually happens only as needed. So, after the application goes idle, GC
does not probably happen. The next GC happens only after the application
goes active again. Assuming that, there aren't many chances to shrink the
heap (maybe only one). If the application calls System.gc() right before
going idle, that may be the only chance. In such cases, the slower process
may not work very well.


> We would
> need to change the UseAdaptiveSizePolicy to do something like
>
>
> if ( average-pause > pause-goal)
>          shrink heap
> else if (average-throughput < throughput-goal) &&
>            ! (UseFreeRatioForParallelGC && heap too empty))  <<<< new
>         grow heap
> else
>         shrink heap
>
>
> Or maybe something like
>
>
> if ( average-pause > pause-goal)
>          shrink heap
> else if (average-throughput < throughput-goal)
>          if  (UseFreeRatioForParallelGC && heap too empty)    <<<< new
>                 shrink heap (policy-to-be-determined)         <<<<
>          else                                                 <<<<
>                 grow heap
> else if (average-throughput < throughput-goal)
>         grow heap to reduce gc overhead
> else
>         shrink heap
>
>
OK. I think the latter may be able to shrink the heap more aggressively.

Do you think the free ratio-based shrinking should happen only when the
throughput goal is met? Using the above scenario, when the application is
working hard (and perhaps GC is working hard as well) and goes idle next
moment, the average throughput at the time of the phase change may not be
good enough for the goal to be met (hence no shrinking).

Thanks,
Hiroshi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20100324/72d17dc0/attachment.html 

From john.coomes at sun.com  Fri Mar 26 02:10:34 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Fri, 26 Mar 2010 09:10:34 +0000
Subject: hg: jdk7/hotspot-gc: Added tag jdk7-b87 for changeset 6b1069f53fbc
Message-ID: <20100326091034.8B9174440E@hg.openjdk.java.net>

Changeset: 82135c848d5f
Author:    mikejwre
Date:      2010-03-25 15:05 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/rev/82135c848d5f

Added tag jdk7-b87 for changeset 6b1069f53fbc

! .hgtags


From john.coomes at sun.com  Fri Mar 26 02:10:38 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Fri, 26 Mar 2010 09:10:38 +0000
Subject: hg: jdk7/hotspot-gc/corba: Added tag jdk7-b87 for changeset
	09a41111a401
Message-ID: <20100326091041.73FF14440F@hg.openjdk.java.net>

Changeset: 39e14d2da687
Author:    mikejwre
Date:      2010-03-25 15:05 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/corba/rev/39e14d2da687

Added tag jdk7-b87 for changeset 09a41111a401

! .hgtags


From john.coomes at sun.com  Fri Mar 26 02:15:37 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Fri, 26 Mar 2010 09:15:37 +0000
Subject: hg: jdk7/hotspot-gc/jaxp: Added tag jdk7-b87 for changeset
	8b493f1aa136
Message-ID: <20100326091537.864C144411@hg.openjdk.java.net>

Changeset: d8ebd1591003
Author:    mikejwre
Date:      2010-03-25 15:05 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxp/rev/d8ebd1591003

Added tag jdk7-b87 for changeset 8b493f1aa136

! .hgtags


From john.coomes at sun.com  Fri Mar 26 02:15:42 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Fri, 26 Mar 2010 09:15:42 +0000
Subject: hg: jdk7/hotspot-gc/jaxws: Added tag jdk7-b87 for changeset
	3febd6fab2ac
Message-ID: <20100326091542.40AF544412@hg.openjdk.java.net>

Changeset: 8c666f8f3565
Author:    mikejwre
Date:      2010-03-25 15:05 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jaxws/rev/8c666f8f3565

Added tag jdk7-b87 for changeset 3febd6fab2ac

! .hgtags


From john.coomes at sun.com  Fri Mar 26 02:15:54 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Fri, 26 Mar 2010 09:15:54 +0000
Subject: hg: jdk7/hotspot-gc/jdk: Added tag jdk7-b87 for changeset 2cafbbe9825e
Message-ID: <20100326091659.A308A44413@hg.openjdk.java.net>

Changeset: b3c69282f6d3
Author:    mikejwre
Date:      2010-03-25 15:05 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/jdk/rev/b3c69282f6d3

Added tag jdk7-b87 for changeset 2cafbbe9825e

! .hgtags


From john.coomes at sun.com  Fri Mar 26 02:22:02 2010
From: john.coomes at sun.com (john.coomes at sun.com)
Date: Fri, 26 Mar 2010 09:22:02 +0000
Subject: hg: jdk7/hotspot-gc/langtools: Added tag jdk7-b87 for changeset
	409db93d19c0
Message-ID: <20100326092212.E561E44415@hg.openjdk.java.net>

Changeset: f9b5d4867a26
Author:    mikejwre
Date:      2010-03-25 15:05 -0700
URL:       http://hg.openjdk.java.net/jdk7/hotspot-gc/langtools/rev/f9b5d4867a26

Added tag jdk7-b87 for changeset 409db93d19c0

! .hgtags


From Jon.Masamitsu at Sun.COM  Fri Mar 26 16:59:39 2010
From: Jon.Masamitsu at Sun.COM (Jon Masamitsu)
Date: Fri, 26 Mar 2010 16:59:39 -0700
Subject: Free ratio based heap shrinking in the parallel collector
In-Reply-To: <d6262f011003241701p6ad9b425y12c5c1236ead9ee7@mail.gmail.com>
References: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
	<4B9E8499.8030904@sun.com>
	<d6262f011003151650v7d58e699gd868cae490f2a6f2@mail.gmail.com>
	<4BA0087A.2040006@Sun.COM>
	<d6262f011003181127x104b729cw8dbdd3040d6a2dd3@mail.gmail.com>
	<4BA3AE88.3090002@sun.com>
	<d6262f011003241701p6ad9b425y12c5c1236ead9ee7@mail.gmail.com>
Message-ID: <56E32EB8-AA26-477B-872F-6AD5C2104900@Sun.COM>


On Mar 24, 2010, at 5:01 PM, Hiroshi Yamauchi wrote:

> Hi Jon,
>
> You're right that adjust_for_throughput() will not shrink the heap.
> Another part of the policy is that the heap is shrunk if the
> throughput goal and the pause time goal are being
> met.   It's a slower process than just enforcing MaxHeapFreeRatio
> and requires a number of GC's (that is, the heap would be
> shrunk down a percentage at each GC).
>
> GC usually happens only as needed. So, after the application goes  
> idle, GC does not probably happen. The next GC happens only after  
> the application goes active again. Assuming that, there aren't many  
> chances to shrink the heap (maybe only one). If the application  
> calls System.gc() right before going idle, that may be the only  
> chance. In such cases, the slower process may not work very well.

Agreed.  The option to use MaxHeapFreeRatio/MinHeapFreeRation on a
System.gc() is needed.  As you say we probably only get the System.gc()
to do anything.

>
> We would
> need to change the UseAdaptiveSizePolicy to do something like
>
>
> if ( average-pause > pause-goal)
>          shrink heap
> else if (average-throughput < throughput-goal) &&
>            ! (UseFreeRatioForParallelGC && heap too empty))  <<<< new
>         grow heap
> else
>         shrink heap
>
>
> Or maybe something like
>
>
> if ( average-pause > pause-goal)
>          shrink heap
> else if (average-throughput < throughput-goal)
>          if  (UseFreeRatioForParallelGC && heap too empty)    <<<< new
>                 shrink heap (policy-to-be-determined)         <<<<
>          else                                                 <<<<
>                 grow heap
> else if (average-throughput < throughput-goal)
>         grow heap to reduce gc overhead
> else
>         shrink heap
>
>
> OK. I think the latter may be able to shrink the heap more  
> aggressively.
>
> Do you think the free ratio-based shrinking should happen only when  
> the throughput goal is met? Using the above scenario, when the  
> application is working hard (and perhaps GC is working hard as well)  
> and goes idle next moment, the average throughput at the time of the  
> phase change may not be good enough for the goal to be met (hence no  
> shrinking).


I don't believe we can handle that case (application suddenly going  
idle and GC does
the right thing).   As you say earlier we might not even have a GC.    
And whose to know
if the going idle isn't to be followed almost immediately by furious  
activity again.  The application has to
make that call.   What we add here is the requirement to limit the  
amount of free space in
the heap even if the throughput goal is not being met.

>
> Thanks,
> Hiroshi
>


From volkan at hatem.net  Sun Mar 28 14:04:46 2010
From: volkan at hatem.net (Volkan Hatem)
Date: Mon, 29 Mar 2010 00:04:46 +0300
Subject: Detecting Garbage Collector Generation
Message-ID: <30e973141003281404m986d9f1gf175a5db78398fc8@mail.gmail.com>

Apologies in advance if this list is not intended for these kind of
questions.

JDK 1.6:

How can I detect to which generation an object belongs? (regular java,
JVMTI, ?)

Is it safe to assume that all generations are allocated contigiously? Hence
detecting starting offset & size of a generation may give me what I'm
looking for?

Where can I find more information about how GC interacts with JVM? In other
words, how can I implement agents which interacts with GC the way JVMTI
does?

This will require more than detecting start/stop of GC collection. What GC
had detected as reachable/unreachable, decision to promote an object etc.

Thanks,
-volkan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20100329/34b643d3/attachment.html 

From Ryan.Highley at sabre-holdings.com  Tue Mar 30 12:54:13 2010
From: Ryan.Highley at sabre-holdings.com (Highley, Ryan)
Date: Tue, 30 Mar 2010 14:54:13 -0500
Subject: G1 Logging and Phases
Message-ID: <32DB14FBADE66B439D6A98D2B01D27EC12E2DB86@sgtulmsp02.Global.ad.sabre.com>

Hello,

 

I have a few validation questions based on the following G1 GC log
excerpt.

 

19.298: [GC pause (young) (initial-mark) 2141M->1957M(4000M)20.433: [GC
concurrent-mark-start]

, 1.1346080 secs]

20.721: [GC pause (young)20.761: [GC concurrent-mark-end, 0.2501590 sec]

 2300M->2119M(4000M), 1.3597880 secs]

22.081: [GC remark, 0.0012060 secs]

22.083: [GC concurrent-count-start]

22.452: [GC pause (young) 2469M->2119M(4000M), 0.3101740 secs]

22.976: [GC concurrent-count-end, 0.8934030]

23.063: [GC cleanup 2435M->639M(4000M), 0.0142190 secs]

23.078: [GC concurrent-cleanup-start]

23.086: [GC concurrent-cleanup-end, 0.0083300]

23.108: [GC pause (young) 660M->338M(4000M), 0.2031580 secs]

23.632: [GC pause (partial) 668M->346M(4000M), 0.3303010 secs]

 

 

>From what I read in concurrentMarkThread.cpp and g1CollectorPolicy.cpp
in the current OpenJDK 7 G1 source code, could I please get the
following confirmed or denied to ensure my understanding is correct?

 

The "GC pause" lines are indeed stop-the-world pauses as blatantly
marked. J

 

The "GC pause (young)" lines are only evacuating young gen regions.

 

The "GC pause (young) (initial-mark)" line is both evacuating young gen
regions and prepping for a full collection with initial marking of
roots.

 

The "remark" line is also a stop-the-world pause, i.e. the G1 "Final
Marking Pause" noted in the G1 whitepaper.

 

The "cleanup" line is also a stop-the-world pause and finalizes the live
object counts and sizes.  The heap allocated size should be the ending
size reported on the "cleanup" line plus any allocations and/or GC
activity that occur during the concurrent cleanup phase.

 

All "concurrent" lines run concurrently with application threads,
similar to the CMS behavior, and in a set of one or more
ConcurrentMarkThread instances.

 

 

Thank you for your help,

 

Ryan

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20100330/d6ca80a1/attachment.html 
-------------- next part --------------
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From jon.masamitsu at oracle.com  Tue Mar 30 21:34:34 2010
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Tue, 30 Mar 2010 21:34:34 -0700
Subject: understanding GC logs
In-Reply-To: <4BB20B1C.4020608@alcatel-lucent.com>
References: <dc6011320909281007k19f52f54v8d086edcd19b0bae@mail.gmail.com>
	<4AC0EEAE.5010705@Sun.COM>
	<dc6011320909281106r398fa9fak9384e89012d0c52d@mail.gmail.com>
	<4AC145AE.30804@sun.com> <4AC148B6.7010608@Sun.COM>
	<4AC1493A.2030004@sun.com> <4B4B3ECB.5090105@alcatel-lucent.com>
	<4B4B937C.4080907@alcatel-lucent.com>
	<4B4B9FBC.4040103@sun.com> <4B4BA6AF.5080300@sun.com>
	<4B50C9BF.8060202@alcatel-lucent.com> <4B50DE95.4060901@Sun.COM>
	<4BABA011.7020801@alcatel-lucent.com>
	<4BB20B1C.4020608@alcatel-lucent.com>
Message-ID: <148A7A57-B616-4CA4-9571-0F0216B0F650@oracle.com>

On 03/30/10 07:30, Shaun Hennessy wrote:
>
> A couple of question related to the GC logs and promotion failure  
> messages
>
> I am running 6u17.
>
> rgv[2]: -server
> argv[3]: -Xms14000m
> argv[4]: -Xmx14000m
> argv[5]: -XX:PermSize=800m
> argv[6]: -XX:NewSize=5600m
> argv[7]: -XX:MaxNewSize=5600m
> argv[8]: -XX:MaxPermSize=800m
> argv[9]: -XX:+DisableExplicitGC
> argv[10]: -XX:+UseConcMarkSweepGC
> argv[11]: -XX:+UseParNewGC
> argv[12]: -XX:+UseCMSCompactAtFullCollection
> argv[13]: -XX:+CMSClassUnloadingEnabled
> argv[28]: -verbose:gc
> argv[29]: -XX:+PrintGCDetails
> argv[30]: -XX:+PrintGCDateStamps
>
>
>
> 4112.744: [GC 4112.744: [ParNew: 4940531K->530787K(5017600K),  
> 0.1455641 secs] 11878708K->7540012K(13619200K), 0.1457559 secs]  
> [Times: user=1.38 sys=0.02, real=0.15 secs]
> 4113.780: [GC 4113.780: [ParNew: 4831587K->372801K(5017600K),  
> 0.2093305 secs] 11840812K->7551390K(13619200K), 0.2095270 secs]  
> [Times: user=1.34 sys=0.07, real=0.21 secs]
>  [Times: user=0.10 sys=0.00, real=0.11 secs]
> 2010-03-24T16:31:56.490-0400: 4114.097: [CMS-concurrent-mark-start]
> 4115.261: [GC 4115.261: [ParNew: 4674075K->364108K(5017600K),  
> 0.0755017 secs] 11852663K->7542736K(13619200K), 0.0756880 secs]  
> [Times: user=0.93 sys=0.00, real=0.08 secs]
> 4115.338: [GC 4115.338: [ParNew: 420064K->323310K(5017600K),  
> 0.1112115 secs] 7598693K->7587370K(13619200K), 0.1113667 secs]  
> [Times: user=0.98 sys=0.02, real=0.11 secs]
> 2010-03-24T16:31:58.647-0400: 4116.254: [CMS-concurrent-mark:  
> 1.909/2.157 secs] [Times: user=31.47 sys=1.55, real=2.16 secs]
> 2010-03-24T16:31:58.647-0400: 4116.254: [CMS-concurrent-preclean- 
> start]
> 2010-03-24T16:31:58.798-0400: 4116.405: [CMS-concurrent-preclean:  
> 0.149/0.151 secs] [Times: user=2.29 sys=0.12, real=0.15 secs]
> 2010-03-24T16:31:58.799-0400: 4116.406: [CMS-concurrent-abortable- 
> preclean-start]
> 4116.460: [GC 4116.460: [ParNew: 4624110K->301464K(5017600K),  
> 0.0914784 secs] 11888170K->7617401K(13619200K), 0.0916679 secs]  
> [Times: user=0.89 sys=0.03, real=0.09 secs]
> 2010-03-24T16:31:59.494-0400: 4117.101: [CMS-concurrent-abortable- 
> preclean: 0.596/0.695 secs] [Times: user=9.88 sys=0.60, real=0.70  
> secs]
> [YG occupancy: 2756990 K (5017600 K)]4117.102: [Rescan (parallel) ,  
> 0.4648394 secs]4117.567: [weak refs processing, 0.0028851  
> secs]4117.570: [class unloading, 0.0240174 secs]4117.594: [scrub  
> symbol & string tables, 0.0898531 secs] [Times: user=1.72 sys=0.37,  
> real=0.58 secs]
> 2010-03-24T16:32:00.079-0400: 4117.686: [CMS-concurrent-sweep-start]
> 4118.116: [GC 4118.116: [ParNew: 4602264K->305089K(5017600K),  
> 0.0712571 secs] 11891816K->7620802K(13619200K), 0.0714474 secs]  
> [Times: user=0.75 sys=0.00, real=0.07 secs]
> 4119.117: [GC 4119.117: [ParNew: 4605889K->263281K(5017600K),  
> 0.0842051 secs] 11665429K->7368947K(13619200K), 0.0843955 secs]  
> [Times: user=0.79 sys=0.01, real=0.08 secs]
> 4125.941: [GC 4125.941: [ParNew: 4936868K->708251K(5017600K),  
> 0.1426036 secs] 9789305K->5612975K(13619200K), 0.1427944 secs]  
> [Times: user=1.56 sys=0.01, real=0.14 secs]
> 4126.893: [GC 4126.894: [ParNew: 5009051K->485783K(5017600K),  
> 0.2210054 secs] 9536611K->5247528K(13619200K), 0.2211964 secs]  
> [Times: user=1.58 sys=0.04, real=0.22 secs]
> 4128.102: [GC 4128.102: [ParNew: 4786583K->455386K(5017600K),  
> 0.0748814 secs] 8588693K->4257495K(13619200K), 0.0750694 secs]  
> [Times: user=0.94 sys=0.00, real=0.08 secs]
> 2010-03-24T16:32:11.951-0400: 4129.558: [CMS-concurrent-sweep:  
> 10.777/11.872 secs] [Times: user=149.77 sys=7.25, real=11.87 secs]
> 2010-03-24T16:32:11.951-0400: 4129.558: [CMS-concurrent-reset-start]
> 2010-03-24T16:32:11.984-0400: 4129.591: [CMS-concurrent-reset:  
> 0.033/0.033 secs] [Times: user=0.04 sys=0.00, real=0.03 secs]  
> 4140.537: [GC 4140.537: [ParNew: 4756186K->539705K(5017600K),  
> 0.0873384 secs] 6572247K->2355767K(13619200K), 0.0875281 secs]  
> [Times: user=1.10 sys=0.00, re
> al=0.09 secs]
>
>
> 1) I no longer seem to get any "CMS-initial-mark"  - is this a  
> change since 6u12?

I'm running

Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
Java HotSpot(TM) Server VM (build 14.3-b01, mixed mode)

and I see entries such as

0.869: [GC [1 CMS-initial-mark: 22901K(229376K)] 28264K(258880K),  
0.0561592 secs] [Times: user=0.05 sys=0.01, real=0.06 secs]

> 2) The rescan portion than is the only non-concurrent correct?  So  
> from the above the application was only STW for 0.58 sec.

The initial-mark is also STW.

The rescan is part of the remark which is STW.  From my run

1.270: [GC[YG occupancy: 15860 K (29504 K)]1.270: [Rescan (parallel) ,  
0.1002467 secs]1.370: [weak refs processing, 0.0000167 secs] [1 CMS- 
remark: 22901K(229376K)] 38761K(258880K), 0.1004598 secs] [Times:  
user=0.11 sys=0.03, real=0.10 secs]


In your entry yes it is .058 sec.

> 3) This may have been a chance from 1.5 to 1.6, but this line also  
> used to display a CMS-remark did it not?

Yes, see my example above.

> 4) Is there a way to have my minor collections also display the full  
> date stamp (ie  2010-03-24T16:31:58.799-0400)
>
>

When I run with -XX:+PrintGCDateStamps I see entries such as

2010-03-30T16:06:55.297-0700: 2.602: [GC 2.602: [ParNew: 29504K- 
 >3264K(29504K), 0.2457302 secs] 52405K->38187K(258880K), 0.2460394  
secs] [Times: user=0.41 sys=0.02, real=0.25 secs]

I don't know why you're not seeing that.
>
> 1270.736: [GC 1270.736: [ParNew: 4872340K->574345K(5017600K),  
> 0.1967262 secs] 7123984K->2972742K(13619200K), 0.1969106 secs]  
> [Times: user=1.45 sys=0.05, re
> al=0.20 secs]
> 1272.024: [GC 1272.024: [ParNew: 4875542K->653139K(5017600K),  
> 0.1334760 secs] 7273939K->3051536K(13619200K), 0.1336582 secs]  
> [Times: user=1.54 sys=0.01, re
> al=0.13 secs]
> 1272.158: [GC 1272.159: [ParNew: 681949K->563105K(5017600K),  
> 0.2187865 secs] 3080347K->3158904K(13619200K), 0.2189362 secs]  
> [Times: user=1.48 sys=0.06, rea
> l=0.22 secs]
> 1273.398: [GC 1273.398: [ParNew: 4863905K->535051K(5017600K),  
> 0.1196808 secs] 7459704K->3130850K(13619200K), 0.1198694 secs]  
> [Times: user=1.51 sys=0.00, re
> al=0.12 secs]
> 1274.461: [GC 1274.461: [ParNew: 4835851K->399744K(5017600K),  
> 0.2861376 secs] 7431650K->3249248K(13619200K), 0.2863296 secs]  
> [Times: user=1.61 sys=0.09, re
> al=0.29 secs]
>
> 5) Why did the middle minor collection occur?  A big allocation?
>

That seems rather suspicious.   It may be a side effect of using JNI  
critical sections.  I don't
know if this is such a case but its the best behavior.

>
>
> - Promotion Failure
> 4896.478: [GC 4896.478: [ParNew: 4894353K->587864K(5017600K),  
> 0.4789909 secs] 8473688K->4268560K(13619200K), 0.4791812 secs]  
> [Times: user=1.00 sys=0.61, real=0.48 secs]
> 4897.812: [GC 4897.812: [ParNew: 4888664K->545903K(5017600K),  
> 0.4105613 secs] 8569360K->4326583K(13619200K), 0.4107560 secs]  
> [Times: user=1.06 sys=0.55, real=0.41 secs]
> 4899.057: [GC 4899.058: [ParNew: 4846703K->638966K(5017600K),  
> 0.2759734 secs] 8627383K->4496987K(13619200K), 0.2761637 secs]  
> [Times: user=1.13 sys=0.36, real=0.28 secs]
> 4900.101: [GC 4900.101: [ParNew: 4939768K->630721K(5017600K),  
> 0.5117751 secs] 8797789K->4607020K(13619200K), 0.5119662 secs]  
> [Times: user=0.84 sys=0.66, real=0.51 secs]
> 4900.615: [GC 4900.615: [ParNew: 651487K->487288K(5017600K),  
> 0.0780183 secs] 4627786K->4463587K(13619200K), 0.0781687 secs]  
> [Times: user=0.96 sys=0.00, real=0.08 secs]
> 4901.581: [GC 4901.581: [ParNew (promotion failed): 4788088K- 
> >4780999K(5017600K), 2.8947499 secs]4904.476: [CMS: 4003090K- 
> >1530872K(8601600K), 7.5122451 secs] 8764387K->1530872K(13619200K),  
> [CMS Perm : 671102K->671102K(819200K)], 10.4072102 secs] [Times:  
> user=11.03 sys=1.09, real=10.41 secs]
> 4913.024: [GC 4913.024: [ParNew: 4300800K->316807K(5017600K),  
> 0.0615917 secs] 5831672K->1847679K(13619200K), 0.0617857 secs]  
> [Times: user=0.74 sys=0.00, real=0.06 secs]
> 4914.015: [GC 4914.015: [ParNew: 4617607K->475077K(5017600K),  
> 0.0771389 secs] 6148479K->2005949K(13619200K), 0.0773290 secs]  
> [Times: user=0.95 sys=0.00, real=0.08 secs]
> 4914.908: [GC 4914.908: [ParNew: 4775877K->586339K(5017600K),  
> 0.0857102 secs] 6306749K->2117211K(13619200K), 0.0859046 secs]  
> [Times: user=1.06 sys=0.00, real=0.09 secs]
> 4915.816: [GC 4915.816: [ParNew: 4887139K->476398K(5017600K),  
> 0.1841627 secs] 6418011K->2152868K(13619200K), 0.1843556 secs]  
> [Times: user=1.32 sys=0.07, real=0.18 secs]
> 6) So here i had a promotion failure, this is due to fragmentation  
> of the tenured generation rather than lack of space?

Fragmentation is the likely problem.  When 6u20 is released try it.   
It does a better job
of keeping fragmentation down.

> 7) Do we need 1 contiguous space in tenured big enough to hold the  
> complete list/size of all objects being promoted, or
>    do multiple spaces get used& the pieces don't all fit?

The free space in the tenured generation is kept in a free list so  
there are multiple chunks.
Don't need 1 contiguous chunk for all the promotions.

> 8) What exactly is occurring during this promotion failed  
> collection?  Based on the next example I assume
>   it's a (successful) scavenge.  What exactly is this - which  
> thread(s) serial / ParallelGCThreads?,
>   STW?, are we simply compacting the tenured gen or are we can  
> actually GC the tenured?

A promotion failure is a scavenge that does not succeed because there  
is not enough
space in the old gen to do all the  needed promotions.  The scavenge  
is in essence
unwound and then a full STW compaction of the entire heap is done.

>
>
>
> promotion failed, and full GC
> 50786.124: [GC 50786.124: [ParNew: 4606713K->338518K(5017600K),  
> 0.0961884 secs] 12303455K->8081859K(13619200K), 0.0963907 secs]  
> [Times: user=0.91 sys=0.01, real=0.10 secs]
> 50787.373: [GC 50787.373: [ParNew: 4639318K->272229K(5017600K),  
> 0.0749353 secs] 12382659K->8053730K(13619200K), 0.0751408 secs]  
> [Times: user=0.75 sys=0.00, real=0.08 secs]
> 50788.483: [GC 50788.483: [ParNew: 4573029K->393397K(5017600K),  
> 0.0837182 secs] 12354530K->8185595K(13619200K), 0.0839321 secs]  
> [Times: user=1.03 sys=0.00, real=0.08 secs]
> 50789.590: [GC 50789.590: [ParNew (promotion failed): 4694264K- 
> >4612345K(5017600K), 1.5974678 secs] 12486461K- 
> >12447305K(13619200K), 1.5976765 secs] [Times : user=2.38 sys=0.20,  
> real=1.60 secs]
> GC locker: Trying a full collection because scavenge failed
> 50791.188: [Full GC 50791.188: [CMS: 7834959K->1227325K(8601600K),  
> 6.7102106 secs] 12447305K->1227325K(13619200K), [CMS Perm : 670478K- 
> >670478K(819200K)], 6.7103417 secs] [Times: user=6.71 sys=0.00,  
> real=6.71 secs]
> 50798.982: [GC 50798.982: [ParNew: 4300800K->217359K(5017600K),  
> 0.0364557 secs] 5528125K->1444685K(13619200K), 0.0366630 secs]  
> [Times: user=0.44 sys=0.00, real=0.04 secs]
> 50800.246: [GC 50800.246: [ParNew: 4518167K->198753K(5017600K),  
> 0.0368620 secs] 5745493K->1426078K(13619200K), 0.0370604 secs]  
> [Times: user=0.46 sys=0.01, real=0.04 secs]
> 9) Probably once I understand what the scavenge is doing will help  
> me understand this case, but logic seems
>  simply enough - fullgc on promotionfailure&scavenge failed.

Yes, full STW compaction.

>
>
>
> promotion and concurrent mode failures
> 53494.424: [GC 53494.424: [ParNew: 4979001K->716800K(5017600K),  
> 0.2120290 secs] 12583633K->8434774K(13619200K), 0.2122200 secs]  
> [Times: user=2.12 sys=0.03, real=0.21 secs]
> 53496.131: [GC 53496.131: [ParNew: 5017600K->605278K(5017600K),  
> 0.2761710 secs] 12735574K->8578720K(13619200K), 0.2763597 secs]  
> [Times: user=1.94 sys=0.08, real=0.28 secs]
>  [Times: user=0.16 sys=0.00, real=0.16 secs]
> 2010-03-25T06:14:58.961-0400: 53496.568: [CMS-concurrent-mark-start]
> 53497.688: [GC 53497.688: [ParNew: 4906078K->545999K(5017600K),  
> 0.0989930 secs] 12879520K->8519441K(13619200K), 0.0991785 secs]  
> [Times: user=1.21 sys=0.02, real=0.10 secs]
> 2010-03-25T06:15:00.188-0400: 53497.795: [CMS-concurrent-mark:  
> 1.107/1.227 secs] [Times: user=15.14 sys=0.42, real=1.23 secs]
> 2010-03-25T06:15:00.188-0400: 53497.795: [CMS-concurrent-preclean- 
> start]
> 2010-03-25T06:15:00.233-0400: 53497.840: [CMS-concurrent-preclean:  
> 0.043/0.045 secs] [Times: user=0.31 sys=0.01, real=0.04 secs]
> 2010-03-25T06:15:00.233-0400: 53497.840: [CMS-concurrent-abortable- 
> preclean-start]
> 2010-03-25T06:15:00.794-0400: 53498.401: [CMS-concurrent-abortable- 
> preclean: 0.541/0.560 secs] [Times: user=6.11 sys=0.22, real=0.56  
> secs]
> [YG occupancy: 3222128 K (5017600 K)]53498.402: [Rescan (parallel) ,  
> 0.4447462 secs]53498.847: [weak refs processing, 0.0028967  
> secs]53498.850: [class unloading, 0.0248904 secs]53498.875: [scrub  
> symbol & string tables, 0.0896937 secs] [Times: user=1.79 sys=0.35,  
> real=0.56 secs]
> 2010-03-25T06:15:01.360-0400: 53498.967: [CMS-concurrent-sweep- 
> start] 53499.350: [GC 53499.350: [ParNew (promotion failed):  
> 4846799K->4718254K(5017600K), 5.3142493 secs]53504.664:  
> [CMS2010-03-25T06:15:11.506-0400: 53509.113:
> [CMS-concurrent-sweep: 4.825/10.146 secs] [Times: user=16.61  
> sys=2.94, real=10.15 secs]
>  (concurrent mode failure): 8087820K->1346631K(8601600K), 11.0573075  
> secs] 12820241K->1346631K(13619200K), [CMS Perm : 670478K- 
> >670478K(819200K)], 16.37177 19 secs] [Times: user=17.62 sys=2.66,  
> real=16.37 secs]
> 53516.713: [GC 53516.714: [ParNew: 4300800K->283359K(5017600K),  
> 0.0498000 secs] 5647431K->1629990K(13619200K), 0.0499965 secs]  
> [Times: user=0.62 sys=0.00, real=0.05 secs]
> 53517.743: [GC 53517.743: [ParNew: 4584343K->340302K(5017600K),  
> 0.0544853 secs] 5930975K->1686933K(13619200K), 0.0546710 secs]  
> [Times: user=0.68 sys=0.00, real=0.05 secs]
> 10)  I think it's just the system is allocating at such at high rate  
> at this point in time ( and we don't use InitiatingOccupancy on this  
> app)
>   so we get close to full on tenured, minor collection came - no  
> room in tenured ---- so even though we don't say "Full GC" in this  
> one,
>   don't you get a Full GC as part of any concurrent-mode-failure?

The promotion failure that happens leads to the concurrent mode failure.
>
>
> promotion failed, scavenge failed, concurrent mode failure
> 86833.016: [GC 86833.017: [ParNew: 4769273K->453398K(5017600K),  
> 0.1316717 secs] 12418197K->8169164K(13619200K), 0.1319220 secs]  
> [Times: user=1.22 sys=0.03, real=0.13 secs]
>  [Times: user=0.14 sys=0.00, real=0.15 secs]
> 2010-03-25T15:30:37.688-0400: 86833.295: [CMS-concurrent-mark-start]
> 86834.751: [GC 86834.751: [ParNew: 4754198K->513298K(5017600K),  
> 0.1250485 secs] 12469964K->8281014K(13619200K), 0.1252553 secs]  
> [Times: user=1.38 sys=0.01, real=0.13 secs]
> 2010-03-25T15:30:39.310-0400: 86834.917: [CMS-concurrent-mark:  
> 1.453/1.621 secs] [Times: user=21.57 sys=1.15, real=1.62 secs]
> 2010-03-25T15:30:39.310-0400: 86834.917: [CMS-concurrent-preclean- 
> start]
> 2010-03-25T15:30:39.650-0400: 86835.258: [CMS-concurrent-preclean:  
> 0.337/0.341 secs] [Times: user=5.30 sys=0.18, real=0.34 secs]
> 2010-03-25T15:30:39.651-0400: 86835.258: [CMS-concurrent-abortable- 
> preclean-start]
> 2010-03-25T15:30:39.864-0400: 86835.471: [CMS-concurrent-abortable- 
> preclean: 0.211/0.214 secs] [Times: user=3.16 sys=0.19, real=0.21  
> secs]
> [YG occupancy: 3329361 K (5017600 K)]86835.500: [Rescan (parallel) ,  
> 0.3868448 secs]86835.887: [weak refs processing, 0.0030042  
> secs]86835.890: [class unloading, 0.0250008 secs]86835.915: [scrub  
> symbol & string tables, 0.0904210 secs] [Times: user=1.85 sys=0.29,  
> real=0.51 secs]
> 2010-03-25T15:30:40.401-0400: 86836.008: [CMS-concurrent-sweep-start]
> 86836.421: [GC 86836.422: [ParNew: 4814154K->680591K(5017600K),  
> 0.2031305 secs] 12581870K->8543701K(13619200K), 0.2033332 secs]  
> [Times: user=1.88 sys=0.04, real=0.20 secs]
> 86836.627: [GC 86836.627: [ParNew (promotion failed): 720747K- 
> >511306K(5017600K), 1.3076955 secs] 8583857K->8560580K(13619200K),  
> 1.3078889 secs] [Times: user=2.66 sys=0.78, real=1.31 secs]
> GC locker: Trying a full collection because scavenge failed
> 86837.935: [Full GC 86837.935: [CMS2010-03-25T15:30:46.850-0400:  
> 86842.457: [CMS-concurrent-sweep: 4.926/6.449 secs] [Times:  
> user=15.24 sys=1.19, real=6.45 secs]
>  (concurrent mode failure): 8049273K->1356962K(8601600K), 9.6514031  
> secs] 8560580K->1356962K(13619200K), [CMS Perm : 670523K- 
> >670523K(819200K)], 9.6515260 secs] [Times: user=9.65 sys=0.00,  
> real=9.65 secs]
> 86848.669: [GC 86848.669: [ParNew: 4301133K->201781K(5017600K),  
> 0.0452702 secs] 5658095K->1558743K(13619200K), 0.0454738 secs]  
> [Times: user=0.57 sys=0.00, real=0.05 secs]
>
> 11) - So here our scavenge failed, - this is what gave us the "Full  
> GC" log message -- the concurrent mode failure
>   was really just a coincidence/timing?  The full gc (triggered by  
> the promotion failure) aborts the tenured CMS collection
> does it not?

The "GC locker" message says that after a JNI critical section was  
exited the GC wanted to
do a scavenge but did not think there was enough room in  the old gen  
so it does a full
STW compaction.  Because a CMS concurrent collection was in progress,  
it is aborted
and that abortion causes the concurrent mode failure to be printed.   
Not a coincidence.
Just telling you that the CMS concurrent collection could not be  
completed for some
reason.


>
>
> thanks,
> Shaun
>
>
>
>
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20100330/27351cfc/attachment-0001.html 
-------------- next part --------------
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From volkan at hatem.net  Tue Mar 30 22:50:00 2010
From: volkan at hatem.net (Volkan Hatem)
Date: Wed, 31 Mar 2010 08:50:00 +0300
Subject: Detecing the generation an object belongs to
Message-ID: <j2p30e973141003302250l77618ee5ha55ab7ed15c36dd@mail.gmail.com>

Hi,

How can I detect to which generation an object belongs? (regular java, JVMTI, ?)

Is it safe to assume that all generations are allocated contigiously?
Hence detecting starting offset & size of a generation may give me
what I'm looking for?

Where can I find more information about how GC interacts with JVM? In
other words, how can I implement agents which interacts with GC the
way JVMTI does?

This will require more than detecting start/stop of GC collection.
What GC had detected as reachable/unreachable, decision to promote an
object etc.

Thanks,
-volkan
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From tony.printezis at oracle.com  Wed Mar 31 09:20:45 2010
From: tony.printezis at oracle.com (Tony Printezis)
Date: Wed, 31 Mar 2010 12:20:45 -0400
Subject: G1 Logging and Phases
In-Reply-To: <3FA4C6109D066541B52E1AF02D28A914093FEF6E@sgtulmsp04.Global.ad.sabre.com>
References: <32DB14FBADE66B439D6A98D2B01D27EC12E2DB86@sgtulmsp02.Global.ad.sabre.com>
	<4BB36FB4.9040903@oracle.com>
	<3FA4C6109D066541B52E1AF02D28A914093FEF6E@sgtulmsp04.Global.ad.sabre.com>
Message-ID: <4BB3765D.2000304@oracle.com>

Yes.

Tony

Bhasin, Vishal wrote:
> Tony, 
>
> So, based on your response - GC pause (young), GC pause (partial), GC
> remark & GC cleanup are stop-the-world, any other lines in log (gc
> activity) that would also be stop-the-world? Thanks!
>
> -----Original Message-----
> From: hotspot-gc-use-bounces at openjdk.java.net
> [mailto:hotspot-gc-use-bounces at openjdk.java.net] On Behalf Of Tony
> Printezis
> Sent: Wednesday, March 31, 2010 10:52 AM
> To: Highley, Ryan
> Cc: hotspot-gc-use at openjdk.java.net
> Subject: Re: G1 Logging and Phases
>
> Ryan,
>
> Hi. See inline.
>
> Highley, Ryan wrote:
>   
>> Hello,
>>
>> I have a few validation questions based on the following G1 GC log 
>> excerpt.
>>
>> 19.298: [GC pause (young) (initial-mark) 2141M->1957M(4000M)20.433: 
>> [GC concurrent-mark-start]
>>
>> , 1.1346080 secs]
>>
>> 20.721: [GC pause (young)20.761: [GC concurrent-mark-end, 0.2501590
>>     
> sec]
>   
>> 2300M->2119M(4000M), 1.3597880 secs]
>>
>> 22.081: [GC remark, 0.0012060 secs]
>>
>> 22.083: [GC concurrent-count-start]
>>
>> 22.452: [GC pause (young) 2469M->2119M(4000M), 0.3101740 secs]
>>
>> 22.976: [GC concurrent-count-end, 0.8934030]
>>
>> 23.063: [GC cleanup 2435M->639M(4000M), 0.0142190 secs]
>>
>> 23.078: [GC concurrent-cleanup-start]
>>
>> 23.086: [GC concurrent-cleanup-end, 0.0083300]
>>
>> 23.108: [GC pause (young) 660M->338M(4000M), 0.2031580 secs]
>>
>> 23.632: [GC pause (partial) 668M->346M(4000M), 0.3303010 secs]
>>
>> From what I read in concurrentMarkThread.cpp and g1CollectorPolicy.cpp
>>     
>
>   
>> in the current OpenJDK 7 G1 source code, could I please get the 
>> following confirmed or denied to ensure my understanding is correct?
>>
>> The "GC pause" lines are indeed stop-the-world pauses as blatantly 
>> marked. J
>>
>>     
> Yes.
>   
>> The "GC pause (young)" lines are only evacuating young gen regions.
>>
>>     
> Yes.
>   
>> The "GC pause (young) (initial-mark)" line is both evacuating young 
>> gen regions and prepping for a full collection with initial marking of
>>     
>
>   
>> roots.
>>
>>
>>     
> Yes, provided by a "full collection" you mean a marking cycle (by "full 
> collections" we refer to GCs that collect the entire heap in a STW
> phase).
>   
>> The "remark" line is also a stop-the-world pause, i.e. the G1 "Final 
>> Marking Pause" noted in the G1 whitepaper.
>>
>>     
> Yes.
>   
>> The "cleanup" line is also a stop-the-world pause and finalizes the 
>> live object counts and sizes.
>>
>>     
> Yes. And it also reclaims regions that have no live objects.
>   
>> The heap allocated size should be the ending size reported on the 
>> "cleanup" line plus any allocations and/or GC activity that occur 
>> during the concurrent cleanup phase.
>>
>>
>>     
> I'm not quite sure what you're trying to say. The line below
>
> 23.063: [GC cleanup 2435M->639M(4000M), 0.0142190 secs]
>
> says that there was a lot of garbage in your heap and the marking phase 
> found 1796MB worth of regions that contained no live objects. So, they 
> were reclaimed during cleanup.
>   
>> All "concurrent" lines run concurrently with application threads, 
>> similar to the CMS behavior, and in a set of one or more 
>> ConcurrentMarkThread instances.
>>
>>     
> There's only one ConcurrentMarkThread (which is the "controller" thread 
> if you want), which spawns parallel workers.
>
> Tony
>   
>> Thank you for your help,
>>
>> Ryan
>>
>>
>>     
> ------------------------------------------------------------------------
>   
>> _______________________________________________
>> hotspot-gc-use mailing list
>> hotspot-gc-use at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>   
>>     
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>   
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From Vishal.Bhasin at sabre-holdings.com  Wed Mar 31 09:20:12 2010
From: Vishal.Bhasin at sabre-holdings.com (Bhasin, Vishal)
Date: Wed, 31 Mar 2010 11:20:12 -0500
Subject: G1 Logging and Phases
In-Reply-To: <4BB36FB4.9040903@oracle.com>
References: <32DB14FBADE66B439D6A98D2B01D27EC12E2DB86@sgtulmsp02.Global.ad.sabre.com>
	<4BB36FB4.9040903@oracle.com>
Message-ID: <3FA4C6109D066541B52E1AF02D28A914093FEF6E@sgtulmsp04.Global.ad.sabre.com>

Tony, 

So, based on your response - GC pause (young), GC pause (partial), GC
remark & GC cleanup are stop-the-world, any other lines in log (gc
activity) that would also be stop-the-world? Thanks!

-----Original Message-----
From: hotspot-gc-use-bounces at openjdk.java.net
[mailto:hotspot-gc-use-bounces at openjdk.java.net] On Behalf Of Tony
Printezis
Sent: Wednesday, March 31, 2010 10:52 AM
To: Highley, Ryan
Cc: hotspot-gc-use at openjdk.java.net
Subject: Re: G1 Logging and Phases

Ryan,

Hi. See inline.

Highley, Ryan wrote:
>
> Hello,
>
> I have a few validation questions based on the following G1 GC log 
> excerpt.
>
> 19.298: [GC pause (young) (initial-mark) 2141M->1957M(4000M)20.433: 
> [GC concurrent-mark-start]
>
> , 1.1346080 secs]
>
> 20.721: [GC pause (young)20.761: [GC concurrent-mark-end, 0.2501590
sec]
>
> 2300M->2119M(4000M), 1.3597880 secs]
>
> 22.081: [GC remark, 0.0012060 secs]
>
> 22.083: [GC concurrent-count-start]
>
> 22.452: [GC pause (young) 2469M->2119M(4000M), 0.3101740 secs]
>
> 22.976: [GC concurrent-count-end, 0.8934030]
>
> 23.063: [GC cleanup 2435M->639M(4000M), 0.0142190 secs]
>
> 23.078: [GC concurrent-cleanup-start]
>
> 23.086: [GC concurrent-cleanup-end, 0.0083300]
>
> 23.108: [GC pause (young) 660M->338M(4000M), 0.2031580 secs]
>
> 23.632: [GC pause (partial) 668M->346M(4000M), 0.3303010 secs]
>
> From what I read in concurrentMarkThread.cpp and g1CollectorPolicy.cpp

> in the current OpenJDK 7 G1 source code, could I please get the 
> following confirmed or denied to ensure my understanding is correct?
>
> The "GC pause" lines are indeed stop-the-world pauses as blatantly 
> marked. J
>
Yes.
>
> The "GC pause (young)" lines are only evacuating young gen regions.
>
Yes.
>
> The "GC pause (young) (initial-mark)" line is both evacuating young 
> gen regions and prepping for a full collection with initial marking of

> roots.
>
>
Yes, provided by a "full collection" you mean a marking cycle (by "full 
collections" we refer to GCs that collect the entire heap in a STW
phase).
>
> The "remark" line is also a stop-the-world pause, i.e. the G1 "Final 
> Marking Pause" noted in the G1 whitepaper.
>
Yes.
>
> The "cleanup" line is also a stop-the-world pause and finalizes the 
> live object counts and sizes.
>
Yes. And it also reclaims regions that have no live objects.
>
> The heap allocated size should be the ending size reported on the 
> "cleanup" line plus any allocations and/or GC activity that occur 
> during the concurrent cleanup phase.
>
>
I'm not quite sure what you're trying to say. The line below

23.063: [GC cleanup 2435M->639M(4000M), 0.0142190 secs]

says that there was a lot of garbage in your heap and the marking phase 
found 1796MB worth of regions that contained no live objects. So, they 
were reclaimed during cleanup.
>
> All "concurrent" lines run concurrently with application threads, 
> similar to the CMS behavior, and in a set of one or more 
> ConcurrentMarkThread instances.
>
There's only one ConcurrentMarkThread (which is the "controller" thread 
if you want), which spawns parallel workers.

Tony
>
>
> Thank you for your help,
>
> Ryan
>
>
------------------------------------------------------------------------
>
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>   
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From yamauchi at google.com  Wed Mar 31 11:26:14 2010
From: yamauchi at google.com (Hiroshi Yamauchi)
Date: Wed, 31 Mar 2010 11:26:14 -0700
Subject: Free ratio based heap shrinking in the parallel collector
In-Reply-To: <56E32EB8-AA26-477B-872F-6AD5C2104900@Sun.COM>
References: <d6262f011003101517mdde3167x80d000b11d0e34ac@mail.gmail.com>
	<4B9E8499.8030904@sun.com>
	<d6262f011003151650v7d58e699gd868cae490f2a6f2@mail.gmail.com>
	<4BA0087A.2040006@Sun.COM>
	<d6262f011003181127x104b729cw8dbdd3040d6a2dd3@mail.gmail.com>
	<4BA3AE88.3090002@sun.com>
	<d6262f011003241701p6ad9b425y12c5c1236ead9ee7@mail.gmail.com>
	<56E32EB8-AA26-477B-872F-6AD5C2104900@Sun.COM>
Message-ID: <v2yd6262f011003311126le13c3d86u787b0e9ed1740a63@mail.gmail.com>

On Fri, Mar 26, 2010 at 4:59 PM, Jon Masamitsu <Jon.Masamitsu at sun.com>wrote:

>
> On Mar 24, 2010, at 5:01 PM, Hiroshi Yamauchi wrote:
>
>  Hi Jon,
>>
>> You're right that adjust_for_throughput() will not shrink the heap.
>> Another part of the policy is that the heap is shrunk if the
>> throughput goal and the pause time goal are being
>> met.   It's a slower process than just enforcing MaxHeapFreeRatio
>> and requires a number of GC's (that is, the heap would be
>> shrunk down a percentage at each GC).
>>
>> GC usually happens only as needed. So, after the application goes idle, GC
>> does not probably happen. The next GC happens only after the application
>> goes active again. Assuming that, there aren't many chances to shrink the
>> heap (maybe only one). If the application calls System.gc() right before
>> going idle, that may be the only chance. In such cases, the slower process
>> may not work very well.
>>
>
> Agreed.  The option to use MaxHeapFreeRatio/MinHeapFreeRation on a
> System.gc() is needed.  As you say we probably only get the System.gc()
> to do anything.


>
>
>> We would
>> need to change the UseAdaptiveSizePolicy to do something like
>>
>>
>> if ( average-pause > pause-goal)
>>         shrink heap
>> else if (average-throughput < throughput-goal) &&
>>           ! (UseFreeRatioForParallelGC && heap too empty))  <<<< new
>>        grow heap
>> else
>>        shrink heap
>>
>>
>> Or maybe something like
>>
>>
>> if ( average-pause > pause-goal)
>>         shrink heap
>> else if (average-throughput < throughput-goal)
>>         if  (UseFreeRatioForParallelGC && heap too empty)    <<<< new
>>                shrink heap (policy-to-be-determined)         <<<<
>>         else                                                 <<<<
>>                grow heap
>> else if (average-throughput < throughput-goal)
>>        grow heap to reduce gc overhead
>> else
>>        shrink heap
>>
>>
>> OK. I think the latter may be able to shrink the heap more aggressively.
>>
>> Do you think the free ratio-based shrinking should happen only when the
>> throughput goal is met? Using the above scenario, when the application is
>> working hard (and perhaps GC is working hard as well) and goes idle next
>> moment, the average throughput at the time of the phase change may not be
>> good enough for the goal to be met (hence no shrinking).
>>
>
>
> I don't believe we can handle that case (application suddenly going idle
> and GC does
> the right thing).   As you say earlier we might not even have a GC.   And
> whose to know
> if the going idle isn't to be followed almost immediately by furious
> activity again.  The application has to
> make that call.   What we add here is the requirement to limit the amount
> of free space in
> the heap even if the throughput goal is not being met.
>

I think we agree that it's a good idea to try to shrink the heap in response
to System.gc() in such a scenario/app.

Since we are talking about a setting where the free ratio flags takes
precedence over the psAdaptiveSizePolicy's throughput goals, the suggested
logic in the original webrev is perhaps not so terrible? If so, I suppose
the logic for UseFreeRatioOnlyInSystemGCForParallelGC does help with that
because it can shrink the heap regardless of the throughput goal. It may
make sense to remove UseFreeRatioForParallelGC and keep
UseFreeRatioOnlyInSystemGCForParallelGC only. If you'd like to see it happen
inside psAdaptiveSizePolicy instead, the current webrev does not work.

Thanks.

Hiroshi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20100331/7ead0c14/attachment.html 

From shaun.hennessy at alcatel-lucent.com  Wed Mar 31 10:45:11 2010
From: shaun.hennessy at alcatel-lucent.com (Shaun Hennessy)
Date: Wed, 31 Mar 2010 13:45:11 -0400
Subject: understanding GC logs
In-Reply-To: <148A7A57-B616-4CA4-9571-0F0216B0F650@oracle.com>
References: <dc6011320909281007k19f52f54v8d086edcd19b0bae@mail.gmail.com>
	<4AC0EEAE.5010705@Sun.COM>
	<dc6011320909281106r398fa9fak9384e89012d0c52d@mail.gmail.com>
	<4AC145AE.30804@sun.com> <4AC148B6.7010608@Sun.COM>
	<4AC1493A.2030004@sun.com> <4B4B3ECB.5090105@alcatel-lucent.com>
	<4B4B937C.4080907@alcatel-lucent.com>
	<4B4B9FBC.4040103@sun.com> <4B4BA6AF.5080300@sun.com>
	<4B50C9BF.8060202@alcatel-lucent.com> <4B50DE95.4060901@Sun.COM>
	<4BABA011.7020801@alcatel-lucent.com>
	<4BB20B1C.4020608@alcatel-lucent.com>
	<148A7A57-B616-4CA4-9571-0F0216B0F650@oracle.com>
Message-ID: <4BB38A27.8050408@alcatel-lucent.com>

An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20100331/34a87f81/attachment-0001.html 
-------------- next part --------------
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use

From jon.masamitsu at oracle.com  Wed Mar 31 12:21:13 2010
From: jon.masamitsu at oracle.com (Jon Masamitsu)
Date: Wed, 31 Mar 2010 12:21:13 -0700
Subject: understanding GC logs
In-Reply-To: <4BB38A27.8050408@alcatel-lucent.com>
References: <dc6011320909281007k19f52f54v8d086edcd19b0bae@mail.gmail.com>
	<4AC0EEAE.5010705@Sun.COM>
	<dc6011320909281106r398fa9fak9384e89012d0c52d@mail.gmail.com>
	<4AC145AE.30804@sun.com> <4AC148B6.7010608@Sun.COM>
	<4AC1493A.2030004@sun.com> <4B4B3ECB.5090105@alcatel-lucent.com>
	<4B4B937C.4080907@alcatel-lucent.com>
	<4B4B9FBC.4040103@sun.com> <4B4BA6AF.5080300@sun.com>
	<4B50C9BF.8060202@alcatel-lucent.com> <4B50DE95.4060901@Sun.COM>
	<4BABA011.7020801@alcatel-lucent.com>
	<4BB20B1C.4020608@alcatel-lucent.com>
	<148A7A57-B616-4CA4-9571-0F0216B0F650@oracle.com>
	<4BB38A27.8050408@alcatel-lucent.com>
Message-ID: <4BB3A0A9.8090306@oracle.com>

Shaun,

I tried redirecting the output with -Xloggc: and I still see
everything I'm expecting.

Who else out there is seeing the problems (1, 2, 3 below) with the
GC logging that Shaun is seeing?

Jon

On 03/31/10 10:45, Shaun Hennessy wrote:
> Hmm just looking around here -- I have 3 different applications all 
> using CMS --
> from glancing at some automated logs and a collections of machines I 
> can find it seems
> that all application on all machines
> 1) Don't log full date timestamps on minor collections
> 2) No longer log the initial-mark
> 3) No longer log the "CMS-remark" in that entry.
>
> All run 6u17 now, and have slightly different parameters..
> I'm pretty sure 3) was also true for me even before 6u17,  I thought 
> 2) did
> work in 6u12, and unsure on 1).     I'll see if I can roll back to confirm
> how 6u12 behaves for me and try stripping our options....
>
> We are mostly a x64 AMD (4140/444/4600) shop -- but I did check a 
> Intel 4170
> and a SPARC v440 and same problems seem to exist:
>
> Here's a typical example from a different app:
>
> # uname -a
> SunOS parking 5.10 Generic_139556-08 i86pc i386 i86pc
>
> 16:44:57,699 INFO  [ServerInfo] Java version: 1.6.0_17,Sun 
> Microsystems Inc.
> 16:44:57,699 INFO  [ServerInfo] Java VM: Java HotSpot(TM) 64-Bit 
> Server VM 14.3-b01,Sun Microsystems Inc.
> 16:44:57,699 INFO  [ServerInfo] OS-System: SunOS 5.10,amd64
>
> argv[0]: ../../jre/bin/amd64/java
> argv[1]: -server
> argv[2]: -DSAM
> argv[3]: -XX:ThreadStackSize=512
> argv[4]: -Xms16384m
> argv[5]: -Xmx16384m
> argv[6]: -XX:PermSize=900m
> argv[7]: -XX:NewSize=4096m
> argv[8]: -XX:MaxNewSize=4096m
> argv[9]: -XX:MaxPermSize=900m
> argv[10]: -XX:+DisableExplicitGC
> argv[11]: -XX:+UseConcMarkSweepGC
> argv[12]: -XX:CMSInitiatingOccupancyFraction=75
> argv[13]: -XX:+UseParNewGC
> argv[14]: -XX:+UseCMSCompactAtFullCollection
> argv[15]: -XX:+CMSClassUnloadingEnabled
> argv[16]: -XX:CMSInitiatingPermOccupancyFraction=95
> argv[17]: 
> -Djavax.xml.stream.XMLOutputFactory=com.bea.xml.stream.XMLOutputFactoryBase
> argv[18]: 
> -Djavax.xml.stream.XMLInputFactory=com.bea.xml.stream.MXParserFactory
> argv[19]: 
> -Djavax.xml.stream.XMLEventFactory=com.bea.xml.stream.EventFactory
> argv[20]: -Dsam.thread.dump=true
> argv[21]: -Xdebug
> argv[22]: -Xnoagent
> argv[23]: -Djava.compiler=NONE
> argv[24]: -XX:+PrintClassHistogram
> argv[25]: -XX:+HeapDumpOnOutOfMemoryError
> argv[26]: -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005
> argv[27]: -Dcom.sun.management.jmxremote
> argv[28]: -Dcom.sun.management.jmxremote.authenticate=false
> argv[29]: -Dcom.sun.management.jmxremote.ssl=false
> argv[30]: -Dcom.sun.management.jmxremote.port=9999
> argv[31]: -verbose:gc
> argv[32]: -XX:+PrintGCDetails
> argv[33]: -XX:+PrintGCDateStamps
> argv[34]: 
> -Djava.security.policy=../../nms/jboss/server/default/conf/server.policy
> argv[35]: -Dcom.timetra.nms.propertyFile=../../nms/config/nms-server.xml
> argv[36]: 
> -Dcom.timetra.nms.propertyFileMetrics=../../nms/config/nms-metrics.xml
> argv[37]: 
> -Dcom.timetra.nms.propertyJaasConfigFile=../../nms/config/SamJaasLogin.config
> argv[38]: -Djava.endorsed.dirs=../../nms/jboss/lib/endorsed
> argv[39]: -Ddrools.compiler=JANINO
> argv[40]: -Dmap.dao.descriptions.directory=../../nms/config/map
> argv[41]: 
> -Xloggc:../../nms/log/server/GC_logs/GC_trace_032610-16:44:56.log
> argv[42]: -classpath
> argv[43]: 
> :../../nms/lib/jdom/jdom.jar:../../nms/lib/installer/nms_installer.jar:../../nms/jboss/bin/run.jar
> argv[44]: 
> com.timetra.nms.installer.appservertools.ApplicationServerStarter
> argv[45]: start
> argv[46]: default
> argv[47]: 138.120.183.17
>
>
>
> Jon Masamitsu wrote:
>> On 03/30/10 07:30, Shaun Hennessy wrote:
>>> A couple of question related to the GC logs and promotion failure 
>>> messages
>>>
>>> I am running 6u17.
>>>
>>> rgv[2]: -server
>>> argv[3]: -Xms14000m
>>> argv[4]: -Xmx14000m
>>> argv[5]: -XX:PermSize=800m
>>> argv[6]: -XX:NewSize=5600m
>>> argv[7]: -XX:MaxNewSize=5600m
>>> argv[8]: -XX:MaxPermSize=800m
>>> argv[9]: -XX:+DisableExplicitGC
>>> argv[10]: -XX:+UseConcMarkSweepGC
>>> argv[11]: -XX:+UseParNewGC
>>> argv[12]: -XX:+UseCMSCompactAtFullCollection
>>> argv[13]: -XX:+CMSClassUnloadingEnabled
>>> argv[28]: -verbose:gc
>>> argv[29]: -XX:+PrintGCDetails
>>> argv[30]: -XX:+PrintGCDateStamps
>>>
>>>
>>>
>>> 4112.744: [GC 4112.744: [ParNew: 4940531K->530787K(5017600K), 
>>> 0.1455641 secs] 11878708K->7540012K(13619200K), 0.1457559 secs] 
>>> [Times: user=1.38 sys=0.02, real=0.15 secs]
>>> 4113.780: [GC 4113.780: [ParNew: 4831587K->372801K(5017600K), 
>>> 0.2093305 secs] 11840812K->7551390K(13619200K), 0.2095270 secs] 
>>> [Times: user=1.34 sys=0.07, real=0.21 secs]
>>>  [Times: user=0.10 sys=0.00, real=0.11 secs]
>>> 2010-03-24T16:31:56.490-0400: 4114.097: [CMS-concurrent-mark-start]
>>> 4115.261: [GC 4115.261: [ParNew: 4674075K->364108K(5017600K), 
>>> 0.0755017 secs] 11852663K->7542736K(13619200K), 0.0756880 secs] 
>>> [Times: user=0.93 sys=0.00, real=0.08 secs]
>>> 4115.338: [GC 4115.338: [ParNew: 420064K->323310K(5017600K), 
>>> 0.1112115 secs] 7598693K->7587370K(13619200K), 0.1113667 secs] 
>>> [Times: user=0.98 sys=0.02, real=0.11 secs]
>>> 2010-03-24T16:31:58.647-0400: 4116.254: [CMS-concurrent-mark: 
>>> 1.909/2.157 secs] [Times: user=31.47 sys=1.55, real=2.16 secs]
>>> 2010-03-24T16:31:58.647-0400: 4116.254: [CMS-concurrent-preclean-start]
>>> 2010-03-24T16:31:58.798-0400: 4116.405: [CMS-concurrent-preclean: 
>>> 0.149/0.151 secs] [Times: user=2.29 sys=0.12, real=0.15 secs]
>>> 2010-03-24T16:31:58.799-0400: 4116.406: 
>>> [CMS-concurrent-abortable-preclean-start]
>>> 4116.460: [GC 4116.460: [ParNew: 4624110K->301464K(5017600K), 
>>> 0.0914784 secs] 11888170K->7617401K(13619200K), 0.0916679 secs] 
>>> [Times: user=0.89 sys=0.03, real=0.09 secs]
>>> *2010-03-24T16:31:59.494-0400: 4117.101: 
>>> [CMS-concurrent-abortable-preclean: 0.596/0.695 secs] [Times: 
>>> user=9.88 sys=0.60, real=0.70 secs]
>>> [YG occupancy: 2756990 K (5017600 K)]4117.102: [Rescan (parallel) , 
>>> 0.4648394 secs]4117.567: [weak refs processing, 0.0028851 
>>> secs]4117.570: [class unloading, 0.0240174 secs]4117.594: [scrub 
>>> symbol & string tables, 0.0898531 secs] [Times: user=1.72 sys=0.37, 
>>> real=0.58 secs]*
>>> 2010-03-24T16:32:00.079-0400: 4117.686: [CMS-concurrent-sweep-start]
>>> 4118.116: [GC 4118.116: [ParNew: 4602264K->305089K(5017600K), 
>>> 0.0712571 secs] 11891816K->7620802K(13619200K), 0.0714474 secs] 
>>> [Times: user=0.75 sys=0.00, real=0.07 secs]
>>> 4119.117: [GC 4119.117: [ParNew: 4605889K->263281K(5017600K), 
>>> 0.0842051 secs] 11665429K->7368947K(13619200K), 0.0843955 secs] 
>>> [Times: user=0.79 sys=0.01, real=0.08 secs]
>>> 4125.941: [GC 4125.941: [ParNew: 4936868K->708251K(5017600K), 
>>> 0.1426036 secs] 9789305K->5612975K(13619200K), 0.1427944 secs] 
>>> [Times: user=1.56 sys=0.01, real=0.14 secs]
>>> 4126.893: [GC 4126.894: [ParNew: 5009051K->485783K(5017600K), 
>>> 0.2210054 secs] 9536611K->5247528K(13619200K), 0.2211964 secs] 
>>> [Times: user=1.58 sys=0.04, real=0.22 secs]
>>> 4128.102: [GC 4128.102: [ParNew: 4786583K->455386K(5017600K), 
>>> 0.0748814 secs] 8588693K->4257495K(13619200K), 0.0750694 secs] 
>>> [Times: user=0.94 sys=0.00, real=0.08 secs]
>>> 2010-03-24T16:32:11.951-0400: 4129.558: [CMS-concurrent-sweep: 
>>> 10.777/11.872 secs] [Times: user=149.77 sys=7.25, real=11.87 secs]
>>> 2010-03-24T16:32:11.951-0400: 4129.558: [CMS-concurrent-reset-start]
>>> 2010-03-24T16:32:11.984-0400: 4129.591: [CMS-concurrent-reset: 
>>> 0.033/0.033 secs] [Times: user=0.04 sys=0.00, real=0.03 secs] 
>>> 4140.537: [GC 4140.537: [ParNew: 4756186K->539705K(5017600K), 
>>> 0.0873384 secs] 6572247K->2355767K(13619200K), 0.0875281 secs] 
>>> [Times: user=1.10 sys=0.00, re
>>> al=0.09 secs]
>>>
>>>
>>> 1) I no longer seem to get any "|CMS-initial-mark"  - is this a 
>>> change since 6u12?
>>> |
>>
>> I'm running
>>
>> Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
>> Java HotSpot(TM) Server VM (build 14.3-b01, mixed mode)
>>
>> and I see entries such as
>>
>> 0.869: [GC [1 CMS-initial-mark: 22901K(229376K)] 28264K(258880K), 
>> 0.0561592 secs] [Times: user=0.05 sys=0.01, real=0.06 secs]
>>
>>> |2) The rescan portion than is the only non-concurrent correct?  So 
>>> from the above the application was only STW for 0.58 sec.
>>> |
>>
>> The initial-mark is also STW.
>>
>> The rescan is part of the remark which is STW.  From my run
>>
>> 1.270: [GC[YG occupancy: 15860 K (29504 K)]1.270: [Rescan (parallel) 
>> , 0.1002467 secs]1.370: [weak refs processing, 0.0000167 secs] [1 
>> CMS-remark: 22901K(229376K)] 38761K(258880K), 0.1004598 secs] [Times: 
>> user=0.11 sys=0.03, real=0.10 secs]
>>
>>
>> In your entry yes it is .058 sec.
>>
>>> |3) This may have been a chance from 1.5 to 1.6, but this line also 
>>> used to display a CMS-remark did it not?|
>>
>> Yes, see my example above.
>>
>>> |4) Is there a way to have my minor collections also display the 
>>> full date stamp (ie  |2010-03-24T16:31:58.799-0400)
>>>
>>>
>>
>> When I run with -XX:+PrintGCDateStamps I see entries such as
>>
>> 2010-03-30T16:06:55.297-0700: 2.602: [GC 2.602: [ParNew: 
>> 29504K->3264K(29504K), 0.2457302 secs] 52405K->38187K(258880K), 
>> 0.2460394 secs] [Times: user=0.41 sys=0.02, real=0.25 secs]
>>
>> I don't know why you're not seeing that.
>>>
>>> 1270.736: [GC 1270.736: [ParNew: 4872340K->574345K(5017600K), 
>>> 0.1967262 secs] 7123984K->2972742K(13619200K), 0.1969106 secs] 
>>> [Times: user=1.45 sys=0.05, re
>>> al=0.20 secs]
>>> 1272.024: [GC 1272.024: [ParNew: 4875542K->653139K(5017600K), 
>>> 0.1334760 secs] 7273939K->3051536K(13619200K), 0.1336582 secs] 
>>> [Times: user=1.54 sys=0.01, re
>>> al=0.13 secs]
>>> *1272.158: [GC 1272.159: [ParNew: 681949K->563105K(5017600K), 
>>> 0.2187865 secs] 3080347K->3158904K(13619200K), 0.2189362 secs] 
>>> [Times: user=1.48 sys=0.06, rea
>>> l=0.22 secs]*
>>> 1273.398: [GC 1273.398: [ParNew: 4863905K->535051K(5017600K), 
>>> 0.1196808 secs] 7459704K->3130850K(13619200K), 0.1198694 secs] 
>>> [Times: user=1.51 sys=0.00, re
>>> al=0.12 secs]
>>> 1274.461: [GC 1274.461: [ParNew: 4835851K->399744K(5017600K), 
>>> 0.2861376 secs] 7431650K->3249248K(13619200K), 0.2863296 secs] 
>>> [Times: user=1.61 sys=0.09, re
>>> al=0.29 secs]
>>>
>>> 5) Why did the middle minor collection occur?  A big allocation?
>>>
>>
>> That seems rather suspicious.   It may be a side effect of using JNI 
>> critical sections.  I don't
>> know if this is such a case but its the best behavior.
>>
>>>
>>>
>>> - Promotion Failure
>>> 4896.478: [GC 4896.478: [ParNew: 4894353K->587864K(5017600K), 
>>> 0.4789909 secs] 8473688K->4268560K(13619200K), 0.4791812 secs] 
>>> [Times: user=1.00 sys=0.61, real=0.48 secs]
>>> 4897.812: [GC 4897.812: [ParNew: 4888664K->545903K(5017600K), 
>>> 0.4105613 secs] 8569360K->4326583K(13619200K), 0.4107560 secs] 
>>> [Times: user=1.06 sys=0.55, real=0.41 secs]
>>> 4899.057: [GC 4899.058: [ParNew: 4846703K->638966K(5017600K), 
>>> 0.2759734 secs] 8627383K->4496987K(13619200K), 0.2761637 secs] 
>>> [Times: user=1.13 sys=0.36, real=0.28 secs]
>>> 4900.101: [GC 4900.101: [ParNew: 4939768K->630721K(5017600K), 
>>> 0.5117751 secs] 8797789K->4607020K(13619200K), 0.5119662 secs] 
>>> [Times: user=0.84 sys=0.66, real=0.51 secs]
>>> 4900.615: [GC 4900.615: [ParNew: 651487K->487288K(5017600K), 
>>> 0.0780183 secs] 4627786K->4463587K(13619200K), 0.0781687 secs] 
>>> [Times: user=0.96 sys=0.00, real=0.08 secs]
>>> *4901.581: [GC 4901.581: [ParNew (promotion failed): 
>>> 4788088K->4780999K(5017600K), 2.8947499 secs]4904.476: [CMS: 
>>> 4003090K->1530872K(8601600K), 7.5122451 secs] 
>>> 8764387K->1530872K(13619200K), [CMS Perm : 
>>> 671102K->671102K(819200K)], 10.4072102 secs] [Times: user=11.03 
>>> sys=1.09, real=10.41 secs]*
>>> 4913.024: [GC 4913.024: [ParNew: 4300800K->316807K(5017600K), 
>>> 0.0615917 secs] 5831672K->1847679K(13619200K), 0.0617857 secs] 
>>> [Times: user=0.74 sys=0.00, real=0.06 secs]
>>> 4914.015: [GC 4914.015: [ParNew: 4617607K->475077K(5017600K), 
>>> 0.0771389 secs] 6148479K->2005949K(13619200K), 0.0773290 secs] 
>>> [Times: user=0.95 sys=0.00, real=0.08 secs]
>>> 4914.908: [GC 4914.908: [ParNew: 4775877K->586339K(5017600K), 
>>> 0.0857102 secs] 6306749K->2117211K(13619200K), 0.0859046 secs] 
>>> [Times: user=1.06 sys=0.00, real=0.09 secs]
>>> 4915.816: [GC 4915.816: [ParNew: 4887139K->476398K(5017600K), 
>>> 0.1841627 secs] 6418011K->2152868K(13619200K), 0.1843556 secs] 
>>> [Times: user=1.32 sys=0.07, real=0.18 secs]
>>> 6) So here i had a promotion failure, this is due to fragmentation 
>>> of the tenured generation rather than lack of space?
>>
>> Fragmentation is the likely problem.  When 6u20 is released try it.  
>> It does a better job
>> of keeping fragmentation down.
>>
>>> 7) Do we need 1 contiguous space in tenured big enough to hold the 
>>> complete list/size of all objects being promoted, or
>>>    do multiple spaces get used& the pieces don't all fit? 
>>
>> The free space in the tenured generation is kept in a free list so 
>> there are multiple chunks.
>> Don't need 1 contiguous chunk for all the promotions.
>>
>>> 8) What exactly is occurring during this promotion failed 
>>> collection?  Based on the next example I assume
>>>   it's a (successful) scavenge.  What exactly is this - which 
>>> thread(s) serial / ParallelGCThreads?,
>>>   STW?, are we simply compacting the tenured gen or are we can 
>>> actually GC the tenured?
>>
>> A promotion failure is a scavenge that does not succeed because there 
>> is not enough
>> space in the old gen to do all the  needed promotions.  The scavenge 
>> is in essence
>> unwound and then a full STW compaction of the entire heap is done.
>>
>>>
>>>
>>>
>>> promotion failed, and full GC
>>> 50786.124: [GC 50786.124: [ParNew: 4606713K->338518K(5017600K), 
>>> 0.0961884 secs] 12303455K->8081859K(13619200K), 0.0963907 secs] 
>>> [Times: user=0.91 sys=0.01, real=0.10 secs]
>>> 50787.373: [GC 50787.373: [ParNew: 4639318K->272229K(5017600K), 
>>> 0.0749353 secs] 12382659K->8053730K(13619200K), 0.0751408 secs] 
>>> [Times: user=0.75 sys=0.00, real=0.08 secs]
>>> 50788.483: [GC 50788.483: [ParNew: 4573029K->393397K(5017600K), 
>>> 0.0837182 secs] 12354530K->8185595K(13619200K), 0.0839321 secs] 
>>> [Times: user=1.03 sys=0.00, real=0.08 secs]
>>> 50789.590: [GC 50789.590: [ParNew (promotion failed): 
>>> 4694264K->4612345K(5017600K), 1.5974678 secs] 
>>> 12486461K->12447305K(13619200K), 1.5976765 secs] [Times : user=2.38 
>>> sys=0.20, real=1.60 secs]
>>> GC locker: Trying a full collection because scavenge failed
>>> 50791.188: [Full GC 50791.188: [CMS: 7834959K->1227325K(8601600K), 
>>> 6.7102106 secs] 12447305K->1227325K(13619200K), [CMS Perm : 
>>> 670478K->670478K(819200K)], 6.7103417 secs] [Times: user=6.71 
>>> sys=0.00, real=6.71 secs]
>>> 50798.982: [GC 50798.982: [ParNew: 4300800K->217359K(5017600K), 
>>> 0.0364557 secs] 5528125K->1444685K(13619200K), 0.0366630 secs] 
>>> [Times: user=0.44 sys=0.00, real=0.04 secs]
>>> 50800.246: [GC 50800.246: [ParNew: 4518167K->198753K(5017600K), 
>>> 0.0368620 secs] 5745493K->1426078K(13619200K), 0.0370604 secs] 
>>> [Times: user=0.46 sys=0.01, real=0.04 secs]
>>> 9) Probably once I understand what the scavenge is doing will help 
>>> me understand this case, but logic seems
>>>  simply enough - fullgc on promotionfailure&scavenge failed.
>>
>> Yes, full STW compaction.
>>
>>>  
>>>
>>>
>>> promotion and concurrent mode failures
>>> 53494.424: [GC 53494.424: [ParNew: 4979001K->716800K(5017600K), 
>>> 0.2120290 secs] 12583633K->8434774K(13619200K), 0.2122200 secs] 
>>> [Times: user=2.12 sys=0.03, real=0.21 secs]
>>> 53496.131: [GC 53496.131: [ParNew: 5017600K->605278K(5017600K), 
>>> 0.2761710 secs] 12735574K->8578720K(13619200K), 0.2763597 secs] 
>>> [Times: user=1.94 sys=0.08, real=0.28 secs]
>>>  [Times: user=0.16 sys=0.00, real=0.16 secs]
>>> 2010-03-25T06:14:58.961-0400: 53496.568: [CMS-concurrent-mark-start]
>>> 53497.688: [GC 53497.688: [ParNew: 4906078K->545999K(5017600K), 
>>> 0.0989930 secs] 12879520K->8519441K(13619200K), 0.0991785 secs] 
>>> [Times: user=1.21 sys=0.02, real=0.10 secs]
>>> 2010-03-25T06:15:00.188-0400: 53497.795: [CMS-concurrent-mark: 
>>> 1.107/1.227 secs] [Times: user=15.14 sys=0.42, real=1.23 secs]
>>> 2010-03-25T06:15:00.188-0400: 53497.795: [CMS-concurrent-preclean-start]
>>> 2010-03-25T06:15:00.233-0400: 53497.840: [CMS-concurrent-preclean: 
>>> 0.043/0.045 secs] [Times: user=0.31 sys=0.01, real=0.04 secs]
>>> 2010-03-25T06:15:00.233-0400: 53497.840: 
>>> [CMS-concurrent-abortable-preclean-start]
>>> 2010-03-25T06:15:00.794-0400: 53498.401: 
>>> [CMS-concurrent-abortable-preclean: 0.541/0.560 secs] [Times: 
>>> user=6.11 sys=0.22, real=0.56 secs]
>>> [YG occupancy: 3222128 K (5017600 K)]53498.402: [Rescan (parallel) , 
>>> 0.4447462 secs]53498.847: [weak refs processing, 0.0028967 
>>> secs]53498.850: [class unloading, 0.0248904 secs]53498.875: [scrub 
>>> symbol & string tables, 0.0896937 secs] [Times: user=1.79 sys=0.35, 
>>> real=0.56 secs]
>>> 2010-03-25T06:15:01.360-0400: 53498.967: 
>>> [CMS-concurrent-sweep-start] 53499.350: [GC 53499.350: [ParNew 
>>> (promotion failed): 4846799K->4718254K(5017600K), 5.3142493 
>>> secs]53504.664: [CMS2010-03-25T06:15:11.506-0400: 53509.113:
>>> [CMS-concurrent-sweep: 4.825/10.146 secs] [Times: user=16.61 
>>> sys=2.94, real=10.15 secs]
>>>  (concurrent mode failure): 8087820K->1346631K(8601600K), 11.0573075 
>>> secs] 12820241K->1346631K(13619200K), [CMS Perm : 
>>> 670478K->670478K(819200K)], 16.37177 19 secs] [Times: user=17.62 
>>> sys=2.66, real=16.37 secs]
>>> 53516.713: [GC 53516.714: [ParNew: 4300800K->283359K(5017600K), 
>>> 0.0498000 secs] 5647431K->1629990K(13619200K), 0.0499965 secs] 
>>> [Times: user=0.62 sys=0.00, real=0.05 secs]
>>> 53517.743: [GC 53517.743: [ParNew: 4584343K->340302K(5017600K), 
>>> 0.0544853 secs] 5930975K->1686933K(13619200K), 0.0546710 secs] 
>>> [Times: user=0.68 sys=0.00, real=0.05 secs]
>>> 10)  I think it's just the system is allocating at such at high rate 
>>> at this point in time ( and we don't use InitiatingOccupancy on this 
>>> app)
>>>   so we get close to full on tenured, minor collection came - no 
>>> room in tenured ---- so even though we don't say "Full GC" in this one,
>>>   don't you get a Full GC as part of any concurrent-mode-failure?
>>
>> The promotion failure that happens leads to the concurrent mode failure.
>>>
>>>
>>> promotion failed, scavenge failed, concurrent mode failure
>>> 86833.016: [GC 86833.017: [ParNew: 4769273K->453398K(5017600K), 
>>> 0.1316717 secs] 12418197K->8169164K(13619200K), 0.1319220 secs] 
>>> [Times: user=1.22 sys=0.03, real=0.13 secs]
>>>  [Times: user=0.14 sys=0.00, real=0.15 secs]
>>> 2010-03-25T15:30:37.688-0400: 86833.295: [CMS-concurrent-mark-start]
>>> 86834.751: [GC 86834.751: [ParNew: 4754198K->513298K(5017600K), 
>>> 0.1250485 secs] 12469964K->8281014K(13619200K), 0.1252553 secs] 
>>> [Times: user=1.38 sys=0.01, real=0.13 secs]
>>> 2010-03-25T15:30:39.310-0400: 86834.917: [CMS-concurrent-mark: 
>>> 1.453/1.621 secs] [Times: user=21.57 sys=1.15, real=1.62 secs]
>>> 2010-03-25T15:30:39.310-0400: 86834.917: [CMS-concurrent-preclean-start]
>>> 2010-03-25T15:30:39.650-0400: 86835.258: [CMS-concurrent-preclean: 
>>> 0.337/0.341 secs] [Times: user=5.30 sys=0.18, real=0.34 secs]
>>> 2010-03-25T15:30:39.651-0400: 86835.258: 
>>> [CMS-concurrent-abortable-preclean-start]
>>> 2010-03-25T15:30:39.864-0400: 86835.471: 
>>> [CMS-concurrent-abortable-preclean: 0.211/0.214 secs] [Times: 
>>> user=3.16 sys=0.19, real=0.21 secs]
>>> [YG occupancy: 3329361 K (5017600 K)]86835.500: [Rescan (parallel) , 
>>> 0.3868448 secs]86835.887: [weak refs processing, 0.0030042 
>>> secs]86835.890: [class unloading, 0.0250008 secs]86835.915: [scrub 
>>> symbol & string tables, 0.0904210 secs] [Times: user=1.85 sys=0.29, 
>>> real=0.51 secs]
>>> 2010-03-25T15:30:40.401-0400: 86836.008: [CMS-concurrent-sweep-start]
>>> 86836.421: [GC 86836.422: [ParNew: 4814154K->680591K(5017600K), 
>>> 0.2031305 secs] 12581870K->8543701K(13619200K), 0.2033332 secs] 
>>> [Times: user=1.88 sys=0.04, real=0.20 secs]
>>> 86836.627: [GC 86836.627: [ParNew (promotion failed): 
>>> 720747K->511306K(5017600K), 1.3076955 secs] 
>>> 8583857K->8560580K(13619200K), 1.3078889 secs] [Times: user=2.66 
>>> sys=0.78, real=1.31 secs]
>>> GC locker: Trying a full collection because scavenge failed
>>> 86837.935: [Full GC 86837.935: [CMS2010-03-25T15:30:46.850-0400: 
>>> 86842.457: [CMS-concurrent-sweep: 4.926/6.449 secs] [Times: 
>>> user=15.24 sys=1.19, real=6.45 secs]
>>>  (concurrent mode failure): 8049273K->1356962K(8601600K), 9.6514031 
>>> secs] 8560580K->1356962K(13619200K), [CMS Perm : 
>>> 670523K->670523K(819200K)], 9.6515260 secs] [Times: user=9.65 
>>> sys=0.00, real=9.65 secs]
>>> 86848.669: [GC 86848.669: [ParNew: 4301133K->201781K(5017600K), 
>>> 0.0452702 secs] 5658095K->1558743K(13619200K), 0.0454738 secs] 
>>> [Times: user=0.57 sys=0.00, real=0.05 secs]
>>>
>>> 11) - So here our scavenge failed, - this is what gave us the "Full 
>>> GC" log message -- the concurrent mode failure
>>>   was really just a coincidence/timing?  The full gc (triggered by 
>>> the promotion failure) aborts the tenured CMS collection
>>> does it not?
>>
>> The "GC locker" message says that after a JNI critical section was 
>> exited the GC wanted to
>> do a scavenge but did not think there was enough room in  the old gen 
>> so it does a full
>> STW compaction.  Because a CMS concurrent collection was in progress, 
>> it is aborted
>> and that abortion causes the concurrent mode failure to be printed. 
>>  Not a coincidence.
>> Just telling you that the CMS concurrent collection could not be 
>> completed for some
>> reason.
>>
>>
>>>
>>>
>>> thanks,
>>> Shaun
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> hotspot-gc-use mailing list
>>> hotspot-gc-use at openjdk.java.net
>>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>>   
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/attachments/20100331/a0d78428/attachment-0001.html 
-------------- next part --------------
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use at openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use