RFR: 8236073: G1: Use SoftMaxHeapSize to guide GC heuristics
maoliang.ml at alibaba-inc.com
Tue Feb 18 12:48:56 UTC 2020
I don't think we need an earlier shrink if we are trying to do it just a
bit later after mixed GCs. For the concurrent uncommit, I already had
a patch http://cr.openjdk.java.net/~luchsh/8236073.webrev/
But need spend sometime to refine it according to Thomas' comments.
From:Stefan Johansson <stefan.johansson at oracle.com>
Send Time:2020 Feb. 18 (Tue.) 17:17
To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>
Cc:hotspot-gc-dev <hotspot-gc-dev at openjdk.java.net>
Subject:Re: RFR: 8236073: G1: Use SoftMaxHeapSize to guide GC heuristics
I’ve also recently been looking at shrinking the heap after the Mixed collections. I totally agree that we should try to uncommit at this point, since the usage should be the lowest. I’m however not convinced that we should only uncommit once. My findings the last time, and what I’m seeing with your patch is some very long pauses when doing the uncommit. To try to avoid those I started looking at doing the uncommit concurrently, but didn’t find enough time to really dig into the details around that. An other thing to investigate would be the suggestion in:
My main point is that we need to ensure that uncommitting memory don’t come with a to high cost.
> 18 feb. 2020 kl. 08:27 skrev Liang Mao <maoliang.ml at alibaba-inc.com>:
> Hi Stefan,
> Thank you for your comments!
> Based on previous discussion, the reasons are as below:
> 1) For the expansion after cm, I think we have the agreement that
> original MinHeapFreeRatio might be too large and predicting the necessary
> size from adaptive IHOP for expansion sounds reasonable and specjbb2015
> have the good result.
> 2) About when to shrink the heap, I think it's a better spot after
> mixed collections. From my observation, the heap use is still at nearly
> peak after remark for most of cases like Alibaba workloads and specjbb2015.
> There could be some senario which contains a lot of humongous regions that
> remark will cleanup considerable regions. But why don't we decide to shrink
> the heap size when most of garbages have been cleaned after mixed GCs. We
> don't need to shrink twice in an old gc cycle. The MaxHeapFreeRatio 70 to
> keep heap capacity with 30% live objects make sence and is unified with full
> gc logic. If we only shrink the heap at remark, the maximum desired capacity
> could be 100/30 times of peak heap usage which is obviously not efficient.
> From:Stefan Johansson <stefan.johansson at oracle.com>
> Send Time:2020 Feb. 17 (Mon.) 18:10
> To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>
> Cc:hotspot-gc-dev <hotspot-gc-dev-bounces at openjdk.java.net>; hotspot-gc-dev <hotspot-gc-dev at openjdk.java.net>
> Subject:Re: RFR: 8236073: G1: Use SoftMaxHeapSize to guide GC heuristics
> Hi Liang,
> I’ve started looking at this patch as well and I have a question regarding the change to not allow shrinking after concurrent mark? Before we could shrink the heap at Remark, but now we only check to expand the heap after the concurrent cycle, why is that? I get that we will be able to shrink even more after the mixed collections but if a lot of regions are freed by the concurrent cycle why not check if we can shrink here?
> Also good to hear you can run SPECjvm2008, we also avoid running any problematic benchmarks.
> > 17 feb. 2020 kl. 10:56 skrev Liang Mao <maoliang.ml at alibaba-inc.com>:
> > Hi Thomas,
> > I am able to run specjvm2008 by excluding the compiler subtests
> > and reproduce the issue that the change commits more memory.
> > The main cause is addressed that the tests have a lot of
> > humongous objects which affect the evaluation of adaptive
> > IHOP. _last_unrestrained_young_size and _last_allocated_bytes
> > used in G1AdaptiveIHOPControl::predict_unstrained_buffer_size are
> > very large. So the expansion after concurrent mark is rather
> > aggressive. I made an enhancement to restrict this uncommon
> > expansion with MinHeapFreeRatio:
> > http://cr.openjdk.java.net/~luchsh/8236073.webrev.4/
> > Thanks,
> > Liang
> > ------------------------------------------------------------------
> > From:Thomas Schatzl <thomas.schatzl at oracle.com>
> > Send Time:2020 Feb. 15 (Sat.) 03:51
> > To:hotspot-gc-dev <hotspot-gc-dev at openjdk.java.net>
> > Subject:Re: RFR: 8236073: G1: Use SoftMaxHeapSize to guide GC heuristics
> > Hi,
> > On 12.02.20 12:16, Thomas Schatzl wrote:
> >> Hi Liang,
> >> On 12.02.20 11:17, Liang Mao wrote:
> >>> Hi Thomas,
> >>> I made a new patch for the issues we listed in JDK-8238686 and
> >>> JDK-8236073:
> >>> http://cr.openjdk.java.net/~luchsh/8236073.webrev.3/
> >> thanks. I only had time to quickly browse the change, and started
> >> building and testing it internally. I will run it through our perf
> >> benchmarks to look for regressions of out-of-box behavior.
> >> I will need a day or two until I can get back to looking at the change
> >> in detail. There is currently something else I need to look at. Sorry.
> > initial results from testing:
> > - gc/g1/TestPeriodicCollection.java fails consistently because the heap
> > does not shrink as expected (but probably this is a test bug as it may
> > expect that uncommit occurs at remark).
> > - memory usage tends to be significantly higher with the change without
> > improving scores.
> > E.g. I have been running specjvm2008 out-of-box with no settings on
> > different machine(s) (32gb ram min), and the build with the changes
> > almost consistently uses more heap (i.e. committed size) than without,
> > in the range of 10% without any performance increase.
> > Specjvm2008 benchmarks are pretty simple application in terms of
> > behavior, i.e. does the same things all the time. This also means that
> > very likely the current sizing is already way beyond the point of
> > diminishing returns (actually, this is a known issue :)); I would prefer
> > if we did not add to that. ;)
> > Unfortunately I lost the graphs I had generated (manually), and I do not
> > have more time available right now so can't show you right now.
> > I started some dacapo 2009 runs (running them for 30 iterations each).
> > Did not have time to look at the changes themselves any further or
> > investigate the reasons for this memory usage increase than I already
> > did earlier; will continue on Tuesday as I'm taking the day off Monday.
> > Thanks,
> > Thomas
More information about the hotspot-gc-dev