Parallelizing the full compaction of CMS
jon.masamitsu at oracle.com
Mon Jun 8 15:58:01 UTC 2015
On 6/8/2015 3:04 AM, Volker Simonis wrote:
> Hi John, Jeremy,
> @John: sorry, but I don't see what's the big problem behind "supported
> and maintained by the community". We already have quite some code in
> the HotSpot which is developed and maintained that way. Take for
> example the C++-Interpreter. It has not been used and supported by
> Sun/Oracle since 1.4 where it was used for the Itanium port.
> Nevertheless it is quite useful for HotSpot ports to new platforms. We
> at SAP used it for our intitial PowerPC port for example. And of
> course we supported it. This includes both fixes as well as the
> implementation of new features (e.g. supports for JSR 292). Another
> user of the C++-interpreter is the Zero-port which is supported mainly
> by RedHat. Again, Oracle does neither build nor test the Zero-port.
> This is all done by RedHat. But it is still a crucial part of the
> OpenJDK for platforms like Linux on zArch, MIPS, Itanium,....
> Then we (i.e. SAP and IBM) have our ports (Linux/PPC64, AIX) and
> RedHat have their AArch64 port. Neither of them is supported by Oracle
> but all of them are pretty well supported by their maintainers. We
> have our own platform categories in the bug system, our OpenJDK
> mailing lists. We do nightly builds and tests and we take all this
> pretty seriously. And that all imposes nearly zero overhead on Oracle
> but rather helps to strengthen the VM on all platforms (e.g. we
> contributed quite some GC-fixes which only showed up on a weak memory
> architecture like Power or helped to clean up the code because we use
> different compiler on other platforms).
> I don't think it is appropriate for an Open Source project to reject a
> new feature just because one of the maintainers doesn't want it. And
> the OpenJDK is an OpenSource project with a distinct development model
> as specified in the OpenJDK bylaws (http://openjdk.java.net/bylaws).
> But I don't want to go on collision course. Until now the cooperation
> between the various contributors in the OpenJDK has always worked
> quite well. And if a feature can really be hidden behind a flag
> without impacting the other configurations that's great. Of course
> Oracle would not have to maintain or even test it (there already exist
> a myriad of unsupported configurations today :). And I think it is
> self-evident that contributors feel responsible for bugs in their code
> and fix them (as we already do today). So the only commitment on
> Oracle part would be to help with the initial review process which I'd
> consider acceptable.
What I think you're saying is that there have been contributions
that are not supported by Oracle and have no cost for Oracle.
And that this parallel full compaction for CMS would be another.
If a bug is reported where parallel full compaction with CMS is turned on
and Oracle is not expected to look at the bug, then I have no problem with
the integration of such a change after the usual code review. And as you
say, there is no need for Oracle to test with it turned on. As you note
I'm only one maintainer. Oracle may have other opinions.
> @Jeremy: could you please post some code so we can see what we're
> really talking about here. It seems that until now nobody ever saw
> these changes. And maybe it helps if several parties are trying
> jointly to 'convince' the maintainer to consider a patch :)
> On Fri, Jun 5, 2015 at 5:56 PM, Jon Masamitsu <jon.masamitsu at oracle.com> wrote:
>> On 6/3/2015 1:00 PM, Jeremy Manson wrote:
>> I'm not on hotspot-gc-dev, so I'm not seeing the responses. But I looked
>> Jon says that "My understanding was that the change was based on a
>> parallelization of the serial mark-sweep-compact (used by CMS for full
>> collections). If it was based on the parallel-old compaction used by
>> ParallelGC, then my bad. It should have been reviewed."
>> His original understanding was correct - we parallelized the serial
>> mark-sweep-compact used by CMS. I'm sorry if I misled anyone there.
>> No problem. It gave me a chance to say again that
>> I'd rather not have to support two implementations of
>> a parallel full stop-the-world collector. We have that
>> issue with the two implementations of a parallel young
>> collector and I'd really rather not go there again.
>> Otherwise, thanks for your thoughtful reply, Volker. In my experience of
>> open source projects, if you can't get a maintainer to agree to consider
>> your patch, then there is really very little that you can do. And I totally
>> get that maintainers don't want / have the time for the burden of
>> understanding huge amounts of complicated code they didn't write!
>> So, the question comes down to what "community supported" means in this
>> case. We can't actually submit anything to Hotspot without approval from
>> someone within Oracle. We have a vested interest in keeping this patch
>> working, but, for major JDK releases, we really only start stress testing to
>> identify problems with it when we upgrade to the latest versions of the JDK,
>> which usually happens some months after Oracle releases it. That could
>> cause some issues for upstream.
>> The upside is that I think we'd be willing to do what it takes, if getting
>> it upstreamed is a possibility. This patch is a huge nuisance, and it is
>> much simpler not to have to do a week or two's worth of forward porting
>> every year. And whatever problems arise are problems that we have to fix
>> Jon, if you are particularly worried, note that we can keep it hidden behind
>> a flag without too much difficulty (and did for a year or two).
>> I don't think that helps much. Unless Google is willing to own any
>> bug reports that come in on CMS when it is turned on. My main
>> concern is long term support. And whether we would have to
>> do our testing with it both turned on and turned off. Both those
>> seem like a big commitment on Oracle's part. Only my opinion,
>> of course.
>> On Wed, Jun 3, 2015 at 2:32 AM, Volker Simonis <volker.simonis at gmail.com>
>>> Hi Jeremy,
>>> thanks for your answer. Maybe you misunderstood Ivan but it is definitely
>>> his and our (i.e. SAP's) intention to make this part of the OpenJDK.
>>> I can also fully understand the pains you have with integrating changes
>>> into the HotSpot. We've gone through this painful process with our ppc64
>>> port a year ago but we think it will finally pay off. After all you (i.e.
>>> Google) used it as the base for your ppc64le port (Alexander Smundak
>>> contributed most of the LE-specific changes).
>>> We're facing the exactly same problems regarding the support of
>>> proprietary changes in a downstream version of the OpenJDK. That was
>>> actually the main reason why we started contributing to the OpenJDK. What we
>>> want to kindly ask you is to make your changes and extensions more
>>> transparently visible to the community if you intend to integrate them into
>>> the OpenJDK. If Hiroshi had posted his changes directly to the hotspot
>>> mailing lists instead of sending them privately to Jon I'm pretty sure we
>>> would have picked them up. And maybe together we would have managed to push
>>> them through (we have some reviewers as well :). After all, there are
>>> already quite a lot of features in the OpenJDK which are supported and
>>> maintained by the community (i.e. RedHat, IBM, SAP, ...) and not by Oracle.
>>> I don't have much experience with other Open Source projects, but with
>>> OpenJDK and especially HotSpot it's pretty pointless to try to contribute
>>> something by writing mails like "..I've implemented this cool feature XXX.
>>> If somebody is interested I can post more details..". You'll almost never
>>> get any feedback to such kind of requests. You'll at least have to post a
>>> complete buildabel and runnable patch against the head revision. I know
>>> that's a lot of overhead and we're having this very same discussion if it is
>>> worth doing it nearly every day for most of the features/additions we are
>>> implementing. Nevertheless, I think it is worth doing it although it
>>> sometimes may take years until you get your changes back in the version you
>>> are using productively.
>>> So coming back to Ivan's initial request, we would kindly ask you to just
>>> publish your changes in their current form against your current version
>>> (8u45 or whatsoever). We'll take a look at them an may invest some time and
>>> effort (also jointly with you if desired) to forward port them to 9 and
>>> propose them for integration into the head revision if that makes sense.
>>> Thus that sound reasonable to you?
>>> Thank you and best regards,
>>> On Tue, Jun 2, 2015 at 5:00 PM, Jeremy Manson <jeremymanson at google.com>
>>>> We'd much rather it be part of OpenJDK, because it is large and fairly
>>>> complicated, so that would save us time and energy forward porting it every
>>>> year or so. It's pretty large (for 8u45, it is a 2600 line change and
>>>> touches 20 files), so it tends to be a lengthy and complicated forward port.
>>>> We would much rather spend that time doing something more interesting.
>>>> Just posting an updated patch whenever we need to update it would be a
>>>> little annoying (although not impossible).
>>>> Hiroshi Yamauchi tried to offer it to (I think it was) Jon Masamitsu a
>>>> few years ago, and Jon didn't know anyone with the bandwidth to review it.
>>>> (It was around the same time we contributed the parallel initial mark). I
>>>> believe that was when CMS was unowned for a while. Perhaps there's a
>>>> different story now? Oracle volunteers?
>>>> (Procedural: One problem is that Hotspot reviews have to go through
>>>> someone at Oracle. We had it written by an OpenJDK author and reviewed by a
>>>> reviewer. We've been using it in production for years without too many
>>>> headaches (although we have yet to inflict^Wlaunch 8u45, which has the
>>>> latest updates, widely). I totally understand that Oracle folks are the
>>>> ones who have to pay the price for a bad check-in, but it's a pretty big
>>>> hurdle for us.)
>>>> On Tue, Jun 2, 2015 at 6:12 AM, Galkin, Ivan <ivan.galkin at sap.com> wrote:
>>>>> in the thread „JEP 248: Make G1 the Default Garbage Collector”  there
>>>>> is an email from Jeremy Manson, who mentions the enhancements made by Google
>>>>> to improve CMS.
>>>>> Especially the parallelizing of full compaction is a great improvement
>>>>> and we at SAP see the strong demand of it.
>>>>> @Jeremy: Are these changes published somewhere? May we have insight into
>>>>> the diffs you've made? We could collaborate in order to try to bring your
>>>>> changes into OpenJDK.
>>>>> Kind regards and thank you in advance,
More information about the hotspot-gc-dev