RFR: JDK-8150736: Excessive disk space used by build system
david.holmes at oracle.com
Tue Oct 4 08:56:42 UTC 2016
On 3/10/2016 6:22 PM, Erik Joelsson wrote:
> On 2016-10-01 14:13, David Holmes wrote:
>> Hi Erik,
>> This is a big change that is hard to digest. I assume the actual
>> images themselves are unaffected by this change?
> I understand that and I have been working on this for some time in the
> background. Yes, I have done quite extensive comparisons during this
> work so there should not be any relevant differences in the images.
> There are some, for example jexec and jspawnhelper are now stripped
> while the old strip logic missed them. I consider that fixing a bug.
> I was about to leave this patch for another release but since the
> schedule was moved, and Magnus is available to review things again, I
> thought it was worth trying to get it in. I believe it will add
> significant value.
>> An initial nit - why use "product-" in the target names? "images-jdk"
>> etc seems perfectly sufficient, and "product" may be a misnomer.
> A while back when we started introducing other images (test-image,
> docs-image, symbols-image) we also renamed the old images target to
> product-images. We left the alias images though since it's fairly well
> established, and introduced the all-images target to build all of them.
> To me it was then natural to create sub targets of product-images rather
> than images for jdk and jre. I do think the new targets are too long
> however and because of that will likely not see much use. Perhaps we can
> add aliases for them too as you suggest.
Why not just jdk-image and jre-image?
>> On 30/09/2016 8:17 PM, Erik Joelsson wrote:
>>> Here is a rather large patch which should make life better for most
>>> people building and developing OpenJDK. The main goal of this patch is
>>> to reduce unnecessary space used by the build. A side goal also become
>>> to figure out a better general strategy for handling native debug info
>>> during the build since that was a large part of the wasted space. These
>>> are the high level changes:
>>> 1. Removed the dist step in the hotspot build (and the corresponding
>>> import step in the jdk build) and linking libjvm directly into
>>> support/modules_libs. This removes 2 extra copies of all the hotspot
>>> native libraries.
>>> 2. Made many files in the exploded image ($(OUTPUT_DIR)/jdk) symlinks
>>> into support/modules_*. Note that binaries and libraries cannot be
>>> symlinks since the image won't run then. Debug symbols and other
>>> files seem to work fine though.
>>> 3. Introduced separate top level targets for building the jdk, jre (and
>>> serverjre) images. A developer can run "make product-images-jdk" and
>>> skip the others. The test targets now depend on just the jdk image
>>> to avoid building extra images that aren't needed for tests.
>>> 4. Changed SetupNativeCompilation to strip binaries by default unless
>>> told not to, and removed the separate strip build step. This lets us
>>> get rid of modules_*-stripped.
>>> 5. Build debug symbols into the OUTPUT_DIR in SetupNativeCompilation
>>> (instead of OBJECT_DIR and then copy to OUTPUT_DIR). For normal jdk
>>> and hotspot libraries, this means the debug symbols end up directly
>>> in support/modules_libs/$MODULE/... and there is only one copy of
>>> 6. If zipping debug symbols, this will also leave the unzipped
>>> debugsymbols in OUTPUT_DIR, where we wouldn't do so before. This
>>> also means that the exploded image will get symlinks to the unzipped
>>> debug symbols regardless of if we are asked to zip them or not. This
>>> in turn means that debugging the exploded image will just work. This
>>> change is space neutral since we were already keeping a copy of the
>>> unzipped debugsymbols in the OBJECT_DIR before, they have just been
>>> moved to a more useful place.
>>> 7. Stop copying debug symbols from test native binaries since we aren't
>>> stripping them anyway. Better to just leave debug information in the
>>> test binaries.
>>> The reduction in size of course varies a lot depending on OS and
>>> configuration, but as an example, on my 64bit Linux workstation
>>> (debug-symbols=zipped) the size of the build directory has gone from
>>> 5.8GB when building "make images" before to 3.8GB when building "make
>>> product-images-jdk". The biggest gain is from only building the jdk
>>> image when only that is needed. A fairer comparison is "make images
>>> test-image" which has gone from 7.3GB to 5.4GB, much due to not
>>> duplicating debug information in test binaries. On Solaris the gain is
>>> generally bigger.
>>> Another nice effect of this is that the exploded image now has debug
>>> information for all native libraries by default, and that handling of
>>> dSYM directories on Macosx should now work properly.
>>> The only significant changes detected by comparison builds is that some
>>> binaries that previously weren't stripped, now are, which they also
>>> should be.
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8150736
>>> Webrev: http://cr.openjdk.java.net/~erikj/8150736/webrev.01/
More information about the build-dev