Proposal: make/Main.gmk Add extra extension/override points to the make file

Steve Groeger GROEGES at uk.ibm.com
Thu Dec 14 15:30:39 UTC 2017


Erik, 
 
Thank you very much for you very constructive comments.  
We do have our own JVM (OpenJ9) which does replace hotspot and which these proposed changes are attempting to implement with as minimal changes needed within the OpenJDK code as possible.
 
I will attempt to implement some of your suggestions within our current extension make file and pottentially use the post extension file. 
Once I have done that I will see if there are still changes needed in the make/Main.gmk and will post them to you and the mailing list for further review.
 
I have made a few comments below. 
 
Thanks
Steve Groeger
Java Runtimes Development
IBM Hursley
IBM United Kingdom Ltd
Tel: (44) 1962 816911 Mobex: 279990 Mobile: 07718 517 129
Fax (44) 1962 816800
Lotus Notes: Steve Groeger/UK/IBM
Internet: groeges at uk.ibm.com
 
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


-----Erik Joelsson <erik.joelsson at oracle.com> wrote: -----
To: Steve Groeger <GROEGES at uk.ibm.com>, build-dev at openjdk.java.net
From: Erik Joelsson <erik.joelsson at oracle.com>
Date: 12/14/2017 10:59
Subject: Re: Proposal: make/Main.gmk Add extra extension/override points to the make file

Hello Steve,

It's certainly an interesting test of the existing extension points when 
someone else needs to use them. The use case of replacing hotspot 
completely is definitely new to me.

First of all, Main.gmk only contains one call to IncludeCustomExtension 
close to the beginning of the file. We have a pattern of adding a second 
at the bottom for some other files so if you feel that would make things 
simpler, feel free to do so. In this case it would be added right before 
the .PHONY declaration and look like this: "$(eval $(call 
IncludeCustomExtension, Main-post.gmk))".

In some of the cases you seem to simply want to add one or more extra 
targets to a list. That kind of extension can often simply be done by 
adding the rule declaration in your custom extension file. In some cases 
it needs to be set in a -post file.

In other cases it seems to be about replacing hotspot completely with 
another jvm. That could be made more explicit with conditionals that 
completely turn off building of hotspot. Not sure which I like best. 
Depends a bit on your actual requirements on some of the things below. 
 From what I can tell, we should be able to collect the relevant parts 
in a handful of JVM_* variables that we can put at the top with one 
comment explaining the need for the overrides. If this is indeed about 
replacing hotspot, that comment needs to make that clear.

See comments below.


On 2017-12-14 10:53, Steve Groeger wrote:
> Hi,
>   
> I would like to propose the changes below to allow the make/Main.gmk file to be more extensible.
> These changes will allow an extension make file to be able to extend or override certain variable and
> targets.
>   
> I would appreciate any feedback. I would also like to find out how I go about opbtaining a sponson
> contributor for this change.
>   
> diff -r 8ca86cfb126f make/Main.gmk
> --- a/make/Main.gmk     Wed Dec 06 15:51:06 2017 -0800
> +++ b/make/Main.gmk     Wed Dec 13 16:27:04 2017 +0000
> @@ -56,6 +56,9 @@
>   # All modules for the current target platform.
>   ALL_MODULES := $(call FindAllModules)
>
> +# If not already set, set the JVM lib target so that the JVM will be built.
> +JVM_LIBS_TARGETS ?= hotspot-$(JVM_VARIANT_MAIN)-libs
> +
I assume this is about replacing hotspot. I would prefer the variable to 
be named JVM_MAIN_LIBS_TARGET as it's for the main variant only.

  This is OK. Happy to rename the variable. I was expecting requests to change the variable names.

>   ################################################################################
>   #################################################################################
>
> @@ -645,7 +648,7 @@
>     generate-exported-symbols: java.base-libs jdk.jdwp.agent-libs
>
>     # Building one JVM variant is enough to start building the other libs
> -  $(LIBS_TARGETS): hotspot-$(JVM_VARIANT_MAIN)-libs
> +  $(LIBS_TARGETS): $(JVM_LIBS_TARGETS)
>
>     $(LAUNCHER_TARGETS): java.base-libs
>
> @@ -721,8 +724,12 @@
>       java.base-jmod: jrtfs-jar $(filter-out java.base-jmod, $(JMOD_TARGETS))
>     endif
>
> -  # Building java.base-jmod requires all of hotspot to be built.
> -  java.base-jmod: hotspot
> +  # If not already set, set the JVM lib target so that the JVM will be built.
> +  JAVA_BASE_JMOD_DEPS ?= hotspot
> +
> +  # Building java.base-jmod requires all of VM (ie hotspot) to be built.
> +  java.base-jmod: $(JAVA_BASE_JMOD_DEPS)
> +
Assuming this is about replacing hotspot. Perhaps rename the variable 
"JVM_MAIN_TARGET" or something similar as it could be reused below. If 
you need to add additional prereqs to java.base-jmod, you can easily do 
so in your custom extension file.

    Yes, we were removing the hotspot target and adding the targets that are required for our own JVMM. 
    I will attempt to just do this in our extension file as suggested. If we need to have the variable
    then it is OK for the rename to "JVM_MAIN_TARGET" 

>
>     # Declare dependencies from <module>-jmod to all other module targets
>     # When creating a BUILDJDK, the java compilation has already been done by the
>
> @@ -748,7 +755,7 @@
>     # in java.base-copy) and tzdb.dat (done in java.base-gendata) to the
>     # appropriate location otherwise jimage, jlink and jmod won't start. This
>     # also applies when creating the buildjdk.
> -  DEFAULT_JMOD_DEPS := java.base-libs java.base-copy java.base-gendata \
> +  DEFAULT_JMOD_DEPS += java.base-libs java.base-copy java.base-gendata \
>         jdk.jlink-launchers
Unless you actually use the CREAET_BUILDJDK feature (typically for cross 
compiling), you could simply add your extra prereqs in a Main-post.gmk 
file like this:

$(JMOD_TARGETS) $(INTERIM_JMOD_TARGETS): <your prereqs>

If you actually do need it, then the above would be necessary and I'm ok 
with that.

   We would be using the CREATE_BUILDJDK as we do cross compiling. I can add the extra prereqs
   in the Main-post.gmk, but this would still need a chnage to Main.gmk to add the use of that
   the Main-post.gmk in this file. Will try and see if this works.

   Will try using Main-post.gmk and see if that provides the results we need.
>   
>
>     # When cross compiling and buildjdk is to be created, depend on creating the
>     # buildjdk instead of the default dependencies.
>
> @@ -808,8 +815,7 @@
>     # populated (java, copy and gendata targets) and the basic libs and launchers
>     # have been built.
>     exploded-image-optimize: java copy gendata java.base-libs java.base-launchers \
> -      buildtools-modules
> -
> +      buildtools-modules $(JVM_IMAGE_TARGETS)
>     bootcycle-images: jdk-image
>   
This looks like you simply need to add extra prereqs to 
exploded-image-optimize. Since that's a phony target, you can just 
declare it in your own file:

exploded-image-optimize: $(JVM_IMAGE_TARGETS)

   OK, thanks. Will give that a try and see if that gives the results we need.

>
>     docs-jdk-api-javadoc: $(GENSRC_TARGETS) rmic
>
> @@ -892,12 +898,17 @@
>   ################################################################################
>   # Virtual targets without recipes
>   
>
> +JVM_TOOLS_TARGETS ?= buildtools-hotspot
>   buildtools: buildtools-langtools interim-langtools interim-rmic \
> -    buildtools-jdk buildtools-hotspot
> +    buildtools-jdk $(JVM_TOOLS_TARGETS)
>   
>
> -hotspot: $(HOTSPOT_VARIANT_TARGETS) hotspot-jsig
> -
> -hotspot-libs: hotspot-jsig
> +# Setup JVM variables if not already set for a different jvm other than hotspot
> +JVM_RULE ?= hotspot
> +JVM_RULE_TARGETS ?= $(HOTSPOT_VARIANT_TARGETS) hotspot-jsig
> +JVM_LIB ?= hotspot-libs
> +JVM_LIB_TARGETS ?= hotspot-jsig
> +$(JVM_RULE): $(JVM_RULE_TARGETS)
> +$(JVM_LIB): $(JVM_LIB_TARGETS)
>   
This one I don't understand the reason for. If replacing hotspot then, 
does it matter that much that there is still a non functioning target 
"hotspot" still in the makefile? Same with "hotspot-libs". If you 
actually want to set values to these variables, why bother with reusing 
these declarations and not just construct your own in your custom 
extension file?

   That is a fair statment. I was tying to reduce the number of targets we had but there
   is no real issue with having non-functioning targets and defining the ones we require
   in our own custom extensions file. Will try that and see if it results in what we need.

>
>   # Create targets hotspot-libs and hotspot-gensrc.
>   $(foreach v, $(JVM_VARIANTS), \
>
> @@ -936,7 +947,7 @@
>   $(foreach m, $(ALL_COPY_MODULES), $(eval $m: $m-copy))
>   
>
>   # Building java.base includes building all of hotspot.
> -java.base: hotspot
> +java.base: $(JVM_BUILD_TARGETS)
>   
This variable needs a default value. Replace with JVM_MAIN_TARGET above.
>
>   demos: demos-jdk
>
> @@ -1003,13 +1014,19 @@
>   # This target builds the documentation image
>   docs-image: docs-jdk
>
> -# This target builds the test image
> -test-image: prepare-test-image test-image-hotspot-jtreg-native \
> +# If not already set, set the targets to built the test image
> +TEST_IMAGE_TARGETS ?= prepare-test-image test-image-hotspot-jtreg-native \
>       test-image-jdk-jtreg-native test-image-failure-handler test-image-hotspot-gtest \
>       test-image-demos-jdk
>
> +# This target builds the test image
> +test-image: $(TEST_IMAGE_TARGETS)
> +
Is this just about removing the *-hotspot-* targets from this list? In 
that case, introduce JVM_TEST_IMAGE_TARGETS.

   Yes, it is about removing the *-hotspot-* targets from this list.
   So you mean have:
   JVM_TEST_IMAGE_TARGETS ?=  test-image-hotspot-jtreg-native test-image-hotspot-gtest
      test-image: prepare-test-image test-image-jdk-jtreg-native test-image-failure-handler \ 
               test-image-demos-jdk $(JVM_TEST_IMAGE_TARGETS)

> +# If not already set, set the targets to built all images
> +ALL_IMAGES_TARGETS ?= product-images test-image docs-image
> +
>
>   # all-images builds all our deliverables as images.
> -all-images: product-images test-image docs-image
> +all-images: $(ALL_IMAGES_TARGETS)
This makes me curious as to what you need to remove. We have typically 
only needed to add targets which can be done from the custom extension file.

    This was strange for me also. All we were removing was the docs-image target.
    Will try this again with out removing that target and see if that causes us any issues.

/Erik
>   
>
>   # all-bundles packages all our deliverables as tar.gz bundles.
>   all-bundles: product-bundles test-bundles docs-bundles
>
> Thanks
> Steve Groeger
> Java Runtimes Development
> IBM Hursley
> IBM United Kingdom Ltd
> Tel: (44) 1962 816911 Mobex: 279990 Mobile: 07718 517 129
> Fax (44) 1962 816800
> Lotus Notes: Steve Groeger/UK/IBM
> Internet: groeges at uk.ibm.com
>   
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU



More information about the build-dev mailing list