Concerns mapping existing dynamic module systems to a 1-1 Module->Layer

mark.reinhold at mark.reinhold at
Wed Mar 8 20:08:39 UTC 2017

2017/3/8 6:09:11 -0800, tjwatson at
> There have been several topics recently discussing concerns in one context 
> or another when trying to use JPMS with an existing module system (like 
> OSGi or JBoss Modules) where the existing module system supports dynamic 
> modules.
> At a high level, the solution involves a 1-1 mapping for Module->Layer. 
>  This allows such a dyamic module system to make use of the dynamic Layer 
> creation to support the existing module system.
> From my experience there are some performance concerns that come in three 
> different categories when using a 1-1 mapping between the Module and 
> Layer.
> ...
> Altogether the overhead appears to be 4-5 ms per module (on a pretty 
> snappy machine).  Caching the discovery of private packages does cut that 
> by at least half.  Most of the remaining overhead can be attributed to the 
> fact that we are using 1-1 Module->Layer mappings in order to support 
> dynamics as well as hiding our own resolution wires that do not fit into 
> the strict rules of JPMS when dealing with split packages or cycles.
> Most of the additional methods David Lloyd is suggesting is to get the 
> functionality needed to do the complete mapping of existing module systems 
> into JPMS.  Essentially I think we are trying to work with JPMS in order 
> to solve the following requirement [1]:
> Interoperation — It must be possible for another module system, such as 
> OSGi, to locate Java modules and resolve them using its own resolver, 
> except possibly for core system modules.

To clarify, this requirement does not imply bidirectional interoperation
with dynamic module systems, as we've been discussing here.  All that
this requirement means is that another module system must be able to read
JPMS module metadata in order to resolve JPMS modules on its own terms.
In the case of OSGi, e.g., a suitably-enhanced framework implementation
could use the `java.lang.module.ModuleFinder` API to locate a module's
definition, and then use the `ModuleReader` API to read its metadata and
content to synthesize a corresponding OSGi bundle that's loaded by an
otherwise-ordinary OSGi class loader.

Support for bidirectional interoperation is not mandated by the agreed
requirements nor by the original JSR submission, as I've noted previously
[2].  I thought we'd found a way to achieve this stretch goal for OSGi,
but this recent discussion suggests that further research and exploration
is required.  I hope you'll understand that I'm reluctant to delay this
JSR any further in order to accommodate such work.

> While the solution we have today achieves this to some extent it is not 
> obvious to me that we have done this in the most useful of ways.  In 
> particular, all the additions David is suggesting are being resisted 
> because they potentially are exposing internal implementation details of 
> the JVM which is not desired by the JVM team.  To be a complete solution 
> for the JBoss case all (most?) of the methods David suggests are needed. 
>  As I stated already, OSGi really only needs the addPackage and 
> addExportsToAll methods to be complete.  But this still requires double 
> resolution work to get us back to the functionality we were at with Java 8 
> and that is only if the suggested methods make it into java 9.

No changes to OSGi implementations, nor to JPMS, should be required in
order for OSGi implementations to achieve the level of functionality that
they have on Java 8.  An OSGi implementation should run just fine on top
of Java 9, without interacting directly with JPMS in any way.  (If it
doesn't then there's a bug somewhere, in which case please report it and
we'll fix it.)  What we've been discussing here goes well beyond the
functionality available on Java 8.

> Taking a certain interpretation of the above requirement seems to imply 
> that we should be able to have a solution that does not require double 
> resolution at the JPMS Layer.  It seems to implies that we should be able 
> to use our own resolver to wire the modules for our own module system how 
> ever we see fit.  But I don't think that is really what we got with the 
> current solution which requires JPMS module resolution and then us using 
> the controller to provide the read edges required by our module system 
> class loaders.  I was not in on the early discussions of JPMS, but was 
> there any thread of discussion about an approach that provided a 
> pre-resolved graph to JPMS?

No, not that I recall.

>                              It seems like such a solution would naturally 
> fit into the spirit of the interoperation requirement.  I acknowledge that 
> such an idea is a drastic turn in the current design and I don't see how 
> they could be contained in any reasonable way.

That would indeed be an enormous change at this point in time, and it
would be justified only by a stretch goal rather than by an agreed

> I think we are left with the Layer Controller now for better or worse.  I 
> hope the overhead of creating and resolving 100s of Layers can be 
> optimized in the future so we can avoid having to do this largely 
> unnecessary resolution work when dealing with existing module systems. 
>  But if the Controller is what we are left with I think the solution needs 
> to be as complete as possible for the usecases it is used for.  Namely to 
> do the adhering of JBoss Modules and OSGi into JPMS.

Sorry, but I remain unwilling to expose the requested low-level primitive
operations, for the reasons previously stated.  If the `Layer.Controller`
API is insufficient as it stands today then we should defer support for
bidirectional interoperation to a future release.

- Mark


More information about the jpms-spec-experts mailing list