<br><font size=1 face="sans-serif"><b>Bryan Atsatt <bryan.atsatt@ORACLE.COM></b></font><tt><font size=2>
wrote on 30/05/2007 19:11:02:<br>
<br>
> Responses inline, and a few clarifications here (I was a bit tired
when<br>
> I finished this last night :^)...<br>
> <br>
> The main point I was trying to make is that resolution must occur
within<br>
> a specific context, but I don't think my example APIs showed that
well.<br>
> I was assuming that ImportResolver had a ModuleContext as a field,
but<br>
> we can make this much cleaner and easier to understand by passing
it as<br>
> an argument:<br>
> <br>
> public abstract class ImportResolver {<br>
> public abstract Module resolve(ModuleDefinition
def,<br>
>
ModuleContext ctx);<br>
> }<br>
> <br>
> And we can really tie this together by adding a convenience method
to<br>
> ModuleContext:<br>
> <br>
> public abstract class ModuleContext {<br>
> ...<br>
> public Module resolve(ModuleDefinition def) {<br>
> return getImportResolver().resolve(def,
this);<br>
> }<br>
> }<br>
> <br>
> Now resolution becomes simply:<br>
> <br>
> context.resolve(definition);<br>
> <br>
> <br>
> (I also left out any use of the ImportPolicy, as it isn't yet clear
to<br>
> me that it should remain a separate type in this model.)<br>
> <br>
> // Bryan<br>
> <br>
> Glyn Normington wrote:<br>
> ><br>
> > *Bryan Atsatt <bryan.atsatt@ORACLE.COM>* wrote on 30/05/2007
07:57:59:<br>
> ><br>
> > > Andy Piper wrote:<br>
> > > > At 23:19 25/05/2007, Stanley M. Ho wrote:<br>
> > > >> Anyway, it seems the EG consensus so
far is to not add import package<br>
> > > >> support. If any EG member disagrees,
please speak up.<br>
> > > ><br>
> > > > Well, it depends on what the solution for
enabling interoperation<br>
> > > > with JSR 291 is.<br>
> > > > Our requirement is that there must be a
solution, if that requires<br>
> > > > import package, so be it. If not then not.<br>
> > ><br>
> > > Exactly.<br>
> > ><br>
> > > I think we can all agree that, at minimum, interoperation
means that<br>
> > > classes and resources are sharable *across* ModuleSystems
at runtime.<br>
> > ><br>
> > > Which implies that *import dependencies must be resolvable
across<br>
> > > multiple ModuleSystem instances*. (BTW, I think we
should change the<br>
> > > state name "PREPARING" to "RESOLVING"
in 7.2.1.)<br>
> ><br>
> > Agreed. We must avoid the trap of thinking that module system
interop.<br>
> > can be achieved by exposing class loaders (as loadClass will
happily<br>
> > load unexported classes).<br>
> ><br>
> > ><br>
> > > So the open issue is the richness of the import "language":
must we<br>
> > > support only lowest-common-denominator, or can we
do better without<br>
> > > over-complicating the design?<br>
> > ><br>
> > > I for one would like to be able to have a single module
express<br>
> > > dependencies on modules from both the same and different
ModuleSystems,<br>
> > > *using the standard semantics of each*. This may be
reaching too far,<br>
> > > but we should at least explore it seriously while
we figure out what<br>
> > > interop means here...<br>
> ><br>
> > At this point, I feel that is likely to be reaching too far,
but I'm<br>
> > happy to play along and see what we can learn along the way.<br>
> ><br>
> > ><br>
> > ><br>
> > > BASICS<br>
> > ><br>
> > > So far, we only know of two different import semantics:
module-name, and<br>
> > > package-name. For discussion, let's call these:<br>
> > ><br>
> > > a. import-module<br>
> > > b. import-package<br>
> > ><br>
> > > So, to start, we could:<br>
> > ><br>
> > > 1. Support declaration of both import types. If 294
supports imports at<br>
> > > all, it should be relatively easy to support both,
since a superpackage<br>
> > > name is a module name, and it contains member
package names. (Compiler<br>
> > > support is clearly the critical issue here, but it
will obviously<br>
> > > require use of the 277 runtime, so the import *type*
should be<br>
> > > transparent to it.) At worst, we'd need two new annotation
types.<br>
> ><br>
> > A superpackage name is a deployment module name in the JSR 277
model of<br>
> > one superpackage per deployment module, but I don't see any reason
why a<br>
> > JSR 291 deployment module should not contain more than one superpackage.<br>
> > So if 294 were to support import, then its import-module would
really be<br>
> > a superpackage import rather than a development module import.<br>
> <br>
> If we end up with nested superpackages, might it make sense to model<br>
> multiple superpackages by enclosing them in a single top-level one?</font></tt>
<br>
<br><tt><font size=2>That is an option, but of course each nested superpackage
has to name its parent, so it wouldn't be possible to combine superpackages
from independent groups or sources without either modifying their superpackage
declarations or getting them to agree on the name of the parent superpackage
and code it themselves.</font></tt>
<br><tt><font size=2><br>
> <br>
> ><br>
> > ><br>
> > > 2. Provide API for both import types (e.g. ImportDependency
has<br>
> > > getModuleName() and getPackageName() methods, one
of which will return<br>
> > > null on a given instance).<br>
> > ><br>
> > > However, we know these are necessary but not sufficient.
Leaving aside<br>
> > > the resolution issue for a moment, support for import-package
also<br>
> > > suggests that we:<br>
> > ><br>
> > > 3. Enable a single module to declare different versions
for each of its<br>
> > > member packages.<br>
> > ><br>
> > > 4. Enable efficient Repository lookup by package name.<br>
> > ><br>
> > > I think these are relatively easy (but *could* be
considered optional).<br>
> > ><br>
> > > We also need:<br>
> > ><br>
> > > 5. Standard Query types for lookup by module and package
name.<br>
> > ><br>
> > ><br>
> > > EXISTING DEPENDENCY RESOLUTION MODEL<br>
> > ><br>
> > > The more interesting issue is dependency resolution.
But this hasn't<br>
> > > been discussed in any real detail, so lets do so before
talking further<br>
> > > about import-package. To simplify this discussion,
I'm ignoring<br>
> > > bundled/custom import policies for now...<br>
> > ><br>
> > > Resolution in the current spec is delegated to the
associated<br>
> > > ModuleSystem instance (7.2.2 #8). While the details
are not spelled out,<br>
> > > the expectation appears to be that<br>
> > > ModuleSystem.getModule(ModuleDefinition) must:<br>
> > ><br>
> > > - Select an initial repository. Call getRepository()
on the input<br>
> > parameter.<br>
> > ><br>
> > > Then, for each ImportDependency of the definition:<br>
> > ><br>
> > > - Select a matching definition. Construct a Query
from the<br>
> > > ImportDependency and use Repository.find() to lookup
a matching<br>
> > > ModuleDefinition.<br>
> > ><br>
> > > - Get an instance. Use def.getModuleSystem().getModule(def).
The<br>
> > > ModuleSystem is expected to return a cached instance
if available, or<br>
> > > create/cache/return one if not.<br>
> ><br>
> > I think there also needs to be some 'resolution context' object
which<br>
> > explicitly denotes a particular resolution so that each module
system<br>
> > can keep track of the state of a resolution. This is required
when two<br>
> > or more imports of a given module from another module system
need to<br>
> > resolve to the same module instance. A resolution context may
also be<br>
> > needed for back-tracking when a set of module instances created
earlier<br>
> > in resolution turn out not to satisfy all the necessary constraints.<br>
> <br>
> Yes. I had (briefly :^) thought that we need only a List<Module>
as<br>
> field in the ImportResolver to hold the resolved modules as we go.
But<br>
> probably we need to keep some additional state along with each Module,<br>
> so a new type may be required. Let's call it ResolutionContext, and<br>
> create one at the start of each resolution:<br>
> <br>
> public abstract class ImportResolver {<br>
> <br>
> public Module resolve(ModuleDefinition def, ModuleContext
modCtx) {<br>
> ResolutionContext resCtx = createResolutionContext();<br>
> return resolve(def, modCtx, resCtx);<br>
> }<br>
> <br>
> protected abstract ResolutionContext createResolutionContext();<br>
> <br>
> protected abstract Module resolve(ModuleDefinition
def,<br>
>
ModuleContext
modCtx,<br>
>
ResolutionContext
resCtx);<br>
> }<br>
> <br>
> Does this approach make sense? Any thoughts on details for<br>
> ResolutionContext?</font></tt>
<br>
<br><tt><font size=2>Seems reasonable. I think ResolutionContext doesn't
need any detail - it can simply be used as the key of a map recording the
resolutions that a module system is currently aware of.</font></tt>
<br><tt><font size=2><br>
> <br>
> ><br>
> > ><br>
> > ><br>
> > > (TBD: The PlatformBinding must be taken into account
somehow during<br>
> > > selection. ModuleDefinition must include an accessor
for it, and either<br>
> > > Repository.find() should implicitly filter them, or
the caller must<br>
> > > construct a Query which will do so. I think we should
add a<br>
> > > CURRENT_PLATFORM constant to Query, which will evaluate
to true if no<br>
> > > binding is present in a definition.)<br>
> > ><br>
> > > (The spec also talks about Repository as the mechanism
of isolation<br>
> > > (6.4). This was the case in the prototype, where the
repository itself<br>
> > > provided caching. It doesn't appear to work with the
current design.<br>
> > > There is no need that I can see to isolate ModuleDefinition<br>
> > > instances--it is Module instances with their associated
loaders that may<br>
> > > require isolation.)<br>
> > ><br>
> > > (Also note that if ImportDependency was itself a Query
subclass, there<br>
> > > would be no need to do any mapping. And since the
ModuleDefinition<br>
> > > subclass must produce ImportDependency instances,
it can even produce<br>
> > > more specialized Query instances if desired.)<br>
> > ><br>
> > ><br>
> > > REFINEMENT<br>
> > ><br>
> > > I think we can improve on the existing model in several
ways:<br>
> > ><br>
> > > A. Provide a model for Module isolation (e.g. for
EE, Applets, etc).<br>
> > ><br>
> > > B. Encapsulate all selection logic in a single mechanism.<br>
> > ><br>
> > > C. Eliminate the overhead of the repository lookup
when a cached<br>
> > > instance exists.<br>
> > ><br>
> > > Let me propose a new class that encapsulates the caching
logic, enables<br>
> > > lookup using Query, and supports multiple instances
for isolation:<br>
> > ><br>
> > > public abstract class ModuleContext {<br>
> > ><br>
> > > // Get the context used to define
JRE modules.<br>
> > > public static ModuleContext getBootstrapContext(){...};<br>
> > ><br>
> > > // Get the context used to define
the main module.<br>
> > > public static ModuleContext getSystemContext(){...};<br>
> > ><br>
> > > // Get all contexts.<br>
> > > public static List<ModuleContext>
getContexts() {...};<br>
> > ><br>
> > > // Add a new context.<br>
> > > public static void addContext(ModuleContext
ctx) {...}<br>
> > ><br>
> > > // Remove a context (error if
== default).<br>
> > > public static boolean removeContext(ModuleContext
ctx) {...}<br>
> > ><br>
> > > // Get the parent context (null
if bootstrap).<br>
> > > public ModuleContext getParentContext(){...}<br>
> > ><br>
> > > // Get the name of this context.<br>
> > > public String getContextName()
{...}<br>
> > ><br>
> > > // Create a new Module instance
and store it in the cache.<br>
> > > public abstract Module createModule(ModuleDefinition
def);<br>
> > ><br>
> > > // Find cached Module instances.
Must check parent first.<br>
> > > public abstract List<Module>
findModules(Query query);<br>
> > ><br>
> > > // Set the context used for JRE
modules.<br>
> > > static void setBootstrapContext(ModuleContext
ctx){...}<br>
> > ><br>
> > > // Set the context used to define
the main module.<br>
> > > static void setSystemContext(ModuleContext
ctx){...}<br>
> > > }<br>
> > ><br>
> > > The JVM will create an appropriate subtype and call<br>
> > > setBootstrapContext(). The launcher will create a
child context and call<br>
> > > setSystemContext(). An EE (or similar) environment
can create/remove new<br>
> > > contexts as needed for application isolation.<br>
> > ><br>
> > > And the resolution algorithm can now check the cache
*first*, before<br>
> > > doing a repository lookup, using the same mechanism
in both. Query<br>
> > > should be used to express *all* selection criteria
(including<br>
> > > attributes, which we have not yet directly supported).<br>
> > ><br>
> > > Caches are no longer tied to ModuleSystem instances.<br>
> > > ModuleSystem.getModule() can become simply createModule().
The normal<br>
> > > implementation of ModuleContext.createModule() just
calls<br>
> > > ModuleSystem.createModule() and caches/returns the
result.<br>
> > ><br>
> > ><br>
> > > This class could easily be made concrete, but it may
be useful to<br>
> > > support subtypes for specialization (e.g. event generation,
lifecycle<br>
> > > management, specialized diagnostics, etc).<br>
> > ><br>
> > ><br>
> > > RESOLUTION MODELS<br>
> > ><br>
> > > The current design requires that each ModuleSystem
provide its own<br>
> > > resolution logic, and that each definition will be
resolved by its<br>
> > > owning ModuleSystem. This model appears to provide
flexibility for<br>
> > > significant differences in implementation, but we
really don't know<br>
> > > enough at this point. Perhaps only an actual second
implementation will<br>
> > > tell us if this provides useful flexibility.<br>
> > ><br>
> > > It wouldn't surprise me if we have to keep tightening
the spec as we go,<br>
> > > in order to remove inconsistencies that arise from
the separate<br>
> > > algorithms. And this may eliminate flexibility to
the point where it is<br>
> > > no longer useful. Much worse, we may not even discover
this until after<br>
> > > the spec and RI are released, if that second implementation
(e.g. OSGi)<br>
> > > is not completed beforehand.<br>
> > ><br>
> > > We should at least consider the obvious alternative:
one algorithm (to<br>
> > > rule them all :^). And I don't mean one hard-coded
algorithm, I mean one<br>
> > > replaceable, extensible class, such as:<br>
> > ><br>
> > > public abstract class ImportResolver {<br>
> > > public abstract Module resolve(ModuleDefinition
def);<br>
> > > }<br>
> > ><br>
> > > And we add a method to ModuleContext to retrieve an
instance:<br>
> > ><br>
> > > public abstract class ModuleContext {<br>
> > > ...<br>
> > > public abstract ImportResolver
getImportResolver();<br>
> > > }<br>
> > ><br>
> > > (Note that ImportResolver is now in a position to
subsume the<br>
> > > functionality of both VisibilityPolicy and ImportOverridePolicy.)<br>
> > ><br>
> > > Repository usage is encapsulated within the implementation
of the<br>
> > > resolve() method. The full resolution algorithm becomes:<br>
> > ><br>
> > > context.getImportResolver().resolve(definition);<br>
> > ><br>
> > > The launcher uses the "system" context for
this. EE, Applets, etc. make<br>
> > > and use their own distinct, isolated instances.<br>
> > ><br>
> > ><br>
> > > RESOLUTION WITH IMPORT-NAME AND IMPORT-PACKAGE<br>
> > ><br>
> > > With this scaffolding in place we can easily take
a phased approach to<br>
> > > supporting import-package: the initial implementation
simply does not<br>
> > > support it at runtime.<br>
> ><br>
> > A phased approach would be particularly beneficial if the initial
phase<br>
> > could be delivered as part of Java 7 and subsequent phases implemented<br>
> > strictly on top of Java 7. But getting the API right up front
might be<br>
> > tricky unless we can spot some really good abstractions or prototype
the<br>
> > later phases sufficiently well. Is that the kind of phasing you
had in<br>
> > mind?<br>
> <br>
> Yes. It would be *extremely* useful to have an OSGi<br>
> implementation/prototype well under way before Java 7 is completed,
so<br>
> that we can fine tune the model as we learn.<br>
> <br>
> ><br>
> > ><br>
> > > A subsequent implementation may support import-package,
but only within<br>
> > > the boundaries of the same ModuleSystem.<br>
> > ><br>
> > > And a full blown implementation may support import-package
across<br>
> > > ModuleSystems.<br>
> > ><br>
> > > We can build in support for selecting/configuring
the ImportResolver as<br>
> > > a service, just as we plan to do with ModuleSystem
(and Repository, I<br>
> > > presume).<br>
> > ><br>
> > ><br>
> > > And maybe, just maybe, we can find a way to abstract
and re-use the<br>
> > > mature resolution logic from the OSGi reference implementation
*as* the<br>
> > > one implementation to rule them all.<br>
> > ><br>
> > > // Bryan<br>
> ><br>
> > Glyn<br>
> ><br>
> ><br>
> > ------------------------------------------------------------------------<br>
> ><br>
> > /<br>
> > /<br>
> ><br>
> > /Unless stated otherwise above:<br>
> > IBM United Kingdom Limited - Registered in England and Wales
with number<br>
> > 741598.<br>
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire
PO6 3AU/<br>
> ><br>
> ><br>
> ><br>
> ><br>
> ><br>
> ><br>
</font></tt>
<br><tt><font size=2>Glyn</font></tt><font size=3 face="sans-serif"><br>
</font>
<br><font size=3 face="sans-serif"><br>
</font>
<hr><font size=2 face="sans-serif"><br>
<i><br>
</i></font>
<p><font size=2 face="sans-serif"><i>Unless stated otherwise above:<br>
IBM United Kingdom Limited - Registered in England and Wales with number
741598. <br>
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU</i></font>
<p><font size=2 face="sans-serif"><br>
</font><font size=3 face="sans-serif"><br>
</font>
<br>
<br><font size=3 face="sans-serif"><br>
</font>