David M. Lloyd
david.lloyd at redhat.com
Thu Sep 17 17:26:07 UTC 2015
On 09/17/2015 09:52 AM, mark.reinhold at oracle.com wrote:
> (splitting your initial reply into separate threads)
> 2015/9/9 1:48 -0700, david.lloyd at redhat.com:
>> On 09/09/2015 11:51 AM, mark.reinhold at oracle.com wrote:
>>> To get the discussion started I suggest we focus first on the high-level
>>> aspects of the design covered in "The State of the Module System" .
>>>  http://openjdk.java.net/projects/jigsaw/spec/sotms/
>> 1) The first thing that jumps out at me (again) is making module
>> declarations be a binary construct produced by the actual programming
>> language. It strikes me as arbitrary as well as redundant (given
>> there's a programmatic API to build descriptors, they could simply be
>> parsed from text nearly as well, and a lot more nicely from the user's
>> perspective). As pointed out in the document, it is already expected
>> that additional tooling would synthesize this file at build time,
>> further decreasing the value of such an idea. The module class file
>> just seems pointless to me.
> One of our requirements is fidelity across all phases. To achieve this,
> javac (or whatever Java compiler you use) must be able to locate and
> interpret descriptions of modules. If modules are not described by
> language constructs that are compiled into class files, then how and
> where would you suggest that a compiler find such information?
They could still be bundled with the artifact, but in clear text format
perhaps. Or even in binary format, but in either case, the descriptor
format should be specific to a specific default module configuration
(e.g. an application module loader) rather than be a hard-wired aspect
of all modules.
To me "fidelity" means the same thing as "integrity": I can build an
artifact and expect it to run on any environment which meets its ABI
requirements in any way. I don't think that hard-coding module
dependency names in the artifact itself is a good way to achieve this
though. Doing so hampers the ability to separate building from
packaging, since if you need to specify different run time environment
parameters in the packaged version, you must crack open and modify the
artifact to do so (new SHA1/MD5 checksums, etc.).
To me a better way to achieve this is a multi-tiered approach, where
each tier depends on the tier before (but not vice-versa).
The build artifact is identified and versioned as today (Maven-style),
and packaged alongside associated build dependency information. This
information is global - i.e. it is the same regardless of where and how
the artifact is used, and the artifact implicitly has a clear ABI.
Building a new artifact only depends on build dependency information
associated with the dependency artifacts. This method of building is
The next tier is a packaging/distribution tier. This pertains
specifically to the environment in which artifacts are utilized. In
this tier, modules are defined which consist of the combination of build
artifacts (with versions) and run-time, distribution-specific dependency
information. The distribution could (and should) be versioned as a
whole; individual module versions within the environment may correspond
with the artifact(s) that comprise it (but this is not necessary). The
distribution information, being versioned, could easily be distributed
wholly or in sections on a system like Maven.
An important aspect of this tier is testing, and its role in managing a
stable module distribution. When a new version of an artifact is
produced and made generally available, before a module may be included
in a distribution it is necessary to determine whether such inclusion
will regress any aspect of the existing infrastructure.
While completely retesting an entire distribution every time a component
or combination of components is updated is generally infeasible, it is
possible to use the distribution's run-time dependency information to
attain a level of confidence regarding the impact of an upgrade, by
working downwards along the dependent graph incrementally from each
changed module until the desired confidence is achieved that consumers
(direct or indirect) are unlikely to be regressed.
The last tier is the run time tier, in which some or all of the module
graph of a distribution is made available for execution, presumably (but
not necessarily) local to the running JVM. No version information is
used at run time outside of an informational capacity.
The reasons that this system is superior to a "flat" system are
hopefully obvious: what you run is not the same as what you build, and
no system exists which can make it be the same, unless you introduce a
central broker to all future Java builds which enforces consistency. A
"version" of a module is really a product of the version of the artifact
and the environment in which the module is executed. It is highly
unlikely that restricting usage of the module system to a single
effective distribution will suffice for most, or even many, use cases.
But this is exactly what will effectively happen by flattening run time
information all the way down to the build time layer: the result will be
one massive, headless module ecosystem whose overall security and
stability is completely indeterminate, which evolves in a completely
unpredictable manner. This is the complete opposite of fidelity in my
mind, and solves nothing of practical value.
> (The statement that "build tools, in particular, can synthesize module
> declarations from information already available in project descriptions"
> in the SotMS document is, I see, overly terse. A tool could suggest an
> initial module declaration with `requires` clauses derived from a
> pom.xml file, or equivalent, but it'd be up to the developer to specify
> the rest. I'll fix this in the next version.)
Yes I agree that a build tool can, at best, only make suggestions based
on information it may have about the target environment (which is
conceptually not very much unless the build tool becomes highly aware of
packaging considerations). Only something like a packaging tool that
has awareness of the distribution as a whole can give really useful
dependency hints for run time though.
More information about the jpms-spec-observers