#LayerPrimitives aka allowing to add private package at runtime to a module ?
David M. Lloyd
david.lloyd at redhat.com
Thu Mar 16 13:39:25 UTC 2017
On 03/16/2017 06:33 AM, Remi Forax wrote:
> This is currently a point where several of us disagree, here is my opinion on it.
Thanks for the reply.
> First, we have to all recognize that making something mutable, the content of a module or anything else, makes maintenance, debugging, etc harder than having the same thing non mutable.
> So the default choice should to choosing the immutable solution apart if there are good argument.
This is a good argument. In my initial JBoss Modules implementation,
modules were immutable. Our first service container was also immutable.
But these models provided too many restrictions to us and to end users
and in the end we found these ideals to be unrealistic, even misguided,
when applied to this kind of thing.
Yes maintenance and debug cost for us is important - more important than
you know, even. But, a framework has one (or a constant number of)
developer(s), whereas it has many, many users - a number which grows
without bound, if a framework is successful. In this light, saving cost
to the developer(s) of the framework (even in this hypothetical
situation) is meaningless if that cost savings means your users have to
make up the difference.
In any event though, it's just an academic debate because modules in the
Jigsaw implementation are not, and never have been, immutable. The
methods on Module and Layer.Controller that do exist could never
function otherwise, and there's the matter of dynamic proxies of course,
which also generate packages at run time. If the JDK must do it for its
framework support classes, it's a safe bet that others need to as well.
To produce an obvious and trivial example, any EJB-compliant
implementation must also have the ability to generate dynamic proxy
classes in the same way that the JDK does (but with a different base
class). JPA uses all kinds of proxies. Many frameworks and specs
require generation of classes at run time. And so on. Every one of
these things have the same security requirements as JDK proxies.
So in order to support the immutability argument, now containers have to
guess every package they might need to generate or load into up front,
and frankly most containers don't work that way. Why? Because until
now, class loading was 100% lazy, and this is has proven to be a *good
> Also, in term of implementation, making the content of a module mutable can be seen as a cache invalidation issue, and we all know that cache invalidation is a hard problem especially when it's on global states.
This I find to be a weaker argument. What cache are we talking about,
and what are the practical problems that make this intractable?
> With JPMS, a module is now part of the Java language, so it seems weird to make it mutable the same way, a java.lang.Class is not mutable in Java.
> But that's not true, in fact a java.lang.Class is mutable in Java, but you can only mutate it using an agent.
It's clear that modules are not to be treated like classes, because (for
example) circularity among modules is considered "bad" whereas
circularity among classes has been shown to be indispensable, and
classes within a class loader or module are loaded and resolved lazily
whereas modules are loaded and resolved aggressively, etc. But I see
your point that only "special" citizens ought to mutate a module. I
agree, but I do not think this should be limited to agents: containers
also have a similar need for similar reasons.
> I'm pretty sure that if we do not make the content of a module mutable, people will hack the JPMS implementation to make it mutable, because during the development you want to be able to add private package like you want to add new method in an existing class without having to restart the program (think things like JRebel or Vert.x here).
> So in my opinion, we should have a mechanism to add private package to a module, there real question is should this mechanism only visible from an agent or should it be also available from the LayerController ?
Even considering Java EE alone requires the framework to have the
ability to do these things. It's not so hard to imagine that there are
many related use cases.
> Aside, there is something that make me uncomfortable in the mails from David or Thomas, I'm maybe wrong, but it seems to me that you are thinking that you will be able to propose to your users to use the JPMS runtime with the current existing modules. I think your are chasing an unicorn/trying to solve a problem nobody care, i maintain several OSGI bundles and i do not want an implementation of OSGI to see them as JPMS modules, not by default because they will not work, the encapsulation model of JPMS is far stricter than the current model (i think it's the same issue with JBoss Module but i've no experience with them). As a user, if i create an OSGI module with a module-info or a bundle manifest that let me specified that this is a JPMS module, then i want it to be seen as a JPMS module but not by default. We have introduced a modulepath which is not a replacement of the classpath exactly for the same reason. So the interropt between OSGI and JPMS or JBoss Module and JPMS should be seen in my opinion with that idea in mind. In that case, supporting fragment that have also a JPMS module can be seen as a corner case, having to iterate over private packages is also not an issue because the existing manifest and the manifest of a JPMS module can be different, having more info, so at runtime the private packages will be available.
There are multiple answers. First, JPMS justified the new reflection
capabilities on a security basis. And believe me, I argued hard against
that. :-) If this basis is to hold up, then containers need to be able
to extend these guarantees and options to their deployments. To quote
Mark, "The lead of Oracle's Java Vulnerability Team estimates that at
least a third of all the vulnerabilities reported since JDK 7 GA would
have been prevented if we'd had the ability to strongly encapsulate
JDK-internal packages." We (JBoss) have had our own share of CVEs as
well, and the same arguments could be applied here just as easily. Why
should containers be second-class when it comes to security? How does
it impact users to say "we've solved a major problem with security but
the solution is only available to code and containers that have not been
written yet, sorry"?
Second, JPMS introduces a new deployment and packaging model. If users
use them, containers will need to support them. And AFAICT Java EE 9 is
still operating under the assumption that this packaging model will be
used by deployments. So in order to interoperate with them, existing
systems (the same existing systems that have the dynamicity requirements
above) have to be able to synthesize a way for such modules to be
deployed and interoperate with the existing systems. The best way is to
have the actual JPMS implementation do this in the normal way.
These problems (and most others that I've raised) are derived from the
idea we have chosen the implementation, and then are deciding what use
cases are valid based on the design parameters of the chosen
implementation. Instead we *must* have a design and implementation
which meets the use cases.
Any pretense of being guided by the requirements has long since been
exposed as being applied inconsistently (at best) as it suites the
implementation. But we can't decide that the use case of
interoperability is invalid just because the implementation does not
presently make it convenient.
More information about the jpms-spec-experts