Updates on Interoperability

Richard S. Hall heavy at UNGOVERNED.ORG
Tue Apr 29 14:10:52 PDT 2008

Bryan Atsatt wrote:
> Richard S. Hall wrote:
>> Bryan Atsatt wrote:
>>> Richard S. Hall wrote:
>>>> Sam Pullara wrote:
>>>>> Did we decide which way it will be compatible?  I'm a little 
>>>>> concerned that implementing something that can use OSGi modules is 
>>>>> a far bigger task that implementing something that OSGi can use.  
>>>>> If we are going for total compatibility it seems like we should 
>>>>> just use OSGi.  Personally I wanted something in Java that was 
>>>>> simpler and cleaner that OSGi-users could leverage if they 
>>>>> wanted.  But I'm coming at this from the 
>>>>> we-need-something-like-maven/gem/ivy camp.
>>>>> The vast majority of people that are going to use this system will 
>>>>> have never heard of OSGI nor care about any of their more subtle 
>>>>> features and just want an easy way to leverage the vast amount of 
>>>>> libraries available (virtually all of them are not OSGi modules 
>>>>> today but are jar files in maven repositories).  We shouldn't be 
>>>>> specing out some sort of uber container but rather a simple way to 
>>>>> tie code to its dependencies.
>>>> If this is all that people wanted, then we could just define 
>>>> repositories and associated metadata and forget about all runtime 
>>>> modularity support, but I strongly disagree that this is all people 
>>>> want or else there wouldn't be such an upsurge in OSGi interest and 
>>>> adoption.
>>>>> I strongly suggest KISS applies to our effort.  I write this 
>>>>> knowing that the committee seems to be heavily in favor of this 
>>>>> alternate view that we implement the kitchen sink.
>>>> I think our interest is similar, but perhaps our conclusions are 
>>>> different. Initially, I think my comments were construed as an 
>>>> argument against split packages in general, but I am really arguing 
>>>> the same as you (I think) that we don't really need split packages 
>>>> to be a concept in our "interoperability API", if that is how we 
>>>> are to view the 277 API. Of course, I am also willing to have the 
>>>> former argument too, but I will leave that for another time. :-)
>>>> From my point of view, I do not think it is wholly reasonable to 
>>>> assume that we want to try to accomplish interoperability across 
>>>> module systems, where that includes arbitrarily sharing split 
>>>> packages across module systems.
>>> Nor do I, but I'm trying to understand the implications if we assume 
>>> that OSGi will continue to split packages.
>>>> Let me try to accurately conjure the peculiarities of split 
>>>> packages in the OSGi module layer...
>>>> We support split packages in two ways, via required bundles (i.e., 
>>>> module dependencies) and via bundle fragments. In general, we 
>>>> recommend that if you split your packages that you attach mandatory 
>>>> attributes on them so that importers do not get them by mistake. By 
>>>> requiring a bundle, you can ignore mandatory attributes and get 
>>>> everything a bundle has to offer and combine it with other required 
>>>> bundles that might be exporting other parts of the split packages. 
>>>> Given this, there is a semi-reasonable chance* that we can tell you 
>>>> which bundles are contributing to the "complete" packages, but we 
>>>> have no way of knowing if/when combined split packages are 
>>>> complete. Further, we have to assume that split package parts 
>>>> offered by different bundles do not overlap, because if we allowed 
>>>> them to overlap then we would have shadowing and ordering issues. 
>>>> Again we have no easy way to verify that they do not overlap, so we 
>>>> just assume they don't.
>>>> Fragments on the other hand are a little trickier, since split 
>>>> packages from them are loaded from the same class loader to provide 
>>>> package private access (which is not afforded for split packages in 
>>>> the require bundle approach). The "host" bundle doesn't really know 
>>>> about fragments and it is possible for fragments to shadow or 
>>>> extend packages in the "host" bundle. Since fragments provide their 
>>>> split packages in an implicit way (i.e., they largely just become 
>>>> part of the host bundle's class path), there is no easy way to 
>>>> determine if a fragment is contributing to a given package or not. 
>>>> (* This feature of fragments also makes it so it is not really 
>>>> possible to know who is contributing classes to a split package in 
>>>> the require bundle case.)
>>> Ok, so first I assume it would make sense for an OSGi Repository to 
>>> simply never return a ModuleDefinition for a fragment.
>> I am not sure I understand your point. Host bundles would be returned 
>> and during the resolve process, fragments might be attached to that 
>> host, thus modifying its content in ways that we cannot predict.
> I simply meant that a fragment would never be returned by itself, as a 
> separate ModuleDefinition.
>>> And the "you don't know what you've got till it's resolved" problem 
>>> is going to be present in any module system for which package name 
>>> is not a required component of a dependency expression. But as long 
>>> as re-export is not the default behavior (it isn't), doesn't this 
>>> really just boil down to the leakage problem? And this is addressed 
>>> in 277 the same as it is in OSGi: you must declare such "leaks" as 
>>> re-exports ("uses"). Not perfect, but probably good enough.
>>> So I think we're left with the issue of a single search visiting 
>>> multiple loaders that can return the same package. This won't happen 
>>> in the fragment case, since the "split" gets mended at runtime, but 
>>> it will in the require-bundle case. And this will manifest as 
>>> multiple entries in the import list which export the same package.
>>> But even this isn't an issue, as long as dependency resolution 
>>> relies solely on declared exports and re-exports of a module, and 
>>> not on what is available by browsing the imports of that module.
>>> So I'm back to my original statement that we just need to 
>>> acknowledge that package duplication may exist in an import list.
>>> Or am I missing something here?
>> I am just not sure why we want to make the fact that some module 
>> systems support split packages a visible concept in our 
>> "interoperability API", when it would be difficult to get 
>> interoperability around split packages across module systems when we 
>> already have issues with split packages within a single module 
>> system. Especially, given my description above, since we would not 
>> easily know which modules were contributing to a split package in OSGi.
> I was just exploring whether or not we need to make it a visible 
> concept in order for cross module system resolution to work correctly. 
> You may note in Stanley's doc that he simply took the position that a 
> JAM definition would fail to resolve against an OSGi definition that 
> split packages. This may be an acceptable solution, but it still 
> requires a means to *detect* that a split has occurred.

Understood and that is precisely the point...it is difficult to 
determine that a split has occurred...perhaps not impossible, but the 
OSGi spec avoided dealing with it due to the inherent complexity. For 
the most part, if you start splitting packages, you are on your own and 
don't get much explicit support from the OSGi framework.

-> richard

>>>> I think that is fairly complete description of the situation, sorry 
>>>> if it is somewhat terse. In summary, I wouldn't assume that we can 
>>>> get reasonable interoperability at this level. If you need these 
>>>> types of features, then you should stick to a single modularity 
>>>> framework...preferably OSGi. ;-)
>>>> But my overall point is, I believe that if we can support 
>>>> interoperability at module- and package-level dependencies across 
>>>> module systems, then I think we would hit most use cases.
>>>> -> richard
>>>>> Sam
>>>>> On Apr 25, 2008, at 1:57 PM, Richard S. Hall wrote:
>>>>>> Bryan Atsatt wrote:
>>>>>>> Richard S. Hall wrote:
>>>>>>>> Gordon Hirsch wrote:
>>>>>>>>>> 1. Map its dependency declarations into a standard runtime 
>>>>>>>>>> representation.
>>>>>>>>>> 2. Support a standard search model over its stored modules, 
>>>>>>>>>> using the runtime dependency expressions.
>>>>>>>>>> 3. Map its stored module data into a standard runtime 
>>>>>>>>>> representation that can be returned by the search.
>>>>>>>>> One challenge lies in defining the "standard runtime 
>>>>>>>>> representations" and "standard search model" in a universal 
>>>>>>>>> enough way to encompass OSGi and other module systems. This 
>>>>>>>>> implies embracing concepts (in these standard representations 
>>>>>>>>> and search model) that were not universally liked by the EG 
>>>>>>>>> early on. (Split packages and package-level import/export come 
>>>>>>>>> to mind.)
>>>>>>>> Package-level import/export seem like a must, but I still don't 
>>>>>>>> see split packages as a necessity.
>>>>>>> Strongly agreed on import-by-package, and that ModuleDefinition 
>>>>>>> requires a method to enumerate exported packages (and probably 
>>>>>>> an exportsPackage(String packageName) method to enable an 
>>>>>>> efficient Query).
>>>>>>> On the split package front, however, it seems a little fuzzy to 
>>>>>>> me. If an OSGi implementation of 277 is also going to remain 
>>>>>>> OSGi Rx compliant, it will still need to support split packages, 
>>>>>>> right? If so, don't we need to surface this fact at the 277 
>>>>>>> level? Perhaps that is as simple as acknowledging that 
>>>>>>> Module.getImportedModules() may return more than one instance 
>>>>>>> that exports a given package.
>>>>>> That all depends on if the 277 API is a subset of OSGi 
>>>>>> functionality or equal to it, I suppose.
>>>>>> -> richard
>>>>>>> // Bryan

More information about the jsr277-eg-observer mailing list