invokevirtual on package private override in different classloader

Karen Kinnear Karen.Kinnear at Sun.COM
Mon Apr 27 10:32:31 PDT 2009


Are there particular languages with which you are involved?
Or specific concerns you have relative to OSGi?

I am explicitly copying John Rose, who has been instrumental
in the multi-language vm efforts, and Alex Buckley, the
current owner of the Java Virtual Machine Specification.
They can better respond to the directions of the specifications
in this context.

For the specific issue you raised, the amendment did not change
the reliance of the JVMS on the Java Language Specification, so
I'll assume your focus is on the  larger context of the question,
and this was just an example.


p.s. don't worry about the monaco.sfbay link, that was just
an explanation that for backward compatibility we are not
changing the package private interpretation for invokevirtual.

Jeffrey Sinclair wrote:
> Karen,
> Thank you for getting back to me with a detailed explanation.
> The concern I have is that there is a clear coupling between the JVM
> specification and the language specification in order to prove the
> correctness of my observations. Whilst I understand that the JVM
> specification is there to support the language specification, I was
> always under the impression that a formal proof of correctness for the
> JVM specification could be done in the absence of the language
> specification.
> To quote the JVM specification itself (section 1.2):
> "The Java virtual machine knows nothing of the Java programming
> language, only of a particular binary format, the class file format. A
> class file contains Java virtual machine instructions (or bytecodes) and
> a symbol table, as well as other ancillary information."
> I appreciate that as the language evolves, additions maybe added to the
> JVM specification to deal with advances in the language, however the
> amendments which you quote appear to break a fundamental principle which
> is explicitly stated in the JVM specification itself, namely the
> awareness required by the JVM on the language.
> My personal opinion is that this decoupling becomes of real importance
> as the number of languages that use the JVM as their implementation
> grows and specifically the rules governing package private is likely to
> impact technologies such as OSGI.
> At the moment I feel that the amendment, whilst it may have strived to
> remove ambiguity, has resulted in a more serious issue and should be
> revoked. Instead the ambiguity should be resolved within the constraints
> of the JVM specification itself.
> I could be way off here, please correct any misunderstanding I have.
> Unfortunately I could not access '' to read up on
> the package private discussions, the lookup fails.
> Regards,
> Jeff
> On Fri, 2009-04-24 at 08:43 -0400, Karen Kinnear wrote:
>>Thanks David for catching that.
>>On Apr 23, 2009, at 6:55 PM, David Holmes - Sun Microsystems wrote:
>>>Hi Karen,
>>>I was okay with the explanation until this point:
>>>>This discussion of access control omits a related restriction on the
>>>>target of a protected field access or method invocation (the target
>>>>must be of class D or a subtype of D). That requirement is checked  
>>>>part of the verification process (§5.4.1); it is not part of link- 
>>>>access control.
>>>>The point here is that the virtual machine in the case of  
>>>>does an accessibility check for the compile-time resolved method,
>>>>Square.getColour() in your example, but does NOT perform an
>>>>accessibility check on the link-time resolved method, or
>>>>CustomSquare.getColour() in your example.
>>>I don't see the relevance of the quoted rule in this case, because  
>>>this is not about the target of protected field/method access.
>>>As far as I can see the reason there is no IllegalAccessError is  
>>>because there is no illegal access:
>>>- statically Printer's call to Square.getColor is fine as they are  
>>>in the same compile-time package
>>>- at run-time because CustomSquare.getColor does not override  
>>>Square.getColor (because they are in different run-time packages),  
>>>CustomSquare.getColor is not selected, but rather Square.getColor is
>>>- At runtime Printer and Square are in the same runtime package, so  
>>>Printer's invocation of Square.getColor is legal - even on a  
>>>CustomSquare instance
>>>In short there is no IllegalAccessError because  
>>>CustomSquare.getColor is not selected to be called.
>>>David Holmes
>>>Karen Kinnear said the following on 04/24/09 07:35:
>>>>Thank you so much for making this easy for me to reproduce.
>>>>I think you have two questions here:
>>>>1) why do you get "red" when you use a different class loader
>>>>for CustomSquare and
>>>>2) why do you not get an IllegalAccessError when calling
>>>>package private CustomSquare.getColour() from a
>>>>different runtime package.
>>>>These are very important questions and use cases related to
>>>>the first caused significant confusion in the past which
>>>>actually led to a specification modification.
>>>>Let me see if I can explain, first why "red", not "blue",
>>>>and second why you don't get an IllegalAccessError.
>>>>First, I too found that Printer and Square are loaded by the
>>>>same class loader, and CustomSquare is loaded by a different class
>>>>loader. I ran a fastdebug vm with the flag: -XX: 
>>>>+PrintSystemDictionaryAtExit. This will print out on exit, the
>>>>internal vm cache of loaded classes including their class loaders.
>>>>To summarize/abbreviate the scenario:
>>>>class Main: // class loader 1
>>>>  Printer.print((Square)CustomSquare instance);
>>>>package 1, Printer.print // class loader 1
>>>>   System.out.println(square.getColour())
>>>>package 1, class Square // class loader 1
>>>>   public getColour() returns "blue"
>>>>package 1, class customSquare // class loader 2
>>>>   package private getColour() returns "red"
>>>>The first question is why does the call in Printer to  
>>>>print "red" when the class customSquare is loaded by
>>>>a different class loader, but print "blue" when loaded by the
>>>>same class loader.
>>>>Let's start with the call in Printer. This translates to
>>>>The Clarifications and Amendments to Java Virtual Machine  
>>>>Specification, 2nd edition definition of invokevirtual says:
>>>>The lookup algorithm used by the invokevirtual  instruction (pages  
>>>>291-292 of the JVMS, 2nd edition) should be revised as follows:
>>>>   Let C be the class of the target of the method invocation. The  
>>>>actual method to be invoked is selected by the following lookup  
>>>>   If C contains a declaration for an instance method M with the  
>>>>same name and descriptor as the resolved method, and M overrides  
>>>>the resolved method , then M is the method to be invoked, and the  
>>>>lookup procedure terminates.
>>>>   Otherwise, if C has a superclass, this same lookup procedure is  
>>>>performed recursively using the direct superclass of C ; the method  
>>>>to be invoked is the result of the recursive invocation of this  
>>>>lookup procedure.
>>>>   Otherwise, an AbstractMethodError is raised.
>>>>The text highlighted in red is the only change. It replaces the text
>>>>   and the resolved method is accessible from C
>>>>in the current specification. See the revision of the definition of  
>>>>method override in the JLS 3rd Edition page for more details.
>>>>// First we do the static compile-time resolution using the  
>>>>constant pool, which in this case refers to Square.getColour()
>>>>The named method is resolved (§
>>>>// Then we do the dynamic run-time resolution using CustomSquare as  
>>>>the // target of the method invocation
>>>>   If C contains a declaration for an instance method M with the  
>>>>same name and descriptor as the resolved method, and M overrides  
>>>>the resolved method , then M is the method to be invoked, and the  
>>>>lookup procedure terminates.
>>>>// The key question is: Does CustomSquare.getColour() override
>>>>// Square.getColour()?
>>>>// I'll get to the definitions of inherit and override in a second.
>>>>// Result is:
>>>>// If they use the same class loader: yes CustomSqure.getColour()  
>>>>overrides Square.getColour()
>>>>// If they use different class loaders: no CustomSquare.getColor()  
>>>>does NOT override Square.getColour()
>>>>   * Otherwise, if C has a superclass, this same lookup procedure  
>>>>is performed recursively using the direct superclass of C ; the  
>>>>method to be invoked is the result of the recursive invocation of  
>>>>this lookup procedure.
>>>>// If CustomSquare.getColour() does NOT override Square.getColour(),
>>>>// then we use the method Square.getColour(), since Square is the
>>>>// direct superclass of CustomSquare.
>>>>Definitions of Inherit and Override:
>>>>Inheritance rules: JLS 3rd edition, 8.4.8 Inheritance, Overriding,  
>>>>and Hiding
>>>> A class C inherits from its direct superclass and direct  
>>>>superinterfaces all non-private methods (whether abstract or not)  
>>>>of the superclass and superinterfaces that are public, protected or  
>>>>declared with default access in the same package as C and are  
>>>>neither overridden (' nor hidden (' by a  
>>>>declaration in the class.
>>>>JLS 3rd edition Overriding (by Instance Methods)
>>>>An instance method m1 declared in a class C overrides another  
>>>>instance method, m2, declared in class A iff all of the following  
>>>>are true:
>>>>     1. C is a subclass of A.
>>>>     2. The signature of m1 is a subsignature (§8.4.2) of the  
>>>>signature of m2.
>>>>     3. Either
>>>>        o m2 is public, protected or declared with default access  
>>>>in the same package as C, or
>>>>        o m1 overrides a method m3, m3 distinct from m1, m3  
>>>>distinct from m2, such that m3 overrides m2.
>>>>Moreover, if m1 is not abstract, then m1 is said to implement any  
>>>>and all declarations of abstract methods that it overrides.
>>>>Note that based on the JVMS 3rd edition transitive overriding  
>>>>rules, we need to do an override check first for the direct  
>>>>superclass and if the current class does not override the direct  
>>>>superclass, recursively for the superclass' superclass.
>>>>So CustomSquare.getColour() does NOT override Square.getColour()  
>>>>it is NOT in the same runtime package as C.
>>>>Second question:
>>>>Why do you not get an IllegalAccessException when invoking package- 
>>>>private CustomSquare.getColour() from Printer.print() which
>>>>is in a different package?
>>>>We had the same question ourselves a couple of years ago and
>>>>researched this.
>>>>If you look closely at JVMS 2nd edition 5.4.4 Access Controls,
>>>>which goes into the details of accessibility checks for methods,
>>>>read the final sentences which say:
>>>>This discussion of access control omits a related restriction on  
>>>>the target of a protected field access or method invocation (the  
>>>>target must be of class D or a subtype of D). That requirement is  
>>>>checked as part of the verification process (§5.4.1); it is not  
>>>>part of link-time access control.
>>>>The point here is that the virtual machine in the case of  
>>>>does an accessibility check for the compile-time resolved method,
>>>>Square.getColour() in your example, but does NOT perform an
>>>>accessibility check on the link-time resolved method, or  
>>>>CustomSquare.getColour() in your example.
>>>>We proposed changing this behavior, and given that it has always been
>>>>this way, changing the specification and the multiple virtual
>>>>machines in the field would be highly likely to break multiple
>>>>shipping applications. So we decided it was not a good change
>>>>to make.
>>>>I hope this helps explain the current behavior, do let me
>>>>know if any of this needs further clarification or if I
>>>>misunderstood your original questions.
>>>>p.s. you can get even more explanation of the inheritance,  
>>>>overriding, invokevirtual, invokespecial, etc. behavior, etc. in
>>>>the evaluation of the following bug:
>>>>p.p.s. The package private discussion is under
>>>>Jeffrey Sinclair wrote:
>>>>>Just to say that there is one slight bug in my reproduction  
>>>>>the result of 'Red' stays the same).
>>>>>My debug statements for which class loader the class was loaded  
>>>>>from is
>>>>>incorrect, they are done in the constructor when they should have  
>>>>>done in a static block in the class.
>>>>>public class Square {
>>>>> static {
>>>>>   System.out.println("Square loaded by classloader: "         +  
>>>>> }
>>>>>This will give the output:
>>>>>Printer loaded by classloader: sun.misc.Launcher 
>>>>>$AppClassLoader at 553f5d07
>>>>>Square loaded by classloader: sun.misc.Launcher 
>>>>>$AppClassLoader at 553f5d07
>>>>>CustomSquare loaded by classloader: at 3ae48e1b
>>>>>Which is still unexpected since I'm getting 'Red'.
>>>>>On Sat, 2009-04-18 at 17:32 +0100, Jeffrey Sinclair wrote:
>>>>>>Please find attached a zip file which packages the example up  
>>>>>>with a
>>>>>>build script that uses javac and a plain old run script.
>>>>>>Simply run followed by (assumes that javac and  
>>>>>>java are
>>>>>>on the PATH).
>>>>>>On Thu, 2009-04-16 at 17:42 -0400, Karen Kinnear wrote:
>>>>>>>Thank you for sending me this information. I have a theory, but I
>>>>>>>would be much more comfortable running the test first. I have   
>>>>>>>with this, but I'd feel much more comfortable duplicating your  
>>>>>>>I do appreciate you offering to package this up using plain old  
>>>>>>>javac/  and
>>>>>>>a run script - I don't have Eclipse installed and much as I've  
>>>>>>>heard it
>>>>>>>is a great product, I don't have the cycles to do that in the  
>>>>>>>near  future.
>>>>>>>thanks so much,
>>>>>>>On Apr 14, 2009, at 5:33 PM, Jeffrey Sinclair wrote:
>>>>>>>>Thanks for getting back to me.
>>>>>>>>I was using 1.6.0_10 and have now tried 1.6.0_13 and get the same
>>>>>>>>result. Specifically I've tried the following versions:
>>>>>>>>java version "1.6.0_10"
>>>>>>>>Java(TM) SE Runtime Environment (build 1.6.0_10-b33)
>>>>>>>>Java HotSpot(TM) 64-Bit Server VM (build 11.0-b15, mixed mode)
>>>>>>>>java version "1.6.0_13"
>>>>>>>>Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
>>>>>>>>Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02, mixed mode)
>>>>>>>>I've attached my source as Eclipse projects. The URL in the  
>>>>>>>>main  method
>>>>>>>>points to the class files generated by the project with the
>>>>>>>>CustomSquare. I've also added some debug info relating to which  
>>>>>>>>loader is being used to load the Square, Printer and Custom  
>>>>>>>>Printer loaded by classloader: sun.misc.Launcher  
>>>>>>>>$AppClassLoader at 553f5d07
>>>>>>>>Square loaded by classloader: at 3ae48e1b
>>>>>>>>CustomSquare loaded by classloader:  
>>>>>>>> at 3ae48e1b
>>>>>>>>I still get Red. The key point is that the CustomSquare is not  
>>>>>>>>at all to the Printer code. If I stick the CustomSquare in the  
>>>>>>>>project as Printer it will then be loaded by the AppClassLoader  
>>>>>>>>and I
>>>>>>>>get blue. However when it is in a completely separate project  
>>>>>>>>and  loaded
>>>>>>>>via the URLClassLoader (and Printer loaded via the  
>>>>>>>>AppClassLoader) I
>>>>>>>>always get Red. If I then change getColour() to be public in  
>>>>>>>>both  Square
>>>>>>>>and CustomSquare I get blue.
>>>>>>>>This is completely baffling me. I may be doing something really  
>>>>>>>>but I can't see it :)
>>>>>>>>Let me know if you can't replicate the issue with the attached  
>>>>>>>>project and I'll try to package it up using plain old javac and  
>>>>>>>>a run
>>>>>>>>Even if you do get blue with the different ClassLoaders for  
>>>>>>>>Printer  and
>>>>>>>>CustomSquare, I would probably argue that this is incorrect but  
>>>>>>>>I have
>>>>>>>>to admit that this argument could be because I've misunderstood  
>>>>>>>>meaning of 'runtime package' in terms of package private  
>>>>>>>>in the JVM spec.
>>>>>>>>On Tue, 2009-04-14 at 15:30 -0400, Karen Kinnear wrote:
>>>>>>>>>Perhaps you can help me duplicate the problem.
>>>>>>>>>First - what does java -version say?
>>>>>>>>>I took the source code appended, and modified the URL setting,
>>>>>>>>>which I believe should just give me your second example, i.e.
>>>>>>>>>CustomSquare instantiated in a different ClassLoader than  
>>>>>>>>>When I run this with Sun's 1.6 or recent 1.7 I get "blue".
>>>>>>>>>What did we do differently?
>>>>>>>>>Jeffrey Sinclair wrote:
>>>>>>>>>>I've been struggling to understand why HotSpot does not throw  
>>>>>>>>>>IllegalAccessError when a call is made to an override of a  
>>>>>>>>>>private method on an instance loaded by a different class  
>>>>>>>>>>loader.  I was
>>>>>>>>>>hoping someone could explain to me in technical detail where  
>>>>>>>>>>I'm  going
>>>>>>>>>>wrong because it appears to be a bug.
>>>>>>>>>>(all source code is pasted at the end of the mail)
>>>>>>>>>>I have a class called Square with a package private method  
>>>>>>>>>>getColour() which returns 'red'. I have a subclass of Square  
>>>>>>>>>>CustomSquare which overrides getColour() to return 'blue'. I  
>>>>>>>>>>another class called Printer which simply prints out  
>>>>>>>>>>getColour()  for the
>>>>>>>>>>Square passed to it. I then have two test cases:
>>>>>>>>>>* Printer.print(Square) is called with a CustomSquare   
>>>>>>>>>>instantiated in
>>>>>>>>>>the same ClassLoader as Printer.
>>>>>>>>>>* Printer.print(Square) is called with a CustomSquare   
>>>>>>>>>>instantiated in
>>>>>>>>>>a different ClassLoader as Printer.
>>>>>>>>>>What I find is that I get 'blue' in the first test (as  
>>>>>>>>>>expected) and
>>>>>>>>>>'red' in the second test (not expected).
>>>>>>>>>>>From the Access Control constraints in the Linking section  
>>>>>>>>>>>of the  JVM
>>>>>>>>>>specification (5.4.4), I as expecting an IllegalAccessError  
>>>>>>>>>>to be  thrown
>>>>>>>>>>because it states that a package private method is accessible  
>>>>>>>>>>to a  class
>>>>>>>>>>if it is declared by a class in the same runtime package.
>>>>>>>>>>My understanding is that Printer is not in the same runtime   
>>>>>>>>>>package as
>>>>>>>>>>CustomSquare which explains why my override does not kick in,  
>>>>>>>>>>but it
>>>>>>>>>>does not explain why an IllegalAccessError is not thrown.
>>>>>>>>>>I was wondering if someone could explain the behaviour to me.
>>>>>>>>>>The source code:
>>>>>>>>>>public class Main {
>>>>>>>>>>public static void main(String[] args) throws Exception {
>>>>>>>>>> URL[] urls = new URL[]{new URL("path/to/CustomSquare")};
>>>>>>>>>> URLClassLoader loader = new URLClassLoader(urls);     Class  
>>>>>>>>>>clazz =
>>>>>>>>>> Printer printer = new Printer();
>>>>>>>>>> printer.print((Square)clazz.newInstance()); // 'red' gets  
>>>>>>>>>>}   }
>>>>>>>>>>public class Printer {
>>>>>>>>>>public void print(Square square) {
>>>>>>>>>> System.out.println(square.getColour());
>>>>>>>>>>public class CustomSquare extends Square {
>>>>>>>>>>public CustomSquare() {
>>>>>>>>>> super(5);
>>>>>>>>>>public String getColour() {
>>>>>>>>>> return "blue";
>>>>>>>>>>public class Square {
>>>>>>>>>>private float length;
>>>>>>>>>>public Square(float length) {
>>>>>>>>>> this.length = length;
>>>>>>>>>>public float calculateArea() {
>>>>>>>>>> return length * length;
>>>>>>>>>>String getColour() {
>>>>>>>>>> return "red";

More information about the hotspot-dev mailing list