Please stop incrementing the classfile version number when there are no format changes
brian.goetz at oracle.com
Sun Oct 13 15:42:39 UTC 2019
Let me try to close off this thread (please) with a summary of the takeaways for interested observers.
There are several pervasive, but wrong, assumptions in the argument being given here (and elsewhere):
- That the only reason to version the classfile is “format changes”
- That format changes are rare
Both of these assumptions are naive.
First, format changes are not so rare. Yes, there has only been one new bytecode added in 13 releases. But 1/13 is still 7.5% — that’s not rare. And have also been three versions (5, 7, and 11) that added new _constant pool forms_. That’s almost 25%! Not rare at all. (If your website failed on 25% of requests, you’d not get away with calling that “rare failures”, you’d be out of business.)
But so-called “format changes” merely scratch the surface of the sorts of dependencies inherent in translation from source to class file, for which versioning is warranted. Additional forms of dependencies include:
- New class file attributes. The JVMS defines a number of attributes that carry important semantic information, including those that control access control decisions, such as NestMembers, PermittedSubtypes, Module (and friends), etc. Yes, the class file format was designed so that unrecognized attributes could be “skipped” without losing your place in parsing — but that doesn’t mean that it is reasonable for a class file processor who claims to understand that class file version to ignore them! New class file attributes are also quite common, with additions in 5, 7, 8, 9, and 11. Not rare, and you should not be parsing (let alone instrumenting) Java 13 classifies if you don’t understand the semantics of the attributes defined for Java 13 classfiles.
- Dependencies between translation strategy and JDK libraries. It is not uncommon that language features require library support; the foreach loop depends on Iterable (5), try-with-resources depends on Autocloseable (7), and lambdas depend on the LambdaMetafactory bootstrap (8). And going forward, most language features will have this characteristic (records, sealed types, pattern matching.) Not bumping the class file version here would be setting a trap for users, because while they might look like they conform to an older class file spec, they won’t run on older JDKs.
- Change in classfile validity. For example, prior to Java 9, it was an error for a method in an interface to have ACC_PRIVATE; now that is valid. While the “format” hasn’t change, the semantics has. This is also not so infrequent.
As you can see, both the assumptions — that “format changes” should be defined by the most narrow interpretation of “can I parse this without losing my place”, and the assumption that changes that require us to bump the class file version are rare — are naive. Even if we didn’t bump the class file version “unless we had to”, such bumps would happen in probably 80% of releases anyway.
People seem to have convinced themselves that it is safe, and therefore should be encouraged, to parse classfiles whose semantics are not understood by the code that is parsing them. (I find it somewhat astonishing that I should have to be making this point, but there it is.) This is the bug in this discussion, and this is exactly why the classfile format is strongly versioned in the first place!
Parsing class files is advanced business, and carries a high responsibility to get it right (including understanding all the attributes), and keep up with changes. Don’t ask us to move more slowly because some parties cannot keep up. That would be unfairly punishing the other 99.9999% of users for the failings of a few.
> On Oct 11, 2019, at 8:45 AM, Mike Hearn <mike at plan99.net> wrote:
> Since the new versioning strategy was introduced, the classfile format
> number increments with every release regardless of whether or not anything
> was actually added. The way the class format is versioned is proving
> disastrous for the community. Please consider a policy change to not
> increment the format unless there are actually new features added.
> We are currently in the following situation:
> Java 11 has serious regressions in SSL support that are fixed, but only in
> Java 13, and not backported in OpenJDK. So we have to use a vendor spin of
> Java 11 with these fixes backported.
> We tried our app with Java 13 but discovered Gradle doesn't work, so we
> can't build it.
> Gradle doesn't work because Groovy doesn't work.
> Groovy doesn't work because it uses ASM and ASM rejects classfiles with the
> new format number. The Groovy fix is here:
> There's no *actual* change needed - simply updating to a new ASM was
> sufficient to fix it.
> ASM doesn't work because it doesn't know if anything important was changed
> in the new classfile version. But in Java 13 nothing actually changed and
> thus the ASM changes are also no-ops:
> Propagating this 1 character version number check change in ASM through the
> entire Java ecosystem dependency stack is a nightmarish task. Many
> libraries fat-jar ASM so you can't even just change it on the classpath to
> fix things ... many projects need to do new releases, and anything
> depending on them needs to do new releases and so on, recursively.
> In practice this means Java 13 is ... just like its predecessors ...
> massively breaking and backwards incompatible for no good reason at all. We
> have no choice but to ignore it for a while until the new ASM propagates,
> by which time Java 14 will be out and the cycle repeats. I think we'll be
> paying vendors for Java 11 backports for a considerable length of time as a
> Ideally the format version wouldn't be incremented unless a semantic change
> would cause old software to mis-interpret the new files without realising
> it. Many types of change e.g. new CP entry types look like they can be
> added without a version change, because if old software encountered them
> they'd throw some more precise kind of exception, and that would scope
> errors to only the files that actually used the new features instead of
> every file compiled by a new release.
> Could this policy please be revisited? It's crushing the communities
> ability to upgrade.
More information about the jdk-dev