What's the status of / relation between "JEP 169: Value Objects" / "Value Types for Java" / "Object Layout"

Gil Tene gil at azulsystems.com
Tue Feb 3 21:08:43 UTC 2015


> On Feb 3, 2015, at 12:20 PM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
> 
> 1) My assumption is that vast majority of classes will benefit from inline storage of their collaborators.  The waste would be in classes that have hot and cold members together, but I think that's the minority? But, for those cases, it would be beneficial to allow out-of-band layout (i.e. today's layout) via annotation or whatever.  

I *think* it's a small (and fragile) minority, rather than the vast majority. That is, I think that auto-magic (implicit optimization without a special factory based instantiation) is limited in potential scope and impact: Without special magic (see #2 below), 100% of current final field assignments are non-inlineable because they (by definition) assign the field to a reference value that refers to an already-allocated-at-assignment-time object.

#2 is hard enough to do for trivial case (those that initialize the field to refer to an in-place instantiated object that does not escape ahead of the assignment, and whose all escaping post-assignemnt references can be safely hunted down and converted). And it is much harder for the plentiful non-trivial cases (the allocation/instantiation are done "far enough apart" such that interleaved operations complexify the analysis).

But even if #2 is solved somehow for some subset of instantiations that the VM can auto-magically determine can be converted to a "placement-new" sort of allocation / construction sequence, there are plenty of final field assignment cases where the final ref field refers to an object that is already referred to elsewhere (before the assignment). E.g. it is quite common for private final ref fields to be initialized from constructor parameters (e.g. HashMap.Entry). It is also common to gave final ref fields assigned to the return values of method calls. And for those final fields to refer to object that are common across multiple referring instances. All those cases are non-inlineable by definition.

> Why do you say this would likely lead to memory waste? If the object is allocated one way or another, it's going to take up space.  The only case i see being problematic is when null is stored in majority of instances.

It's not nulls. It's a reference to something that isn't "inlineable". Like setting the final field to a reference to an already-allocated-at-assignement-time object.

If the VM can determine ahead of time (ahead of instantiating any instances of a containing class) that ALL final field assignments performed in the future will be made from in-place instantiations that can be safely and auto-magically converted to placement-news, then there will be no wasted space. However, for every class where some late-discovered (discovered after an instance has been created) situation leads to a loss of the optimization, potential memory waste is also there.

It's true that the memory waste would be contained to the cases where the reference is assigned to a non-"placement-new" result, and that in mixed situations the pre-allocated space in the containing object will still be used. It's just hard to guess how much of each case there is in real code. 

> I guess this boils down to whether one thinks it's more likely, by default, that objects embedded in others are hot/cold in terms of access; I think it's more common for them to be accessed together and so default should cater to that.
> 
> When/if inlined layout is available, next logical thing one may request is specifying layout *order* to try and place commonly accessed data on same cacheline.  This is less important for streaming cases, but would be nice for random walks.

I think cache line co-placement makes sense for fields (e.g. an @Together thing to mirror the @Contended stuff), but I doubt that it makes much sense for objects, mostly due to size (not that many realistic objects will fit together in one cache line anyway). A natural "inline in declaration order" approach for intrinsic objects is probably just as good as anything else, and will require no special annotation.

> 
> 2) yes, this basically requires a placement-new like thing to be implemented in the VM.  No disagreement that it's intrusive.
> 

It's more than intrusive,. It's incompatible with the semantics of all current object-instntiating code, making auto-magical optimization that coverts current instantiations to placement-new a "very hard" thing.

Unfortunately, both #1 and #2 would need to be addressed for an optimization to apply, so it's enough for one to be "too hard" for non of it to fly.

> sent from my phone
> 
> On Feb 3, 2015 2:10 PM, "Gil Tene" <gil at azulsystems.com> wrote:
> 
>> On Feb 3, 2015, at 9:13 AM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
>> 
>> Gil, not sure if you saw my reply to Volker, but I agree -- I was simply asking why request this optimization via explicit syntax and not do it automatically in the runtime (with all the same restrictions, caveats, fine print, etc).
> 
> Well, there are a few reasons why this is "very hard":
> 
> 1. It's "hard" to discern at runtime that an object assigned to a final reference field can/should be "inlined" into it's containing object:
> 
> 1.1 Object-Inlining decisions are not instance-specific, they are class-global.
> 
> 1.1.1 If a single containing instance inlines such an object, all instances of the same class must spend the space.
> 
> 1.1.2 If a single "containing" instance does not inline the object referred to by the final reference field, NONE of the instances of the containing class can gain from the deference-avoiding (dead reckoning) optimization.
> 
> 1.1.3 If we tried to inline all final reference declared objects without a specific declaration of intent, we'd likely end up with a lot  of wasted space.
> 
> 2. Construction semantics: Even if you could auto-discern the need to "inline" the objects, you face a hard problem is dealing with how the "inlined" object (not the final ref pointing to it) is constructed and initialized:
> 
> 2.1 Any factory-based instantiation of the referred to object is by-definition not inline-able (the factory based creation with a specific inlining intent and enough provided context, such as constructWithin(), being an exception).
> 
> 2.2 Current non-factory based instantiation options in Java (new, reflection, methodHandles) all perform their own internal allocation of instance storage, and would not be able to use the "inlined" space without some serious surgery. Even if such surgery to all internal instantiation forms was done, getting the target instance location to the instantiation code is also "very hard", given that instantiation logically occurs before the assignment of the resulting reference to the final field, and many operations can happen in between. Approaches that attempt to override a sequence of operations (e.g. the simplest stuff like new; dup; push arg1; push arg2; invoke_special; putfield (into final ref field), or much more complicated ones...) such that the intermediate heap reference is never exposed and can be replaced with a reference to the already-allocated space only work in trivial situations, and tend to fail in all sorts of interesting common-case ways (e.g. when perfectly innocent instrumentation is involved).
> 
> Now "very hard" probably does not mean impossible. But it has so much open ended stuff that makes it a huge thing to tackle safely. Given the fact that much of the context of org.ObjectLayout benefits will never be auto-discernible without using explicit statements to describe the expected semantics (using a StructuredArray to state optimization-enabling limiting semantics, for example), adding explicit intent declaration (as opposed to auto-optimization) for intrinsic objects seems natural there.
> 
>> 
>> On Tue, Feb 3, 2015 at 11:58 AM, Gil Tene <gil at azulsystems.com> wrote:
>> A couple of point here, specific to org.ObjectLayout (http://objectlayout.org):
>> 
>> Declaration:
>> The ObjectLayout @Intrinsic (http://objectlayout.org) annotation is used for declaring what you refer to as "inline" objects. It is specifically not intended to be a layout control directive, but an optimization hint. Whether or not a JVM "inlines" the intrinsic object within the containing one, and how/where that "inlining" happens becomes a JVM-specific implementation concern, and no a semantic one.
>> 
>> Field initialization:
>> 
>> Implicit, undeclared choices to "inline" all final referenced fields fail very quickly when attempted in practice. E.g. final fields can (and often will) be set to refer to pre-existing-at-construction-time objects, which are (by definition) impossible to "inline". In addition, there are many common uses final reference fields where "inlining" is no possible because the actual object size of the referred-to object is not a global constant (e.g. it will be set to a construction-time or parameter-based choice of subclass).
>> 
>> We've given Intrinsic Object initialization a lot of thought in org.ObjectLayout. The dedicated initialization API is there to assure several things, including exact-type (the field's specific declared type) choice. It could turn into a less-verbose version in some future JDK if language support was added (e.g. to avoid mentioning the needed-only-to-conform-with-syntax things like "this", and the field name in the constructWithin() call), but I expect the semantics to need to be similar even if the syntax was made less verbose.
>> 
>> -- Gil.
>> 
>> > On Feb 3, 2015, at 8:40 AM, Vitaly Davidovich <vitalyd at gmail.com> wrote:
>> >
>> > Hi Volker,
>> >
>> > Sorry, I may have been unclear in my question.  As you say, ObjectLayout
>> > requires that you annotate the fields that you'd like inlined and then also
>> > use special API to construct those objects.  I'm wondering whether,
>> > instead, all private final fields are automatically inlined, and only cases
>> > where you'd like to layout the field out-of-band would require annotation.
>> > This would be controlled via a cmdline flag, as you say, similar to perhaps
>> > how compressed oops are enabled (or not).  Note that I'm talking about
>> > purely layout of reference types, not value types.
>> >
>> > The "concern" with having to explicitly annotate and use dedicated APIs to
>> > opt-in is that adoption will be fairly low, whereas I think most of the
>> > time one would want inlined storage layout.
>> >
>> > Thanks
>> >
>> >
>> > On Tue, Feb 3, 2015 at 11:29 AM, Volker Simonis <volker.simonis at gmail.com>
>> > wrote:
>> >
>> >> Hi Vitaly,
>> >>
>> >> for PackedObjects/ObjectLayout you need to specially annotate the
>> >> classes and/or fields which you want to allocate "inline". Once you've
>> >> done that you have no choice with the PackedObjects approach.
>> >> ObjectLayout is a little special here, because it can run with any
>> >> Java VM in which case it will still use the default reference model.
>> >> But it can potentially be optimized by some VM's to provide the flat
>> >> object layout. I expect these optimizations to be controllable by a
>> >> command line option.
>> >>
>> >> With the "Value Types for Java" [1] approach you'll have the
>> >> possiblitly to express the behavior right in Java like in the
>> >> following example from [1]:
>> >>
>> >> final __ByValue class Point {
>> >> static Point origin = __MakeValue(0, 0);
>> >>
>> >> I think the default will always be "reference semantics" in Java but
>> >> with various degrees of freedom to optionally choose value semantics.
>> >>
>> >> Regards,
>> >> Volker
>> >>
>> >> [1] http://cr.openjdk.java.net/~jrose/values/values-0.html
>> >>
>> >> On Mon, Feb 2, 2015 at 9:19 PM, Vitaly Davidovich <vitalyd at gmail.com>
>> >> wrote:
>> >>> Volker (or anyone else for that matter),
>> >>>
>> >>> Just curious -- do you envision "inline" layout of objects as something
>> >> one
>> >>> would have to opt-in or as the default layout for all objects in a heap?
>> >> It
>> >>> seems like this should be the default (assuming zero to minimal overhead
>> >> for
>> >>> loading the references) as I think wanting "out of line" allocations is
>> >> more
>> >>> rare.
>> >>>
>> >>> Thanks
>> >>>
>> >>> On Mon, Feb 2, 2015 at 2:06 PM, Volker Simonis <volker.simonis at gmail.com
>> >>>
>> >>> wrote:
>> >>>>
>> >>>> Hi Brian,
>> >>>>
>> >>>> thanks a lot for your detailed answer and apologies for the late reply
>> >>>> (I was a little distracted by FOSDEM :)
>> >>>>
>> >>>> All your comments have been clear and reasonable and are much
>> >>>> appreciated. Please find my additional answers inline:
>> >>>>
>> >>>> On Thu, Jan 29, 2015 at 6:05 PM, Brian Goetz <brian.goetz at oracle.com>
>> >>>> wrote:
>> >>>>>> Question: is JEP 169 still under active development or has it been
>> >>>>>> merged into the more general "Value types for Java" proposal below?
>> >>>>>
>> >>>>>
>> >>>>> It has been merged into the more general Value Types for Java
>> >> proposal.
>> >>>>>
>> >>>>
>> >>>> Then maybe this JEP should be closed to avoid further confusion?
>> >>>>
>> >>>>>> The "Value types for Java" approach clearly seems to be the most
>> >>>>>> general but also the most complex proposal.
>> >>>>>
>> >>>>>
>> >>>>> For some meanings of "complex".  It is certainly the most intrusive
>> >> and
>> >>>>> large; new bytecodes, new type signatures.  But from a user-model
>> >>>>> perspective, value types are actually fairly simple.
>> >>>>>
>> >>>>>> It's out of scope for Java
>> >>>>>> 9 and still questionable for Java 10 and above. The "PackedObject"
>> >> and
>> >>>>>> "ObjectLayout" approaches are clearly simpler and more limited in
>> >>>>>> scope as they only concentrate on better object layout.
>> >>>>>
>> >>>>>
>> >>>>> To your list, I'd add: Project Panama, the sister project to Valhalla.
>> >>>>> Panama focuses on interop with native code and data, including layout
>> >>>>> specification.  A key goal of Packed was to be able to access off-heap
>> >>>>> native data in its native format, rather than marshalling it across
>> >> the
>> >>>>> JNI
>> >>>>> boundary.  Panama is focused on this problem as well, but aims to
>> >> treat
>> >>>>> it
>> >>>>> as a separate problem from Java object layout, resulting in what we
>> >>>>> believe
>> >>>>> to be a cleaner decomposition of the two concerns.
>> >>>>>
>> >>>>
>> >>>> Your right. I somehow missed to look at Panama more deeply because I
>> >>>> always thought it is only about FFI. John Rose nicely explains the
>> >>>> various parts of Panama in this mail
>> >>>>
>> >> http://mail.openjdk.java.net/pipermail/panama-dev/2014-October/000042.html
>> >>>> where he also mentions the intention of Panama to create new flatter
>> >>>> data layouts in the Heap and the relation of Panama to PackedObjects
>> >>>> and ObjectLayout.
>> >>>>
>> >>>>> Packed is an interesting mix of memory density (object embedding and
>> >>>>> packed
>> >>>>> arrays) and native interop.  But mixing the two goals also has costs;
>> >>>>> our
>> >>>>> approach is to separate them into orthogonal concerns, and we think
>> >> that
>> >>>>> Valhalla and Panama do just that.  So in many ways, while a larger
>> >>>>> project,
>> >>>>> the combination of Valhalla+Panama addresses the problem that Packed
>> >>>>> did, in
>> >>>>> a cleaner way.
>> >>>>>
>> >>>>>> Question: is there a chance to get a some sort of Java-only but
>> >>>>>> transparently optimizable structure package like "ObjectLayout" into
>> >>>>>> Java early (i.e. Java 9)?
>> >>>>>
>> >>>>>
>> >>>>> It would depend on a lot of things -- including the level of readiness
>> >>>>> of
>> >>>>> the design and implementation, and the overlap with anticipated future
>> >>>>> features.  We've reviewed some of the early design of ObjectLayout and
>> >>>>> provided feedback to the projects architects; currently, I think it's
>> >> in
>> >>>>> the
>> >>>>> "promising exploration" stage, but I think multiple rounds of
>> >>>>> simplification
>> >>>>> are needed before it is ready to be considered for "everybody's Java."
>> >>>>> But
>> >>>>> if the choice is to push something that's not ready into 9, or to wait
>> >>>>> longer -- there's not actually a choice to be made there.
>> >>>>>
>> >>>>> I appreciate the desire to "get something you can use now", but we
>> >> have
>> >>>>> to
>> >>>>> be prepared to support whatever we push into Java for the next 20
>> >> years,
>> >>>>> and
>> >>>>> deal with the additional constraints it generates -- which can be an
>> >>>>> enormous cost.  (Even thought the direct cost is mostly borne by
>> >> Oracle,
>> >>>>> the
>> >>>>> indirect cost is borne by everyone, in the form of slower progress on
>> >>>>> everything else.)  So I am very wary of the motivation of "well,
>> >>>>> something
>> >>>>> better is coming, but this works now, so can we push it in?"  I'd
>> >> prefer
>> >>>>> to
>> >>>>> focus on answering whether this is right thing for Java for the next
>> >> 20
>> >>>>> years.
>> >>>>>
>> >>>>>> In my eyes this wouldn't contradict with a more general solution like
>> >>>>>> the one proposed in the "Value types for Java" approach while still
>> >>>>>> offering quite significant performance improvements for quite a big
>> >>>>>> range of problems.
>> >>>>>
>> >>>>>
>> >>>>> The goals of the ObjectLayout effort has overlap with, but also
>> >> differs
>> >>>>> from, the goals of Valhalla.  And herein is the problem; neither
>> >>>>> generalizes
>> >>>>> the other, and I don't think we do the user base a great favor by
>> >>>>> pursuing
>> >>>>> two separate neither-coincident-nor-orthogonal approaches.  I suspect,
>> >>>>> though, that after a few rounds of simplification, ObjectLayout could
>> >>>>> morph
>> >>>>> into something that fit either coincidently or orthogonally with the
>> >>>>> Valhalla work -- which would be great.  But, as you know, our
>> >> resources
>> >>>>> are
>> >>>>> limited, so we (Oracle) can't really afford to invest in both.  And
>> >> such
>> >>>>> simplification takes time -- getting to that "aha" moment when you
>> >>>>> realize
>> >>>>> you can simplify something is generally an incompressible process.
>> >>>>>
>> >>>>>> Question: what would be the right place to propose something like the
>> >>>>>> "ObjectLayout" library for Java 9/10? Would that fit within the
>> >>>>>> umbrella of the Valhalla project or would it be done within its own
>> >>>>>> project / under it's own JEP?
>> >>>>>
>> >>>>>
>> >>>>> Suggesting a version number at this point would be putting the cart
>> >>>>> before
>> >>>>> the horse (you'll note that we've not even proposed a version number
>> >> for
>> >>>>> Valhalla; the closest we've gotten to that is "after 9".)
>> >>>>>
>> >>>>> OpenJDK Projects are a tool for building a community around a body of
>> >>>>> work;
>> >>>>> JEPs are a project-management tool for defining, scoping, and tracking
>> >>>>> the
>> >>>>> progress of a feature.  Given where OL is, it would be reasonable to
>> >>>>> start a
>> >>>>> Project, which would become the nexus of collaboration that could
>> >>>>> eventually
>> >>>>> produce a JEP.
>> >>>>>
>> >>>>
>> >>>> That sounds reasonable. I'll speak with the ObjectLayout people to
>> >>>> hear what they think about starting a new OpenJDK project.
>> >>>>
>> >>>> And after all you've said before, I also came to the conclusion that
>> >>>> investigating ways of new in-heap object layouts seem to be better of
>> >>>> in the Panama project. So I'll also ask there what they think about
>> >>>> it. Maybe ObjectLayout could become part of Panama or maybe we could
>> >>>> just start a new subproject of Panama with the same/similar goals.
>> >>>>
>> >>>>> Hope this helps,
>> >>>>> -Brian
>> >>>>
>> >>>> It really did!
>> >>>>
>> >>>> Thanks again,
>> >>>> Volker
>> >>>> _______________________________________________
>> >>>> mlvm-dev mailing list
>> >>>> mlvm-dev at openjdk.java.net
>> >>>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> mlvm-dev mailing list
>> >>> mlvm-dev at openjdk.java.net
>> >>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> >>>
>> >> _______________________________________________
>> >> mlvm-dev mailing list
>> >> mlvm-dev at openjdk.java.net
>> >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> >>
>> 
>> 
> 



More information about the valhalla-dev mailing list