Integrating native Open GL code in JavaFX and alternatives

Sven Reimers sven.reimers at
Mon Jan 28 07:48:32 PST 2013

The link

does not work for "outsiders" - I think you are hinting at


On Mon, Jan 28, 2013 at 4:16 PM, Artem Ananiev <artem.ananiev at>wrote:

> On 1/26/2013 12:04 AM, Scott Palmer wrote:
>> On 2013-01-25, at 2:16 PM, Artem Ananiev <artem.ananiev at>
>> wrote:
>>  A few comments from my side:
>>> 1. If a native Window/Scene handle is provided, it will be enough
>>> to embed whatever you need. It will even be possible to embed D3D
>>> content into JavaFX running with OpenGL pipeline :)
>> There will be issues in JavaFX that we can't work around as easily as
>> we code in Swing. For example clipping.  How can my video preview
>> overlay participate in a Scene such that I can put it inside a
>> ScrollPane or SplitPane?  With Swing I could manage this by placing a
>> heavyweight component in the UI.  Rectangular clipping was easy to
>> deal with by wrapping in another heavyweight component - the OS
>> windowing system took care of it.  Later Swing got smarter about
>> mixing heavy/light components.  JavaFX doesn't have something
>> comparable.
> To use heavyweight component in a scroll/split pane, we need heavyweight
> scroll/split pane :) In other words, such scenarios don't work well in
> Swing too.
> Do you mean limited heavyweight/lightweight component mixing, introduced
> to 6u10 and the forward-ported to 7? Note the word "limited" here: the
> mixing is really limited, it doesn't cover non-rectangular shapes,
> transparent components, and many more. It actually works fine for simplest
> scenarios only. The code we put to AWT to support the limited mixing is not
> something I would like to see in JavaFX.
>  The question is what is the native handle? On Windows, HWND is the
>>> answer. On Mac: NSView or CALayer?
>> CALayer seems to be the way things are going. NSView was the Java 6
>> native layer, now in Java 7 it is CALayer.  But this is native code…
>> let there be a way to get either from a platform specific structure.
> Believe it or not, we will support both CALayer-based and NSView-based
> embedded windows in JDK8 (and probably next JDK7 updates). Check the
> following JDK bug for details:
> Classes are not public (java.*, javax.*) APIs, though, but are supposed to
> be backwards compatible. If we implement this functionality in JavaFX,
> we'll likely support both CALayers and NSViews.
>  In the latter case we'll also need mechanism to flush embedded
>>> layer content into the parent (window) layer, and the thing will
>>> get complicated.
>>> 2. Another challenge with heavyweight embedding is integration into
>>> FX scene. Focus traversal, transforms, effects, modality, etc.
>> Can be depending on the use case.  For a video preview panel, I
>> wouldn't' need most of that.  #3 would be best.
>>  3. Alternatively, we can try to implement semi-heavyweight
>>> embedding. What we need is to make Prism be able to render
>>> externally managed textures (GL, D3D). Input events will be handled
>>> as for any other FX nodes, transforms/effects will also be of no
>>> problem. What's questionable here is a) performance b) sharing GL
>>> context or D3D device between FX and application code. I'm not a
>>> graphics expert, but suspect there may be problems in this area.
>> I agree.  But I think it is worth getting a graphics expert (I'm not
>> a OpenGL/D3D person either) to think about it over the weekend :-)
>> Cheers,
>> Scott
> Thanks,
> Artem
>  Thanks,
>>> Artem
>>> On 1/25/2013 6:28 PM, Scott Palmer wrote:
>>>> +1
>>>> I see two basic needs:
>>>> 1: A JAWT equivalent so we can at least get a handle to the Stage
>>>> window.
>>>> 2: Some means of embedding our own rendering surface.  Likely via a
>>>> "texture" described in the native format for the graphics pipeline
>>>> used (OpenGL or Direct3D)
>>>> In our Swing product (on Windows) we grab the native window handle of
>>>> a heavyweight 'placholder' via JAWT and blit a bitmap directly to it
>>>> - hardware assisted with a colorspace conversion, if I recall
>>>> correctly.  JavaFX offers no equivalent.
>>>> I know that JavaFX will likely never allow the concept of embedding a
>>>> heavyweight native window into a Scene.  I don't think that would be
>>>> the right approach for it anyway.  But we can't even make a separate
>>>> preview window natively and give it a proper parent window with the
>>>> current APIs.
>>>> Scott
>>>> On 2013-01-25, at 4:14 AM, Robert Krüger <krueger at>
>>>> wrote:
>>>>  Hi,
>>>>> just to avoid, this gets buried in the other long thread where it
>>>>> originated from (Migrating commercial applications from Swing to JFX),
>>>>> I'm posting this again as a separate thread. I hope this is OK.
>>>>> We are evaluating migrating our Swing Application which includes a
>>>>> video player built using native code for decoding, rendering and
>>>>> effects to JavaFX. Performance requirements are very ambitious (like
>>>>> being close to the playback performance of the best native player, in
>>>>> our case Quicktime for most formats we care about, for high definition
>>>>> video).
>>>>> Currently there is no way to do the same thing in JavaFX, which the
>>>>> JOGL folks have already run into themselves (see
>>>>> ).
>>>>> We would either need support for getting a native Open GL context to
>>>>> draw to a JFX component (panel, canvas whatever), so we can use native
>>>>> code to do all sorts of stuff that happens in our video player or we
>>>>> would need support for the following operations in JFX:
>>>>> - shaders for converting pixel formats (e.g. YUV422->RGB, YUV420->RGB
>>>>> ....)
>>>>> - shaders and frame buffer objects (render to texture) for applying
>>>>> layered effects to video frames
>>>>> - pixel buffer objects for transferring video images to textures (and
>>>>> back)
>>>>> - interoperability with opencl and cuda (processing of video frames
>>>>> with opencl/cuda and rendering with opengl)
>>>>> - reuse of opengl code across platforms (including android and iOS)
>>>>> We achieve that currently by having cross-platform C code that is
>>>>> connected to each platform via a rather thin layer for communication
>>>>> (e.g. to be able implement callbacks for OpenGL rendering parameters
>>>>> controlled by a Java-UI). My guess is that it is more likely that you
>>>>> will be able to expose the native context than providing all the
>>>>> listed things. It is also a maybe a philosophical question whether you
>>>>> want that but IMHO you close the door on an interesting range of
>>>>> applications if you don't provide anything like that in some way
>>>>> because you cannot implement a serious media application of a certain
>>>>> kind without it and why reinvent the wheel? IMHO Java technologies in
>>>>> the multi-media area (e.g. JavaSound, JMF) have in many cases suffered
>>>>> from being good ideas but not making it to a production-quality state
>>>>> because doing all that would have meant reinventing the wheel, which
>>>>> in complex areas like the one described here is simply not feasible so
>>>>> they IMHO remained being mostly of educational value for academic
>>>>> environments. Acknowledging that in some areas it does make sense to
>>>>> have some native code doesn't diminish the value of Java as a
>>>>> platform. Having a lot of portable code in an application is still a
>>>>> lot better than having one application per platform. Quite the
>>>>> contrary, to me it would make Java/JavaFX more grown-up in the desktop
>>>>> area if that was supported some more.
>>>>> Sorry for being a bit philosophical in the end ;-).
>>>>> Thanks a lot for your help. I'm really hoping we can come to the
>>>>> conclusion that migrating to JavaFX is the right way to go for us.
>>>>> Robert

Sven Reimers

* Senior Expert Software Architect
* NetBeans Dream Team Member:
* NetBeans Governance Board Member:
* Community Leader  NetBeans:
                              Desktop Java:
* Duke's Choice Award Winner 2009
* Blog:

* LinkedIn:

Join the NetBeans Groups:
* LinkedIn:
* Oracle:

More information about the openjfx-dev mailing list