JavaFX Media issues

John Hendrikx hjohn at
Fri Aug 9 03:23:15 PDT 2013

On 9/08/2013 03:48, Scott Palmer wrote:
> I have heard rumors of people being able to play HD video via Canvas. 
>  I have tried everything and can't come close. (Yes, I have been 
> careful about the pixel format.)  I mean, it looks like it is working 
> for a few seconds, but then as the memory fills with the Canvas 
> backlog it can lead to the GC using a lot more CPU, thus reducing the 
> ability for Canvas to process its command queue even further, well it 
> just collapses in on itself  and dies.
> Is your app able to do *anything* else while the video is playing? 
>  The slightest delay to the rendering and that Canvas buffering bug 
> kills the app.  Not that it would matter if it could keep up, because 
> the off-screen thing is also a deal breaker.
My app is basically just a video watching system -- while video is 
playing (in the bottom most layer of a StackPane), it shows overlays 
with time / position, possibly a menu (where you can download a matching 
subtitle for the video and adjust settings like playback speed and 
brightness).  These overlays are smoothly faded in / faded out just with 
the opacity property of another Pane that is at a higher level in the 
main StackPane.

None of this is really CPU intensive, nor is there a huge SceneGraph to 
deal with (I'm guessing its smaller than 100 nodes while video playback 
is running).  See picture here for an idea of what's going on while 
playback is occuring:

The Stage used is a non-transparent one -- I mention this because a 
transparent change performs a lot worse than a non-transparent one.  
Before I used the Canvas solution, I'd actually just playback video in a 
java.awt.Frame with a transparent JavaFX Stage on top.   I'd have 2 
windows, with the transparent JavaFX Stage on top that would show 
overlays for the video, while the other window would show the video 
using java.awt.Canvas, accessed directly by VLC.

I do run into this issue every now and then though:

It really needs to be fixed or Canvas is simply not safely usable in its 
current form (and IMHO never has been since its release then).

> Of course 25fps is well below the 60fps required for full-speed video. 
>  I suspect it is the frame rate more than the frame size that matters 
> here.  Plain old, standard definition, interlaced, 60 fields per 
> second will probably kill most apps with the current Canvas 
> implementation.
I don't think I have aany 60fps video, if you can point me to something 
I can download that VLC can handle that is of the size you're using I 
could test with the setup I have here.  With 1920x1080 HD Video, the CPU 
uses 8% according to Task Manager (low-end quad core xeon, about 1 year 
old).  I'm running a standard Java VM (b99), no other tweaking, with 
default memory settings, 256 MB heap, Task Manager claims a working set 
of  +/- 600 MB, but some native memory might be involved -- when 
playback stops, working set drops to 340 MB, so there's definitely a lot 
going on.

It's solid though once playback starts (usually it only locks up at the 
start of playback, if it locks up) and can run for hours.  No frame 
stuttering (I'd notice this immediately on a horizontal pan of one of 
the test videos I use).  Even with a lot of other things going on on the 
same machine (although not by my Java process) playback stays solid -- I 
often have this machine running at 50-75% CPU for extended periods while 
enjoying a Video on the 3rd screen.

I'd agree though that more than doubling the frame rate is gonna be a 
huge impact.  I'm not certain if interlaced is going to cause any 
additional problems, I'm assuming that's handled by VLC and that I'd 
still be copying a full buffer for each frame (or do you mean 60 FPS 
interlaced = 120 FPS?).
> We need something better.  I proposed having at least a JNI method so 
> we could get native window handles from Stages but didn't get any 
> traction either.  Security was brought up as a concern, which I 
> totally do NOT understand as the use of JNI means you are out of the 
> Java sandbox already.  If we had native window handles we might be 
> able get our own window for rendering to at least interact nicely with 
> the Stage.  (We already did this successfully in Swing via JAWT and a 
> heavyweight component that we paint to from native code)
Getting a WindowHandle for a Stage would be great -- however, I think I 
actually hacked this in at one point, and then tried playing back video 
on a JavaFX Stage -- the video however always ended up in the 
foreground, obliterating any JavaFX rendering.  That would need to be 
solved as well if it is still an issue.

I've actually tried almost every video player solution that I could find 
(all on Windows), including DirectShow (using DSJ), GStreamer-java, 
MPlayer (in seperate window), VLCJ, MediaPlayer Classic Home Cinema and 
Xuggler.  All of them except VLCJ had severe issues, ranging from simply 
being immature and crashing (GStreamer-java, MPlayer) to having 
insufficient accessible controls (DSJ, MediaPlayer Classic) to only 
being able to play stereo output, no 5.1 etc (Xuggler) or being 
complicated to use (DSJ mainly (DirectShow sucks), Xuggler somewhat).

Best regards,

More information about the openjfx-dev mailing list