<AWT Dev> [OpenJDK 2D-Dev] [8] Review request for 8011059 [macosx] Make JDK demos look perfect on retina displays

Jim Graham james.graham at oracle.com
Wed Nov 6 16:09:59 PST 2013

On 11/6/13 3:09 PM, Sergey Bylokhov wrote:
> Hi , Jim.
> On 07.11.2013 1:40, Jim Graham wrote:
>> On 11/6/13 5:19 AM, Alexander Scherbatiy wrote:
>>> On 11/6/2013 5:39 AM, Jim Graham wrote:
>>>> Why is getScaledInstance() being consulted here?  It seems a misuse of
>>>> that method.
>>>      We need to introduce a new API that allows developers to return
>>> scaled images for HiDPI displays.
>> Then we should introduce new API.
> Well during draw of the image we should request appropriate scale image
> from the user, who possible know better how its image can be scaled.
> What can be better, than a method which "Creates a scaled version of
> this image."?

That method "returns a scaled version of *this* image".  It specifically 
refers to recombining the pixels of the image you would normally get 
into a new image that, by default, renders at the requested size.

The method has parameters to control the pixel replication/elimination 

This is a pixel manipulation method pure and simple.  It is also not 
anywhere documented that it is used by drawImage(), in fact it is 
documented that the developer call this method and then the result is 
for the developer to use when they call drawImage().

drawImage() should *NOT* be using this method on its own, and the method 
should not be involved in choosing alternate designs.  It is for 
manually resampling an image into a new permanent image.

>>>      The good way is to allow a user to override a method like
>>> getScaledImage() which has either scaledFactor or width and height
>>> parameters and returns necessary scaled images.
>> If that is a new API with new semantics, then I agree.
>>>      But the  getScaledImage(width, height) method looks similar to the
>>> getScaledInstance(width, height, hints). It also can confuse a user
>>> which method should be overridden to return
>>>      the scaled version of an image.
>> A better name would help.  I don't think width/height are really the
>> best parameters for such a method since you want an image that can be
>> substituted and the original image has already established an aspect
>> ratio.  A single resolution would have been preferable.
> But it can be used independently, if the graphics is scaled in one
> direction. And to allow the user to decide how to scale the picture
> depending on how it will be expanded. And there is no direct
> relationship with retina and its scale factor.

Currently we only have one case of HiDPI displays (retina) and they have 
the same pixel scaling in both directions.  It is true that a more 
complete API would allow for having different resolutions in X and Y, 
though I'm not sure that will ever be a case needed in practice.

> We can move deeper and request here the actual size of the image
> including transform of the graphics (<=TRANSFORM_TRANSLATESCALE). So the
> user can apply IMAGE_SCALING hint to the transformed sg2d and all images
> will have a chance to use custom scale instead of  some scaling from the
> sg2d.

I see no reason for this.  We are trying to solve image fidelity for 
retina displays here, (and hopefully whatever Windows comes up with for 
similar purposes).  The graphics object already has plenty of 
information at its disposal to determine how to scale images.

I also don't think we want to take the transform into consideration.  We 
are either "on a hi res display" or we are not on such a display and we 
should use the variant of the image that is appropriate for that 
display, even if they are scaled up or down by 2x or more - we don't 
want an image to suddenly change its look just because they are 
animating its size (though we could offer a hint in the future that 
might indicate that they would like us to do that, I don't think it 
should be the standard or default behavior and I am not sure anyone 
would ever actually want it in the first place as it creates a visual 
discontinuity in what should be a smooth seamless process).

>> And there are 10 years of potential overrides that developers could
>> have created that could be even worse than the default implementation
>> from a performance perspective.  "Whether or not the method is
>> overridden" is a bad heuristic from the get go and it is much, much
>> more inappropriate to apply to a legacy method that has been in the
>> wild for over a decade.
> If user overrides this method and provides some specific scale algorithm
> is not what we want from the beginning?

No, that method is provided for developers to call to get "scaled 
versions" of an image.  If they've provided some custom code - of 
unknown performance capability - in their libraries and apps then they 
have probably not expected this to ever be used in any automatic sense 
by drawImage().

> "Whether or not the method is overridden" come in to play because
> default scale operation in SG2D and the results of default
> implementation of getScaledInstance is quite similar, but performance is
> extremely different.
> So it can be rephrased as "Whether or not the default scale algorithm is
> use", so in the fix we use default from the S2G2 not Image.

They came into play because you were calling that method in a place that 
you should never have been calling that method.

>> The suggestion to add a new hint to a decade old API is unwise.
>> Additionally, the hint requested "SCALE_CUSTOM" could mean anything
>> and is documented so loosely that resolution swapping is only one of
>> an infinite number of responses an application could make.  The spec
>> of that hint also violates the contract of the getScaledInstance()
>> method which states that the returned instance will be of the
>> requested size, but you are planning on returning an image that can
>> have a different size.
> It return the image which is rendered to appropriate width and height by
> default. It is not necessary to be always the same sized image. Moreover
> it can be a different size image in case of negative width/height.
> This hint will be enabled on the retina device by default, but can be
> enabled by the user as well for any graphics

Fact: getScaledInstance() *MUST ALWAYS* return an image that is the 
requested WxH.  It has no other option.  you can't override that *API 
REQUIREMENT* in the documentation of a hint.  Read the documentation for 
it: "A new Image object is returned which will render the image at the 
specified width and height by default."  In other words, if you call 
g.drawImage(thenewscaledinstance, x, y) (i.e. the version with no w,h 
parameters in the drawImage() signature) then it will draw at the size 
requested when "thenewscaledinstance" was created.

> In general an idea of the fix is: The image know better how it could be
> scaled
>   - It could use some pixel replicate scale filters
>   - it could use different files images
>   - it could redraw itself from scratch if it use vectors graphics.
> The only parameters it needed is width and height.

You are not understanding that it is a new scaled version of *THIS* 
image.  The pixels from the original image must be used.  This is about 
pixel replication algorithms, not about redesigning a representation.

> The CUSTOM hint is used for two reasons
>   - to use default scale from graphics and to skip default in image
>   - to explicitly document new usage of getScaledInstance().
> And additionally about compatibility you can compare this version of the
> fix and version of jdk7(where demos was rewritten). If I understand it
> correctly nothing should be changed in demos in the current version of
> the fix(except new x2 images.)
> http://closedjdk.us.oracle.com/jdk7u/jdk7u-dev/jdk/src/closed/rev/335cba03699d

I am not suggesting that we change the demos.  I am suggesting that we 
handle this internally for now with an option to provide a developer to 
inspect and/or intercept the process when we can create new API.

There is no reason our own internal toolkit images could not provide an 
internal API that only our toolkits have access to in JDK7/8 which is 
very similar to the way that you are using getScaledInstance() here, but 
it is using private internal interfaces that won't be confused with 
legacy public API and won't run into legacy performance issues and won't 
redesign a legacy API to do a job it was never intended or designed to 
do and which it was specifically designed and specified to not do.

In other words:

package sun.awt.image;

public interface MultiResImage {
     public Image getResolutionVariant(float resolution);

public class MacImage extends Image implements MultiResImage {
     public Image getResolutionVariant(float resolution) {
         if (resolution >= 2f && ImageAt2x != null) {
             return ImageAt2x;
         return this;

SG2D.drawImage() {
     if (img instanceof MultiResImage && dest.pixelscale != 1) {
         img = ((MultiResImage) img).getResolutionVariant(dest.pixelscale);


>>>> The @2x mechanism should be based on different API.  I guess it would
>>>> have to be internal-only for 8.0 and could be exposed to allow
>>>> developers to call it and possibly to be a provider for it in JDK9...
>>>>             ...jim
>>>> On 10/31/13 9:19 AM, Alexander Scherbatiy wrote:
>>>>>   Could you review the updated fix:
>>>>> http://cr.openjdk.java.net/~alexsch/8011059/webrev.04/
>>>>>     The reflection is used to skip the Image.getScaledInstance()
>>>>> method
>>>>> call if it is not overridden by a user.
>>>>> On 10/29/2013 11:08 PM, Sergey Bylokhov wrote:
>>>>>> Hi, Alexander.
>>>>>> The fix looks fine to me in general. But there is at least one issue.
>>>>>> I build you fix and test it:
>>>>>>  - Consuming of cpu increased by 500 times Java2Demo on images tab.
>>>>>>  - FPS is dropped from 220(jdk8)/35(jdk7u40) to 15 in guimark2. Note
>>>>>> that jdk6 has the same FPS(15) on my system.
>>>>>     The main problem is that the Image.SCALE_DEFAULT hint is used to
>>>>> retrieve a scaled image from Image.getScaledInstance() method.
>>>>>     It always uses the ReplicateScaleFilter for images which
>>>>> getScaledInstance() method has not been overridden.
>>>>>     The ReplicateScaleFilter creates a lot of arrays and consumes the
>>>>> CPU during the image parsing.
>>>>>     The better fix would be to introduce the new Image.SCALE_CUSTOM
>>>>> hint
>>>>> which could be used to get a scaled image and does not use filters by
>>>>> default.
>>>>>     But it should be a separated bug with a new CCC request.
>>>>>    Thanks,
>>>>>    Alexandr.
>>>>>> On 29.10.2013 20:45, Alexander Scherbatiy wrote:
>>>>>>>   Could you review the updated fix:
>>>>>>> http://cr.openjdk.java.net/~alexsch/8011059/webrev.03
>>>>>>> On 10/28/2013 2:33 PM, Artem Ananiev wrote:
>>>>>>>> Hi, Alexander,
>>>>>>>> a few comments:
>>>>>>>> 1. SunGraphics2D.java:3076 - should isHiDPIImage() be used here?
>>>>>>>      The isHiDPIImage() method is used to check that the
>>>>>>> drawHiDPIImage should be called like:
>>>>>>>         if (isHiDPIImage(img)) {
>>>>>>>             return drawHiDPIImage(...);
>>>>>>>         }
>>>>>>>> 2. I'm not sure that the proposed getScaledImageName()
>>>>>>>> implementation in ScalableToolkitImage works perfectly for URLs
>>>>>>>> like
>>>>>>>> this:
>>>>>>>> http://www.exampmle.com/dir/image
>>>>>>>> In this case it will try to find 2x image here:
>>>>>>>> http://www.example@2x.com/dir/image
>>>>>>>> which doesn't look correct.
>>>>>>>        Fixed. Only path part of a URL is converted to path2x.
>>>>>>>> 3. RenderingHints spec references Retina or non-Retina displays,
>>>>>>>> which should be removed.
>>>>>>>        Fixed.
>>>>>>>     - devScale is used instead of transform parsing in the
>>>>>>> drawHiDPIImage() method just to not have performance regression more
>>>>>>> than 2 times on HiDPI displays
>>>>>>>     - LWCToolkit.ScalableToolkitImage is made public for the fix
>>>>>>> 8024926 [macosx] AquaIcon HiDPI support.
>>>>>>>     Thanks,
>>>>>>>     Alexandr.
>>>>>>>> Thanks,
>>>>>>>> Artem
>>>>>>>> On 10/25/2013 5:18 PM, Alexander Scherbatiy wrote:
>>>>>>>>>    Could you review the updated fix:
>>>>>>>>> http://cr.openjdk.java.net/~alexsch/8011059/webrev.02/
>>>>>>>>>    - Scaled image width and height are transformed according to
>>>>>>>>> the
>>>>>>>>> AffineTransform type.
>>>>>>>>>    - ToolkitImage subclass is used to hold @2x image instance.
>>>>>>>>>   Thanks,
>>>>>>>>>   Alexandr.
>>>>>>>>> On 10/23/2013 7:24 PM, Alexander Scherbatiy wrote:
>>>>>>>>>>   Could you review the updated fix:
>>>>>>>>>> http://cr.openjdk.java.net/~alexsch/8011059/webrev.01/
>>>>>>>>>>   The JCK failures has been resolved:
>>>>>>>>>>     - Some tests tries to draw an image with Integer.MAX_VALUE
>>>>>>>>>> width
>>>>>>>>>> or height. Passing large values to image.getScaledImage(width,
>>>>>>>>>> height,
>>>>>>>>>> hints).
>>>>>>>>>>        leads that an Image filter is not able to create necessary
>>>>>>>>>> arrays.  The fix uses the original image if width or height are
>>>>>>>>>> equal
>>>>>>>>>> to Integer.MAX_VALUE.
>>>>>>>>>>     - Using Image.SCALE_DEFAULT hint for the
>>>>>>>>>> getScaledImage(width,
>>>>>>>>>> height, hints) method to get the high resolution image interferes
>>>>>>>>>> with
>>>>>>>>>>        JCK tests that expect that the scaled image by certain
>>>>>>>>>> algorithm is returned. This is fixed by invoking the
>>>>>>>>>> super.getScaledImage(width, height, hints)
>>>>>>>>>>        method in ToolkitImage in case if a high resolution
>>>>>>>>>> image is
>>>>>>>>>> not set.
>>>>>>>>>>     Thanks,
>>>>>>>>>>     Alexandr.
>>>>>>>>>> On 10/22/2013 1:31 PM, Alexander Scherbatiy wrote:
>>>>>>>>>>> Hello,
>>>>>>>>>>> Could you review the fix:
>>>>>>>>>>>   bug: https://bugs.openjdk.java.net/browse/JDK-8011059
>>>>>>>>>>>   webrev: http://cr.openjdk.java.net/~alexsch/8011059/webrev.00
>>>>>>>>>>>   The IMAGE_SCALING rendering hint is added to the
>>>>>>>>>>> RenderingHints
>>>>>>>>>>> class.
>>>>>>>>>>>   Enabling the image scaling rendering hint forces the
>>>>>>>>>>> SunGraphics2D
>>>>>>>>>>> to use getScaledInstance(width, height, hints) method
>>>>>>>>>>>   from Image class with SCALE_DEFAULT hint.
>>>>>>>>>>>   By default the image scaling rendering hint is enabled on
>>>>>>>>>>> HiDPI
>>>>>>>>>>> display and disabled for standard displays.
>>>>>>>>>>>   User can override the getScaledInstance(width, height, hints)
>>>>>>>>>>> method and return necessary high resolution image
>>>>>>>>>>>   according to the given image width and height.
>>>>>>>>>>>   For example:
>>>>>>>>>>>   ---------------------
>>>>>>>>>>>         final Image highResolutionImage =
>>>>>>>>>>>                 new BufferedImage(2 * WIDTH, 2 * HEIGHT,
>>>>>>>>>>> BufferedImage.TYPE_INT_RGB);
>>>>>>>>>>>         Image image = new BufferedImage(WIDTH, HEIGHT,
>>>>>>>>>>> BufferedImage.TYPE_INT_RGB) {
>>>>>>>>>>>             @Override
>>>>>>>>>>>             public Image getScaledInstance(int width, int
>>>>>>>>>>> height,
>>>>>>>>>>> int
>>>>>>>>>>> hints) {
>>>>>>>>>>>                 if ((hints & Image.SCALE_DEFAULT) != 0) {
>>>>>>>>>>>                     return (width <= WIDTH && height <= HEIGHT)
>>>>>>>>>>>                             ? this : highResolutionImage;
>>>>>>>>>>>                 }
>>>>>>>>>>>                 return super.getScaledInstance(width, height,
>>>>>>>>>>> hints);
>>>>>>>>>>>             }
>>>>>>>>>>>         };
>>>>>>>>>>>   ---------------------
>>>>>>>>>>>   The LWCToolkit and ToolkitImage classes are patched to
>>>>>>>>>>> automatically get provided image at 2x.ext images on MacOSX.
>>>>>>>>>>>   There are no significant changes in the Java2D demo to make it
>>>>>>>>>>> look
>>>>>>>>>>> perfect on Retina displays.
>>>>>>>>>>>   It needs only to put necessary images with the @2x postfix and
>>>>>>>>>>> they
>>>>>>>>>>> will be automatically drawn.
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Alexandr.

More information about the awt-dev mailing list