value and {@code value}. I more like the
latter, it's compact and better readable
- you have mixed U+1234 and \u1234 style. Why?
- often you use '\' for '\', but not ever (e.g. '\t'). I think we
can use always '\'. There should not be so much developers in the world
who can't decode ISO-8859-1 or UTF-xx.
(aren't there many places in the JDK's javadoc where others have used
ISO-8859-1 characters ?)
- there are many "Character#xYz" or even "java.lang.Character#xYz" or "
"java.lang.Abc#xYz" " links left. They could be truncated for better
readability and smaller source size
- I would like to see backwards-referring like:
public static final int MIN_CODE_POINT = MIN_VALUE;
public static final int MIN_SUPPLEMENTARY_CODE_POINT = MAX_VALUE + 1;
-Ulf
Am 01.09.2009 00:11, martinrb at google.com schrieb:
> Changeset: db5d6b4cbc11
> Author: martin
> Date: 2009-08-31 15:00 -0700
> URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/db5d6b4cbc11
>
> 6860431: Character.isSurrogate(char ch)
> Summary: Add new method Character.isSurrogate(char ch)
> Reviewed-by: sherman, darcy, okutsu
>
>
From maurizio.cimadamore at sun.com Tue Sep 1 07:01:43 2009
From: maurizio.cimadamore at sun.com (maurizio.cimadamore at sun.com)
Date: Tue, 01 Sep 2009 14:01:43 +0000
Subject: hg: jdk7/tl/langtools: 6650759: Inference of formal type parameter
(unused in formal parameters) is not performed
Message-ID: <20090901140148.62F2C12978@hg.openjdk.java.net>
Changeset: dda7e13f09fb
Author: mcimadamore
Date: 2009-09-01 14:53 +0100
URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/dda7e13f09fb
6650759: Inference of formal type parameter (unused in formal parameters) is not performed
Summary: propagate inference constraints from 15.12.2.7 to 15.12.2.8
Reviewed-by: jjg
! src/share/classes/com/sun/tools/javac/code/Type.java
! src/share/classes/com/sun/tools/javac/comp/Infer.java
! test/tools/javac/generics/inference/6302954/T6476073.java
! test/tools/javac/generics/inference/6638712/T6638712b.out
! test/tools/javac/generics/inference/6638712/T6638712e.out
+ test/tools/javac/generics/inference/6650759/T6650759a.java
+ test/tools/javac/generics/inference/6650759/T6650759b.java
+ test/tools/javac/generics/inference/6650759/T6650759c.java
+ test/tools/javac/generics/inference/6650759/T6650759d.java
+ test/tools/javac/generics/inference/6650759/T6650759e.java
+ test/tools/javac/generics/inference/6650759/T6650759f.java
+ test/tools/javac/generics/inference/6650759/T6650759g.java
+ test/tools/javac/generics/inference/6650759/T6650759h.java
+ test/tools/javac/generics/inference/6650759/T6650759i.java
+ test/tools/javac/generics/inference/6650759/T6650759j.java
+ test/tools/javac/generics/inference/6650759/T6650759k.java
+ test/tools/javac/generics/inference/6650759/T6650759l.java
+ test/tools/javac/generics/inference/6650759/T6650759m.java
+ test/tools/javac/generics/inference/6650759/T6650759m.out
From jonathan.gibbons at sun.com Tue Sep 1 11:43:17 2009
From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com)
Date: Tue, 01 Sep 2009 18:43:17 +0000
Subject: hg: jdk7/tl/langtools: 6877763: update langtools/test/Makefile for
JPRT
Message-ID: <20090901184328.8E6481298A@hg.openjdk.java.net>
Changeset: 40a1327a5283
Author: jjg
Date: 2009-09-01 11:35 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/40a1327a5283
6877763: update langtools/test/Makefile for JPRT
Reviewed-by: ohair
! test/Makefile
From Ulf.Zibis at gmx.de Tue Sep 1 18:55:16 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Wed, 02 Sep 2009 03:55:16 +0200
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <4A9CDB81.1050500@gmx.de>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<4A9CDB81.1050500@gmx.de>
Message-ID: <4A9DD084.6010406@gmx.de>
I have separated the "cosmetics":
https://bugs.openjdk.java.net/attachment.cgi?id=140&action=diff
https://bugs.openjdk.java.net/attachment.cgi?id=141&action=diff
-Ulf
Am 01.09.2009 10:29, Ulf Zibis schrieb:
> Martin,
>
> I like how you have enhanced javadoc for Character class.
>
> Some comments:
> - You more like uppercased hex literals, so we are 2. :-)
> - you have mixed value and {@code value}. I more like the
> latter, it's compact and better readable
> - you have mixed U+1234 and \u1234 style. Why?
> - often you use '\' for '\', but not ever (e.g. '\t'). I think we
> can use always '\'. There should not be so much developers in the
> world who can't decode ISO-8859-1 or UTF-xx.
> (aren't there many places in the JDK's javadoc where others have used
> ISO-8859-1 characters ?)
> - there are many "Character#xYz" or even "java.lang.Character#xYz" or
> " "java.lang.Abc#xYz" " links left. They could be truncated for better
> readability and smaller source size
> - I would like to see backwards-referring like:
> public static final int MIN_CODE_POINT = MIN_VALUE;
> public static final int MIN_SUPPLEMENTARY_CODE_POINT = MAX_VALUE + 1;
>
> -Ulf
>
>
> Am 01.09.2009 00:11, martinrb at google.com schrieb:
>> Changeset: db5d6b4cbc11
>> Author: martin
>> Date: 2009-08-31 15:00 -0700
>> URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/db5d6b4cbc11
>>
>> 6860431: Character.isSurrogate(char ch)
>> Summary: Add new method Character.isSurrogate(char ch)
>> Reviewed-by: sherman, darcy, okutsu
>>
>>
>
>
From martinrb at google.com Tue Sep 1 20:21:12 2009
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 1 Sep 2009 20:21:12 -0700
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <4A9CDB81.1050500@gmx.de>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<4A9CDB81.1050500@gmx.de>
Message-ID: <1ccfd1c10909012021g78d4fa3cx5f6ab0792c3ba688@mail.gmail.com>
On Tue, Sep 1, 2009 at 01:29, Ulf Zibis value and {@code value}. I more like the
> latter, it's compact and better readable
{@code is now the preferred way. I tried to modify the methods I changed,
but didn't try to change the whole file.
A brave person such as yourself could try to
become "code janitor" for the whole jdk.
- you have mixed U+1234 and \u1234 style. Why?
They are different things. U+1234 describes a Unicode character or
codepoint,
while '\u1234' is a char (code unit, not code point).
See Unicode glossary.
> - often you use '\' for '\', but not ever (e.g. '\t'). I think we can
> use always '\'. There should not be so much developers in the world who
> can't decode ISO-8859-1 or UTF-xx.
We try hard to keep source code ASCII. Sorry, the world is adopting UTF-8,
but the transition is rather slow. Maybe in 10 years we can go UTF-8
everywhere.
(aren't there many places in the JDK's javadoc where others have used
> ISO-8859-1 characters ?)
> - there are many "Character#xYz" or even "java.lang.Character#xYz" or "
> "java.lang.Abc#xYz" " links left. They could be truncated for better
> readability and smaller source size
I agree.
>
> - I would like to see backwards-referring like:
> public static final int MIN_CODE_POINT = MIN_VALUE;
> public static final int MIN_SUPPLEMENTARY_CODE_POINT = MAX_VALUE + 1;
Those would work, but would add to the confusion
between code points and UTF-16 code units.
Notice how "MAX_VALUE + 1" looks like an oxymoron.
Martin
>
>
> -Ulf
>
>
> Am 01.09.2009 00:11, martinrb at google.com schrieb:
>
>> Changeset: db5d6b4cbc11
>> Author: martin
>> Date: 2009-08-31 15:00 -0700
>> URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/db5d6b4cbc11
>>
>> 6860431: Character.isSurrogate(char ch)
>> Summary: Add new method Character.isSurrogate(char ch)
>> Reviewed-by: sherman, darcy, okutsu
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090901/f1d83648/attachment.html
From uncle.alice at gmail.com Tue Sep 1 22:39:13 2009
From: uncle.alice at gmail.com (Alan Moore)
Date: Tue, 1 Sep 2009 22:39:13 -0700
Subject: Regex named-group and backreference syntax
Message-ID: '\uD800'.
*
* @since 1.5
*/
public static final char MIN_SURROGATE = MIN_HIGH_SURROGATE;
> A brave person such as yourself could try to
> become "code janitor" for the whole jdk.
In this case it should be simple to replace ... against
{@code ...} on the whole JDK. My problem is, that I don't have the
CPU-power to build the JDK, and check the whole javadoc if it would have
broken.
>
>
> - you have mixed U+1234 and \u1234 style. Why?
>
>
> They are different things. U+1234 describes a Unicode character or
> codepoint,
> while '\u1234' is a char (code unit, not code point).
> See Unicode glossary.
Yes, after a closer look I can see the point, so I corrected their usage
where I thought, it was wrong.
But what's about using {@code U+10000}, found for
MIN_SUPPLEMENTARY_CODE_POINT javadoc ?
"U+10000" is not valid java code, but I must admit, that it looks better
than "0x010000"
Maybe we must use U+10000 here.
>
>
> - often you use '\' for '\', but not ever (e.g. '\t'). I think
> we can use always '\'. There should not be so much developers in
> the world who can't decode ISO-8859-1 or UTF-xx.
>
>
> We try hard to keep source code ASCII. Sorry, the world is adopting
> UTF-8,
> but the transition is rather slow. Maybe in 10 years we can go UTF-8
> everywhere.
I have been fallen into a trap: '\' *is* ASCII, it's '\u005C'. so is
there any reason remaining on '\' ???
>
>
>
>
> - I would like to see backwards-referring like:
> public static final int MIN_CODE_POINT = MIN_VALUE;
> public static final int MIN_SUPPLEMENTARY_CODE_POINT = MAX_VALUE
> + 1;
>
>
> Those would work, but would add to the confusion
> between code points and UTF-16 code units.
> Notice how "MAX_VALUE + 1" looks like an oxymoron.
>
;-)
But I don't have any problem as I don't have using Byte.MAX_VALUE + 1.
The real source of the confusion is elsewhere, i.e. imagine we would
have class Integer managing 16 + 32 bit values.
Maybe it would become more clear adding MAX_SUPPLEMENTARY_CODE_POINT for
*consistency* and having following order:
(Note that I added " of type {@code int}", similar to description of
MIN_VALUE.)
/**
* The minimum value of a
*
* Unicode code point, constant {@code U+0000}
* of type {@code int}.
*
* @since 1.5
*/
public static final int MIN_CODE_POINT = MIN_VALUE;
/**
* The minimum value of a
*
* Unicode supplementary code point, constant {@code U+10000}
* of type {@code int}.
*
* @since 1.5
*/
public static final int MIN_SUPPLEMENTARY_CODE_POINT = MAX_VALUE + 1;
/**
* The maximum value of a
*
* Unicode code point, constant {@code U+10FFFF}
* of type {@code int}.
*
* @since 1.5
*/
public static final int MAX_CODE_POINT = 0X10FFFF;
/**
* The maximum value of a
*
* Unicode supplementary code point, constant {@code U+10FFFF}
* of type {@code int}.
*
* @since 1.7
*/
public static final int MAX_SUPPLEMENTARY_CODE_POINT = MAX_CODE_POINT;
-Ulf
> Martin
>
>
>
>
> -Ulf
>
>
> Am 01.09.2009 00:11, martinrb at google.com
> '\uD800'.
> *
> * @since 1.5
> */
> public static final char MIN_SURROGATE = MIN_HIGH_SURROGATE;
>
>
I would have used {@code here if I could figure out
how to make it work
("\" shows up literally in the generated output).
>
> A brave person such as yourself could try to
> become "code janitor" for the whole jdk.
>
>
> In this case it should be simple to replace ... against {@code
> ...} on the whole JDK. My problem is, that I don't have the CPU-power to
> build the JDK, and check the whole javadoc if it would have broken.
>
Building the javadoc requires a lot of memory -
a javadoc bug - someone could try to fix that...
- you have mixed U+1234 and \u1234 style. Why?
They are different things. U+1234 describes a Unicode character or
codepoint,
while '\u1234' is a char (code unit, not code point).
See Unicode glossary.
Yes, after a closer look I can see the point, so I corrected their usage
> where I thought, it was wrong.
> But what's about using {@code U+10000}, found for
> MIN_SUPPLEMENTARY_CODE_POINT javadoc ?
> "U+10000" is not valid java code, but I must admit, that it looks better
> than "0x010000"
> Maybe we must use U+10000 here.
>
>
>
>> - often you use '\' for '\', but not ever (e.g. '\t'). I think we can
>> use always '\'. There should not be so much developers in the world who
>> can't decode ISO-8859-1 or UTF-xx.
>
>
> We try hard to keep source code ASCII. Sorry, the world is adopting UTF-8,
> but the transition is rather slow. Maybe in 10 years we can go UTF-8
> everywhere.
>
>
> I have been fallen into a trap: '\' *is* ASCII, it's '\u005C'. so is
> there any reason remaining on '\' ???
>
\uXXXX has special meaning in java source files, even in comments.
- I would like to see backwards-referring like: public static final int
MIN_CODE_POINT = MIN_VALUE;
> public static final int MIN_SUPPLEMENTARY_CODE_POINT = MAX_VALUE + 1;
Those would work, but would add to the confusion
between code points and UTF-16 code units.
Notice how "MAX_VALUE + 1" looks like an oxymoron.
;-)
> But I don't have any problem as I don't have using Byte.MAX_VALUE + 1.
> The real source of the confusion is elsewhere, i.e. imagine we would have
> class Integer managing 16 + 32 bit values.
>
> Maybe it would become more clear adding MAX_SUPPLEMENTARY_CODE_POINT for
> *consistency* and having following order:
>
MAX_SUPPLEMENTARY_CODE_POINT would not be a bad thing to have,
but not compelling enough for the effort involved with any change to java
se.
People who use these constants anyways have to understand the
model - that code points are divided into bmp and supplementary.
Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090902/8dd2100d/attachment.html
From david.lloyd at redhat.com Wed Sep 2 09:40:09 2009
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 02 Sep 2009 11:40:09 -0500
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net> <4A9CDB81.1050500@gmx.de> <1ccfd1c10909012021g78d4fa3cx5f6ab0792c3ba688@mail.gmail.com> <4A9E27BF.8000905@gmx.de>
<1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com>
Message-ID: <4A9E9FE9.7060107@redhat.com>
On 09/02/2009 11:27 AM, Martin Buchholz wrote:
> On Wed, Sep 2, 2009 at 01:07, Ulf Zibis '\uD800'.
> *
> * @since 1.5
> */
> public static final char MIN_SURROGATE = MIN_HIGH_SURROGATE;
>
> I would have used {@code here if I could figure out
> how to make it work
> ("\" shows up literally in the generated output).
Why not just do {@code \uD800}? I'm like 60% sure that would work just
fine. :-)
- DML
From mr at sun.com Wed Sep 2 09:43:40 2009
From: mr at sun.com (Mark Reinhold)
Date: Wed, 02 Sep 2009 09:43:40 -0700
Subject: Regex named-group and backreference syntax
In-Reply-To: uncle.alice@gmail.com; Wed, 02 Sep 2009 01:58:46 PDT;
'\uD800'.
>> *
>> * @since 1.5
>> */
>> public static final char MIN_SURROGATE = MIN_HIGH_SURROGATE;
>>
>> I would have used {@code here if I could figure out
>> how to make it work
>> ("\" shows up literally in the generated output).
>>
>
> Why not just do {@code \uD800}? I'm like 60% sure that would work just
> fine. :-)
>
I'm pretty sure it would fail. Prove me wrong!
Searching the JDK sources for regex
^ *\*.*\\u[0-9a-fA-F]{4}
is a good way to find javadoc bugs, e.g.
http://java.sun.com/javase/6/docs/api/java/lang/String.html#toLowerCase()
Martin
> - DML
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090902/2d3399cb/attachment.html
From david.lloyd at redhat.com Wed Sep 2 10:11:53 2009
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Wed, 02 Sep 2009 12:11:53 -0500
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<4A9CDB81.1050500@gmx.de>
<1ccfd1c10909012021g78d4fa3cx5f6ab0792c3ba688@mail.gmail.com>
<4A9E27BF.8000905@gmx.de>
<1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com>
<4A9E9FE9.7060107@redhat.com>
<1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com>
Message-ID: <4A9EA759.3050804@redhat.com>
On 09/02/2009 12:03 PM, Martin Buchholz wrote:
> On Wed, Sep 2, 2009 at 09:40, David M. Lloyd '\uD800'.
> *
> * @since 1.5
> */
> public static final char MIN_SURROGATE = MIN_HIGH_SURROGATE;
>
>
> I would have used {@code here if I could figure out
> how to make it work
> ("\" shows up literally in the generated output).
Could this be seen as a bug? IMHO the {@code construct should be a 100%
replaceable for . Any Bug Id ?
>
>
>
>> A brave person such as yourself could try to
>> become "code janitor" for the whole jdk.
>
> In this case it should be simple to replace ...
> against {@code ...} on the whole JDK. My problem is, that I don't
> have the CPU-power to build the JDK, and check the whole javadoc
> if it would have broken.
>
>
> Building the javadoc requires a lot of memory -
> a javadoc bug - someone could try to fix that...
... but as we see, simply replacing ... against {@code ...}
in the whole JDK would not work. :-(
>
> Yes, after a closer look I can see the point, so I corrected their
> usage where I thought, it was wrong.
> But what's about using {@code U+10000}, found for
> MIN_SUPPLEMENTARY_CODE_POINT javadoc ?
> "U+10000" is not valid java code, but I must admit, that it looks
> better than "0x010000"
> Maybe we must use U+10000 here.
>
No answer. Does that mean, {@code U+10000} is valid?
>
>> - I would like to see backwards-referring like: public static final
>> int MIN_CODE_POINT = MIN_VALUE;
>>
>> public static final int MIN_SUPPLEMENTARY_CODE_POINT =
>> MAX_VALUE + 1;
>>
>>
>> Those would work, but would add to the confusion
>> between code points and UTF-16 code units.
>> Notice how "MAX_VALUE + 1" looks like an oxymoron.
>>
> ;-)
> But I don't have any problem as I don't have using Byte.MAX_VALUE + 1.
> The real source of the confusion is elsewhere, i.e. imagine we
> would have class Integer managing 16 + 32 bit values.
>
> Maybe it would become more clear adding
> MAX_SUPPLEMENTARY_CODE_POINT for *consistency* and having
> following order:
>
>
> MAX_SUPPLEMENTARY_CODE_POINT would not be a bad thing to have,
> but not compelling enough for the effort involved with any change to
> java se.
:-(
... but changing the order would decrease the confusion ?
-Ulf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090902/0eeedd83/attachment.html
From Ulf.Zibis at gmx.de Wed Sep 2 12:46:52 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Wed, 02 Sep 2009 21:46:52 +0200
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <4A9EA759.3050804@redhat.com>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<4A9CDB81.1050500@gmx.de>
<1ccfd1c10909012021g78d4fa3cx5f6ab0792c3ba688@mail.gmail.com>
<4A9E27BF.8000905@gmx.de>
<1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com>
<4A9E9FE9.7060107@redhat.com>
<1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com>
<4A9EA759.3050804@redhat.com>
Message-ID: <4A9ECBAC.7060303@gmx.de>
Am 02.09.2009 19:11, David M. Lloyd schrieb:
> On 09/02/2009 12:03 PM, Martin Buchholz wrote:
>> On Wed, Sep 2, 2009 at 09:40, David M. Lloyd > > wrote:
>> Why not just do {@code \uD800}? I'm like 60% sure that would work
>> just fine. :-)
>>
>>
>> I'm pretty sure it would fail. Prove me wrong!
>> Searching the JDK sources for regex
>> ^ *\*.*\\u[0-9a-fA-F]{4}
>> is a good way to find javadoc bugs, e.g.
>> http://java.sun.com/javase/6/docs/api/java/lang/String.html#toLowerCase()
>>
>
> Ah, you're right. It worked in my previewer but not in the actual
> javadoc. It's pretty bad that that sequence has special meaning but
> you can't escape a \ with another \. I guess in the worst case you
> could always do \u005CD800 or something like that.
>
Looks little better, but not much. Did somebody tried it (Martin)?
If it works in a previewer, is there any chance to change the javadoc
spec, staying backwards compatible?
-Ulf
From mr at sun.com Wed Sep 2 12:47:42 2009
From: mr at sun.com (Mark Reinhold)
Date: Wed, 02 Sep 2009 12:47:42 -0700
Subject: Regex named-group and backreference syntax
In-Reply-To: gnu_andrew@member.fsf.org; Wed, 02 Sep 2009 18:55:49 BST;
<17c6771e0909021055r78d75d60i3dc80c61d64b4f00@mail.gmail.com>
Message-ID: <20090902194742.C8A3E1B37@eggemoggin.niobe.net>
> Date: Wed, 02 Sep 2009 18:55:49 +0100
> From: Andrew John Hughes
> 2009/9/2 Mark Reinhold :
>> I'd say ${name} is the best option here given that it will be familiar,
>> at least, to Perl and .NET developers.
>
> It's also used for substitution in shell and Makefiles :)
That too!
- Mark
From Ulf.Zibis at gmx.de Wed Sep 2 13:12:10 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Wed, 02 Sep 2009 22:12:10 +0200
Subject: hg: jdk7/tl/jdk: 16 new changesets
In-Reply-To: <20090830070022.D6A6B12820@hg.openjdk.java.net>
References: <20090830070022.D6A6B12820@hg.openjdk.java.net>
Message-ID: <4A9ED19A.7050608@gmx.de>
Am 30.08.2009 08:54, tim.bell at sun.com schrieb:
> Changeset: 4c6a5ea563ba
> Author: peytoia
> Date: 2009-07-30 14:45 +0900
> URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/4c6a5ea563ba
>
> 6866243: Javadoc for java.lang.Character still refers to Unicode 4 instead of 5
> Reviewed-by: okutsu
>
> ! src/share/classes/java/lang/Character.java
>
There is still a reference to Unicode 4. See comment in method
toUpperCaseCharArray(int codePoint).
-Ulf
From martinrb at google.com Wed Sep 2 13:29:16 2009
From: martinrb at google.com (Martin Buchholz)
Date: Wed, 2 Sep 2009 13:29:16 -0700
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <4A9ECBAC.7060303@gmx.de>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<4A9CDB81.1050500@gmx.de>
<1ccfd1c10909012021g78d4fa3cx5f6ab0792c3ba688@mail.gmail.com>
<4A9E27BF.8000905@gmx.de>
<1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com>
<4A9E9FE9.7060107@redhat.com>
<1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com>
<4A9EA759.3050804@redhat.com> <4A9ECBAC.7060303@gmx.de>
Message-ID: <1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
On Wed, Sep 2, 2009 at 12:46, Ulf Zibis wrote:
> Am 02.09.2009 19:11, David M. Lloyd schrieb:
>>
>> On 09/02/2009 12:03 PM, Martin Buchholz wrote:
>>>
>>> On Wed, Sep 2, 2009 at 09:40, David M. Lloyd >> > wrote:
>>> ? ?Why not just do {@code \uD800}? ?I'm like 60% sure that would work
>>> ? ?just fine. :-)
>>>
>>>
>>> I'm pretty sure it would fail. ? Prove me wrong!
>>> Searching the JDK sources for regex
>>> ^ *\*.*\\u[0-9a-fA-F]{4}
>>> is a good way to find javadoc bugs, e.g.
>>> http://java.sun.com/javase/6/docs/api/java/lang/String.html#toLowerCase()
>>
>> Ah, you're right. ?It worked in my previewer but not in the actual
>> javadoc. ?It's pretty bad that that sequence has special meaning but you
>> can't escape a \ with another \. ?I guess in the worst case you could always
>> do \u005CD800 or something like that.
>>
>
> Looks little better, but not much. Did somebody tried it (Martin)?
Well.... learn something new every day.
Let's turn this into a fix.
It's yet another "turkish i" bug.
http://cr.openjdk.java.net/~martin/webrevs/openjdk7/javadoc-unicode-escapes/
Xueming, please file a bug and review.
Synopsis: Unreadable \uXXXX in javadoc
Description: Replace \uXXXX by \u005CXXXX, or simply delete
Martin
> If it works in a previewer, is there any chance to change the javadoc spec,
> staying backwards compatible?
>
> -Ulf
>
>
>
From Ulf.Zibis at gmx.de Wed Sep 2 14:58:25 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Wed, 02 Sep 2009 23:58:25 +0200
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<4A9CDB81.1050500@gmx.de>
<1ccfd1c10909012021g78d4fa3cx5f6ab0792c3ba688@mail.gmail.com>
<4A9E27BF.8000905@gmx.de>
<1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com>
<4A9E9FE9.7060107@redhat.com>
<1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com>
<4A9EA759.3050804@redhat.com> <4A9ECBAC.7060303@gmx.de>
<1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
Message-ID: <4A9EEA81.8000501@gmx.de>
Am 02.09.2009 22:29, Martin Buchholz schrieb:
> On Wed, Sep 2, 2009 at 12:46, Ulf Zibis wrote:
>
>> Am 02.09.2009 19:11, David M. Lloyd schrieb:
>>
>>> On 09/02/2009 12:03 PM, Martin Buchholz wrote:
>>>
>>>> On Wed, Sep 2, 2009 at 09:40, David M. Lloyd >>> > wrote:
>>>> Why not just do {@code \uD800}? I'm like 60% sure that would work
>>>> just fine. :-)
>>>>
>>>>
>>>> I'm pretty sure it would fail. Prove me wrong!
>>>> Searching the JDK sources for regex
>>>> ^ *\*.*\\u[0-9a-fA-F]{4}
>>>> is a good way to find javadoc bugs, e.g.
>>>> http://java.sun.com/javase/6/docs/api/java/lang/String.html#toLowerCase()
>>>>
>>> Ah, you're right. It worked in my previewer but not in the actual
>>> javadoc. It's pretty bad that that sequence has special meaning but you
>>> can't escape a \ with another \. I guess in the worst case you could always
>>> do \u005CD800 or something like that.
>>>
>>>
>> Looks little better, but not much. Did somebody tried it (Martin)?
>>
>
> Well.... learn something new every day.
> Let's turn this into a fix.
> It's yet another "turkish i" bug.
>
> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/javadoc-unicode-escapes/
>
> Xueming, please file a bug and review.
>
> Synopsis: Unreadable \uXXXX in javadoc
> Description: Replace \uXXXX by \u005CXXXX, or simply delete
>
Doesn't {@code "t\u005Cu0131tle"} work too ?
If not, can this be seen as a bug ?
> Searching the JDK sources for regex
> ^ *\*.*\\u[0-9a-fA-F]{4}
Doesn't it make sense to cover all occurrences in JDK ?
-Ulf
From martinrb at google.com Wed Sep 2 15:10:04 2009
From: martinrb at google.com (Martin Buchholz)
Date: Wed, 2 Sep 2009 15:10:04 -0700
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <4A9EEA81.8000501@gmx.de>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<1ccfd1c10909012021g78d4fa3cx5f6ab0792c3ba688@mail.gmail.com>
<4A9E27BF.8000905@gmx.de>
<1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com>
<4A9E9FE9.7060107@redhat.com>
<1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com>
<4A9EA759.3050804@redhat.com> <4A9ECBAC.7060303@gmx.de>
<1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
<4A9EEA81.8000501@gmx.de>
Message-ID: <1ccfd1c10909021510y41649b2ep650885c17a41ace5@mail.gmail.com>
On Wed, Sep 2, 2009 at 14:58, Ulf Zibis wrote:
> Am 02.09.2009 22:29, Martin Buchholz schrieb:
> Doesn't {@code "t\u005Cu0131tle"} work too ?
I think so. This would make it easier to globally
replace with {@code.
Here's a bit of Emacs Lisp to get you started:
(defun tt-code ()
(interactive)
(query-replace-regexp "<\\(tt\\|code\\)>\\([^<>\\\\]+\\)"
"{@code \\2}"))
>> Searching the JDK sources for regex
>> ^ *\*.*\\u[0-9a-fA-F]{4}
>
>
> Doesn't it make sense to cover all occurrences in JDK ?
In non-public javadoc
(that is, javadoc which is usually read directly instead of via the
generated html)
it is mostly harmless, and more readable as well.
Martin
From Xueming.Shen at Sun.COM Wed Sep 2 15:21:12 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Wed, 02 Sep 2009 15:21:12 -0700
Subject: Regex named-group and backreference syntax
In-Reply-To: <20090902194742.C8A3E1B37@eggemoggin.niobe.net>
References: <20090902194742.C8A3E1B37@eggemoggin.niobe.net>
Message-ID: <4A9EEFD8.3060002@sun.com>
RFE#6878475 has been filed to trace this issue.
At this point we are going to update the doc and implementation to
(1)disallow the group name started with digit character
(2)use ${name} for the group reference in replacement string.
The \k and ${n} can leave for further discussion (whether or not we
really
need this NEW feature, while there is existing syntax that can do the
job just fine).
Sherman
Mark Reinhold wrote:
>> Date: Wed, 02 Sep 2009 18:55:49 +0100
>> From: Andrew John Hughes
>>
>
>
>> 2009/9/2 Mark Reinhold :
>>
>>> I'd say ${name} is the best option here given that it will be familiar,
>>> at least, to Perl and .NET developers.
>>>
>> It's also used for substitution in shell and Makefiles :)
>>
>
> That too!
>
> - Mark
>
From Ulf.Zibis at gmx.de Wed Sep 2 15:46:24 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Thu, 03 Sep 2009 00:46:24 +0200
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <1ccfd1c10909021510y41649b2ep650885c17a41ace5@mail.gmail.com>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<1ccfd1c10909012021g78d4fa3cx5f6ab0792c3ba688@mail.gmail.com>
<4A9E27BF.8000905@gmx.de>
<1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com>
<4A9E9FE9.7060107@redhat.com>
<1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com>
<4A9EA759.3050804@redhat.com> <4A9ECBAC.7060303@gmx.de>
<1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
<4A9EEA81.8000501@gmx.de>
<1ccfd1c10909021510y41649b2ep650885c17a41ace5@mail.gmail.com>
Message-ID: <4A9EF5C0.60509@gmx.de>
Am 03.09.2009 00:10, Martin Buchholz schrieb:
> On Wed, Sep 2, 2009 at 14:58, Ulf Zibis wrote:
>
>> Doesn't {@code "t\u005Cu0131tle"} work too ?
>>
>
> I think so.
So can I securely replace all occurrences of ... by {@code
...} using \u005CuXXXX in Character.java ?
-Ulf
From irisg at alum.mit.edu Wed Sep 2 16:04:14 2009
From: irisg at alum.mit.edu (irisg at alum.mit.edu)
Date: Wed, 2 Sep 2009 16:04:14 -0700
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4A9EC69F.6000509@sun.com>
References: <4A9EC69F.6000509@sun.com>
Message-ID: <000301ca2c21$b1395d30$13ac1790$@mit.edu>
Hi, Mandy.
> This is related to 6857194: Add hotspot new perf counters to aid class
> loading performance measurement.
[ ... ]
> Webrev:
> http://cr.openjdk.java.net/~mchung/6878481/webrev.00/
Looks good.
iris
From forax at univ-mlv.fr Wed Sep 2 16:52:25 2009
From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=)
Date: Thu, 03 Sep 2009 01:52:25 +0200
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <000301ca2c21$b1395d30$13ac1790$@mit.edu>
References: <4A9EC69F.6000509@sun.com>
<000301ca2c21$b1395d30$13ac1790$@mit.edu>
Message-ID: <4A9F0539.2070104@univ-mlv.fr>
Le 03/09/2009 01:04, irisg at alum.mit.edu a ?crit :
> Hi, Mandy.
>
>
>> This is related to 6857194: Add hotspot new perf counters to aid class
>> loading performance measurement.
>>
> [ ... ]
>
>
>> Webrev:
>> http://cr.openjdk.java.net/~mchung/6878481/webrev.00/
>>
> Looks good.
>
> iris
>
In PerfCounter.java,
- static field perf should be final.
- newConstantPerfCounter() seems unused, it's perhaps for a future usage ?
- newPerfCounter(String) should be declared with default visibility
(not private),
currently the compiler generates useless accessors (access$000 etc).
- fields
private String name;
private LongBuffer lb;
should be declared final to avoid race problem,
even if I'am not sure this can happen here because all PerfCounter
constructor calls are done in a block i.e under the
protection of a lock.
R?mi
From uncle.alice at gmail.com Wed Sep 2 16:46:31 2009
From: uncle.alice at gmail.com (Alan Moore)
Date: Wed, 2 Sep 2009 16:46:31 -0700
Subject: Regex named-group and backreference syntax
In-Reply-To: <4A9EEFD8.3060002@sun.com>
References: <20090902194742.C8A3E1B37@eggemoggin.niobe.net>
<4A9EEFD8.3060002@sun.com>
Message-ID:
On Wed, Sep 2, 2009 at 3:21 PM, Xueming Shen wrote:
>
> RFE#6878475 has been filed to trace this issue.
>
> At this point we are going to update the doc and implementation to
> (1)disallow the group name started with digit character
> (2)use ${name} for the group reference in replacement string.
Great! Thank you.
> The \k and ${n} can leave for further discussion (whether or not we
> really need this NEW feature, while there is existing syntax that can
> do the job just fine).
Agreed--it's a separate issue. But at least now we'll have the option.
From David.Holmes at Sun.COM Wed Sep 2 17:08:03 2009
From: David.Holmes at Sun.COM (David Holmes - Sun Microsystems)
Date: Thu, 03 Sep 2009 10:08:03 +1000
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4A9EC69F.6000509@sun.com>
References: <4A9EC69F.6000509@sun.com>
Message-ID: <4A9F08E3.8020406@sun.com>
Hi Mandy,
Mandy Chung said the following on 09/03/09 05:25:
> This is related to 6857194: Add hotspot new perf counters to aid class
> loading performance measurement.
>
> It's useful to add performance counters in the library code so that perf
> data from the JDK and VM can be collected and output in a unified way
> (written in the jvmstat shared memory buffer). I add a simple
> sun.misc.PerfCounter class to maintain the list of perf counters for the
> library to use. This fix only instruments the class loading and jar/zip
> to collect simple basic metrics. Additional perf counters will be
> added in the future.
>
> Webrev:
> http://cr.openjdk.java.net/~mchung/6878481/webrev.00/
What is the performance overhead here? This seems like a lot of extra
code if you don't want to read the counters. Is there not some way to
only conditionally enable them?
We're sticking in a lot of System.nanoTime() calls and on some systems
it isn't necessarily cheap eg Windows - see 6440250. Even if it is
cheap, there are potentially a lot of them in classloading intensive
situations.
David
From Alan.Bateman at Sun.COM Thu Sep 3 01:54:55 2009
From: Alan.Bateman at Sun.COM (Alan Bateman)
Date: Thu, 03 Sep 2009 09:54:55 +0100
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4A9EC69F.6000509@sun.com>
References: <4A9EC69F.6000509@sun.com>
Message-ID: <4A9F845F.9060809@sun.com>
Mandy Chung wrote:
> This is related to 6857194: Add hotspot new perf counters to aid class
> loading performance measurement.
>
> It's useful to add performance counters in the library code so that
> perf data from the JDK and VM can be collected and output in a unified
> way (written in the jvmstat shared memory buffer). I add a simple
> sun.misc.PerfCounter class to maintain the list of perf counters for
> the library to use. This fix only instruments the class loading and
> jar/zip to collect simple basic metrics. Additional perf counters
> will be added in the future.
>
> Webrev:
> http://cr.openjdk.java.net/~mchung/6878481/webrev.00/
>
> Thanks
> Mandy
It's good to see the use of the instrumentation buffer extended to the
libraries.
I'm not sure how sun.misc.Perf behaves when running with
-XX:-UsePerfData. You've probably checked this anyway, but just in case...
It would be good to include a comment to define the meaning of each of
the counters. For example, in ClassLoader.loadClass, it's not clear if
you mean to include the time to check if the class has already been
loaded (just on that one, if the lookup is not meant to be included then
it would reduce the calls to nanoTimes a bit).
Is it necessary to have separate counters for zip and JAR? If I'm
reading this correctly, then zipFiles and jarFiles will both be
incremented when a JAR file is opened - is that right?
I agree with R?mi's comment that perf, name, and lb can be final.
Would addElapsedTime(start) be better named as addElapsedTimeFrom(start)
to make it clear that it doesn't add "start" to the elapsed time, but
rather the elapsed time from the given starting time?
This might sound crazy, but do the updates need to be synchronized? I
don't think there is any synchronization in the VM. I agree with David's
concern about the overhead of the calls to nanoTimes (which I'm sure you
are measuring) but for the synchronization there are places where both a
counter and an elapsed time are updated.
-Alan.
From forax at univ-mlv.fr Thu Sep 3 02:48:09 2009
From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=)
Date: Thu, 03 Sep 2009 11:48:09 +0200
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4A9F845F.9060809@sun.com>
References: <4A9EC69F.6000509@sun.com> <4A9F845F.9060809@sun.com>
Message-ID: <4A9F90D9.9070301@univ-mlv.fr>
Le 03/09/2009 10:54, Alan Bateman a ?crit :
> Mandy Chung wrote:
>> This is related to 6857194: Add hotspot new perf counters to aid
>> class loading performance measurement.
>>
>> It's useful to add performance counters in the library code so that
>> perf data from the JDK and VM can be collected and output in a
>> unified way (written in the jvmstat shared memory buffer). I add a
>> simple sun.misc.PerfCounter class to maintain the list of perf
>> counters for the library to use. This fix only instruments the class
>> loading and jar/zip to collect simple basic metrics. Additional
>> perf counters will be added in the future.
>>
>> Webrev:
>> http://cr.openjdk.java.net/~mchung/6878481/webrev.00/
>>
>> Thanks
>> Mandy
> It's good to see the use of the instrumentation buffer extended to the
> libraries.
>
> I'm not sure how sun.misc.Perf behaves when running with
> -XX:-UsePerfData. You've probably checked this anyway, but just in
> case...
>
> It would be good to include a comment to define the meaning of each of
> the counters. For example, in ClassLoader.loadClass, it's not clear if
> you mean to include the time to check if the class has already been
> loaded (just on that one, if the lookup is not meant to be included
> then it would reduce the calls to nanoTimes a bit).
>
> Is it necessary to have separate counters for zip and JAR? If I'm
> reading this correctly, then zipFiles and jarFiles will both be
> incremented when a JAR file is opened - is that right?
>
> I agree with R?mi's comment that perf, name, and lb can be final.
>
> Would addElapsedTime(start) be better named as
> addElapsedTimeFrom(start) to make it clear that it doesn't add "start"
> to the elapsed time, but rather the elapsed time from the given
> starting time?
>
> This might sound crazy, but do the updates need to be synchronized? I
> don't think there is any synchronization in the VM. I agree with
> David's concern about the overhead of the calls to nanoTimes (which
> I'm sure you are measuring) but for the synchronization there are
> places where both a counter and an elapsed time are updated.
Hi Alan,
Classloading is now done in parallele, zip and jar can be loaded by
different threads.
"for the synchronization there are places where both a counter and an
elapsed time are updated"
=> the problem is that these synchronisations are done on two different
monitors.
I don't see how to remove them easily.
Furthermore I think that all synchronized can not be easily replaced by
some atomics operations
(unsafe.put* or unsafe.compareAndSwap) because long in a ByteBuffer
aren't volatile.
Perhaps with the Doug Lea's Fences (scheduled to be introduced in jdk7)
but I'm not sure.
>
> -Alan.
R?mi
From Alan.Bateman at Sun.COM Thu Sep 3 02:56:04 2009
From: Alan.Bateman at Sun.COM (Alan Bateman)
Date: Thu, 03 Sep 2009 10:56:04 +0100
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4A9F90D9.9070301@univ-mlv.fr>
References: <4A9EC69F.6000509@sun.com> <4A9F845F.9060809@sun.com>
<4A9F90D9.9070301@univ-mlv.fr>
Message-ID: <4A9F92B4.6080201@sun.com>
R?mi Forax wrote:
> :
> Hi Alan,
> Classloading is now done in parallele, zip and jar can be loaded by
> different threads.
>
> "for the synchronization there are places where both a counter and an
> elapsed time are updated"
> => the problem is that these synchronisations are done on two
> different monitors.
> I don't see how to remove them easily.
>
> Furthermore I think that all synchronized can not be easily replaced
> by some atomics operations
> (unsafe.put* or unsafe.compareAndSwap) because long in a ByteBuffer
> aren't volatile.
> Perhaps with the Doug Lea's Fences (scheduled to be introduced in
> jdk7) but I'm not sure.
I understand - I'm just posing the question because correctness isn't a
major issue here and perhaps the trade-off could be considered.
-Alan.
From forax at univ-mlv.fr Thu Sep 3 03:59:48 2009
From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=)
Date: Thu, 03 Sep 2009 12:59:48 +0200
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4A9F92B4.6080201@sun.com>
References: <4A9EC69F.6000509@sun.com> <4A9F845F.9060809@sun.com>
<4A9F90D9.9070301@univ-mlv.fr> <4A9F92B4.6080201@sun.com>
Message-ID: <4A9FA1A4.4000401@univ-mlv.fr>
Le 03/09/2009 11:56, Alan Bateman a ?crit :
> R?mi Forax wrote:
>> :
>> Hi Alan,
>> Classloading is now done in parallele, zip and jar can be loaded by
>> different threads.
>>
>> "for the synchronization there are places where both a counter and an
>> elapsed time are updated"
>> => the problem is that these synchronisations are done on two
>> different monitors.
>> I don't see how to remove them easily.
>>
>> Furthermore I think that all synchronized can not be easily replaced
>> by some atomics operations
>> (unsafe.put* or unsafe.compareAndSwap) because long in a ByteBuffer
>> aren't volatile.
>> Perhaps with the Doug Lea's Fences (scheduled to be introduced in
>> jdk7) but I'm not sure.
> I understand - I'm just posing the question because correctness isn't
> a major issue here and perhaps the trade-off could be considered.
>
> -Alan.
Writing longs on a 32 bits system leads to really surprising results :)
R?mi
From Ulf.Zibis at gmx.de Thu Sep 3 06:22:33 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Thu, 03 Sep 2009 15:22:33 +0200
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<4A9CDB81.1050500@gmx.de>
<1ccfd1c10909012021g78d4fa3cx5f6ab0792c3ba688@mail.gmail.com>
<4A9E27BF.8000905@gmx.de>
<1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com>
<4A9E9FE9.7060107@redhat.com>
<1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com>
<4A9EA759.3050804@redhat.com> <4A9ECBAC.7060303@gmx.de>
<1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
Message-ID: <4A9FC319.3030508@gmx.de>
Am 02.09.2009 22:29, Martin Buchholz schrieb:
> On Wed, Sep 2, 2009 at 12:46, Ulf Zibis wrote:
>
>> Am 02.09.2009 19:11, David M. Lloyd schrieb:
>>
>>> On 09/02/2009 12:03 PM, Martin Buchholz wrote:
>>>
>>>> On Wed, Sep 2, 2009 at 09:40, David M. Lloyd >>> > wrote:
>>>> Why not just do {@code \uD800}? I'm like 60% sure that would work
>>>> just fine. :-)
>>>>
>>>>
>>>> I'm pretty sure it would fail. Prove me wrong!
>>>> Searching the JDK sources for regex
>>>> ^ *\*.*\\u[0-9a-fA-F]{4}
>>>> is a good way to find javadoc bugs, e.g.
>>>> http://java.sun.com/javase/6/docs/api/java/lang/String.html#toLowerCase()
>>>>
>>> Ah, you're right. It worked in my previewer but not in the actual
>>> javadoc. It's pretty bad that that sequence has special meaning but you
>>> can't escape a \ with another \. I guess in the worst case you could always
>>> do \u005CD800 or something like that.
>>>
>>>
>> Looks little better, but not much. Did somebody tried it (Martin)?
>>
>
> Well.... learn something new every day.
> Let's turn this into a fix.
> It's yet another "turkish i" bug.
>
> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/javadoc-unicode-escapes/
>
> Xueming, please file a bug and review.
>
> Synopsis: Unreadable \uXXXX in javadoc
> Description: Replace \uXXXX by \u005CXXXX, or simply delete
>
>
BTW regarding javadoc of class String:
- replace(char,char) and replace(CharSequence,CharSequence) should refer
to one another by @see....
- replace(CharSequence,CharSequence) should state, if original is
returned in case of no match.
-Ulf
From gnu_andrew at member.fsf.org Thu Sep 3 07:43:23 2009
From: gnu_andrew at member.fsf.org (Andrew John Hughes)
Date: Thu, 3 Sep 2009 15:43:23 +0100
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4A9F90D9.9070301@univ-mlv.fr>
References: <4A9EC69F.6000509@sun.com> <4A9F845F.9060809@sun.com>
<4A9F90D9.9070301@univ-mlv.fr>
Message-ID: <17c6771e0909030743l4a861ccwfdef158ea48962e6@mail.gmail.com>
2009/9/3 R?mi Forax :
> Le 03/09/2009 10:54, Alan Bateman a ?crit :
>>
>> Mandy Chung wrote:
>>>
>>> This is related to 6857194: Add hotspot new perf counters to aid class
>>> loading performance measurement.
>>>
>>> It's useful to add performance counters in the library code so that perf
>>> data from the JDK and VM can be collected and output in a unified way
>>> (written in the jvmstat shared memory buffer). ?I add a simple
>>> sun.misc.PerfCounter class to maintain the list of perf counters for the
>>> library to use. ?This fix only instruments the class loading and jar/zip to
>>> collect simple basic metrics. ? Additional perf counters will be added in
>>> the future.
>>>
>>> Webrev:
>>> ? ?http://cr.openjdk.java.net/~mchung/6878481/webrev.00/
>>>
>>> Thanks
>>> Mandy
>>
>> It's good to see the use of the instrumentation buffer extended to the
>> libraries.
>>
>> I'm not sure how sun.misc.Perf behaves when running with -XX:-UsePerfData.
>> You've probably checked this anyway, but just in case...
>>
>> It would be good to include a comment to define the meaning of each of the
>> counters. For example, in ClassLoader.loadClass, it's not clear if you mean
>> to include the time to check if the class has already been loaded (just on
>> that one, if the lookup is not meant to be included then it would reduce the
>> calls to nanoTimes a bit).
>>
>> Is it necessary to have separate counters for zip and JAR? If I'm reading
>> this correctly, then zipFiles and jarFiles will both be incremented when a
>> JAR file is opened - is that right?
>>
>> I agree with R?mi's comment that perf, name, and lb can be final.
>>
>> Would addElapsedTime(start) be better named as addElapsedTimeFrom(start)
>> to make it clear that it doesn't add "start" to the elapsed time, but rather
>> the elapsed time from the given starting time?
>>
>> This might sound crazy, but do the updates need to be synchronized? I
>> don't think there is any synchronization in the VM. I agree with David's
>> concern about the overhead of the calls to nanoTimes (which I'm sure you are
>> measuring) but for the synchronization there are places where both a counter
>> and an elapsed time are updated.
>
> Hi Alan,
> Classloading is now done in parallele, zip and jar can be loaded by
> different threads.
>
> "for the synchronization there are places where both a counter and an
> elapsed time are updated"
> => the problem is that these synchronisations are done on two different
> monitors.
> ? ? ?I don't see how to remove them easily.
>
> Furthermore I think that all synchronized can not be easily replaced by some
> atomics operations
> (unsafe.put* or unsafe.compareAndSwap) because long in a ByteBuffer aren't
> volatile.
> Perhaps with the Doug Lea's Fences (scheduled to be introduced in jdk7) but
> I'm not sure.
>
>>
>> -Alan.
>
> R?mi
>
The use of synchronized at present seems flawed as only the set
methods are protected and not the get. So lb.get could be called
while in an lb.put call. I don't see the reason for using a
LongBuffer either, as only index 0 is ever used. Why not use an
AtomicLong?
What is the purpose of d3dAvailable? I don't know this piece of code
so I'm not sure what it's counting, but it will presumably be zero
forever on non-Windows systems.
--
Andrew :-)
Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)
Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net
PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
From Alan.Bateman at Sun.COM Thu Sep 3 08:44:22 2009
From: Alan.Bateman at Sun.COM (Alan Bateman)
Date: Thu, 03 Sep 2009 16:44:22 +0100
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <17c6771e0909030743l4a861ccwfdef158ea48962e6@mail.gmail.com>
References: <4A9EC69F.6000509@sun.com> <4A9F845F.9060809@sun.com>
<4A9F90D9.9070301@univ-mlv.fr>
<17c6771e0909030743l4a861ccwfdef158ea48962e6@mail.gmail.com>
Message-ID: <4A9FE456.5040409@sun.com>
Andrew John Hughes wrote:
> :
> The use of synchronized at present seems flawed as only the set
> methods are protected and not the get. So lb.get could be called
> while in an lb.put call. I don't see the reason for using a
> LongBuffer either, as only index 0 is ever used. Why not use an
> AtomicLong?
>
The LongBuffer is a direct buffer with a reference to memory in the
PerfData instrumentation buffer. This is the shared memory that jvmstat
clients (jstat, VisualVM , ..) will attach to in order to monitor the VM.
> What is the purpose of d3dAvailable? I don't know this piece of code
> so I'm not sure what it's counting, but it will presumably be zero
> forever on non-Windows systems.
>
Mandy can say more about this, but it essentially allows jstat to see if
the target VM is using the D3D pipeline or not.
-Alan.
From Mandy.Chung at Sun.COM Thu Sep 3 09:13:01 2009
From: Mandy.Chung at Sun.COM (Mandy Chung)
Date: Thu, 03 Sep 2009 09:13:01 -0700
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4A9FE456.5040409@sun.com>
References: <4A9EC69F.6000509@sun.com> <4A9F845F.9060809@sun.com>
<4A9F90D9.9070301@univ-mlv.fr>
<17c6771e0909030743l4a861ccwfdef158ea48962e6@mail.gmail.com>
<4A9FE456.5040409@sun.com>
Message-ID: <4A9FEB0D.3040403@sun.com>
Alan Bateman wrote:
> Andrew John Hughes wrote:
>> :
>> The use of synchronized at present seems flawed as only the set
>> methods are protected and not the get. So lb.get could be called
>> while in an lb.put call. I don't see the reason for using a
>> LongBuffer either, as only index 0 is ever used. Why not use an
>> AtomicLong?
>>
> The LongBuffer is a direct buffer with a reference to memory in the
> PerfData instrumentation buffer. This is the shared memory that
> jvmstat clients (jstat, VisualVM , ..) will attach to in order to
> monitor the VM.
I can add synchronized in the get method. The PerfData instrumentation
buffer is designed to be read by another process such as jstat tool so
that we can monitor a running VM without interfering its execution.
>
>> What is the purpose of d3dAvailable? I don't know this piece of code
>> so I'm not sure what it's counting, but it will presumably be zero
>> forever on non-Windows systems.
>>
> Mandy can say more about this, but it essentially allows jstat to see
> if the target VM is using the D3D pipeline or not.
>
Exactly. D3D pipeline is not always available on all Windows machines
and it depends on the graphic board, the OS version, and other
configuration. We have been doing startup performance measurement on
various systems and there is perf difference with D3D available or not.
Adding a counter (it should be a constant counter) to indicate if D3D
pipeline is available would be useful.
Note that d3dAvailable is only added when running on Windows. On
Solaris/Linux, ClientCounters class will not be initialized and thus
d3dAvailable is not added. I should add a comment to describe about
platform-specific counters or optional counters.
Mandy
> -Alan.
From gnu_andrew at member.fsf.org Thu Sep 3 10:34:18 2009
From: gnu_andrew at member.fsf.org (Andrew John Hughes)
Date: Thu, 3 Sep 2009 18:34:18 +0100
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4A9FEB0D.3040403@sun.com>
References: <4A9EC69F.6000509@sun.com> <4A9F845F.9060809@sun.com>
<4A9F90D9.9070301@univ-mlv.fr>
<17c6771e0909030743l4a861ccwfdef158ea48962e6@mail.gmail.com>
<4A9FE456.5040409@sun.com> <4A9FEB0D.3040403@sun.com>
Message-ID: <17c6771e0909031034k24e70919qf5cc9f107e492acc@mail.gmail.com>
2009/9/3 Mandy Chung :
> Alan Bateman wrote:
>>
>> Andrew John Hughes wrote:
>>>
>>> :
>>> The use of synchronized at present seems flawed as only the set
>>> methods are protected and not the get. ?So lb.get could be called
>>> while in an lb.put call. ?I don't see the reason for using a
>>> LongBuffer either, as only index 0 is ever used. ?Why not use an
>>> AtomicLong?
>>>
>>
>> The LongBuffer is a direct buffer with a reference to memory in the
>> PerfData instrumentation buffer. This is the shared memory that jvmstat
>> clients (jstat, VisualVM , ..) will attach to in order to monitor the VM.
>
> I can add synchronized in the get method. ?The PerfData instrumentation
> buffer is designed to be read by another process such as jstat tool so that
> we can monitor a running VM without ?interfering its execution.
I see now it's been generated from the Perf class so I can understand
why the LongBuffer is being used.
As to locking get(), it gets a little more interesting if the values
are being retrieved from a different process. However, presumably the
get() method is invoked on the VM being probed rather than the VM
running jstat, etc. otherwise the wrong buffer would be read. So the
lock should still serve its purpose.
>>
>>> What is the purpose of d3dAvailable? ?I don't know this piece of code
>>> so I'm not sure what it's counting, but it will presumably be zero
>>> forever on non-Windows systems.
>>>
>>
>> Mandy can say more about this, but it essentially allows jstat to see if
>> the target VM is using the D3D pipeline or not.
>>
> Exactly. ?D3D pipeline is not always available on all Windows machines and
> it depends on the graphic board, the OS version, and other configuration.
> ?We have been doing startup performance measurement on various systems and
> there is perf difference with D3D available or not. ?Adding a counter (it
> should be a constant counter) to indicate if D3D pipeline is available would
> be useful.
>
> Note that d3dAvailable is only added when running on Windows. ?On
> Solaris/Linux, ClientCounters class will not be initialized and thus
> d3dAvailable is not added. ?I should add a comment to describe about
> platform-specific counters or optional counters.
>
Thanks, this makes sense now.
> Mandy
>>
>> -Alan.
>
>
--
Andrew :-)
Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)
Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net
PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
From Xueming.Shen at Sun.COM Thu Sep 3 10:49:38 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Thu, 03 Sep 2009 10:49:38 -0700
Subject: Regex named-group and backreference syntax
In-Reply-To:
References: <20090902194742.C8A3E1B37@eggemoggin.niobe.net>
<4A9EEFD8.3060002@sun.com>
Message-ID: <4AA001B2.6080103@sun.com>
Hi
Guess you might be interested to help review the change for this RFE,
here is the webrev.
http://cr.openjdk.java.net/~sherman/6878475/webrev
Since this is an API update, the change will need to go through a review
process before it can
be integrated into the workspace, so it might take a little while.
Regards,
Sherman
Alan Moore wrote:
> On Wed, Sep 2, 2009 at 3:21 PM, Xueming Shen wrote:
>
>> RFE#6878475 has been filed to trace this issue.
>>
>> At this point we are going to update the doc and implementation to
>> (1)disallow the group name started with digit character
>> (2)use ${name} for the group reference in replacement string.
>>
>
> Great! Thank you.
>
>
>> The \k and ${n} can leave for further discussion (whether or not we
>> really need this NEW feature, while there is existing syntax that can
>> do the job just fine).
>>
>
> Agreed--it's a separate issue. But at least now we'll have the option.
>
From Mandy.Chung at Sun.COM Thu Sep 3 11:02:02 2009
From: Mandy.Chung at Sun.COM (Mandy Chung)
Date: Thu, 03 Sep 2009 11:02:02 -0700
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4A9F845F.9060809@sun.com>
References: <4A9EC69F.6000509@sun.com> <4A9F845F.9060809@sun.com>
Message-ID: <4AA0049A.6050900@sun.com>
Alan Bateman wrote:
> Mandy Chung wrote:
>> This is related to 6857194: Add hotspot new perf counters to aid
>> class loading performance measurement.
>>
>> It's useful to add performance counters in the library code so that
>> perf data from the JDK and VM can be collected and output in a
>> unified way (written in the jvmstat shared memory buffer). I add a
>> simple sun.misc.PerfCounter class to maintain the list of perf
>> counters for the library to use. This fix only instruments the class
>> loading and jar/zip to collect simple basic metrics. Additional
>> perf counters will be added in the future.
>>
>> Webrev:
>> http://cr.openjdk.java.net/~mchung/6878481/webrev.00/
>>
>> Thanks
>> Mandy
> It's good to see the use of the instrumentation buffer extended to the
> libraries.
>
> I'm not sure how sun.misc.Perf behaves when running with
> -XX:-UsePerfData. You've probably checked this anyway, but just in
> case...
Adding -XX:-UsePerfData option works fine but I am yet to understand the
logics since this should be different than the case with a small
PerfDataMemorySize where the perf counters are still created but in the
C heap. I'm looking into the hotspot perfdata implementation further.
>
> It would be good to include a comment to define the meaning of each of
> the counters. For example, in ClassLoader.loadClass, it's not clear if
> you mean to include the time to check if the class has already been
> loaded (just on that one, if the lookup is not meant to be included
> then it would reduce the calls to nanoTimes a bit).
>
As David points out the high number of calls to System.nanoTime, I
should refine the fix not to include the lookup time. I'll update the
fix and also describes the meaning of each counter.
> Is it necessary to have separate counters for zip and JAR? If I'm
> reading this correctly, then zipFiles and jarFiles will both be
> incremented when a JAR file is opened - is that right?
>
That's right, both zipFiles and jarFiles counters are incremented.
> I agree with R?mi's comment that perf, name, and lb can be final.
>
Agreed.
> Would addElapsedTime(start) be better named as
> addElapsedTimeFrom(start) to make it clear that it doesn't add "start"
> to the elapsed time, but rather the elapsed time from the given
> starting time?
>
Ok.
> This might sound crazy, but do the updates need to be synchronized? I
> don't think there is any synchronization in the VM. I agree with
> David's concern about the overhead of the calls to nanoTimes (which
> I'm sure you are measuring) but for the synchronization there are
> places where both a counter and an elapsed time are updated.
>
Yes, I ran the server benchmarks on Windows XP that shows negligible
overhead. I have to rerun the solaris-i586 measurement as I got some
high standard deviation. I'm composing my reply to David's email.
With the synchronization, we might miss some updates. It's a tradeoff
between correctness and performance overhead (I have to find out the
reasons why it was decided not to have synchronization in the hotspot
implementation). Is the counter update a fast operation - updating the
direct buffer? I think so and thus this should not cause high lock
contention on the PerfCounter.
Thanks
Mandy
> -Alan.
From martinrb at google.com Thu Sep 3 11:10:42 2009
From: martinrb at google.com (Martin Buchholz)
Date: Thu, 3 Sep 2009 11:10:42 -0700
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <4A9FC319.3030508@gmx.de>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<1ccfd1c10909012021g78d4fa3cx5f6ab0792c3ba688@mail.gmail.com>
<4A9E27BF.8000905@gmx.de>
<1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com>
<4A9E9FE9.7060107@redhat.com>
<1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com>
<4A9EA759.3050804@redhat.com> <4A9ECBAC.7060303@gmx.de>
<1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
<4A9FC319.3030508@gmx.de>
Message-ID: <1ccfd1c10909031110p6d38eb9dv9787ff88f3b4d1b7@mail.gmail.com>
I agree that the spec can be clarified.
Another case is the similar
http://java.sun.com/javase/6/docs/api/java/lang/String.html#replace(char, char)
which states "Returns a new string", but then goes on to contradict itself,
although the meaning is clear.
Webrev please.
Martin
On Thu, Sep 3, 2009 at 06:22, Ulf Zibis wrote:
>
>
> Am 02.09.2009 22:29, Martin Buchholz schrieb:
>>
>> On Wed, Sep 2, 2009 at 12:46, Ulf Zibis wrote:
>>
>>>
>>> Am 02.09.2009 19:11, David M. Lloyd schrieb:
>>>
>>>>
>>>> On 09/02/2009 12:03 PM, Martin Buchholz wrote:
>>>>
>>>>>
>>>>> On Wed, Sep 2, 2009 at 09:40, David M. Lloyd >>>> > wrote:
>>>>> ? Why not just do {@code \uD800}? ?I'm like 60% sure that would work
>>>>> ? just fine. :-)
>>>>>
>>>>>
>>>>> I'm pretty sure it would fail. ? Prove me wrong!
>>>>> Searching the JDK sources for regex
>>>>> ^ *\*.*\\u[0-9a-fA-F]{4}
>>>>> is a good way to find javadoc bugs, e.g.
>>>>>
>>>>> http://java.sun.com/javase/6/docs/api/java/lang/String.html#toLowerCase()
>>>>>
>>>>
>>>> Ah, you're right. ?It worked in my previewer but not in the actual
>>>> javadoc. ?It's pretty bad that that sequence has special meaning but you
>>>> can't escape a \ with another \. ?I guess in the worst case you could
>>>> always
>>>> do \u005CD800 or something like that.
>>>>
>>>>
>>>
>>> Looks little better, but not much. Did somebody tried it (Martin)?
>>>
>>
>> Well.... learn something new every day.
>> Let's turn this into a fix.
>> It's yet another "turkish i" bug.
>>
>>
>> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/javadoc-unicode-escapes/
>>
>> Xueming, please file a bug and review.
>>
>> Synopsis: Unreadable \uXXXX in javadoc
>> Description: Replace \uXXXX by \u005CXXXX, or simply delete
>>
>>
>
> BTW regarding javadoc of class String:
> - replace(char,char) and replace(CharSequence,CharSequence) should refer to
> one another by @see....
> - replace(CharSequence,CharSequence) should state, if original is returned
> in case of no match.
>
> -Ulf
>
>
>
>
From Mandy.Chung at Sun.COM Thu Sep 3 11:36:48 2009
From: Mandy.Chung at Sun.COM (Mandy Chung)
Date: Thu, 03 Sep 2009 11:36:48 -0700
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4A9F08E3.8020406@sun.com>
References: <4A9EC69F.6000509@sun.com> <4A9F08E3.8020406@sun.com>
Message-ID: <4AA00CC0.9000402@sun.com>
David Holmes - Sun Microsystems wrote:
> Hi Mandy,
>
> Mandy Chung said the following on 09/03/09 05:25:
>> This is related to 6857194: Add hotspot new perf counters to aid
>> class loading performance measurement.
>>
>> It's useful to add performance counters in the library code so that
>> perf data from the JDK and VM can be collected and output in a
>> unified way (written in the jvmstat shared memory buffer). I add a
>> simple sun.misc.PerfCounter class to maintain the list of perf
>> counters for the library to use. This fix only instruments the class
>> loading and jar/zip to collect simple basic metrics. Additional
>> perf counters will be added in the future.
>>
>> Webrev:
>> http://cr.openjdk.java.net/~mchung/6878481/webrev.00/
>
> What is the performance overhead here? This seems like a lot of extra
> code if you don't want to read the counters. Is there not some way to
> only conditionally enable them?
I ran the server benchmarks and the Windows XP result shows negligible
difference. I'm rerunning my solaris-i586 measurement (I got high diff
in the standard deviation from my previous runs).
Windows XP SP 3 Intel Core 2 CPU @3.0Ghz, 1.96 GB RAM
==============================================================================
mchung.baseline.server:
Benchmark Samples Mean Stdev Geomean
Weight
reference_server 11 7591.49 38.91
jetstream 11 125.58 1.24 0.10
scimark 11 619.94 4.86 0.15
specjbb2000 11 94540.14 493.47 0.15
specjbb2005 11 43320.15 193.86 0.25
specjvm98 11 577.06 1.74 0.15
volano25 11 45661.45 1146.69 0.20
==============================================================================
mchung.perfcounters:
Benchmark Samples Mean Stdev %Diff P
Significant
reference_server 11 7588.94 48.89 -0.03
0.893 *
jetstream 11 125.57 0.88 -0.00
0.990 *
scimark 11 616.95 5.78 -0.48
0.203 *
specjbb2000 11 94040.74 385.93 -0.53
0.016 *
specjbb2005 11 43166.00 191.27 -0.36
0.075 *
specjvm98 11 577.39 1.69 0.06
0.654 *
volano25 11 46119.64 1259.95 1.00
0.383 *
==============================================================================
* - Not Significant: A non-zero %Diff for the mean could be noise. If the
%Diff is 0, an actual difference may still exist. In either case, more
samples would be needed to detect an actual difference in sample
means.
To provide a way to enable I can't think of a better way other than
adding the conditional check in the new calls I add.
>
> We're sticking in a lot of System.nanoTime() calls and on some systems
> it isn't necessarily cheap eg Windows - see 6440250. Even if it is
> cheap, there are potentially a lot of them in classloading intensive
> situations.
>
Thanks for the CR#. FYI. The hotspot VM has a "sun.os.hrt.frequency"
perf counter to store the QueryPerformanceFrequency. The system used
for the benchmark runs has the value of 2,992,550,000.
I am going to find a system using ACPI PMT and run the benchmarks to
measure its overhead after revising the fix to reduce the number of
System.nanoTime() calls.
Thanks
Mandy
> David
From martinrb at google.com Thu Sep 3 11:47:30 2009
From: martinrb at google.com (Martin Buchholz)
Date: Thu, 3 Sep 2009 11:47:30 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
Message-ID: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
Taking this discussion public.
It has happened that concurrently there are two fixes in progress
for long suffering RFE
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4206909
Given the number of votes and support in third party java zip
packages, jdk should certainly support it.
Sherman has a webrev here:
http://cr.openjdk.java.net/~sherman/zipflush/webrev/
Brandon Long at Google has contributed an implementation
that has a webrev here:
http://cr.openjdk.java.net/~martin/webrevs/openjdk7/Z_SYNC_FLUSH/
I've been doing some reviewing and thinking about this myself.
The zlib and Deflater API is hard to understand.
It seems to me that the proper place to specify the flushing behavior is
when calling setInput, not when calling one of the deflate methods.
I'm imagining something like
setInput(byte[]b, int off, int len, int flags)
or more java-esque
enum FlushingBehavior {
NO_FLUSH, SYNC_FLUSH, FULL_FLUSH, FINISH }
setInput(byte[]b, int off, int len, FlushingBehavior)
Martin
On Thu, Aug 27, 2009 at 23:46, Xueming Shen wrote:
> Another long over-due zip RFE
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4206909
> with 93 votes.
>
> Background info:
>
> zlib has following 6 allowed flush values when invoking deflate()/inflate
>
> #define Z_NO_FLUSH ? ? ?0
> #define Z_PARTIAL_FLUSH 1 /* will be removed, use Z_SYNC_FLUSH instead */
> #define Z_SYNC_FLUSH ? ?2
> #define Z_FULL_FLUSH ? ?3
> #define Z_FINISH ? ? ? ?4
> #define Z_BLOCK ? ? ? ? 5
>
> The values are not really used by inflate(), so we only need to worry about
> the
> deflate().
>
> Our implementation now uses Z_NO_FLUSH for all deflation except when
> "finish()"
> is invoked, in which case we use Z_FINISH.
>
> When Z_NO_FLUSH is used, the deflater may accumulate/pend/hold input data in
> its internal
> buffer for better compression (in which the existing output data might not
> be a "complete"
> block), some applications (as those mentioned in the bug report)however
> might need to flush
> all pending data out so the output data is a "complete" block, therefor the
> corresponding
> de-compressor can work on those data.
>
> Z_SYNC_FLUSH and Z_FULL_FLUSH achieve this. The difference is that the
> "full_flush" will
> reset the existing hash table and start a new one for the coming new data,
> which is really
> not very useful. And the only difference compared to current "reset()" is
> the reset() will
> need to reallocate the memory, not a big deal.
>
> See zlib.h for more details (?)
>
> The change in Deflater is actually relatively easy, see the webrev
>
> http://cr.openjdk.java.net/~sherman/zipflush/webrev
>
> I would like to hear your opinion on
>
> (1)I'm proposing to simply add a "flush()" method to support the
> Z_SYNC_FLUSH only. The
> alternative is to have a flush(int) method + 2 constant sync_flush and
> full_flush)
>
> (2)What should we do to the flush() method in DeflaterOutputStream class.
> Ideally it should
> be overridden like in the webrev, with API doc to explain that it does 2
> things (1) flush
> the deflater (2)flush the underlying outputstream, I believe this is what
> the developer is
> expecting and what the API should have been implemented at first place.
> However this definitely
> is an "incompatible" change, given the existing "flush the underlying os
> only" behavior has
> been in place for decade. I just wonder which approach we should take. I
> tend to leave it
> un-touched and do our best to "educate" the developer to go with
>
> ?79 ? ? static class FlushableDOS extends DeflaterOutputStream {
> ?80 ? ? ? ? public FlushableDOS(OutputStream out) {
> ?81 ? ? ? ? ? ? super(out);
> ?82 ? ? ? ? }
> ?83 ?84 ? ? ? ? public void flush() throws IOException {
> ?85 ? ? ? ? ? ? int len = 0;
> ?86 ? ? ? ? ? ? while ((len = def.flush(buf, 0, buf.length)) > 0) {
> ?87 ? ? ? ? ? ? ? ? out.write(buf, 0, len);
> ?88 ? ? ? ? ? ? }
> ?89 ? ? ? ? ? ? out.flush();
> ?90 ? ? ? ? }
> ?91 ? ? }
>
> We can add something in the class description to explain about the "flush"
>
> (3)During the debug, I noticed the InflaterInputStream.available() issue.
> The method is currently specified as "0 if EOF 1 otherwise", which does not
> look right. The BufferedReader.readLine() in test case Flush()is blocked in
> StreamDecoder#325, the inReady() at ln#323 keeps returning true for the
> inputstream (InflaterInputStream). I believe this available() should be
> specified/implemented as
>
> 0 if EOF, in.available() if inf.needInput() is true, otherwise 1.
>
> Opinion? This is going to be a separate bugid, if we agree to fix it. Btw,
> how to write above spec in English?:-)
>
> Sherman
>
>
>
>
>
>
>
>
>
>
>
From Ulf.Zibis at gmx.de Thu Sep 3 12:51:39 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Thu, 03 Sep 2009 21:51:39 +0200
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <1ccfd1c10909031110p6d38eb9dv9787ff88f3b4d1b7@mail.gmail.com>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<1ccfd1c10909012021g78d4fa3cx5f6ab0792c3ba688@mail.gmail.com>
<4A9E27BF.8000905@gmx.de>
<1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com>
<4A9E9FE9.7060107@redhat.com>
<1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com>
<4A9EA759.3050804@redhat.com> <4A9ECBAC.7060303@gmx.de>
<1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
<4A9FC319.3030508@gmx.de>
<1ccfd1c10909031110p6d38eb9dv9787ff88f3b4d1b7@mail.gmail.com>
Message-ID: <4AA01E4B.8020500@gmx.de>
Am 03.09.2009 20:10, Martin Buchholz schrieb:
> I agree that the spec can be clarified.
> Another case is the similar
>
> http://java.sun.com/javase/6/docs/api/java/lang/String.html#replace(char, char)
>
> which states "Returns a new string", but then goes on to contradict itself,
> although the meaning is clear.
>
> Webrev please.
>
>
Martin,
I was in the assumption, that you could add this clarification to your
CR as you are involved anyway on finishing the javadoc of String. As you
may be know, webrev script is a pain for me, as I'm on Windows. CYGWIN
is buggy in that case, so each 2nd time I try this, Windows comes to
hang because of C heap overflow.
I like to finish the Character class and file the changes via patch on
https://bugs.openjdk.java.net. So I would be happy If you could answer
my question, if replacing all occurrences of ... by {@code
...} using \u005CuXXXX is secure.
-Ulf
From Xueming.Shen at Sun.COM Thu Sep 3 15:02:51 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Thu, 03 Sep 2009 15:02:51 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <20090903204137.GB15971@google.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<20090903204137.GB15971@google.com>
Message-ID: <4AA03D0B.1030206@sun.com>
Same as the proposed spec/impl change to
InflaterInputStream.isAvailable(), it is also a risky/incompatible
change to override DeflaterOutputStream.flush() method to flush the
underlying deflater. Existing apps
that are using DOS.flush() (directly or in-directly) will not be happy
to see the overall compression ratio gets
worse after migrating to JDK7. I don't have any data to back this
assumption, given DOS has been used for
decade, this risk for sure is real. This of course contradicts to the
request of this RFE which expects/assumes
that the DOS.flush() should sync_flush the underlying deflater.
Basically the change in Deflater alone is "good" enough to provide A
solution for this issue (developer currently
has to use nasty/hack workaround), it's easy to override DOS.flush
(and/or IIS.isAvailble()) to whatever you
like if the existing is not what you want. Having said that, I still
hope the consensus is that we should just do
the RIGHT thing:-)
As you said, the only benefit of adding a pair of syncFlush()/flushed is
to match the existing finish()/finished,
personally I feel the finish()/finished() control flow() is
"complicated" enough, why add another one. Still fell
the best choice is to simply expose what the zlib provides, I'm still
debating with myself if the Z_FINISH should
be included as one of the flush options as well, someone might prefer
not to use that finish()/finished() control
flow...
Sherman
Brandon Long wrote:
> On 09/03/09 Martin Buchholz uttered the following other thing:
>
>> The zlib and Deflater API is hard to understand.
>> It seems to me that the proper place to specify the flushing behavior is
>> when calling setInput, not when calling one of the deflate methods.
>>
>> I'm imagining something like
>> setInput(byte[]b, int off, int len, int flags)
>> or more java-esque
>> enum FlushingBehavior {
>> NO_FLUSH, SYNC_FLUSH, FULL_FLUSH, FINISH }
>> setInput(byte[]b, int off, int len, FlushingBehavior)
>>
>
> Looking at DeflaterOutputStream, setInput is called in write, but flush
> is in a very separate place.
>
> We're using this to implement a compressed line oriented network
> protocol, and so we're likely to have buffering (multiple writes) and
> then a flush when we're done, we're not building up the entire response
> and calling write just once.
>
> So, I'm not sure how you'd get the expected "stream" behavior out of
> DeflaterOutputStream if you put the flushing behavior on setInput.
>
> Having Deflater.deflate take a flush type more closely mimics the
> zlib/jzlib deflate function. Having a separate flush call more closely
> matches finish/finished on the API.
>
> GNU's classpath "fixes" this behavior, and their API is a flush call on
> Deflater (though no flushed, so they're Deflater.deflate probably loops)
>
> Looking at Android's JDK, it has the same bug as the OpenJDK (no flush).
>
> Anyways, after Martin's review, I changed it to syncFlush and passed
> tightened up some other things.. and dropped the
> InflaterInputStream.available change which was similar to yours (he felt
> that would be harder to get in). I guess it partially depends on
> whether we're targetting OpenJDK 6 or 7.
>
> Brandon
>
>
>> Martin
>>
>> On Thu, Aug 27, 2009 at 23:46, Xueming Shen wrote:
>>
>>> Another long over-due zip RFE
>>> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4206909
>>> with 93 votes.
>>>
>>> Background info:
>>>
>>> zlib has following 6 allowed flush values when invoking deflate()/inflate
>>>
>>> #define Z_NO_FLUSH 0
>>> #define Z_PARTIAL_FLUSH 1 /* will be removed, use Z_SYNC_FLUSH instead */
>>> #define Z_SYNC_FLUSH 2
>>> #define Z_FULL_FLUSH 3
>>> #define Z_FINISH 4
>>> #define Z_BLOCK 5
>>>
>>> The values are not really used by inflate(), so we only need to worry about
>>> the
>>> deflate().
>>>
>>> Our implementation now uses Z_NO_FLUSH for all deflation except when
>>> "finish()"
>>> is invoked, in which case we use Z_FINISH.
>>>
>>> When Z_NO_FLUSH is used, the deflater may accumulate/pend/hold input data in
>>> its internal
>>> buffer for better compression (in which the existing output data might not
>>> be a "complete"
>>> block), some applications (as those mentioned in the bug report)however
>>> might need to flush
>>> all pending data out so the output data is a "complete" block, therefor the
>>> corresponding
>>> de-compressor can work on those data.
>>>
>>> Z_SYNC_FLUSH and Z_FULL_FLUSH achieve this. The difference is that the
>>> "full_flush" will
>>> reset the existing hash table and start a new one for the coming new data,
>>> which is really
>>> not very useful. And the only difference compared to current "reset()" is
>>> the reset() will
>>> need to reallocate the memory, not a big deal.
>>>
>>> See zlib.h for more details (?)
>>>
>>> The change in Deflater is actually relatively easy, see the webrev
>>>
>>> http://cr.openjdk.java.net/~sherman/zipflush/webrev
>>>
>>> I would like to hear your opinion on
>>>
>>> (1)I'm proposing to simply add a "flush()" method to support the
>>> Z_SYNC_FLUSH only. The
>>> alternative is to have a flush(int) method + 2 constant sync_flush and
>>> full_flush)
>>>
>>> (2)What should we do to the flush() method in DeflaterOutputStream class.
>>> Ideally it should
>>> be overridden like in the webrev, with API doc to explain that it does 2
>>> things (1) flush
>>> the deflater (2)flush the underlying outputstream, I believe this is what
>>> the developer is
>>> expecting and what the API should have been implemented at first place.
>>> However this definitely
>>> is an "incompatible" change, given the existing "flush the underlying os
>>> only" behavior has
>>> been in place for decade. I just wonder which approach we should take. I
>>> tend to leave it
>>> un-touched and do our best to "educate" the developer to go with
>>>
>>> 79 static class FlushableDOS extends DeflaterOutputStream {
>>> 80 public FlushableDOS(OutputStream out) {
>>> 81 super(out);
>>> 82 }
>>> 83 84 public void flush() throws IOException {
>>> 85 int len = 0;
>>> 86 while ((len = def.flush(buf, 0, buf.length)) > 0) {
>>> 87 out.write(buf, 0, len);
>>> 88 }
>>> 89 out.flush();
>>> 90 }
>>> 91 }
>>>
>>> We can add something in the class description to explain about the "flush"
>>>
>>> (3)During the debug, I noticed the InflaterInputStream.available() issue.
>>> The method is currently specified as "0 if EOF 1 otherwise", which does not
>>> look right. The BufferedReader.readLine() in test case Flush()is blocked in
>>> StreamDecoder#325, the inReady() at ln#323 keeps returning true for the
>>> inputstream (InflaterInputStream). I believe this available() should be
>>> specified/implemented as
>>>
>>> 0 if EOF, in.available() if inf.needInput() is true, otherwise 1.
>>>
>>> Opinion? This is going to be a separate bugid, if we agree to fix it. Btw,
>>> how to write above spec in English?:-)
>>>
>>> Sherman
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>
>
From martinrb at google.com Thu Sep 3 15:15:33 2009
From: martinrb at google.com (Martin Buchholz)
Date: Thu, 3 Sep 2009 15:15:33 -0700
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <4AA01E4B.8020500@gmx.de>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com>
<4A9E9FE9.7060107@redhat.com>
<1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com>
<4A9EA759.3050804@redhat.com> <4A9ECBAC.7060303@gmx.de>
<1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
<4A9FC319.3030508@gmx.de>
<1ccfd1c10909031110p6d38eb9dv9787ff88f3b4d1b7@mail.gmail.com>
<4AA01E4B.8020500@gmx.de>
Message-ID: <1ccfd1c10909031515o77fc310fn844d372ee03b4925@mail.gmail.com>
On Thu, Sep 3, 2009 at 12:51, Ulf Zibis wrote:
> Am 03.09.2009 20:10, Martin Buchholz schrieb:
> I like to finish the Character class and file the changes via patch on
> https://bugs.openjdk.java.net. So I would be happy If you could answer my
> question, if replacing all occurrences of ... by {@code ...}
> using ?\u005CuXXXX is secure.
Here is my recommendation:
I think it is worthwhile to convert => {@code
I have been doing this in files that I "own",
but have not tried to make a more pervasive change.
I support such a more pervasive change,
but it should be done with a script that gets a fair amount
of scrutiny first, and will have to be restricted to those
parts of the jdk that are truly maintained by the openjdk team,
not code for which the true master lives elsewhere.
I think we could get agreement to change at least core libraries.
As part of this project, you would need to be able to run
full builds including the javadoc, and verify that the
output html is not damaged by the process.
You could ignore a particular ... snippet if the
text in between contains troublesome characters such as
\ & < > @
Martin
From blong at google.com Thu Sep 3 13:41:37 2009
From: blong at google.com (Brandon Long)
Date: Thu, 3 Sep 2009 13:41:37 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
Message-ID: <20090903204137.GB15971@google.com>
On 09/03/09 Martin Buchholz uttered the following other thing:
> The zlib and Deflater API is hard to understand.
> It seems to me that the proper place to specify the flushing behavior is
> when calling setInput, not when calling one of the deflate methods.
>
> I'm imagining something like
> setInput(byte[]b, int off, int len, int flags)
> or more java-esque
> enum FlushingBehavior {
> NO_FLUSH, SYNC_FLUSH, FULL_FLUSH, FINISH }
> setInput(byte[]b, int off, int len, FlushingBehavior)
Looking at DeflaterOutputStream, setInput is called in write, but flush
is in a very separate place.
We're using this to implement a compressed line oriented network
protocol, and so we're likely to have buffering (multiple writes) and
then a flush when we're done, we're not building up the entire response
and calling write just once.
So, I'm not sure how you'd get the expected "stream" behavior out of
DeflaterOutputStream if you put the flushing behavior on setInput.
Having Deflater.deflate take a flush type more closely mimics the
zlib/jzlib deflate function. Having a separate flush call more closely
matches finish/finished on the API.
GNU's classpath "fixes" this behavior, and their API is a flush call on
Deflater (though no flushed, so they're Deflater.deflate probably loops)
Looking at Android's JDK, it has the same bug as the OpenJDK (no flush).
Anyways, after Martin's review, I changed it to syncFlush and passed
tightened up some other things.. and dropped the
InflaterInputStream.available change which was similar to yours (he felt
that would be harder to get in). I guess it partially depends on
whether we're targetting OpenJDK 6 or 7.
Brandon
> Martin
>
> On Thu, Aug 27, 2009 at 23:46, Xueming Shen wrote:
> > Another long over-due zip RFE
> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4206909
> > with 93 votes.
> >
> > Background info:
> >
> > zlib has following 6 allowed flush values when invoking deflate()/inflate
> >
> > #define Z_NO_FLUSH ? ? ?0
> > #define Z_PARTIAL_FLUSH 1 /* will be removed, use Z_SYNC_FLUSH instead */
> > #define Z_SYNC_FLUSH ? ?2
> > #define Z_FULL_FLUSH ? ?3
> > #define Z_FINISH ? ? ? ?4
> > #define Z_BLOCK ? ? ? ? 5
> >
> > The values are not really used by inflate(), so we only need to worry about
> > the
> > deflate().
> >
> > Our implementation now uses Z_NO_FLUSH for all deflation except when
> > "finish()"
> > is invoked, in which case we use Z_FINISH.
> >
> > When Z_NO_FLUSH is used, the deflater may accumulate/pend/hold input data in
> > its internal
> > buffer for better compression (in which the existing output data might not
> > be a "complete"
> > block), some applications (as those mentioned in the bug report)however
> > might need to flush
> > all pending data out so the output data is a "complete" block, therefor the
> > corresponding
> > de-compressor can work on those data.
> >
> > Z_SYNC_FLUSH and Z_FULL_FLUSH achieve this. The difference is that the
> > "full_flush" will
> > reset the existing hash table and start a new one for the coming new data,
> > which is really
> > not very useful. And the only difference compared to current "reset()" is
> > the reset() will
> > need to reallocate the memory, not a big deal.
> >
> > See zlib.h for more details (?)
> >
> > The change in Deflater is actually relatively easy, see the webrev
> >
> > http://cr.openjdk.java.net/~sherman/zipflush/webrev
> >
> > I would like to hear your opinion on
> >
> > (1)I'm proposing to simply add a "flush()" method to support the
> > Z_SYNC_FLUSH only. The
> > alternative is to have a flush(int) method + 2 constant sync_flush and
> > full_flush)
> >
> > (2)What should we do to the flush() method in DeflaterOutputStream class.
> > Ideally it should
> > be overridden like in the webrev, with API doc to explain that it does 2
> > things (1) flush
> > the deflater (2)flush the underlying outputstream, I believe this is what
> > the developer is
> > expecting and what the API should have been implemented at first place.
> > However this definitely
> > is an "incompatible" change, given the existing "flush the underlying os
> > only" behavior has
> > been in place for decade. I just wonder which approach we should take. I
> > tend to leave it
> > un-touched and do our best to "educate" the developer to go with
> >
> > ?79 ? ? static class FlushableDOS extends DeflaterOutputStream {
> > ?80 ? ? ? ? public FlushableDOS(OutputStream out) {
> > ?81 ? ? ? ? ? ? super(out);
> > ?82 ? ? ? ? }
> > ?83 ?84 ? ? ? ? public void flush() throws IOException {
> > ?85 ? ? ? ? ? ? int len = 0;
> > ?86 ? ? ? ? ? ? while ((len = def.flush(buf, 0, buf.length)) > 0) {
> > ?87 ? ? ? ? ? ? ? ? out.write(buf, 0, len);
> > ?88 ? ? ? ? ? ? }
> > ?89 ? ? ? ? ? ? out.flush();
> > ?90 ? ? ? ? }
> > ?91 ? ? }
> >
> > We can add something in the class description to explain about the "flush"
> >
> > (3)During the debug, I noticed the InflaterInputStream.available() issue.
> > The method is currently specified as "0 if EOF 1 otherwise", which does not
> > look right. The BufferedReader.readLine() in test case Flush()is blocked in
> > StreamDecoder#325, the inReady() at ln#323 keeps returning true for the
> > inputstream (InflaterInputStream). I believe this available() should be
> > specified/implemented as
> >
> > 0 if EOF, in.available() if inf.needInput() is true, otherwise 1.
> >
> > Opinion? This is going to be a separate bugid, if we agree to fix it. Btw,
> > how to write above spec in English?:-)
> >
> > Sherman
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
--
"I think, therefore, I am confused." -- RAW
http://www.fiction.net/blong/
From blong at google.com Thu Sep 3 15:19:44 2009
From: blong at google.com (Brandon Long)
Date: Thu, 3 Sep 2009 15:19:44 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA03D0B.1030206@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<20090903204137.GB15971@google.com> <4AA03D0B.1030206@sun.com>
Message-ID: <20090903221944.GB10024@google.com>
On 09/03/09 Xueming Shen uttered the following other thing:
>
> Same as the proposed spec/impl change to
> InflaterInputStream.isAvailable(), it is also a risky/incompatible
> change to override DeflaterOutputStream.flush() method to flush the
> underlying deflater. Existing apps
> that are using DOS.flush() (directly or in-directly) will not be happy
> to see the overall compression ratio gets
> worse after migrating to JDK7. I don't have any data to back this
> assumption, given DOS has been used for
> decade, this risk for sure is real. This of course contradicts to the
> request of this RFE which expects/assumes
> that the DOS.flush() should sync_flush the underlying deflater.
>
> Basically the change in Deflater alone is "good" enough to provide A
> solution for this issue (developer currently
> has to use nasty/hack workaround), it's easy to override DOS.flush
> (and/or IIS.isAvailble()) to whatever you
> like if the existing is not what you want. Having said that, I still
> hope the consensus is that we should just do
> the RIGHT thing:-)
Agreed, I hope making it do the right thing is the result. People wrote
an entire pure-java implementation of zlib to work around this problem.
I was the fourth team at Google to run into it separately. I don't
think anyone who explicitly calls "flush" will get what the expect.
And I wouldn't expect the overall compression ratio to get that much
worse, unless you are calling flush a really large amount of time.
Z_FULL_FLUSH would be a much bigger change, and clearly the wrong thing
to do on a DeflaterOutputStream.flush.
> As you said, the only benefit of adding a pair of syncFlush()/flushed is
> to match the existing finish()/finished,
> personally I feel the finish()/finished() control flow() is
> "complicated" enough, why add another one. Still fell
> the best choice is to simply expose what the zlib provides, I'm still
> debating with myself if the Z_FINISH should
> be included as one of the flush options as well, someone might prefer
> not to use that finish()/finished() control
> flow...
The biggest challenge is that you have to loop your calls to deflate
with FLUSH (and FINISH) till it completes, and knowing when you're done
is complicated, the finished/flushed methods simplify that... but if the
entire api was updated, its possible it could be reworked to make the
external looping unnecessary... though that may involve using more
memory and maybe cpu by using a growable output buffer instead of the
fixed size output byte array.
Anyways, hence the "stick to the minimal changes that match the existing
API", if someone decides that there needs to be a java.util.zip2
(java.util.compress?) that fixes up the whole API to make a cleaner
programmatic interface, that's way beyond my scope.
Brandon
> Sherman
>
> Brandon Long wrote:
>> On 09/03/09 Martin Buchholz uttered the following other thing:
>>
>>> The zlib and Deflater API is hard to understand.
>>> It seems to me that the proper place to specify the flushing behavior is
>>> when calling setInput, not when calling one of the deflate methods.
>>>
>>> I'm imagining something like
>>> setInput(byte[]b, int off, int len, int flags)
>>> or more java-esque
>>> enum FlushingBehavior {
>>> NO_FLUSH, SYNC_FLUSH, FULL_FLUSH, FINISH }
>>> setInput(byte[]b, int off, int len, FlushingBehavior)
>>>
>>
>> Looking at DeflaterOutputStream, setInput is called in write, but flush
>> is in a very separate place.
>>
>> We're using this to implement a compressed line oriented network
>> protocol, and so we're likely to have buffering (multiple writes) and
>> then a flush when we're done, we're not building up the entire response
>> and calling write just once.
>>
>> So, I'm not sure how you'd get the expected "stream" behavior out of
>> DeflaterOutputStream if you put the flushing behavior on setInput.
>>
>> Having Deflater.deflate take a flush type more closely mimics the
>> zlib/jzlib deflate function. Having a separate flush call more closely
>> matches finish/finished on the API.
>>
>> GNU's classpath "fixes" this behavior, and their API is a flush call on
>> Deflater (though no flushed, so they're Deflater.deflate probably loops)
>>
>> Looking at Android's JDK, it has the same bug as the OpenJDK (no flush).
>>
>> Anyways, after Martin's review, I changed it to syncFlush and passed
>> tightened up some other things.. and dropped the
>> InflaterInputStream.available change which was similar to yours (he felt
>> that would be harder to get in). I guess it partially depends on
>> whether we're targetting OpenJDK 6 or 7.
>>
>> Brandon
>>
>>> Martin
>>>
>>> On Thu, Aug 27, 2009 at 23:46, Xueming Shen wrote:
>>>
>>>> Another long over-due zip RFE
>>>> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4206909
>>>> with 93 votes.
>>>>
>>>> Background info:
>>>>
>>>> zlib has following 6 allowed flush values when invoking deflate()/inflate
>>>>
>>>> #define Z_NO_FLUSH 0
>>>> #define Z_PARTIAL_FLUSH 1 /* will be removed, use Z_SYNC_FLUSH instead */
>>>> #define Z_SYNC_FLUSH 2
>>>> #define Z_FULL_FLUSH 3
>>>> #define Z_FINISH 4
>>>> #define Z_BLOCK 5
>>>>
>>>> The values are not really used by inflate(), so we only need to worry about
>>>> the
>>>> deflate().
>>>>
>>>> Our implementation now uses Z_NO_FLUSH for all deflation except when
>>>> "finish()"
>>>> is invoked, in which case we use Z_FINISH.
>>>>
>>>> When Z_NO_FLUSH is used, the deflater may accumulate/pend/hold input data in
>>>> its internal
>>>> buffer for better compression (in which the existing output data might not
>>>> be a "complete"
>>>> block), some applications (as those mentioned in the bug report)however
>>>> might need to flush
>>>> all pending data out so the output data is a "complete" block, therefor the
>>>> corresponding
>>>> de-compressor can work on those data.
>>>>
>>>> Z_SYNC_FLUSH and Z_FULL_FLUSH achieve this. The difference is that the
>>>> "full_flush" will
>>>> reset the existing hash table and start a new one for the coming new data,
>>>> which is really
>>>> not very useful. And the only difference compared to current "reset()" is
>>>> the reset() will
>>>> need to reallocate the memory, not a big deal.
>>>>
>>>> See zlib.h for more details (?)
>>>>
>>>> The change in Deflater is actually relatively easy, see the webrev
>>>>
>>>> http://cr.openjdk.java.net/~sherman/zipflush/webrev
>>>>
>>>> I would like to hear your opinion on
>>>>
>>>> (1)I'm proposing to simply add a "flush()" method to support the
>>>> Z_SYNC_FLUSH only. The
>>>> alternative is to have a flush(int) method + 2 constant sync_flush and
>>>> full_flush)
>>>>
>>>> (2)What should we do to the flush() method in DeflaterOutputStream class.
>>>> Ideally it should
>>>> be overridden like in the webrev, with API doc to explain that it does 2
>>>> things (1) flush
>>>> the deflater (2)flush the underlying outputstream, I believe this is what
>>>> the developer is
>>>> expecting and what the API should have been implemented at first place.
>>>> However this definitely
>>>> is an "incompatible" change, given the existing "flush the underlying os
>>>> only" behavior has
>>>> been in place for decade. I just wonder which approach we should take. I
>>>> tend to leave it
>>>> un-touched and do our best to "educate" the developer to go with
>>>>
>>>> 79 static class FlushableDOS extends DeflaterOutputStream {
>>>> 80 public FlushableDOS(OutputStream out) {
>>>> 81 super(out);
>>>> 82 }
>>>> 83 84 public void flush() throws IOException {
>>>> 85 int len = 0;
>>>> 86 while ((len = def.flush(buf, 0, buf.length)) > 0) {
>>>> 87 out.write(buf, 0, len);
>>>> 88 }
>>>> 89 out.flush();
>>>> 90 }
>>>> 91 }
>>>>
>>>> We can add something in the class description to explain about the "flush"
>>>>
>>>> (3)During the debug, I noticed the InflaterInputStream.available() issue.
>>>> The method is currently specified as "0 if EOF 1 otherwise", which does not
>>>> look right. The BufferedReader.readLine() in test case Flush()is blocked in
>>>> StreamDecoder#325, the inReady() at ln#323 keeps returning true for the
>>>> inputstream (InflaterInputStream). I believe this available() should be
>>>> specified/implemented as
>>>>
>>>> 0 if EOF, in.available() if inf.needInput() is true, otherwise 1.
>>>>
>>>> Opinion? This is going to be a separate bugid, if we agree to fix it. Btw,
>>>> how to write above spec in English?:-)
>>>>
>>>> Sherman
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>>
--
"Have been unavoidably detained by the world.
Expect us when you see us." -- Neil Gaiman, _Stardust_
http://www.fiction.net/blong/
From martinrb at google.com Thu Sep 3 16:06:38 2009
From: martinrb at google.com (Martin Buchholz)
Date: Thu, 3 Sep 2009 16:06:38 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA03D0B.1030206@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<20090903204137.GB15971@google.com> <4AA03D0B.1030206@sun.com>
Message-ID: <1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
Here are review comments on your webrev
http://cr.openjdk.java.net/~sherman/zipflush/webrev
---
308 * Returns actual number of bytes of compressed data.
=>
308 * Returns actual number of bytes of compressed data written
to the output buffer.
---
{@linke needsInput() needsInput}
=>
{@link #needsInput()}
---
accumualte => accumulate
(yes, I know this is zlib's typo)
---
achieve the maximize => achieve the best
---
comparessed => compressed
---
358 * so far (In particular the{@link #needInput needInput}
returns {code@ true}
add missing space
code@ => @code
needInput => needsInput
---
178 public int available() throws IOException {
179 ensureOpen();
180 if (reachEOF) {
181 return 0;
182 } else {
183 if (inf.needsInput())
184 return in.available();
Even if you could deal with the compatibility problems of applications
relying on the 1/0 existing specified behavior, this code
seems wrong because an inflater might possibly shrink the input.
Maybe you can use something like deflateBounds?
---
8 if ("-client".equals(args[0]))
Confusing, because of hotspot flag with same name.
Are you trying to find a launcher bug?
How about "client" instead of "-client"?
Better yet, implement Flush as a single java program.
---
Martin
From martinrb at google.com Thu Sep 3 16:32:58 2009
From: martinrb at google.com (Martin Buchholz)
Date: Thu, 3 Sep 2009 16:32:58 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA03D0B.1030206@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<20090903204137.GB15971@google.com> <4AA03D0B.1030206@sun.com>
Message-ID: <1ccfd1c10909031632r4842b841p9ca6d081ee25e72b@mail.gmail.com>
A related mini-rfe would be to provide access to zlib's deflateBounds
so that a caller can produce an appropriately sized buffer
for deflation.
From Xueming.Shen at Sun.COM Thu Sep 3 16:31:49 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Thu, 03 Sep 2009 16:31:49 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<20090903204137.GB15971@google.com> <4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
Message-ID: <4AA051E5.5060605@sun.com>
Thanks for the comment. webrev has been updated accordingly. Flush class
is a quick hack to verify the
change, it needs to be auto run one if putback.
We might want to separate the proposed "incompatible" change of
DOS.flush() into a separate rfe/bug,
if we can not have a consensus if we should change the existing/default
behavior, and integrate the change
of Deflater first after we all agree on the interface. IIS.isAvailable()
definitely is a separate issue.
Sherman
Martin Buchholz wrote:
> Here are review comments on your webrev
> http://cr.openjdk.java.net/~sherman/zipflush/webrev
>
> ---
>
> 308 * Returns actual number of bytes of compressed data.
> =>
> 308 * Returns actual number of bytes of compressed data written
> to the output buffer.
>
> ---
>
> {@linke needsInput() needsInput}
> =>
> {@link #needsInput()}
>
> ---
>
> accumualte => accumulate
>
> (yes, I know this is zlib's typo)
>
> ---
>
> achieve the maximize => achieve the best
>
> ---
>
> comparessed => compressed
>
> ---
>
> 358 * so far (In particular the{@link #needInput needInput}
> returns {code@ true}
>
> add missing space
> code@ => @code
> needInput => needsInput
>
> ---
> 178 public int available() throws IOException {
> 179 ensureOpen();
> 180 if (reachEOF) {
> 181 return 0;
> 182 } else {
> 183 if (inf.needsInput())
> 184 return in.available();
>
> Even if you could deal with the compatibility problems of applications
> relying on the 1/0 existing specified behavior, this code
> seems wrong because an inflater might possibly shrink the input.
> Maybe you can use something like deflateBounds?
> ---
> 8 if ("-client".equals(args[0]))
>
> Confusing, because of hotspot flag with same name.
> Are you trying to find a launcher bug?
> How about "client" instead of "-client"?
>
> Better yet, implement Flush as a single java program.
>
> ---
> Martin
>
From martinrb at google.com Thu Sep 3 16:55:22 2009
From: martinrb at google.com (Martin Buchholz)
Date: Thu, 3 Sep 2009 16:55:22 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA051E5.5060605@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<20090903204137.GB15971@google.com> <4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
Message-ID: <1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
On Thu, Sep 3, 2009 at 16:31, Xueming Shen wrote:
> Thanks for the comment. webrev has been updated accordingly. Flush class is
> a quick hack to verify the
> change, it needs to be auto run one if putback.
Perhaps you would like to take Brandon's test InflateIn_DeflateOut.java?
> We might want to separate the proposed "incompatible" change of DOS.flush()
> into a separate rfe/bug,
Aside from incompatibility, some users might want SYNC_FLUSH,
some might want FULL_FLUSH. I think it's too big a change.
So just add a new method on DeflaterOutputStream to sync flush
current output.
> if we can not have a consensus if we should change the existing/default
> behavior, and integrate the change
> of Deflater first after we all agree on the interface. IIS.isAvailable()
> definitely is a separate issue.
I think others have tried and failed to change the behavior of
various isAvailable methods in the past.
Compatibility sucks, but users love it.
---
Given that you, Xueming are the current nominal owner of java.util.zip,
and we can't include both fixes in the JDK,
perhaps you would like to drive this, while getting input from us,
and being able to use anything from our fix?
Martin
From martinrb at google.com Thu Sep 3 17:24:08 2009
From: martinrb at google.com (Martin Buchholz)
Date: Thu, 3 Sep 2009 17:24:08 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA051E5.5060605@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<20090903204137.GB15971@google.com> <4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
Message-ID: <1ccfd1c10909031724u22e365a0m14cea3b685686956@mail.gmail.com>
The obvious alternative to built-in zlib
in the jdk is jzlib
http://www.jcraft.com/jzlib/
a pure-java implementation of zlib
under an apparently GPL-compatible license
that does support flush modes.
Some people are using it for that reason.
Non-scientific benchmarks at Google suggest
it is twice as slow as JDK Deflater,
so we may not want to throw out C zlib just yet.
Martin
From blong at google.com Thu Sep 3 17:08:59 2009
From: blong at google.com (Brandon Long)
Date: Thu, 3 Sep 2009 17:08:59 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<20090903204137.GB15971@google.com> <4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
Message-ID: <20090904000859.GC10024@google.com>
On 09/03/09 Martin Buchholz uttered the following other thing:
>
> ---
> 178 public int available() throws IOException {
> 179 ensureOpen();
> 180 if (reachEOF) {
> 181 return 0;
> 182 } else {
> 183 if (inf.needsInput())
> 184 return in.available();
>
> Even if you could deal with the compatibility problems of applications
> relying on the 1/0 existing specified behavior, this code
> seems wrong because an inflater might possibly shrink the input.
> Maybe you can use something like deflateBounds?
My version just returned 1 in this case, though even that might not be
true, but it was no worse than the current always return 1. Its a
pretty grey area, maybe deflateBounds would get closer to the truth.
Another option would be to try and to inflate a single byte or some
bounded number.
Brandon
--
"Its much more fun to be sand than oil in the machinery of life."
http://www.fiction.net/blong/
From Xueming.Shen at Sun.COM Thu Sep 3 17:55:26 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Thu, 03 Sep 2009 17:55:26 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <1ccfd1c10909031724u22e365a0m14cea3b685686956@mail.gmail.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<20090903204137.GB15971@google.com> <4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031724u22e365a0m14cea3b685686956@mail.gmail.com>
Message-ID: <4AA0657E.7060903@sun.com>
Martin Buchholz wrote:
> The obvious alternative to built-in zlib
> in the jdk is jzlib
>
> http://www.jcraft.com/jzlib/
>
> a pure-java implementation of zlib
> under an apparently GPL-compatible license
> that does support flush modes.
> Some people are using it for that reason.
>
> Non-scientific benchmarks at Google suggest
> it is twice as slow as JDK Deflater,
> so we may not want to throw out C zlib just yet.
>
>
jzlib owner claims the number one reason we have zlib in this world is
java deflater does
not export zlib's sync_flush and the previous maintainer refused to add
it after so many
years:-)
no, you can't throw out the c zlib even after we have a java version
benchmarked
faster (in 8?), vm and pack still need the native version.
From Ulf.Zibis at gmx.de Thu Sep 3 18:37:49 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Fri, 04 Sep 2009 03:37:49 +0200
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <1ccfd1c10909031515o77fc310fn844d372ee03b4925@mail.gmail.com>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com>
<4A9E9FE9.7060107@redhat.com>
<1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com>
<4A9EA759.3050804@redhat.com> <4A9ECBAC.7060303@gmx.de>
<1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
<4A9FC319.3030508@gmx.de>
<1ccfd1c10909031110p6d38eb9dv9787ff88f3b4d1b7@mail.gmail.com>
<4AA01E4B.8020500@gmx.de>
<1ccfd1c10909031515o77fc310fn844d372ee03b4925@mail.gmail.com>
Message-ID: <4AA06F6D.2080809@gmx.de>
Am 04.09.2009 00:15, Martin Buchholz schrieb:
> On Thu, Sep 3, 2009 at 12:51, Ulf Zibis wrote:
>
>> Am 03.09.2009 20:10, Martin Buchholz schrieb:
>>
>
>
>> I like to finish the Character class and file the changes via patch on
>> https://bugs.openjdk.java.net. So I would be happy If you could answer my
>> question, if replacing all occurrences of ... by {@code ...}
>> using \u005CuXXXX is secure.
>>
>
> Here is my recommendation:
>
> I think it is worthwhile to convert => {@code
>
I have tried {@code '\u005CuD800'} in Character.java and thanks NetBeans
I easily could create a partly-JDK javadoc (on about 100 classes) in
surprising short time. The resulting html works fine / is not damaged
(in Firefox). Thanks for your motivation.
BTW, in String class there are some warnings: @spec is an unknown tag.
In Character class you have inserted a typo: '\u007f'" at method
valueOf(char) (the additional " )
> I have been doing this in files that I "own",
> but have not tried to make a more pervasive change.
> I support such a more pervasive change,
> but it should be done with a script that gets a fair amount
> of scrutiny first, and will have to be restricted to those
> parts of the jdk that are truly maintained by the openjdk team,
> not code for which the true master lives elsewhere.
> I think we could get agreement to change at least core libraries.
>
> As part of this project, you would need to be able to run
> full builds
Yes, I should start working on Linux. I guess it would take me 3-4 week
to setup a full running system, which fully satisfies my needs. For this
reason I'm hesitating, because I want to progress with my charset ideas
as fast as possible.
On the other hand, it would be worthwhile there are some people who keep
developing on Windows to assure, that java stays working on Windows,
even under unusual JDK development. ;-) (BTW, I've found and filed
several Windows related bugs regarding NetBeans)
-Ulf
P.S.: You have the habit to use rare English vocabulary, especially in
this post, so I profit in increasing my knowledge in English.
From martinrb at google.com Thu Sep 3 18:52:52 2009
From: martinrb at google.com (Martin Buchholz)
Date: Thu, 3 Sep 2009 18:52:52 -0700
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <4AA06F6D.2080809@gmx.de>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com>
<4A9EA759.3050804@redhat.com> <4A9ECBAC.7060303@gmx.de>
<1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
<4A9FC319.3030508@gmx.de>
<1ccfd1c10909031110p6d38eb9dv9787ff88f3b4d1b7@mail.gmail.com>
<4AA01E4B.8020500@gmx.de>
<1ccfd1c10909031515o77fc310fn844d372ee03b4925@mail.gmail.com>
<4AA06F6D.2080809@gmx.de>
Message-ID: <1ccfd1c10909031852v2a09e439y4010841598d28a6e@mail.gmail.com>
On Thu, Sep 3, 2009 at 18:37, Ulf Zibis wrote:
> Am 04.09.2009 00:15, Martin Buchholz schrieb:
> In Character class you have inserted a typo: '\u007f'" at method
> valueOf(char) (the additional " )
When you say "you", I assume you are using the "royal you".
That change was actually made by Joe Darcy.
Martin
From martinrb at google.com Thu Sep 3 19:05:00 2009
From: martinrb at google.com (Martin Buchholz)
Date: Thu, 3 Sep 2009 19:05:00 -0700
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <1ccfd1c10909031852v2a09e439y4010841598d28a6e@mail.gmail.com>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<4A9EA759.3050804@redhat.com> <4A9ECBAC.7060303@gmx.de>
<1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
<4A9FC319.3030508@gmx.de>
<1ccfd1c10909031110p6d38eb9dv9787ff88f3b4d1b7@mail.gmail.com>
<4AA01E4B.8020500@gmx.de>
<1ccfd1c10909031515o77fc310fn844d372ee03b4925@mail.gmail.com>
<4AA06F6D.2080809@gmx.de>
<1ccfd1c10909031852v2a09e439y4010841598d28a6e@mail.gmail.com>
Message-ID: <1ccfd1c10909031905j634b184dy83a17e18558cfb05@mail.gmail.com>
Joe, here's a fix for the stray double quote.
I've used {@code with \u005C with the hope of making Ulf happy.
As usual, we'll need bug filed, etc...
diff --git a/src/share/classes/java/lang/Character.java
b/src/share/classes/java/lang/Character.java
--- a/src/share/classes/java/lang/Character.java
+++ b/src/share/classes/java/lang/Character.java
@@ -2587,9 +2587,9 @@
* significantly better space and time performance by caching
* frequently requested values.
*
- * This method will always cache values in the range '\u0000'
- * to '\u007f'", inclusive, and may cache other values outside
- * of this range.
+ * This method will always cache values in the range {@code
+ * '\u005Cu0000'} to {@code '\u005Cu007f'}, inclusive, and may
+ * cache other values outside of this range.
*
* @param c a char value.
* @return a Character instance representing c.
From weijun.wang at sun.com Fri Sep 4 00:09:45 2009
From: weijun.wang at sun.com (weijun.wang at sun.com)
Date: Fri, 04 Sep 2009 07:09:45 +0000
Subject: hg: jdk7/tl/jdk: 2 new changesets
Message-ID: <20090904071039.9227012BD5@hg.openjdk.java.net>
Changeset: ee5300e1835a
Author: weijun
Date: 2009-09-04 14:58 +0800
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/ee5300e1835a
6876328: different names for the same digest algorithms breaks jarsigner
Reviewed-by: mullan
! src/share/classes/sun/security/tools/JarSigner.java
+ test/sun/security/tools/jarsigner/nameclash.sh
Changeset: 98ad1322051e
Author: weijun
Date: 2009-09-04 14:59 +0800
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/98ad1322051e
6871847: AlgorithmId.get("SHA256withECDSA") not available
Reviewed-by: vinnie
! src/share/classes/sun/security/x509/AlgorithmId.java
+ test/sun/security/x509/AlgorithmId/SHA256withECDSA.java
From Ulf.Zibis at gmx.de Fri Sep 4 02:14:53 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Fri, 04 Sep 2009 11:14:53 +0200
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <1ccfd1c10909031852v2a09e439y4010841598d28a6e@mail.gmail.com>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com>
<4A9EA759.3050804@redhat.com> <4A9ECBAC.7060303@gmx.de>
<1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
<4A9FC319.3030508@gmx.de>
<1ccfd1c10909031110p6d38eb9dv9787ff88f3b4d1b7@mail.gmail.com>
<4AA01E4B.8020500@gmx.de>
<1ccfd1c10909031515o77fc310fn844d372ee03b4925@mail.gmail.com>
<4AA06F6D.2080809@gmx.de>
<1ccfd1c10909031852v2a09e439y4010841598d28a6e@mail.gmail.com>
Message-ID: <4AA0DA8D.9070408@gmx.de>
Am 04.09.2009 03:52, Martin Buchholz schrieb:
> When you say "you", I assume you are using the "royal you".
> That change was actually made by Joe Darcy.
>
> Martin
>
Oops sorry, I only recognized javadoc update from 1.6.0_14 to 1.7.0 b70,
so as this particular change was in similar flavour of your changes in
6860431, I actually and regrettably erroneously thought, that You
personally was the author.
I receive my just deserts, it's cold and raining today.
-Ulf
From martinrb at google.com Fri Sep 4 08:10:21 2009
From: martinrb at google.com (Martin Buchholz)
Date: Fri, 4 Sep 2009 08:10:21 -0700
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <4AA0DA8D.9070408@gmx.de>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<4A9ECBAC.7060303@gmx.de>
<1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
<4A9FC319.3030508@gmx.de>
<1ccfd1c10909031110p6d38eb9dv9787ff88f3b4d1b7@mail.gmail.com>
<4AA01E4B.8020500@gmx.de>
<1ccfd1c10909031515o77fc310fn844d372ee03b4925@mail.gmail.com>
<4AA06F6D.2080809@gmx.de>
<1ccfd1c10909031852v2a09e439y4010841598d28a6e@mail.gmail.com>
<4AA0DA8D.9070408@gmx.de>
Message-ID: <1ccfd1c10909040810i3bdeb991l34e165b299477d1f@mail.gmail.com>
On Fri, Sep 4, 2009 at 02:14, Ulf Zibis wrote:
> Oops sorry, I only recognized javadoc update from 1.6.0_14 to 1.7.0 b70,
Do you know about "hg blame" ?
From Ulf.Zibis at gmx.de Fri Sep 4 12:04:33 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Fri, 04 Sep 2009 21:04:33 +0200
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <1ccfd1c10909040810i3bdeb991l34e165b299477d1f@mail.gmail.com>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<4A9ECBAC.7060303@gmx.de>
<1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
<4A9FC319.3030508@gmx.de>
<1ccfd1c10909031110p6d38eb9dv9787ff88f3b4d1b7@mail.gmail.com>
<4AA01E4B.8020500@gmx.de>
<1ccfd1c10909031515o77fc310fn844d372ee03b4925@mail.gmail.com>
<4AA06F6D.2080809@gmx.de>
<1ccfd1c10909031852v2a09e439y4010841598d28a6e@mail.gmail.com>
<4AA0DA8D.9070408@gmx.de>
<1ccfd1c10909040810i3bdeb991l34e165b299477d1f@mail.gmail.com>
Message-ID: <4AA164C1.20507@gmx.de>
Am 04.09.2009 17:10, Martin Buchholz schrieb:
> On Fri, Sep 4, 2009 at 02:14, Ulf Zibis wrote:
>
>
>> Oops sorry, I only recognized javadoc update from 1.6.0_14 to 1.7.0 b70,
>>
>
> Do you know about "hg blame" ?
>
Good idea. :-)
I rarely used it. Thanks for remembering me.
-Ulf
From joe.darcy at sun.com Fri Sep 4 13:19:49 2009
From: joe.darcy at sun.com (joe.darcy at sun.com)
Date: Fri, 04 Sep 2009 20:19:49 +0000
Subject: hg: jdk7/tl/jdk: 6873951: test/java/lang/reflect/Generics/Probe.java
fails.
Message-ID: <20090904202035.25AB812FEA@hg.openjdk.java.net>
Changeset: c34f92a47245
Author: darcy
Date: 2009-09-04 13:11 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/c34f92a47245
6873951: test/java/lang/reflect/Generics/Probe.java fails.
Reviewed-by: alanb
! test/java/lang/reflect/Generics/Probe.java
From Joe.Darcy at Sun.COM Fri Sep 4 13:34:26 2009
From: Joe.Darcy at Sun.COM (Joseph D. Darcy)
Date: Fri, 04 Sep 2009 13:34:26 -0700
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <1ccfd1c10909031905j634b184dy83a17e18558cfb05@mail.gmail.com>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net>
<4A9EA759.3050804@redhat.com> <4A9ECBAC.7060303@gmx.de>
<1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com>
<4A9FC319.3030508@gmx.de>
<1ccfd1c10909031110p6d38eb9dv9787ff88f3b4d1b7@mail.gmail.com>
<4AA01E4B.8020500@gmx.de>
<1ccfd1c10909031515o77fc310fn844d372ee03b4925@mail.gmail.com>
<4AA06F6D.2080809@gmx.de>
<1ccfd1c10909031852v2a09e439y4010841598d28a6e@mail.gmail.com>
<1ccfd1c10909031905j634b184dy83a17e18558cfb05@mail.gmail.com>
Message-ID: <4AA179D2.7050407@sun.com>
Hello.
Martin Buchholz wrote:
> Joe, here's a fix for the stray double quote.
>
Mea culpa!
> I've used {@code with \u005C with the hope of making Ulf happy.
>
> As usual, we'll need bug filed, etc...
>
I've filed bug 6879368 "Remove stray quote in Character javadoc" for
this issue.
> diff --git a/src/share/classes/java/lang/Character.java
> b/src/share/classes/java/lang/Character.java
> --- a/src/share/classes/java/lang/Character.java
> +++ b/src/share/classes/java/lang/Character.java
> @@ -2587,9 +2587,9 @@
> * significantly better space and time performance by caching
> * frequently requested values.
> *
> - * This method will always cache values in the range '\u0000'
> - * to '\u007f'", inclusive, and may cache other values outside
> - * of this range.
> + * This method will always cache values in the range {@code
> + * '\u005Cu0000'} to {@code '\u005Cu007f'}, inclusive, and may
> + * cache other values outside of this range.
> *
> * @param c a char value.
> * @return a Character instance representing c.
>
I approve this going back.
Thanks,
-Joe
From martinrb at google.com Fri Sep 4 13:59:01 2009
From: martinrb at google.com (martinrb at google.com)
Date: Fri, 04 Sep 2009 20:59:01 +0000
Subject: hg: jdk7/tl/jdk: 6879368: Remove stray quote in Character javadoc
Message-ID: <20090904205940.3BF69E013@hg.openjdk.java.net>
Changeset: 704296144175
Author: martin
Date: 2009-09-04 13:44 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/704296144175
6879368: Remove stray quote in Character javadoc
Summary: Remove stray quote in Character.valueOf javadoc, using Ulf's \u005CuXXXX technique
Reviewed-by: darcy
! src/share/classes/java/lang/Character.java
From Xueming.Shen at Sun.COM Fri Sep 4 15:18:56 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Fri, 04 Sep 2009 15:18:56 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<20090903204137.GB15971@google.com> <4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
Message-ID: <4AA19250.2060803@sun.com>
I would like to take the conservative (read compatible) approach for
this issue.
(1)expose zlib's flush mode Z_NO_FLUSH, Z_SYNC_FLUSH and Z_FULL_FLUSH
(2)add deflate(byte[] b, int off, int len, int flush)
(3)document the existing deflate() continues to use Z_NO_FLUSH
(4)add DeflaterOutputStream.flush(int flushmode)
and leave DeflaterOutputStream.flush() un-touched.
http://cr.openjdk.java.net/~sherman/zipflush/webrev
So the sophisticated Deflater/DeflaterOutputStream user can now use all
3 flush flavors
by using the newly added APIs explicitly. "naive" Deflater/DOS user
(means no need for
the sync/full_flush) and existing application can continue live with/use
the existing API
without any compatibility concern. The only disadvantage of this
approach is the
"in-direct flush" use scenario (means the DOS is passed around and the
inherited flush()
is invoked by other wrapping classes) will not be benefited from this
change, they have
to override the flush() methold before passing the DOS object around,
like what I do
in the test case Flush()
static class MyDeflaterOutputStream extends DeflaterOutputStream {
public MyDeflaterOutputStream(OutputStream out) {
super(out);
}
public void flush() throws IOException {
flush(Deflater.SYNC_FLUSH);
}
}
A little inconvenient, but seen like worth the price of being
compatible, for now. And we
still have the choice to do whatever we want to do with this method in
the future should the
feedback show I'm wrong and the compatibility does not matter.
Opinion? If you guys agree (please also review the webrev) I can start
the "process".
Sherman
From blong at google.com Fri Sep 4 15:57:14 2009
From: blong at google.com (Brandon Long)
Date: Fri, 4 Sep 2009 15:57:14 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA19250.2060803@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<20090903204137.GB15971@google.com> <4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com>
Message-ID: <20090904225714.GB26286@google.com>
In the new Deflater.deflate, you are checking valid flush values against
NO_FLUSH twice (and missing FULL_FLUSH)
The new DeflaterOutputStream.flush, you loop as long as some output is
generated, which can call deflate an extra time. Normally, code that
does this loops until there is no def.needsInput and the output is less
than the size of the output buffer. I would have thought it would have
generated extra output, but then the loop would never finish... so I
guess this must work.
Overall, I'm least happy with #4, since I feel it leaves a bug. flush()
on a stream should flush everything I've written to the stream. This
bug is that it currently doesn't, and this doesn't fix it. It makes it
possible for people to fix it (which isn't possible currently without
using a completely separate implementation), but it doesn't fix the bug.
Brandon
On 09/04/09 Xueming Shen uttered the following other thing:
>
> I would like to take the conservative (read compatible) approach for
> this issue.
>
> (1)expose zlib's flush mode Z_NO_FLUSH, Z_SYNC_FLUSH and Z_FULL_FLUSH
> (2)add deflate(byte[] b, int off, int len, int flush)
> (3)document the existing deflate() continues to use Z_NO_FLUSH
> (4)add DeflaterOutputStream.flush(int flushmode)
> and leave DeflaterOutputStream.flush() un-touched.
>
> http://cr.openjdk.java.net/~sherman/zipflush/webrev
>
> So the sophisticated Deflater/DeflaterOutputStream user can now use all
> 3 flush flavors
> by using the newly added APIs explicitly. "naive" Deflater/DOS user
> (means no need for
> the sync/full_flush) and existing application can continue live with/use
> the existing API
> without any compatibility concern. The only disadvantage of this
> approach is the
> "in-direct flush" use scenario (means the DOS is passed around and the
> inherited flush()
> is invoked by other wrapping classes) will not be benefited from this
> change, they have
> to override the flush() methold before passing the DOS object around,
> like what I do
> in the test case Flush()
>
> static class MyDeflaterOutputStream extends DeflaterOutputStream {
> public MyDeflaterOutputStream(OutputStream out) {
> super(out);
> }
>
> public void flush() throws IOException {
> flush(Deflater.SYNC_FLUSH);
> }
> }
>
> A little inconvenient, but seen like worth the price of being
> compatible, for now. And we
> still have the choice to do whatever we want to do with this method in
> the future should the
> feedback show I'm wrong and the compatibility does not matter.
>
> Opinion? If you guys agree (please also review the webrev) I can start
> the "process".
>
> Sherman
>
>
--
"Quantum Mechanics: The dreams stuff is made of" -- Steven Wright
http://www.fiction.net/blong/
From Xueming.Shen at Sun.COM Fri Sep 4 18:02:08 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Fri, 04 Sep 2009 18:02:08 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <20090904225714.GB26286@google.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<20090903204137.GB15971@google.com> <4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
Message-ID: <4AA1B890.7000703@sun.com>
Thanks for the comments.
Brandon Long wrote:
> In the new Deflater.deflate, you are checking valid flush values against
> NO_FLUSH twice (and missing FULL_FLUSH)
>
good catch, a bad copy/paste. More unit tests will be written to catch
this kind of problem after
we finalize the APIs.
> The new DeflaterOutputStream.flush, you loop as long as some output is
> generated, which can call deflate an extra time. Normally, code that
> does this loops until there is no def.needsInput and the output is less
> than the size of the output buffer. I would have thought it would have
> generated extra output, but then the loop would never finish... so I
> guess this must work.
>
>
The current code works. But your suggestion of using len < b.length
(actually it means zlib's
availble_out > 0) seems a better and correct approach, which can same
an extra round. A
quick scan of the zlib deflate/flush_pending shows it should work, I
will update to that
after give the code a more careful read and run some tests. Given
each/every write() loops
till needsInput() returns true, I don't think we need to consider this
in flush().
> Overall, I'm least happy with #4, since I feel it leaves a bug. flush()
> on a stream should flush everything I've written to the stream. This
> bug is that it currently doesn't, and this doesn't fix it. It makes it
> possible for people to fix it (which isn't possible currently without
> using a completely separate implementation), but it doesn't fix the bug.
>
Understood and "agreed":-) I'm wiling to change position under more
pressure:-) And we
can add that anytime. It's better than put it in now and have to take it
out later or add in some
ugly workaround.
Sherman
> Brandon
>
> On 09/04/09 Xueming Shen uttered the following other thing:
>
>> I would like to take the conservative (read compatible) approach for
>> this issue.
>>
>> (1)expose zlib's flush mode Z_NO_FLUSH, Z_SYNC_FLUSH and Z_FULL_FLUSH
>> (2)add deflate(byte[] b, int off, int len, int flush)
>> (3)document the existing deflate() continues to use Z_NO_FLUSH
>> (4)add DeflaterOutputStream.flush(int flushmode)
>> and leave DeflaterOutputStream.flush() un-touched.
>>
>> http://cr.openjdk.java.net/~sherman/zipflush/webrev
>>
>> So the sophisticated Deflater/DeflaterOutputStream user can now use all
>> 3 flush flavors
>> by using the newly added APIs explicitly. "naive" Deflater/DOS user
>> (means no need for
>> the sync/full_flush) and existing application can continue live with/use
>> the existing API
>> without any compatibility concern. The only disadvantage of this
>> approach is the
>> "in-direct flush" use scenario (means the DOS is passed around and the
>> inherited flush()
>> is invoked by other wrapping classes) will not be benefited from this
>> change, they have
>> to override the flush() methold before passing the DOS object around,
>> like what I do
>> in the test case Flush()
>>
>> static class MyDeflaterOutputStream extends DeflaterOutputStream {
>> public MyDeflaterOutputStream(OutputStream out) {
>> super(out);
>> }
>>
>> public void flush() throws IOException {
>> flush(Deflater.SYNC_FLUSH);
>> }
>> }
>>
>> A little inconvenient, but seen like worth the price of being
>> compatible, for now. And we
>> still have the choice to do whatever we want to do with this method in
>> the future should the
>> feedback show I'm wrong and the compatibility does not matter.
>>
>> Opinion? If you guys agree (please also review the webrev) I can start
>> the "process".
>>
>> Sherman
>>
>>
>>
>
>
From martinrb at google.com Fri Sep 4 18:21:33 2009
From: martinrb at google.com (Martin Buchholz)
Date: Fri, 4 Sep 2009 18:21:33 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA1B890.7000703@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<20090903204137.GB15971@google.com> <4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
<4AA1B890.7000703@sun.com>
Message-ID: <1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
On Fri, Sep 4, 2009 at 18:02, Xueming Shen wrote:
> Thanks for the comments.
>
> Brandon Long wrote:
>>
>> In the new Deflater.deflate, you are checking valid flush values against
>> NO_FLUSH twice (and missing FULL_FLUSH)
>>
>
> good catch, a bad copy/paste. More unit tests will be written to catch this
> kind of problem after
> we finalize the APIs.
>
>> The new DeflaterOutputStream.flush, you loop as long as some output is
>> generated, which can call deflate an extra time. ?Normally, code that
>> does this loops until there is no def.needsInput and the output is less
>> than the size of the output buffer. ?I would have thought it would have
>> generated extra output, but then the loop would never finish... so I
>> guess this must work.
>>
>>
>
> The current code works. ?But your suggestion of using len < b.length
> (actually it means zlib's
> availble_out > ?0) seems a better and correct approach, which can same an
> extra round. A
> quick scan of the zlib deflate/flush_pending shows it should work, I will
> update to that
> after give the code a more careful read and run some tests. Given each/every
> write() loops
> till needsInput() returns true, I don't think we need to consider this in
> flush().
>
>> Overall, I'm least happy with #4, since I feel it leaves a bug. ?flush()
>> on a stream should flush everything I've written to the stream. ?This
>> bug is that it currently doesn't, and this doesn't fix it. ?It makes it
>> possible for people to fix it (which isn't possible currently without
>> using a completely separate implementation), but it doesn't fix the bug.
>>
I think "bug" is going too far. There are OutputStreams that can't dispose
of previously read data without reading some more first. It depends on
the transformation. DeflaterOutputStream is in a gray area -
it "could" write all the data to its underlying stream, but at the cost of
sacrificing some compression (which is its purpose).
> Understood and "agreed":-) I'm wiling to change position under more
> pressure:-) ?And we
> can add that anytime. It's better than put it in now and have to take it out
> later or add in some
> ugly workaround.
Maybe we should understand the risk. Doing a SYNC_FLUSH on every
DOS.flush() won't cause the compression/decompression to "break",
you will just have very slow and very bad compression.
How bad could it be? Suppose we test with random data,
and doing a SYNC_FLUSH on every byte? Presumably the "compressed"
output will be larger than the input by some factor. If that factor
is close to 1,
then it's probably OK...It's "only" a performance problem.
Anyways, I am leaning towards changing DOS to do the Right Thing.
Martin
From blong at google.com Sat Sep 5 00:23:50 2009
From: blong at google.com (Brandon Long)
Date: Sat, 5 Sep 2009 00:23:50 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<20090903204137.GB15971@google.com> <4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
Message-ID:
On Fri, Sep 4, 2009 at 6:21 PM, Martin Buchholz wrote:
> On Fri, Sep 4, 2009 at 18:02, Xueming Shen wrote:
>>> Overall, I'm least happy with #4, since I feel it leaves a bug. ?flush()
>>> on a stream should flush everything I've written to the stream. ?This
>>> bug is that it currently doesn't, and this doesn't fix it. ?It makes it
>>> possible for people to fix it (which isn't possible currently without
>>> using a completely separate implementation), but it doesn't fix the bug.
>
> I think "bug" is going too far. ?There are OutputStreams that can't dispose
> of previously read data without reading some more first. ?It depends on
> the transformation. ?DeflaterOutputStream is in a gray area -
> it "could" write all the data to its underlying stream, but at the cost of
> sacrificing some compression (which is its purpose).
I think you're saying read when you mean write... and what is the
point of calling flush if it doesn't mean to actually flush what I
wrote?
As for bug, how about :
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4077821
and
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4032746
Which both speak that it breaks the OutputStream interface (and both
were closed as "not reproducible?)
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4206909
The doc for Flushable.flush says:
Flushes this stream by writing any buffered output to the
underlying stream.
I think the lack of flush actually working on a DeflaterOutputStream
is the bug that we're attempting to fix.
>> Understood and "agreed":-) I'm wiling to change position under more
>> pressure:-) ?And we
>> can add that anytime. It's better than put it in now and have to take it out
>> later or add in some
>> ugly workaround.
>
> Maybe we should understand the risk. ?Doing a SYNC_FLUSH on every
> DOS.flush() won't cause the compression/decompression to "break",
> you will just have very slow and very bad compression.
> How bad could it be? ?Suppose we test with random data,
> and doing a SYNC_FLUSH on every byte? ?Presumably the "compressed"
> output will ?be larger than the input by some factor. ?If that factor
> is close to 1,
> then it's probably OK...It's "only" a performance problem.
>
> Anyways, I am leaning towards changing DOS to do the Right Thing.
I imagine that doing it on every byte would be horrible, but I imagine
there is some trade-off point at some number of bytes, past which it
won't make much difference.
Running some tests, it usually takes about 50 bytes between flushes
before the compression is "reasonable", and it achieves equivalent to
"no flushes" at about 500-2000 bytes, depending on input (at level 6,
at least). At less than 10 bytes, the compressed stream is larger.
Lots of variability, though.
Brandon
From alan.bateman at sun.com Sat Sep 5 08:07:05 2009
From: alan.bateman at sun.com (alan.bateman at sun.com)
Date: Sat, 05 Sep 2009 15:07:05 +0000
Subject: hg: jdk7/tl/jdk: 4 new changesets
Message-ID: <20090905150831.EE952E07F@hg.openjdk.java.net>
Changeset: 3f87b755b1c8
Author: alanb
Date: 2009-09-04 18:15 +0100
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/3f87b755b1c8
6873621: (file) FileStore.supportsFileAttributeView(Class type) returns wrong result
Reviewed-by: andrew
! src/share/sample/nio/file/Xdd.java
! src/solaris/classes/sun/nio/fs/LinuxFileStore.java
! src/solaris/classes/sun/nio/fs/SolarisFileStore.java
! src/solaris/classes/sun/nio/fs/UnixFileStore.java
! src/windows/classes/sun/nio/fs/WindowsFileStore.java
! test/java/nio/file/FileStore/Basic.java
Changeset: 05ea733a7ae2
Author: alanb
Date: 2009-09-04 18:17 +0100
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/05ea733a7ae2
6868627: (spec) Files.walkFileTree doesn't make it clear that uncaught errors and exceptions are propagated
Reviewed-by: sherman
! src/share/classes/java/nio/file/Files.java
! src/share/classes/java/nio/file/SimpleFileVisitor.java
Changeset: 87a2ef2439bc
Author: alanb
Date: 2009-09-04 22:22 +0100
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/87a2ef2439bc
6432567: PIT : com/sun/jdi/BadHandshakeTest.java fails due to java.net.ConnectException
Reviewed-by: tbell, ohair, dcubed, andrew
! src/share/transport/socket/socketTransport.c
! test/com/sun/jdi/BadHandshakeTest.java
Changeset: 7afdf9d0bc2c
Author: alanb
Date: 2009-09-05 15:57 +0100
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/7afdf9d0bc2c
Merge
From martinrb at google.com Sat Sep 5 09:24:05 2009
From: martinrb at google.com (Martin Buchholz)
Date: Sat, 5 Sep 2009 09:24:05 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To:
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
Message-ID: <1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
On Sat, Sep 5, 2009 at 00:23, Brandon Long wrote:
> On Fri, Sep 4, 2009 at 6:21 PM, Martin Buchholz wrote:
>> On Fri, Sep 4, 2009 at 18:02, Xueming Shen wrote:
>>>> Overall, I'm least happy with #4, since I feel it leaves a bug. ?flush()
>>>> on a stream should flush everything I've written to the stream. ?This
>>>> bug is that it currently doesn't, and this doesn't fix it. ?It makes it
>>>> possible for people to fix it (which isn't possible currently without
>>>> using a completely separate implementation), but it doesn't fix the bug.
>>
>> I think "bug" is going too far. ?There are OutputStreams that can't dispose
>> of previously read data without reading some more first. ?It depends on
>> the transformation. ?DeflaterOutputStream is in a gray area -
>> it "could" write all the data to its underlying stream, but at the cost of
>> sacrificing some compression (which is its purpose).
>
> I think you're saying read when you mean write... and what is the
> point of calling flush if it doesn't mean to actually flush what I
> wrote?
I was thinking from the OutputStream's point of view - it "reads"
data when its "write" method is invoked. I guess "receive" is a
better word.
Suppose you have a ByteSwappingOutputStream whose contract
was to swap every other pair of bytes. If it has received an odd
number of bytes, its flush method cannot write the last byte,
because it needs another byte first. You can interpret flush's
contract as saying to write all buffered output that is "ready"
to go out *unmodified* - it might not be expected that flush()
causes different bytes to be written just so that they can be
written *now*. But we are proposing that DOS.flush()
does in fact modify the bytes written, possibly in a way that
violates DOS's contract, which is to compress the data.
Although DeflaterOutputStream's contract is not really clear -
is its contract to maximally compress the output, or merely to produce
output that can be read later by an Inflater?
That said, I still lean towards changing DOS.flush as suggested,
but I also understand why conservative maintainers might not.
> As for bug, how about :
>
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4077821
>
> and
>
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4032746
Those are some ancient bugs!
... that I don't recall having looked at in my
years of part-time maintenance of java.util.zip.
Martin
>
> Which both speak that it breaks the OutputStream interface (and both
> were closed as "not reproducible?)
>
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4206909
>
> The doc for Flushable.flush says:
> ? ? ? ? ? Flushes this stream by writing any buffered output to the
> underlying stream.
>
> I think the lack of flush actually working on a DeflaterOutputStream
> is the bug that we're attempting to fix.
From Ulf.Zibis at gmx.de Sat Sep 5 12:54:30 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Sat, 05 Sep 2009 21:54:30 +0200
Subject: hg: jdk7/tl/jdk: 6879368: Remove stray quote in Character javadoc
In-Reply-To: <20090904205940.3BF69E013@hg.openjdk.java.net>
References: <20090904205940.3BF69E013@hg.openjdk.java.net>
Message-ID: <4AA2C1F6.8000403@gmx.de>
Am 04.09.2009 22:59, martinrb at google.com schrieb:
> Changeset: 704296144175
> Author: martin
> Date: 2009-09-04 13:44 -0700
> URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/704296144175
>
> 6879368: Remove stray quote in Character javadoc
> Summary: Remove stray quote in Character.valueOf javadoc, using Ulf's \u005CuXXXX technique
> Reviewed-by: darcy
>
> ! src/share/classes/java/lang/Character.java
>
>
>
To be accurate: It's David M. Lloyd's technique!
-Ulf
From Ulf.Zibis at gmx.de Sun Sep 6 14:31:28 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Sun, 06 Sep 2009 23:31:28 +0200
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <4AA01E4B.8020500@gmx.de>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net> <1ccfd1c10909012021g78d4fa3cx5f6ab0792c3ba688@mail.gmail.com> <4A9E27BF.8000905@gmx.de> <1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com> <4A9E9FE9.7060107@redhat.com> <1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com> <4A9EA759.3050804@redhat.com>
<4A9ECBAC.7060303@gmx.de> <1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com> <4A9FC319.3030508@gmx.de> <1ccfd1c10909031110p6d38eb9dv9787ff88f3b4d1b7@mail.gmail.com>
<4AA01E4B.8020500@gmx.de>
Message-ID: <4AA42A30.6080703@gmx.de>
Cosmetics for class Character is finished (there have been javadoc
errors in String+AbstractStringbuilder too):
https://bugs.openjdk.java.net/attachment.cgi?id=145&action=diff
-Ulf
Am 03.09.2009 21:51, Ulf Zibis schrieb:
> Am 03.09.2009 20:10, Martin Buchholz schrieb:
>> I agree that the spec can be clarified.
>> Another case is the similar
>>
>> http://java.sun.com/javase/6/docs/api/java/lang/String.html#replace(char,
>> char)
>>
>> which states "Returns a new string", but then goes on to contradict
>> itself,
>> although the meaning is clear.
>>
>> Webrev please.
>>
>>
>
> Martin,
>
> I was in the assumption, that you could add this clarification to your
> CR as you are involved anyway on finishing the javadoc of String. As
> you may be know, webrev script is a pain for me, as I'm on Windows.
> CYGWIN is buggy in that case, so each 2nd time I try this, Windows
> comes to hang because of C heap overflow.
>
> I like to finish the Character class and file the changes via patch on
> https://bugs.openjdk.java.net. So I would be happy If you could answer
> my question, if replacing all occurrences of ... by
> {@code ...} using \u005CuXXXX is secure.
>
> -Ulf
>
>
>
From Ulf.Zibis at gmx.de Mon Sep 7 05:30:03 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Mon, 07 Sep 2009 14:30:03 +0200
Subject: hg: jdk7/tl/jdk: 6860431: Character.isSurrogate(char ch)
In-Reply-To: <4AA42A30.6080703@gmx.de>
References: <20090831221217.2CEFA12912@hg.openjdk.java.net> <1ccfd1c10909012021g78d4fa3cx5f6ab0792c3ba688@mail.gmail.com> <4A9E27BF.8000905@gmx.de> <1ccfd1c10909020927v74fe5ceekc91f4e4a4724a273@mail.gmail.com> <4A9E9FE9.7060107@redhat.com> <1ccfd1c10909021003o7b060a23ge700680cd75b07bf@mail.gmail.com> <4A9EA759.3050804@redhat.com> <4A9ECBAC.7060303@gmx.de> <1ccfd1c10909021329i34005b1bi5816e695d71a174d@mail.gmail.com> <4A9FC319.3030508@gmx.de> <1ccfd1c10909031110p6d38eb9dv9787ff88f3b4d1b7@mail.gmail.com> <4AA01E4B.8020500@gmx.de>
<4AA42A30.6080703@gmx.de>
Message-ID: <4AA4FCCB.3010107@gmx.de>
Am 06.09.2009 23:31, Ulf Zibis schrieb:
> Cosmetics for class Character is finished (there have been javadoc
> errors in String+AbstractStringbuilder too):
> https://bugs.openjdk.java.net/attachment.cgi?id=145&action=diff
>
Update:
https://bugs.openjdk.java.net/attachment.cgi?id=146&action=diff
Benefit from package private helpers:
https://bugs.openjdk.java.net/attachment.cgi?id=147&action=diff
Removed usages of class Surrogate where ever possible:
https://bugs.openjdk.java.net/attachment.cgi?id=148&action=diff
-Ulf
From Ulf.Zibis at gmx.de Mon Sep 7 06:01:00 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Mon, 07 Sep 2009 15:01:00 +0200
Subject: review request for 6798511/6860431: Include functionality of
Surrogate in Character -> use scenarios
In-Reply-To: <4A9578C4.8060801@sun.com>
References: <4A95079A.8080803@gmx.de> <4A9578C4.8060801@sun.com>
Message-ID: <4AA5040C.8050701@gmx.de>
Am 26.08.2009 20:02, Xueming Shen schrieb:
> Ulf,
>
> "get rid of sun.nio.cs.Surrogate" itself is not a sufficient enough
> reason to add bunch of new supplementary
> support related APIs into Character and CharBuffer classes. You
> probably need to demonstrate more use
> scenarios to show/prove why these new APIs are needed, why they can
> not be easily achieved by using existing
> APIs and especially you should ask yourself first if these APIs
> will/should be used/needed by "general public"
> or they are just "specifically" needed by your current
> application/project.
I have made a search for "surrogate" on sun.font and sun.text.* packages.
I've found > 400 matches.
In search on java.* I found another 291 matches.
Looking deeper in the classes you can find many use scenarios.
E.g. class sun.text.normalizer.UTF16 more ore less is a duplicate of
sun.nio.cs.Surrogate.
>
> I'm not saying we can NOT add isBMP() (I know icu4j's UCharacter class
> does have one), just
> believe it's arguably not necessary.
>
> Same as the pair
>
> -- static char highSurrogate(int codePoint);
--> sun.text.normalizer.UTF16.getLeadSurrogate(int char32)
> -- static char lowSurrogate(int codePoint);
--> sun.text.normalizer.UTF16.getTrailSurrogate(int char32)
> -- CharBuffer.putCodePoint(int)
Maybe it would be better to add appendCodePoint(int cp) to Charsequence,
of something similar.
See: sun.text.normalizer.UTF16.append(StringBuffer target, int char32)
See: java.text.CharacterIterator.CodePointIterator
-Ulf
From tim.bell at sun.com Mon Sep 7 09:44:52 2009
From: tim.bell at sun.com (tim.bell at sun.com)
Date: Mon, 07 Sep 2009 16:44:52 +0000
Subject: hg: jdk7/tl: Added tag jdk7-b71 for changeset 4c36e9853dda
Message-ID: <20090907164453.2B39CE0F1@hg.openjdk.java.net>
Changeset: 378f57273f09
Author: xdono
Date: 2009-09-03 10:52 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/rev/378f57273f09
Added tag jdk7-b71 for changeset 4c36e9853dda
! .hgtags
From tim.bell at sun.com Mon Sep 7 09:51:22 2009
From: tim.bell at sun.com (tim.bell at sun.com)
Date: Mon, 07 Sep 2009 16:51:22 +0000
Subject: hg: jdk7/tl/corba: 4 new changesets
Message-ID: <20090907165126.8A819E0F8@hg.openjdk.java.net>
Changeset: 8001ba2bf10d
Author: andrew
Date: 2009-08-20 01:28 +0100
URL: http://hg.openjdk.java.net/jdk7/tl/corba/rev/8001ba2bf10d
6873059: Explicitly use -source 6 -target 6 when compiling with the boot jdk javac
Summary: The bootstrap javac currently uses the default source and targets of the boot javac
Reviewed-by: jjg, ohair
! make/common/shared/Defs-java.gmk
Changeset: 04414f276160
Author: xdono
Date: 2009-08-24 17:25 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/corba/rev/04414f276160
Merge
Changeset: 3f1ef7f899ea
Author: andrew
Date: 2009-09-01 23:44 +0100
URL: http://hg.openjdk.java.net/jdk7/tl/corba/rev/3f1ef7f899ea
6878106: Synchronize CORBA and JDK makefiles where possible
Summary: Add recent changes to the JDK makefile to the CORBA makefile
Reviewed-by: jjg, never
! make/common/shared/Defs-java.gmk
Changeset: c793a3120926
Author: xdono
Date: 2009-09-03 10:52 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/corba/rev/c793a3120926
Added tag jdk7-b71 for changeset 3f1ef7f899ea
! .hgtags
From tim.bell at sun.com Mon Sep 7 09:59:41 2009
From: tim.bell at sun.com (tim.bell at sun.com)
Date: Mon, 07 Sep 2009 16:59:41 +0000
Subject: hg: jdk7/tl/hotspot: Added tag jdk7-b71 for changeset 50a95aa4a247
Message-ID: <20090907165947.3A9F5E0FF@hg.openjdk.java.net>
Changeset: 6e6427f797c0
Author: xdono
Date: 2009-09-03 10:52 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/hotspot/rev/6e6427f797c0
Added tag jdk7-b71 for changeset 50a95aa4a247
! .hgtags
From tim.bell at sun.com Mon Sep 7 10:15:50 2009
From: tim.bell at sun.com (tim.bell at sun.com)
Date: Mon, 07 Sep 2009 17:15:50 +0000
Subject: hg: jdk7/tl/jaxp: Added tag jdk7-b71 for changeset ff94d8ce0dad
Message-ID: <20090907171552.7AF24E104@hg.openjdk.java.net>
Changeset: 37c805b6156f
Author: xdono
Date: 2009-09-03 10:52 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jaxp/rev/37c805b6156f
Added tag jdk7-b71 for changeset ff94d8ce0dad
! .hgtags
From tim.bell at sun.com Mon Sep 7 10:22:01 2009
From: tim.bell at sun.com (tim.bell at sun.com)
Date: Mon, 07 Sep 2009 17:22:01 +0000
Subject: hg: jdk7/tl/jaxws: Added tag jdk7-b71 for changeset 03314cf56a72
Message-ID: <20090907172204.27D39E109@hg.openjdk.java.net>
Changeset: 4c990aa99bc0
Author: xdono
Date: 2009-09-03 10:52 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jaxws/rev/4c990aa99bc0
Added tag jdk7-b71 for changeset 03314cf56a72
! .hgtags
From tim.bell at sun.com Mon Sep 7 10:29:02 2009
From: tim.bell at sun.com (tim.bell at sun.com)
Date: Mon, 07 Sep 2009 17:29:02 +0000
Subject: hg: jdk7/tl/jdk: 9 new changesets
Message-ID: <20090907173119.D25E8E10E@hg.openjdk.java.net>
Changeset: 80368890a2a0
Author: andrew
Date: 2009-08-18 19:50 +0100
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/80368890a2a0
6873059: Explicitly use -source 6 -target 6 when compiling with the boot jdk javac
Summary: The bootstrap javac currently uses the default source and targets of the boot javac
Reviewed-by: ohair
! make/common/shared/Defs-java.gmk
Changeset: 43465920bf47
Author: xdono
Date: 2009-08-18 19:53 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/43465920bf47
Merge
- test/java/util/concurrent/ConcurrentLinkedQueue/ConcurrentQueueLoops.java
- test/java/util/concurrent/ConcurrentLinkedQueue/LoopHelpers.java
Changeset: b3aac0db5586
Author: tbell
Date: 2009-08-21 12:12 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/b3aac0db5586
6705913: freetype_versioncheck.exe - Unable To Locate Component
Summary: Update freetype_versioncheck to deal with newer Visual Studio releases
Reviewed-by: ohair
! make/tools/freetypecheck/Makefile
! make/tools/freetypecheck/freetypecheck.c
Changeset: e0b26d347302
Author: xdono
Date: 2009-08-24 17:26 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/e0b26d347302
Merge
Changeset: b3f3240135f0
Author: xdono
Date: 2009-09-01 13:03 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/b3f3240135f0
Merge
- src/share/classes/sun/nio/ch/AbstractFuture.java
Changeset: ce3fde68c495
Author: xdono
Date: 2009-09-03 10:53 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/ce3fde68c495
Added tag jdk7-b71 for changeset b3f3240135f0
! .hgtags
Changeset: ed0863629d28
Author: tbell
Date: 2009-09-03 18:32 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/ed0863629d28
Merge
- src/share/native/java/util/zip/zlib-1.1.3/ChangeLog
- src/share/native/java/util/zip/zlib-1.1.3/README
- src/share/native/java/util/zip/zlib-1.1.3/compress.c
- src/share/native/java/util/zip/zlib-1.1.3/deflate.c
- src/share/native/java/util/zip/zlib-1.1.3/deflate.h
- src/share/native/java/util/zip/zlib-1.1.3/doc/algorithm.doc
- src/share/native/java/util/zip/zlib-1.1.3/example.c
- src/share/native/java/util/zip/zlib-1.1.3/gzio.c
- src/share/native/java/util/zip/zlib-1.1.3/infblock.c
- src/share/native/java/util/zip/zlib-1.1.3/infblock.h
- src/share/native/java/util/zip/zlib-1.1.3/infcodes.c
- src/share/native/java/util/zip/zlib-1.1.3/infcodes.h
- src/share/native/java/util/zip/zlib-1.1.3/inffast.c
- src/share/native/java/util/zip/zlib-1.1.3/inffast.h
- src/share/native/java/util/zip/zlib-1.1.3/inffixed.h
- src/share/native/java/util/zip/zlib-1.1.3/inflate.c
- src/share/native/java/util/zip/zlib-1.1.3/inftrees.c
- src/share/native/java/util/zip/zlib-1.1.3/inftrees.h
- src/share/native/java/util/zip/zlib-1.1.3/infutil.c
- src/share/native/java/util/zip/zlib-1.1.3/infutil.h
- src/share/native/java/util/zip/zlib-1.1.3/minigzip.c
- src/share/native/java/util/zip/zlib-1.1.3/trees.c
- src/share/native/java/util/zip/zlib-1.1.3/trees.h
- src/share/native/java/util/zip/zlib-1.1.3/uncompr.c
- src/share/native/java/util/zip/zlib-1.1.3/zadler32.c
- src/share/native/java/util/zip/zlib-1.1.3/zconf.h
- src/share/native/java/util/zip/zlib-1.1.3/zcrc32.c
- src/share/native/java/util/zip/zlib-1.1.3/zlib.h
- src/share/native/java/util/zip/zlib-1.1.3/zutil.c
- src/share/native/java/util/zip/zlib-1.1.3/zutil.h
- test/java/util/concurrent/LinkedBlockingQueue/LastElement.java
- test/java/util/concurrent/LinkedBlockingQueue/OfferRemoveLoops.java
Changeset: 658a4255c797
Author: tbell
Date: 2009-09-04 17:07 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/658a4255c797
Merge
Changeset: abb69e8b1774
Author: tbell
Date: 2009-09-06 23:14 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/abb69e8b1774
Merge
From tim.bell at sun.com Mon Sep 7 10:44:03 2009
From: tim.bell at sun.com (tim.bell at sun.com)
Date: Mon, 07 Sep 2009 17:44:03 +0000
Subject: hg: jdk7/tl/langtools: 2 new changesets
Message-ID: <20090907174410.44916E11F@hg.openjdk.java.net>
Changeset: d434aa041b52
Author: xdono
Date: 2009-09-03 10:53 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/d434aa041b52
Added tag jdk7-b71 for changeset 33c8c38e1757
! .hgtags
Changeset: 90c28923e449
Author: tbell
Date: 2009-09-03 18:34 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/90c28923e449
Merge
- test/tools/javac/innerClassFile/Driver.java
- test/tools/javac/meth/InvokeMH_BAD68.java
- test/tools/javac/meth/InvokeMH_BAD72.java
- test/tools/javac/quid/QuotedIdent_BAD61.java
- test/tools/javac/quid/QuotedIdent_BAD62.java
- test/tools/javac/quid/QuotedIdent_BAD63.java
From Xueming.Shen at Sun.COM Mon Sep 7 11:27:27 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Mon, 07 Sep 2009 11:27:27 -0700
Subject: review request for 6798511/6860431: Include functionality of
Surrogate in Character -> use scenarios
In-Reply-To: <4AA5040C.8050701@gmx.de>
References: <4A95079A.8080803@gmx.de> <4A9578C4.8060801@sun.com>
<4AA5040C.8050701@gmx.de>
Message-ID: <4AA5508F.1090307@sun.com>
Ulf,
sun.text.normalizer might not be a good example, it is "independent
third-party" package we licensed. Usually
we try our best to keep this kind of package as-is, for maintenance
reason, you don't want to make the trivial
update allover the places again and again whenever there is a version
upgrade. Yes, I did make some modification
when dropped in the package when I was the owner years ago, that was for
functionally better fit with the rest
of the code and mostly for better performance (especially faster
startup). You should be able to "smell" that is
part of the icu4j, so it has its own utf16 handling:-)
sherman
Ulf Zibis wrote:
>
> Looking deeper in the classes you can find many use scenarios.
> E.g. class sun.text.normalizer.UTF16 more ore less is a duplicate of
> sun.nio.cs.Surrogate.
>
>>
>> I'm not saying we can NOT add isBMP() (I know icu4j's UCharacter
>> class does have one), just
>> believe it's arguably not necessary.
>>
>> Same as the pair
>>
>> -- static char highSurrogate(int codePoint);
>
> --> sun.text.normalizer.UTF16.getLeadSurrogate(int char32)
>
>> -- static char lowSurrogate(int codePoint);
>
> --> sun.text.normalizer.UTF16.getTrailSurrogate(int char32)
>
>> -- CharBuffer.putCodePoint(int)
>
> Maybe it would be better to add appendCodePoint(int cp) to
> Charsequence, of something similar.
> See: sun.text.normalizer.UTF16.append(StringBuffer target, int char32)
> See: java.text.CharacterIterator.CodePointIterator
>
>
>
> -Ulf
>
>
From Ulf.Zibis at gmx.de Mon Sep 7 12:44:23 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Mon, 07 Sep 2009 21:44:23 +0200
Subject: review request for 6798511/6860431: Include functionality of
Surrogate in Character -> use scenarios
In-Reply-To: <4AA5508F.1090307@sun.com>
References: <4A95079A.8080803@gmx.de> <4A9578C4.8060801@sun.com>
<4AA5040C.8050701@gmx.de> <4AA5508F.1090307@sun.com>
Message-ID: <4AA56297.2040501@gmx.de>
Sherman,
thanks for insisting on icu4j. I must admit, I newer heard about it. :-(
I took a short look on http://icu-project.org/download/4.2.html#ICU4J
etc. Very interesting stuff.
Now I'm thinking: why they don't use Character class? Do they miss those
3..4 methods? ;-)
May be it's time to talk with them, so they could reduce their
footprint, which would reduce JDK footprint in a row.
-Ulf
Am 07.09.2009 20:27, Xueming Shen schrieb:
>
> Ulf,
>
> sun.text.normalizer might not be a good example, it is "independent
> third-party" package we licensed. Usually
> we try our best to keep this kind of package as-is, for maintenance
> reason, you don't want to make the trivial
> update allover the places again and again whenever there is a version
> upgrade. Yes, I did make some modification
> when dropped in the package when I was the owner years ago, that was
> for functionally better fit with the rest
> of the code and mostly for better performance (especially faster
> startup). You should be able to "smell" that is
> part of the icu4j, so it has its own utf16 handling:-)
>
> sherman
>
>
> Ulf Zibis wrote:
>>
>> Looking deeper in the classes you can find many use scenarios.
>> E.g. class sun.text.normalizer.UTF16 more ore less is a duplicate of
>> sun.nio.cs.Surrogate.
>>
>>>
>>> I'm not saying we can NOT add isBMP() (I know icu4j's UCharacter
>>> class does have one), just
>>> believe it's arguably not necessary.
>>>
>>> Same as the pair
>>>
>>> -- static char highSurrogate(int codePoint);
>>
>> --> sun.text.normalizer.UTF16.getLeadSurrogate(int char32)
>>
>>> -- static char lowSurrogate(int codePoint);
>>
>> --> sun.text.normalizer.UTF16.getTrailSurrogate(int char32)
>>
>>> -- CharBuffer.putCodePoint(int)
>>
>> Maybe it would be better to add appendCodePoint(int cp) to
>> Charsequence, of something similar.
>> See: sun.text.normalizer.UTF16.append(StringBuffer target, int char32)
>> See: java.text.CharacterIterator.CodePointIterator
>>
>>
>>
>> -Ulf
>>
>>
>
>
From jonathan.gibbons at sun.com Tue Sep 8 11:20:35 2009
From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com)
Date: Tue, 08 Sep 2009 18:20:35 +0000
Subject: hg: jdk7/tl/langtools: 6419701: DefaultFileManager clean up:
URI.create; ...
Message-ID: <20090908182040.7DCC1E24D@hg.openjdk.java.net>
Changeset: 35e29f51a7c3
Author: jjg
Date: 2009-09-08 11:12 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/35e29f51a7c3
6419701: DefaultFileManager clean up: URI.create
6483788: DefaultFileManager.ZipFileObject.toUri() fails to escape space characters
6501502: JSR 199: FileObject.toUri should return file:///c:/ or file:/c:/ not file://c:/
6877206: JavaFileObject.toUri returns bogus URI (win)
6877223: tests @ignored because of issues with File.toURI on Windows
Reviewed-by: mcimadamore, alanb
! src/share/classes/com/sun/tools/javac/file/BaseFileObject.java
! src/share/classes/com/sun/tools/javac/file/JavacFileManager.java
! src/share/classes/com/sun/tools/javac/file/RegularFileObject.java
! src/share/classes/com/sun/tools/javac/file/SymbolArchive.java
! src/share/classes/com/sun/tools/javac/file/ZipArchive.java
! src/share/classes/com/sun/tools/javac/file/ZipFileIndexArchive.java
! src/share/classes/com/sun/tools/javac/jvm/ClassReader.java
! src/share/classes/com/sun/tools/javadoc/ClassDocImpl.java
! test/tools/javac/Diagnostics/6769027/tester.properties
! test/tools/javac/api/6440333/T6440333.java
! test/tools/javac/api/Sibling.java
+ test/tools/javac/api/T6483788.java
+ test/tools/javac/api/T6501502.java
+ test/tools/javac/api/T6877206.java
From jonathan.gibbons at sun.com Tue Sep 8 11:38:59 2009
From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com)
Date: Tue, 08 Sep 2009 18:38:59 +0000
Subject: hg: jdk7/tl/langtools: 6879346: files have Windows newlines
Message-ID: <20090908183904.4E176E254@hg.openjdk.java.net>
Changeset: dd98acd9f717
Author: jjg
Date: 2009-09-08 11:29 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/dd98acd9f717
6879346: files have Windows newlines
Reviewed-by: darcy
! src/share/classes/com/sun/tools/doclets/internal/toolkit/resources/doclet.xml
! test/com/sun/javadoc/testCRLineSeparator/TestCRLineSeparator.java
! test/com/sun/javadoc/testCRLineSeparator/pkg/MyClass.java
! test/com/sun/javadoc/testHref/package-list
! test/com/sun/javadoc/testLinkOption/testNewLineInLink/package.html
! test/com/sun/javadoc/testNoPackagesFile/TestNoPackagesFile.java
! test/com/sun/javadoc/testOverridenMethods/TestMultiInheritence.java
! test/com/sun/javadoc/testRelativeLinks/pkg/package.html
! test/com/sun/javadoc/testTaglets/TestTaglets.java
! test/com/sun/javadoc/testWarnings/pkg/package.html
! test/tools/javah/SubClassConsts.win
From jonathan.gibbons at sun.com Tue Sep 8 11:52:20 2009
From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com)
Date: Tue, 08 Sep 2009 18:52:20 +0000
Subject: hg: jdk7/tl/langtools: 6879371: javap does not close internal default
file manager
Message-ID: <20090908185225.9D5B1E259@hg.openjdk.java.net>
Changeset: 261c54b2312e
Author: jjg
Date: 2009-09-08 11:43 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/261c54b2312e
6879371: javap does not close internal default file manager
Reviewed-by: darcy
! src/share/classes/com/sun/tools/javap/JavapTask.java
+ test/tools/javap/T6879371.java
From mandy.chung at sun.com Tue Sep 8 13:12:35 2009
From: mandy.chung at sun.com (mandy.chung at sun.com)
Date: Tue, 08 Sep 2009 20:12:35 +0000
Subject: hg: jdk7/tl/jdk: 7 new changesets
Message-ID: <20090908201437.B63FAE2D9@hg.openjdk.java.net>
Changeset: fdf11ce72e8e
Author: mchung
Date: 2009-08-06 11:25 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/fdf11ce72e8e
4917309: (cl) Reduce internal usage of ClassNotFoundExceptions during class-loading
Summary: Change findBootstrapClass to return null instead of throwing CNFE if class not found
Reviewed-by: alanb, dholmes, iris
! src/share/classes/java/lang/ClassLoader.java
! src/share/javavm/export/jvm.h
! src/share/native/java/lang/ClassLoader.c
Changeset: 3323e6c925f9
Author: mchung
Date: 2009-08-06 16:35 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/3323e6c925f9
6864028: Update the java launcher to use the new entry point JVM_FindClassFromBootLoader
Summary: Update the java launcher to use the new entry point JVM_FindClassFromBootLoader
Reviewed-by: ksrini
! src/share/bin/java.h
! src/solaris/bin/java_md.c
! src/windows/bin/java_md.c
Changeset: 1f1c824e6244
Author: mchung
Date: 2009-08-24 10:33 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/1f1c824e6244
Merge
- src/share/classes/com/sun/crypto/provider/JarVerifier.java
- src/share/classes/javax/swing/plaf/basic/DesktopIconMover.java
- src/share/classes/sun/nio/ch/AbstractFuture.java
- src/share/classes/sun/security/pkcs11/JarVerifier.java
- src/windows/classes/sun/security/mscapi/JarVerifier.java
Changeset: 799731b1cd03
Author: mchung
Date: 2009-08-27 12:58 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/799731b1cd03
Merge
- src/share/native/java/util/zip/zlib-1.1.3/ChangeLog
- src/share/native/java/util/zip/zlib-1.1.3/README
- src/share/native/java/util/zip/zlib-1.1.3/compress.c
- src/share/native/java/util/zip/zlib-1.1.3/deflate.c
- src/share/native/java/util/zip/zlib-1.1.3/deflate.h
- src/share/native/java/util/zip/zlib-1.1.3/doc/algorithm.doc
- src/share/native/java/util/zip/zlib-1.1.3/example.c
- src/share/native/java/util/zip/zlib-1.1.3/gzio.c
- src/share/native/java/util/zip/zlib-1.1.3/infblock.c
- src/share/native/java/util/zip/zlib-1.1.3/infblock.h
- src/share/native/java/util/zip/zlib-1.1.3/infcodes.c
- src/share/native/java/util/zip/zlib-1.1.3/infcodes.h
- src/share/native/java/util/zip/zlib-1.1.3/inffast.c
- src/share/native/java/util/zip/zlib-1.1.3/inffast.h
- src/share/native/java/util/zip/zlib-1.1.3/inffixed.h
- src/share/native/java/util/zip/zlib-1.1.3/inflate.c
- src/share/native/java/util/zip/zlib-1.1.3/inftrees.c
- src/share/native/java/util/zip/zlib-1.1.3/inftrees.h
- src/share/native/java/util/zip/zlib-1.1.3/infutil.c
- src/share/native/java/util/zip/zlib-1.1.3/infutil.h
- src/share/native/java/util/zip/zlib-1.1.3/minigzip.c
- src/share/native/java/util/zip/zlib-1.1.3/trees.c
- src/share/native/java/util/zip/zlib-1.1.3/trees.h
- src/share/native/java/util/zip/zlib-1.1.3/uncompr.c
- src/share/native/java/util/zip/zlib-1.1.3/zadler32.c
- src/share/native/java/util/zip/zlib-1.1.3/zconf.h
- src/share/native/java/util/zip/zlib-1.1.3/zcrc32.c
- src/share/native/java/util/zip/zlib-1.1.3/zlib.h
- src/share/native/java/util/zip/zlib-1.1.3/zutil.c
- src/share/native/java/util/zip/zlib-1.1.3/zutil.h
- test/java/util/concurrent/LinkedBlockingQueue/LastElement.java
- test/java/util/concurrent/LinkedBlockingQueue/OfferRemoveLoops.java
Changeset: dfb5cf81d8cd
Author: mchung
Date: 2009-09-03 16:09 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/dfb5cf81d8cd
Merge
Changeset: b9b7f56bdfa3
Author: mchung
Date: 2009-09-04 15:44 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/b9b7f56bdfa3
Merge
Changeset: 466915134131
Author: mchung
Date: 2009-09-08 12:59 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/466915134131
Merge
From jonathan.gibbons at sun.com Tue Sep 8 14:07:28 2009
From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com)
Date: Tue, 08 Sep 2009 21:07:28 +0000
Subject: hg: jdk7/tl/langtools: 5093723: REGRESSION: ClassCastException in
SingleIndexWriter
Message-ID: <20090908210735.49896E2E4@hg.openjdk.java.net>
Changeset: ebb6ad5a95bb
Author: jjg
Date: 2009-09-08 13:53 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/ebb6ad5a95bb
5093723: REGRESSION: ClassCastException in SingleIndexWriter
Reviewed-by: jjg
Contributed-by: ahe at google.com
! src/share/classes/com/sun/tools/doclets/formats/html/HtmlDocletWriter.java
! src/share/classes/com/sun/tools/doclets/formats/html/resources/standard.properties
+ test/com/sun/javadoc/5093723/DocumentedClass.java
+ test/com/sun/javadoc/5093723/T5093723.java
+ test/com/sun/javadoc/5093723/UndocumentedClass.java
From jonathan.gibbons at sun.com Tue Sep 8 14:19:13 2009
From: jonathan.gibbons at sun.com (jonathan.gibbons at sun.com)
Date: Tue, 08 Sep 2009 21:19:13 +0000
Subject: hg: jdk7/tl/langtools: 6709246: ClassCastException in javadoc
Message-ID: <20090908211923.6C716E2EB@hg.openjdk.java.net>
Changeset: 071a4e36cd87
Author: jjg
Date: 2009-09-08 14:08 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/langtools/rev/071a4e36cd87
6709246: ClassCastException in javadoc
Reviewed-by: jjg
Contributed-by: ahe at google.com
! src/share/classes/com/sun/tools/javadoc/AnnotationDescImpl.java
+ test/tools/javadoc/annotations/missing/Main.java
+ test/tools/javadoc/annotations/missing/somepackage/MissingAnnotationClass.java
From martinrb at google.com Tue Sep 8 16:39:09 2009
From: martinrb at google.com (martinrb at google.com)
Date: Tue, 08 Sep 2009 23:39:09 +0000
Subject: hg: jdk7/tl/jdk: 6850958: Honor -XX:OnOutOfMemoryError when array
size exceeds VM limit
Message-ID: <20090908233956.9A942E308@hg.openjdk.java.net>
Changeset: 0d50d40a4a39
Author: martin
Date: 2009-09-08 14:33 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/0d50d40a4a39
6850958: Honor -XX:OnOutOfMemoryError when array size exceeds VM limit
Summary: Test hotspot/jvmti fix 6850957 using ProcessBuilder test infrastructure
Reviewed-by: tbell, dholmes, alanb, ysr
! test/java/lang/ProcessBuilder/Basic.java
From martinrb at google.com Tue Sep 8 18:52:40 2009
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 8 Sep 2009 18:52:40 -0700
Subject: review request for 6798511/6860431: Include functionality of
Surrogate in Character -> use scenarios
In-Reply-To: <4AA56297.2040501@gmx.de>
References: <4A95079A.8080803@gmx.de> <4A9578C4.8060801@sun.com>
<4AA5040C.8050701@gmx.de> <4AA5508F.1090307@sun.com>
<4AA56297.2040501@gmx.de>
Message-ID: <1ccfd1c10909081852t1b31e4ccv20121c625591373e@mail.gmail.com>
I wish there had been more collaboration between the
jdk developers (including myself) and the icu
developers in the past. There has certainly been
a fair amount of duplication of effort in the
character encoding area.
>>> Maybe it would be better to add appendCodePoint(int cp) to Charsequence,
>>> of something similar.
You only get one chance to get an interface right, unfortunately.
Martin
From Ulf.Zibis at gmx.de Wed Sep 9 04:21:53 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Wed, 09 Sep 2009 13:21:53 +0200
Subject: review request for 6798511/6860431: Include functionality of
Surrogate in Character -> use scenarios
In-Reply-To: <1ccfd1c10909081852t1b31e4ccv20121c625591373e@mail.gmail.com>
References: <4A95079A.8080803@gmx.de> <4A9578C4.8060801@sun.com>
<4AA5040C.8050701@gmx.de> <4AA5508F.1090307@sun.com>
<4AA56297.2040501@gmx.de>
<1ccfd1c10909081852t1b31e4ccv20121c625591373e@mail.gmail.com>
Message-ID: <4AA78FD1.1020509@gmx.de>
Am 09.09.2009 03:52, Martin Buchholz schrieb:
> I wish there had been more collaboration between the
> jdk developers (including myself) and the icu
> developers in the past. There has certainly been
> a fair amount of duplication of effort in the
> character encoding area.
>
>
>>>> Maybe it would be better to add appendCodePoint(int cp) to Charsequence,
>>>> of something similar.
>>>>
>
> You only get one chance to get an interface right, unfortunately.
>
Shame on me, I temporarily forgot this. :-[
-Ulf
From sean.mullan at sun.com Wed Sep 9 09:52:58 2009
From: sean.mullan at sun.com (sean.mullan at sun.com)
Date: Wed, 09 Sep 2009 16:52:58 +0000
Subject: hg: jdk7/tl/jdk: 2 new changesets
Message-ID: <20090909165352.6752BE36B@hg.openjdk.java.net>
Changeset: 8252729d51a3
Author: mullan
Date: 2009-09-09 09:54 -0400
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/8252729d51a3
6745437: Add option to only check revocation of end-entity certificate in a chain of certificates
6869739: Cannot check revocation of single certificate without validating the entire chain
Reviewed-by: xuelei
+ src/share/classes/sun/security/action/GetBooleanSecurityPropertyAction.java
! src/share/classes/sun/security/provider/certpath/Builder.java
! src/share/classes/sun/security/provider/certpath/CertId.java
! src/share/classes/sun/security/provider/certpath/CrlRevocationChecker.java
! src/share/classes/sun/security/provider/certpath/DistributionPointFetcher.java
! src/share/classes/sun/security/provider/certpath/ForwardBuilder.java
+ src/share/classes/sun/security/provider/certpath/OCSP.java
! src/share/classes/sun/security/provider/certpath/OCSPChecker.java
! src/share/classes/sun/security/provider/certpath/OCSPRequest.java
! src/share/classes/sun/security/provider/certpath/OCSPResponse.java
! src/share/classes/sun/security/provider/certpath/PKIXCertPathValidator.java
! src/share/classes/sun/security/provider/certpath/SunCertPathBuilder.java
! src/share/classes/sun/security/x509/AccessDescription.java
Changeset: 7b85ef3d752e
Author: mullan
Date: 2009-09-09 09:59 -0400
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/7b85ef3d752e
Merge
From Xueming.Shen at Sun.COM Wed Sep 9 11:54:35 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Wed, 09 Sep 2009 11:54:35 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
<1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
Message-ID: <4AA7F9EB.5090200@sun.com>
Martin, Brandon, Alan
I still believe the best choice at this moment is to be a little more
conservative, given the DOS.flush() has been
working in this behavior for decade. The proposed methods are good
enough to serve the functionality required,
I don't think it is worth breaking the compatibility in this case simply
because "it would be better...". We definitely
can consider to change the position should the feedback show this is not
the case.
I will start the CCC if you guys are OK with the latest wording. The doc
has been changed "slightly" compared to
last version, thanks for the comment from all of you.
http://cr.openjdk.java.net/~sherman/zipflush/webrev
Sherman
From blong at google.com Wed Sep 9 12:34:39 2009
From: blong at google.com (Brandon Long)
Date: Wed, 9 Sep 2009 12:34:39 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA7F9EB.5090200@sun.com>
References: <1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
<1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
<4AA7F9EB.5090200@sun.com>
Message-ID: <20090909193439.GA23862@google.com>
I really don't understand this. I think just about everyone would
expect that flush on an output stream means send all the data I've
written. There are a large number of people who have stumbled on this
bug and several others, and the fact that it wasn't fixed when it was
discovered 10 years ago is justification for not fixing it now?
I imagine a lot of people will stumble on this in the future, instead of
it just working as expected.
This is up against some unknown fraction of people who are for some
reason happy with the current broken behavior, and would dislike paying
a 10-50% penalty in compression because their code calls flush when they
don't actually need to?
At the very least, this misfeature needs to be called out in the
documentation for the DeflaterOutputStream class.
Brandon
On 09/09/09 Xueming Shen uttered the following other thing:
> Martin, Brandon, Alan
>
> I still believe the best choice at this moment is to be a little more
> conservative, given the DOS.flush() has been
> working in this behavior for decade. The proposed methods are good
> enough to serve the functionality required,
> I don't think it is worth breaking the compatibility in this case simply
> because "it would be better...". We definitely
> can consider to change the position should the feedback show this is not
> the case.
>
> I will start the CCC if you guys are OK with the latest wording. The doc
> has been changed "slightly" compared to
> last version, thanks for the comment from all of you.
>
> http://cr.openjdk.java.net/~sherman/zipflush/webrev
>
> Sherman
--
"I need to either get a new hard drive or clean my home dir."
-- Mark Notarus
http://www.fiction.net/blong/
From Joe.Darcy at Sun.COM Wed Sep 9 13:40:48 2009
From: Joe.Darcy at Sun.COM (Joe Darcy)
Date: Wed, 09 Sep 2009 13:40:48 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
Message-ID: <4AA812D0.309@sun.com>
Hello.
For JDK 7, I think it is high-time the platform included a class like
java.util.Objects to hold commonly-written utility methods. For
example, a two-argument static equals method that returned true if both
arguments are null, returns false is one argument is null, and otherwise
returns the result of calling arg1.equals(arg2) (6797535 "Add shared
two argument static equals method to the platform").
A static hashCode method returning 0 for null and the value of
arg.hashCode() has also been suggested.
A set of
static int compareTo(int, int)
static int compareTo(long, long)
....
methods probably belongs somewhere in the platform too.
What other utility methods would have broad enough use and applicability
to go into a common java.util class?
-Joe
From Alan.Bateman at Sun.COM Wed Sep 9 13:57:17 2009
From: Alan.Bateman at Sun.COM (Alan Bateman)
Date: Wed, 09 Sep 2009 21:57:17 +0100
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA7F9EB.5090200@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
<1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
<4AA7F9EB.5090200@sun.com>
Message-ID: <4AA816AD.7090306@sun.com>
Xueming Shen wrote:
> Martin, Brandon, Alan
>
> I still believe the best choice at this moment is to be a little more
> conservative, given the DOS.flush() has been
> working in this behavior for decade. The proposed methods are good
> enough to serve the functionality required,
> I don't think it is worth breaking the compatibility in this case
> simply because "it would be better...". We definitely
> can consider to change the position should the feedback show this is
> not the case.
>
> I will start the CCC if you guys are OK with the latest wording. The
> doc has been changed "slightly" compared to
> last version, thanks for the comment from all of you.
>
> http://cr.openjdk.java.net/~sherman/zipflush/webrev
>
> Sherman
The Deflater proposal feels right and fits well with the existing API.
The DeflaterOutputStream.flush(int) proposal is workable but it will
clearly be a hard sell to get developers to extend DeflaterOutputStream
and override the no-arg flush method (which they will need to do to get
this to work with existing code/libraries). How you considered
overriding the flush() method to use Z_SYNC_FLUSH but have some way to
restore existing behavior in the event that it causes severe performance
or other issues for someone migrating an existing application to jdk7? I
hate suggesting yet another system property to get us out of jail. Aside
from an application calling flush a lot, are there any other behavioral
implications (say for GZIPOutputStream?).
-Alan.
From gnu_andrew at member.fsf.org Wed Sep 9 14:22:04 2009
From: gnu_andrew at member.fsf.org (Andrew John Hughes)
Date: Wed, 9 Sep 2009 22:22:04 +0100
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA812D0.309@sun.com>
References: <4AA812D0.309@sun.com>
Message-ID: <17c6771e0909091422n57ed252bh4749571a07d2dd41@mail.gmail.com>
2009/9/9 Joe Darcy :
> Hello.
>
> For JDK 7, I think it is high-time the platform included a class like
> java.util.Objects to hold commonly-written utility methods. ?For example, a
> two-argument static equals method that returned true if both arguments are
> null, returns false is one argument is null, and otherwise returns the
> result of calling arg1.equals(arg2) ?(6797535 "Add shared two argument
> static equals method to the platform").
>
> A static hashCode method returning 0 for null and the value of
> arg.hashCode() has also been suggested.
>
> A set of
>
> ? static int compareTo(int, int)
> ? static int compareTo(long, long)
> ? ....
>
> methods probably belongs somewhere in the platform too.
>
> What other utility methods would have broad enough use and applicability to
> go into a common java.util class?
>
> -Joe
>
Given you've listed utility methods for two Object methods, equals and
hashCode, toString seems an obvious one to handle as well:
public static String toString(Object o)
throws IllegalAccessException
{
Class c = o.getClass();
StringBuilder b = new StringBuilder(c.getName());
b.append("[");
for (Field f : c.getDeclaredFields())
{
f.setAccessible(true);
b.append(f.getName() + "=" + f.get());
b.append(",");
}
b.replace(b.length() - 1, b.length(), "]");
return b.toString();
}
Maybe there's also a useful utility implementation of clone too, but I
can't think of one offhand.
--
Andrew :-)
Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)
Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net
PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
From Xueming.Shen at Sun.COM Wed Sep 9 14:19:30 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Wed, 09 Sep 2009 14:19:30 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <20090909193439.GA23862@google.com>
References: <1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
<1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
<4AA7F9EB.5090200@sun.com> <20090909193439.GA23862@google.com>
Message-ID: <4AA81BE2.5080702@sun.com>
I sometime too feels the same frustration when get kicked by the
compatibility, especially if you have to
back out hundreds of lines of code just because it is "incompatible". I
guess someone said it right a while
ago, the compatibility sucks, but yes, MOST end user/developer like it.
I disagree with the statement that "we are not fixing it". As I said in
my last email, the proposed change does
address the issue and provide a reasonable solution to the problem that
can't not be solved with the existing
APIs. My understanding of the key issue here is that the existing
DeflaterOutputStream and Deflater do not
provide the method(s) to sync_flush the underlying deflater, which is a
must for some use scenario. I believe
most developers should be happy as long as such methods are provided, it
is NOT necessary to be the flush().
Yes, the justification for me to go with the conservative approach here
is exactly the "behavior has been in this
way for 10 years". Let me share my consideration here again.
For existing running application, the DOS.flush() has not been working
the way someone might expect for
decade, so I don't think there is any application is sitting tightly and
patiently over there for years to just
wait for us to "fix" the DOS.flush() behavior and their app will then
magically work as "expected". They
either go with the ugly workaround or go use jzlib.
For developer that are waiting this to be fix so that they can simply
use the default zlib included in java in their
application, the proposed solution just works fine, use the flush(int)
when needed, they have to explicitly invoke
A flush method anyway, why it can not be the flush(int) (or say, why it
must be the flush()).
The only benefit of having a sync_flush flush() in DOS is that you don't
have to override the default flush() when
you have to pass the DOS around and "flush()" will be invoked
"indirectly" by other part/layer of the code. The
price to pay for this "convenience" is the incompatibility. The
applications that works for years might suddenly
experience the "regression" when migrated to JDK7. We DONT know if there
are really such applications there
and how important they are until we hit it real world. At this point,
the merit of doing this seems not worth it.
I can see other benefit of doing this. If we change the behavior of
DOS.flush() in JDK7 and do not see lots
of "regression" complain over there, it will be then easy to backport
part of the change (means the DOS.flush()
only) into 6 releases.
I agree it might be a better to have some thing in the DOS api to
explain the situation, how about to have
a note like below to add into the flush(int) API doc.
Note: The inherited flush() method only flushes the output stream
without flushing the compressor(), it
should be overridden to do flush(Deflater.SYNC_FLUSH) or
flush(Deflate.FULL_FLUSH) if desirable.
sherman
Brandon Long wrote:
> I really don't understand this. I think just about everyone would
> expect that flush on an output stream means send all the data I've
> written. There are a large number of people who have stumbled on this
> bug and several others, and the fact that it wasn't fixed when it was
> discovered 10 years ago is justification for not fixing it now?
>
> I imagine a lot of people will stumble on this in the future, instead of
> it just working as expected.
>
> This is up against some unknown fraction of people who are for some
> reason happy with the current broken behavior, and would dislike paying
> a 10-50% penalty in compression because their code calls flush when they
> don't actually need to?
>
> At the very least, this misfeature needs to be called out in the
> documentation for the DeflaterOutputStream class.
>
> Brandon
>
> On 09/09/09 Xueming Shen uttered the following other thing:
>
>> Martin, Brandon, Alan
>>
>> I still believe the best choice at this moment is to be a little more
>> conservative, given the DOS.flush() has been
>> working in this behavior for decade. The proposed methods are good
>> enough to serve the functionality required,
>> I don't think it is worth breaking the compatibility in this case simply
>> because "it would be better...". We definitely
>> can consider to change the position should the feedback show this is not
>> the case.
>>
>> I will start the CCC if you guys are OK with the latest wording. The doc
>> has been changed "slightly" compared to
>> last version, thanks for the comment from all of you.
>>
>> http://cr.openjdk.java.net/~sherman/zipflush/webrev
>>
>> Sherman
>>
>
>
From martinrb at google.com Wed Sep 9 14:30:04 2009
From: martinrb at google.com (Martin Buchholz)
Date: Wed, 9 Sep 2009 14:30:04 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <20090909193439.GA23862@google.com>
References: <1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
<1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
<4AA7F9EB.5090200@sun.com> <20090909193439.GA23862@google.com>
Message-ID: <1ccfd1c10909091430r1812d1b8l73365683ad1c98d5@mail.gmail.com>
JDK has often chosen compatibility over doing something Right.
I think the risk of performance degradation due to frequent calls to
flush() is higher than one might think at first. Calls to flush may be
made on one's behalf by some middleware between the user
and the DOS. E.g. someone might wrap an auto-flushing PrintStream
around a DOS, which causes surprisingly many calls to flush
(Although see java puzzler "Char'ed beyond recognition")
Venerable System.out is an example of an auto-flushing PrintStream.
As I said previously, I am a bit on the fence with this issue myself,
but given Xueming's reluctance, let's agree to do things his way.
Users (like Google) who want this functionality *now* will need
to develop a private version of this change in any case for use with
java 6.
Martin
On Wed, Sep 9, 2009 at 12:34, Brandon Long wrote:
> I really don't understand this. ?I think just about everyone would
> expect that flush on an output stream means send all the data I've
> written. ?There are a large number of people who have stumbled on this
> bug and several others, and the fact that it wasn't fixed when it was
> discovered 10 years ago is justification for not fixing it now?
>
> I imagine a lot of people will stumble on this in the future, instead of
> it just working as expected.
>
> This is up against some unknown fraction of people who are for some
> reason happy with the current broken behavior, and would dislike paying
> a 10-50% penalty in compression because their code calls flush when they
> don't actually need to?
>
> At the very least, this misfeature needs to be called out in the
> documentation for the DeflaterOutputStream class.
>
> Brandon
>
> On 09/09/09 Xueming Shen uttered the following other thing:
>> Martin, Brandon, Alan
>>
>> I still believe the best choice at this moment is to be a little more
>> conservative, given the DOS.flush() has been
>> working in this behavior for decade. The proposed methods are good
>> enough to serve the functionality required,
>> I don't think it is worth breaking the compatibility in this case simply
>> because "it would be better...". We definitely
>> can consider to change the position should the feedback show this is not
>> the case.
>>
>> I will start the CCC if you guys are OK with the latest wording. The doc
>> has been changed "slightly" compared to
>> last version, thanks for the comment from all of you.
>>
>> http://cr.openjdk.java.net/~sherman/zipflush/webrev
>>
>> Sherman
>
> --
> ?"I need to either get a new hard drive or clean my home dir."
> ? ? ? ?-- Mark Notarus
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? http://www.fiction.net/blong/
>
From lance.andersen at sun.com Wed Sep 9 14:53:09 2009
From: lance.andersen at sun.com (lance.andersen at sun.com)
Date: Wed, 09 Sep 2009 21:53:09 +0000
Subject: hg: jdk7/tl/jdk: 6737212: Fixed javadoc warning messages in RowSet
classes
Message-ID: <20090909215352.F154BE392@hg.openjdk.java.net>
Changeset: f1eb4c28b313
Author: lancea
Date: 2009-09-09 20:15 -0400
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/f1eb4c28b313
6737212: Fixed javadoc warning messages in RowSet classes
Reviewed-by: darcy
! src/share/classes/com/sun/rowset/JdbcRowSetResourceBundle.java
! src/share/classes/com/sun/rowset/JoinRowSetImpl.java
! src/share/classes/com/sun/rowset/internal/WebRowSetXmlReader.java
! src/share/classes/javax/sql/rowset/BaseRowSet.java
From mlists at juma.me.uk Wed Sep 9 15:26:46 2009
From: mlists at juma.me.uk (Ismael Juma)
Date: Wed, 9 Sep 2009 22:26:46 +0000 (UTC)
Subject: What methods should go into a java.util.Objects class in JDK 7?
References: <4AA812D0.309@sun.com>
Message-ID:
Joe Darcy writes:
> A static hashCode method returning 0 for null and the value of
> arg.hashCode() has also been suggested.
>
> A set of
>
> static int compareTo(int, int)
> static int compareTo(long, long)
> ....
>
> methods probably belongs somewhere in the platform too.
Although I think a java.util.Objects is probably a good idea, the compareTo
methods for primitives should probably go to their own primitive class and be
called compare to match the one in Double:
public static int compare(double d1, double d2)
Best,
Ismael
From forax at univ-mlv.fr Wed Sep 9 16:29:18 2009
From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=)
Date: Thu, 10 Sep 2009 01:29:18 +0200
Subject: What methods should go into a java.util.Objects class in JDK
7?
In-Reply-To: <4AA812D0.309@sun.com>
References: <4AA812D0.309@sun.com>
Message-ID: <4AA83A4E.7000901@univ-mlv.fr>
Le 09/09/2009 22:40, Joe Darcy a ?crit :
> Hello.
>
> For JDK 7, I think it is high-time the platform included a class like
> java.util.Objects to hold commonly-written utility methods. For
> example, a two-argument static equals method that returned true if
> both arguments are null, returns false is one argument is null, and
> otherwise returns the result of calling arg1.equals(arg2) (6797535
> "Add shared two argument static equals method to the platform").
>
> A static hashCode method returning 0 for null and the value of
> arg.hashCode() has also been suggested.
>
> A set of
>
> static int compareTo(int, int)
> static int compareTo(long, long)
> ....
>
> methods probably belongs somewhere in the platform too.
>
> What other utility methods would have broad enough use and
> applicability to go into a common java.util class?
>
> -Joe
Some hashCode builder like:
http://commons.apache.org/lang/api/org/apache/commons/lang/builder/HashCodeBuilder.html
(without the methods that use reflection).
It will force all VMs to have a better escape analysis :)
R?mi
From kevinb at google.com Wed Sep 9 16:25:57 2009
From: kevinb at google.com (Kevin Bourrillion)
Date: Wed, 9 Sep 2009 16:25:57 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <17c6771e0909091422n57ed252bh4749571a07d2dd41@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<17c6771e0909091422n57ed252bh4749571a07d2dd41@mail.gmail.com>
Message-ID: <108fcdeb0909091625h5cc8c370t5d0a16445d7eb46a@mail.gmail.com>
-1 on reflection-based stuff in the Objects class.
equal() and hashCode() are the only two things we saw fit to include in our
Objects class, and actually, our hashCode() is identical to
Arrays.hashCode() (but is varargs).
http://google-collections.googlecode.com/svn/trunk/javadoc/com/google/common/base/Objects.html
Please do add the compare() methods to all the rest of the wrapper types; we
would use those a lot.
When it comes to cloning, I tried very hard once to provide something useful
-- a "clone any object without knowing its concrete type" method -- and it
was a disaster. A total mess of reflection, and no one wanted to use it
anyway. Cloning in Java is discredited.
So I'm left with only equal() for your new Objects class. By the way, the
best implementation is a == b || (a != null && a.equals(b)).
On Wed, Sep 9, 2009 at 2:22 PM, Andrew John Hughes <
gnu_andrew at member.fsf.org> wrote:
> 2009/9/9 Joe Darcy :
> > Hello.
> >
> > For JDK 7, I think it is high-time the platform included a class like
> > java.util.Objects to hold commonly-written utility methods. For example,
> a
> > two-argument static equals method that returned true if both arguments
> are
> > null, returns false is one argument is null, and otherwise returns the
> > result of calling arg1.equals(arg2) (6797535 "Add shared two argument
> > static equals method to the platform").
> >
> > A static hashCode method returning 0 for null and the value of
> > arg.hashCode() has also been suggested.
> >
> > A set of
> >
> > static int compareTo(int, int)
> > static int compareTo(long, long)
> > ....
> >
> > methods probably belongs somewhere in the platform too.
> >
> > What other utility methods would have broad enough use and applicability
> to
> > go into a common java.util class?
> >
> > -Joe
> >
>
> Given you've listed utility methods for two Object methods, equals and
> hashCode, toString seems an obvious one to handle as well:
>
> public static String toString(Object o)
> throws IllegalAccessException
> {
> Class c = o.getClass();
> StringBuilder b = new StringBuilder(c.getName());
> b.append("[");
> for (Field f : c.getDeclaredFields())
> {
> f.setAccessible(true);
> b.append(f.getName() + "=" + f.get());
> b.append(",");
> }
> b.replace(b.length() - 1, b.length(), "]");
> return b.toString();
> }
>
> Maybe there's also a useful utility implementation of clone too, but I
> can't think of one offhand.
> --
> Andrew :-)
>
> Free Java Software Engineer
> Red Hat, Inc. (http://www.redhat.com)
>
> Support Free Java!
> Contribute to GNU Classpath and the OpenJDK
> http://www.gnu.org/software/classpath
> http://openjdk.java.net
>
> PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
> Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
>
--
Kevin Bourrillion @ Google
internal: http://go/javalibraries
google-collections.googlecode.com
google-guice.googlecode.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090909/be13273d/attachment.html
From Joe.Darcy at Sun.COM Wed Sep 9 16:36:41 2009
From: Joe.Darcy at Sun.COM (Joe Darcy)
Date: Wed, 09 Sep 2009 16:36:41 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <108fcdeb0909091625h5cc8c370t5d0a16445d7eb46a@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<17c6771e0909091422n57ed252bh4749571a07d2dd41@mail.gmail.com>
<108fcdeb0909091625h5cc8c370t5d0a16445d7eb46a@mail.gmail.com>
Message-ID: <4AA83C09.30700@sun.com>
Kevin Bourrillion wrote:
> -1 on reflection-based stuff in the Objects class.
>
> equal() and hashCode() are the only two things we saw fit to include
> in our Objects class, and actually, our hashCode() is identical to
> Arrays.hashCode() (but is varargs).
>
> http://google-collections.googlecode.com/svn/trunk/javadoc/com/google/common/base/Objects.html
Hmm, java.util.Arrays.{hashCode, deepHashCode} could be changed to be
var-args methods in JDK 7. Possibly the toString methods on Object[]
would benefit from being var-args too.
>
> Please do add the compare() methods to all the rest of the wrapper
> types; we would use those a lot.
I was considering putting all the compare/compareTo methods for
primitives in one class; that way the type of the operands doesn't have
to be expressed in a typically call site.
>
> When it comes to cloning, I tried very hard once to provide something
> useful -- a "clone any object without knowing its concrete type"
> method -- and it was a disaster. A total mess of reflection, and no
> one wanted to use it anyway. Cloning in Java is discredited.
>
> So I'm left with only equal() for your new Objects class. By the way,
> the best implementation is a == b || (a != null && a.equals(b)).
Thanks for the tip,
-Joe
>
>
>
> On Wed, Sep 9, 2009 at 2:22 PM, Andrew John Hughes
> > wrote:
>
> 2009/9/9 Joe Darcy >:
> > Hello.
> >
> > For JDK 7, I think it is high-time the platform included a class
> like
> > java.util.Objects to hold commonly-written utility methods. For
> example, a
> > two-argument static equals method that returned true if both
> arguments are
> > null, returns false is one argument is null, and otherwise
> returns the
> > result of calling arg1.equals(arg2) (6797535 "Add shared two
> argument
> > static equals method to the platform").
> >
> > A static hashCode method returning 0 for null and the value of
> > arg.hashCode() has also been suggested.
> >
> > A set of
> >
> > static int compareTo(int, int)
> > static int compareTo(long, long)
> > ....
> >
> > methods probably belongs somewhere in the platform too.
> >
> > What other utility methods would have broad enough use and
> applicability to
> > go into a common java.util class?
> >
> > -Joe
> >
>
> Given you've listed utility methods for two Object methods, equals and
> hashCode, toString seems an obvious one to handle as well:
>
> public static String toString(Object o)
> throws IllegalAccessException
> {
> Class c = o.getClass();
> StringBuilder b = new StringBuilder(c.getName());
> b.append("[");
> for (Field f : c.getDeclaredFields())
> {
> f.setAccessible(true);
> b.append(f.getName() + "=" + f.get());
> b.append(",");
> }
> b.replace(b.length() - 1, b.length(), "]");
> return b.toString();
> }
>
> Maybe there's also a useful utility implementation of clone too, but I
> can't think of one offhand.
> --
> Andrew :-)
>
> Free Java Software Engineer
> Red Hat, Inc. (http://www.redhat.com)
>
> Support Free Java!
> Contribute to GNU Classpath and the OpenJDK
> http://www.gnu.org/software/classpath
> http://openjdk.java.net
>
> PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
> Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
>
>
>
>
> --
> Kevin Bourrillion @ Google
> internal: http://go/javalibraries
> google-collections.googlecode.com
>
> google-guice.googlecode.com
>
From forax at univ-mlv.fr Wed Sep 9 16:48:44 2009
From: forax at univ-mlv.fr (=?UTF-8?B?UsOpbWkgRm9yYXg=?=)
Date: Thu, 10 Sep 2009 01:48:44 +0200
Subject: What methods should go into a java.util.Objects class in JDK
7?
In-Reply-To: <17c6771e0909091422n57ed252bh4749571a07d2dd41@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<17c6771e0909091422n57ed252bh4749571a07d2dd41@mail.gmail.com>
Message-ID: <4AA83EDC.4090100@univ-mlv.fr>
Le 09/09/2009 23:22, Andrew John Hughes a ?crit :
> Given you've listed utility methods for two Object methods, equals and
> hashCode, toString seems an obvious one to handle as well:
>
> public static String toString(Object o)
> throws IllegalAccessException
> {
> Class c = o.getClass();
> StringBuilder b = new StringBuilder(c.getName());
> b.append("[");
> for (Field f : c.getDeclaredFields())
> {
> f.setAccessible(true);
> b.append(f.getName() + "=" + f.get());
> b.append(",");
> }
> b.replace(b.length() - 1, b.length(), "]");
> return b.toString();
> }
>
> Maybe there's also a useful utility implementation of clone too, but I
> can't think of one offhand.
>
Arghhh,
b.append(f.getName() + "=" + f.get());
shoud be
b.append(f.getName()).append("=").append(f.get(o));
And I think there is a problem if the object has no field.
R?mi
From gnu_andrew at member.fsf.org Wed Sep 9 16:43:57 2009
From: gnu_andrew at member.fsf.org (Andrew John Hughes)
Date: Thu, 10 Sep 2009 00:43:57 +0100
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA83EDC.4090100@univ-mlv.fr>
References: <4AA812D0.309@sun.com>
<17c6771e0909091422n57ed252bh4749571a07d2dd41@mail.gmail.com>
<4AA83EDC.4090100@univ-mlv.fr>
Message-ID: <17c6771e0909091643o1fe399edt4b29b2632f9fe12e@mail.gmail.com>
2009/9/10 R?mi Forax :
> Le 09/09/2009 23:22, Andrew John Hughes a ?crit :
>>
>> Given you've listed utility methods for two Object methods, equals and
>> hashCode, toString seems an obvious one to handle as well:
>>
>> public static String toString(Object o)
>> ? throws IllegalAccessException
>> {
>> ? Class ?c = o.getClass();
>> ? StringBuilder b = new StringBuilder(c.getName());
>> ? b.append("[");
>> ? for (Field f : c.getDeclaredFields())
>> ? {
>> ? ? f.setAccessible(true);
>> ? ? b.append(f.getName() + "=" + f.get());
>> ? ? b.append(",");
>> ? }
>> ? b.replace(b.length() - 1, b.length(), "]");
>> ? return b.toString();
>> }
>>
>> Maybe there's also a useful utility implementation of clone too, but I
>> can't think of one offhand.
>>
>
> Arghhh,
>
> b.append(f.getName() + "=" + f.get());
>
> shoud be
>
> b.append(f.getName()).append("=").append(f.get(o));
>
>
> And I think there is a problem if the object has no field.
>
> R?mi
>
Yeah, it was meant to be just a proof of concept, not bulletproof
efficient code for actual use, but thanks for the code review :)
Gives output something like: Utils[bing=5,boom=boom,isDone=false]
I end up writing so many toString methods that are return
getClass().getName() + "[field1=" + field + ",field2=" + field2 + "]";
that it would make things easier. Not as efficient as implementing it
yourself, agreed, but I find I usually call these methods only when
debugging anyway.
--
Andrew :-)
Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)
Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net
PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
From forax at univ-mlv.fr Wed Sep 9 17:04:08 2009
From: forax at univ-mlv.fr (=?UTF-8?B?UsOpbWkgRm9yYXg=?=)
Date: Thu, 10 Sep 2009 02:04:08 +0200
Subject: What methods should go into a java.util.Objects class in JDK
7?
In-Reply-To: <17c6771e0909091643o1fe399edt4b29b2632f9fe12e@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<17c6771e0909091422n57ed252bh4749571a07d2dd41@mail.gmail.com>
<4AA83EDC.4090100@univ-mlv.fr>
<17c6771e0909091643o1fe399edt4b29b2632f9fe12e@mail.gmail.com>
Message-ID: <4AA84278.7060403@univ-mlv.fr>
Le 10/09/2009 01:43, Andrew John Hughes a ?crit :
> 2009/9/10 R?mi Forax:
>
>> Le 09/09/2009 23:22, Andrew John Hughes a ?crit :
>>
>>> Given you've listed utility methods for two Object methods, equals and
>>> hashCode, toString seems an obvious one to handle as well:
>>>
>>> public static String toString(Object o)
>>> throws IllegalAccessException
>>> {
>>> Class c = o.getClass();
>>> StringBuilder b = new StringBuilder(c.getName());
>>> b.append("[");
>>> for (Field f : c.getDeclaredFields())
>>> {
>>> f.setAccessible(true);
>>> b.append(f.getName() + "=" + f.get());
>>> b.append(",");
>>> }
>>> b.replace(b.length() - 1, b.length(), "]");
>>> return b.toString();
>>> }
>>>
>>> Maybe there's also a useful utility implementation of clone too, but I
>>> can't think of one offhand.
>>>
>>>
>> Arghhh,
>>
>> b.append(f.getName() + "=" + f.get());
>>
>> shoud be
>>
>> b.append(f.getName()).append("=").append(f.get(o));
>>
>>
>> And I think there is a problem if the object has no field.
>>
>> R?mi
>>
>>
> Yeah, it was meant to be just a proof of concept, not bulletproof
> efficient code for actual use, but thanks for the code review :)
> Gives output something like: Utils[bing=5,boom=boom,isDone=false]
> I end up writing so many toString methods that are return
> getClass().getName() + "[field1=" + field + ",field2=" + field2 + "]";
> that it would make things easier. Not as efficient as implementing it
> yourself, agreed, but I find I usually call these methods only when
> debugging anyway.
>
I agree about the debugging purpose, that why I don't agree
with Kevin when he said that java.util.Objects should not use
reflection.
Perhaps the method can be renamed to: toDebugString.
R?mi
PS: the code above also forget inherited fields.
From scolebourne at joda.org Wed Sep 9 16:54:33 2009
From: scolebourne at joda.org (Stephen Colebourne)
Date: Thu, 10 Sep 2009 00:54:33 +0100
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA812D0.309@sun.com>
References: <4AA812D0.309@sun.com>
Message-ID: <4b4f45e00909091654t7891127bh24cc1df9c38bec7d@mail.gmail.com>
2009/9/9 Joe Darcy :
> What other utility methods would have broad enough use and applicability to
> go into a common java.util class?
Well, my first selection is here
http://www.jroller.com/scolebourne/entry/jdk_7_method_suggestions
To keep the discussion here valid too, I'll paste it below:
Stephen
JDK 7 utility methods
There is lots of prior art in the field of general utility method
libraries for Java. Perhaps the best known is Apache Commons Lang.
Lets choose some methods (other than string utilities) that might be
appropriate for the JDK from ObjectUtils.
1) Null-safe equals check.
boolean Objects.equals(Object object1, Object object2)
Returns true if both are null, or both are non-null and equal.
2) Null-safe hash code.
int Objects.hashCode(Object object)
Returns the hash code of the object, or zero if the object is null.
3) Null-safe toString.
String Objects.toString(Object object)
Returns the toString of the object, or "" if the object is null.
4) Null-safe toString with specified default.
String Objects.toString(Object object, String defaultStr)
Returns the toString of the object, or defaultStr if the object is null.
5) Null-safe object defaulting.
T Objects.defaultNull(T object, T defaultValue)
Returns the object, or defaultValue if the object is null.
(Of course, the Elvis operator is a much clearer approach for this
common case...)
6) Get maximum and minimum.
T Objects.max(T comparable1, T comparable2)
T Objects.min(T comparable1, T comparable2)
Returns the maximum/minimum object, returning the non-null object if
either is null, or null if both are null.
Generics allow the return type to check the input is also a Comparable.
These would appear to be the obvious methods to add to an Objects utility class.
The following are missing methods on Integer/Long:
7) Compare two primitives.
int Byte.compare(byte value1, byte value2)
int Short.compare(short value1, short value2)
int Integer.compare(int value1, int value2)
int Long.compare(long value1, long value2)
Safely returns the comparison (-1/0/1) indicator for two primitives.
(These methods already exist on Float/Double).
The methods from SystemUtils can be very useful in reducing arbitrary
strings in code:
8) Get common environment variables.
File System.getJavaIoTempDir()
File System.getJavaHomeDir()
File System.getUserHomeDir()
File System.getUserDir()
Returns the java environment variables. There are lots more in
SystemUtils that could be added.
Some more null handling methods for primitives - again, Elvis would be
a better solution:
9) Convert wrapper to primitive avoiding NPE.
boolean Boolean.booleanValue(Boolean obj, boolean defaultValue)
char Character.charValue(Character obj, char defaultValue)
byte Byte.byteValue(Byte obj)
byte Byte.byteValue(Byte obj, byte defaultValue)
short Short.shortValue(Short obj)
short Short.shortValue(Short obj, short defaultValue)
int Integer.intValue(Integer obj)
int Integer.intValue(Integer obj, int defaultValue)
long Long.longValue(Long obj)
long Long.longValue(Long obj, long defaultValue)
float Float.floatValue(Float obj)
float Float.floatValue(Float obj, float defaultValue)
double Double.doubleValue(Double obj)
double Double.doubleValue(Double obj, double defaultValue)
Safe ways to convert a wrapper to a primitive.
The numeric ones return zero if the default argument isn't specified.
There is also a case for methods to convert arrays of wrappers to
arrays of primitives.
These should probably be on a Booleans utility class as per BooleanUtils.
10) Boolean methods for clearer code.
boolean Booleans.isTrue(Boolean booleanObj)
boolean Booleans.isFalse(Boolean booleanObj)
Return true as per the method name, false if null.
boolean Booleans.isNotTrue(Boolean booleanObj)
boolean Booleans.isNotFalse(Boolean booleanObj)
Return true as per the method name, true if null.
11) Negate handling null.
Boolean Booleans.negate(Boolean booleanObj)
TRUE returns FALSE, FALSE returns TRUE, null returns null.
11) Boolean arithmetic.
boolean Booleans.and(boolean[] array)
boolean Booleans.or(boolean[] array)
boolean Booleans.xor(boolean[] array)
boolean Booleans.and(Boolean[] array)
boolean Booleans.or(Boolean[] array)
boolean Booleans.xor(Boolean[] array)
Performs the stated binary maths.
12) Character comparison.
boolean Character.equalsIgnoreCase(char ch1, char ch2)
Compares two characters ignoring case.
These should be constants for empty arrays on common classes:
13) Empty array constants.
Boolean[] Boolean.EMPTY_ARRAY
boolean[] Boolean.EMPTY_PRIMITIVE_ARRAY
Character[] Character.EMPTY_ARRAY
char[] Character.EMPTY_PRIMITIVE_ARRAY
Byte[] Byte.EMPTY_ARRAY
byte[] Byte.EMPTY_PRIMITIVE_ARRAY
Short[] Short.EMPTY_ARRAY
short[] Short.EMPTY_PRIMITIVE_ARRAY
Integer[] Integer.EMPTY_ARRAY
int[] Integer.EMPTY_PRIMITIVE_ARRAY
Long[] Long.EMPTY_ARRAY
long[] Long.EMPTY_PRIMITIVE_ARRAY
Float[] Float.EMPTY_ARRAY
float[] Float.EMPTY_PRIMITIVE_ARRAY
Double[] Double.EMPTY_ARRAY
double[] Double.EMPTY_PRIMITIVE_ARRAY
String[] String.EMPTY_ARRAY
Class[] Class.EMPTY_ARRAY
Object[] Objects.EMPTY_OBJECT_ARRAY
Return true as per the method name, true if null.
Or a better solution might be a method on Class:
T[] Class.emptyArray();
This allows code like:
Boolean.class.emptyArray();
14) Array size.
boolean Arrays.isEmpty(Object[] array)
int Arrays.size(Object[] array)
Where a null array is empty/size-zero.
15) Collection size.
boolean Collections.isEmpty(Collection coll)
int Collections.size(Collection coll)
boolean Collections.isEmpty(Map map)
int Collections.size(Map map)
Where a null collection is empty/size-zero.
Or, even better, add a new interface Sized with a single method to get
the size. This would be retrofitted to Collection, Map, String and
arrays (of course, technically, thats a language change...).
And some Class utilities for less NPEs when outputting the class name
in debugging:
16) NPE safe way to get class name.
String Class.getName(Class cls)
String Class.getSimpleName(Class cls)
String Class.getPackageName(Class cls)
Returns the class name, or null if the input is null.
Again, the null-safe operators would avoid this kind of specific method.
Locale could do with some love, as per LocaleUtils:
17) Parse a locale string.
Locale parse(String localeStr)
Parses the locale string to a locale.
16) Country/Language lists.
List Locale.countriesByLanguage(String langaugeStr)
List Locale.languagesByCountry(String countryStr)
Extracts just countries or languages.
A better solution would be two new classes Country and Language. These
would be useful in other places in the JDK, where a locale is used and
it should be a country or language.
And the big one that I've missed - safe maths, as per Joda-Time FieldUtils:
18) Add/subtract/multiply/cast safely.
int Math.safeToInt(long value)
int Math.safeNegate(int value)
long Math.safeNegate(long value)
int Math.safeAdd(int value1, int value2)
long Math.safeAdd(long value1, int value2)
long Math.safeAdd(long value1, long value2)
int Math.safeSubtract(int value1, int value2)
long Math.safeSubtract(long value1, int value2)
long Math.safeSubtract(long value1, long value2)
int Math.safeMultiply(int value1, int value2)
long Math.safeMultiply(long value1, int value2)
long Math.safeMultiply(long value1, long value2)
These perform the specified mathematical operation, but throw
ArithmeticException, rather than failing quietly, if overflow occurs.
I should also note that there are a whole heap of BigDecimal,
BigInteger and String utilities that could be added.
From martinrb at google.com Wed Sep 9 17:04:10 2009
From: martinrb at google.com (Martin Buchholz)
Date: Wed, 9 Sep 2009 17:04:10 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4b4f45e00909091654t7891127bh24cc1df9c38bec7d@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<4b4f45e00909091654t7891127bh24cc1df9c38bec7d@mail.gmail.com>
Message-ID: <1ccfd1c10909091704i1f4f75dfm4ae1988d0903cd99@mail.gmail.com>
Google would like to contribute implementations of the methods below.
Here's a webrev:
http://cr.openjdk.java.net/~martin/webrevs/openjdk7/compare/
We have tests (not yet included) but they would need to be jtreg-ified.
Is there any junit support yet for jtreg?
Martin
On Wed, Sep 9, 2009 at 16:54, Stephen Colebourne wrote:
> 2009/9/9 Joe Darcy :
> The following are missing methods on Integer/Long:
>
> 7) Compare two primitives.
> int Byte.compare(byte value1, byte value2)
> int Short.compare(short value1, short value2)
> int Integer.compare(int value1, int value2)
> int Long.compare(long value1, long value2)
> Safely returns the comparison (-1/0/1) indicator for two primitives.
> (These methods already exist on Float/Double).
From martinrb at google.com Wed Sep 9 17:15:48 2009
From: martinrb at google.com (Martin Buchholz)
Date: Wed, 9 Sep 2009 17:15:48 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA812D0.309@sun.com>
References: <4AA812D0.309@sun.com>
Message-ID: <1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
Here's a bunch of static hashCode methods for primitives:
http://cr.openjdk.java.net/~martin/webrevs/openjdk7/hashCode/
This doesn't include a
public static int hashCode(object x) { return x == null ? 0 : x.hashCode(); }
that I think was also being asked for, and which might be a candidate
for Objects.
Martin
From Joe.Darcy at Sun.COM Wed Sep 9 17:34:57 2009
From: Joe.Darcy at Sun.COM (Joe Darcy)
Date: Wed, 09 Sep 2009 17:34:57 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <1ccfd1c10909091704i1f4f75dfm4ae1988d0903cd99@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<4b4f45e00909091654t7891127bh24cc1df9c38bec7d@mail.gmail.com>
<1ccfd1c10909091704i1f4f75dfm4ae1988d0903cd99@mail.gmail.com>
Message-ID: <4AA849B1.1090005@sun.com>
Martin Buchholz wrote:
> Google would like to contribute implementations of the methods below.
> Here's a webrev:
>
> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/compare/
>
Wherever these compare methods go, into the N wrapper classes or into
java.util.Compare, I think the instance method compareTo should be
rewritten in terms of the new static compare. In particular,
422 public int compareTo(Short anotherShort) {
423 return this.value - anotherShort.value;
424 }
should get replaced with
422 public int compareTo(Short anotherShort) {
423 return compare(this.value, anotherShort.value);
424 }
This consolidates the comparison logic in a single method per type and
reuses the existing implementation tests.
(Later in JDK 7, a set of "unsignedCompare" methods are possible
additions to the platform too.)
> We have tests (not yet included) but they would need to be jtreg-ified.
> Is there any junit support yet for jtreg?
>
No.
Once the home of the new methods is determined, I'll file a ccc request
so these changes can go back after any additional testing is settled.
Thanks,
-Joe
> Martin
>
> On Wed, Sep 9, 2009 at 16:54, Stephen Colebourne wrote:
>
>> 2009/9/9 Joe Darcy :
>> The following are missing methods on Integer/Long:
>>
>> 7) Compare two primitives.
>> int Byte.compare(byte value1, byte value2)
>> int Short.compare(short value1, short value2)
>> int Integer.compare(int value1, int value2)
>> int Long.compare(long value1, long value2)
>> Safely returns the comparison (-1/0/1) indicator for two primitives.
>> (These methods already exist on Float/Double).
>>
From martinrb at google.com Wed Sep 9 17:35:10 2009
From: martinrb at google.com (Martin Buchholz)
Date: Wed, 9 Sep 2009 17:35:10 -0700
Subject: Race condition in Class.getName()
Message-ID: <1ccfd1c10909091735ndf28f91j1813c26203a60967@mail.gmail.com>
It is believed that the java memory model allows Class.getName()
to return null.
This is one of those methods with an intentional data race.
Probably this has not been seen in practice because only
a perverse or adversarial runtime would load the "name" field
twice, out-of-order, as it seems to be allowed to.
diff --git a/src/share/classes/java/lang/Class.java
b/src/share/classes/java/lang/Class.java
--- a/src/share/classes/java/lang/Class.java
+++ b/src/share/classes/java/lang/Class.java
@@ -565,8 +565,9 @@
* represented by this object.
*/
public String getName() {
+ String name = this.name;
if (name == null)
- name = getName0();
+ this.name = name = getName0();
return name;
}
Martin
From Joe.Darcy at Sun.COM Wed Sep 9 17:40:10 2009
From: Joe.Darcy at Sun.COM (Joe Darcy)
Date: Wed, 09 Sep 2009 17:40:10 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
Message-ID: <4AA84AEA.7050008@sun.com>
Martin Buchholz wrote:
> Here's a bunch of static hashCode methods for primitives:
>
> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/hashCode/
>
Hmm, I'd be more inclined to group these methods into Objects as part of
a "universal hash" functionality.
> This doesn't include a
>
> public static int hashCode(object x) { return x == null ? 0 : x.hashCode(); }
>
> that I think was also being asked for, and which might be a candidate
> for Objects.
Yes, I was thinking of that method for Objects.
-Joe
PS This talk of hashing remind me of a small spec cleanup:
4245470 algorithm of java.lang.Byte.hashCode() is not specified
http://bugs.sun.com/view_bug.do?bug_id=4245470
From forax at univ-mlv.fr Wed Sep 9 17:54:13 2009
From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=)
Date: Thu, 10 Sep 2009 02:54:13 +0200
Subject: What methods should go into a java.util.Objects class in JDK
7?
In-Reply-To: <1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
Message-ID: <4AA84E35.4060207@univ-mlv.fr>
Le 10/09/2009 02:15, Martin Buchholz a ?crit :
> Here's a bunch of static hashCode methods for primitives:
>
> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/hashCode/
>
> This doesn't include a
>
> public static int hashCode(object x) { return x == null ? 0 : x.hashCode(); }
>
> that I think was also being asked for, and which might be a candidate
> for Objects.
>
> Martin
>
Martin,
You forget java.lang.Boolean in your patch set :)
R?mi
From martinrb at google.com Wed Sep 9 18:08:48 2009
From: martinrb at google.com (Martin Buchholz)
Date: Wed, 9 Sep 2009 18:08:48 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA849B1.1090005@sun.com>
References: <4AA812D0.309@sun.com>
<4b4f45e00909091654t7891127bh24cc1df9c38bec7d@mail.gmail.com>
<1ccfd1c10909091704i1f4f75dfm4ae1988d0903cd99@mail.gmail.com>
<4AA849B1.1090005@sun.com>
Message-ID: <1ccfd1c10909091808s772d454epfe7a1057ccb9ca9f@mail.gmail.com>
On Wed, Sep 9, 2009 at 17:34, Joe Darcy wrote:
> Martin Buchholz wrote:
>>
>> Google would like to contribute implementations of the methods below.
>> Here's a webrev:
>>
>> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/compare/
>>
>
> Wherever these compare methods go, into the N wrapper classes or into
> java.util.Compare,
Given the precedent of Double.compare(double,double) and
Float.compare(float,float), I think the compare methods need
to go into the primitive wrapper classes.
I think the instance method compareTo should be rewritten
> in terms of the new static compare. ?In particular,
>
> 422 ? ? public int compareTo(Short anotherShort) {
> 423 ? ? ? ? return this.value - anotherShort.value;
> 424 ? ? }
>
> should get replaced with
>
> 422 ? ? public int compareTo(Short anotherShort) {
> 423 ? ? ? ? return compare(this.value, anotherShort.value);
> 424 ? ? }
As you wish.
Webrev updated.
http://cr.openjdk.java.net/~martin/webrevs/openjdk7/compare/
> This consolidates the comparison logic in a single method per type and
> reuses the existing implementation tests.
> (Later in JDK 7, a set of "unsignedCompare" methods are possible additions
> to the platform too.)
>
>> We have tests (not yet included) but they would need to be jtreg-ified.
>> Is there any junit support yet for jtreg?
>>
>
> No.
>
> Once the home of the new methods is determined, I'll file a ccc request so
> these changes can go back after any additional testing is settled.
There's not a whole lot of testing work here, but we will provide it if
it CCC is approved.
Martin
From Joe.Darcy at Sun.COM Wed Sep 9 18:34:32 2009
From: Joe.Darcy at Sun.COM (Joe Darcy)
Date: Wed, 09 Sep 2009 18:34:32 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <1ccfd1c10909091808s772d454epfe7a1057ccb9ca9f@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<4b4f45e00909091654t7891127bh24cc1df9c38bec7d@mail.gmail.com>
<1ccfd1c10909091704i1f4f75dfm4ae1988d0903cd99@mail.gmail.com>
<4AA849B1.1090005@sun.com>
<1ccfd1c10909091808s772d454epfe7a1057ccb9ca9f@mail.gmail.com>
Message-ID: <4AA857A8.1010700@sun.com>
Martin Buchholz wrote:
> On Wed, Sep 9, 2009 at 17:34, Joe Darcy wrote:
>
>> Martin Buchholz wrote:
>>
>>> Google would like to contribute implementations of the methods below.
>>> Here's a webrev:
>>>
>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/compare/
>>>
>>>
>> Wherever these compare methods go, into the N wrapper classes or into
>> java.util.Compare,
>>
>
> Given the precedent of Double.compare(double,double) and
> Float.compare(float,float), I think the compare methods need
> to go into the primitive wrapper classes.
>
> I think the instance method compareTo should be rewritten
>
>> in terms of the new static compare. In particular,
>>
>> 422 public int compareTo(Short anotherShort) {
>> 423 return this.value - anotherShort.value;
>> 424 }
>>
>> should get replaced with
>>
>> 422 public int compareTo(Short anotherShort) {
>> 423 return compare(this.value, anotherShort.value);
>> 424 }
>>
>
> As you wish.
>
> Webrev updated.
> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/compare/
>
I approve these changes in an engineering sense; they are a fix for bug
6582946 Add suite of compare(T, T) methods for ints, longs etc
>> This consolidates the comparison logic in a single method per type and
>> reuses the existing implementation tests.
>>
>
>
>
>> (Later in JDK 7, a set of "unsignedCompare" methods are possible additions
>> to the platform too.)
>>
>>
>>> We have tests (not yet included) but they would need to be jtreg-ified.
>>> Is there any junit support yet for jtreg?
>>>
>>>
>> No.
>>
>> Once the home of the new methods is determined, I'll file a ccc request so
>> these changes can go back after any additional testing is settled.
>>
>
> There's not a whole lot of testing work here, but we will provide it if
> it CCC is approved.
>
>
I've created a ccc for this and will let you know when it is approved.
-Joe
From martinrb at google.com Wed Sep 9 19:11:28 2009
From: martinrb at google.com (Martin Buchholz)
Date: Wed, 9 Sep 2009 19:11:28 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA84E35.4060207@univ-mlv.fr>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
<4AA84E35.4060207@univ-mlv.fr>
Message-ID: <1ccfd1c10909091911l67ffaf66xf0bb3733f4071224@mail.gmail.com>
On Wed, Sep 9, 2009 at 17:54, R?mi Forax wrote:
> Le 10/09/2009 02:15, Martin Buchholz a ?crit :
>>
>> Here's a bunch of static hashCode methods for primitives:
>>
>> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/hashCode/
>>
>> This doesn't include a
>>
>> public static int hashCode(object x) { return x == null ? 0 :
>> x.hashCode(); }
>>
>> that I think was also being asked for, and which might be a candidate
>> for Objects.
>>
>> Martin
>>
>
> Martin,
> You forget java.lang.Boolean in your patch set :)
R?mi, merci beaucoup.
Les Booleans, je les avaient oubli?e.
Mes patchs pour les classes wrapper,
tous les deux, maintenant incluent Boolean.
http://cr.openjdk.java.net/~martin/webrevs/openjdk7/hashCode/
http://cr.openjdk.java.net/~martin/webrevs/openjdk7/compare/
Martin
> R?mi
>
From martinrb at google.com Wed Sep 9 19:28:15 2009
From: martinrb at google.com (Martin Buchholz)
Date: Wed, 9 Sep 2009 19:28:15 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA84AEA.7050008@sun.com>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
<4AA84AEA.7050008@sun.com>
Message-ID: <1ccfd1c10909091928o2797106bqf433af4bc6adb294@mail.gmail.com>
I've updated the hashCode patch to include Booleans
and similar refactoring as in compare.
We could do worse than to do hashCode in a totally
analogous fashion to compare, but I could also see
them incorporated into a static HashCodes class.
If you put one into Objects, then the primitive hashCode
methods wouldn't belong in there, defeating your aim
of keeping them together.
Probably we want to have hash code combining methods.
Sigh... nothing is ever easy.
Martin
On Wed, Sep 9, 2009 at 17:40, Joe Darcy wrote:
> Martin Buchholz wrote:
>>
>> Here's a bunch of static hashCode methods for primitives:
>>
>> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/hashCode/
>>
>
> Hmm, I'd be more inclined to group these methods into Objects as part of a
> "universal hash" functionality.
>
>> This doesn't include a
>>
>> public static int hashCode(object x) { return x == null ? 0 :
>> x.hashCode(); }
>>
>> that I think was also being asked for, and which might be a candidate
>> for Objects.
>
> Yes, I was thinking of that method for Objects.
>
> -Joe
>
> PS This talk of hashing remind me of a small spec cleanup:
> 4245470 algorithm of java.lang.Byte.hashCode() is not specified
> http://bugs.sun.com/view_bug.do?bug_id=4245470
>
From martinrb at google.com Wed Sep 9 19:42:29 2009
From: martinrb at google.com (Martin Buchholz)
Date: Wed, 9 Sep 2009 19:42:29 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA84AEA.7050008@sun.com>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
<4AA84AEA.7050008@sun.com>
Message-ID: <1ccfd1c10909091942p41683f0bl46d901126c61e8d5@mail.gmail.com>
On Wed, Sep 9, 2009 at 17:40, Joe Darcy wrote:
>
> PS This talk of hashing remind me of a small spec cleanup:
> 4245470 algorithm of java.lang.Byte.hashCode() is not specified
> http://bugs.sun.com/view_bug.do?bug_id=4245470
>
http://cr.openjdk.java.net/~martin/webrevs/openjdk7/hashCode-spec/
Martin
From Joe.Darcy at Sun.COM Wed Sep 9 20:01:37 2009
From: Joe.Darcy at Sun.COM (Joe Darcy)
Date: Wed, 09 Sep 2009 20:01:37 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <1ccfd1c10909091942p41683f0bl46d901126c61e8d5@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
<4AA84AEA.7050008@sun.com>
<1ccfd1c10909091942p41683f0bl46d901126c61e8d5@mail.gmail.com>
Message-ID: <4AA86C11.7080409@sun.com>
Martin Buchholz wrote:
> On Wed, Sep 9, 2009 at 17:40, Joe Darcy wrote:
>
>> PS This talk of hashing remind me of a small spec cleanup:
>> 4245470 algorithm of java.lang.Byte.hashCode() is not specified
>> http://bugs.sun.com/view_bug.do?bug_id=4245470
>>
>>
>
> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/hashCode-spec/
>
> Martin
>
I'll work on the archeology to verify this behavior has been consistent
over time and thus is reasonable to tighten the spec in these cases.
-Joe
From reinier at zwitserloot.com Wed Sep 9 20:45:09 2009
From: reinier at zwitserloot.com (Reinier Zwitserloot)
Date: Thu, 10 Sep 2009 05:45:09 +0200
Subject: What methods should go into a java.util.Objects class in JDK 7?
Message-ID: <505CEA25-1E45-40BC-BFF9-EC0959E6EC05@zwitserloot.com>
Late to the thread, so many one-liners to various ideas uttered on the
list:
--
This is great stuff folks. As long as I get my String.join (or
Collections.join, or Arrays.join, or Whatever.join, as long as there's
a join!), of course :)
--
The compareTo methods for Integer/Short/Byte/Long (Character?) should
clearly go in the respective wrapper classes for consistency's sake,
but there's nothing in the way of ALSO making a utilities class of
sorts that contains all the primitive compareTo methods as well, if
that seems useful.
--
A trick I'm reverting to rather a lot of late is to find my own jar
file by getting a URL to my own class file as a resource, resolved
against my own class, and then parsing this URL for the jar bit of it.
(MyClassName.class.getResource("MyClassName.class")). There are a
bunch of caveats here, mostly revolving around the notion that you may
not have been loaded as a jar file, but as a directory, or via the
network, or from a database, or who knows how? - but that's what
returning "null" is for, surely. Something to consider, though I'm not
entirely convinced this hack should be made legitimate by turning it
into a library method. I doubt it's a good fit for Objects, in any case.
--
Whatever hashCode() implementation is decided on, it should probably
mirror the way IDEs generate it when you ask them to. Specifically:
Use deepHashCode for object arrays. Yes, this means you'll get
StackOverflows if an array contains itself. This is either (A) the
correct action, or (B) ArrayList is broken, as it has the same
behaviour.
Split up longs via XOR, and doubles via toLongBits -> XOR. There's
loads of alternative algorithms that purport to be more efficient, but
without specific domain knowledge, xor is hard to beat.
Project Lombok sticks to this plan, which is mostly derived from
Effective Java, IIRC. Same goes for equals and toString.
--
How does Joe Average Java Programmer tell the difference between
Objects.equals(), which simply adds null friendliness, and a
reflection-based pseudo-automagical hashCode builder?
--
Changing Arrays.deepEquals and friends to varargs has the same problem
as Arrays.asList for 'list literals': What do list literals and null-
friendly equals/hashCode/toString implements have to do with arrays?
Nothing. There's a reason google went out of their way to dress up
"ImmutableList.of(a, b, c);" as a library, after all. Even without the
Collections.unmodifiableList() wrapper call, Arrays.asList() is
unwieldly and a pretty bad implementation leakage - you're trying to
build an immutable list; the fact that you're doing so by varargsing
your arguments into an array and turning THAT into a list is not
something you should be confronted with when reading code. I'd much
prefer a solution that did not involve java.util.Arrays.
--
toString() is not supposed to be something you ever rely upon, and I
furthermore have a rather hard time imagining anybody can possibly be
relying on the random-esque default toString() implementation, which
combines a memory pointer with a class name. So, would it be an option
to change toString()'s implementation to do the sane thing?
Arrays.toString (deepToString? That can cause StackOverflows...) for
arrays, and some reflection-based magic otherwise? Changing equals()
and hashCode() is not so simply modified without causing serious
backwards compatibility issues, unfortunately.
--Reinier Zwitserloot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090910/fc701e1c/attachment.html
From mikro2nd at gmail.com Thu Sep 10 01:24:16 2009
From: mikro2nd at gmail.com (Mike Morris)
Date: Thu, 10 Sep 2009 10:24:16 +0200
Subject: What methods should go into a java.util.Objects class in JDK
7?
In-Reply-To: <4AA812D0.309@sun.com>
References: <4AA812D0.309@sun.com>
Message-ID: <4AA8B7B0.5070009@gmail.com>
Joe Darcy wrote:
> For JDK 7, I think it is high-time the platform included a class like
> java.util.Objects to hold commonly-written utility methods. For
...
> What other utility methods would have broad enough use and applicability
> to go into a common java.util class?
Ah! Let's just go back to coding in C, then. I cannot see any
justification for everything and the kitchen-sink being glommed into the
Java APIs. If people want these sort of static methods, they're
pretty-much all available in other libraries (Apache Commons, etc.)
Must they /really/ be added to the morass that the Java APIs have become?
(Sorry: As a non-contributor I probably don't have the mana to make such
a comment, but this proposal really punches a lot of my buttons.)
--
mike morris :: mikro2nd (at) gmail (dot) com
http://mikro2nd.net/
http://mikro2nd.net/blog/planb/
http://mikro2nd.net/blog/mike/
This email is [X]bloggable [ ]ask-first [ ]private
From Ulf.Zibis at gmx.de Thu Sep 10 01:56:28 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Thu, 10 Sep 2009 10:56:28 +0200
Subject: What methods should go into a java.util.Objects class in JDK
7?
In-Reply-To: <4AA84278.7060403@univ-mlv.fr>
References: <4AA812D0.309@sun.com> <17c6771e0909091422n57ed252bh4749571a07d2dd41@mail.gmail.com> <4AA83EDC.4090100@univ-mlv.fr> <17c6771e0909091643o1fe399edt4b29b2632f9fe12e@mail.gmail.com>
<4AA84278.7060403@univ-mlv.fr>
Message-ID: <4AA8BF3C.7080601@gmx.de>
Am 10.09.2009 02:04, R?mi Forax schrieb:
> Le 10/09/2009 01:43, Andrew John Hughes a ?crit :
> I agree about the debugging purpose, that why I don't agree
> with Kevin when he said that java.util.Objects should not use
> reflection.
>
> Perhaps the method can be renamed to: toDebugString.
>
> R?mi
> PS: the code above also forget inherited fields.
>
Maybe another class: java.util.Debug / DebugHelpers / DebugUtils
I guess, there are many ideas around to fill such a class with useful
things.
-Ulf
From Ulf.Zibis at gmx.de Thu Sep 10 02:43:45 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Thu, 10 Sep 2009 11:43:45 +0200
Subject: What methods should go into a java.util.Objects class in JDK
7?
In-Reply-To: <4b4f45e00909091654t7891127bh24cc1df9c38bec7d@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<4b4f45e00909091654t7891127bh24cc1df9c38bec7d@mail.gmail.com>
Message-ID: <4AA8CA51.8000705@gmx.de>
Am 10.09.2009 01:54, Stephen Colebourne schrieb:
>
> There is also a case for methods to convert arrays of wrappers to
> arrays of primitives.
>
+1
- I guess, you mean also vice versa
- Maybe would become superfluous, if HotSpot internally would compile to
primitive arrays. (just a raw thought)
19) re-usability of java.nio.Buffer (saves multiple instantiation):
- X-Buffer#clear(x[] buf); clears all pointers
- X-Buffer#reset(x[] buf); only resets pos to mark (maybe limit to array
size), allows to process different data in same/similar/synchronous manner.
-Ulf
From Ulf.Zibis at gmx.de Thu Sep 10 02:45:24 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Thu, 10 Sep 2009 11:45:24 +0200
Subject: What methods should go into a java.util.Objects class in JDK
7?
In-Reply-To: <1ccfd1c10909091704i1f4f75dfm4ae1988d0903cd99@mail.gmail.com>
References: <4AA812D0.309@sun.com> <4b4f45e00909091654t7891127bh24cc1df9c38bec7d@mail.gmail.com>
<1ccfd1c10909091704i1f4f75dfm4ae1988d0903cd99@mail.gmail.com>
Message-ID: <4AA8CAB4.5060403@gmx.de>
Am 10.09.2009 02:04, Martin Buchholz schrieb:
> Is there any junit support yet for jtreg?
>
+1 to have this.
-Ulf
From Ulf.Zibis at gmx.de Thu Sep 10 02:58:24 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Thu, 10 Sep 2009 11:58:24 +0200
Subject: What methods should go into a java.util.Objects class in JDK
7?
In-Reply-To: <1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
Message-ID: <4AA8CDC0.4000004@gmx.de>
Am 10.09.2009 02:15, Martin Buchholz schrieb:
> Here's a bunch of static hashCode methods for primitives:
>
> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/hashCode/
>
Can't this be solved by extending auto-boxing capabilities? :
long l = 1234567890;
int hash = l.hashCode();
-Ulf
From Ulf.Zibis at gmx.de Thu Sep 10 04:07:16 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Thu, 10 Sep 2009 13:07:16 +0200
Subject: What methods should go into a java.util.Objects class in JDK
7?
In-Reply-To: <505CEA25-1E45-40BC-BFF9-EC0959E6EC05@zwitserloot.com>
References: <505CEA25-1E45-40BC-BFF9-EC0959E6EC05@zwitserloot.com>
Message-ID: <4AA8DDE4.8050909@gmx.de>
Am 10.09.2009 05:45, Reinier Zwitserloot schrieb:
>
> A trick I'm reverting to rather a lot of late is to find my own jar
> file by getting a URL to my own class file as a resource, resolved
> against my own class, and then parsing this URL for the jar bit of it.
> (MyClassName.class.getResource("MyClassName.class")). There are a
> bunch of caveats here, mostly revolving around the notion that you may
> not have been loaded as a jar file, but as a directory, or via the
> network, or from a database, or who knows how? - but that's what
> returning "null" is for, surely. Something to consider, though I'm not
> entirely convinced this hack should be made legitimate by turning it
> into a library method. I doubt it's a good fit for Objects, in any case.
I guess you mean:
MyClassName.class.getResource("MyClassSimpleName.class")
I would like File(URI uri) to accept jar URI's, so it would be possible
to get a nio channel access to a jared resource.
Or even better:
MyClassName.class.getResourceAsChannel("MyResourceName"))
-Ulf
From gnu_andrew at member.fsf.org Thu Sep 10 06:01:24 2009
From: gnu_andrew at member.fsf.org (Andrew John Hughes)
Date: Thu, 10 Sep 2009 14:01:24 +0100
Subject: PATCH: Tired of waiting for rt.jar to build?
In-Reply-To: <1ccfd1c10904291325p5f3b40e8y36e214e54f82a1a9@mail.gmail.com>
References: <1ccfd1c10904271754s6f56feafub82c97c80109f023@mail.gmail.com>
<1ccfd1c10904291325p5f3b40e8y36e214e54f82a1a9@mail.gmail.com>
Message-ID: <17c6771e0909100601v1779eadaqec6bc7f4071a44fb@mail.gmail.com>
2009/4/29 Martin Buchholz :
> Since writing this, I have learned, to my horror, that the
> behavior of the -C flag differs from the behavior in tar in that
>
> - -C is not sticky - it applies only to the one following argument
>
> - the path is relative to the JDK's current directory, not the
> previous -C directory.
>
> despite assurances from jar(1)
>
> ? ? ? ? ?-C ?dir
> ? ? ? ? ? ? Temporarily changes directories (cd dir) during execution of the
> ? ? ? ? ? ? jar command while processing the following inputfiles argument.
> ? ? ? ? ? ? Its operation is intended to be similar to the -C option of the
> ? ? ? ? ? ? UNIX tar utility.
>
> If you squint, you can see that it says "argument", not "arguments".
>
> Martin
>
> On Mon, Apr 27, 2009 at 17:54, Martin Buchholz wrote:
>> ?I believe the better fix would be
>> to eviscerate the code that handles the "-C" flag and do it right,
>
>
>> Someone who cares about the Makefiles can also try to remove the
>> 16000 gratuitous -C flags that makes jar's life "jar hell".
>>
>> Martin
>>
>
Martin,
Thanks a lot for this patch. We've seen good speedups with it applied.
The thread here reads very strangely; did a number of mails go to a
different mail list or only to private mail addresses?
There's also seems to be no mention of this being applied to OpenJDK6:
http://hg.openjdk.java.net/jdk6/jdk6/jdk/rev/b35f1e5075a4
That changeset seems to have been merged together with several others;
was there a reason the changesets were not imported individually so as
to retain the history?
Thanks,
--
Andrew :-)
Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)
Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net
PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
From gnu_andrew at member.fsf.org Thu Sep 10 06:11:16 2009
From: gnu_andrew at member.fsf.org (Andrew John Hughes)
Date: Thu, 10 Sep 2009 14:11:16 +0100
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA8BF3C.7080601@gmx.de>
References: <4AA812D0.309@sun.com>
<17c6771e0909091422n57ed252bh4749571a07d2dd41@mail.gmail.com>
<4AA83EDC.4090100@univ-mlv.fr>
<17c6771e0909091643o1fe399edt4b29b2632f9fe12e@mail.gmail.com>
<4AA84278.7060403@univ-mlv.fr> <4AA8BF3C.7080601@gmx.de>
Message-ID: <17c6771e0909100611p7af03905v60051d36bf4cf1de@mail.gmail.com>
2009/9/10 Ulf Zibis :
> Am 10.09.2009 02:04, R?mi Forax schrieb:
>>
>> Le 10/09/2009 01:43, Andrew John Hughes a ?crit : ?I agree about the
>> debugging purpose, that why I don't agree
>> with Kevin when he said that java.util.Objects should not use
>> reflection.
>>
>> Perhaps the method can be renamed to: toDebugString.
>>
>> R?mi
>> PS: the code above also forget inherited fields.
>>
> Maybe another class: java.util.Debug / DebugHelpers / DebugUtils
>
> I guess, there are many ideas around to fill such a class with useful
> things.
>
>
> -Ulf
>
>
>
Yeah you are probably right, my suggestion seems to come from a
different avenue than the others here.
It's probably because personally I've never used library method to do
such simple null checks; it might have fixed a few bugs for me along
the road, but I just prefer to check such things locally than add in
the overhead of a method call.
--
Andrew :-)
Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)
Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net
PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
From gnu_andrew at member.fsf.org Thu Sep 10 06:12:10 2009
From: gnu_andrew at member.fsf.org (Andrew John Hughes)
Date: Thu, 10 Sep 2009 14:12:10 +0100
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA8CA51.8000705@gmx.de>
References: <4AA812D0.309@sun.com>
<4b4f45e00909091654t7891127bh24cc1df9c38bec7d@mail.gmail.com>
<4AA8CA51.8000705@gmx.de>
Message-ID: <17c6771e0909100612k83b1168o3c3bb651a7f93ceb@mail.gmail.com>
2009/9/10 Ulf Zibis :
> Am 10.09.2009 01:54, Stephen Colebourne schrieb:
>>
>> There is also a case for methods to convert arrays of wrappers to
>> arrays of primitives.
>>
>
> +1
> - I guess, you mean also vice versa
> - Maybe would become superfluous, if HotSpot internally would compile to
> primitive arrays. (just a raw thought)
>
Except that what's being proposed here results in a spec. change and
will be available regardless of VM. We shouldn't go on relying on
features of certain VMs.
There are other VMs than HotSpot you know! :)
> 19) re-usability of java.nio.Buffer (saves multiple instantiation):
> - X-Buffer#clear(x[] buf); clears all pointers
> - X-Buffer#reset(x[] buf); only resets pos to mark (maybe limit to array
> size), allows to process different data in same/similar/synchronous manner.
>
> -Ulf
>
>
>
>
>
--
Andrew :-)
Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)
Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net
PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
From martinrb at google.com Thu Sep 10 08:01:11 2009
From: martinrb at google.com (Martin Buchholz)
Date: Thu, 10 Sep 2009 08:01:11 -0700
Subject: PATCH: Tired of waiting for rt.jar to build?
In-Reply-To: <17c6771e0909100601v1779eadaqec6bc7f4071a44fb@mail.gmail.com>
References: <1ccfd1c10904271754s6f56feafub82c97c80109f023@mail.gmail.com>
<1ccfd1c10904291325p5f3b40e8y36e214e54f82a1a9@mail.gmail.com>
<17c6771e0909100601v1779eadaqec6bc7f4071a44fb@mail.gmail.com>
Message-ID: <1ccfd1c10909100801m529824a6rb184fb8cb95cd33@mail.gmail.com>
Hi Andrew,
I don't recall all the details, but I probably backported all-at-once
because it was a little easier for me to do so - it reflected the
engineering that was actually done. I care more about
the quality of the openjdk7 mercurial history. In this case
the information *is* available to allow a future maintainer
to do archaeological investigations. I agree it would have
been a little cleaner to backport each change individually.
You also correctly notice that not all discussions end up
on core-libs-dev. In part this is a cultural heritage - tradition
is private, not public, peer review. I have been trying,
and have been encouraging others, to make discussions
more public, even when we might perceive them to be
uninteresting to others. (What can be said about a backport,
except whether it's worth doing, one might think.)
Thanks for the culture change prodding.
Martin
On Thu, Sep 10, 2009 at 06:01, Andrew John
Hughes wrote:
> 2009/4/29 Martin Buchholz :
>> Since writing this, I have learned, to my horror, that the
>> behavior of the -C flag differs from the behavior in tar in that
>>
>> - -C is not sticky - it applies only to the one following argument
>>
>> - the path is relative to the JDK's current directory, not the
>> previous -C directory.
>>
>> despite assurances from jar(1)
>>
>> ? ? ? ? ?-C ?dir
>> ? ? ? ? ? ? Temporarily changes directories (cd dir) during execution of the
>> ? ? ? ? ? ? jar command while processing the following inputfiles argument.
>> ? ? ? ? ? ? Its operation is intended to be similar to the -C option of the
>> ? ? ? ? ? ? UNIX tar utility.
>>
>> If you squint, you can see that it says "argument", not "arguments".
>>
>> Martin
>>
>> On Mon, Apr 27, 2009 at 17:54, Martin Buchholz wrote:
>>> ?I believe the better fix would be
>>> to eviscerate the code that handles the "-C" flag and do it right,
>>
>>
>>> Someone who cares about the Makefiles can also try to remove the
>>> 16000 gratuitous -C flags that makes jar's life "jar hell".
>>>
>>> Martin
>>>
>>
>
> Martin,
>
> Thanks a lot for this patch. ?We've seen good speedups with it applied.
>
> The thread here reads very strangely; did a number of mails go to a
> different mail list or only to private mail addresses?
> There's also seems to be no mention of this being applied to OpenJDK6:
> http://hg.openjdk.java.net/jdk6/jdk6/jdk/rev/b35f1e5075a4
> That changeset seems to have been merged together with several others;
> was there a reason the changesets were not imported individually so as
> to retain the history?
>
> Thanks,
> --
> Andrew :-)
>
> Free Java Software Engineer
> Red Hat, Inc. (http://www.redhat.com)
>
> Support Free Java!
> Contribute to GNU Classpath and the OpenJDK
> http://www.gnu.org/software/classpath
> http://openjdk.java.net
>
> PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
> Fingerprint: F8EF F1EA 401E 2E60 15FA ?7927 142C 2591 94EF D9D8
>
From gnu_andrew at member.fsf.org Thu Sep 10 08:17:01 2009
From: gnu_andrew at member.fsf.org (Andrew John Hughes)
Date: Thu, 10 Sep 2009 16:17:01 +0100
Subject: PATCH: Tired of waiting for rt.jar to build?
In-Reply-To: <1ccfd1c10909100801m529824a6rb184fb8cb95cd33@mail.gmail.com>
References: <1ccfd1c10904271754s6f56feafub82c97c80109f023@mail.gmail.com>
<1ccfd1c10904291325p5f3b40e8y36e214e54f82a1a9@mail.gmail.com>
<17c6771e0909100601v1779eadaqec6bc7f4071a44fb@mail.gmail.com>
<1ccfd1c10909100801m529824a6rb184fb8cb95cd33@mail.gmail.com>
Message-ID: <17c6771e0909100817h3ad22efbjb1c5f92e1dd7642e@mail.gmail.com>
2009/9/10 Martin Buchholz :
> Hi Andrew,
>
> I don't recall all the details, but I probably backported all-at-once
> because it was a little easier for me to do so - it reflected the
> engineering that was actually done. ? I care more about
> the quality of the openjdk7 mercurial history. ?In this case
> the information *is* available to allow a future maintainer
> to do archaeological investigations. ?I agree it would have
> been a little cleaner to backport each change individually.
>
Yeah, the OpenJDK6 version history is already a complete mess, so I
don't suppose it makes that much difference :)
Neither are ideal because everything before b24 is missing from OpenJDK7 too.
The problem was more that this was never mentioned in the thread, so
we didn't pick up on this until Andrew Haley spotted it today. I
remember reading the thread at the time, but because there was no
apparent conclusion publicly, I missed out on it actually being
committed and wasn't just another patch in a big pile of pending
stuff...
> You also correctly notice that not all discussions end up
> on core-libs-dev. ?In part this is a cultural heritage - tradition
> is private, not public, peer review. ?I have been trying,
> and have been encouraging others, to make discussions
> more public, even when we might perceive them to be
> uninteresting to others. ?(What can be said about a backport,
> except whether it's worth doing, one might think.)
>
> Thanks for the culture change prodding.
>
I'm aware of this endemic culture at Sun (see
http://blog.fuseyism.com/index.php/2009/09/08/im-so-tired/), and I
very much appreciate your efforts to do things publicly. Believe me,
it hasn't gone unnoticed, and the prodding was aimed more at the
authors of the missing messages in that thread... :)
FWIW, I'm pretty sure I'm not alone in saying I'd much prefer to
receive these 'uninteresting' mails. There are plenty of mailing
lists and mail filters to allow us to cherry-pick what we want to
read, but we can't recreate missing e-mails.
> Martin
>
> On Thu, Sep 10, 2009 at 06:01, Andrew John
> Hughes wrote:
>> 2009/4/29 Martin Buchholz :
>>> Since writing this, I have learned, to my horror, that the
>>> behavior of the -C flag differs from the behavior in tar in that
>>>
>>> - -C is not sticky - it applies only to the one following argument
>>>
>>> - the path is relative to the JDK's current directory, not the
>>> previous -C directory.
>>>
>>> despite assurances from jar(1)
>>>
>>> ? ? ? ? ?-C ?dir
>>> ? ? ? ? ? ? Temporarily changes directories (cd dir) during execution of the
>>> ? ? ? ? ? ? jar command while processing the following inputfiles argument.
>>> ? ? ? ? ? ? Its operation is intended to be similar to the -C option of the
>>> ? ? ? ? ? ? UNIX tar utility.
>>>
>>> If you squint, you can see that it says "argument", not "arguments".
>>>
>>> Martin
>>>
>>> On Mon, Apr 27, 2009 at 17:54, Martin Buchholz wrote:
>>>> ?I believe the better fix would be
>>>> to eviscerate the code that handles the "-C" flag and do it right,
>>>
>>>
>>>> Someone who cares about the Makefiles can also try to remove the
>>>> 16000 gratuitous -C flags that makes jar's life "jar hell".
>>>>
>>>> Martin
>>>>
>>>
>>
>> Martin,
>>
>> Thanks a lot for this patch. ?We've seen good speedups with it applied.
>>
>> The thread here reads very strangely; did a number of mails go to a
>> different mail list or only to private mail addresses?
>> There's also seems to be no mention of this being applied to OpenJDK6:
>> http://hg.openjdk.java.net/jdk6/jdk6/jdk/rev/b35f1e5075a4
>> That changeset seems to have been merged together with several others;
>> was there a reason the changesets were not imported individually so as
>> to retain the history?
>>
>> Thanks,
>> --
>> Andrew :-)
>>
>> Free Java Software Engineer
>> Red Hat, Inc. (http://www.redhat.com)
>>
>> Support Free Java!
>> Contribute to GNU Classpath and the OpenJDK
>> http://www.gnu.org/software/classpath
>> http://openjdk.java.net
>>
>> PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
>> Fingerprint: F8EF F1EA 401E 2E60 15FA ?7927 142C 2591 94EF D9D8
>>
>
--
Andrew :-)
Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)
Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net
PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
From aph at redhat.com Thu Sep 10 08:25:53 2009
From: aph at redhat.com (Andrew Haley)
Date: Thu, 10 Sep 2009 16:25:53 +0100
Subject: PATCH: Tired of waiting for rt.jar to build?
In-Reply-To: <17c6771e0909100817h3ad22efbjb1c5f92e1dd7642e@mail.gmail.com>
References: <1ccfd1c10904271754s6f56feafub82c97c80109f023@mail.gmail.com> <1ccfd1c10904291325p5f3b40e8y36e214e54f82a1a9@mail.gmail.com> <17c6771e0909100601v1779eadaqec6bc7f4071a44fb@mail.gmail.com> <1ccfd1c10909100801m529824a6rb184fb8cb95cd33@mail.gmail.com>
<17c6771e0909100817h3ad22efbjb1c5f92e1dd7642e@mail.gmail.com>
Message-ID: <4AA91A81.4040908@redhat.com>
Andrew John Hughes wrote:
> FWIW, I'm pretty sure I'm not alone in saying I'd much prefer to
> receive these 'uninteresting' mails.
This particular back-port mail would have been very interesting to
us, since it has a huge impact on build times on some architectures.
Andrew.
From Joe.Darcy at Sun.COM Thu Sep 10 08:58:42 2009
From: Joe.Darcy at Sun.COM (Joseph D. Darcy)
Date: Thu, 10 Sep 2009 08:58:42 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA8B7B0.5070009@gmail.com>
References: <4AA812D0.309@sun.com> <4AA8B7B0.5070009@gmail.com>
Message-ID: <4AA92232.5000107@sun.com>
Mike Morris wrote:
> Joe Darcy wrote:
>
>> For JDK 7, I think it is high-time the platform included a class like
>> java.util.Objects to hold commonly-written utility methods. For
> ...
>> What other utility methods would have broad enough use and
>> applicability to go into a common java.util class?
>
> Ah! Let's just go back to coding in C, then. I cannot see any
> justification for everything and the kitchen-sink being glommed into
> the Java APIs. If people want these sort of static methods, they're
> pretty-much all available in other libraries (Apache Commons, etc.)
The methods being discussed are generaly short and while easy to write,
all of them aren't so easy to write that they cannot have bugs. To me,
the existence of these methods in multiple other libraries argues for
putting robust version of them in the JDK since it is not worthwhile
adding a library dependency just to get a static two argument equals method.
-Joe
From schulz at e-spirit.de Thu Sep 10 00:37:39 2009
From: schulz at e-spirit.de (Schulz, Stefan)
Date: Thu, 10 Sep 2009 09:37:39 +0200
Subject: What methods should go into a java.util.Objects class in JDK 7?
Message-ID: <7D2077BFF677D2429DDDEE095D9A48AC13259F4A@osiris2.e-spirit.de>
> So I'm left with only equal() for your new Objects class. By the way,
> the best implementation is a == b || (a != null && a.equals(b)).
I disagree, as this one does not handle arrays properly.
I'd either like equal to handle deep equality on arrays or have a deepEqual utility method that takes arbitrary objects.
Cheers,
Stefan
From martinrb at google.com Thu Sep 10 09:34:15 2009
From: martinrb at google.com (Martin Buchholz)
Date: Thu, 10 Sep 2009 09:34:15 -0700
Subject: PATCH: Tired of waiting for rt.jar to build?
In-Reply-To: <4AA91A81.4040908@redhat.com>
References: <1ccfd1c10904271754s6f56feafub82c97c80109f023@mail.gmail.com>
<1ccfd1c10904291325p5f3b40e8y36e214e54f82a1a9@mail.gmail.com>
<17c6771e0909100601v1779eadaqec6bc7f4071a44fb@mail.gmail.com>
<1ccfd1c10909100801m529824a6rb184fb8cb95cd33@mail.gmail.com>
<17c6771e0909100817h3ad22efbjb1c5f92e1dd7642e@mail.gmail.com>
<4AA91A81.4040908@redhat.com>
Message-ID: <1ccfd1c10909100934i6c6a6a28v2c461b9a4d3ad499@mail.gmail.com>
On Thu, Sep 10, 2009 at 08:25, Andrew Haley wrote:
> Andrew John Hughes wrote:
>
>> FWIW, I'm pretty sure I'm not alone in saying I'd much prefer to
>> receive these 'uninteresting' mails.
>
> This particular back-port mail would have been very interesting to
> us, since it has a huge impact on build times on some architectures.
The mercurial history of openjdk6 is sadly incomplete,
even when compared to openjdk7.
But going forward, we have hg in place, and
anyone who is managing a downstream release
probably should be examining recent changesets
anyways. Humans are flaky; hg is going to
be a more reliable source of what's gone in.
Martin
From Joe.Darcy at Sun.COM Thu Sep 10 09:53:53 2009
From: Joe.Darcy at Sun.COM (Joseph D. Darcy)
Date: Thu, 10 Sep 2009 09:53:53 -0700
Subject: PATCH: Tired of waiting for rt.jar to build?
In-Reply-To: <1ccfd1c10909100934i6c6a6a28v2c461b9a4d3ad499@mail.gmail.com>
References: <1ccfd1c10904271754s6f56feafub82c97c80109f023@mail.gmail.com>
<1ccfd1c10904291325p5f3b40e8y36e214e54f82a1a9@mail.gmail.com>
<17c6771e0909100601v1779eadaqec6bc7f4071a44fb@mail.gmail.com>
<1ccfd1c10909100801m529824a6rb184fb8cb95cd33@mail.gmail.com>
<17c6771e0909100817h3ad22efbjb1c5f92e1dd7642e@mail.gmail.com>
<4AA91A81.4040908@redhat.com>
<1ccfd1c10909100934i6c6a6a28v2c461b9a4d3ad499@mail.gmail.com>
Message-ID: <4AA92F21.3060305@sun.com>
Martin Buchholz wrote:
> On Thu, Sep 10, 2009 at 08:25, Andrew Haley wrote:
>
>> Andrew John Hughes wrote:
>>
>>
>>> FWIW, I'm pretty sure I'm not alone in saying I'd much prefer to
>>> receive these 'uninteresting' mails.
>>>
>> This particular back-port mail would have been very interesting to
>> us, since it has a huge impact on build times on some architectures.
>>
>
> The mercurial history of openjdk6 is sadly incomplete,
> even when compared to openjdk7.
>
Due to efforts by Kelly, OpenJDK 6 has Mercurial history since it
branched off from OpenJDK 7. Given the original use of teamware, there
is not fully accurate changeset-level reconstruction from the beginning,
but there is at least build-level information for all OpenJDK 6 builds.
-Joe
From Xueming.Shen at Sun.COM Thu Sep 10 11:47:54 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Thu, 10 Sep 2009 11:47:54 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA816AD.7090306@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
<1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
<4AA7F9EB.5090200@sun.com> <4AA816AD.7090306@sun.com>
Message-ID: <4AA949DA.9010707@sun.com>
I think we have enough discussion on this topic, it's time to make a
final decision. Seems like we are all happy on the
changes in Deflater and new DOS.flush(mode), the only difference is
whether or not we should change the existing
behavior of DOS.flush() should use Z_SYNC_FLUSH by default.
Brandon is very firm on the "yes" side, strongly believe this is a MUST.
Martin is on the fence, leaning towards "yes"
Sherman is on the "no" side
So if there is no other vote coming with strong opinion, your vote will
be important:-) Are you also leaning to "yes"
or "strongly" suggesting a yes? I need strong "yes" to change my mind.
The current proposal has the advantage of still keeping the door open to
change the behavior if the new APIs
really can not make the developer to achieve what they want to do or
someone can present a strong case that
the compatibility is really not a concern (we are all assuming there are
some applications over there might be
broken by this).
The latest webrev is at
http://cr.openjdk.java.net/~sherman/zipflush/webrev
I added a note to describe the behavior of DOS.flush(), as Brandon
suggested as his "at the very least".
Sherman
Alan Bateman wrote:
> The Deflater proposal feels right and fits well with the existing API.
> The DeflaterOutputStream.flush(int) proposal is workable but it will
> clearly be a hard sell to get developers to extend
> DeflaterOutputStream and override the no-arg flush method (which they
> will need to do to get this to work with existing code/libraries). How
> you considered overriding the flush() method to use Z_SYNC_FLUSH but
> have some way to restore existing behavior in the event that it causes
> severe performance or other issues for someone migrating an existing
> application to jdk7? I hate suggesting yet another system property to
> get us out of jail. Aside from an application calling flush a lot, are
> there any other behavioral implications (say for GZIPOutputStream?).
>
> -Alan.
>
>
>
>
From blong at google.com Thu Sep 10 12:52:09 2009
From: blong at google.com (Brandon Long)
Date: Thu, 10 Sep 2009 12:52:09 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA949DA.9010707@sun.com>
References: <1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
<1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
<4AA7F9EB.5090200@sun.com> <4AA816AD.7090306@sun.com>
<4AA949DA.9010707@sun.com>
Message-ID: <20090910195209.GA9731@google.com>
The only use case that can't be worked around is if someone is relying
on code which either sub-classes DeflaterOutputStream or uses a
non-injected DeflaterOutputStream, where they can't replace it with a
fixed version. Both of these use cases probably aren't that common,
since if they ever needed a working flush, they would have had to work
around this bug a while back.
Brandon
On 09/10/09 Xueming Shen uttered the following other thing:
> I think we have enough discussion on this topic, it's time to make a
> final decision. Seems like we are all happy on the
> changes in Deflater and new DOS.flush(mode), the only difference is
> whether or not we should change the existing
> behavior of DOS.flush() should use Z_SYNC_FLUSH by default.
>
> Brandon is very firm on the "yes" side, strongly believe this is a MUST.
> Martin is on the fence, leaning towards "yes"
> Sherman is on the "no" side
>
> So if there is no other vote coming with strong opinion, your vote will
> be important:-) Are you also leaning to "yes"
> or "strongly" suggesting a yes? I need strong "yes" to change my mind.
>
> The current proposal has the advantage of still keeping the door open to
> change the behavior if the new APIs
> really can not make the developer to achieve what they want to do or
> someone can present a strong case that
> the compatibility is really not a concern (we are all assuming there are
> some applications over there might be broken by this).
>
> The latest webrev is at
>
> http://cr.openjdk.java.net/~sherman/zipflush/webrev
>
> I added a note to describe the behavior of DOS.flush(), as Brandon
> suggested as his "at the very least".
>
> Sherman
>
> Alan Bateman wrote:
>> The Deflater proposal feels right and fits well with the existing API.
>> The DeflaterOutputStream.flush(int) proposal is workable but it will
>> clearly be a hard sell to get developers to extend
>> DeflaterOutputStream and override the no-arg flush method (which they
>> will need to do to get this to work with existing code/libraries). How
>> you considered overriding the flush() method to use Z_SYNC_FLUSH but
>> have some way to restore existing behavior in the event that it causes
>> severe performance or other issues for someone migrating an existing
>> application to jdk7? I hate suggesting yet another system property to
>> get us out of jail. Aside from an application calling flush a lot, are
>> there any other behavioral implications (say for GZIPOutputStream?).
>>
>> -Alan.
>>
>>
>>
>>
--
"I'd like to take this opportunity to thank all my code reviewers.
Without you ignoring my requests, I couldn't have made the list."
-- Wei-Hwa Huang
From david.lloyd at redhat.com Thu Sep 10 13:58:18 2009
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Thu, 10 Sep 2009 15:58:18 -0500
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA949DA.9010707@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com> <4AA03D0B.1030206@sun.com> <1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com> <4AA051E5.5060605@sun.com> <1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com> <4AA19250.2060803@sun.com>
<20090904225714.GB26286@google.com> <4AA1B890.7000703@sun.com> <1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com> <1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com> <4AA7F9EB.5090200@sun.com>
<4AA816AD.7090306@sun.com> <4AA949DA.9010707@sun.com>
Message-ID: <4AA9686A.4070608@redhat.com>
On 09/10/2009 01:47 PM, Xueming Shen wrote:
> I think we have enough discussion on this topic, it's time to make a
> final decision. Seems like we are all happy on the
> changes in Deflater and new DOS.flush(mode), the only difference is
> whether or not we should change the existing
> behavior of DOS.flush() should use Z_SYNC_FLUSH by default.
The question is, is there really such code out there? It can't really have
any useful semantics that I can see. My irrelevant vote is firmly on "yes"
for this.
- DML
From Xueming.Shen at Sun.COM Thu Sep 10 14:51:55 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Thu, 10 Sep 2009 14:51:55 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA9686A.4070608@redhat.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
<1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
<4AA7F9EB.5090200@sun.com> <4AA816AD.7090306@sun.com>
<4AA949DA.9010707@sun.com> <4AA9686A.4070608@redhat.com>
Message-ID: <4AA974FB.5050201@sun.com>
David M. Lloyd wrote:
> On 09/10/2009 01:47 PM, Xueming Shen wrote:
>> I think we have enough discussion on this topic, it's time to make a
>> final decision. Seems like we are all happy on the
>> changes in Deflater and new DOS.flush(mode), the only difference is
>> whether or not we should change the existing
>> behavior of DOS.flush() should use Z_SYNC_FLUSH by default.
>
> The question is, is there really such code out there? It can't really
> have any useful semantics that I can see. My irrelevant vote is
> firmly on "yes" for this.
>
> - DML
-The question is, is there really such code out there?
I honestly don't know, this is why it is called "compatibility risk",
similar to the investment risk in the stock market, you
lose or gain only after you put the real money in:-) Though my feeling
is it is very "likely" this change is going to break
someone's code given DOS is designed/implemented to be easily used as a
"normal" OutputStream.
-It can't really have any useful semantics that I can see
DOS abstracts two functionalities, a deflater and an output stream, both
the deflater and output stream have their own
"flush" semantics. A flush on the deflater changes the final output,
degrades the compression, basically it changes the
"function" of the deflater. A flush on the OutputStream however is
supposed to push out the "buffered" output (accumulated
for whatever reason, mostly for the i/o performance), it does not change
the final result/output. So arguably it should not be
a too bad idea to also provide two types of "flush" to flush the
deflater and/or to flush the output stream, the existing
DOS.flush() "accidentally" provides the semantics of the outputstream's
flush(), with the help of the newly added flush(mode),
now the developer have the choice of doing whatever "flush" they want to do.
Agree it's arguable that if the stream type of flush is really needed
for DOS in real world, we can debate what should be
the default behavior (the most useful scenario) of the DOS.flush() , if
we are designing/implementing a new DOS class. The
question we need to answer now is should we change the existing default
behavior of DOS.flush(), all existing/running/working
application that uses DOS today is based on (happy with) the
assumption/expectation that it does a "outputstream flush", it
does not change the final compression output. So while we fix the
problem of "can't not flush the deflater of the DOS", do we
have to or say is it worthing to break the expectation of all existing
users. Some time even a "byte" difference of the output result
is a BIG deal.
Every vote counts:-)
sherman
From david.lloyd at redhat.com Thu Sep 10 15:13:19 2009
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Thu, 10 Sep 2009 17:13:19 -0500
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA974FB.5050201@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
<1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
<4AA7F9EB.5090200@sun.com> <4AA816AD.7090306@sun.com>
<4AA949DA.9010707@sun.com> <4AA9686A.4070608@redhat.com>
<4AA974FB.5050201@sun.com>
Message-ID: <4AA979FF.5060708@redhat.com>
On 09/10/2009 04:51 PM, Xueming Shen wrote:
> David M. Lloyd wrote:
>> On 09/10/2009 01:47 PM, Xueming Shen wrote:
>>> I think we have enough discussion on this topic, it's time to make a
>>> final decision. Seems like we are all happy on the
>>> changes in Deflater and new DOS.flush(mode), the only difference is
>>> whether or not we should change the existing
>>> behavior of DOS.flush() should use Z_SYNC_FLUSH by default.
>>
>> The question is, is there really such code out there? It can't really
>> have any useful semantics that I can see. My irrelevant vote is
>> firmly on "yes" for this.
>>
>> - DML
> -The question is, is there really such code out there?
>
> I honestly don't know, this is why it is called "compatibility risk",
> similar to the investment risk in the stock market, you
> lose or gain only after you put the real money in:-) Though my feeling
> is it is very "likely" this change is going to break
> someone's code given DOS is designed/implemented to be easily used as a
> "normal" OutputStream.
>
> -It can't really have any useful semantics that I can see
>
> DOS abstracts two functionalities, a deflater and an output stream, both
> the deflater and output stream have their own
> "flush" semantics. A flush on the deflater changes the final output,
> degrades the compression, basically it changes the
> "function" of the deflater. A flush on the OutputStream however is
> supposed to push out the "buffered" output (accumulated
> for whatever reason, mostly for the i/o performance), it does not change
> the final result/output. So arguably it should not be
> a too bad idea to also provide two types of "flush" to flush the
> deflater and/or to flush the output stream, the existing
> DOS.flush() "accidentally" provides the semantics of the outputstream's
> flush(), with the help of the newly added flush(mode),
> now the developer have the choice of doing whatever "flush" they want to
> do.
This is not unusual in the JDK or elsewhere - that a stream calls the
underlying stream's flush() method I mean - and I think in the (unlikely)
event that someone wants to flush the underlying stream only, they could
simply call .flush() directly on that stream. And furthermore, if they
want to *prevent* the flush() method from calling the underlying flush(),
they can simply interpose a FlushIgnoringOutputStream which changes flush()
to a no-op and passes other calls through. So I'd say "no" to having a
separate flush method that does *that*.
That all said, I guess I should qualify my "yes" with the statement that if
the final decision were ultimately "no", it would be really, really trivial
for someone to subclass or wrap DOS with a version which does the full sync
flush instead of the pass-thru that your current webrev has. And come to
think of it, if you're going to provide your own deflater (which often
happens) you need to subclass DOS anyway to avoid leaking Deflaters on
stream close (since the "usesDefaultDeflater"-based auto-cleanup mechanism
is defeated in this case). So maybe it wouldn't be so bad to leave it as is...
- DML
From forax at univ-mlv.fr Thu Sep 10 15:32:51 2009
From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=)
Date: Fri, 11 Sep 2009 00:32:51 +0200
Subject: What methods should go into a java.util.Objects class in JDK
7?
In-Reply-To: <1ccfd1c10909091911l67ffaf66xf0bb3733f4071224@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
<4AA84E35.4060207@univ-mlv.fr>
<1ccfd1c10909091911l67ffaf66xf0bb3733f4071224@mail.gmail.com>
Message-ID: <4AA97E93.5000506@univ-mlv.fr>
Le 10/09/2009 04:11, Martin Buchholz a ?crit :
> On Wed, Sep 9, 2009 at 17:54, R?mi Forax wrote:
>
>> Le 10/09/2009 02:15, Martin Buchholz a ?crit :
>>
>>> Here's a bunch of static hashCode methods for primitives:
>>>
>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/hashCode/
>>>
>>> This doesn't include a
>>>
>>> public static int hashCode(object x) { return x == null ? 0 :
>>> x.hashCode(); }
>>>
>>> that I think was also being asked for, and which might be a candidate
>>> for Objects.
>>>
>>> Martin
>>>
>>>
>> Martin,
>> You forget java.lang.Boolean in your patch set :)
>>
> R?mi, merci beaucoup.
> Les Booleans, je les avaient oubli?e.
>
> Mes patchs pour les classes wrapper,
> tous les deux, maintenant incluent Boolean.
>
> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/hashCode/
> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/compare/
>
> Martin
>
Wow, you're able to write in French.
From now onwards, I will write my email to you in French :)
R?mi
>
>> R?mi
>>
>>
From Joe.Darcy at Sun.COM Thu Sep 10 15:48:29 2009
From: Joe.Darcy at Sun.COM (Joe Darcy)
Date: Thu, 10 Sep 2009 15:48:29 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA86C11.7080409@sun.com>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
<4AA84AEA.7050008@sun.com>
<1ccfd1c10909091942p41683f0bl46d901126c61e8d5@mail.gmail.com>
<4AA86C11.7080409@sun.com>
Message-ID: <4AA9823D.60507@sun.com>
On 09/09/09 08:01 PM, Joe Darcy wrote:
> Martin Buchholz wrote:
>> On Wed, Sep 9, 2009 at 17:40, Joe Darcy wrote:
>>
>>> PS This talk of hashing remind me of a small spec cleanup:
>>> 4245470 algorithm of java.lang.Byte.hashCode() is not specified
>>> http://bugs.sun.com/view_bug.do?bug_id=4245470
>>>
>>>
>>
>> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/hashCode-spec/
>>
>> Martin
>>
>
> I'll work on the archeology to verify this behavior has been
> consistent over time and thus is reasonable to tighten the spec in
> these cases.
>
> -Joe
Martin,
I noticed that the Character class should be similarly tightened:
/**
* Returns a hash code for this {@code Character}; equal to the result
* of invoking {@code intValue()}.
*
* @return a hash code value for this {@code Character}
*/
public int hashCode()
I've done some checking and all of the following Sun JDKs implement the
"intValue" hashCode for Byte, Character, and Short:
1.1
1.2
1.3.0
1.3.1
1,4.0
1.4.1
1.4.2
1.5
1.6
so I think a change to all three wrapper classes is safe to go back from
a behavioral compatibility perspective :-)
-Joe
From martinrb at google.com Thu Sep 10 16:40:25 2009
From: martinrb at google.com (Martin Buchholz)
Date: Thu, 10 Sep 2009 16:40:25 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA9823D.60507@sun.com>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
<4AA84AEA.7050008@sun.com>
<1ccfd1c10909091942p41683f0bl46d901126c61e8d5@mail.gmail.com>
<4AA86C11.7080409@sun.com> <4AA9823D.60507@sun.com>
Message-ID: <1ccfd1c10909101640r465ac11albc414bf6573c215b@mail.gmail.com>
On Thu, Sep 10, 2009 at 15:48, Joe Darcy wrote:
> On 09/09/09 08:01 PM, Joe Darcy wrote:
> I noticed that the Character class should be similarly tightened:
Right you are. Webrev regenerated.
Martin
From Joe.Darcy at Sun.COM Thu Sep 10 16:48:00 2009
From: Joe.Darcy at Sun.COM (Joe Darcy)
Date: Thu, 10 Sep 2009 16:48:00 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <1ccfd1c10909101640r465ac11albc414bf6573c215b@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
<4AA84AEA.7050008@sun.com>
<1ccfd1c10909091942p41683f0bl46d901126c61e8d5@mail.gmail.com>
<4AA86C11.7080409@sun.com> <4AA9823D.60507@sun.com>
<1ccfd1c10909101640r465ac11albc414bf6573c215b@mail.gmail.com>
Message-ID: <4AA99030.8040904@sun.com>
Martin Buchholz wrote:
> On Thu, Sep 10, 2009 at 15:48, Joe Darcy wrote:
>
>> On 09/09/09 08:01 PM, Joe Darcy wrote:
>> I noticed that the Character class should be similarly tightened:
>>
>
> Right you are. Webrev regenerated.
>
> Martin
>
One typo inherited from my earlier suggestion (mea culpa!): as the
Character class is not a java.lang.Number, it does not have an intValue
method. Character does have a charValue method and referring to that
method in the specification of hashCode should suffice.
I approve this change going back with the correction to the Character class.
Thanks,
-Joe
From martinrb at google.com Thu Sep 10 16:56:29 2009
From: martinrb at google.com (Martin Buchholz)
Date: Thu, 10 Sep 2009 16:56:29 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA99030.8040904@sun.com>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
<4AA84AEA.7050008@sun.com>
<1ccfd1c10909091942p41683f0bl46d901126c61e8d5@mail.gmail.com>
<4AA86C11.7080409@sun.com> <4AA9823D.60507@sun.com>
<1ccfd1c10909101640r465ac11albc414bf6573c215b@mail.gmail.com>
<4AA99030.8040904@sun.com>
Message-ID: <1ccfd1c10909101656sdcc6643uf4ff06cf76b05f30@mail.gmail.com>
On Thu, Sep 10, 2009 at 16:48, Joe Darcy wrote:
> Martin Buchholz wrote:
>>
>> On Thu, Sep 10, 2009 at 15:48, Joe Darcy wrote:
>>
>>>
>>> On 09/09/09 08:01 PM, Joe Darcy wrote:
>>> I noticed that the Character class should be similarly tightened:
>>>
>>
>> Right you are. ?Webrev regenerated.
>>
>> Martin
>>
>
> One typo inherited from my earlier suggestion (mea culpa!): as the Character
> class is not a java.lang.Number, it does not have an intValue method.
> ?Character does have a charValue method and referring to that method in the
> specification of hashCode should suffice.
>
> I approve this change going back with the correction to the Character class.
Oops! so many little mistakes. Webrev regenerated.
Martin
From scolebourne at joda.org Thu Sep 10 17:01:47 2009
From: scolebourne at joda.org (Stephen Colebourne)
Date: Fri, 11 Sep 2009 01:01:47 +0100
Subject: What methods should go into a java.util.Objects class in JDK
7?
In-Reply-To: <4AA812D0.309@sun.com>
References: <4AA812D0.309@sun.com>
Message-ID: <4AA9936B.5000006@joda.org>
Joe Darcy wrote:
> What other utility methods would have broad enough use and applicability
> to go into a common java.util class?
Joe,
You've asked for a call for ideas, but not given any indication of
process. Are you going to evaluate everything that comes in and pick the
best a la Coin? Or allow anyone to send in patches? Who decides what is
in and what is out?
Since its obviously not going to be decided by a Java SE 7 JSR, it would
be useful to know the answers, before we all waste lots of time on
discussions about methods that might or might not be used.
Stephen
From Joe.Darcy at Sun.COM Thu Sep 10 17:03:47 2009
From: Joe.Darcy at Sun.COM (Joe Darcy)
Date: Thu, 10 Sep 2009 17:03:47 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <1ccfd1c10909101656sdcc6643uf4ff06cf76b05f30@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
<4AA84AEA.7050008@sun.com>
<1ccfd1c10909091942p41683f0bl46d901126c61e8d5@mail.gmail.com>
<4AA86C11.7080409@sun.com> <4AA9823D.60507@sun.com>
<1ccfd1c10909101640r465ac11albc414bf6573c215b@mail.gmail.com>
<4AA99030.8040904@sun.com>
<1ccfd1c10909101656sdcc6643uf4ff06cf76b05f30@mail.gmail.com>
Message-ID: <4AA993E3.20506@sun.com>
Martin Buchholz wrote:
> On Thu, Sep 10, 2009 at 16:48, Joe Darcy wrote:
>> Martin Buchholz wrote:
>>> On Thu, Sep 10, 2009 at 15:48, Joe Darcy wrote:
>>>
>>>> On 09/09/09 08:01 PM, Joe Darcy wrote:
>>>> I noticed that the Character class should be similarly tightened:
>>>>
>>> Right you are. Webrev regenerated.
>>>
>>> Martin
>>>
>> One typo inherited from my earlier suggestion (mea culpa!): as the Character
>> class is not a java.lang.Number, it does not have an intValue method.
>> Character does have a charValue method and referring to that method in the
>> specification of hashCode should suffice.
>>
>> I approve this change going back with the correction to the Character class.
>
> Oops! so many little mistakes. Webrev regenerated.
Looks good.
-Joe
From Ulf.Zibis at gmx.de Fri Sep 11 00:00:40 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Fri, 11 Sep 2009 09:00:40 +0200
Subject: What methods should go into a java.util.Objects class in JDK
7?
In-Reply-To: <1ccfd1c10909101656sdcc6643uf4ff06cf76b05f30@mail.gmail.com>
References: <4AA812D0.309@sun.com> <1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com> <4AA84AEA.7050008@sun.com> <1ccfd1c10909091942p41683f0bl46d901126c61e8d5@mail.gmail.com> <4AA86C11.7080409@sun.com>
<4AA9823D.60507@sun.com> <1ccfd1c10909101640r465ac11albc414bf6573c215b@mail.gmail.com> <4AA99030.8040904@sun.com>
<1ccfd1c10909101656sdcc6643uf4ff06cf76b05f30@mail.gmail.com>
Message-ID: <4AA9F598.9050703@gmx.de>
Am 11.09.2009 01:56, Martin Buchholz schrieb:
> On Thu, Sep 10, 2009 at 16:48, Joe Darcy wrote:
>
>> I approve this change going back with the correction to the Character
>> class.
>>
>
> Oops! so many little mistakes. Webrev regenerated.
>
Is that webrev public anywhere ?
(Hopefully you consider my "cosmetics" and some of my changes from [1]
some day)
-Ulf
[1] https://bugs.openjdk.java.net/show_bug.cgi?id=100104
From David.Holmes at Sun.COM Fri Sep 11 01:57:12 2009
From: David.Holmes at Sun.COM (David Holmes - Sun Microsystems)
Date: Fri, 11 Sep 2009 18:57:12 +1000
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA9F598.9050703@gmx.de>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
<4AA84AEA.7050008@sun.com>
<1ccfd1c10909091942p41683f0bl46d901126c61e8d5@mail.gmail.com>
<4AA86C11.7080409@sun.com> <4AA9823D.60507@sun.com>
<1ccfd1c10909101640r465ac11albc414bf6573c215b@mail.gmail.com>
<4AA99030.8040904@sun.com>
<1ccfd1c10909101656sdcc6643uf4ff06cf76b05f30@mail.gmail.com>
<4AA9F598.9050703@gmx.de>
Message-ID: <4AAA10E8.3010103@sun.com>
Ulf Zibis said the following on 09/11/09 17:00:
> Am 11.09.2009 01:56, Martin Buchholz schrieb:
>> On Thu, Sep 10, 2009 at 16:48, Joe Darcy wrote:
>>
>>> I approve this change going back with the correction to the Character
>>> class.
>>>
>> Oops! so many little mistakes. Webrev regenerated.
>>
>
> Is that webrev public anywhere ?
All Martin's webrevs are public:
http://cr.openjdk.java.net/~martin/webrevs/openjdk7/
This one was:
http://cr.openjdk.java.net/~martin/webrevs/openjdk7/hashCode
-David
> (Hopefully you consider my "cosmetics" and some of my changes from [1]
> some day)
>
> -Ulf
>
> [1] https://bugs.openjdk.java.net/show_bug.cgi?id=100104
>
>
From Alan.Bateman at Sun.COM Fri Sep 11 02:42:13 2009
From: Alan.Bateman at Sun.COM (Alan Bateman)
Date: Fri, 11 Sep 2009 10:42:13 +0100
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AA949DA.9010707@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
<1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
<4AA7F9EB.5090200@sun.com> <4AA816AD.7090306@sun.com>
<4AA949DA.9010707@sun.com>
Message-ID: <4AAA1B75.6060309@sun.com>
Xueming Shen wrote:
> I think we have enough discussion on this topic, it's time to make a
> final decision. Seems like we are all happy on the
> changes in Deflater and new DOS.flush(mode), the only difference is
> whether or not we should change the existing
> behavior of DOS.flush() should use Z_SYNC_FLUSH by default.
>
> Brandon is very firm on the "yes" side, strongly believe this is a MUST.
> Martin is on the fence, leaning towards "yes"
> Sherman is on the "no" side
>
> So if there is no other vote coming with strong opinion, your vote
> will be important:-) Are you also leaning to "yes"
> or "strongly" suggesting a yes? I need strong "yes" to change my mind.
This is tough call. It is clearly desirable to change flush() to work
the way that developers expect. On the other hand there is always risk
with changing long-standing behavior. The risk here is probably low but
it's impossible to quantify. Martin makes a good observation that it may
cause problems for code that wraps the stream with an auto-flushing
PrintStream or PrintWriter. You also made a good observation that just
changing the number of bytes written could cause breakage (in some
fragile environment) - we just don't know! So on balance, I think you
are probably right to take the conservative line on this one.
I know you've had enough discussion on this but I wonder if you might be
open to considering new constructors for DeflaterOutputStream (and maybe
GZIPOutputStream) as an alternative to the flush(int) method. This might
fit better with this API, in particular for cases where the intended
usage is known. It could be as simple as a "boolean syncFlush" parameter
to indicate how uncompressed buffered bytes are handled when flushing
(default to false for existing constructors so existing behavior
preserved). The boolean parameter is just an opening bid - clearly
there are alternatives. The point is that it makes it easy and more
obvious as how to get a DeflaterOutputStream suitable for interactive cases.
-Alan.
From Vladimir.Yaroslavskiy at Sun.COM Fri Sep 11 03:35:22 2009
From: Vladimir.Yaroslavskiy at Sun.COM (Vladimir Yaroslavskiy)
Date: Fri, 11 Sep 2009 14:35:22 +0400
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
Quicksort
Message-ID: <4AAA27EA.8090902@Sun.COM>
Hello All,
I'd like to share with you new Dual-Pivot Quicksort which is
faster than the known implementations (theoretically and
experimental). I'd like to propose to replace the JDK's
Quicksort implementation by new one.
Description
-----------
The classical Quicksort algorithm uses the following scheme:
1. Pick an element P, called a pivot, from the array.
2. Reorder the array so that all elements, which are less than
the pivot, come before the pivot and all elements greater than
the pivot come after it (equal values can go either way). After
this partitioning, the pivot element is in its final position.
3. Recursively sort the sub-array of lesser elements and the
sub-array of greater elements.
The invariant of classical Quicksort is:
[ <= p | >= p ]
There are several modifications of the schema:
[ < p | = p | > p ] or [ = p | < p | > p | = p ]
But all of them use *one* pivot.
The new Dual-Pivot Quicksort uses *two* pivots elements in this manner:
1. Pick an elements P1, P2, called pivots from the array.
2. Assume that P1 <= P2, otherwise swap it.
3. Reorder the array into three parts: those less than the smaller
pivot, those larger than the larger pivot, and in between are
those elements between (or equal to) the two pivots.
4. Recursively sort the sub-arrays.
The invariant of the Dual-Pivot Quicksort is:
[ < P1 | P1 <= & <= P2 } > P2 ]
The new Quicksort is faster than current implementation of Quicksort
in JDK (L. Bentley and M. Douglas McIlroy) and classical Quicksort.
The full description of the Dual-Pivot Quicksort you can find
on my page: http://iaroslavski.narod.ru/quicksort
Performance tests
-----------------
Here is result of running on different types of input data:
Client VM all 85% organ 0..1 0..4
random ascend descend equal equal pipes random 010101 random
Dual-Pivot: 16.83 5.31 5.47 0.35 0.68 10.59 1.06 1.02 2.18
Bentley's: 19.77 9.08 10.13 0.63 1.12 13.22 1.63 1.08 2.49
Server VM all 85% organ 0..1 0..4
random ascend descend equal equal pipes random 010101 random
Dual-Pivot: 23.94 6.68 6.63 0.43 0.62 17.14 1.42 1.96 3.41
Bentley's: 25.20 10.18 10.32 2.07 1.33 16.72 2.95 1.82 3.39
The a lot of other tests have been run under client and server mode.
The most interesting is BentleyBasher test framework. It runs battery
of tests for all cases:
{ 100, 1000, 10000, 1000000 } x
{ sawtooth, rand, stagger, plateau, shuffle } x
{ ident, reverse, reverse_front, reverse_back, sort, dither}
where
100, ... , 1000000 - array length
sawtooth: x[i] =i%m
rand: x[i] = rand() % m
stagger: x[i] = (i*m + i) % n
plateau: x[i] = min(i, m)
shuffle: x[i] = rand()%m? (j+=2): (k+=2)
ident(x) - a copy of x
reverse(x, 0, n) - reversed copy
reverse_front(x, 0, n/2) - front half reversed
reverse_back(x, n/2, n) - back half reversed
sort(x) - an ordered copy
dither(x) - add i%5 to x[i]
Here is the result of execution:
Server VM: http://spreadsheets.google.com/pub?key=t_EAWUkQ4mD3BIbOv8Fa-AQ&output=html
Client VM: http://spreadsheets.google.com/pub?key=tdiMo8xleTxd23nKUObcz0Q&single=true&gid=0&output=html
Mathematical investigations
---------------------------
It is proved that for the Dual-Pivot Quicksort the average number of
comparisons is 2*n*ln(n), the average number of swaps is 0.8*n*ln(n),
whereas classical Quicksort algorithm has 2*n*ln(n) and 1*n*ln(n)
respectively. Full mathematical proof see in attached proof.txt
and proof_add.txt files. Theoretical results are also confirmed
by experimental counting of the operations.
Diff between current and new implementation of Quicksort
--------------------------------------------------------
Here is the link to the diff for java.util.Arrays class:
http://cr.openjdk.java.net/~alanb/6880672/webrev.00
If you like to look and play with new algorithm,
please, take attached class DualPivotQuicksort.java
Feedback
--------
Also I'd like to share a feedback from Joshua Bloch and
Jon Bentley who spent a lot of time investigating this
algorithm, who gave me many advices and tips how to
make new Quicksort better.
-------- Original Message --------
Subject: Re: Integration of new Dual-Pivot Quicksort into JDK 7
Date: Thu, 10 Sep 2009 07:20:11 -0700
From: Joshua Bloch
Jon also says that Vladimir should make every reasonable improvement to
the basic method before checking in the code. In his words, "It would be
horrible to put the new code into the library, and then have someone
else come along and speed it up by another 20% by using standard
techniques." I believe it's not unlikely that this code may end up
getting ported to many languages and widely deployed in much the manner
of Bentley and McIlroy's fine sort (which is nearing 20 successful years
in the field). Jon will help Vladimir do this.
-------- Original Message --------
Subject: Dual-Pivot Quicksort: Next Steps
Date: Wed, 09 Sep 2009 15:02:25 -0400
From: Jon Bentley
Vladimir, Josh,
I *finally* feel like I understand what is going on. Now that I
(think that) I see it, it seems straightforward and obvious.
Tony Hoare developed Quicksort in the early 1960s. I was very
proud to make minor contributions to a particularly clean (binary)
quicksort in the mid 1980s, to a relatively straightforward, industrial
strength Quicksort with McIlroy in the early 1990s, and then to
algorithms and data structures for strings with Sedgewick in the mid 1990s.
I think that Vladimir's contributions to Quicksort go way beyond
anything that I've ever done, and rank up there with Hoare's original
design and Sedgewick's analysis. I feel so privileged to play a very,
very minor role in helping Vladimir with the most excellent work!
-----------------------------------------------
Let me know, if you have any questions/comments.
Thank you,
Vladimir
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: proof_add.txt
Url: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090911/6c3ba31b/attachment-0002.txt
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: DualPivotQuicksort.java
Url: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090911/6c3ba31b/attachment-0001.ksh
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: proof.txt
Url: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090911/6c3ba31b/attachment-0003.txt
From forax at univ-mlv.fr Fri Sep 11 06:32:08 2009
From: forax at univ-mlv.fr (=?ISO-8859-1?Q?R=E9mi_Forax?=)
Date: Fri, 11 Sep 2009 15:32:08 +0200
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
Quicksort
In-Reply-To: <4AAA27EA.8090902@Sun.COM>
References: <4AAA27EA.8090902@Sun.COM>
Message-ID: <4AAA5158.8060202@univ-mlv.fr>
just my two cents :
In the loop tagged "sorting" and "equals element",
a[k] can be stored in a local variable to avoid to recompute it several
time.
The algorithm use two constants: 37 and 13,
I think they should be declared as private static final.
R?mi
Le 11/09/2009 12:35, Vladimir Yaroslavskiy a ?crit :
> Hello All,
>
> I'd like to share with you new Dual-Pivot Quicksort which is
> faster than the known implementations (theoretically and
> experimental). I'd like to propose to replace the JDK's
> Quicksort implementation by new one.
>
> Description
> -----------
> The classical Quicksort algorithm uses the following scheme:
>
> 1. Pick an element P, called a pivot, from the array.
> 2. Reorder the array so that all elements, which are less than
> the pivot, come before the pivot and all elements greater than
> the pivot come after it (equal values can go either way). After
> this partitioning, the pivot element is in its final position.
> 3. Recursively sort the sub-array of lesser elements and the
> sub-array of greater elements.
>
> The invariant of classical Quicksort is:
>
> [ <= p | >= p ]
>
> There are several modifications of the schema:
>
> [ < p | = p | > p ] or [ = p | < p | > p | = p ]
>
> But all of them use *one* pivot.
>
> The new Dual-Pivot Quicksort uses *two* pivots elements in this manner:
>
> 1. Pick an elements P1, P2, called pivots from the array.
> 2. Assume that P1 <= P2, otherwise swap it.
> 3. Reorder the array into three parts: those less than the smaller
> pivot, those larger than the larger pivot, and in between are
> those elements between (or equal to) the two pivots.
> 4. Recursively sort the sub-arrays.
>
> The invariant of the Dual-Pivot Quicksort is:
>
> [ < P1 | P1 <= & <= P2 } > P2 ]
>
> The new Quicksort is faster than current implementation of Quicksort
> in JDK (L. Bentley and M. Douglas McIlroy) and classical Quicksort.
>
> The full description of the Dual-Pivot Quicksort you can find
> on my page: http://iaroslavski.narod.ru/quicksort
>
> Performance tests
> -----------------
> Here is result of running on different types of input data:
>
> Client VM all 85% organ 0..1 0..4
> random ascend descend equal equal pipes random 010101
> random
> Dual-Pivot: 16.83 5.31 5.47 0.35 0.68 10.59 1.06 1.02 2.18
> Bentley's: 19.77 9.08 10.13 0.63 1.12 13.22 1.63 1.08 2.49
>
> Server VM all 85% organ 0..1 0..4
> random ascend descend equal equal pipes random 010101
> random
> Dual-Pivot: 23.94 6.68 6.63 0.43 0.62 17.14 1.42 1.96 3.41
> Bentley's: 25.20 10.18 10.32 2.07 1.33 16.72 2.95 1.82 3.39
>
> The a lot of other tests have been run under client and server mode.
> The most interesting is BentleyBasher test framework. It runs battery
> of tests for all cases:
>
> { 100, 1000, 10000, 1000000 } x
> { sawtooth, rand, stagger, plateau, shuffle } x
> { ident, reverse, reverse_front, reverse_back, sort, dither}
>
> where
>
> 100, ... , 1000000 - array length
>
> sawtooth: x[i] =i%m
> rand: x[i] = rand() % m
> stagger: x[i] = (i*m + i) % n
> plateau: x[i] = min(i, m)
> shuffle: x[i] = rand()%m? (j+=2): (k+=2)
>
> ident(x) - a copy of x
> reverse(x, 0, n) - reversed copy
> reverse_front(x, 0, n/2) - front half reversed
> reverse_back(x, n/2, n) - back half reversed
> sort(x) - an ordered copy
> dither(x) - add i%5 to x[i]
>
> Here is the result of execution:
> Server VM:
> http://spreadsheets.google.com/pub?key=t_EAWUkQ4mD3BIbOv8Fa-AQ&output=html
>
> Client VM:
> http://spreadsheets.google.com/pub?key=tdiMo8xleTxd23nKUObcz0Q&single=true&gid=0&output=html
>
>
> Mathematical investigations
> ---------------------------
> It is proved that for the Dual-Pivot Quicksort the average number of
> comparisons is 2*n*ln(n), the average number of swaps is 0.8*n*ln(n),
> whereas classical Quicksort algorithm has 2*n*ln(n) and 1*n*ln(n)
> respectively. Full mathematical proof see in attached proof.txt
> and proof_add.txt files. Theoretical results are also confirmed
> by experimental counting of the operations.
>
> Diff between current and new implementation of Quicksort
> --------------------------------------------------------
>
> Here is the link to the diff for java.util.Arrays class:
> http://cr.openjdk.java.net/~alanb/6880672/webrev.00
>
> If you like to look and play with new algorithm,
> please, take attached class DualPivotQuicksort.java
>
> Feedback
> --------
>
> Also I'd like to share a feedback from Joshua Bloch and
> Jon Bentley who spent a lot of time investigating this
> algorithm, who gave me many advices and tips how to
> make new Quicksort better.
>
> -------- Original Message --------
> Subject: Re: Integration of new Dual-Pivot Quicksort into JDK 7
> Date: Thu, 10 Sep 2009 07:20:11 -0700
> From: Joshua Bloch
>
> Jon also says that Vladimir should make every reasonable improvement to
> the basic method before checking in the code. In his words, "It would be
> horrible to put the new code into the library, and then have someone
> else come along and speed it up by another 20% by using standard
> techniques." I believe it's not unlikely that this code may end up
> getting ported to many languages and widely deployed in much the manner
> of Bentley and McIlroy's fine sort (which is nearing 20 successful years
> in the field). Jon will help Vladimir do this.
>
> -------- Original Message --------
> Subject: Dual-Pivot Quicksort: Next Steps
> Date: Wed, 09 Sep 2009 15:02:25 -0400
> From: Jon Bentley
>
> Vladimir, Josh,
> I *finally* feel like I understand what is going on. Now that I
> (think that) I see it, it seems straightforward and obvious.
> Tony Hoare developed Quicksort in the early 1960s. I was very
> proud to make minor contributions to a particularly clean (binary)
> quicksort in the mid 1980s, to a relatively straightforward, industrial
> strength Quicksort with McIlroy in the early 1990s, and then to
> algorithms and data structures for strings with Sedgewick in the mid
> 1990s.
> I think that Vladimir's contributions to Quicksort go way beyond
> anything that I've ever done, and rank up there with Hoare's original
> design and Sedgewick's analysis. I feel so privileged to play a very,
> very minor role in helping Vladimir with the most excellent work!
>
> -----------------------------------------------
>
> Let me know, if you have any questions/comments.
>
> Thank you,
> Vladimir
From Ulf.Zibis at gmx.de Fri Sep 11 06:41:04 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Fri, 11 Sep 2009 15:41:04 +0200
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
Quicksort
In-Reply-To: <4AAA27EA.8090902@Sun.COM>
References: <4AAA27EA.8090902@Sun.COM>
Message-ID: <4AAA5370.3010704@gmx.de>
Very interesting stuff.
Does one have tried (theoretically and/or experimental) P1+P2+P3,
P1+P2+P3+P2, ... segmentation ?
Maybe coefficient A has a minimum below 0.8.
-Ulf
Am 11.09.2009 12:35, Vladimir Yaroslavskiy schrieb:
> Hello All,
>
> I'd like to share with you new Dual-Pivot Quicksort which is
> faster than the known implementations (theoretically and
> experimental). I'd like to propose to replace the JDK's
> Quicksort implementation by new one.
>
> Description
> -----------
> The classical Quicksort algorithm uses the following scheme:
>
> 1. Pick an element P, called a pivot, from the array.
> 2. Reorder the array so that all elements, which are less than
> the pivot, come before the pivot and all elements greater than
> the pivot come after it (equal values can go either way). After
> this partitioning, the pivot element is in its final position.
> 3. Recursively sort the sub-array of lesser elements and the
> sub-array of greater elements.
>
> The invariant of classical Quicksort is:
>
> [ <= p | >= p ]
>
> There are several modifications of the schema:
>
> [ < p | = p | > p ] or [ = p | < p | > p | = p ]
>
> But all of them use *one* pivot.
>
> The new Dual-Pivot Quicksort uses *two* pivots elements in this manner:
>
> 1. Pick an elements P1, P2, called pivots from the array.
> 2. Assume that P1 <= P2, otherwise swap it.
> 3. Reorder the array into three parts: those less than the smaller
> pivot, those larger than the larger pivot, and in between are
> those elements between (or equal to) the two pivots.
> 4. Recursively sort the sub-arrays.
>
> The invariant of the Dual-Pivot Quicksort is:
>
> [ < P1 | P1 <= & <= P2 } > P2 ]
>
> The new Quicksort is faster than current implementation of Quicksort
> in JDK (L. Bentley and M. Douglas McIlroy) and classical Quicksort.
>
> The full description of the Dual-Pivot Quicksort you can find
> on my page: http://iaroslavski.narod.ru/quicksort
>
> Performance tests
> -----------------
> Here is result of running on different types of input data:
>
> Client VM all 85% organ 0..1 0..4
> random ascend descend equal equal pipes random 010101
> random
> Dual-Pivot: 16.83 5.31 5.47 0.35 0.68 10.59 1.06 1.02 2.18
> Bentley's: 19.77 9.08 10.13 0.63 1.12 13.22 1.63 1.08 2.49
>
> Server VM all 85% organ 0..1 0..4
> random ascend descend equal equal pipes random 010101
> random
> Dual-Pivot: 23.94 6.68 6.63 0.43 0.62 17.14 1.42 1.96 3.41
> Bentley's: 25.20 10.18 10.32 2.07 1.33 16.72 2.95 1.82 3.39
>
> The a lot of other tests have been run under client and server mode.
> The most interesting is BentleyBasher test framework. It runs battery
> of tests for all cases:
>
> { 100, 1000, 10000, 1000000 } x
> { sawtooth, rand, stagger, plateau, shuffle } x
> { ident, reverse, reverse_front, reverse_back, sort, dither}
>
> where
>
> 100, ... , 1000000 - array length
>
> sawtooth: x[i] =i%m
> rand: x[i] = rand() % m
> stagger: x[i] = (i*m + i) % n
> plateau: x[i] = min(i, m)
> shuffle: x[i] = rand()%m? (j+=2): (k+=2)
>
> ident(x) - a copy of x
> reverse(x, 0, n) - reversed copy
> reverse_front(x, 0, n/2) - front half reversed
> reverse_back(x, n/2, n) - back half reversed
> sort(x) - an ordered copy
> dither(x) - add i%5 to x[i]
>
> Here is the result of execution:
> Server VM:
> http://spreadsheets.google.com/pub?key=t_EAWUkQ4mD3BIbOv8Fa-AQ&output=html
>
> Client VM:
> http://spreadsheets.google.com/pub?key=tdiMo8xleTxd23nKUObcz0Q&single=true&gid=0&output=html
>
>
> Mathematical investigations
> ---------------------------
> It is proved that for the Dual-Pivot Quicksort the average number of
> comparisons is 2*n*ln(n), the average number of swaps is 0.8*n*ln(n),
> whereas classical Quicksort algorithm has 2*n*ln(n) and 1*n*ln(n)
> respectively. Full mathematical proof see in attached proof.txt
> and proof_add.txt files. Theoretical results are also confirmed
> by experimental counting of the operations.
>
> Diff between current and new implementation of Quicksort
> --------------------------------------------------------
>
> Here is the link to the diff for java.util.Arrays class:
> http://cr.openjdk.java.net/~alanb/6880672/webrev.00
>
> If you like to look and play with new algorithm,
> please, take attached class DualPivotQuicksort.java
>
> Feedback
> --------
>
> Also I'd like to share a feedback from Joshua Bloch and
> Jon Bentley who spent a lot of time investigating this
> algorithm, who gave me many advices and tips how to
> make new Quicksort better.
>
> -------- Original Message --------
> Subject: Re: Integration of new Dual-Pivot Quicksort into JDK 7
> Date: Thu, 10 Sep 2009 07:20:11 -0700
> From: Joshua Bloch
>
> Jon also says that Vladimir should make every reasonable improvement to
> the basic method before checking in the code. In his words, "It would be
> horrible to put the new code into the library, and then have someone
> else come along and speed it up by another 20% by using standard
> techniques." I believe it's not unlikely that this code may end up
> getting ported to many languages and widely deployed in much the manner
> of Bentley and McIlroy's fine sort (which is nearing 20 successful years
> in the field). Jon will help Vladimir do this.
>
> -------- Original Message --------
> Subject: Dual-Pivot Quicksort: Next Steps
> Date: Wed, 09 Sep 2009 15:02:25 -0400
> From: Jon Bentley
>
> Vladimir, Josh,
> I *finally* feel like I understand what is going on. Now that I
> (think that) I see it, it seems straightforward and obvious.
> Tony Hoare developed Quicksort in the early 1960s. I was very
> proud to make minor contributions to a particularly clean (binary)
> quicksort in the mid 1980s, to a relatively straightforward, industrial
> strength Quicksort with McIlroy in the early 1990s, and then to
> algorithms and data structures for strings with Sedgewick in the mid
> 1990s.
> I think that Vladimir's contributions to Quicksort go way beyond
> anything that I've ever done, and rank up there with Hoare's original
> design and Sedgewick's analysis. I feel so privileged to play a very,
> very minor role in helping Vladimir with the most excellent work!
>
> -----------------------------------------------
>
> Let me know, if you have any questions/comments.
>
> Thank you,
> Vladimir
From iaroslavski at mail.ru Fri Sep 11 06:42:12 2009
From: iaroslavski at mail.ru (Vladimir Iaroslavski)
Date: Fri, 11 Sep 2009 17:42:12 +0400
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
Quicksort
In-Reply-To: <4AAA5370.3010704@gmx.de>
References: <4AAA27EA.8090902@Sun.COM> <4AAA5370.3010704@gmx.de>
Message-ID: <4AAA53B4.20702@mail.ru>
Hi Ulf,
Sure, I have tried it, see:
-------- Original Message --------
Subject: Re: Dual-Pivot Quicksort Analysis
Date: Wed, 09 Sep 2009 18:45:46 +0400
From: Vladimir Yaroslavskiy
Organization: Sun Microsystems, Inc.
To: Jon Bentley
CC: Joshua Bloch
Hello,
I've converted the pseudocode into Java and here is the result:
I counted the comps and swaps for array with length n = 2'000'000,
did it 77 times and took the average value:
Classic Quicksort: comps - 52254956
swaps - 26277985
Dual-Pivot Quicksort: comps - 52246553
swaps - 20977937
These values correlate with math proof: the number of comps
the same (2*n*ln(n)) and ratio for swaps is 20977937/26277985
= 0.7983 (theoretical is 0.8/1.0 = 0.8).
I don't have "few words" explanation wh ythe new algorithm is quicker,
just the ratio swaps/comps/partitions is better for Dual-Pivot Quicksort.
If we consider common case when we have (k-1) pivots and divide an
array into k parts, we can ask, if the case k=3 (Dual-Pivot Quicksort)
is the best? May be k=4,5 are better?
It's easy to define recurrence relation for the count of operations
when k=3,4,... but mathematical conclusion will be too complex.
But to find out the answer for this question, I run pseudocode
(in Java, see attached class) for k=4,5 and got:
Quicksort with 3 pivots/4 parts: comps - 54399943
swaps - 24224254
Quicksort with 4 pivots/5 parts: comps - 57241615
swaps - 28659728
So, you can see that "3 pivots/4 parts" is better than
Classic Quicksort for swaps only, "4 pivots/5 parts" is
the looser, and "Dual-Pivot Quicksort" is real winner.
I feel it is expectable result: if we have many pivots
we spent a lot of time on sorting them and dividing into
more parts doesn't save. In case with 2 pivots, we use
only 1 comparison and 1/2 swaps to sort it out.
BTW, the dividing into 3 parts instead of two also shows
better results for MergeSort (but it's not actual for JDK
because TimSort is faster).
Ulf Zibis wrote:
> Very interesting stuff.
>
> Does one have tried (theoretically and/or experimental) P1+P2+P3,
> P1+P2+P3+P2, ... segmentation ?
> Maybe coefficient A has a minimum below 0.8.
>
> -Ulf
>
> Am 11.09.2009 12:35, Vladimir Yaroslavskiy schrieb:
>> Hello All,
>>
>> I'd like to share with you new Dual-Pivot Quicksort which is
>> faster than the known implementations (theoretically and
>> experimental). I'd like to propose to replace the JDK's
>> Quicksort implementation by new one.
>>
>> Description
>> -----------
>> The classical Quicksort algorithm uses the following scheme:
>>
>> 1. Pick an element P, called a pivot, from the array.
>> 2. Reorder the array so that all elements, which are less than
>> the pivot, come before the pivot and all elements greater than
>> the pivot come after it (equal values can go either way). After
>> this partitioning, the pivot element is in its final position.
>> 3. Recursively sort the sub-array of lesser elements and the
>> sub-array of greater elements.
>>
>> The invariant of classical Quicksort is:
>>
>> [ <= p | >= p ]
>>
>> There are several modifications of the schema:
>>
>> [ < p | = p | > p ] or [ = p | < p | > p | = p ]
>>
>> But all of them use *one* pivot.
>>
>> The new Dual-Pivot Quicksort uses *two* pivots elements in this manner:
>>
>> 1. Pick an elements P1, P2, called pivots from the array.
>> 2. Assume that P1 <= P2, otherwise swap it.
>> 3. Reorder the array into three parts: those less than the smaller
>> pivot, those larger than the larger pivot, and in between are
>> those elements between (or equal to) the two pivots.
>> 4. Recursively sort the sub-arrays.
>>
>> The invariant of the Dual-Pivot Quicksort is:
>>
>> [ < P1 | P1 <= & <= P2 } > P2 ]
>>
>> The new Quicksort is faster than current implementation of Quicksort
>> in JDK (L. Bentley and M. Douglas McIlroy) and classical Quicksort.
>>
>> The full description of the Dual-Pivot Quicksort you can find
>> on my page: http://iaroslavski.narod.ru/quicksort
>>
>> Performance tests
>> -----------------
>> Here is result of running on different types of input data:
>>
>> Client VM all 85% organ 0..1 0..4
>> random ascend descend equal equal pipes random 010101
>> random
>> Dual-Pivot: 16.83 5.31 5.47 0.35 0.68 10.59 1.06 1.02 2.18
>> Bentley's: 19.77 9.08 10.13 0.63 1.12 13.22 1.63 1.08 2.49
>>
>> Server VM all 85% organ 0..1 0..4
>> random ascend descend equal equal pipes random 010101
>> random
>> Dual-Pivot: 23.94 6.68 6.63 0.43 0.62 17.14 1.42 1.96 3.41
>> Bentley's: 25.20 10.18 10.32 2.07 1.33 16.72 2.95 1.82 3.39
>>
>> The a lot of other tests have been run under client and server mode.
>> The most interesting is BentleyBasher test framework. It runs battery
>> of tests for all cases:
>>
>> { 100, 1000, 10000, 1000000 } x
>> { sawtooth, rand, stagger, plateau, shuffle } x
>> { ident, reverse, reverse_front, reverse_back, sort, dither}
>>
>> where
>>
>> 100, ... , 1000000 - array length
>>
>> sawtooth: x[i] =i%m
>> rand: x[i] = rand() % m
>> stagger: x[i] = (i*m + i) % n
>> plateau: x[i] = min(i, m)
>> shuffle: x[i] = rand()%m? (j+=2): (k+=2)
>>
>> ident(x) - a copy of x
>> reverse(x, 0, n) - reversed copy
>> reverse_front(x, 0, n/2) - front half reversed
>> reverse_back(x, n/2, n) - back half reversed
>> sort(x) - an ordered copy
>> dither(x) - add i%5 to x[i]
>>
>> Here is the result of execution:
>> Server VM:
>> http://spreadsheets.google.com/pub?key=t_EAWUkQ4mD3BIbOv8Fa-AQ&output=html
>>
>> Client VM:
>> http://spreadsheets.google.com/pub?key=tdiMo8xleTxd23nKUObcz0Q&single=true&gid=0&output=html
>>
>>
>> Mathematical investigations
>> ---------------------------
>> It is proved that for the Dual-Pivot Quicksort the average number of
>> comparisons is 2*n*ln(n), the average number of swaps is 0.8*n*ln(n),
>> whereas classical Quicksort algorithm has 2*n*ln(n) and 1*n*ln(n)
>> respectively. Full mathematical proof see in attached proof.txt
>> and proof_add.txt files. Theoretical results are also confirmed
>> by experimental counting of the operations.
>>
>> Diff between current and new implementation of Quicksort
>> --------------------------------------------------------
>>
>> Here is the link to the diff for java.util.Arrays class:
>> http://cr.openjdk.java.net/~alanb/6880672/webrev.00
>>
>> If you like to look and play with new algorithm,
>> please, take attached class DualPivotQuicksort.java
>>
>> Feedback
>> --------
>>
>> Also I'd like to share a feedback from Joshua Bloch and
>> Jon Bentley who spent a lot of time investigating this
>> algorithm, who gave me many advices and tips how to
>> make new Quicksort better.
>>
>> -------- Original Message --------
>> Subject: Re: Integration of new Dual-Pivot Quicksort into JDK 7
>> Date: Thu, 10 Sep 2009 07:20:11 -0700
>> From: Joshua Bloch
>>
>> Jon also says that Vladimir should make every reasonable improvement to
>> the basic method before checking in the code. In his words, "It would be
>> horrible to put the new code into the library, and then have someone
>> else come along and speed it up by another 20% by using standard
>> techniques." I believe it's not unlikely that this code may end up
>> getting ported to many languages and widely deployed in much the manner
>> of Bentley and McIlroy's fine sort (which is nearing 20 successful years
>> in the field). Jon will help Vladimir do this.
>>
>> -------- Original Message --------
>> Subject: Dual-Pivot Quicksort: Next Steps
>> Date: Wed, 09 Sep 2009 15:02:25 -0400
>> From: Jon Bentley
>>
>> Vladimir, Josh,
>> I *finally* feel like I understand what is going on. Now that I
>> (think that) I see it, it seems straightforward and obvious.
>> Tony Hoare developed Quicksort in the early 1960s. I was very
>> proud to make minor contributions to a particularly clean (binary)
>> quicksort in the mid 1980s, to a relatively straightforward, industrial
>> strength Quicksort with McIlroy in the early 1990s, and then to
>> algorithms and data structures for strings with Sedgewick in the mid
>> 1990s.
>> I think that Vladimir's contributions to Quicksort go way beyond
>> anything that I've ever done, and rank up there with Hoare's original
>> design and Sedgewick's analysis. I feel so privileged to play a very,
>> very minor role in helping Vladimir with the most excellent work!
>>
>> -----------------------------------------------
>>
>> Let me know, if you have any questions/comments.
>>
>> Thank you,
>> Vladimir
From iaroslavski at mail.ru Fri Sep 11 06:43:36 2009
From: iaroslavski at mail.ru (Vladimir Iaroslavski)
Date: Fri, 11 Sep 2009 17:43:36 +0400
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
Quicksort
In-Reply-To: <4AAA5158.8060202@univ-mlv.fr>
References: <4AAA27EA.8090902@Sun.COM> <4AAA5158.8060202@univ-mlv.fr>
Message-ID: <4AAA5408.8050706@mail.ru>
sure, will do it
R?mi Forax wrote:
> just my two cents :
> In the loop tagged "sorting" and "equals element",
> a[k] can be stored in a local variable to avoid to recompute it several
> time.
>
> The algorithm use two constants: 37 and 13,
> I think they should be declared as private static final.
>
> R?mi
>
> Le 11/09/2009 12:35, Vladimir Yaroslavskiy a ?crit :
>> Hello All,
>>
>> I'd like to share with you new Dual-Pivot Quicksort which is
>> faster than the known implementations (theoretically and
>> experimental). I'd like to propose to replace the JDK's
>> Quicksort implementation by new one.
>>
>> Description
>> -----------
>> The classical Quicksort algorithm uses the following scheme:
>>
>> 1. Pick an element P, called a pivot, from the array.
>> 2. Reorder the array so that all elements, which are less than
>> the pivot, come before the pivot and all elements greater than
>> the pivot come after it (equal values can go either way). After
>> this partitioning, the pivot element is in its final position.
>> 3. Recursively sort the sub-array of lesser elements and the
>> sub-array of greater elements.
>>
>> The invariant of classical Quicksort is:
>>
>> [ <= p | >= p ]
>>
>> There are several modifications of the schema:
>>
>> [ < p | = p | > p ] or [ = p | < p | > p | = p ]
>>
>> But all of them use *one* pivot.
>>
>> The new Dual-Pivot Quicksort uses *two* pivots elements in this manner:
>>
>> 1. Pick an elements P1, P2, called pivots from the array.
>> 2. Assume that P1 <= P2, otherwise swap it.
>> 3. Reorder the array into three parts: those less than the smaller
>> pivot, those larger than the larger pivot, and in between are
>> those elements between (or equal to) the two pivots.
>> 4. Recursively sort the sub-arrays.
>>
>> The invariant of the Dual-Pivot Quicksort is:
>>
>> [ < P1 | P1 <= & <= P2 } > P2 ]
>>
>> The new Quicksort is faster than current implementation of Quicksort
>> in JDK (L. Bentley and M. Douglas McIlroy) and classical Quicksort.
>>
>> The full description of the Dual-Pivot Quicksort you can find
>> on my page: http://iaroslavski.narod.ru/quicksort
>>
>> Performance tests
>> -----------------
>> Here is result of running on different types of input data:
>>
>> Client VM all 85% organ 0..1 0..4
>> random ascend descend equal equal pipes random 010101
>> random
>> Dual-Pivot: 16.83 5.31 5.47 0.35 0.68 10.59 1.06 1.02 2.18
>> Bentley's: 19.77 9.08 10.13 0.63 1.12 13.22 1.63 1.08 2.49
>>
>> Server VM all 85% organ 0..1 0..4
>> random ascend descend equal equal pipes random 010101
>> random
>> Dual-Pivot: 23.94 6.68 6.63 0.43 0.62 17.14 1.42 1.96 3.41
>> Bentley's: 25.20 10.18 10.32 2.07 1.33 16.72 2.95 1.82 3.39
>>
>> The a lot of other tests have been run under client and server mode.
>> The most interesting is BentleyBasher test framework. It runs battery
>> of tests for all cases:
>>
>> { 100, 1000, 10000, 1000000 } x
>> { sawtooth, rand, stagger, plateau, shuffle } x
>> { ident, reverse, reverse_front, reverse_back, sort, dither}
>>
>> where
>>
>> 100, ... , 1000000 - array length
>>
>> sawtooth: x[i] =i%m
>> rand: x[i] = rand() % m
>> stagger: x[i] = (i*m + i) % n
>> plateau: x[i] = min(i, m)
>> shuffle: x[i] = rand()%m? (j+=2): (k+=2)
>>
>> ident(x) - a copy of x
>> reverse(x, 0, n) - reversed copy
>> reverse_front(x, 0, n/2) - front half reversed
>> reverse_back(x, n/2, n) - back half reversed
>> sort(x) - an ordered copy
>> dither(x) - add i%5 to x[i]
>>
>> Here is the result of execution:
>> Server VM:
>> http://spreadsheets.google.com/pub?key=t_EAWUkQ4mD3BIbOv8Fa-AQ&output=html
>>
>> Client VM:
>> http://spreadsheets.google.com/pub?key=tdiMo8xleTxd23nKUObcz0Q&single=true&gid=0&output=html
>>
>>
>> Mathematical investigations
>> ---------------------------
>> It is proved that for the Dual-Pivot Quicksort the average number of
>> comparisons is 2*n*ln(n), the average number of swaps is 0.8*n*ln(n),
>> whereas classical Quicksort algorithm has 2*n*ln(n) and 1*n*ln(n)
>> respectively. Full mathematical proof see in attached proof.txt
>> and proof_add.txt files. Theoretical results are also confirmed
>> by experimental counting of the operations.
>>
>> Diff between current and new implementation of Quicksort
>> --------------------------------------------------------
>>
>> Here is the link to the diff for java.util.Arrays class:
>> http://cr.openjdk.java.net/~alanb/6880672/webrev.00
>>
>> If you like to look and play with new algorithm,
>> please, take attached class DualPivotQuicksort.java
>>
>> Feedback
>> --------
>>
>> Also I'd like to share a feedback from Joshua Bloch and
>> Jon Bentley who spent a lot of time investigating this
>> algorithm, who gave me many advices and tips how to
>> make new Quicksort better.
>>
>> -------- Original Message --------
>> Subject: Re: Integration of new Dual-Pivot Quicksort into JDK 7
>> Date: Thu, 10 Sep 2009 07:20:11 -0700
>> From: Joshua Bloch
>>
>> Jon also says that Vladimir should make every reasonable improvement to
>> the basic method before checking in the code. In his words, "It would be
>> horrible to put the new code into the library, and then have someone
>> else come along and speed it up by another 20% by using standard
>> techniques." I believe it's not unlikely that this code may end up
>> getting ported to many languages and widely deployed in much the manner
>> of Bentley and McIlroy's fine sort (which is nearing 20 successful years
>> in the field). Jon will help Vladimir do this.
>>
>> -------- Original Message --------
>> Subject: Dual-Pivot Quicksort: Next Steps
>> Date: Wed, 09 Sep 2009 15:02:25 -0400
>> From: Jon Bentley
>>
>> Vladimir, Josh,
>> I *finally* feel like I understand what is going on. Now that I
>> (think that) I see it, it seems straightforward and obvious.
>> Tony Hoare developed Quicksort in the early 1960s. I was very
>> proud to make minor contributions to a particularly clean (binary)
>> quicksort in the mid 1980s, to a relatively straightforward, industrial
>> strength Quicksort with McIlroy in the early 1990s, and then to
>> algorithms and data structures for strings with Sedgewick in the mid
>> 1990s.
>> I think that Vladimir's contributions to Quicksort go way beyond
>> anything that I've ever done, and rank up there with Hoare's original
>> design and Sedgewick's analysis. I feel so privileged to play a very,
>> very minor role in helping Vladimir with the most excellent work!
>>
>> -----------------------------------------------
>>
>> Let me know, if you have any questions/comments.
>>
>> Thank you,
>> Vladimir
From Ulf.Zibis at gmx.de Fri Sep 11 06:47:16 2009
From: Ulf.Zibis at gmx.de (Ulf Zibis)
Date: Fri, 11 Sep 2009 15:47:16 +0200
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
Quicksort
In-Reply-To: <4AAA5158.8060202@univ-mlv.fr>
References: <4AAA27EA.8090902@Sun.COM> <4AAA5158.8060202@univ-mlv.fr>
Message-ID: <4AAA54E4.7090107@gmx.de>
Am 11.09.2009 15:32, R?mi Forax schrieb:
> just my two cents :
> In the loop tagged "sorting" and "equals element",
> a[k] can be stored in a local variable to avoid to recompute it
> several time.
I add 2 cents more:
Doesn't HotSpot see this, and optimize accordingly?
IMO: It's time that javac should do such optimization, as there are less
optimized VM's in the world.
-Ulf
>
> The algorithm use two constants: 37 and 13,
> I think they should be declared as private static final.
>
> R?mi
>
>
> Le 11/09/2009 12:35, Vladimir Yaroslavskiy a ?crit :
>> Hello All,
>>
>> I'd like to share with you new Dual-Pivot Quicksort which is
>> faster than the known implementations (theoretically and
>> experimental). I'd like to propose to replace the JDK's
>> Quicksort implementation by new one.
>>
>> Description
>> -----------
>> The classical Quicksort algorithm uses the following scheme:
>>
>> 1. Pick an element P, called a pivot, from the array.
>> 2. Reorder the array so that all elements, which are less than
>> the pivot, come before the pivot and all elements greater than
>> the pivot come after it (equal values can go either way). After
>> this partitioning, the pivot element is in its final position.
>> 3. Recursively sort the sub-array of lesser elements and the
>> sub-array of greater elements.
>>
>> The invariant of classical Quicksort is:
>>
>> [ <= p | >= p ]
>>
>> There are several modifications of the schema:
>>
>> [ < p | = p | > p ] or [ = p | < p | > p | = p ]
>>
>> But all of them use *one* pivot.
>>
>> The new Dual-Pivot Quicksort uses *two* pivots elements in this manner:
>>
>> 1. Pick an elements P1, P2, called pivots from the array.
>> 2. Assume that P1 <= P2, otherwise swap it.
>> 3. Reorder the array into three parts: those less than the smaller
>> pivot, those larger than the larger pivot, and in between are
>> those elements between (or equal to) the two pivots.
>> 4. Recursively sort the sub-arrays.
>>
>> The invariant of the Dual-Pivot Quicksort is:
>>
>> [ < P1 | P1 <= & <= P2 } > P2 ]
>>
>> The new Quicksort is faster than current implementation of Quicksort
>> in JDK (L. Bentley and M. Douglas McIlroy) and classical Quicksort.
>>
>> The full description of the Dual-Pivot Quicksort you can find
>> on my page: http://iaroslavski.narod.ru/quicksort
>>
>> Performance tests
>> -----------------
>> Here is result of running on different types of input data:
>>
>> Client VM all 85% organ 0..1 0..4
>> random ascend descend equal equal pipes random 010101
>> random
>> Dual-Pivot: 16.83 5.31 5.47 0.35 0.68 10.59 1.06 1.02 2.18
>> Bentley's: 19.77 9.08 10.13 0.63 1.12 13.22 1.63 1.08
>> 2.49
>>
>> Server VM all 85% organ 0..1 0..4
>> random ascend descend equal equal pipes random 010101
>> random
>> Dual-Pivot: 23.94 6.68 6.63 0.43 0.62 17.14 1.42 1.96 3.41
>> Bentley's: 25.20 10.18 10.32 2.07 1.33 16.72 2.95 1.82
>> 3.39
>>
>> The a lot of other tests have been run under client and server mode.
>> The most interesting is BentleyBasher test framework. It runs battery
>> of tests for all cases:
>>
>> { 100, 1000, 10000, 1000000 } x
>> { sawtooth, rand, stagger, plateau, shuffle } x
>> { ident, reverse, reverse_front, reverse_back, sort, dither}
>>
>> where
>>
>> 100, ... , 1000000 - array length
>>
>> sawtooth: x[i] =i%m
>> rand: x[i] = rand() % m
>> stagger: x[i] = (i*m + i) % n
>> plateau: x[i] = min(i, m)
>> shuffle: x[i] = rand()%m? (j+=2): (k+=2)
>>
>> ident(x) - a copy of x
>> reverse(x, 0, n) - reversed copy
>> reverse_front(x, 0, n/2) - front half reversed
>> reverse_back(x, n/2, n) - back half reversed
>> sort(x) - an ordered copy
>> dither(x) - add i%5 to x[i]
>>
>> Here is the result of execution:
>> Server VM:
>> http://spreadsheets.google.com/pub?key=t_EAWUkQ4mD3BIbOv8Fa-AQ&output=html
>>
>> Client VM:
>> http://spreadsheets.google.com/pub?key=tdiMo8xleTxd23nKUObcz0Q&single=true&gid=0&output=html
>>
>>
>> Mathematical investigations
>> ---------------------------
>> It is proved that for the Dual-Pivot Quicksort the average number of
>> comparisons is 2*n*ln(n), the average number of swaps is 0.8*n*ln(n),
>> whereas classical Quicksort algorithm has 2*n*ln(n) and 1*n*ln(n)
>> respectively. Full mathematical proof see in attached proof.txt
>> and proof_add.txt files. Theoretical results are also confirmed
>> by experimental counting of the operations.
>>
>> Diff between current and new implementation of Quicksort
>> --------------------------------------------------------
>>
>> Here is the link to the diff for java.util.Arrays class:
>> http://cr.openjdk.java.net/~alanb/6880672/webrev.00
>>
>> If you like to look and play with new algorithm,
>> please, take attached class DualPivotQuicksort.java
>>
>> Feedback
>> --------
>>
>> Also I'd like to share a feedback from Joshua Bloch and
>> Jon Bentley who spent a lot of time investigating this
>> algorithm, who gave me many advices and tips how to
>> make new Quicksort better.
>>
>> -------- Original Message --------
>> Subject: Re: Integration of new Dual-Pivot Quicksort into JDK 7
>> Date: Thu, 10 Sep 2009 07:20:11 -0700
>> From: Joshua Bloch
>>
>> Jon also says that Vladimir should make every reasonable improvement to
>> the basic method before checking in the code. In his words, "It would be
>> horrible to put the new code into the library, and then have someone
>> else come along and speed it up by another 20% by using standard
>> techniques." I believe it's not unlikely that this code may end up
>> getting ported to many languages and widely deployed in much the manner
>> of Bentley and McIlroy's fine sort (which is nearing 20 successful years
>> in the field). Jon will help Vladimir do this.
>>
>> -------- Original Message --------
>> Subject: Dual-Pivot Quicksort: Next Steps
>> Date: Wed, 09 Sep 2009 15:02:25 -0400
>> From: Jon Bentley
>>
>> Vladimir, Josh,
>> I *finally* feel like I understand what is going on. Now that I
>> (think that) I see it, it seems straightforward and obvious.
>> Tony Hoare developed Quicksort in the early 1960s. I was very
>> proud to make minor contributions to a particularly clean (binary)
>> quicksort in the mid 1980s, to a relatively straightforward, industrial
>> strength Quicksort with McIlroy in the early 1990s, and then to
>> algorithms and data structures for strings with Sedgewick in the mid
>> 1990s.
>> I think that Vladimir's contributions to Quicksort go way beyond
>> anything that I've ever done, and rank up there with Hoare's original
>> design and Sedgewick's analysis. I feel so privileged to play a very,
>> very minor role in helping Vladimir with the most excellent work!
>>
>> -----------------------------------------------
>>
>> Let me know, if you have any questions/comments.
>>
>> Thank you,
>> Vladimir
>
>
From iaroslavski at mail.ru Fri Sep 11 07:27:25 2009
From: iaroslavski at mail.ru (Vladimir Iaroslavski)
Date: Fri, 11 Sep 2009 18:27:25 +0400
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
Quicksort
In-Reply-To: <4AAA54E4.7090107@gmx.de>
References: <4AAA27EA.8090902@Sun.COM> <4AAA5158.8060202@univ-mlv.fr>
<4AAA54E4.7090107@gmx.de>
Message-ID: <4AAA5E4D.6050909@mail.ru>
Hi,
I've tried to use local variable int ak = a[k] in the loop
and not found saving of time. There are 3 occurrences of a[k]
in each loop, but only two can be changed because third
comes after line: swap(a, k, great--);
As summary: I'm changing only hardcoded constant by
private static final variables.
Thank you,
Vladimir
Ulf Zibis wrote:
> Am 11.09.2009 15:32, R?mi Forax schrieb:
>> just my two cents :
>> In the loop tagged "sorting" and "equals element",
>> a[k] can be stored in a local variable to avoid to recompute it
>> several time.
>
> I add 2 cents more:
> Doesn't HotSpot see this, and optimize accordingly?
> IMO: It's time that javac should do such optimization, as there are less
> optimized VM's in the world.
>
> -Ulf
>
>
>>
>> The algorithm use two constants: 37 and 13,
>> I think they should be declared as private static final.
>>
>> R?mi
>>
>>
>> Le 11/09/2009 12:35, Vladimir Yaroslavskiy a ?crit :
>>> Hello All,
>>>
>>> I'd like to share with you new Dual-Pivot Quicksort which is
>>> faster than the known implementations (theoretically and
>>> experimental). I'd like to propose to replace the JDK's
>>> Quicksort implementation by new one.
>>>
>>> Description
>>> -----------
>>> The classical Quicksort algorithm uses the following scheme:
>>>
>>> 1. Pick an element P, called a pivot, from the array.
>>> 2. Reorder the array so that all elements, which are less than
>>> the pivot, come before the pivot and all elements greater than
>>> the pivot come after it (equal values can go either way). After
>>> this partitioning, the pivot element is in its final position.
>>> 3. Recursively sort the sub-array of lesser elements and the
>>> sub-array of greater elements.
>>>
>>> The invariant of classical Quicksort is:
>>>
>>> [ <= p | >= p ]
>>>
>>> There are several modifications of the schema:
>>>
>>> [ < p | = p | > p ] or [ = p | < p | > p | = p ]
>>>
>>> But all of them use *one* pivot.
>>>
>>> The new Dual-Pivot Quicksort uses *two* pivots elements in this manner:
>>>
>>> 1. Pick an elements P1, P2, called pivots from the array.
>>> 2. Assume that P1 <= P2, otherwise swap it.
>>> 3. Reorder the array into three parts: those less than the smaller
>>> pivot, those larger than the larger pivot, and in between are
>>> those elements between (or equal to) the two pivots.
>>> 4. Recursively sort the sub-arrays.
>>>
>>> The invariant of the Dual-Pivot Quicksort is:
>>>
>>> [ < P1 | P1 <= & <= P2 } > P2 ]
>>>
>>> The new Quicksort is faster than current implementation of Quicksort
>>> in JDK (L. Bentley and M. Douglas McIlroy) and classical Quicksort.
>>>
>>> The full description of the Dual-Pivot Quicksort you can find
>>> on my page: http://iaroslavski.narod.ru/quicksort
>>>
>>> Performance tests
>>> -----------------
>>> Here is result of running on different types of input data:
>>>
>>> Client VM all 85% organ 0..1 0..4
>>> random ascend descend equal equal pipes random 010101
>>> random
>>> Dual-Pivot: 16.83 5.31 5.47 0.35 0.68 10.59 1.06 1.02 2.18
>>> Bentley's: 19.77 9.08 10.13 0.63 1.12 13.22 1.63 1.08
>>> 2.49
>>>
>>> Server VM all 85% organ 0..1 0..4
>>> random ascend descend equal equal pipes random 010101
>>> random
>>> Dual-Pivot: 23.94 6.68 6.63 0.43 0.62 17.14 1.42 1.96 3.41
>>> Bentley's: 25.20 10.18 10.32 2.07 1.33 16.72 2.95 1.82
>>> 3.39
>>>
>>> The a lot of other tests have been run under client and server mode.
>>> The most interesting is BentleyBasher test framework. It runs battery
>>> of tests for all cases:
>>>
>>> { 100, 1000, 10000, 1000000 } x
>>> { sawtooth, rand, stagger, plateau, shuffle } x
>>> { ident, reverse, reverse_front, reverse_back, sort, dither}
>>>
>>> where
>>>
>>> 100, ... , 1000000 - array length
>>>
>>> sawtooth: x[i] =i%m
>>> rand: x[i] = rand() % m
>>> stagger: x[i] = (i*m + i) % n
>>> plateau: x[i] = min(i, m)
>>> shuffle: x[i] = rand()%m? (j+=2): (k+=2)
>>>
>>> ident(x) - a copy of x
>>> reverse(x, 0, n) - reversed copy
>>> reverse_front(x, 0, n/2) - front half reversed
>>> reverse_back(x, n/2, n) - back half reversed
>>> sort(x) - an ordered copy
>>> dither(x) - add i%5 to x[i]
>>>
>>> Here is the result of execution:
>>> Server VM:
>>> http://spreadsheets.google.com/pub?key=t_EAWUkQ4mD3BIbOv8Fa-AQ&output=html
>>>
>>> Client VM:
>>> http://spreadsheets.google.com/pub?key=tdiMo8xleTxd23nKUObcz0Q&single=true&gid=0&output=html
>>>
>>>
>>> Mathematical investigations
>>> ---------------------------
>>> It is proved that for the Dual-Pivot Quicksort the average number of
>>> comparisons is 2*n*ln(n), the average number of swaps is 0.8*n*ln(n),
>>> whereas classical Quicksort algorithm has 2*n*ln(n) and 1*n*ln(n)
>>> respectively. Full mathematical proof see in attached proof.txt
>>> and proof_add.txt files. Theoretical results are also confirmed
>>> by experimental counting of the operations.
>>>
>>> Diff between current and new implementation of Quicksort
>>> --------------------------------------------------------
>>>
>>> Here is the link to the diff for java.util.Arrays class:
>>> http://cr.openjdk.java.net/~alanb/6880672/webrev.00
>>>
>>> If you like to look and play with new algorithm,
>>> please, take attached class DualPivotQuicksort.java
>>>
>>> Feedback
>>> --------
>>>
>>> Also I'd like to share a feedback from Joshua Bloch and
>>> Jon Bentley who spent a lot of time investigating this
>>> algorithm, who gave me many advices and tips how to
>>> make new Quicksort better.
>>>
>>> -------- Original Message --------
>>> Subject: Re: Integration of new Dual-Pivot Quicksort into JDK 7
>>> Date: Thu, 10 Sep 2009 07:20:11 -0700
>>> From: Joshua Bloch
>>>
>>> Jon also says that Vladimir should make every reasonable improvement to
>>> the basic method before checking in the code. In his words, "It would be
>>> horrible to put the new code into the library, and then have someone
>>> else come along and speed it up by another 20% by using standard
>>> techniques." I believe it's not unlikely that this code may end up
>>> getting ported to many languages and widely deployed in much the manner
>>> of Bentley and McIlroy's fine sort (which is nearing 20 successful years
>>> in the field). Jon will help Vladimir do this.
>>>
>>> -------- Original Message --------
>>> Subject: Dual-Pivot Quicksort: Next Steps
>>> Date: Wed, 09 Sep 2009 15:02:25 -0400
>>> From: Jon Bentley
>>>
>>> Vladimir, Josh,
>>> I *finally* feel like I understand what is going on. Now that I
>>> (think that) I see it, it seems straightforward and obvious.
>>> Tony Hoare developed Quicksort in the early 1960s. I was very
>>> proud to make minor contributions to a particularly clean (binary)
>>> quicksort in the mid 1980s, to a relatively straightforward, industrial
>>> strength Quicksort with McIlroy in the early 1990s, and then to
>>> algorithms and data structures for strings with Sedgewick in the mid
>>> 1990s.
>>> I think that Vladimir's contributions to Quicksort go way beyond
>>> anything that I've ever done, and rank up there with Hoare's original
>>> design and Sedgewick's analysis. I feel so privileged to play a very,
>>> very minor role in helping Vladimir with the most excellent work!
>>>
>>> -----------------------------------------------
>>>
>>> Let me know, if you have any questions/comments.
>>>
>>> Thank you,
>>> Vladimir
From straxus at gmail.com Fri Sep 11 07:39:11 2009
From: straxus at gmail.com (Ryan Slobojan)
Date: Fri, 11 Sep 2009 10:39:11 -0400
Subject: What methods should go into a java.util.Objects class in JDK 7?
Message-ID: <9F2CEE46-89BD-407C-B68C-00B819ABAE73@gmail.com>
> Hello.
>
> For JDK 7, I think it is high-time the platform included a class like
> java.util.Objects to hold commonly-written utility methods. For
> example, a two-argument static equals method that returned true if
> both
> arguments are null, returns false is one argument is null, and
> otherwise
> returns the result of calling arg1.equals(arg2) (6797535 "Add shared
> two argument static equals method to the platform").
>
> A static hashCode method returning 0 for null and the value of
> arg.hashCode() has also been suggested.
>
> A set of
>
> static int compareTo(int, int)
> static int compareTo(long, long)
> ....
>
> methods probably belongs somewhere in the platform too.
>
> What other utility methods would have broad enough use and
> applicability
> to go into a common java.util class?
>
> -Joe
Hi,
As a note, I've just put up an article about this on InfoQ soliciting
feedback from the wider Java community:
http://www.infoq.com/news/2009/09/jdk7-java-utils-object
Thanks,
Ryan Slobojan
From Mandy.Chung at Sun.COM Fri Sep 11 11:14:10 2009
From: Mandy.Chung at Sun.COM (Mandy Chung)
Date: Fri, 11 Sep 2009 11:14:10 -0700
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4A9EC69F.6000509@sun.com>
References: <4A9EC69F.6000509@sun.com>
Message-ID: <4AAA9372.3070200@Sun.com>
Alan, David, Remi, Iris,
Thanks for the review and the revised webrev at:
http://cr.openjdk.java.net/~mchung/6878481/webrev.01/
I incorporate the comments you have and minimize the number of
System.nanoTime() calls and also remove the number of opened jars perf
counters. Please let me know if you have further comments.
I added /usepmtimer in boot.ini on my system (thanks David for the
suggestion). The performance run shows insignificant perf impact.
Windows XP SP 3 Intel Core 2 CPU @3.0Ghz, 1.96 GB RAM
==============================================================================
mchung.baseline.server:
Benchmark Samples Mean Stdev Geomean
Weight
reference_server 20 7615.99 123.75
jetstream 20 131.78 29.37 0.10
scimark 20 618.00 6.35 0.15
specjbb2000 20 93732.74 540.37 0.15
specjbb2005 20 43062.47 182.61 0.25
specjvm98 20 575.46 1.94 0.15
volano25 20 46464.90 1164.30 0.20
==============================================================================
mchung.perfcounters:
Benchmark Samples Mean Stdev %Diff P
Significant
reference_server 20 7575.65 34.80 -0.53 0.174
*
jetstream 20 124.67 1.88 -5.40 0.293
*
scimark 20 619.47 5.19 0.24 0.426
*
specjbb2000 20 93573.49 663.77 -0.17 0.411
*
specjbb2005 20 43095.83 190.47 0.08 0.575
*
specjvm98 20 576.44 3.05 0.17 0.233
*
volano25 20 46064.85 1034.46 -0.86 0.258
*
==============================================================================
* - Not Significant: A non-zero %Diff for the mean could be noise. If the
%Diff is 0, an actual difference may still exist. In either case,
more
samples would be needed to detect an actual difference in sample
means.
Thanks
Mandy
Mandy Chung wrote:
> This is related to 6857194: Add hotspot new perf counters to aid class
> loading performance measurement.
>
> It's useful to add performance counters in the library code so that perf
> data from the JDK and VM can be collected and output in a unified way
> (written in the jvmstat shared memory buffer). I add a simple
> sun.misc.PerfCounter class to maintain the list of perf counters for the
> library to use. This fix only instruments the class loading and jar/zip
> to collect simple basic metrics. Additional perf counters will be
> added in the future.
>
> Webrev:
> http://cr.openjdk.java.net/~mchung/6878481/webrev.00/
>
> Thanks
> Mandy
From graehl at isi.edu Fri Sep 11 12:09:48 2009
From: graehl at isi.edu (Jonathan Graehl)
Date: Fri, 11 Sep 2009 19:09:48 +0000 (UTC)
Subject: Replacement of Quicksort in java.util.Arrays with new
=?utf-8?b?RHVhbC1QaXZvdAlRdWlja3NvcnQ=?=
References: <4AAA27EA.8090902@Sun.COM>
Message-ID:
Nice.
> int third = len / div;
>
> // "medians"
> int m1 = left + third;
> int m2 = right - third;
>
> if (m1 <= left) {
> m1 = left + 1;
> }
> if (m2 >= right) {
> m2 = right - 1;
> }
I'd suggest this instead:
int third = len / div;
third=third>0?third:1;
// "medians"
int m1 = left + third;
int m2 = right - third;
From lgeller at feedroom.com Fri Sep 11 12:14:16 2009
From: lgeller at feedroom.com (Leonid Geller)
Date: Fri, 11 Sep 2009 19:14:16 +0000 (UTC)
Subject: Replacement of Quicksort in java.util.Arrays with new
=?utf-8?b?RHVhbC1QaXZvdAlRdWlja3NvcnQ=?=
References: <4AAA27EA.8090902@Sun.COM>
Message-ID:
As an observation, why not expand the new algorithm to N-Pivot
where N = round(ln(array length)).
This should lower the average sort cost even lower.
From Joe.Darcy at Sun.COM Fri Sep 11 13:04:50 2009
From: Joe.Darcy at Sun.COM (Joseph D. Darcy)
Date: Fri, 11 Sep 2009 13:04:50 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA993E3.20506@sun.com>
References: <4AA812D0.309@sun.com>
<1ccfd1c10909091715w431554fldf2e9f7f7ce17599@mail.gmail.com>
<4AA84AEA.7050008@sun.com>
<1ccfd1c10909091942p41683f0bl46d901126c61e8d5@mail.gmail.com>
<4AA86C11.7080409@sun.com> <4AA9823D.60507@sun.com>
<1ccfd1c10909101640r465ac11albc414bf6573c215b@mail.gmail.com>
<4AA99030.8040904@sun.com>
<1ccfd1c10909101656sdcc6643uf4ff06cf76b05f30@mail.gmail.com>
<4AA993E3.20506@sun.com>
Message-ID: <4AAAAD62.70805@sun.com>
Joe Darcy wrote:
> Martin Buchholz wrote:
>> On Thu, Sep 10, 2009 at 16:48, Joe Darcy wrote:
>>> Martin Buchholz wrote:
>>>> On Thu, Sep 10, 2009 at 15:48, Joe Darcy wrote:
>>>>
>>>>> On 09/09/09 08:01 PM, Joe Darcy wrote:
>>>>> I noticed that the Character class should be similarly tightened:
>>>>>
>>>> Right you are. Webrev regenerated.
>>>>
>>>> Martin
>>>>
>>> One typo inherited from my earlier suggestion (mea culpa!): as the
>>> Character
>>> class is not a java.lang.Number, it does not have an intValue method.
>>> Character does have a charValue method and referring to that method
>>> in the
>>> specification of hashCode should suffice.
>>>
>>> I approve this change going back with the correction to the
>>> Character class.
>>
>> Oops! so many little mistakes. Webrev regenerated.
>
> Looks good.
>
> -Joe
>
Martin,
Just to be clear, I approve this change of tightening the hashCode
specifications for Byte, Short, and Character going back into JDK 7.
Thanks,
-Joe
From Joe.Darcy at Sun.COM Fri Sep 11 13:07:31 2009
From: Joe.Darcy at Sun.COM (Joseph D. Darcy)
Date: Fri, 11 Sep 2009 13:07:31 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA857A8.1010700@sun.com>
References: <4AA812D0.309@sun.com>
<4b4f45e00909091654t7891127bh24cc1df9c38bec7d@mail.gmail.com>
<1ccfd1c10909091704i1f4f75dfm4ae1988d0903cd99@mail.gmail.com>
<4AA849B1.1090005@sun.com>
<1ccfd1c10909091808s772d454epfe7a1057ccb9ca9f@mail.gmail.com>
<4AA857A8.1010700@sun.com>
Message-ID: <4AAAAE03.5010002@sun.com>
Joe Darcy wrote:
> Martin Buchholz wrote:
>> On Wed, Sep 9, 2009 at 17:34, Joe Darcy wrote:
>>
>>> Martin Buchholz wrote:
>>>
>>>> Google would like to contribute implementations of the methods below.
>>>> Here's a webrev:
>>>>
>>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/compare/
>>>>
>>>>
>>> Wherever these compare methods go, into the N wrapper classes or into
>>> java.util.Compare,
>>>
>>
>> Given the precedent of Double.compare(double,double) and
>> Float.compare(float,float), I think the compare methods need
>> to go into the primitive wrapper classes.
>>
>> I think the instance method compareTo should be rewritten
>>
>>> in terms of the new static compare. In particular,
>>>
>>> 422 public int compareTo(Short anotherShort) {
>>> 423 return this.value - anotherShort.value;
>>> 424 }
>>>
>>> should get replaced with
>>>
>>> 422 public int compareTo(Short anotherShort) {
>>> 423 return compare(this.value, anotherShort.value);
>>> 424 }
>>>
>>
>> As you wish.
>>
>> Webrev updated.
>> http://cr.openjdk.java.net/~martin/webrevs/openjdk7/compare/
>>
>
> I approve these changes in an engineering sense; they are a fix for bug
>
> 6582946 Add suite of compare(T, T) methods for ints, longs etc
>
>>> This consolidates the comparison logic in a single method per type and
>>> reuses the existing implementation tests.
>>>
>>
>>
>>
>>> (Later in JDK 7, a set of "unsignedCompare" methods are possible
>>> additions
>>> to the platform too.)
>>>
>>>
>>>> We have tests (not yet included) but they would need to be
>>>> jtreg-ified.
>>>> Is there any junit support yet for jtreg?
>>>>
>>>>
>>> No.
>>>
>>> Once the home of the new methods is determined, I'll file a ccc
>>> request so
>>> these changes can go back after any additional testing is settled.
>>>
>>
>> There's not a whole lot of testing work here, but we will provide it if
>> it CCC is approved.
>>
>>
> I've created a ccc for this and will let you know when it is approved.
>
> -Joe
>
Martin,
The paperwork has been handled; I approve your set of compare methods
going back.
Thanks,
-Joe
From Xueming.Shen at Sun.COM Fri Sep 11 14:05:04 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Fri, 11 Sep 2009 14:05:04 -0700
Subject: Codereview request for #6881337
Message-ID: <4AAABB80.9060203@sun.com>
Martin,
Would you please help review the change?
http://cr.openjdk.java.net/~sherman/6881337/webrev
Somehow the development team messed up our change in ZipEntry.java in
b64 in their putback. This fix is to restore the api/impl.
The bug report probably has not been pushed out yet. Here is the copy/paste.
Thanks,
Sherman
*Synopsis*: ZipEntry.setComment() was accidentally changed back to old spec/impl in jdk7-b64
CrPrint: http://bt2ws.central.sun.com/CrPrint?id=6881337
Monaco: http://monaco.sfbay.sun.com/detail.jsf?cr=6881337
*Change Request ID*: 6881337
*Synopsis*: ZipEntry.setComment() was accidentally changed back to old spec/impl in jdk7-b64
Product: java
Category: java
Subcategory: classes_util_jarzip
Type: Defect
Subtype:
Status: 5-Cause Known
Substatus:
Priority: 3-Medium
Introduced In Release:
Introduced In Build:
Responsible Manager: frances.ho at sun.com
Responsible Engineer: xueming.shen at sun.com
Initial Evaluator: java_util_bugs at sun.com
Keywords:
=== *Description* ============================================================
Yes - I agree that this is part of a mismerge made by me, while getting the change needed for kernel to setTime, I also took 1.6 version of setComment, which was unrelated.
You should restore setComment to its previous form.
/Andy
Xueming Shen wrote:
> > It appears the deployment team might have "accidentally" changed it back to the old spec/impl
> > while fixing their problem #6625963 "3 months ago", see the diff at
> >
> > http://hg.openjdk.java.net/jdk7/jdk7/jdk/diff/9053bcc8eef0/src/share/classes/java/util/zip/ZipEntry.java
> >
> > The changeset summary/tag in the ZipEntry.java is
> >
> > 6797688: Umbrella: Merge all JDK 6u4 - 6u12 deployment code into JDK7
> >
> > (I'm not sure if this is accurate or not)
> >
> > Andy, you are listed as the owner of this fix, can you confirm?
> >
> > Sherman
> >
> > Dmitry Miltsov wrote:
> >
>
>> >> Hello Xueming,
>> >>
>> >> According to CCC 4244499 the following assertion was removed for the spec
>> >> on java.util.zip.ZipEntry since b57:
>> >> "Throws: IllegalArgumentException - if the length of the specified comment
>> >> string is greater than 0xFFFF bytes"
>> >>
>> >> However since b64 this assertion has returned to the ZipEntry specification and
>> >> is contradicting to the new assertion "If the length of the specified comment
>> >> string is greater than 0xFFFF bytes after encoding, only the first 0xFFFF bytes
>> >> are output to the ZIP file entry."
>> >>
>> >> Could you please clarify the issue ? Also it seems that the returned assertion
>> >> about IllegalArgumentException has not been approved through the CCC process.
>> >>
>> >> Thanks,
>> >> Dmitry
>>
*** (#1 of 1): 2009-09-11 18:48:33 GMT+00:00 xueming.shen at sun.com
=== *Public Comments* ========================================================
=== *Comments* ===============================================================
=== *Evaluation* =============================================================
Need to restore the spec and the implementation.
*** (#1 of 1): 2009-09-11 18:49:36 GMT+00:00 xueming.shen at sun.com
=== *Suggested Fix* ==========================================================
=== *Workaround* =============================================================
=== *Justification* ==========================================================
Priority changed from [] to [3-Medium]
regression
xueming.shen at sun.com 2009-09-11 18:48:33 GMT
*** (#1 of 1): 2009-09-11 18:48:33 GMT+00:00 xueming.shen at sun.com
=== *Additional Details* =====================================================
Targeted Release: 7
Commit To Fix In Build:
Fixed In Build:
Integrated In Build:
Verified In Build:
See Also:
Duplicate of:
Hooks:
Hook1:
Hook2:
Hook3:
Hook4:
Hook5:
Hook6:
Interest List:
Program Management:
Root Cause:
Is a Security Vulnerability?: No
Fix Affects Documentation: No
Fix Affects Localization: No
Reported by:
=== *History* ================================================================
Date Submitted: 2009-09-11 18:48:32 GMT+00:00
Submitted By: xueming.shen at sun.com
Status Changed Date Updated Updated By
5-Cause Known 2009-09-11 18:49:36 GMT+00:00 xueming.shen at sun.com
=== *Solution* ===============================================================
From martinrb at google.com Fri Sep 11 15:34:33 2009
From: martinrb at google.com (Martin Buchholz)
Date: Fri, 11 Sep 2009 15:34:33 -0700
Subject: Codereview request for #6881337
In-Reply-To: <4AAABB80.9060203@sun.com>
References: <4AAABB80.9060203@sun.com>
Message-ID: <1ccfd1c10909111534j5dee631r17910c19f9c120d1@mail.gmail.com>
Approved.
(It would be nice for all changes to
openjdk7 core libraries to leave a trace on core-libs-dev,
so that they can get review here.)
Martin
On Fri, Sep 11, 2009 at 14:05, Xueming Shen wrote:
> Martin,
>
> Would you please help review the change?
>
> http://cr.openjdk.java.net/~sherman/6881337/webrev
>
> Somehow the development team messed up our change in ZipEntry.java in b64 in
> their putback. This fix is to restore the api/impl.
> The bug report probably has not been pushed out yet. Here is the copy/paste.
>
> Thanks,
> Sherman
>
> *Synopsis*: ZipEntry.setComment() was accidentally changed back to old
> spec/impl in jdk7-b64
>
> CrPrint: http://bt2ws.central.sun.com/CrPrint?id=6881337
> Monaco: http://monaco.sfbay.sun.com/detail.jsf?cr=6881337
>
> ? ?*Change Request ID*: 6881337
>
> *Synopsis*: ZipEntry.setComment() was accidentally changed back to old
> spec/impl in jdk7-b64
>
> ?Product: java
> ?Category: java
> ?Subcategory: classes_util_jarzip
> ?Type: Defect
> ?Subtype: ?Status: 5-Cause Known
> ?Substatus: ?Priority: 3-Medium
> ?Introduced In Release: ?Introduced In Build: ?Responsible Manager:
> frances.ho at sun.com
> ?Responsible Engineer: xueming.shen at sun.com
> ?Initial Evaluator: java_util_bugs at sun.com
> ?Keywords:
> === *Description*
> ============================================================
> Yes - I agree that this is part of a mismerge made by me, while getting the
> change needed for kernel to setTime, I also took 1.6 version of setComment,
> which was unrelated.
> You should restore setComment to its previous form.
>
> /Andy
>
> Xueming Shen wrote:
>
>
>> > It appears the deployment team might have "accidentally" changed it back
>> > to the old spec/impl
>> > while fixing their problem #6625963 "3 months ago", see the diff at
>> >
>> >
>> > http://hg.openjdk.java.net/jdk7/jdk7/jdk/diff/9053bcc8eef0/src/share/classes/java/util/zip/ZipEntry.java
>> >
>> > The changeset summary/tag in the ZipEntry.java is
>> >
>> > 6797688: Umbrella: Merge all JDK 6u4 - 6u12 deployment code into JDK7
>> >
>> > (I'm not sure if this is accurate or not)
>> >
>> > Andy, you are listed as the owner of this fix, can you confirm?
>> >
>> > Sherman
>> >
>> > Dmitry Miltsov wrote:
>> >
>>
>>>
>>> >> Hello Xueming,
>>> >>
>>> >> According to CCC 4244499 the following assertion was removed for the
>>> >> spec
>>> >> on java.util.zip.ZipEntry since b57:
>>> >> "Throws: IllegalArgumentException - if the length of the specified
>>> >> comment
>>> >> string is greater than 0xFFFF bytes"
>>> >>
>>> >> However since b64 this assertion has returned to the ZipEntry
>>> >> specification and
>>> >> is contradicting to the new assertion "If the length of the specified
>>> >> comment
>>> >> string is greater than 0xFFFF bytes after encoding, only the first
>>> >> 0xFFFF bytes
>>> >> are output to the ZIP file entry."
>>> >>
>>> >> Could you please clarify the issue ? Also it seems that the returned
>>> >> assertion
>>> >> about IllegalArgumentException has not been approved through the CCC
>>> >> process.
>>> >>
>>> >> Thanks,
>>> >> Dmitry
>>>
>
> *** (#1 of 1): 2009-09-11 18:48:33 GMT+00:00 xueming.shen at sun.com
>
>
> === *Public Comments*
> ========================================================
>
> === *Comments*
> ===============================================================
>
> === *Evaluation*
> =============================================================
> Need to restore the spec and the implementation.
>
> *** (#1 of 1): 2009-09-11 18:49:36 GMT+00:00 xueming.shen at sun.com
>
>
> === *Suggested Fix*
> ==========================================================
>
> === *Workaround*
> =============================================================
>
> === *Justification*
> ==========================================================
> Priority changed from [] to [3-Medium]
> regression
> xueming.shen at sun.com 2009-09-11 18:48:33 GMT
>
> *** (#1 of 1): 2009-09-11 18:48:33 GMT+00:00 xueming.shen at sun.com
>
>
> === *Additional Details*
> =====================================================
> ? ? ? Targeted Release: 7
> ? ? ? Commit To Fix In Build: ? ? ? Fixed In Build: ? ? ? Integrated In
> Build: ? ? ? Verified In Build: ?See Also: ?Duplicate of: ?Hooks:
> ? ? ? Hook1: ? ? ? Hook2: ? ? ? Hook3: ? ? ? Hook4: ? ? ? Hook5:
> Hook6: ?Interest List: ?Program Management: ?Root Cause: ?Is a Security
> Vulnerability?: No
> ?Fix Affects Documentation: No
> ?Fix Affects Localization: No
> ?Reported by:
> === *History*
> ================================================================
> ? ? ? Date Submitted: 2009-09-11 18:48:32 GMT+00:00
> ? ? ? Submitted By: xueming.shen at sun.com
>
> ? ? ? Status Changed ? ?Date Updated ? ? ? ? ? ? ? ? ?Updated By
> ? ? ? 5-Cause Known ? ? 2009-09-11 18:49:36 GMT+00:00 xueming.shen at sun.com
>
>
> === *Solution*
> ===============================================================
>
>
>
From vadim123 at gmail.com Fri Sep 11 15:48:56 2009
From: vadim123 at gmail.com (Vadim Ponomarenko)
Date: Fri, 11 Sep 2009 22:48:56 +0000 (UTC)
Subject: Replacement of Quicksort in java.util.Arrays with new
=?utf-8?b?RHVhbC1QaXZvdAlRdWlja3NvcnQ=?=
References: <4AAA27EA.8090902@Sun.COM>
Message-ID:
Vladimir Yaroslavskiy writes:
> I'd like to share with you new Dual-Pivot Quicksort which is
> faster than the known implementations (theoretically and
> experimental). I'd like to propose to replace the JDK's
> Quicksort implementation by new one.
This is a great idea; as a mathematician I immediately want to generalize it.
Unfortunately, I lack the computing skill to test my generalization in
practice.
Suppose that instead of 2 pivots we have m (randomly chosen) pivots, where
m=sqrt(n+1)-1.
Step 1: Sort the m pivots, recursively.
Step 2: Build a balanced binary tree from the pivots.
Step 3: Sort the remaining n-m elements into (m+1) categories, using the BBT.
Step 4: Sort each of the (m+1) categories, recursively.
Let S(n) denote the number of steps needed for this algorithm. Calculating
roughly:
Step 1 requires S(m) steps.
Step 2 requires m steps (it is easy since the pivots are already sorted).
Step 3 requires (n-m)(log m) comparisons.
Step 4 requires, on average, (m+1)S((n-m)/(m+1)) steps.
m was chosen so that (n-m)/(m+1)=m, so S(n)=(m+2)S(m)+m+(n-m)(log m) <= (m+2)
S(m)+ n log m. Crudely, we replace both m and m+2 by sqrt(n), to get as
approximation:
S(n) <= 0.5 n log n + n^0.5 [ 0.25 n^0.5 log n + n^0.25 [ 0.125 n^0.25 log n
+...]]] = 0.5 n log n [1 + 0.5 + 0.25 + ...] = n log n. This appears to be
better than 2 n log n. This argument can be made precise (the only really
questionable issue is replacing m+2 by m) by first getting a crude but
correct upper bound for 2S(m), which gets absorbed into the -mlogm term.
Unfortunately, I'm not up to the detailed comparison vs. swap analysis as
done by Mr. Yaroslavskiy, but it seems to me that, if nothing else, having
an efficient algorithm to partition the non-pivot elements should provide a
savings.
Vadim Ponomarenko
Dept. of Mathematics and Statistics
San Diego State University
From neal at gafter.com Fri Sep 11 16:08:49 2009
From: neal at gafter.com (Neal Gafter)
Date: Fri, 11 Sep 2009 16:08:49 -0700
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
Quicksort
In-Reply-To:
References: <4AAA27EA.8090902@Sun.COM>
Message-ID: <15e8b9d20909111608u4e463176rf8021f5ed866420a@mail.gmail.com>
Vadim-
It would be very interesting if something along these lines could be made
practical. It isn't obvious how to do step (3) in place. Either you end up
allocating extra storage to do it, in which case you might as well have used
a merge sort, or you end up doing some extra shuffling around of the data,
in which case it is probably more expensive than the dual-partition
version. Perhaps there is some technique I'm not thinking of.
Cheers,
Neal
On Fri, Sep 11, 2009 at 3:48 PM, Vadim Ponomarenko wrote:
> Vladimir Yaroslavskiy writes:
> > I'd like to share with you new Dual-Pivot Quicksort which is
> > faster than the known implementations (theoretically and
> > experimental). I'd like to propose to replace the JDK's
> > Quicksort implementation by new one.
>
> This is a great idea; as a mathematician I immediately want to generalize
> it.
> Unfortunately, I lack the computing skill to test my generalization in
> practice.
>
> Suppose that instead of 2 pivots we have m (randomly chosen) pivots, where
> m=sqrt(n+1)-1.
> Step 1: Sort the m pivots, recursively.
> Step 2: Build a balanced binary tree from the pivots.
> Step 3: Sort the remaining n-m elements into (m+1) categories, using the
> BBT.
> Step 4: Sort each of the (m+1) categories, recursively.
>
> Let S(n) denote the number of steps needed for this algorithm. Calculating
> roughly:
> Step 1 requires S(m) steps.
> Step 2 requires m steps (it is easy since the pivots are already sorted).
> Step 3 requires (n-m)(log m) comparisons.
> Step 4 requires, on average, (m+1)S((n-m)/(m+1)) steps.
>
> m was chosen so that (n-m)/(m+1)=m, so S(n)=(m+2)S(m)+m+(n-m)(log m) <=
> (m+2)
> S(m)+ n log m. Crudely, we replace both m and m+2 by sqrt(n), to get as
> approximation:
> S(n) <= 0.5 n log n + n^0.5 [ 0.25 n^0.5 log n + n^0.25 [ 0.125 n^0.25 log
> n
> +...]]] = 0.5 n log n [1 + 0.5 + 0.25 + ...] = n log n. This appears to
> be
> better than 2 n log n. This argument can be made precise (the only really
> questionable issue is replacing m+2 by m) by first getting a crude but
> correct upper bound for 2S(m), which gets absorbed into the -mlogm term.
>
> Unfortunately, I'm not up to the detailed comparison vs. swap analysis as
> done by Mr. Yaroslavskiy, but it seems to me that, if nothing else, having
> an efficient algorithm to partition the non-pivot elements should provide a
> savings.
>
> Vadim Ponomarenko
> Dept. of Mathematics and Statistics
> San Diego State University
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090911/86766558/attachment.html
From Xueming.Shen at Sun.COM Fri Sep 11 16:13:30 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Fri, 11 Sep 2009 16:13:30 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AAA1B75.6060309@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<4AA03D0B.1030206@sun.com>
<1ccfd1c10909031606w28309d4cj4f6ea9309264db7c@mail.gmail.com>
<4AA051E5.5060605@sun.com>
<1ccfd1c10909031655r187530cap6871c15b8f595e57@mail.gmail.com>
<4AA19250.2060803@sun.com> <20090904225714.GB26286@google.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
<1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
<4AA7F9EB.5090200@sun.com> <4AA816AD.7090306@sun.com>
<4AA949DA.9010707@sun.com> <4AAA1B75.6060309@sun.com>
Message-ID: <4AAAD99A.4050505@sun.com>
I'm definitely open for any better choice, I'm not in hurry to close
this one up. Sure, I hope we can
reach a consensus before I go on vacation this weekend:-)
So here is the planB alan is proposing, no API doc added yet, but you
should know what it means. I will add the doc
in later if we agree on this approach.
http://cr.openjdk.java.net/~sherman/zipflush/webrev.planB/src/share/classes/java/util/zip/DeflaterOutputStream.java.sdiff.html
We probably should figure out want to do with the GZipOutputStream...it
probably needs something as well...we can figure
out it later.
So, Martin, Brandon, David and anyone interested,
Is this something you guys can accept ? It makes the life easy for
developer that needs the sync_flush for the flush(),
you don't need to override/subclass anything, just makes it clear what
you want when creating the DOS by picking
the appropriate constructor, nothing more. Since we keep the default
behavior un-changed, it is also compatible:-)
It does not add the the full_sync, sure we can consider to use a enum
instead of the boolean parameter, but since
no one is asking for it now and the enum looks a little "too modern"
compared to the rest of the API, we might
want to stay with the simple "boolean doSyncFlush".
Opinion?
sherman
Funny, I was "beaten up" today for one of my other changes, I was
literally asked "where was it documented/highlighted
that this fix was going to introduce an incompatibility", for a much
much "less" incompatible change:-)
Alan Bateman wrote:
> Xueming Shen wrote:
>> I think we have enough discussion on this topic, it's time to make a
>> final decision. Seems like we are all happy on the
>> changes in Deflater and new DOS.flush(mode), the only difference is
>> whether or not we should change the existing
>> behavior of DOS.flush() should use Z_SYNC_FLUSH by default.
>>
>> Brandon is very firm on the "yes" side, strongly believe this is a MUST.
>> Martin is on the fence, leaning towards "yes"
>> Sherman is on the "no" side
>>
>> So if there is no other vote coming with strong opinion, your vote
>> will be important:-) Are you also leaning to "yes"
>> or "strongly" suggesting a yes? I need strong "yes" to change my mind.
> This is tough call. It is clearly desirable to change flush() to work
> the way that developers expect. On the other hand there is always risk
> with changing long-standing behavior. The risk here is probably low
> but it's impossible to quantify. Martin makes a good observation that
> it may cause problems for code that wraps the stream with an
> auto-flushing PrintStream or PrintWriter. You also made a good
> observation that just changing the number of bytes written could cause
> breakage (in some fragile environment) - we just don't know! So on
> balance, I think you are probably right to take the conservative line
> on this one.
>
> I know you've had enough discussion on this but I wonder if you might
> be open to considering new constructors for DeflaterOutputStream (and
> maybe GZIPOutputStream) as an alternative to the flush(int) method.
> This might fit better with this API, in particular for cases where the
> intended usage is known. It could be as simple as a "boolean
> syncFlush" parameter to indicate how uncompressed buffered bytes are
> handled when flushing (default to false for existing constructors so
> existing behavior preserved). The boolean parameter is just an
> opening bid - clearly there are alternatives. The point is that it
> makes it easy and more obvious as how to get a DeflaterOutputStream
> suitable for interactive cases.
>
> -Alan.
From vadim123 at gmail.com Fri Sep 11 16:17:33 2009
From: vadim123 at gmail.com (Vadim Ponomarenko)
Date: Fri, 11 Sep 2009 23:17:33 +0000 (UTC)
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
=?utf-8?b?CVF1aWNrc29ydA==?=
References: <4AAA27EA.8090902@Sun.COM>
<15e8b9d20909111608u4e463176rf8021f5ed866420a@mail.gmail.com>
Message-ID:
> Vadim-It would be very interesting if something along these lines could be
made practical.? It isn't obvious how to do step (3) in place.? Either you
end up allocating extra storage to do it, in which case you might as well
have used a merge sort, or you end up doing some extra shuffling around
of the data, in which case it is probably more expensive than the
dual-partition version.? Perhaps there is some technique I'm not
thinking of.Cheers,Neal
I didn't think of space requirements, but naively it seems to me that the
two-pivot method of categorizing the non-pivot elements in place (which
admittedly I don't entirely understand) would work just as well with the
m-pivot method.
From Joe.Darcy at Sun.COM Fri Sep 11 16:25:31 2009
From: Joe.Darcy at Sun.COM (Joe Darcy)
Date: Fri, 11 Sep 2009 16:25:31 -0700
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <4AA9936B.5000006@joda.org>
References: <4AA812D0.309@sun.com> <4AA9936B.5000006@joda.org>
Message-ID: <4AAADC6B.7070807@sun.com>
Stephen Colebourne wrote:
> Joe Darcy wrote:
>> What other utility methods would have broad enough use and
>> applicability to go into a common java.util class?
>
> Joe,
> You've asked for a call for ideas, but not given any indication of
> process. Are you going to evaluate everything that comes in and pick
> the best a la Coin? Or allow anyone to send in patches?
Those options are not mutually exclusive; patches are welcome subject to
the usual terms and conditions.
> Who decides what is in and what is out?
This is a little side project of mine and I wanted to get some feedback
before preparing a formal change for review, possibly including patches
from others.
-Joe
From xueming.shen at sun.com Fri Sep 11 16:45:15 2009
From: xueming.shen at sun.com (xueming.shen at sun.com)
Date: Fri, 11 Sep 2009 23:45:15 +0000
Subject: hg: jdk7/tl/jdk: 6881337: ZipEntry.setComment() was accidentally
changed back to old spec/impl in jdk7-b64
Message-ID: <20090911234603.9E71112044@hg.openjdk.java.net>
Changeset: 46406871599c
Author: sherman
Date: 2009-09-11 16:36 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/46406871599c
6881337: ZipEntry.setComment() was accidentally changed back to old spec/impl in jdk7-b64
Summary: restored the correct spec and implementation of setComment
Reviewed-by: martin
! src/share/classes/java/util/zip/ZipEntry.java
From Joe.Darcy at Sun.COM Fri Sep 11 16:48:48 2009
From: Joe.Darcy at Sun.COM (Joe Darcy)
Date: Fri, 11 Sep 2009 16:48:48 -0700
Subject: Race condition in Class.getName()
In-Reply-To: <1ccfd1c10909091735ndf28f91j1813c26203a60967@mail.gmail.com>
References: <1ccfd1c10909091735ndf28f91j1813c26203a60967@mail.gmail.com>
Message-ID: <4AAAE1E0.6020200@sun.com>
Martin Buchholz wrote:
> It is believed that the java memory model allows Class.getName()
> to return null.
> This is one of those methods with an intentional data race.
> Probably this has not been seen in practice because only
> a perverse or adversarial runtime would load the "name" field
> twice, out-of-order, as it seems to be allowed to.
>
> diff --git a/src/share/classes/java/lang/Class.java
> b/src/share/classes/java/lang/Class.java
> --- a/src/share/classes/java/lang/Class.java
> +++ b/src/share/classes/java/lang/Class.java
> @@ -565,8 +565,9 @@
> * represented by this object.
> */
> public String getName() {
> + String name = this.name;
> if (name == null)
> - name = getName0();
> + this.name = name = getName0();
> return name;
> }
>
> Martin
>
Hello.
The change looks fine to me. Do you want a bug filed, etc.?
-Joe
From martinrb at google.com Fri Sep 11 17:22:50 2009
From: martinrb at google.com (Martin Buchholz)
Date: Fri, 11 Sep 2009 17:22:50 -0700
Subject: Race condition in Class.getName()
In-Reply-To: <4AAAE1E0.6020200@sun.com>
References: <1ccfd1c10909091735ndf28f91j1813c26203a60967@mail.gmail.com>
<4AAAE1E0.6020200@sun.com>
Message-ID: <1ccfd1c10909111722k6920b384ub1848c1a8d7ac1d@mail.gmail.com>
On Fri, Sep 11, 2009 at 16:48, Joe Darcy wrote:
> Martin Buchholz wrote:
>>
>> It is believed that the java memory model allows Class.getName()
>> to return null.
>> This is one of those methods with an intentional data race.
>> Probably this has not been seen in practice because only
>> a perverse or adversarial runtime would load the "name" field
>> twice, out-of-order, as it seems to be allowed to.
>>
>> diff --git a/src/share/classes/java/lang/Class.java
>> b/src/share/classes/java/lang/Class.java
>> --- a/src/share/classes/java/lang/Class.java
>> +++ b/src/share/classes/java/lang/Class.java
>> @@ -565,8 +565,9 @@
>> ? ? ?* ? ? ? ? ?represented by this object.
>> ? ? ?*/
>> ? ? public String getName() {
>> + ? ? ? ?String name = this.name;
>> ? ? ? ? if (name == null)
>> - ? ? ? ? ? ?name = getName0();
>> + ? ? ? ? ? ?this.name = name = getName0();
>> ? ? ? ? return name;
>> ? ? }
>>
>> Martin
>>
>
> Hello.
>
> The change looks fine to me. ?Do you want a bug filed, etc.?
Yup. I will commit as soon as I get a bug + synopsis.
From Joe.Darcy at Sun.COM Fri Sep 11 17:37:09 2009
From: Joe.Darcy at Sun.COM (Joe Darcy)
Date: Fri, 11 Sep 2009 17:37:09 -0700
Subject: Race condition in Class.getName()
In-Reply-To: <1ccfd1c10909111722k6920b384ub1848c1a8d7ac1d@mail.gmail.com>
References: <1ccfd1c10909091735ndf28f91j1813c26203a60967@mail.gmail.com>
<4AAAE1E0.6020200@sun.com>
<1ccfd1c10909111722k6920b384ub1848c1a8d7ac1d@mail.gmail.com>
Message-ID: <4AAAED35.8080809@sun.com>
Martin Buchholz wrote:
> On Fri, Sep 11, 2009 at 16:48, Joe Darcy wrote:
>
>> Martin Buchholz wrote:
>>
>>> It is believed that the java memory model allows Class.getName()
>>> to return null.
>>> This is one of those methods with an intentional data race.
>>> Probably this has not been seen in practice because only
>>> a perverse or adversarial runtime would load the "name" field
>>> twice, out-of-order, as it seems to be allowed to.
>>>
>>> diff --git a/src/share/classes/java/lang/Class.java
>>> b/src/share/classes/java/lang/Class.java
>>> --- a/src/share/classes/java/lang/Class.java
>>> +++ b/src/share/classes/java/lang/Class.java
>>> @@ -565,8 +565,9 @@
>>> * represented by this object.
>>> */
>>> public String getName() {
>>> + String name = this.name;
>>> if (name == null)
>>> - name = getName0();
>>> + this.name = name = getName0();
>>> return name;
>>> }
>>>
>>> Martin
>>>
>>>
>> Hello.
>>
>> The change looks fine to me. Do you want a bug filed, etc.?
>>
>
> Yup. I will commit as soon as I get a bug + synopsis.
>
6881442 (reflect) Race condition in Class.getName()
-Joe
From gokdogan at gmail.com Sat Sep 12 03:20:03 2009
From: gokdogan at gmail.com (Goktug Gokdogan)
Date: Sat, 12 Sep 2009 13:20:03 +0300
Subject: What methods should go into a java.util.Objects class in JDK 7?
In-Reply-To: <108fcdeb0909091625h5cc8c370t5d0a16445d7eb46a@mail.gmail.com>
References: <4AA812D0.309@sun.com>
<17c6771e0909091422n57ed252bh4749571a07d2dd41@mail.gmail.com>
<108fcdeb0909091625h5cc8c370t5d0a16445d7eb46a@mail.gmail.com>
Message-ID:
Similarly, I've previously written reflection-based #clone, #equals, and
#fillRandomData methods working on POJOs. They were useful especially in
testing the data binding codes, ie. X original = new X();,
fillRandomData(original);
mapper.toGUI( clone(original) ); //Clone object so original data can not
be modified
X rv = new X();
mapper.fromGUI(rv);
assertTrue( equals(original, rv) );
But I can not imagine much use in a production code. If other use cases can
be suggested, we could consider putting into Objects class.
On Thu, Sep 10, 2009 at 2:25 AM, Kevin Bourrillion wrote:
> -1 on reflection-based stuff in the Objects class.
>
> equal() and hashCode() are the only two things we saw fit to include in our
> Objects class, and actually, our hashCode() is identical to
> Arrays.hashCode() (but is varargs).
>
>
> http://google-collections.googlecode.com/svn/trunk/javadoc/com/google/common/base/Objects.html
>
> Please do add the compare() methods to all the rest of the wrapper types;
> we would use those a lot.
>
> When it comes to cloning, I tried very hard once to provide something
> useful -- a "clone any object without knowing its concrete type" method --
> and it was a disaster. A total mess of reflection, and no one wanted to use
> it anyway. Cloning in Java is discredited.
>
> So I'm left with only equal() for your new Objects class. By the way, the
> best implementation is a == b || (a != null && a.equals(b)).
>
>
>
>
> On Wed, Sep 9, 2009 at 2:22 PM, Andrew John Hughes <
> gnu_andrew at member.fsf.org> wrote:
>
>> 2009/9/9 Joe Darcy :
>> > Hello.
>> >
>> > For JDK 7, I think it is high-time the platform included a class like
>> > java.util.Objects to hold commonly-written utility methods. For
>> example, a
>> > two-argument static equals method that returned true if both arguments
>> are
>> > null, returns false is one argument is null, and otherwise returns the
>> > result of calling arg1.equals(arg2) (6797535 "Add shared two argument
>> > static equals method to the platform").
>> >
>> > A static hashCode method returning 0 for null and the value of
>> > arg.hashCode() has also been suggested.
>> >
>> > A set of
>> >
>> > static int compareTo(int, int)
>> > static int compareTo(long, long)
>> > ....
>> >
>> > methods probably belongs somewhere in the platform too.
>> >
>> > What other utility methods would have broad enough use and applicability
>> to
>> > go into a common java.util class?
>> >
>> > -Joe
>> >
>>
>> Given you've listed utility methods for two Object methods, equals and
>> hashCode, toString seems an obvious one to handle as well:
>>
>> public static String toString(Object o)
>> throws IllegalAccessException
>> {
>> Class c = o.getClass();
>> StringBuilder b = new StringBuilder(c.getName());
>> b.append("[");
>> for (Field f : c.getDeclaredFields())
>> {
>> f.setAccessible(true);
>> b.append(f.getName() + "=" + f.get());
>> b.append(",");
>> }
>> b.replace(b.length() - 1, b.length(), "]");
>> return b.toString();
>> }
>>
>> Maybe there's also a useful utility implementation of clone too, but I
>> can't think of one offhand.
>> --
>> Andrew :-)
>>
>> Free Java Software Engineer
>> Red Hat, Inc. (http://www.redhat.com)
>>
>> Support Free Java!
>> Contribute to GNU Classpath and the OpenJDK
>> http://www.gnu.org/software/classpath
>> http://openjdk.java.net
>>
>> PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
>> Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
>>
>
>
>
> --
> Kevin Bourrillion @ Google
> internal: http://go/javalibraries
> google-collections.googlecode.com
> google-guice.googlecode.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090912/28a0e64e/attachment.html
From gokdogan at gmail.com Sat Sep 12 04:43:24 2009
From: gokdogan at gmail.com (Goktug Gokdogan)
Date: Sat, 12 Sep 2009 14:43:24 +0300
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
Quicksort
In-Reply-To: <4AAA5E4D.6050909@mail.ru>
References: <4AAA27EA.8090902@Sun.COM> <4AAA5158.8060202@univ-mlv.fr>
<4AAA54E4.7090107@gmx.de> <4AAA5E4D.6050909@mail.ru>
Message-ID:
My absolutely non-scientific preliminary tests indicate Arrays.sort performs
better with the same input (5000 items) in nearly all runs (-client).
public static void main(String[] args) {
final int n = 5000;
int[] array = new int[n];
int[] array2 = new int[n];
Random random = new Random();
for (int i = 0; i < n; i++) {
int r = random.nextInt(n);
array[i] = r;
array2[i] = r;
}
long st1 = System.nanoTime();
Arrays.sort(array2);
long st2 = System.nanoTime();
DualPivotQuicksort.sort(array);
long end = System.nanoTime();
System.out.println(st2 - st1);
System.out.println(end - st2);
}
I tried changing which sort runs first, but the results did not change. Over
100.000 items, dual-pivot consistently beats Arrays.sort.
Well, this test is not a good reference though :)
On Fri, Sep 11, 2009 at 5:27 PM, Vladimir Iaroslavski
wrote:
> Hi,
>
> I've tried to use local variable int ak = a[k] in the loop
> and not found saving of time. There are 3 occurrences of a[k]
> in each loop, but only two can be changed because third
> comes after line: swap(a, k, great--);
>
> As summary: I'm changing only hardcoded constant by
> private static final variables.
>
> Thank you,
> Vladimir
>
>
> Ulf Zibis wrote:
>
>> Am 11.09.2009 15:32, R?mi Forax schrieb:
>>
>>> just my two cents :
>>> In the loop tagged "sorting" and "equals element",
>>> a[k] can be stored in a local variable to avoid to recompute it several
>>> time.
>>>
>>
>> I add 2 cents more:
>> Doesn't HotSpot see this, and optimize accordingly?
>> IMO: It's time that javac should do such optimization, as there are less
>> optimized VM's in the world.
>>
>> -Ulf
>>
>>
>>
>>> The algorithm use two constants: 37 and 13,
>>> I think they should be declared as private static final.
>>>
>>> R?mi
>>>
>>>
>>> Le 11/09/2009 12:35, Vladimir Yaroslavskiy a ?crit :
>>>
>>>> Hello All,
>>>>
>>>> I'd like to share with you new Dual-Pivot Quicksort which is
>>>> faster than the known implementations (theoretically and
>>>> experimental). I'd like to propose to replace the JDK's
>>>> Quicksort implementation by new one.
>>>>
>>>> Description
>>>> -----------
>>>> The classical Quicksort algorithm uses the following scheme:
>>>>
>>>> 1. Pick an element P, called a pivot, from the array.
>>>> 2. Reorder the array so that all elements, which are less than
>>>> the pivot, come before the pivot and all elements greater than
>>>> the pivot come after it (equal values can go either way). After
>>>> this partitioning, the pivot element is in its final position.
>>>> 3. Recursively sort the sub-array of lesser elements and the
>>>> sub-array of greater elements.
>>>>
>>>> The invariant of classical Quicksort is:
>>>>
>>>> [ <= p | >= p ]
>>>>
>>>> There are several modifications of the schema:
>>>>
>>>> [ < p | = p | > p ] or [ = p | < p | > p | = p ]
>>>>
>>>> But all of them use *one* pivot.
>>>>
>>>> The new Dual-Pivot Quicksort uses *two* pivots elements in this manner:
>>>>
>>>> 1. Pick an elements P1, P2, called pivots from the array.
>>>> 2. Assume that P1 <= P2, otherwise swap it.
>>>> 3. Reorder the array into three parts: those less than the smaller
>>>> pivot, those larger than the larger pivot, and in between are
>>>> those elements between (or equal to) the two pivots.
>>>> 4. Recursively sort the sub-arrays.
>>>>
>>>> The invariant of the Dual-Pivot Quicksort is:
>>>>
>>>> [ < P1 | P1 <= & <= P2 } > P2 ]
>>>>
>>>> The new Quicksort is faster than current implementation of Quicksort
>>>> in JDK (L. Bentley and M. Douglas McIlroy) and classical Quicksort.
>>>>
>>>> The full description of the Dual-Pivot Quicksort you can find
>>>> on my page: http://iaroslavski.narod.ru/quicksort
>>>>
>>>> Performance tests
>>>> -----------------
>>>> Here is result of running on different types of input data:
>>>>
>>>> Client VM all 85% organ 0..1 0..4
>>>> random ascend descend equal equal pipes random 010101
>>>> random
>>>> Dual-Pivot: 16.83 5.31 5.47 0.35 0.68 10.59 1.06 1.02 2.18
>>>> Bentley's: 19.77 9.08 10.13 0.63 1.12 13.22 1.63 1.08 2.49
>>>>
>>>> Server VM all 85% organ 0..1 0..4
>>>> random ascend descend equal equal pipes random 010101
>>>> random
>>>> Dual-Pivot: 23.94 6.68 6.63 0.43 0.62 17.14 1.42 1.96 3.41
>>>> Bentley's: 25.20 10.18 10.32 2.07 1.33 16.72 2.95 1.82 3.39
>>>>
>>>> The a lot of other tests have been run under client and server mode.
>>>> The most interesting is BentleyBasher test framework. It runs battery
>>>> of tests for all cases:
>>>>
>>>> { 100, 1000, 10000, 1000000 } x
>>>> { sawtooth, rand, stagger, plateau, shuffle } x
>>>> { ident, reverse, reverse_front, reverse_back, sort, dither}
>>>>
>>>> where
>>>>
>>>> 100, ... , 1000000 - array length
>>>>
>>>> sawtooth: x[i] =i%m
>>>> rand: x[i] = rand() % m
>>>> stagger: x[i] = (i*m + i) % n
>>>> plateau: x[i] = min(i, m)
>>>> shuffle: x[i] = rand()%m? (j+=2): (k+=2)
>>>>
>>>> ident(x) - a copy of x
>>>> reverse(x, 0, n) - reversed copy
>>>> reverse_front(x, 0, n/2) - front half reversed
>>>> reverse_back(x, n/2, n) - back half reversed
>>>> sort(x) - an ordered copy
>>>> dither(x) - add i%5 to x[i]
>>>>
>>>> Here is the result of execution:
>>>> Server VM:
>>>> http://spreadsheets.google.com/pub?key=t_EAWUkQ4mD3BIbOv8Fa-AQ&output=html
>>>> Client VM:
>>>> http://spreadsheets.google.com/pub?key=tdiMo8xleTxd23nKUObcz0Q&single=true&gid=0&output=html
>>>>
>>>> Mathematical investigations
>>>> ---------------------------
>>>> It is proved that for the Dual-Pivot Quicksort the average number of
>>>> comparisons is 2*n*ln(n), the average number of swaps is 0.8*n*ln(n),
>>>> whereas classical Quicksort algorithm has 2*n*ln(n) and 1*n*ln(n)
>>>> respectively. Full mathematical proof see in attached proof.txt
>>>> and proof_add.txt files. Theoretical results are also confirmed
>>>> by experimental counting of the operations.
>>>>
>>>> Diff between current and new implementation of Quicksort
>>>> --------------------------------------------------------
>>>>
>>>> Here is the link to the diff for java.util.Arrays class:
>>>> http://cr.openjdk.java.net/~alanb/6880672/webrev.00
>>>>
>>>> If you like to look and play with new algorithm,
>>>> please, take attached class DualPivotQuicksort.java
>>>>
>>>> Feedback
>>>> --------
>>>>
>>>> Also I'd like to share a feedback from Joshua Bloch and
>>>> Jon Bentley who spent a lot of time investigating this
>>>> algorithm, who gave me many advices and tips how to
>>>> make new Quicksort better.
>>>>
>>>> -------- Original Message --------
>>>> Subject: Re: Integration of new Dual-Pivot Quicksort into JDK 7
>>>> Date: Thu, 10 Sep 2009 07:20:11 -0700
>>>> From: Joshua Bloch
>>>>
>>>> Jon also says that Vladimir should make every reasonable improvement to
>>>> the basic method before checking in the code. In his words, "It would be
>>>> horrible to put the new code into the library, and then have someone
>>>> else come along and speed it up by another 20% by using standard
>>>> techniques." I believe it's not unlikely that this code may end up
>>>> getting ported to many languages and widely deployed in much the manner
>>>> of Bentley and McIlroy's fine sort (which is nearing 20 successful years
>>>> in the field). Jon will help Vladimir do this.
>>>>
>>>> -------- Original Message --------
>>>> Subject: Dual-Pivot Quicksort: Next Steps
>>>> Date: Wed, 09 Sep 2009 15:02:25 -0400
>>>> From: Jon Bentley
>>>>
>>>> Vladimir, Josh,
>>>> I *finally* feel like I understand what is going on. Now that I
>>>> (think that) I see it, it seems straightforward and obvious.
>>>> Tony Hoare developed Quicksort in the early 1960s. I was very
>>>> proud to make minor contributions to a particularly clean (binary)
>>>> quicksort in the mid 1980s, to a relatively straightforward, industrial
>>>> strength Quicksort with McIlroy in the early 1990s, and then to
>>>> algorithms and data structures for strings with Sedgewick in the mid
>>>> 1990s.
>>>> I think that Vladimir's contributions to Quicksort go way beyond
>>>> anything that I've ever done, and rank up there with Hoare's original
>>>> design and Sedgewick's analysis. I feel so privileged to play a very,
>>>> very minor role in helping Vladimir with the most excellent work!
>>>>
>>>> -----------------------------------------------
>>>>
>>>> Let me know, if you have any questions/comments.
>>>>
>>>> Thank you,
>>>> Vladimir
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090912/3bdeedac/attachment-0001.html
From gokdogan at gmail.com Sat Sep 12 05:13:17 2009
From: gokdogan at gmail.com (Goktug Gokdogan)
Date: Sat, 12 Sep 2009 15:13:17 +0300
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
Quicksort
In-Reply-To:
References: <4AAA27EA.8090902@Sun.COM> <4AAA5158.8060202@univ-mlv.fr>
<4AAA54E4.7090107@gmx.de> <4AAA5E4D.6050909@mail.ru>
Message-ID:
Sorry :( . Please ignore previous post. Warming-up yield to better results
in dual-pivot's favor.
On Sat, Sep 12, 2009 at 2:43 PM, Goktug Gokdogan wrote:
> My absolutely non-scientific preliminary tests indicate Arrays.sort
> performs better with the same input (5000 items) in nearly all runs
> (-client).
> public static void main(String[] args) {
> final int n = 5000;
> int[] array = new int[n];
> int[] array2 = new int[n];
> Random random = new Random();
> for (int i = 0; i < n; i++) {
> int r = random.nextInt(n);
> array[i] = r;
> array2[i] = r;
> }
>
> long st1 = System.nanoTime();
> Arrays.sort(array2);
> long st2 = System.nanoTime();
> DualPivotQuicksort.sort(array);
> long end = System.nanoTime();
> System.out.println(st2 - st1);
> System.out.println(end - st2);
> }
>
> I tried changing which sort runs first, but the results did not change.
> Over 100.000 items, dual-pivot consistently beats Arrays.sort.
>
> Well, this test is not a good reference though :)
>
> On Fri, Sep 11, 2009 at 5:27 PM, Vladimir Iaroslavski > wrote:
>
>> Hi,
>>
>> I've tried to use local variable int ak = a[k] in the loop
>> and not found saving of time. There are 3 occurrences of a[k]
>> in each loop, but only two can be changed because third
>> comes after line: swap(a, k, great--);
>>
>> As summary: I'm changing only hardcoded constant by
>> private static final variables.
>>
>> Thank you,
>> Vladimir
>>
>>
>> Ulf Zibis wrote:
>>
>>> Am 11.09.2009 15:32, R?mi Forax schrieb:
>>>
>>>> just my two cents :
>>>> In the loop tagged "sorting" and "equals element",
>>>> a[k] can be stored in a local variable to avoid to recompute it several
>>>> time.
>>>>
>>>
>>> I add 2 cents more:
>>> Doesn't HotSpot see this, and optimize accordingly?
>>> IMO: It's time that javac should do such optimization, as there are less
>>> optimized VM's in the world.
>>>
>>> -Ulf
>>>
>>>
>>>
>>>> The algorithm use two constants: 37 and 13,
>>>> I think they should be declared as private static final.
>>>>
>>>> R?mi
>>>>
>>>>
>>>> Le 11/09/2009 12:35, Vladimir Yaroslavskiy a ?crit :
>>>>
>>>>> Hello All,
>>>>>
>>>>> I'd like to share with you new Dual-Pivot Quicksort which is
>>>>> faster than the known implementations (theoretically and
>>>>> experimental). I'd like to propose to replace the JDK's
>>>>> Quicksort implementation by new one.
>>>>>
>>>>> Description
>>>>> -----------
>>>>> The classical Quicksort algorithm uses the following scheme:
>>>>>
>>>>> 1. Pick an element P, called a pivot, from the array.
>>>>> 2. Reorder the array so that all elements, which are less than
>>>>> the pivot, come before the pivot and all elements greater than
>>>>> the pivot come after it (equal values can go either way). After
>>>>> this partitioning, the pivot element is in its final position.
>>>>> 3. Recursively sort the sub-array of lesser elements and the
>>>>> sub-array of greater elements.
>>>>>
>>>>> The invariant of classical Quicksort is:
>>>>>
>>>>> [ <= p | >= p ]
>>>>>
>>>>> There are several modifications of the schema:
>>>>>
>>>>> [ < p | = p | > p ] or [ = p | < p | > p | = p ]
>>>>>
>>>>> But all of them use *one* pivot.
>>>>>
>>>>> The new Dual-Pivot Quicksort uses *two* pivots elements in this manner:
>>>>>
>>>>> 1. Pick an elements P1, P2, called pivots from the array.
>>>>> 2. Assume that P1 <= P2, otherwise swap it.
>>>>> 3. Reorder the array into three parts: those less than the smaller
>>>>> pivot, those larger than the larger pivot, and in between are
>>>>> those elements between (or equal to) the two pivots.
>>>>> 4. Recursively sort the sub-arrays.
>>>>>
>>>>> The invariant of the Dual-Pivot Quicksort is:
>>>>>
>>>>> [ < P1 | P1 <= & <= P2 } > P2 ]
>>>>>
>>>>> The new Quicksort is faster than current implementation of Quicksort
>>>>> in JDK (L. Bentley and M. Douglas McIlroy) and classical Quicksort.
>>>>>
>>>>> The full description of the Dual-Pivot Quicksort you can find
>>>>> on my page: http://iaroslavski.narod.ru/quicksort
>>>>>
>>>>> Performance tests
>>>>> -----------------
>>>>> Here is result of running on different types of input data:
>>>>>
>>>>> Client VM all 85% organ 0..1
>>>>> 0..4
>>>>> random ascend descend equal equal pipes random 010101
>>>>> random
>>>>> Dual-Pivot: 16.83 5.31 5.47 0.35 0.68 10.59 1.06 1.02
>>>>> 2.18
>>>>> Bentley's: 19.77 9.08 10.13 0.63 1.12 13.22 1.63 1.08
>>>>> 2.49
>>>>>
>>>>> Server VM all 85% organ 0..1
>>>>> 0..4
>>>>> random ascend descend equal equal pipes random 010101
>>>>> random
>>>>> Dual-Pivot: 23.94 6.68 6.63 0.43 0.62 17.14 1.42 1.96
>>>>> 3.41
>>>>> Bentley's: 25.20 10.18 10.32 2.07 1.33 16.72 2.95 1.82
>>>>> 3.39
>>>>>
>>>>> The a lot of other tests have been run under client and server mode.
>>>>> The most interesting is BentleyBasher test framework. It runs battery
>>>>> of tests for all cases:
>>>>>
>>>>> { 100, 1000, 10000, 1000000 } x
>>>>> { sawtooth, rand, stagger, plateau, shuffle } x
>>>>> { ident, reverse, reverse_front, reverse_back, sort, dither}
>>>>>
>>>>> where
>>>>>
>>>>> 100, ... , 1000000 - array length
>>>>>
>>>>> sawtooth: x[i] =i%m
>>>>> rand: x[i] = rand() % m
>>>>> stagger: x[i] = (i*m + i) % n
>>>>> plateau: x[i] = min(i, m)
>>>>> shuffle: x[i] = rand()%m? (j+=2): (k+=2)
>>>>>
>>>>> ident(x) - a copy of x
>>>>> reverse(x, 0, n) - reversed copy
>>>>> reverse_front(x, 0, n/2) - front half reversed
>>>>> reverse_back(x, n/2, n) - back half reversed
>>>>> sort(x) - an ordered copy
>>>>> dither(x) - add i%5 to x[i]
>>>>>
>>>>> Here is the result of execution:
>>>>> Server VM:
>>>>> http://spreadsheets.google.com/pub?key=t_EAWUkQ4mD3BIbOv8Fa-AQ&output=html
>>>>> Client VM:
>>>>> http://spreadsheets.google.com/pub?key=tdiMo8xleTxd23nKUObcz0Q&single=true&gid=0&output=html
>>>>>
>>>>> Mathematical investigations
>>>>> ---------------------------
>>>>> It is proved that for the Dual-Pivot Quicksort the average number of
>>>>> comparisons is 2*n*ln(n), the average number of swaps is 0.8*n*ln(n),
>>>>> whereas classical Quicksort algorithm has 2*n*ln(n) and 1*n*ln(n)
>>>>> respectively. Full mathematical proof see in attached proof.txt
>>>>> and proof_add.txt files. Theoretical results are also confirmed
>>>>> by experimental counting of the operations.
>>>>>
>>>>> Diff between current and new implementation of Quicksort
>>>>> --------------------------------------------------------
>>>>>
>>>>> Here is the link to the diff for java.util.Arrays class:
>>>>> http://cr.openjdk.java.net/~alanb/6880672/webrev.00
>>>>>
>>>>> If you like to look and play with new algorithm,
>>>>> please, take attached class DualPivotQuicksort.java
>>>>>
>>>>> Feedback
>>>>> --------
>>>>>
>>>>> Also I'd like to share a feedback from Joshua Bloch and
>>>>> Jon Bentley who spent a lot of time investigating this
>>>>> algorithm, who gave me many advices and tips how to
>>>>> make new Quicksort better.
>>>>>
>>>>> -------- Original Message --------
>>>>> Subject: Re: Integration of new Dual-Pivot Quicksort into JDK 7
>>>>> Date: Thu, 10 Sep 2009 07:20:11 -0700
>>>>> From: Joshua Bloch
>>>>>
>>>>> Jon also says that Vladimir should make every reasonable improvement to
>>>>> the basic method before checking in the code. In his words, "It would
>>>>> be
>>>>> horrible to put the new code into the library, and then have someone
>>>>> else come along and speed it up by another 20% by using standard
>>>>> techniques." I believe it's not unlikely that this code may end up
>>>>> getting ported to many languages and widely deployed in much the manner
>>>>> of Bentley and McIlroy's fine sort (which is nearing 20 successful
>>>>> years
>>>>> in the field). Jon will help Vladimir do this.
>>>>>
>>>>> -------- Original Message --------
>>>>> Subject: Dual-Pivot Quicksort: Next Steps
>>>>> Date: Wed, 09 Sep 2009 15:02:25 -0400
>>>>> From: Jon Bentley
>>>>>
>>>>> Vladimir, Josh,
>>>>> I *finally* feel like I understand what is going on. Now that I
>>>>> (think that) I see it, it seems straightforward and obvious.
>>>>> Tony Hoare developed Quicksort in the early 1960s. I was very
>>>>> proud to make minor contributions to a particularly clean (binary)
>>>>> quicksort in the mid 1980s, to a relatively straightforward, industrial
>>>>> strength Quicksort with McIlroy in the early 1990s, and then to
>>>>> algorithms and data structures for strings with Sedgewick in the mid
>>>>> 1990s.
>>>>> I think that Vladimir's contributions to Quicksort go way beyond
>>>>> anything that I've ever done, and rank up there with Hoare's original
>>>>> design and Sedgewick's analysis. I feel so privileged to play a very,
>>>>> very minor role in helping Vladimir with the most excellent work!
>>>>>
>>>>> -----------------------------------------------
>>>>>
>>>>> Let me know, if you have any questions/comments.
>>>>>
>>>>> Thank you,
>>>>> Vladimir
>>>>>
>>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090912/280f7a30/attachment.html
From gokdogan at gmail.com Sat Sep 12 09:22:41 2009
From: gokdogan at gmail.com (Goktug Gokdogan)
Date: Sat, 12 Sep 2009 19:22:41 +0300
Subject: Replacement of Quicksort in java.util.Arrays with new
Dual-Pivot
In-Reply-To:
References:
Message-ID:
In fact, at first I intentionally skipped the warming-up code, thinking that
use of non-jvm optimized code could give some idea if more fine-tuning
required in the implementation. Then I realized that results also includes
the class loading times :(. So I added code that will just load classes by
sorting an empty array. After change Array.sort was still giving competitive
results at 1000 items. (points to my previous theory about fine-tuning?)
Finally, I decided give the code into hot-spots ingenious hands, dual-pivot
sorted 5000 items in about %35 shorter times.Good job!
2009/9/12 Vladimir Iaroslavski
> Hello,
>
> As you mentioned, warm up is needed.
> Add just several lines as here:
> ...
> Random random = new Random();
>
> DualPivotQuicksort.sort(array);
> DualPivotQuicksort.sort(array);
> Arrays.sort(array2);
> Arrays.sort(array2);
>
> for (int i = 0; i < n; i++) {
> int r = random.nextInt(n);
> array[i] = r;
> array2[i] = r;
> }
> ...
> after that I see ratio of time:
>
> 943761 - Arrays
> 534816 - DualPivotQuicksort
>
> do you?
>
> In original your version where you have only one call
> of Quicksorts, a lot of time is spent for loading classes.
>
> And I also note that it is better to run algorithms many times
> (say, 50-100 times) and take average (or minimum) time.
>
> Thank you,
> Vladimir
>
> P.S. Benchmarking in Java is "Dark Art"
>
> -----Original Message-----
> From: Goktug Gokdogan
> To: Vladimir Iaroslavski
> Date: Sat, 12 Sep 2009 15:13:17 +0300
> Subject: Re: Replacement of Quicksort in java.util.Arrays with new
> Dual-Pivot
> Quicksort
>
> > Sorry :( . Please ignore previous post. Warming-up yield to better
> results
> > in dual-pivot's favor.
> >
> > On Sat, Sep 12, 2009 at 2:43 PM, Goktug Gokdogan
> wrote:
> >
> > > My absolutely non-scientific preliminary tests indicate Arrays.sort
> > > performs better with the same input (5000 items) in nearly all runs
> > > (-client).
> > > public static void main(String[] args) {
> > > final int n = 5000;
> > > int[] array = new int[n];
> > > int[] array2 = new int[n];
> > > Random random = new Random();
> > > for (int i = 0; i < n; i++) {
> > > int r = random.nextInt(n);
> > > array[i] = r;
> > > array2[i] = r;
> > > }
> > >
> > > long st1 = System.nanoTime();
> > > Arrays.sort(array2);
> > > long st2 = System.nanoTime();
> > > DualPivotQuicksort.sort(array);
> > > long end = System.nanoTime();
> > > System.out.println(st2 - st1);
> > > System.out.println(end - st2);
> > > }
> > >
> > > I tried changing which sort runs first, but the results did not change.
> > > Over 100.000 items, dual-pivot consistently beats Arrays.sort.
> > >
> > > Well, this test is not a good reference though :)
> > >
> > > On Fri, Sep 11, 2009 at 5:27 PM, Vladimir Iaroslavski <
> iaroslavski at mail.ru
> > > > wrote:
> > >
> > >> Hi,
> > >>
> > >> I've tried to use local variable int ak = a[k] in the loop
> > >> and not found saving of time. There are 3 occurrences of a[k]
> > >> in each loop, but only two can be changed because third
> > >> comes after line: swap(a, k, great--);
> > >>
> > >> As summary: I'm changing only hardcoded constant by
> > >> private static final variables.
> > >>
> > >> Thank you,
> > >> Vladimir
> > >>
> > >> Ulf Zibis wrote:
> > >>
> > >>> Am 11.09.2009 15:32, R?mi Forax schrieb:
> > >>>
> > >>>> just my two cents :
> > >>>> In the loop tagged "sorting" and "equals element",
> > >>>> a[k] can be stored in a local variable to avoid to recompute it
> several
> > >>>> time.
> > >>>
> > >>> I add 2 cents more:
> > >>> Doesn't HotSpot see this, and optimize accordingly?
> > >>> IMO: It's time that javac should do such optimization, as there are
> less
> > >>> optimized VM's in the world.
> > >>>
> > >>> -Ulf
> > >>>
> > >>>> The algorithm use two constants: 37 and 13,
> > >>>> I think they should be declared as private static final.
> > >>>>
> > >>>> R?mi
> > >>>>
> > >>>> Le 11/09/2009 12:35, Vladimir Yaroslavskiy a ?crit :
> > >>>>
> > >>>>> Hello All,
> > >>>>>
> > >>>>> I'd like to share with you new Dual-Pivot Quicksort which is
> > >>>>> faster than the known implementations (theoretically and
> > >>>>> experimental). I'd like to propose to replace the JDK's
> > >>>>> Quicksort implementation by new one.
> > >>>>>
> > >>>>> Description
> > >>>>> -----------
> > >>>>> The classical Quicksort algorithm uses the following scheme:
> > >>>>>
> > >>>>> 1. Pick an element P, called a pivot, from the array.
> > >>>>> 2. Reorder the array so that all elements, which are less than
> > >>>>> the pivot, come before the pivot and all elements greater than
> > >>>>> the pivot come after it (equal values can go either way). After
> > >>>>> this partitioning, the pivot element is in its final position.
> > >>>>> 3. Recursively sort the sub-array of lesser elements and the
> > >>>>> sub-array of greater elements.
> > >>>>>
> > >>>>> The invariant of classical Quicksort is:
> > >>>>>
> > >>>>> [ <= p | >= p ]
> > >>>>>
> > >>>>> There are several modifications of the schema:
> > >>>>>
> > >>>>> [ < p | = p | > p ] or [ = p | < p | > p | = p ]
> > >>>>>
> > >>>>> But all of them use *one* pivot.
> > >>>>>
> > >>>>> The new Dual-Pivot Quicksort uses *two* pivots elements in this
> manner:
> > >>>>>
> > >>>>> 1. Pick an elements P1, P2, called pivots from the array.
> > >>>>> 2. Assume that P1 <= P2, otherwise swap it.
> > >>>>> 3. Reorder the array into three parts: those less than the smaller
> > >>>>> pivot, those larger than the larger pivot, and in between are
> > >>>>> those elements between (or equal to) the two pivots.
> > >>>>> 4. Recursively sort the sub-arrays.
> > >>>>>
> > >>>>> The invariant of the Dual-Pivot Quicksort is:
> > >>>>>
> > >>>>> [ < P1 | P1 <= & <= P2 } > P2 ]
> > >>>>>
> > >>>>> The new Quicksort is faster than current implementation of
> Quicksort
> > >>>>> in JDK (L. Bentley and M. Douglas McIlroy) and classical Quicksort.
> > >>>>>
> > >>>>> The full description of the Dual-Pivot Quicksort you can find
> > >>>>> on my page: http://iaroslavski.narod.ru/quicksort
> > >>>>>
> > >>>>> Performance tests
> > >>>>> -----------------
> > >>>>> Here is result of running on different types of input data:
> > >>>>>
> > >>>>> Client VM all 85% organ 0..1
> > >>>>> 0..4
> > >>>>> random ascend descend equal equal pipes random 010101
> > >>>>> random
> > >>>>> Dual-Pivot: 16.83 5.31 5.47 0.35 0.68 10.59 1.06 1.02
> > >>>>> 2.18
> > >>>>> Bentley's: 19.77 9.08 10.13 0.63 1.12 13.22 1.63 1.08
> > >>>>> 2.49
> > >>>>>
> > >>>>> Server VM all 85% organ 0..1
> > >>>>> 0..4
> > >>>>> random ascend descend equal equal pipes random 010101
> > >>>>> random
> > >>>>> Dual-Pivot: 23.94 6.68 6.63 0.43 0.62 17.14 1.42 1.96
> > >>>>> 3.41
> > >>>>> Bentley's: 25.20 10.18 10.32 2.07 1.33 16.72 2.95 1.82
> > >>>>> 3.39
> > >>>>>
> > >>>>> The a lot of other tests have been run under client and server
> mode.
> > >>>>> The most interesting is BentleyBasher test framework. It runs
> battery
> > >>>>> of tests for all cases:
> > >>>>>
> > >>>>> { 100, 1000, 10000, 1000000 } x
> > >>>>> { sawtooth, rand, stagger, plateau, shuffle } x
> > >>>>> { ident, reverse, reverse_front, reverse_back, sort, dither}
> > >>>>>
> > >>>>> where
> > >>>>>
> > >>>>> 100, ... , 1000000 - array length
> > >>>>>
> > >>>>> sawtooth: x[i] =i%m
> > >>>>> rand: x[i] = rand() % m
> > >>>>> stagger: x[i] = (i*m + i) % n
> > >>>>> plateau: x[i] = min(i, m)
> > >>>>> shuffle: x[i] = rand()%m? (j+=2): (k+=2)
> > >>>>>
> > >>>>> ident(x) - a copy of x
> > >>>>> reverse(x, 0, n) - reversed copy
> > >>>>> reverse_front(x, 0, n/2) - front half reversed
> > >>>>> reverse_back(x, n/2, n) - back half reversed
> > >>>>> sort(x) - an ordered copy
> > >>>>> dither(x) - add i%5 to x[i]
> > >>>>>
> > >>>>> Here is the result of execution:
> > >>>>> Server VM:
> > >>>>>
> http://spreadsheets.google.com/pub?key=t_EAWUkQ4mD3BIbOv8Fa-AQ&output=html
> > >>>>> Client VM:
> > >>>>>
> http://spreadsheets.google.com/pub?key=tdiMo8xleTxd23nKUObcz0Q&single=true&gid=0&output=html
> > >>>>>
> > >>>>> Mathematical investigations
> > >>>>> ---------------------------
> > >>>>> It is proved that for the Dual-Pivot Quicksort the average number
> of
> > >>>>> comparisons is 2*n*ln(n), the average number of swaps is
> 0.8*n*ln(n),
> > >>>>> whereas classical Quicksort algorithm has 2*n*ln(n) and 1*n*ln(n)
> > >>>>> respectively. Full mathematical proof see in attached proof.txt
> > >>>>> and proof_add.txt files. Theoretical results are also confirmed
> > >>>>> by experimental counting of the operations.
> > >>>>>
> > >>>>> Diff between current and new implementation of Quicksort
> > >>>>> --------------------------------------------------------
> > >>>>>
> > >>>>> Here is the link to the diff for java.util.Arrays class:
> > >>>>> http://cr.openjdk.java.net/~alanb/6880672/webrev.00
> > >>>>>
> > >>>>> If you like to look and play with new algorithm,
> > >>>>> please, take attached class DualPivotQuicksort.java
> > >>>>>
> > >>>>> Feedback
> > >>>>> --------
> > >>>>>
> > >>>>> Also I'd like to share a feedback from Joshua Bloch and
> > >>>>> Jon Bentley who spent a lot of time investigating this
> > >>>>> algorithm, who gave me many advices and tips how to
> > >>>>> make new Quicksort better.
> > >>>>>
> > >>>>> -------- Original Message --------
> > >>>>> Subject: Re: Integration of new Dual-Pivot Quicksort into JDK 7
> > >>>>> Date: Thu, 10 Sep 2009 07:20:11 -0700
> > >>>>> From: Joshua Bloch
> > >>>>>
> > >>>>> Jon also says that Vladimir should make every reasonable
> improvement to
> > >>>>> the basic method before checking in the code. In his words, "It
> would
> > >>>>> be
> > >>>>> horrible to put the new code into the library, and then have
> someone
> > >>>>> else come along and speed it up by another 20% by using standard
> > >>>>> techniques." I believe it's not unlikely that this code may end up
> > >>>>> getting ported to many languages and widely deployed in much the
> manner
> > >>>>> of Bentley and McIlroy's fine sort (which is nearing 20 successful
> > >>>>> years
> > >>>>> in the field). Jon will help Vladimir do this.
> > >>>>>
> > >>>>> -------- Original Message --------
> > >>>>> Subject: Dual-Pivot Quicksort: Next Steps
> > >>>>> Date: Wed, 09 Sep 2009 15:02:25 -0400
> > >>>>> From: Jon Bentley
> > >>>>>
> > >>>>> Vladimir, Josh,
> > >>>>> I *finally* feel like I understand what is going on. Now that
> I
> > >>>>> (think that) I see it, it seems straightforward and obvious.
> > >>>>> Tony Hoare developed Quicksort in the early 1960s. I was very
> > >>>>> proud to make minor contributions to a particularly clean (binary)
> > >>>>> quicksort in the mid 1980s, to a relatively straightforward,
> industrial
> > >>>>> strength Quicksort with McIlroy in the early 1990s, and then to
> > >>>>> algorithms and data structures for strings with Sedgewick in the
> mid
> > >>>>> 1990s.
> > >>>>> I think that Vladimir's contributions to Quicksort go way
> beyond
> > >>>>> anything that I've ever done, and rank up there with Hoare's
> original
> > >>>>> design and Sedgewick's analysis. I feel so privileged to play a
> very,
> > >>>>> very minor role in helping Vladimir with the most excellent work!
> > >>>>>
> > >>>>> -----------------------------------------------
> > >>>>>
> > >>>>> Let me know, if you have any questions/comments.
> > >>>>>
> > >>>>> Thank you,
> > >>>>> Vladimir
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090912/e540adfb/attachment-0001.html
From jjb at google.com Sat Sep 12 11:52:12 2009
From: jjb at google.com (Joshua Bloch)
Date: Sat, 12 Sep 2009 11:52:12 -0700
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
Quicksort
In-Reply-To: <4AAA27EA.8090902@Sun.COM>
References: <4AAA27EA.8090902@Sun.COM>
Message-ID: <17b2302a0909121152v421611a8l4a3d25357991bb0e@mail.gmail.com>
To amplify my previous statement, I think this is a great piece of work!
Vladimir is to be commended. I also think that it may well get
substantially faster as Vladimir continues to make minor algorithmic
modifications. Jon Bentley has made many fine suggestions that Vladimir will
try out.There are also some lower-level performance tweaks that may well
yield improved performance (e.g., eliminate the div parameter from
dualPivotQuicksort, eliminate the division from algorithm). It is, of
course, possible that this performance work won't yield significant
benefits, but I suspect otherwise.
Josh
P.S. When all of this is done, the code could use some minor reformatting
to match JDK standards, but I wouldn't worry about this until the
performance work is done.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090912/c98ae95e/attachment.html
From martinrb at google.com Sat Sep 12 14:45:20 2009
From: martinrb at google.com (Martin Buchholz)
Date: Sat, 12 Sep 2009 14:45:20 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AAAD99A.4050505@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
<1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
<4AA7F9EB.5090200@sun.com> <4AA816AD.7090306@sun.com>
<4AA949DA.9010707@sun.com> <4AAA1B75.6060309@sun.com>
<4AAAD99A.4050505@sun.com>
Message-ID: <1ccfd1c10909121445p7361dbe9l6ce7e79e178b1519@mail.gmail.com>
On Fri, Sep 11, 2009 at 16:13, Xueming Shen wrote:
>
> I'm definitely open for any better choice, I'm not in hurry to close this
> one up. Sure, I hope we can
> reach a consensus before I go on vacation this weekend:-)
>
> So here is the planB alan is proposing, no API doc added yet, but you should
> know what it means. I will add the doc
> in later if we agree on this approach.
>
> http://cr.openjdk.java.net/~sherman/zipflush/webrev.planB/src/share/classes/java/util/zip/DeflaterOutputStream.java.sdiff.html
I vote for this approach - i.e. plan B as proposed by Alan.
I no longer advocate making partial flush the default,
as a result of my own arguments for the other side.
The use case for full flush is partial recoverability from future corruption
of the data. If the competing java zib libraries are supporting this
flush variant,
we probably should too, for completeness.
David wrote:
> This is not unusual in the JDK or elsewhere - that a stream calls the underlying stream's flush() method I mean - and I think in the (unlikely) event that someone wants to flush the underlying stream only, they could simply call .flush() directly on that stream.
David: In this case the deflater is not an "underlying stream", and
the proposal was to
have flush() change the nature of the bytes written, while normally
OutputStream.flush merely causes
the same bytes to be written more promptly.
Martin "Every change is an incompatible change"
>
> We probably should figure out want to do with the GZipOutputStream...it
> probably needs something as well...we can figure
> out it later.
>
> So, Martin, Brandon, David and anyone interested,
>
> Is this something you guys can accept ? It makes the life easy for developer
> that needs the sync_flush for the flush(),
> you don't need to override/subclass anything, just makes it clear what you
> want when creating the DOS by picking
> the appropriate constructor, nothing more. Since we keep the default
> behavior un-changed, it is also compatible:-)
>
> It does not add the the full_sync, sure we can consider to use a enum
> instead of the boolean parameter, but since
> no one is asking for it now and the enum looks a little "too modern"
> compared to the rest of the API, we might
> want to stay with the simple "boolean doSyncFlush".
>
> Opinion?
>
> sherman
>
> Funny, I was "beaten up" today for one of my other changes, I was literally
> asked "where was it documented/highlighted
> that this fix was going to introduce an incompatibility", for a much much
> "less" incompatible change:-)
>
>
> Alan Bateman wrote:
>>
>> Xueming Shen wrote:
>>>
>>> I think we have enough discussion on this topic, it's time to make a
>>> final decision. Seems like we are all happy on the
>>> changes in Deflater and new DOS.flush(mode), the only difference is
>>> whether or not we should change the existing
>>> behavior of DOS.flush() should use Z_SYNC_FLUSH by default.
>>>
>>> Brandon is very firm on the "yes" side, strongly believe this is a MUST.
>>> Martin is on the fence, leaning towards "yes"
>>> Sherman is on the "no" side
>>>
>>> So if there is no other vote coming with strong opinion, your vote will
>>> be important:-) Are you also leaning to "yes"
>>> or "strongly" suggesting a yes? I need ?strong "yes" to change my mind.
>>
>> This is tough call. It is clearly desirable to change flush() to work the
>> way that developers expect. On the other hand there is always risk with
>> changing long-standing behavior. The risk here is probably low but it's
>> impossible to quantify. Martin makes a good observation that it may cause
>> problems for code that wraps the stream with an auto-flushing PrintStream or
>> PrintWriter. You also made a good observation that just changing the number
>> of bytes written could cause breakage (in some fragile environment) - we
>> just don't know! So on balance, I think you are probably right to take the
>> conservative line on this one.
>>
>> I know you've had enough discussion on this but I wonder if you might be
>> open to considering new constructors for DeflaterOutputStream (and maybe
>> GZIPOutputStream) as an alternative to the flush(int) method. This might fit
>> better with this API, in particular for cases where the intended usage is
>> known. It could be as simple as a "boolean syncFlush" parameter to indicate
>> how uncompressed buffered bytes are handled when flushing (default to false
>> for existing constructors so existing behavior preserved). ?The boolean
>> parameter is just an opening bid - clearly there are alternatives. The point
>> is that it makes it easy and more obvious as how to get a
>> DeflaterOutputStream suitable for interactive cases.
>>
>> -Alan.
>
>
From martinrb at google.com Sat Sep 12 15:32:32 2009
From: martinrb at google.com (martinrb at google.com)
Date: Sat, 12 Sep 2009 22:32:32 +0000
Subject: hg: jdk7/tl/jdk: 6881442: (reflect) Race condition in Class.getName()
Message-ID: <20090912223309.8736A12067@hg.openjdk.java.net>
Changeset: 020a0fed38c9
Author: martin
Date: 2009-09-12 15:30 -0700
URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/020a0fed38c9
6881442: (reflect) Race condition in Class.getName()
Summary: only read "name" field racily once
Reviewed-by: darcy
! src/share/classes/java/lang/Class.java
From Alan.Bateman at Sun.COM Sun Sep 13 01:07:15 2009
From: Alan.Bateman at Sun.COM (Alan Bateman)
Date: Sun, 13 Sep 2009 09:07:15 +0100
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4AAA9372.3070200@Sun.com>
References: <4A9EC69F.6000509@sun.com> <4AAA9372.3070200@Sun.com>
Message-ID: <4AACA833.9030204@sun.com>
Mandy Chung wrote:
> Alan, David, Remi, Iris,
>
> Thanks for the review and the revised webrev at:
> http://cr.openjdk.java.net/~mchung/6878481/webrev.01/
>
> I incorporate the comments you have and minimize the number of
> System.nanoTime() calls and also remove the number of opened jars perf
> counters. Please let me know if you have further comments.
This looks much better, in particular the overhead in ClassLoader is
much reduced. Also, I see you've removed the special counter for JAR
files, which make sense.
Minor nit but I assume you can initialize perf as:
private static final Perf perf =
AccessController.doPrivileged(new Perf.GetPerfAction());
Also minor nit: in the javadoc of the PerfCounter's public methods - I
assume you meant to use /** instead of /*.
Method naming is hard (and often subjective) but there are updates like
this:
PerfCounter.getParentDelegationTime.inc(t1 - t0);
which might be easier to read as:
PerfCounter.getParentDelegationCounter().addTime(t1 - t0)
Also:
PerfCounter.getZipFileCount().inc();
which might be cleaer as:
PerfCounter.getZipFileCounter().increment();
Just a suggestion of course. Otherwise, looks okay to me and I'm looking
forward to seeing the output once these changes are in.
-Alan.
From gokdogan at gmail.com Sun Sep 13 08:23:57 2009
From: gokdogan at gmail.com (Goktug Gokdogan)
Date: Sun, 13 Sep 2009 18:23:57 +0300
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
Quicksort
In-Reply-To: <4AAA5E4D.6050909@mail.ru>
References: <4AAA27EA.8090902@Sun.COM> <4AAA5158.8060202@univ-mlv.fr>
<4AAA54E4.7090107@gmx.de> <4AAA5E4D.6050909@mail.ru>
Message-ID:
I reviewed the code. A few simple tricks helped me to speed-up code in my
tests:1. Falling back-to insertion sort at <17 instead of <27 (JDK 6
Arrays.sort falls back <7)
2. (a[great] > pivot2) test is more likely to fail compared to (k < great)
in the while loop, so exchange them (ie. use while (a[great] > pivot2 && k <
great)).
These two changes together improved performance around 4-5 percent in my
laptop.
Hopefully, I will dig algorithm further for other suggestions; if I can
spare more time this week.
Congratulations again for your algorithm!
Goktug
On Fri, Sep 11, 2009 at 5:27 PM, Vladimir Iaroslavski
wrote:
> Hi,
>
> I've tried to use local variable int ak = a[k] in the loop
> and not found saving of time. There are 3 occurrences of a[k]
> in each loop, but only two can be changed because third
> comes after line: swap(a, k, great--);
>
> As summary: I'm changing only hardcoded constant by
> private static final variables.
>
> Thank you,
> Vladimir
>
>
> Ulf Zibis wrote:
>
>> Am 11.09.2009 15:32, R?mi Forax schrieb:
>>
>>> just my two cents :
>>> In the loop tagged "sorting" and "equals element",
>>> a[k] can be stored in a local variable to avoid to recompute it several
>>> time.
>>>
>>
>> I add 2 cents more:
>> Doesn't HotSpot see this, and optimize accordingly?
>> IMO: It's time that javac should do such optimization, as there are less
>> optimized VM's in the world.
>>
>> -Ulf
>>
>>
>>
>>> The algorithm use two constants: 37 and 13,
>>> I think they should be declared as private static final.
>>>
>>> R?mi
>>>
>>>
>>> Le 11/09/2009 12:35, Vladimir Yaroslavskiy a ?crit :
>>>
>>>> Hello All,
>>>>
>>>> I'd like to share with you new Dual-Pivot Quicksort which is
>>>> faster than the known implementations (theoretically and
>>>> experimental). I'd like to propose to replace the JDK's
>>>> Quicksort implementation by new one.
>>>>
>>>> Description
>>>> -----------
>>>> The classical Quicksort algorithm uses the following scheme:
>>>>
>>>> 1. Pick an element P, called a pivot, from the array.
>>>> 2. Reorder the array so that all elements, which are less than
>>>> the pivot, come before the pivot and all elements greater than
>>>> the pivot come after it (equal values can go either way). After
>>>> this partitioning, the pivot element is in its final position.
>>>> 3. Recursively sort the sub-array of lesser elements and the
>>>> sub-array of greater elements.
>>>>
>>>> The invariant of classical Quicksort is:
>>>>
>>>> [ <= p | >= p ]
>>>>
>>>> There are several modifications of the schema:
>>>>
>>>> [ < p | = p | > p ] or [ = p | < p | > p | = p ]
>>>>
>>>> But all of them use *one* pivot.
>>>>
>>>> The new Dual-Pivot Quicksort uses *two* pivots elements in this manner:
>>>>
>>>> 1. Pick an elements P1, P2, called pivots from the array.
>>>> 2. Assume that P1 <= P2, otherwise swap it.
>>>> 3. Reorder the array into three parts: those less than the smaller
>>>> pivot, those larger than the larger pivot, and in between are
>>>> those elements between (or equal to) the two pivots.
>>>> 4. Recursively sort the sub-arrays.
>>>>
>>>> The invariant of the Dual-Pivot Quicksort is:
>>>>
>>>> [ < P1 | P1 <= & <= P2 } > P2 ]
>>>>
>>>> The new Quicksort is faster than current implementation of Quicksort
>>>> in JDK (L. Bentley and M. Douglas McIlroy) and classical Quicksort.
>>>>
>>>> The full description of the Dual-Pivot Quicksort you can find
>>>> on my page: http://iaroslavski.narod.ru/quicksort
>>>>
>>>> Performance tests
>>>> -----------------
>>>> Here is result of running on different types of input data:
>>>>
>>>> Client VM all 85% organ 0..1 0..4
>>>> random ascend descend equal equal pipes random 010101
>>>> random
>>>> Dual-Pivot: 16.83 5.31 5.47 0.35 0.68 10.59 1.06 1.02 2.18
>>>> Bentley's: 19.77 9.08 10.13 0.63 1.12 13.22 1.63 1.08 2.49
>>>>
>>>> Server VM all 85% organ 0..1 0..4
>>>> random ascend descend equal equal pipes random 010101
>>>> random
>>>> Dual-Pivot: 23.94 6.68 6.63 0.43 0.62 17.14 1.42 1.96 3.41
>>>> Bentley's: 25.20 10.18 10.32 2.07 1.33 16.72 2.95 1.82 3.39
>>>>
>>>> The a lot of other tests have been run under client and server mode.
>>>> The most interesting is BentleyBasher test framework. It runs battery
>>>> of tests for all cases:
>>>>
>>>> { 100, 1000, 10000, 1000000 } x
>>>> { sawtooth, rand, stagger, plateau, shuffle } x
>>>> { ident, reverse, reverse_front, reverse_back, sort, dither}
>>>>
>>>> where
>>>>
>>>> 100, ... , 1000000 - array length
>>>>
>>>> sawtooth: x[i] =i%m
>>>> rand: x[i] = rand() % m
>>>> stagger: x[i] = (i*m + i) % n
>>>> plateau: x[i] = min(i, m)
>>>> shuffle: x[i] = rand()%m? (j+=2): (k+=2)
>>>>
>>>> ident(x) - a copy of x
>>>> reverse(x, 0, n) - reversed copy
>>>> reverse_front(x, 0, n/2) - front half reversed
>>>> reverse_back(x, n/2, n) - back half reversed
>>>> sort(x) - an ordered copy
>>>> dither(x) - add i%5 to x[i]
>>>>
>>>> Here is the result of execution:
>>>> Server VM:
>>>> http://spreadsheets.google.com/pub?key=t_EAWUkQ4mD3BIbOv8Fa-AQ&output=html
>>>> Client VM:
>>>> http://spreadsheets.google.com/pub?key=tdiMo8xleTxd23nKUObcz0Q&single=true&gid=0&output=html
>>>>
>>>> Mathematical investigations
>>>> ---------------------------
>>>> It is proved that for the Dual-Pivot Quicksort the average number of
>>>> comparisons is 2*n*ln(n), the average number of swaps is 0.8*n*ln(n),
>>>> whereas classical Quicksort algorithm has 2*n*ln(n) and 1*n*ln(n)
>>>> respectively. Full mathematical proof see in attached proof.txt
>>>> and proof_add.txt files. Theoretical results are also confirmed
>>>> by experimental counting of the operations.
>>>>
>>>> Diff between current and new implementation of Quicksort
>>>> --------------------------------------------------------
>>>>
>>>> Here is the link to the diff for java.util.Arrays class:
>>>> http://cr.openjdk.java.net/~alanb/6880672/webrev.00
>>>>
>>>> If you like to look and play with new algorithm,
>>>> please, take attached class DualPivotQuicksort.java
>>>>
>>>> Feedback
>>>> --------
>>>>
>>>> Also I'd like to share a feedback from Joshua Bloch and
>>>> Jon Bentley who spent a lot of time investigating this
>>>> algorithm, who gave me many advices and tips how to
>>>> make new Quicksort better.
>>>>
>>>> -------- Original Message --------
>>>> Subject: Re: Integration of new Dual-Pivot Quicksort into JDK 7
>>>> Date: Thu, 10 Sep 2009 07:20:11 -0700
>>>> From: Joshua Bloch
>>>>
>>>> Jon also says that Vladimir should make every reasonable improvement to
>>>> the basic method before checking in the code. In his words, "It would be
>>>> horrible to put the new code into the library, and then have someone
>>>> else come along and speed it up by another 20% by using standard
>>>> techniques." I believe it's not unlikely that this code may end up
>>>> getting ported to many languages and widely deployed in much the manner
>>>> of Bentley and McIlroy's fine sort (which is nearing 20 successful years
>>>> in the field). Jon will help Vladimir do this.
>>>>
>>>> -------- Original Message --------
>>>> Subject: Dual-Pivot Quicksort: Next Steps
>>>> Date: Wed, 09 Sep 2009 15:02:25 -0400
>>>> From: Jon Bentley
>>>>
>>>> Vladimir, Josh,
>>>> I *finally* feel like I understand what is going on. Now that I
>>>> (think that) I see it, it seems straightforward and obvious.
>>>> Tony Hoare developed Quicksort in the early 1960s. I was very
>>>> proud to make minor contributions to a particularly clean (binary)
>>>> quicksort in the mid 1980s, to a relatively straightforward, industrial
>>>> strength Quicksort with McIlroy in the early 1990s, and then to
>>>> algorithms and data structures for strings with Sedgewick in the mid
>>>> 1990s.
>>>> I think that Vladimir's contributions to Quicksort go way beyond
>>>> anything that I've ever done, and rank up there with Hoare's original
>>>> design and Sedgewick's analysis. I feel so privileged to play a very,
>>>> very minor role in helping Vladimir with the most excellent work!
>>>>
>>>> -----------------------------------------------
>>>>
>>>> Let me know, if you have any questions/comments.
>>>>
>>>> Thank you,
>>>> Vladimir
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090913/acf830b6/attachment-0001.html
From david.lloyd at redhat.com Sun Sep 13 08:49:50 2009
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Sun, 13 Sep 2009 10:49:50 -0500
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <1ccfd1c10909121445p7361dbe9l6ce7e79e178b1519@mail.gmail.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com> <4AA1B890.7000703@sun.com> <1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com> <1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com> <4AA7F9EB.5090200@sun.com>
<4AA816AD.7090306@sun.com> <4AA949DA.9010707@sun.com>
<4AAA1B75.6060309@sun.com> <4AAAD99A.4050505@sun.com>
<1ccfd1c10909121445p7361dbe9l6ce7e79e178b1519@mail.gmail.com>
Message-ID: <4AAD149E.2020906@redhat.com>
On 09/12/2009 04:45 PM, Martin Buchholz wrote:
> On Fri, Sep 11, 2009 at 16:13, Xueming Shen wrote:
> David wrote:
>> This is not unusual in the JDK or elsewhere - that a stream calls the underlying stream's flush() method I mean - and I think in the (unlikely) event that someone wants to flush the underlying stream only, they could simply call .flush() directly on that stream.
>
> David: In this case the deflater is not an "underlying stream", and the proposal was to
> have flush() change the nature of the bytes written, while normally OutputStream.flush merely causes
> the same bytes to be written more promptly.
In this paragraph I was referring to the real underlying OutputStream
though, not the Deflater (afaict the existing implementation just calls
through to the underlying stream currently).
The second paragraph is where I talk about the Deflater. And anyway I
change my vote from "yes-kinda-no-but-qualified-yes" in favor of the plan B
patch which I like a lot.
- DML
From oleg.anashkin at gmail.com Sun Sep 13 09:32:09 2009
From: oleg.anashkin at gmail.com (Oleg Anashkin)
Date: Sun, 13 Sep 2009 16:32:09 +0000 (UTC)
Subject: Replacement of Quicksort in java.util.Arrays with new
=?utf-8?b?RHVhbC1QaXZvdAlRdWlja3NvcnQ=?=
References: <4AAA27EA.8090902@Sun.COM>
Message-ID:
Hello Vladimir,
First thing that came to mind - have you thought about extrapolating this
approach to more pivots? If 2-pivot algorithm is faster than 1-pivot, then
3-pivot might be even faster, right? Can the number of pivots be chosen as a
function of array size (to mitigate overhead)?
Thanks,
Oleg
From Alan.Bateman at Sun.COM Sun Sep 13 10:11:16 2009
From: Alan.Bateman at Sun.COM (Alan Bateman)
Date: Sun, 13 Sep 2009 18:11:16 +0100
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <1ccfd1c10909121445p7361dbe9l6ce7e79e178b1519@mail.gmail.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
<1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
<4AA7F9EB.5090200@sun.com> <4AA816AD.7090306@sun.com>
<4AA949DA.9010707@sun.com> <4AAA1B75.6060309@sun.com>
<4AAAD99A.4050505@sun.com>
<1ccfd1c10909121445p7361dbe9l6ce7e79e178b1519@mail.gmail.com>
Message-ID: <4AAD27B4.4040507@sun.com>
Martin Buchholz wrote:
> :
> The use case for full flush is partial recoverability from future corruption
> of the data. If the competing java zib libraries are supporting this
> flush variant,
> we probably should too, for completeness.
>
It's worth considering although we get into detail as to how the flush
mode is expressed in the constructor (an int is awkward because of the
existing size parameter, and java.util.zip was well into adolescence
when enums were introduced). The other option is to just ignore it as
the proposed Deflater.deflate method supports full flush and it's not
too difficult to create your own OutputStream that uses Deflater.
-Alan.
From David.Holmes at Sun.COM Sun Sep 13 16:08:21 2009
From: David.Holmes at Sun.COM (David Holmes - Sun Microsystems)
Date: Mon, 14 Sep 2009 09:08:21 +1000
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4AACA833.9030204@sun.com>
References: <4A9EC69F.6000509@sun.com> <4AAA9372.3070200@Sun.com>
<4AACA833.9030204@sun.com>
Message-ID: <4AAD7B65.7010109@sun.com>
Just to add 2c to Alan's method naming comments:
Alan Bateman said the following on 09/13/09 18:07:
> Method naming is hard (and often subjective) but there are updates like
> this:
> PerfCounter.getParentDelegationTime.inc(t1 - t0);
> which might be easier to read as:
> PerfCounter.getParentDelegationCounter().addTime(t1 - t0)
There's always a problem with method naming when a thing can be both a
counter and a time-tracker. I would agree to use context specific
methods even at the expense of some redundancy in the public API eg:
addTime(long interval)
addElapsedTimeFrom(long startTime)
> Also:
> PerfCounter.getZipFileCount().inc();
> which might be cleaer as:
> PerfCounter.getZipFileCounter().increment();
I'd also suggest (in a convention used elsewhere eg
java.util.concurrent.atomic.*) that you reserve increment() for addition
of 1 and use add(long val) for the general case.
> Just a suggestion of course. Otherwise, looks okay to me and I'm looking
> forward to seeing the output once these changes are in.
Agreed - all looks good.
Thanks,
David
From Mandy.Chung at Sun.COM Sun Sep 13 18:31:03 2009
From: Mandy.Chung at Sun.COM (Mandy Chung)
Date: Sun, 13 Sep 2009 18:31:03 -0700
Subject: Review Request for 6878481: Add performance counters in the JDK
In-Reply-To: <4AAD7B65.7010109@sun.com>
References: <4A9EC69F.6000509@sun.com> <4AAA9372.3070200@Sun.com>
<4AACA833.9030204@sun.com> <4AAD7B65.7010109@sun.com>
Message-ID: <4AAD9CD7.8030405@sun.com>
Alan, David,
Thanks for the review. Comments inlined below.
Alan Bateman wrote:
> Minor nit but I assume you can initialize perf as:
> private static final Perf perf =
> AccessController.doPrivileged(new Perf.GetPerfAction());
Fixed.
David Holmes wrote:
> Just to add 2c to Alan's method naming comments:
>
> Alan Bateman said the following on 09/13/09 18:07:
>> Method naming is hard (and often subjective) but there are updates
>> like this:
>> PerfCounter.getParentDelegationTime.inc(t1 - t0);
>> which might be easier to read as:
>> PerfCounter.getParentDelegationCounter().addTime(t1 - t0)
>
>
> There's always a problem with method naming when a thing can be both a
> counter and a time-tracker. I would agree to use context specific
> methods even at the expense of some redundancy in the public API eg:
>
> addTime(long interval)
> addElapsedTimeFrom(long startTime)
>
>> Also:
>> PerfCounter.getZipFileCount().inc();
>> which might be cleaer as:
>> PerfCounter.getZipFileCounter().increment();
>
> I'd also suggest (in a convention used elsewhere eg
> java.util.concurrent.atomic.*) that you reserve increment() for
> addition of 1 and use add(long val) for the general case.
>
I agree that addTime(long interval), increment(), and add(long val) are
better method names. I'll make the change.
As for PerfCounter.getParentDelegationTime() vs
getParentDelegationCounter(), I would prefer to keep
getParentDelegationTime as the method name so that it's clear that this
counter is a time tracker. Similarly, it's clear that getZipFileCount()
is a counter to keep track of zip file count.
Thanks
Mandy
From Mandy.Chung at Sun.COM Sun Sep 13 18:32:29 2009
From: Mandy.Chung at Sun.COM (Mandy Chung)
Date: Sun, 13 Sep 2009 18:32:29 -0700
Subject: Review Request for 6879044
Message-ID: <4AAD9D2D.6000502@sun.com>
6879044: Eliminate the dependency of logging from the JRE core/awt/swing classes
Webrev:
http://cr.openjdk.java.net/~mchung/6879044/webrev.00/
Summary:
1. A new sun.util.logging.PlatformLogger class that will handle the log messages in a similar way as Logger but it will only delegate to java.util.logging only when it is enabled. LogManager and LogRecord are modified to support the platform loggers. The users of PlatformLogger will continue to run if java.util.logging classes do not exist.
2. AWT, 2D, Swing, and a few java.util classes are modified to use PlatformLogger instead of Logger. Although many files are modified, the change is mostly replacement with classname.
AWT statically creates a number of loggers. Running a simple AWT Framer application with JDK 7 b71 creates 79 loggers on solaris-i586 and 34 loggers on windows-i586. SwingSet2 creates a total of 85 loggers including a few non-awt ones on solaris-i586 and 35 on windows-i586).
Although the memory usage might not be very high (especially with this fix), I don't see the need of having many fine-grained loggers. This fix doesn't address this the number of AWT loggers. I file a separate CR (6880089) to revisit it.
Startup Performance:
This change does not have significant startup performance improvement, as expected. However, it does reduce the number of loaded classes (Framer app loads 16 fewer classes and jedit loads 13 fewer classes).
Thanks
Mandy
From son.two at gmail.com Sun Sep 13 20:44:27 2009
From: son.two at gmail.com (Oleg Sukhodolsky)
Date: Mon, 14 Sep 2009 07:44:27 +0400
Subject: Review Request for 6879044
In-Reply-To: <4AAD9D2D.6000502@sun.com>
References: <4AAD9D2D.6000502@sun.com>
Message-ID: <271e55d20909132044k3b6b7a5bw86e87a5e21a2c34e@mail.gmail.com>
On Mon, Sep 14, 2009 at 5:32 AM, Mandy Chung wrote:
> 6879044: Eliminate the dependency of logging from the JRE core/awt/swing
> classes
>
> Webrev:
> ?http://cr.openjdk.java.net/~mchung/6879044/webrev.00/
>
> Summary:
> 1. A new sun.util.logging.PlatformLogger class that will handle the log
> messages in a similar way as Logger but it will only delegate to
> java.util.logging only when it is enabled. ?LogManager and LogRecord are
> modified to support the platform loggers. ?The users of PlatformLogger will
> continue to run if java.util.logging classes do not exist.
evaluation for 6879044 says:
Add a PlatformLogger class that mimics the default logging behavior
(output the log message to System.err with simple formatting) and
creates a java.util.logging.Logger only when the logging facility is
used by the application or a user-defined configuration is supplied.
which of two descriptions is correct one?
Also I wonder if it is ok to get possible better modularization by
adding dependency to Sun's internal API into public packages.
> 2. AWT, 2D, Swing, and a few java.util classes are modified to use
> PlatformLogger instead of Logger. ?Although many files are modified, the
> change is mostly replacement with classname.
Well, at least in AWTEvent you have changed static initialization of
logger to lazy one.
Perhaps it is better to keep a static one to minimize your changes.
Oleg.
>
> AWT statically creates a number of loggers. Running a simple AWT Framer
> application with JDK 7 b71 creates 79 loggers on solaris-i586 and 34 loggers
> on windows-i586. SwingSet2 creates a total of 85 loggers including a few
> non-awt ones on solaris-i586 and 35 on windows-i586).
> Although the memory usage might not be very high (especially with this fix),
> I don't see the need of having many fine-grained loggers. ?This fix doesn't
> address this the number of AWT loggers. I file a separate CR (6880089) to
> revisit it.
>
> Startup Performance:
> This change does not have significant startup performance improvement, as
> expected. ?However, it does reduce the number of loaded classes (Framer app
> loads 16 fewer classes and jedit loads 13 fewer classes).
>
>
> Thanks
> Mandy
>
>
>
From Xueming.Shen at Sun.COM Sun Sep 13 21:12:22 2009
From: Xueming.Shen at Sun.COM (Xueming Shen)
Date: Sun, 13 Sep 2009 21:12:22 -0700
Subject: 4206909 - adding Z_SYNC_FLUSH support to deflaters
In-Reply-To: <4AAD27B4.4040507@sun.com>
References: <1ccfd1c10909031147g28b856fbrdaefa5fbaa411515@mail.gmail.com>
<4AA1B890.7000703@sun.com>
<1ccfd1c10909041821w12905254w1a3515e9b42d2ee3@mail.gmail.com>
<1ccfd1c10909050924v656431d0j527558cb8ef9502f@mail.gmail.com>
<4AA7F9EB.5090200@sun.com> <4AA816AD.7090306@sun.com>
<4AA949DA.9010707@sun.com> <4AAA1B75.6060309@sun.com>
<4AAAD99A.4050505@sun.com>
<1ccfd1c10909121445p7361dbe9l6ce7e79e178b1519@mail.gmail.com>
<4AAD27B4.4040507@sun.com>
Message-ID: <4AADC2A6.2090909@sun.com>
The ZOutputStream of the "competing" jzlib has its "flush" control in
its write() method,
it invokes "defalter.deflate()" with a "flush" parameter, the
"flush"parameter is a protected
instance variable of the ZOutputStream class with a default value of
z_no_flush. So its
subclass can set whatever value is desirable. Its default "flush()" does
the same thing we do,
out.flush():-) So obviously all the "happy" jzlib users do not feel too
bad to write their own
subclass to achieve whatever they can not get from our DOS/deflater.
While I still feel my original proposal is very close what the jzlib
offers. I'm happy with
Alan's approach, and I suggest we simply do the z_no_flush/z_sync_flush.
It should
not be too difficult for sophisticated developer to write their own DOS
with the new
deflater class.
sherman
Alan Bateman wrote:
> Martin Buchholz wrote:
>> :
>> The use case for full flush is partial recoverability from future
>> corruption
>> of the data. If the competing java zib libraries are supporting this
>> flush variant,
>> we probably should too, for completeness.
>>
> It's worth considering although we get into detail as to how the flush
> mode is expressed in the constructor (an int is awkward because of the
> existing size parameter, and java.util.zip was well into adolescence
> when enums were introduced). The other option is to just ignore it as
> the proposed Deflater.deflate method supports full flush and it's not
> too difficult to create your own OutputStream that uses Deflater.
>
> -Alan.
From Alan.Bateman at Sun.COM Mon Sep 14 02:17:07 2009
From: Alan.Bateman at Sun.COM (Alan Bateman)
Date: Mon, 14 Sep 2009 10:17:07 +0100
Subject: Review Request for 6879044
In-Reply-To: <4AAD9D2D.6000502@sun.com>
References: <4AAD9D2D.6000502@sun.com>
Message-ID: <4AAE0A13.3080709@sun.com>
Mandy Chung wrote:
> 6879044: Eliminate the dependency of logging from the JRE
> core/awt/swing classes
>
> Webrev:
> http://cr.openjdk.java.net/~mchung/6879044/webrev.00/
The approach seems reasonable to me. It is a bit strange to redirect the
platform logging to j.u.logging if something up the stack starts logging
but I think we can live with this because these loggers are for
debugging purposes.
The changes to the AWT code are mostly mechanical, so I've only skimmed
these (assuming that someone from the AWT team with do a detailed check).
Have you tested this with a security manager set? I'm just wondering if
PlatformLogger's static initializer should do the property lookup in a
doPrivileged block. Also in the LoggerProxy for the line separator (BTW:
lineSeparator can be final).
Is isLoggingEnabled() used anywhere? I can't see it and wonder if it
should be removed. If it is used, then I assume it needs to be
synchronized with redirectPlatformLoggers.
You allow the PlatformLoggers to be GC'ed but the entries will remain in
the loggers map. I don't think this is a big deal because the AWT
classes will not be unloaded but might be worth putting a note in a
comment to re-visit this some time.
The static initializer in the forwarding proxy seems a bit messy. It
might look neater if changed to something like:
private static final Class loggerClass =
getClass("java.lang.Logger");
private static final Method getLoggerMethod = getMethod(loggerClass,
"getLogger", String.class);
where getClass does the Class.forName, returning null if CNF, getMethod
returns null if passed a null class, etc.
Minor nit but there are a few style/formatting issues in PlatformLogger.
For example, in JavaLogger there isn't a blank line between methods. It
might be worthing taking a pass over this to have it as neat as possible.
Do you have a bugID to track updating the http protocol handler? Jessie
did push some changes to eliminate the static dependency on logging and
it would be good if that code used PlatformLogger.
-Alan.
From Weijun.Wang at Sun.COM Mon Sep 14 05:40:53 2009
From: Weijun.Wang at Sun.COM (Max (Weijun) Wang)
Date: Mon, 14 Sep 2009 20:40:53 +0800
Subject: Date precision on Windows
Message-ID:
Hi All
Recently one of my Kerberos tests always reports "replay detected"
error. Turns out the time precision on Windows is quite low.
I've written this tiny program. It calls new Date().getTime() as fast
as possible and try to see if the output is continuous:
class A {
public static void main(String[] rags) throws Exception {
int[] x = new int[100];
long l = new java.util.Date().getTime();
while (true) {
long m = new java.util.Date().getTime();
if (m - l >= 100) break;
x[(int)(m-l)]++;
}
for (int i=0; i<100; i++) {
System.out.printf("%5d %d\n", i, x[i]);
}
}
}
On Linux and Solaris, there are only 2 to 5 zeros in the output, which
shows the time is quite continuous on these systems. However, on
Windows (a x64 system), only 1/16th are non zeros, which shows the
maximum precision on Windows looks like 16 millisec.
Can this be enhanced?
Thanks
Max
From aph at redhat.com Mon Sep 14 06:27:35 2009
From: aph at redhat.com (Andrew Haley)
Date: Mon, 14 Sep 2009 14:27:35 +0100
Subject: Date precision on Windows
In-Reply-To:
References:
Message-ID: <4AAE44C7.5000208@redhat.com>
Max (Weijun) Wang wrote:
> Recently one of my Kerberos tests always reports "replay detected"
> error. Turns out the time precision on Windows is quite low.
>
> I've written this tiny program. It calls new Date().getTime() as fast as
> possible and try to see if the output is continuous:
>
> class A {
> public static void main(String[] rags) throws Exception {
> int[] x = new int[100];
> long l = new java.util.Date().getTime();
> while (true) {
> long m = new java.util.Date().getTime();
> if (m - l >= 100) break;
> x[(int)(m-l)]++;
> }
> for (int i=0; i<100; i++) {
> System.out.printf("%5d %d\n", i, x[i]);
> }
> }
> }
>
> On Linux and Solaris, there are only 2 to 5 zeros in the output, which
> shows the time is quite continuous on these systems. However, on Windows
> (a x64 system), only 1/16th are non zeros, which shows the maximum
> precision on Windows looks like 16 millisec.
Please, someone tell me Windows isn't still using the old 18.2 Hz
DOS timer for its system clock. Nooooo....
Andrew.
From Alan.Bateman at Sun.COM Mon Sep 14 06:38:23 2009
From: Alan.Bateman at Sun.COM (Alan Bateman)
Date: Mon, 14 Sep 2009 14:38:23 +0100
Subject: Date precision on Windows
In-Reply-To:
References:
Message-ID: <4AAE474F.4070608@sun.com>
Max (Weijun) Wang wrote:
> Hi All
>
> Recently one of my Kerberos tests always reports "replay detected"
> error. Turns out the time precision on Windows is quite low.
>
> I've written this tiny program. It calls new Date().getTime() as fast
> as possible and try to see if the output is continuous:
>
> class A {
> public static void main(String[] rags) throws Exception {
> int[] x = new int[100];
> long l = new java.util.Date().getTime();
> while (true) {
> long m = new java.util.Date().getTime();
> if (m - l >= 100) break;
> x[(int)(m-l)]++;
> }
> for (int i=0; i<100; i++) {
> System.out.printf("%5d %d\n", i, x[i]);
> }
> }
> }
>
> On Linux and Solaris, there are only 2 to 5 zeros in the output, which
> shows the time is quite continuous on these systems. However, on
> Windows (a x64 system), only 1/16th are non zeros, which shows the
> maximum precision on Windows looks like 16 millisec.
>
> Can this be enhanced?
>
> Thanks
> Max
>
David Holmes has a good write-up on this topic here:
http://blogs.sun.com/dholmes/entry/inside_the_hotspot_vm_clocks
(the no-arg Date constructor uses System.currentTimeMillis)
-Alan.
From jeroen at sumatra.nl Mon Sep 14 06:42:23 2009
From: jeroen at sumatra.nl (Jeroen Frijters)
Date: Mon, 14 Sep 2009 15:42:23 +0200
Subject: Date precision on Windows
In-Reply-To: <4AAE44C7.5000208@redhat.com>
References:
<4AAE44C7.5000208@redhat.com>
Message-ID:
Andrew Haley wrote:
> Please, someone tell me Windows isn't still using the old 18.2 Hz DOS
> timer for its system clock. Nooooo....
Not that it is true, but why would that be bad?
By default (most) multi core/cpu Windows systems run at a 15 ms timer interrupt interval (10 ms for single cpu/core systems), but programs can request a smaller interval (down to 1 ms).
Running the timer interrupt at a lower frequency helps if you care about throughput and/or battery life.
Regards,
Jeroen
From Vladimir.Yaroslavskiy at Sun.COM Mon Sep 14 08:07:30 2009
From: Vladimir.Yaroslavskiy at Sun.COM (Vladimir Yaroslavskiy)
Date: Mon, 14 Sep 2009 19:07:30 +0400
Subject: Replacement of Quicksort in java.util.Arrays with Dual-Pivot
Quicksort: improvements
In-Reply-To:
References:
Message-ID: <4AAE5C32.1060502@Sun.COM>
Hi Team,
Thank you very much for your feedback and comments!
I collect here all hints and tips for improvement of
the new algorithm. New version of Quicksort is attached.
And now my investigations:
1. [Jon Bentley]
> Use the median of 2k+1 elements for pivots.
> We can choose a sample of five elements, sort them, and then
> partition around the 2nd and 4th sorted elements.
Yes, this helps, I can't say that we win a lot of time in general,
but it works really faster for input data like this 012012012...,
012301230..., 01234012340.... And more important thing: this
approach allows me to eliminate "div" parameter in the implementation.
As it was suggested, I take middle 5 elements with indexes: s, 2s, 3s,
4s, 5s (where s is length/6), sort them and take 2s and 4s elements
as pivots. Also I don't need to compare pivots because they are sorted.
2. [Jon Bentley]
> How about doing a test for pivot1 == pivot2, and then
> separately handling that special case? equal elements. Right now, the
> inner loop in the "// sorting" section compares every element both to
> pivot1 and pivot2.
Good catch! It reduces time of sorting equal elements (or almost equal)
from 347 to 228 and doesn't change time for other cases.
3. [Jon Bentley]
> Vladimir is currently using the first scheme that we tried.
> I wonder if it might be faster (or clearer) to try an invariant like
> | < | M | ? | M | > |
> where M denotes middle elements with the property
> Pivot1 <= M <= Pivot2
> Vladimir, do you think that you might be able to get a better invariant?
Sure, I tried other invariant like this:
| < | M | > | ? |
but it is slower than current.
4.1 [Oleg Anashkin]
> First thing that came to mind - have you thought about extrapolating this
> approach to more pivots? If 2-pivot algorithm is faster than 1-pivot, then
> 3-pivot might be even faster, right? Can the number of pivots be chosen as
> a function of array size (to mitigate overhead)?
and
4.2 [Leonid Geller]
> As an observation, why not expand the new algorithm to N-Pivot
> where N = round(ln(array length)).
> This should lower the average sort cost even lower.
Here is the experimental results for several pivots.
It's easy to define recurrence relation and write code
to count the comparisons and swap:
Classic Quicksort: comps - 52254956
swaps - 26277985
Dual-Pivot Quicksort: comps - 52246553
swaps - 20977937
Quicksort with 3 pivots: comps - 54399943
swaps - 24224254
Quicksort with 4 pivots: comps - 57241615
swaps - 28659728
You can see that "3 pivots" is better than Classic Quicksort for
swaps only, "4 pivots" is the looser, and "Dual-Pivot Quicksort"
is real winner. If we have many pivots we spent a lot of time on
sorting them and dividing into more parts doesn't save. In case
with 2 pivots, we use only 1 comparison and 1/2 swaps to sort it out.
5. [R?mi Forax]
> The algorithm use two constants: 37 and 13,
> I think they should be declared as private static final.
The constants are: 27 (not 37) and 13, and I made it as
private static final int. (but 27 is changed to 17, see below)
6. [R?mi Forax]
> In the loop tagged "sorting" and "equals element", a[k] can be
> stored in a local variable to avoid to recompute it several time.
Adding a local variable doesn't save time, but it was done in the
line of other improvement (see below). Thank you for pointing to it!
7. [Vladimir Yaroslavskiy]
I very often look at swap function and don't like it. I tried to
eliminate it and add these 3 lines into the code as inline, but
no effect. Then I looked from other side, we have a lot of code
such as:
for (int k=0; ...) {
if (a[k] < ...) {
swap(a, k, less++)
}
...
}
You can see that we retrieve a[k] many times, so I suggest doing it
like this without swap function:
for (int k=0; ...) {
int ak = a[k]
if (ak < ...) {
a[k] = a[less];
a[less++] = ak;
}
...
}
What we have achieve:
1. No swap function call
2. Two assignments instead of three
3. No local variable temp
This approach shows very nice results especially for server mode:
new: 100 / old: 126
8. [Jonathan Graehl]
> suggest this:
> int third = len / div;
> third=third>0?third:1;
Very elegant, but unfortunately I could not apply this,
because the schema of choosing pivot elements has been changed.
9. [Goktug Gokdogan]
> 1. Falling back-to insertion sort at < 17 instead of < 27.
> 2. (a[great] > pivot2) test is more likely to fail compared to
> (k < great) in the while loop, so exchange them (ie. use
> while (a[great] > pivot2 && k < great)).
I tried 17 as boundary for insertion sort: in some cases it is better,
in other no. But in average I don't see that time is increased.
I set up 17 in new version. Note that in my first version the value
was 17, but later it was changed to 25 -> 27.
while (a[great] > pivot2 && k < great)) - we win 3-4% ! Thank you!
------------------------------------------------------------------
I hope that I don't miss something what you suggested.
If you find missed items or have other comments/suggestions,
please, let me know. And the end I would like provide the
execution times for standard input data (all times should
be divided by 1000 to get seconds):
Client VM Server VM
random random
dpq: 16221 dpq: 19893
jdk: 20162 jdk: 24199
ascendant ascendant
dpq: 5686 dpq: 7952
jdk: 9414 jdk: 11748
descendant descendant
dpq: 5921 dpq: 7924
jdk: 9736 jdk: 11777
all equal all equal
dpq: 261 dpq: 500
jdk: 734 jdk: 1140
repeated 50 repeated 50
dpq: 3450 dpq: 4239
jdk: 4207 jdk: 5874
organ pipes organ pipes
dpq: 9364 dpq: 11451
jdk: 12858 jdk: 16534
random 01 random 01
dpq: 1474 dpq: 1423
jdk: 1747 jdk: 2591
010101 010101
dpq: 1092 dpq: 1052
jdk: 1140 jdk: 2076
012012 012012
dpq: 771 dpq: 1035
jdk: 1220 jdk: 1980
012301 012301
dpq: 1140 dpq: 1393
jdk: 1606 jdk: 2876
012340 012340
dpq: 1464 dpq: 1826
jdk: 1947 jdk: 3366
random 4 random 4
dpq: 1855 dpq: 2423
jdk: 2258 jdk: 3148
random 1000 random 1000
dpq: 8008 dpq: 9213
jdk: 8521 jdk: 10363
You can see that now Dual-Pivots Quicksort wins in all nominations
(in previous letter it looses in server mode in few cases).
Alan,
I'll send you updated Arrays.java little bit later.
Thank you,
Vladimir
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: DualPivotQuicksort753.java
Url: http://mail.openjdk.java.net/pipermail/core-libs-dev/attachments/20090914/a1332afe/attachment.ksh
From Artem.Ananiev at Sun.COM Mon Sep 14 03:08:55 2009
From: Artem.Ananiev at Sun.COM (Artem Ananiev)
Date: Mon, 14 Sep 2009 14:08:55 +0400
Subject: Review Request for 6879044
In-Reply-To: <4AAD9D2D.6000502@sun.com>
References: <4AAD9D2D.6000502@sun.com>
Message-ID: <4AAE1637.70005@sun.com>
Mandy Chung wrote:
> 6879044: Eliminate the dependency of logging from the JRE core/awt/swing
> classes
>
> Webrev:
> http://cr.openjdk.java.net/~mchung/6879044/webrev.00/
>
> Summary:
> 1. A new sun.util.logging.PlatformLogger class that will handle the log
> messages in a similar way as Logger but it will only delegate to
> java.util.logging only when it is enabled. LogManager and LogRecord are
> modified to support the platform loggers. The users of PlatformLogger
> will continue to run if java.util.logging classes do not exist.
>
> 2. AWT, 2D, Swing, and a few java.util classes are modified to use
> PlatformLogger instead of Logger. Although many files are modified, the
> change is mostly replacement with classname.
I have quickly looked through AWT/2D changes and haven't found anything
suspicious. A funny typo is noticed in X11 key logging (see XToolkit or
XWindow for examples): logger name is sun.awt.X11.kye instead of
sun.awt.X11.key - but anyway it's not your fault :)
> AWT statically creates a number of loggers. Running a simple AWT Framer
> application with JDK 7 b71 creates 79 loggers on solaris-i586 and 34
> loggers on windows-i586. SwingSet2 creates a total of 85 loggers
> including a few non-awt ones on solaris-i586 and 35 on windows-i586).
> Although the memory usage might not be very high (especially with this
> fix), I don't see the need of having many fine-grained loggers. This
> fix doesn't address this the number of AWT loggers. I file a separate CR
> (6880089) to revisit it.
Thanks.
> Startup Performance:
> This change does not have significant startup performance improvement,
> as expected. However, it does reduce the number of loaded classes
> (Framer app loads 16 fewer classes and jedit loads 13 fewer classes).
>
>
> Thanks
> Mandy
Thanks,
Artem
From Yuri.Nesterenko at Sun.COM Mon Sep 14 03:22:09 2009
From: Yuri.Nesterenko at Sun.COM (Yuri Nesterenko)
Date: Mon, 14 Sep 2009 14:22:09 +0400
Subject: Review Request for 6879044
In-Reply-To: <4AAE1637.70005@sun.com>
References: <4AAD9D2D.6000502@sun.com> <4AAE1637.70005@sun.com>
Message-ID: <4AAE1951.6040907@sun.com>
Artem Ananiev wrote:
>
> Mandy Chung wrote:
>> 6879044: Eliminate the dependency of logging from the JRE
>> core/awt/swing classes
>
...
> I have quickly looked through AWT/2D changes and haven't found anything
> suspicious. A funny typo is noticed in X11 key logging (see XToolkit or
> XWindow for examples): logger name is sun.awt.X11.kye instead of
> sun.awt.X11.key - but anyway it's not your fault :)
It's not a typo, it's a decision!!! I forgot why I did it though but
there was a good reason.
...Oh, there it is: it's extremely easy to find
in the code full of everything "key".
-yan
From opinali at gmail.com Mon Sep 14 08:18:31 2009
From: opinali at gmail.com (Osvaldo Doederlein)
Date: Mon, 14 Sep 2009 12:18:31 -0300
Subject: Date precision on Windows
In-Reply-To:
References:
<4AAE44C7.5000208@redhat.com>
Message-ID: