For any TimeUnit {@code unit}, * {@code unit.convert(Duration.ofNanos(n))} * is equivalent to * {@code unit.convert(n, NANOSECONDS)}, and * {@code unit.convert(Duration.of(n, unit.toChronoUnit()))} * is equivalent to {@code n} (in the absence of overflow). * * @param duration the time duration * @return the converted duration in this unit, * or {@code Long.MIN_VALUE} if conversion would negatively overflow, * or {@code Long.MAX_VALUE} if it would positively overflow. * @throws NullPointerException if {@code duration} is null * @see Duration#of(long,TemporalUnit) * @since 11 */ public long convert(Duration duration) { final long secs = duration.getSeconds(); final int nano = duration.getNano(); final long s, candidate; if ((s = scale) == SECOND_SCALE) return (secs < 0 && nano > 0) ? secs + 1 : secs; if (s > SECOND_SCALE) { long div = secs / secRatio; if (secs < 0 && nano > 0 && div * secRatio == secs) div++; return div; } return (secs >= 0) ? ((secs > maxSecs || (candidate = secs * secRatio + nano / s) < 0) ? Long.MAX_VALUE : candidate) : ((secs < - maxSecs - 1 || (candidate = secs * secRatio + (nano + s - 1) / s) > 0) ? Long.MIN_VALUE : candidate); } But is it really correct? Here are some tests as well: /** * convert(Duration) roundtrips with Duration.ofXXXX and Duration.of(long, ChronoUnit) */ public void testConvertDuration_roundtripDurationOf() { long n = ThreadLocalRandom.current().nextLong(); assertEquals(n, NANOSECONDS.convert(Duration.ofNanos(n))); assertEquals(n, NANOSECONDS.convert(Duration.of(n, ChronoUnit.NANOS))); assertEquals(n, MILLISECONDS.convert(Duration.ofMillis(n))); assertEquals(n, MILLISECONDS.convert(Duration.of(n, ChronoUnit.MILLIS))); assertEquals(n, SECONDS.convert(Duration.ofSeconds(n))); assertEquals(n, SECONDS.convert(Duration.of(n, ChronoUnit.SECONDS))); n /= 60; assertEquals(n, MINUTES.convert(Duration.ofMinutes(n))); assertEquals(n, MINUTES.convert(Duration.of(n, ChronoUnit.MINUTES))); n /= 60; assertEquals(n, HOURS.convert(Duration.ofHours(n))); assertEquals(n, HOURS.convert(Duration.of(n, ChronoUnit.HOURS))); n /= 24; assertEquals(n, DAYS.convert(Duration.ofDays(n))); assertEquals(n, DAYS.convert(Duration.of(n, ChronoUnit.DAYS))); } /** * convert(Duration.ofNanos(n)) agrees with convert(n, NANOSECONDS) */ public void testConvertDuration_roundtripDurationOfNanos() { // Test values near unit transitions and near overflow. LongStream.concat( Arrays.stream(TimeUnit.values()).mapToLong(u -> u.toNanos(1)), LongStream.of(Long.MAX_VALUE, Long.MIN_VALUE)) .flatMap(n -> LongStream.of(n, n + 1, n - 1)) .flatMap(n -> LongStream.of(n, n + 1_000_000_000, n - 1_000_000_000)) .flatMap(n -> LongStream.of(n, -n)) // .peek(System.err::println) .forEach(n -> Arrays.stream(TimeUnit.values()).forEach( u -> assertEquals(u.convert(n, NANOSECONDS), u.convert(Duration.ofNanos(n))))); } /** * convert(Duration) doesn't misbehave near Long.MAX_VALUE and Long.MIN_VALUE. */ public void testConvertDuration_nearOverflow() { ChronoUnit NANOS = ChronoUnit.NANOS; Duration maxDuration = Duration.ofSeconds(Long.MAX_VALUE, 999_999_999); Duration minDuration = Duration.ofSeconds(Long.MIN_VALUE, 0); for (TimeUnit u : TimeUnit.values()) { ChronoUnit cu = u.toChronoUnit(); long r; if (u.toNanos(1) > SECONDS.toNanos(1)) { r = u.toNanos(1) / SECONDS.toNanos(1); assertThrows(ArithmeticException.class, () -> Duration.of(Long.MAX_VALUE, cu), () -> Duration.of(Long.MIN_VALUE, cu)); } else { r = 1; Duration max = Duration.of(Long.MAX_VALUE, cu); Duration min = Duration.of(Long.MIN_VALUE, cu); assertEquals(Long.MAX_VALUE, u.convert(max)); assertEquals(Long.MAX_VALUE - 1, u.convert(max.minus(1, NANOS))); assertEquals(Long.MAX_VALUE - 1, u.convert(max.minus(1, cu))); assertEquals(Long.MIN_VALUE, u.convert(min)); assertEquals(Long.MIN_VALUE + 1, u.convert(min.plus(1, NANOS))); assertEquals(Long.MIN_VALUE + 1, u.convert(min.plus(1, cu))); assertEquals(Long.MAX_VALUE, u.convert(max.plus(1, NANOS))); if (u != SECONDS) { assertEquals(Long.MAX_VALUE, u.convert(max.plus(1, cu))); assertEquals(Long.MIN_VALUE, u.convert(min.minus(1, NANOS))); assertEquals(Long.MIN_VALUE, u.convert(min.minus(1, cu))); } } assertEquals(Long.MAX_VALUE / r, u.convert(maxDuration)); assertEquals(Long.MIN_VALUE / r, u.convert(minDuration)); } } On Thu, May 31, 2018 at 12:32 AM, Stephen Colebourne wrote: > I'm not convinced TimeUnit::toDuration(long amount) has enough value. > We don't have a similar method on ChronoUnit > > Duration.of(amount, timeUnit.toChronoUnit()) seems sufficient. Maybe > document this in the convert(Duration) method? > > Stephen > > > On 31 May 2018 at 01:19, Martin Buchholz wrote: > > v.0.2 has both conversion methods in TimeUnit. The unexpected weirdness > is > > that convert(Duration) saturates while toDuration throws > > ArithmeticException, but both seem author-culture-consistent. Perhaps > > TimeUnit#toDuration doesn't provide enough value in view of the existing > > Duration.of and TimeUnit#toChronoUnit. And most of the time you'd > expect to > > convert from Duration to long, just before calling a TimeUnit based > method. > > > > /** > > * Converts the given time duration to this unit. > > * > > * @param duration the time duration > > * @return the converted duration in this unit, > > * or {@code Long.MIN_VALUE} if conversion would negatively overflow, > > * or {@code Long.MAX_VALUE} if it would positively overflow. > > * @throws NullPointerException if {@code duration} is null > > */ > > public long convert(Duration duration) { > > long s = convert(duration.getSeconds(), SECONDS); > > if (s == Long.MIN_VALUE) return s; > > long n = convert(duration.getNano(), NANOSECONDS); > > assert n >= 0 && n < 1_000_000_000; > > return (s + n < s) ? Long.MAX_VALUE : s + n; > > } > > > > /** > > * Converts the given time duration in this unit to a Duration. > > * > > * @param duration the time duration > > * @return the time duration represented as a Duration > > * @throws ArithmeticException if the duration cannot be represented > > * as a Duration due to numeric overflow > > */ > > public Duration toDuration(long duration) { > > return Duration.of(duration, toChronoUnit()); > > } > > > From brent.christian at oracle.com Mon Jun 4 22:10:20 2018 From: brent.christian at oracle.com (Brent Christian) Date: Mon, 4 Jun 2018 15:10:20 -0700 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: Message-ID: Hi, Roger A few things I noticed in src/java.base/share/classes/java/lang/System.java: * Properties can also be set via setProperties(Properties p), though that method is not mentioned in the doc changes (other than to setProperties() itself). 750 * setting a standard property after initialization 's' -> 'S' in "setting" 788 * setting a standard property after initialization using 's' -> 'S' in "setting" 789 * {@link #getProperties()} or {@link #setProperty(String, String)} The doc change for getProperties() also mentions "clearProperty()", but that one is not mentioned here. The rest looks fine to me. Thanks, -Brent On 6/4/18 6:32 AM, Roger Riggs wrote: > Please review a change to make the values of java.home, user.home, > user.dir, and user.name > effectively read-only for internal use.? The values are cached during > initialization and the > cached values are used. > > Webrev: > ? http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ > > Issue: > ? https://bugs.openjdk.java.net/browse/JDK-8066709 > > CSR: > ? https://bugs.openjdk.java.net/browse/JDK-8204235 > > Thanks, Roger > > From weijun.wang at oracle.com Mon Jun 4 23:24:38 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Tue, 5 Jun 2018 07:24:38 +0800 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <4dcfbb49-a0dc-e69f-d44b-2faa2189c5de@Oracle.com> References: <8D9FFD19-4501-46A8-9E7E-D3A622CF81EC@oracle.com> <4dcfbb49-a0dc-e69f-d44b-2faa2189c5de@Oracle.com> Message-ID: Hi Roger Thanks for the explanation. Another question: Before this change, a SecurityException might be thrown when getProperty() is called, does you new code simulate this behavior? Or in these cases this is unnecessary? Also, looks like all change is inside java.base, do you have a suggestion we use it in other JDK modules? Thanks Max > On Jun 5, 2018, at 3:54 AM, Roger Riggs wrote: > > Hi Max, > > On 6/4/2018 11:41 AM, Wang Weijun wrote: >> Not a native English speaker, so my feeling might be incorrect. >> >> Will someone interpret this as that System.getProperty() will return a cached value? >> > I don't think so, it should be clear that the values are cached at initialization or first use. >> >> I would say ?Although getProperty() always returns the last value set by setProperty() (I assume this is the current behavior), it is not uncommon that consumers of a system property may read it once and cache the value it for later use, which means setting the property may not have the desired effect?. >> > The purpose of the change is to clarify the behavior of the java.base module internal access to the properties. > How applications handle properties is beyond the intended scope. >> >> I also don?t think it?s worth listing the 4 property names in the spec. Quite some other system properties are also cached. Listing them there could further suggests that calling getProperty() on them will return the cached value. >> > It seemed worthwhile to highlight the specific properties affected and the release note should be specific. > I moved them to an implNote as Alan suggested. > > Thanks, Roger > >> >> Thanks >> Max >> >> >>> ? 2018?6?4????9:32?Roger Riggs >>> ??? >>> >>> Please review a change to make the values of java.home, user.home, user.dir, and user.name >>> effectively read-only for internal use. The values are cached during initialization and the >>> cached values are used. >>> >>> Webrev: >>> >>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>> >>> >>> Issue: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>> >>> >>> CSR: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8204235 >>> >>> >>> Thanks, Roger >>> >>> >>> > From martinrb at google.com Tue Jun 5 00:05:24 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Jun 2018 17:05:24 -0700 Subject: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries In-Reply-To: References: Message-ID: Hej Deepak, This looks alright, but you really need to add that trailing newline on the test file (a Martin pet peeve). I wonder if instead we invest a little more work, but backport the entire tck directory. All tests should pass except for those that test features not yet backported. On Mon, Jun 4, 2018 at 4:46 AM, Deepak Kejriwal wrote: > Hi all, > > > > Please review the fix for JDK8u Backport of https://bugs.openjdk.java.net/ > browse/JDK-8186171 > > Webrev: http://cr.openjdk.java.net/~rpatil/8186171/webrev.00/ > > > > Summary(also added to backport bug description): > > > > The back port for test files is not clean back port as all tests files are > extending JSR166TestCase.java which was added to JDK 9 as part of HYPERLINK > "https://bugs.openjdk.java.net/browse/JDK-8146467"JDK-8146467: Integrate > JSR 166 jck tests into JDK repo. This is not present in JDK8 version. > > Therefore, I have extracted test are relevant to the fix done JDK-8186171 > and created a new test file Bug8186171Test.java that contains below two > test methods: > > > > . testBug8186171NonDeterministic : This method is copy of > "testBug8186171" present in "MapTest.java" of original changeset > http://hg.openjdk.java.net/jdk10/master/rev/3f5f9bc0bdc2. As it is based > on randomization and runs 1000 times, the method name is suffixed with > "NonDeterministic". > > . testBug8186171Deterministic : This is a new method that runs > single time as it produces the exact scenario mentioned in JDK-8186171. > Therefore, it is not needed to run multiple times to produce the scenario > mentioned in bug. Hence the method name is suffixed with "Deterministic" > > > > Regards, > > Deepak > From stuart.marks at oracle.com Tue Jun 5 00:09:13 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 4 Jun 2018 17:09:13 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> Message-ID: <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> On 6/3/18 8:55 AM, Alan Bateman wrote: > On 03/06/2018 13:11, David Holmes wrote: >> >> Any suggestions as to how to do that in a practical and effective way? >> >> "As if done by the highly-dangerous, long-deprecated and finally removed >> Thread.stop(Throwable)" >> >> ? ;-) >> >> In all seriousness I hate to do anything that might suggest these are valid >> API's even for tools. Maybe we should deprecate them as well (separate RFE of >> course). > These date back to JPDA in JDK 1.2 (it was JVMDI at the time, pre-dates JVM TI) > and would need research to see if there are debuggers or maybe fault injection > tools are using it. > > For Stuart's current effort then it will need a bit of text, maybe borrowing old > spec where the operation was specified to "stop" a thread with an asynchronous > exception. Warnings about the dangers are probably appropriate but these are of > course interfaces that developers are unlikely to ever use directly. The spec for com.sun.jdk.ThreadReference.stop(Throwable) seems sufficiently descriptive to stand alone if the @see Thread.stop(Throwable) cross reference is simply removed. Are there any other changes that need to be done as part of this changeset? s'marks From david.holmes at oracle.com Tue Jun 5 00:56:57 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 5 Jun 2018 10:56:57 +1000 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: <647D84B5-F364-46F4-A80B-8854A3F889B1@oracle.com> Message-ID: Hi Roger, On 5/06/2018 5:55 AM, Roger Riggs wrote: > Hi Lance, > > On 6/4/2018 12:09 PM, Lance Andersen wrote: >> Hi Roger, >> >> Overall, it looks very good. >> >> A couple of quick thoughts, nothing that needs any action unless you >> feel the desire :-) >> >> Line 671 in System.java, it mentions ?standard? system properties. >> ?(as does the existing Implementation note) but it does not really >> specify what a standard system property is (though it can be assumed) >> in getProperties overview. ?Not a big deal, just thought I would point >> it out in case you had a alternative thought. > Yes, it is a bit vague and the documentation of properties is spread > over many packages and classes. > 'Standard' generally refers to any property defined in the scope of Java > SE or the documented JDK properties. > (For example, those subject to the CSR process). The "standard properties" are those documented in the table in System.getProperties() following "This set of system properties always includes values for the following keys:" and which is followed shortly after by: "In addition to the standard system properties, the system properties may include the following keys:" These are the properties that must be provided by all implementations of the Java platform. David ----- >> >> Do you think we should had a mention in getProperty() about this >> cached properties (or do we think the reference to getProperties() is >> enough) > Hard to say,? there will be a release note, but the properties methods > are well known and it is > likely no one reads the javadoc anymore.? getProperty delegates to > getProperties for most of its behavior. > > Adding a repeat of the API note to System.setProperties and > System.setProperty might be more to the point, > warning against setting standard system properties via the API. > I added the apinote to each of the 6 methods for setting, clearing, or > getting properties. If that seems excessive, > I'm open to removing it from those that provide the least value. > > Thanks, Roger > >> >> Best >> Lance >>> On Jun 4, 2018, at 9:32 AM, Roger Riggs >> > wrote: >>> >>> Please review a change to make the values of java.home, user.home, >>> user.dir, and user.name >>> effectively read-only for internal use.? The values are cached during >>> initialization and the >>> cached values are used. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>> >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>> >>> CSR: >>> https://bugs.openjdk.java.net/browse/JDK-8204235 >>> >>> Thanks, Roger >>> >>> >> >> >> >> >> Lance >> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle?Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > From stuart.marks at oracle.com Tue Jun 5 01:52:55 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Mon, 4 Jun 2018 18:52:55 -0700 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: Message-ID: <5f6cb9eb-f95b-309b-433e-63dde8614948@oracle.com> On 6/4/18 6:32 AM, Roger Riggs wrote: > Please review a change to make the values of java.home, user.home, user.dir, and > user.name > effectively read-only for internal use.? The values are cached during > initialization and the > cached values are used. > > Webrev: > ? http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ > > Issue: > ? https://bugs.openjdk.java.net/browse/JDK-8066709 > > CSR: > ? https://bugs.openjdk.java.net/browse/JDK-8204235 Hi Roger, Overall I think the intent of the change is a good one, as is the reimplementation of internal uses of system properties to use an internal API instead. I think it would be good to have an explicit initialization of the SystemProperty class (or whatever it ends up being called) instead of implicitly initializing via a static initializer. If the class is initialized too early, for some reason, things would go wrong. Should there be an error, a warning, or assertion checking to make sure that none of the cached values are null? They should all be non-null, right? s'marks From martinrb at google.com Tue Jun 5 03:52:26 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Jun 2018 20:52:26 -0700 Subject: Durations in existing JDK APIs In-Reply-To: References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: Looks like if you switch representation for negative Duration, much of the hair goes away. This version also optimizes for NANOSECONDS, which is very likely to be the target in practice: public long convert(Duration duration) { long secs = duration.getSeconds(); int nano = duration.getNano(); if (secs < 0 && nano > 0) { // use representation compatible with integer division secs++; nano -= SECOND_SCALE; } final long s; if ((s = scale) >= SECOND_SCALE) return (s == SECOND_SCALE) ? secs : secs / secRatio; long val = secs * secRatio + ((s == NANO_SCALE) ? nano : nano / s); return ((secs | nano) >= 0) ? ((secs > maxSecs || val < 0) ? Long.MAX_VALUE : val) : ((secs < -maxSecs || val > 0) ? Long.MIN_VALUE : val); } On Mon, Jun 4, 2018 at 2:47 PM, Martin Buchholz wrote: > TimeUnit#toDuration is gone. > > Here's a non-strawman attempt at TimeUnit#convert(Duration). > I was astonished how hard it was to get this really correct instead of > merely mostly correct. > For negative numbers, the long/int representation of Duration is sort-of a > "floored" division instead of java's default "truncating" division. > > public long convert(Duration duration) { > final long secs = duration.getSeconds(); > final int nano = duration.getNano(); > final long s, candidate; > if ((s = scale) == SECOND_SCALE) > return (secs < 0 && nano > 0) ? secs + 1 : secs; > if (s > SECOND_SCALE) { > long div = secs / secRatio; > if (secs < 0 && nano > 0 && div * secRatio == secs) > div++; > return div; > } > return (secs >= 0) > ? ((secs > maxSecs > || (candidate = secs * secRatio + nano / s) < 0) > ? Long.MAX_VALUE : candidate) > : ((secs < - maxSecs - 1 > || (candidate = secs * secRatio + (nano + s - 1) / s) > 0) > ? Long.MIN_VALUE : candidate); > } > > From ivan.gerasimov at oracle.com Tue Jun 5 04:26:48 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 4 Jun 2018 21:26:48 -0700 Subject: RFR 8203768 : Avoid reallocation in java.base/unix/classes/java/lang/ProcessImpl.java Message-ID: <09e6208b-52ab-461e-fae6-5cc1c6f671b7@oracle.com> Hello! When closing a Process, its stdout and stderr are drained: The remaining bytes are copied into an array, and then the out/err stream is replaced with ByteArrayInputStream(). In a case when a fewer number of bytes were read then the Stream.available() suggested, the array is reallocated with Arrays.copyOf(). It is possible to avoid reallocation, and use three-args ByteArrayInputStream() constructor to specify a portion of the array used. Would you please help review this trivial fix? BUGURL: https://bugs.openjdk.java.net/browse/JDK-8203768 WEBREV: http://cr.openjdk.java.net/~igerasim/8203768/00/webrev/ All existing tests pass on all supported platforms. Thanks in advance! -- With kind regards, Ivan Gerasimov From martinrb at google.com Tue Jun 5 04:55:21 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Jun 2018 21:55:21 -0700 Subject: RFR 8203768 : Avoid reallocation in java.base/unix/classes/java/lang/ProcessImpl.java In-Reply-To: <09e6208b-52ab-461e-fae6-5cc1c6f671b7@oracle.com> References: <09e6208b-52ab-461e-fae6-5cc1c6f671b7@oracle.com> Message-ID: Hej Ivan, IIRC I wrote the first iteration of this code. I would expect that almost always either the buffer will be empty or the call to available() is accurate and the read will actually read everything in one chunk, so the byte array will be perfectly allocated with no need for re-allocation in practice. Do you have evidence it would be otherwise in practice? But suppose available() incorrectly claims that a million bytes are available, but you only read 10. Then there's a tradeoff - the cost of reallocation versus the risk that the mostly empty backing array will be retained for a long time if the Process object is not garbage collected. I'm leaning towards the status quo. On Mon, Jun 4, 2018 at 9:26 PM, Ivan Gerasimov wrote: > Hello! > > When closing a Process, its stdout and stderr are drained: The remaining > bytes are copied into an array, and then the out/err stream is replaced > with ByteArrayInputStream(). > > In a case when a fewer number of bytes were read then the > Stream.available() suggested, the array is reallocated with Arrays.copyOf(). > > It is possible to avoid reallocation, and use three-args > ByteArrayInputStream() constructor to specify a portion of the array used. > > Would you please help review this trivial fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8203768 > WEBREV: http://cr.openjdk.java.net/~igerasim/8203768/00/webrev/ > > All existing tests pass on all supported platforms. > > Thanks in advance! > > -- > With kind regards, > Ivan Gerasimov > > From nishit.jain at oracle.com Tue Jun 5 05:13:08 2018 From: nishit.jain at oracle.com (Nishit Jain) Date: Tue, 5 Jun 2018 10:43:08 +0530 Subject: RFR JDK-8203872: Upgrading JDK with latest available LSR data from IANA Message-ID: <6aa57b43-c15e-cfee-cb32-922e4eec8489@oracle.com> Hi, Please review the fix for JDK-8203872. Bug: https://bugs.openjdk.java.net/browse/JDK-8203872 Webrev: http://cr.openjdk.java.net/~nishjain/8203872/webrev.01/ Fix: Updated the LSR data to the version 2017-08-15 Regards, Nishit Jain From ivan.gerasimov at oracle.com Tue Jun 5 05:31:00 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 4 Jun 2018 22:31:00 -0700 Subject: RFR 8203768 : Avoid reallocation in java.base/unix/classes/java/lang/ProcessImpl.java In-Reply-To: References: <09e6208b-52ab-461e-fae6-5cc1c6f671b7@oracle.com> Message-ID: Hej Martin! On 6/4/18 9:55 PM, Martin Buchholz wrote: > Hej Ivan, > > IIRC I wrote the first iteration of this code. > > I would expect that almost always either the buffer will be empty or > the call to available() is accurate and the read will actually read > everything in one chunk, so the byte array will be perfectly allocated > with no need for re-allocation in practice. Do you have evidence it > would be otherwise in practice? > No evidence. It is just an observation from reading code. > But suppose available() incorrectly claims that a million bytes are > available, but you only read 10. Then there's a tradeoff - the cost > of reallocation versus the risk that the mostly empty backing array > will be retained for a long time if the Process object is not garbage > collected. > On the other hand, if available() claims a million bytes, and then only 999999990 bytes were read, there will be mostly unnecessary allocation and copying. > I'm leaning towards the status quo. > Okay, let's leave it as is :) It was meant mostly for cleaning up the code, so if there is a doubt, then it's better keep what we have. With kind regards, Ivan > > On Mon, Jun 4, 2018 at 9:26 PM, Ivan Gerasimov > > wrote: > > Hello! > > When closing a Process, its stdout and stderr are drained: The > remaining bytes are copied into an array, and then the out/err > stream is replaced with ByteArrayInputStream(). > > In a case when a fewer number of bytes were read then the > Stream.available() suggested, the array is reallocated with > Arrays.copyOf(). > > It is possible to avoid reallocation, and use three-args > ByteArrayInputStream() constructor to specify a portion of the > array used. > > Would you please help review this trivial fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8203768 > > WEBREV: http://cr.openjdk.java.net/~igerasim/8203768/00/webrev/ > > > All existing tests pass on all supported platforms. > > Thanks in advance! > > -- > With kind regards, > Ivan Gerasimov > > -- With kind regards, Ivan Gerasimov From xueming.shen at oracle.com Tue Jun 5 05:38:45 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 04 Jun 2018 22:38:45 -0700 Subject: RFR JDK-8200530: '\r' is not supported as "newline" in java.util.jar.Manifest. Message-ID: <5B1621E5.80301@oracle.com> Hi, Please help review the changeset for JDK-8200530. "newline" is specified as |CR LF | LF | CR|(/not followed by/|LF|) in Jar spec [1] from the very beginning but our implementation actually never supports "\r"/CR (not followed by LF) case. The proposed change here is to add CR as an individual supported "newline"/line separator. issue: https://bugs.openjdk.java.net/browse/JDK-8200530 webrev: http://cr.openjdk.java.net/~sherman/8200530/webrev Thanks, Sherman [1] https://docs.oracle.com/javase/10/docs/specs/jar/jar.html From Alan.Bateman at oracle.com Tue Jun 5 06:11:05 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 5 Jun 2018 07:11:05 +0100 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: On 05/06/2018 01:09, Stuart Marks wrote: > : > > The spec for com.sun.jdk.ThreadReference.stop(Throwable) seems > sufficiently descriptive to stand alone if the @see > Thread.stop(Throwable) cross reference is simply removed. > > Are there any other changes that need to be done as part of this > changeset? The source file that is used to generate the JDWP protocol code and the spec is in make/data/jdwp/jdwp.spec. The JVM TI spec is at src/hotspot/share/prims/jvmti.xml. It should be okay to skip those for this change-set, assuming it is followed up quickly with another change-set to update those specs. -Alan From Alan.Bateman at oracle.com Tue Jun 5 06:19:22 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 5 Jun 2018 07:19:22 +0100 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <26cb79b2-6ccc-f7a1-4cc0-3c13eaf60f43@Oracle.com> References: <7e57ecce-a455-8f5d-f6f5-0bb965a86dc6@oracle.com> <26cb79b2-6ccc-f7a1-4cc0-3c13eaf60f43@Oracle.com> Message-ID: <8a1ad331-411b-3e0b-58d1-36f1caad3dac@oracle.com> On 04/06/2018 20:59, Roger Riggs wrote: > : > >> >> Are the changes to SocksSocketImpl correct? I may have missed >> something but the original called System.getProperty("user.dir") in a >> privileged block so I'm wondering if getUserNameChecked is needed. > The existing code in SocksSocketImpl is inconsistent with respect to > access to user.name; some flows > use doPriv to access the property and others did not.? If someone > familiar with the Socks networking function > can recommend the proper access, it can be revised.? The intent was to > have the same security checks > as before. The original code at L181 is using GetPropertyAction.privilegedGetProperty so it looks like it reads the value of the property in a privileged block. The replacement code is doing an explicit permission check. If I read the original code correctly then it should only be doing a permission check for the proxy case. So I think it needs to be checked, another set of eyes would be useful. -Alan From ivan.gerasimov at oracle.com Tue Jun 5 06:23:00 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 4 Jun 2018 23:23:00 -0700 Subject: RFR 8204310 : Simpler RandomAccessFile.setLength() on Windows Message-ID: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> Hello! When a file size is modified via RandomAccessFile.setLength(int) on Windows, it is currently done in three steps: SetFilePointer(), SetEndOfFile(), SetFilePointerEx(). First two can be combined in one call of SetFileInformationByHandle(), which will make the code shorter and more aligned with Unix variant (ftruncate64 is used on Unix systems, which behaves close to SetFileInformationByHandle(_,FileEndOfFileInfo,_)). Would you please help review this trivial fix? BUGURL: https://bugs.openjdk.java.net/browse/JDK-8204310 WEBREV: http://cr.openjdk.java.net/~igerasim/8204310/00/webrev/ All the existing tests pass Ok. -- With kind regards, Ivan Gerasimov From li.jiang at oracle.com Tue Jun 5 06:30:16 2018 From: li.jiang at oracle.com (li.jiang at oracle.com) Date: Tue, 5 Jun 2018 14:30:16 +0800 Subject: [11] RFR: 8202026 8193552 8204269 : ISO 4217 Amendment #165 # 166 #167 Update In-Reply-To: <875fa321-9b73-17f4-c63e-b542e32b6424@oracle.com> References: <5d50ddec-884c-e78c-4c2c-b9e11d8c3236@oracle.com> <62217aa4-16c5-2ec2-15f7-177d0906c53b@oracle.com> <2099b4c0-b97a-4c1a-f291-fec8f7beb294@oracle.com> <9181f81a-00f4-475c-553f-fa155439fa15@oracle.com> <875fa321-9b73-17f4-c63e-b542e32b6424@oracle.com> Message-ID: <156571a8-d7ab-2c51-39ea-3242e33399f0@oracle.com> Hi Naoto, I removed the obsoleted currency LTL and LVL from tablea1.txt, added them into otherCodes in ValidateISO4217.java. new webrev as below: http://cr.openjdk.java.net/~ljiang/8202026/webrev.03/ Thanks, Leo On 05/06/2018 2:27 AM, naoto.sato at oracle.com wrote: > Hi Leo, > > Change looks good. One leftover from the previous: > > >> in CurrencyData.properties. This applies to tabela1.txt as well. > > Can you please clean those LV/LT entries in tablea1.txt as well? > > Naoto > > On 6/4/18 7:41 AM, li.jiang at oracle.com wrote: >> Hi Naoto, >> >> Pls review the updated code: >> >> http://cr.openjdk.java.net/~ljiang/8202026/webrev.02/ >> >> - As suggested, clean up the old transition dates. >> - Update copyright year. >> - ISO 4217 Amendment #167 was published today, which discards the >> #166, so withdraw the change for #166 in webrev.02. >> >> Bug for #167: >> https://bugs.openjdk.java.net/browse/JDK-8204269 >> >> Test passed on mach5. >> >> Thanks, >> Leo >> >> On 26/04/2018 1:00 AM, naoto.sato at oracle.com wrote: >>> Hi Leo, >>> >>> Although JDK11 is slated in 09/2018, enabling amendment 166 now is >>> technically a bug, as it will be effective from June 4. Please use >>> the transition mechanism in >>> make/data/currency/CurrencyData.properties and >>> test/jdk/java/util/Currency/tablea1.txt. >>> >>> OTOH, there are old (past) transition entries. I would clean up those >>> entries, such as: >>> >>> ??326 # LATVIA >>> ??327 LV=LVL;2013-12-31-22-00-00;EUR >>> >>> in CurrencyData.properties. This applies to tabela1.txt as well. >>> >>> Naoto >>> >>> On 4/24/18 8:52 PM, Leo Jiang wrote: >>>> Forgot to mention, the tests in Currency fold are passed on Mach5. >>>> >>>> -Leo >>>> >>>> On 04/25/2018 09:33 AM, Leo Jiang wrote: >>>>> Hi, >>>>> >>>>> Please review the changes to address the ISO 4217 Amendment 165 166 >>>>> update. >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8193552? 165 >>>>> https://bugs.openjdk.java.net/browse/JDK-8202026? 166 >>>>> >>>>> CR: >>>>> http://cr.openjdk.java.net/~ljiang/8202026/webrev.00/ >>>>> >>>>> >>>>> Detail: >>>>> #165 >>>>> From: >>>>> MAURITANIA??? Ouguiya??? MRO??? 478??? 2 >>>>> To: >>>>> MAURITANIA??? Ouguiya??? MRU??? 929??? 2 >>>>> >>>>> #166 >>>>> From: >>>>> VENEZUELA (BOLIVARIAN REPUBLIC OF)??? Bol?var??? VEF??? 937??? 2 >>>>> To: >>>>> VENEZUELA (BOLIVARIAN REPUBLIC OF)??? Bol?var Soberano??? VES 928??? 2 >>>>> >>>>> >>>>> Thanks, >>>>> Leo From takiguc at linux.vnet.ibm.com Tue Jun 5 06:59:01 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Tue, 05 Jun 2018 15:59:01 +0900 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: <81453d0cbe05477ea558917de263ada2@sap.com> References: <0c73caec8810b9844a463343bdaec064@linux.vnet.ibm.com> <81453d0cbe05477ea558917de263ada2@sap.com> Message-ID: Hello Matthias and Christoph. Thank you for your explanations. I did not have enough knowledge about "visibility". I created following patches. ================================ diff -r 02934b0d661b src/java.base/share/native/libjimage/NativeImageBuffer.cpp --- a/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Wed May 30 14:46:28 2018 +0200 +++ b/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Tue Jun 05 12:10:41 2018 +0900 @@ -39,7 +39,9 @@ #include "imageFile.hpp" #include "inttypes.hpp" #include "jimage.hpp" +#if !defined(_AIX) #include "osSupport.hpp" +#endif #include "jdk_internal_jimage_NativeImageBuffer.h" diff -r 02934b0d661b src/java.base/unix/native/include/jni_md.h --- a/src/java.base/unix/native/include/jni_md.h Wed May 30 14:46:28 2018 +0200 +++ b/src/java.base/unix/native/include/jni_md.h Tue Jun 05 12:10:41 2018 +0900 @@ -29,7 +29,8 @@ #ifndef __has_attribute #define __has_attribute(x) 0 #endif -#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) +#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) \ + || (defined(_AIX) && (defined(__xlC__) && (__xlC__ >= 0xD01))) #ifdef ARM #define JNIEXPORT __attribute__((externally_visible,visibility("default"))) #define JNIIMPORT __attribute__((externally_visible,visibility("default"))) ================================ If "osSupport.hpp" was included, XLC++ compiler complained. I could not understand, which part was invalid... I'm not sure, "osSupport.hpp" is really required on NativeImageBuffer.cpp or not... I added __xlC__ macro to determine XLC/C++'s version# into jni_md.h. [1] 0xD01 means 13.1 by hexadecimal. I checked symbol table by "dump -Tv -X64". [2] It seemed it was fine, some symbols were hidden. Does someone review my code? [1] https://www.ibm.com/support/knowledgecenter/en/SSGH2K_13.1.3/com.ibm.xlc1313.aix.doc/compiler_ref/xlmacros.html [2] https://www.ibm.com/developerworks/aix/library/au-aix-symbol-visibility/index.html On 2018-06-01 21:12, Baesken, Matthias wrote: > Hi , my change 8202322 just handled the fact that the > visibility - flags are not supported with xlc 12.1 , so setting > them generated a TON of compile - time warnings . > > The introduction of the "-qvisibility=hidden" came with the > mapfile removal changes : > > 8200358: Remove mapfiles for JDK executables > http://hg.openjdk.java.net/jdk/jdk/rev/210cf224b690 > > 8200178: Remove mapfiles for JDK native libraries > http://hg.openjdk.java.net/jdk/jdk/rev/396ea30afbd5 > > I guess it might need further testing+adjustments to make the > "visibility hiding" work nicely with XLC13 , but currently we > build only with XLC12 . > > As a workaround you might want to remove the "-qvisibility=hidden" > setting for XLC 13 as well , like I did for XLC12 with the change > 8202322 . > > > Best regards, Matthias > > > > >> -----Original Message----- >> From: Langer, Christoph >> Sent: Freitag, 1. Juni 2018 10:57 >> To: Ichiroh Takiguchi >> Cc: Baesken, Matthias ; 'build- >> dev at openjdk.java.net' ; ppc-aix-port- >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, >> Goetz >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support >> on xlc 12.1 >> >> Hi Ichiroh, >> >> we do not use the XLC 13 compiler on AIX yet here at SAP and I believe >> nobody of my colleagues has played with it yet. So you are on a new >> playground here ?? >> >> However, I believe the idea in OpenJDK with the abolition of map files >> is that >> symbols should be invisible externally unless they are declared >> exported, >> e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the >> correct >> default and whatever JNIEXPORT expands to should contain the right >> attributes to get that symbol visible. >> >> Can you check if either my assumption is completely wrong, JNIEXPORT >> does >> not expand to the right thing, XLC 13 has a bug or maybe just sume >> specific >> required symbols are not declared correctly? >> >> Best regards >> Christoph >> >> > -----Original Message----- >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] >> > Sent: Donnerstag, 31. Mai 2018 09:55 >> > To: Langer, Christoph >> > Cc: Baesken, Matthias ; 'build- >> > dev at openjdk.java.net' ; ppc-aix-port- >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, >> > Goetz >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 >> > >> > Hello. >> > 8202322 was integrated into jdk-11+15. >> > I'm using XLC 13.1.3 on AIX 7.1.4. >> > Build was failed because of "-qvisibility=hidden" on >> > make/lib/LibCommon.gmk. >> > According to "XL C/C++ for AIX 13.1.3" documentation [1], >> > "-qvisibility=hidden" cannot create shared libraries entry points. >> > For example, libverify.so was there, but entry points were not resolved >> > by "-lverify" option. >> > I think it should be "-qvisibility=default" (I tried, it worked) >> > or "-qvisibility=protected" (I had not tried) ? >> > I'm not familiar with -qvisibility option, but I'd like to find out >> > right way. >> > >> > [1] >> > >> https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm. >> > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html >> > >> > On 2018-05-16 16:08, Langer, Christoph wrote: >> > > Hi Matthias, >> > > >> > > yes, reviewed. >> > > >> > > Best regards >> > > Christoph >> > > >> > > From: Baesken, Matthias >> > > Sent: Mittwoch, 16. Mai 2018 09:06 >> > > To: Langer, Christoph ; >> > > 'build-dev at openjdk.java.net' ; >> > > ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net >> > > Cc: Lindenmaier, Goetz >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on >> > > xlc 12.1 >> > > >> > > Hi Christoph can I add you as second reviewer (other reviewer was >> > > Erik Joelsson) can push the change ? >> > > >> > > Best regards, Matthias >> > > >> > > >> > > >> > > From: Langer, Christoph >> > > Sent: Donnerstag, 26. April 2018 16:38 >> > > To: Baesken, Matthias >> > > >; >> > > 'build-dev at openjdk.java.net' >> > > >; >> > > ppc-aix-port-dev at openjdk.java.net> > dev at openjdk.java.net>; >> > > core-libs-dev at openjdk.java.net> dev at openjdk.java.net> >> > > Cc: Simonis, Volker >> > > > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on >> > > xlc 12.1 >> > > >> > > Hi Matthias, >> > > >> > > to me the change in principal looks good. >> > > >> > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. >> > > extract major number before the first dot, then compare numerically) - >> > > but maybe it is too complicated and the current single version compare >> > > suits our needs ? >> > > >> > > Best regards >> > > Christoph >> > > >> > > From: Baesken, Matthias >> > > Sent: Donnerstag, 26. April 2018 16:14 >> > > To: 'build-dev at openjdk.java.net' >> > > >; >> > > ppc-aix-port-dev at openjdk.java.net> > dev at openjdk.java.net>; >> > > core-libs-dev at openjdk.java.net> dev at openjdk.java.net> >> > > Cc: Langer, Christoph >> > > >; >> Simonis, >> > > Volker > >> > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc >> > > 12.1 >> > > >> > > Hello , could you please review this small adjustment to the symbol >> > > visibility compilation settings on AIX ? >> > > Currently we use XLC 12.1 to compile JDK on AIX . >> > > >> > > However XLC 12.1 does not support the "-qvisibility=hidden" >> > > setting currently set on AIX. >> > > It was introduced with XLC 13.1 . Christoph found some info about it >> > > here : >> > > >> > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- >> > visibility-part2/index.html >> > > >> > > Setting it only generates hundreds of warnings in the build log , >> > > warnings look like this : >> > > XlC12.1 >> > > >> > > bash-4.4\$ xlC -qversion >> > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) >> > > Version: 12.01.0000.0019 >> > > >> > > bash-4.4\$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list >> > > of valid options. >> > > >> > > Compare to XLC13.1 >> > > >> > > bash-3.00\$ xlC -qversion >> > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) >> > > Version: 13.01.0000.0008 >> > > bash-3.00\$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc >> > > bash-3.00\$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> > > >> > > >> > > So it is better to avoid setting these flags when using xlc12.1 . >> > > Please review : >> > > >> > > Bug : >> > > >> > > https://bugs.openjdk.java.net/browse/JDK-8202322 >> > > >> > > Change : >> > > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ >> > > >> > > >> > > Best regards, Matthias From jan.lahoda at oracle.com Tue Jun 5 07:29:01 2018 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Tue, 5 Jun 2018 09:29:01 +0200 Subject: RFR: JDK-8203891: Upgrade JOpt Simple to 5.0.4 In-Reply-To: <799bd973-3d11-b4cc-2f16-f4dc6383b4b2@oracle.com> References: <5B0FBC37.2090103@oracle.com> <799bd973-3d11-b4cc-2f16-f4dc6383b4b2@oracle.com> Message-ID: <5B163BBD.9060001@oracle.com> On 4.6.2018 19:40, mandy chung wrote: > Hi Jan, > > On 5/31/18 2:11 AM, Jan Lahoda wrote: >> Hi, >> >> I'd like to upgrade the JOpt Simple library we are using to version >> 5.0.4. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203891 >> Complete webrev: >> http://cr.openjdk.java.net/~jlahoda/8203891/webrev.00/complete/ >> >> Delta webrev only showing (all) JDK changes in JOpt Simple and related >> changes in tests needed for the upgrade, etc.: >> http://cr.openjdk.java.net/~jlahoda/8203891/webrev.00/joptsimple.delta/ > >> Probably the biggest issue with this upgrade is that for two >> subsequent parameters: >> "--libs=", "/tmp" >> "/tmp" used to be interpreted as the parameter of "libs", but now the >> "libs" parameter is empty (as there's nothing behind the '='). > > This is a reasonable fix and no whitespace is expected following `=`. > >> See the changes to test/jdk/tools/jmod/JmodTest.java for an example. >> Hopefully, this is a reasonable change. > > 114 "--libs=" + libDir.toString(), > > Alternatively, you can take out `=` and run it as jmod --libs /tmp > "--libs", libDir.toString(), Yes, that would work as well, but there are already invocations like this in the test. So I opted for using "--libs=" + ... and similar, so that both variants are covered by the test. Thanks, Jan > > Mandy From Alan.Bateman at oracle.com Tue Jun 5 09:06:29 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 5 Jun 2018 10:06:29 +0100 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <26cb79b2-6ccc-f7a1-4cc0-3c13eaf60f43@Oracle.com> References: <7e57ecce-a455-8f5d-f6f5-0bb965a86dc6@oracle.com> <26cb79b2-6ccc-f7a1-4cc0-3c13eaf60f43@Oracle.com> Message-ID: On 04/06/2018 20:59, Roger Riggs wrote: > Hi Alan, > > Updated webrev in place: > http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ The splitting of the note into @apiNote and @implNote mostly looks good. I think it should be "cached by the Java virtual machine" rather than "cached internally". It would be good to fix inconsistent line length while you are there. I still think we need to find a better name for SystemProperty and it would be good to re-visit how this is initialized in initPhase1. I think this is Stuart's point too. It would be nice to call it with "props" so that it captures the initial value of the interesting properties. A second best would be to a SystemProperties.capture() or something explicit. There's something not quite right with the change to SocksSocketImpl that I mentioned in the other mail. -Alan From james.laskey at oracle.com Tue Jun 5 12:17:18 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Tue, 5 Jun 2018 09:17:18 -0300 Subject: RFR JDK-8200530: '\r' is not supported as "newline" in java.util.jar.Manifest. In-Reply-To: <5B1621E5.80301@oracle.com> References: <5B1621E5.80301@oracle.com> Message-ID: <514183E1-18ED-499C-B42F-14853BED8966@oracle.com> Attributes.java:380 nit - assign c at decl - only test len if decremented >>>>>> byte c; if ((c = lbuf[--len]) != '\n' && c != '\r') { throw new IOException("line too long"); } if (len > 0 && lbuf[len-1] == '\r') { --len; } if (len == 0) { break; } ====== byte c = lbuf[--len]; if (c != '\n' && c != '\r') { throw new IOException("line too long"); } if (len > 0 && lbuf[len-1] == '\r' && --len == 0) { break; } <<<<<< Manifest.java:208 nit - same as above >>>>>> byte c; if ((c = lbuf[--len]) != '\n' && c != '\r') { throw new IOException("manifest line too long"); } if (len > 0 && lbuf[len-1] == '\r') { --len; } if (len == 0 && skipEmptyLines) { continue; } ====== byte c = lbuf[--len]; if (c != '\n' && c != '\r') { throw new IOException("manifest line too long"); } if (len > 0 && lbuf[len-1] == '\r' && --len == 0 && skipEmptyLines) { continue; } <<<<<< Manifest.java:396 nit - Shouldn?t c already be loaded with tbuf[tpos-1] (or tbuf[tpos-2]) if ?\r\n?) >>>>>> if ((c = tbuf[tpos-1]) == '\n' || c == '\r') { break; } ====== if (c == '\n' || c == '\r') { break; } <<<<<< +1 Cheers, ? Jim > On Jun 5, 2018, at 2:38 AM, Xueming Shen wrote: > > Hi, > > Please help review the changeset for JDK-8200530. > > "newline" is specified as |CR LF | LF | CR|(/not followed by/|LF|) in Jar spec [1] from > the very beginning but our implementation actually never supports "\r"/CR (not > followed by LF) case. The proposed change here is to add CR as an individual > supported "newline"/line separator. > > issue: https://bugs.openjdk.java.net/browse/JDK-8200530 > webrev: http://cr.openjdk.java.net/~sherman/8200530/webrev > > Thanks, > Sherman > > > [1] https://docs.oracle.com/javase/10/docs/specs/jar/jar.html From adam.farley at uk.ibm.com Tue Jun 5 13:46:08 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Tue, 5 Jun 2018 14:46:08 +0100 Subject: RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers Message-ID: Hi All, Native memory allocation for DBBs is tracked in java.nio.Bits, but that only includes what the user thinks they are allocating. When the VM adds extra memory to the allocation amount this extra bit is not represented in the Bits total. A cursory glance shows, minimum, that we round the requested memory quantity up to the heap word size in the Unsafe.allocateMemory code, and something to do with nmt_header_size in os:malloc() (os.cpp) too. On its own, and in small quantities, align_up(sz, HeapWordSize) isn't that big of an issue. But when you allocate a lot of DBBs, and coupled with the nmt_header_size business, it makes the Bits values wrong. The more DBB allocations, the more inaccurate those numbers will be. To get the "+X", it seems to me that the best option would be to introduce an native method in Bits that fetches "X" directly from Hotspot, using the same code that Hotspot uses (so we'd have to abstract-out the Hotspot logic that adds X to the memory quantity). This way, anyone modifying the Hotspot logic won't risk rendering the Bits logic wrong again. That's only one way to fix the accuracy problem here though. Suggestions welcome. Best Regards Adam Farley Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From tprintezis at twitter.com Tue Jun 5 14:09:03 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Tue, 5 Jun 2018 07:09:03 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> Message-ID: Hey Alan, Any thoughts on this? (with apologies for the ping) Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 30, 2018 at 5:16:44 PM, Peter Levart (peter.levart at gmail.com) wrote: I thought there would be some hint from Alan about which of the two paths we should take for more refinement. The Tony's: http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ Or the Alan's with my mods: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ Regards, Peter On 05/30/18 22:46, Tony Printezis wrote: Hi all, Any more thoughts on this? (with apologies for the ping) Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 18, 2018 at 3:58:57 PM, Tony Printezis (tprintezis at twitter.com) wrote: Hi again, Stylistically, I strongly prefer this version over the previous one (the one with the doubly-linked list of JdkThreadLocal entries) you had posted. This one is a lot cleaner. A few suggestions: * I?m not crazy about the fact that the users have to override calculateInitialValue() instead of initialValue() as it will be a bit error-prone. If they accidentally override initialValue() then the per-thread registering is not going to happen. Maybe I?m overthinking this. One way to get round this is for each JdkThreadLocal instance to delegate calls to get(), set(), and remove() to a private ThreadLocal instance, which can in turn delegate initialValue() to the outer instance. (Hope this makes sense?) I did a quick prototype of this and it seems to work, but I didn?t heavily test it. * I would prefer if the method users override actually took the thread-local value as a parameter, i.e., protected void threadTerminated(T value). This is very easy to add: protected void threadTerminated(T value) { } private void threadTerminated() { threadTerminated(get()); } and I think it will be easier to use, as most uses will probably need to do get() anyway. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 18, 2018 at 4:23:19 AM, Peter Levart (peter.levart at gmail.com) wrote: It's good to have alternative implementations on the table. Here's another variant that is space neutral for threads that don't use JdkThreadLocal, but also scales to many JdkThreadLocal instances: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ Now we only need an arbiter to decide which one! :-) Regards, Peter On 05/17/18 22:39, Peter Levart wrote: Hi Tony, If we anticipate only small number of JdkThreadLocal(s) (there will be only one for the start) then this is a plausible solution. Then perhaps this limitation should be written somewhere in the JdkThreadLocal javadoc so that one day somebody will not be tempted to use JdkThreadLocal(s) en masse. Let's say there will be a few more JdkThreadLocal(s) in the future. Are we willing to pay for a few lookups into a ThreadLocalMap at each and every thread's exit even though such thread did not register a mapping for any JdkThreadLocal? Is an additional reference field in each and every ThreadLocalMap (one per Thread that uses thread locals) a bigger price to pay? I don't know. Will let others comment on this. Otherwise the code looks good. Just a couple of observations: Since private static method JdkThreadLocal.add is only called from JdkThreadLocal constructor with just constructed instance (this), there's no possibility for it to be called twice or more times with the same instance. The check for duplicates could go away then, right? You keep an array of Entry objects which are just wrappers for JdkThreadLocal objects. Are you already planning for Entry to become a WeakReference? Otherwise you could just keep JdkThreadLocal objects in the array directly. Regards, Peter On 05/17/18 20:25, Tony Printezis wrote: Hi all, I have a new version of the code for your consideration: http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ I basically combined our two approaches. The usage is as Alan had proposed it: Users have to use JdkThreadLocal (which is only available within java.base) and override threadTerminated(). However, I keep track of JdkThreadLocal instances globally (as I did before) and not per-thread. This way we don?t need to add any unnecessary complexity to ThreadLocalMap. Currently, I don?t allow entries to be purged if the JdkThreadLocal instance becomes otherwise unreachable. I can easily add that functionality if needed (I can use WeakReferences for that). However, for the uses we?re considering, is it really necessary? Thoughts? Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 14, 2018 at 12:40:28 PM, Tony Printezis (tprintezis at twitter.com) wrote: Peter, In my proposal, you can register the exit hook in the ThreadLocal c?tor, so it?s almost as nice as Alan?s in that respect (and it doesn't require an extra field per ThreadLocal plus two extra fields per JdkEntry). :-) But, I do like the addition of the JdkEntry list to avoid unnecessarily iterating over all the map entries (which was my main concern with Alan?s original webrev). I?ll be totally happy with a version of this. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On May 12, 2018 at 6:44:08 AM, Peter Levart (peter.levart at gmail.com) wrote: Hi, On 05/11/18 16:13, Alan Bateman wrote: On 08/05/2018 16:07, Tony Printezis wrote: Hi all, Following the discussion on this a few weeks ago, here?s the first version of the change: http://cr.openjdk.java.net/~tonyp/8202788/webrev.0/ I think the consensus was that it?d be easier if the exit hooks were only available within java.base. Is it enough that I added the functionality to the jdk.internal.misc package? (And is jdk.internal.misc the best place for this?) I?ll also add a test for the new functionality. But let?s first come up with an approach that everyone is happy with. :-) Peter's approach in early April was clean (and we should come to the getIfPresent discussion) but it adds a field to Thread for the callback list. If I read your approach correctly, you are avoiding that by maintaining an array of hooks in ThreadLocalExitHooks. Another approach to try is a java.base-internal ThreadLocal that defines a method to be invoked when a thread terminates. Something like the following: http://cr.openjdk.java.net/~alanb/8202788/webrev/index.html -Alan >From the API perspective, Alan's suggestion is the most attractive one as it puts the method to where it belongs - into the ThreadLocal instance. But the implementation could be improved a bit. I don't like the necessity to iterate over all ThreadLocal(s) to filter out JdkThreadLocal(s). There might be a situation where there's plenty of ThreadLocal(s) registered per exiting thread which would produce a spike in CPU processing at thread exit. The way to avoid this would be to maintain a separate linked list of entries that contains just those with JdkThreadLocal(s). Like in this modification of Alan's patch, for example: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.01/ (Only ThreadLocal class is modified from Alan's patch) What do you think? Regards, Peter From Roger.Riggs at Oracle.com Tue Jun 5 14:18:23 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jun 2018 10:18:23 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <5f6cb9eb-f95b-309b-433e-63dde8614948@oracle.com> References: <5f6cb9eb-f95b-309b-433e-63dde8614948@oracle.com> Message-ID: Hi Stuart, On 6/4/2018 9:52 PM, Stuart Marks wrote: > > > On 6/4/18 6:32 AM, Roger Riggs wrote: >> Please review a change to make the values of java.home, user.home, >> user.dir, and user.name >> effectively read-only for internal use.? The values are cached during >> initialization and the >> cached values are used. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >> >> Issue: >> ?? https://bugs.openjdk.java.net/browse/JDK-8066709 >> >> CSR: >> ?? https://bugs.openjdk.java.net/browse/JDK-8204235 > > Hi Roger, > > Overall I think the intent of the change is a good one, as is the > reimplementation of internal uses of system properties to use an > internal API instead. > > I think it would be good to have an explicit initialization of the > SystemProperty class (or whatever it ends up being called) instead of > implicitly initializing via a static initializer. If the class is > initialized too early, for some reason, things would go wrong. > That would be a no-op;? I want the values of the properties to be final statics which means they have to be initialized by the static initializer. > Should there be an error, a warning, or assertion checking to make > sure that none of the cached values are null? They should all be > non-null, right? Yes, null will be a fatal InternalError. Thanks, Roger > > s'marks From Roger.Riggs at Oracle.com Tue Jun 5 14:57:03 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jun 2018 10:57:03 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: <8D9FFD19-4501-46A8-9E7E-D3A622CF81EC@oracle.com> <4dcfbb49-a0dc-e69f-d44b-2faa2189c5de@Oracle.com> Message-ID: Hi Max, On 6/4/2018 7:24 PM, Weijun Wang wrote: > Hi Roger > > Thanks for the explanation. > > Another question: Before this change, a SecurityException might be thrown when getProperty() is called, does you new code simulate this behavior? Or in these cases this is unnecessary? Many of the usages were already in doPriv or GetPropertyAction flows so no security check was being done. In a few cases, the existing doPriv is in a different file (e.g. calls to SunEntries.getDeviceFile(utl)). Those would be safer to leave as is, trading off the consistency with confidence about the property check (or add the property access check). The usage in SocksSocketImpl needs a second look, Alan pointed out a difference in behavior. The use of user.name in MimeTable and MailToURLConnection were missing a security check. > Also, looks like all change is inside java.base, do you have a suggestion we use it in other JDK modules? Not at this time. Roger > > Thanks > Max > >> On Jun 5, 2018, at 3:54 AM, Roger Riggs wrote: >> >> Hi Max, >> >> On 6/4/2018 11:41 AM, Wang Weijun wrote: >>> Not a native English speaker, so my feeling might be incorrect. >>> >>> Will someone interpret this as that System.getProperty() will return a cached value? >>> >> I don't think so, it should be clear that the values are cached at initialization or first use. >>> I would say ?Although getProperty() always returns the last value set by setProperty() (I assume this is the current behavior), it is not uncommon that consumers of a system property may read it once and cache the value it for later use, which means setting the property may not have the desired effect?. >>> >> The purpose of the change is to clarify the behavior of the java.base module internal access to the properties. >> How applications handle properties is beyond the intended scope. >>> I also don?t think it?s worth listing the 4 property names in the spec. Quite some other system properties are also cached. Listing them there could further suggests that calling getProperty() on them will return the cached value. >>> >> It seemed worthwhile to highlight the specific properties affected and the release note should be specific. >> I moved them to an implNote as Alan suggested. >> >> Thanks, Roger >> >>> Thanks >>> Max >>> >>> >>>> ? 2018?6?4????9:32?Roger Riggs >>>> ??? >>>> >>>> Please review a change to make the values of java.home, user.home, user.dir, and user.name >>>> effectively read-only for internal use. The values are cached during initialization and the >>>> cached values are used. >>>> >>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>>> >>>> >>>> Issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>>> >>>> >>>> CSR: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8204235 >>>> >>>> >>>> Thanks, Roger >>>> >>>> >>>> From Roger.Riggs at Oracle.com Tue Jun 5 15:01:22 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jun 2018 11:01:22 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <12B561C2-8EE6-4DAE-A352-D53ED485F53C@oracle.com> References: <647D84B5-F364-46F4-A80B-8854A3F889B1@oracle.com> <12B561C2-8EE6-4DAE-A352-D53ED485F53C@oracle.com> Message-ID: Hi Lance, The "unless otherwise specified" part is important so reworeded as: ???? * Changing any standard system property may have unpredictable results ???? * unless otherwise specified. Thanks, Roger On 6/4/2018 4:39 PM, Lance Andersen wrote: > Hi Roger, > > > > In System.java > > ????? > changing any standard system property may have unpredictable > effects. > > ???????? > > Perhaps capitalize ?changing' and maybe consider ?results? vs ?effects? > > > > HTH > > Best > Lance >> On Jun 4, 2018, at 3:55 PM, Roger Riggs > > wrote: >> >> Hi Lance, >> >> On 6/4/2018 12:09 PM, Lance Andersen wrote: >>> Hi Roger, >>> >>> Overall, it looks very good. >>> >>> A couple of quick thoughts, nothing that needs any action unless you >>> feel the desire :-) >>> >>> Line 671 in System.java, it mentions ?standard? system properties. >>> ?(as does the existing Implementation note) but it does not really >>> specify what a standard system property is (though it can be >>> assumed) in getProperties overview. ?Not a big deal, just thought I >>> would point it out in case you had a alternative thought. >> Yes, it is a bit vague and the documentation of properties is spread >> over many packages and classes. >> 'Standard' generally refers to any property defined in the scope of >> Java SE or the documented JDK properties. >> (For example, those subject to the CSR process). >>> >>> Do you think we should had a mention in getProperty() about this >>> cached properties (or do we think the reference to getProperties() >>> is enough) >> Hard to say,? there will be a release note, but the properties >> methods are well known and it is >> likely no one reads the javadoc anymore.? getProperty delegates to >> getProperties for most of its behavior. >> >> Adding a repeat of the API note to System.setProperties and >> System.setProperty might be more to the point, >> warning against setting standard system properties via the API. >> I added the apinote to each of the 6 methods for setting, clearing, >> or getting properties. If that seems excessive, >> I'm open to removing it from those that provide the least value. >> >> Thanks, Roger >> >>> >>> Best >>> Lance >>>> On Jun 4, 2018, at 9:32 AM, Roger Riggs >>> > wrote: >>>> >>>> Please review a change to make the values of java.home, user.home, >>>> user.dir, and user.name >>>> effectively read-only for internal use.? The values are cached >>>> during initialization and the >>>> cached values are used. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>>> >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>>> >>>> CSR: >>>> https://bugs.openjdk.java.net/browse/JDK-8204235 >>>> >>>> Thanks, Roger >>>> >>>> >>> >>> >>> >>> >>> Lance >>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle?Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> >>> >>> >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From chris.hegarty at oracle.com Tue Jun 5 15:05:42 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 5 Jun 2018 16:05:42 +0100 Subject: RFR: JDK-8203891: Upgrade JOpt Simple to 5.0.4 In-Reply-To: <5B0FBC37.2090103@oracle.com> References: <5B0FBC37.2090103@oracle.com> Message-ID: <18b569fb-e7ab-f0db-e30e-51e7ef299aea@oracle.com> On 31/05/18 10:11, Jan Lahoda wrote: > Hi, > > I'd like to upgrade the JOpt Simple library we are using to version 5.0.4. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203891 > Complete webrev: > http://cr.openjdk.java.net/~jlahoda/8203891/webrev.00/complete/ > > Delta webrev only showing (all) JDK changes in JOpt Simple and related > changes in tests needed for the upgrade, etc.: > http://cr.openjdk.java.net/~jlahoda/8203891/webrev.00/joptsimple.delta/ > > Probably the biggest issue with this upgrade is that for two subsequent > parameters: > "--libs=", "/tmp" > "/tmp" used to be interpreted as the parameter of "libs", but now the > "libs" parameter is empty (as there's nothing behind the '='). See the > changes to test/jdk/tools/jmod/JmodTest.java for an example. Hopefully, > this is a reasonable change. > > How does this look? Thanks Jan, I think this looks good. -Chris. From Roger.Riggs at Oracle.com Tue Jun 5 15:10:09 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jun 2018 11:10:09 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <8a1ad331-411b-3e0b-58d1-36f1caad3dac@oracle.com> References: <7e57ecce-a455-8f5d-f6f5-0bb965a86dc6@oracle.com> <26cb79b2-6ccc-f7a1-4cc0-3c13eaf60f43@Oracle.com> <8a1ad331-411b-3e0b-58d1-36f1caad3dac@oracle.com> Message-ID: <1cb3e4f9-da59-78d5-a8ff-049d1bed7f4c@Oracle.com> Hi Alan, On 6/5/2018 2:19 AM, Alan Bateman wrote: > On 04/06/2018 20:59, Roger Riggs wrote: >> : >> >>> >>> Are the changes to SocksSocketImpl correct? I may have missed >>> something but the original called System.getProperty("user.dir") in >>> a privileged block so I'm wondering if getUserNameChecked is needed. >> The existing code in SocksSocketImpl is inconsistent with respect to >> access to user.name; some flows >> use doPriv to access the property and others did not.? If someone >> familiar with the Socks networking function >> can recommend the proper access, it can be revised.? The intent was >> to have the same security checks >> as before. > The original code at L181 is using > GetPropertyAction.privilegedGetProperty so it looks like it reads the > value of the property in a privileged block. The replacement code is > doing an explicit permission check. If I read the original code > correctly then it should only be doing a permission check for the > proxy case. So I think it needs to be checked, another set of eyes > would be useful. Yes, that case did not need the propertyAccess check. (The local getUsername method could be simplified since applicationSetProxy can never be true (assuming no reflection setting it)). Thanks, Roger > > -Alan From Roger.Riggs at Oracle.com Tue Jun 5 15:14:20 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jun 2018 11:14:20 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: <7e57ecce-a455-8f5d-f6f5-0bb965a86dc6@oracle.com> <26cb79b2-6ccc-f7a1-4cc0-3c13eaf60f43@Oracle.com> Message-ID: <73a07938-6e9f-03e8-8919-f1ee3836ffff@Oracle.com> Hi Alan, On 6/5/2018 5:06 AM, Alan Bateman wrote: > On 04/06/2018 20:59, Roger Riggs wrote: >> Hi Alan, >> >> Updated webrev in place: >> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ > The splitting of the note into @apiNote and @implNote mostly looks > good. I think it should be "cached by the Java virtual machine" rather > than "cached internally". It would be good to fix inconsistent line > length while you are there. I'm not keen on the abuse of 'Java Virtual Machine' to mean the entire java runtime. These values are part of the runtime state, not specifically the JVM. > > I still think we need to find a better name for SystemProperty and it > would be good to re-visit how this is initialized in initPhase1. I can suggest StaticProperties? (but I think it will change back in a future enhancement). > I think this is Stuart's point too. It would be nice to call it with > "props" so that it captures the initial value of the interesting > properties. A second best would be to a SystemProperties.capture() or > something explicit. The values should be static final Strings so they need to be initialized in the static initializer. The code can be made more complicated by adding another class but that does not seem warranted. System initialization code is know to be sensitive to ordering, etc. Thanks, Roger > > There's something not quite right with the change to SocksSocketImpl > that I mentioned in the other mail. > > -Alan From xueming.shen at oracle.com Tue Jun 5 15:46:51 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 05 Jun 2018 08:46:51 -0700 Subject: RFR JDK-8200530: '\r' is not supported as "newline" in java.util.jar.Manifest. In-Reply-To: <514183E1-18ED-499C-B42F-14853BED8966@oracle.com> References: <5B1621E5.80301@oracle.com> <514183E1-18ED-499C-B42F-14853BED8966@oracle.com> Message-ID: <5B16B06B.6050109@oracle.com> Thanks Jim! webrev has been updated as suggested. http://cr.openjdk.java.net/~sherman/8200530/webrev -sherman On 06/05/2018 05:17 AM, Jim Laskey wrote: > Attributes.java:380 nit > > - assign c at decl > - only test len if decremented > > byte c; > if ((c = lbuf[--len]) != '\n'&& c != '\r') { > throw new IOException("line too long"); > } > if (len> 0&& lbuf[len-1] == '\r') { > --len; > } > if (len == 0) { > break; > } > ====== > byte c = lbuf[--len]; > if (c != '\n'&& c != '\r') { > throw new IOException("line too long"); > } > if (len> 0&& lbuf[len-1] == '\r'&& --len == 0) { > break; > } > <<<<<< > > > Manifest.java:208 nit > > - same as above > > byte c; > if ((c = lbuf[--len]) != '\n'&& c != '\r') { > throw new IOException("manifest line too long"); > } > if (len> 0&& lbuf[len-1] == '\r') { > --len; > } > if (len == 0&& skipEmptyLines) { > continue; > } > ====== > byte c = lbuf[--len]; > if (c != '\n'&& c != '\r') { > throw new IOException("manifest line too long"); > } > if (len> 0&& lbuf[len-1] == '\r'&& --len == 0&& skipEmptyLines) { > continue; > } > <<<<<< > > Manifest.java:396 nit > > - Shouldn?t c already be loaded with tbuf[tpos-1] (or tbuf[tpos-2]) if ?\r\n?) > > if ((c = tbuf[tpos-1]) == '\n' || c == '\r') { > break; > } > ====== > if (c == '\n' || c == '\r') { > break; > } > <<<<<< > > > +1 > > Cheers, > > ? Jim > > >> On Jun 5, 2018, at 2:38 AM, Xueming Shen wrote: >> >> Hi, >> >> Please help review the changeset for JDK-8200530. >> >> "newline" is specified as |CR LF | LF | CR|(/not followed by/|LF|) in Jar spec [1] from >> the very beginning but our implementation actually never supports "\r"/CR (not >> followed by LF) case. The proposed change here is to add CR as an individual >> supported "newline"/line separator. >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8200530 >> webrev: http://cr.openjdk.java.net/~sherman/8200530/webrev >> >> Thanks, >> Sherman >> >> >> [1] https://docs.oracle.com/javase/10/docs/specs/jar/jar.html From naoto.sato at oracle.com Tue Jun 5 15:48:22 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Tue, 5 Jun 2018 08:48:22 -0700 Subject: [11] RFR: 8202026 8193552 8204269 : ISO 4217 Amendment #165 # 166 #167 Update In-Reply-To: <156571a8-d7ab-2c51-39ea-3242e33399f0@oracle.com> References: <5d50ddec-884c-e78c-4c2c-b9e11d8c3236@oracle.com> <62217aa4-16c5-2ec2-15f7-177d0906c53b@oracle.com> <2099b4c0-b97a-4c1a-f291-fec8f7beb294@oracle.com> <9181f81a-00f4-475c-553f-fa155439fa15@oracle.com> <875fa321-9b73-17f4-c63e-b542e32b6424@oracle.com> <156571a8-d7ab-2c51-39ea-3242e33399f0@oracle.com> Message-ID: <37c4fc7a-a6ac-0dc4-be12-a5f7f764e743@oracle.com> Looks good. Naoto On 6/4/18 11:30 PM, li.jiang at oracle.com wrote: > Hi Naoto, > > I removed the obsoleted currency LTL and LVL from tablea1.txt, added > them into otherCodes in ValidateISO4217.java. > > new webrev as below: > http://cr.openjdk.java.net/~ljiang/8202026/webrev.03/ > > Thanks, > Leo > > On 05/06/2018 2:27 AM, naoto.sato at oracle.com wrote: >> Hi Leo, >> >> Change looks good. One leftover from the previous: >> >> ?>> in CurrencyData.properties. This applies to tabela1.txt as well. >> >> Can you please clean those LV/LT entries in tablea1.txt as well? >> >> Naoto >> >> On 6/4/18 7:41 AM, li.jiang at oracle.com wrote: >>> Hi Naoto, >>> >>> Pls review the updated code: >>> >>> http://cr.openjdk.java.net/~ljiang/8202026/webrev.02/ >>> >>> - As suggested, clean up the old transition dates. >>> - Update copyright year. >>> - ISO 4217 Amendment #167 was published today, which discards the >>> #166, so withdraw the change for #166 in webrev.02. >>> >>> Bug for #167: >>> https://bugs.openjdk.java.net/browse/JDK-8204269 >>> >>> Test passed on mach5. >>> >>> Thanks, >>> Leo >>> >>> On 26/04/2018 1:00 AM, naoto.sato at oracle.com wrote: >>>> Hi Leo, >>>> >>>> Although JDK11 is slated in 09/2018, enabling amendment 166 now is >>>> technically a bug, as it will be effective from June 4. Please use >>>> the transition mechanism in >>>> make/data/currency/CurrencyData.properties and >>>> test/jdk/java/util/Currency/tablea1.txt. >>>> >>>> OTOH, there are old (past) transition entries. I would clean up >>>> those entries, such as: >>>> >>>> ??326 # LATVIA >>>> ??327 LV=LVL;2013-12-31-22-00-00;EUR >>>> >>>> in CurrencyData.properties. This applies to tabela1.txt as well. >>>> >>>> Naoto >>>> >>>> On 4/24/18 8:52 PM, Leo Jiang wrote: >>>>> Forgot to mention, the tests in Currency fold are passed on Mach5. >>>>> >>>>> -Leo >>>>> >>>>> On 04/25/2018 09:33 AM, Leo Jiang wrote: >>>>>> Hi, >>>>>> >>>>>> Please review the changes to address the ISO 4217 Amendment 165 >>>>>> 166 update. >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8193552? 165 >>>>>> https://bugs.openjdk.java.net/browse/JDK-8202026? 166 >>>>>> >>>>>> CR: >>>>>> http://cr.openjdk.java.net/~ljiang/8202026/webrev.00/ >>>>>> >>>>>> >>>>>> Detail: >>>>>> #165 >>>>>> From: >>>>>> MAURITANIA??? Ouguiya??? MRO??? 478??? 2 >>>>>> To: >>>>>> MAURITANIA??? Ouguiya??? MRU??? 929??? 2 >>>>>> >>>>>> #166 >>>>>> From: >>>>>> VENEZUELA (BOLIVARIAN REPUBLIC OF)??? Bol?var??? VEF??? 937??? 2 >>>>>> To: >>>>>> VENEZUELA (BOLIVARIAN REPUBLIC OF)??? Bol?var Soberano??? VES >>>>>> 928??? 2 >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Leo From thomas.stuefe at gmail.com Tue Jun 5 16:10:02 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 5 Jun 2018 18:10:02 +0200 Subject: RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: References: Message-ID: On Tue, Jun 5, 2018 at 3:46 PM, Adam Farley8 wrote: > Hi All, > > Native memory allocation for DBBs is tracked in java.nio.Bits, but that > only includes what the user thinks they are allocating. > Which is exactly what I would expect as a user... > When the VM adds extra memory to the allocation amount this extra bit is > not represented in the Bits total. A cursory glance > shows, minimum, that we round the requested memory quantity up to the heap > word size in the Unsafe.allocateMemory code which I do not understand either - why do we do this? After all, normal allocations from inside hotspot do not get aligned up in size, and the java doc to Unsafe allocateMemory does not state anything about the size being aligned. In addition to questioning the align up of the user requested size, I would be in favor of adding a new NMT tag for these, maybe "mtUnsafe"? That would be an easy fix. >, and > something to do with nmt_header_size in os:malloc() (os.cpp) too. That is mighty unspecific and also wrong. The align-up mentioned above goes into the size reported by Bits; the nmt header size does not. > > On its own, and in small quantities, align_up(sz, HeapWordSize) isn't that > big of an issue. But when you allocate a lot of DBBs, > and coupled with the nmt_header_size business, it makes the Bits values > wrong. The more DBB allocations, the more inaccurate those > numbers will be. To be annoyingly precise, it will never be more wrong than 1:7 on 64bit machines :) - if all memory requested via Unsafe.allocateMemory would be of size 1 byte. > > To get the "+X", it seems to me that the best option would be to introduce > an native method in Bits that fetches "X" directly > from Hotspot, using the same code that Hotspot uses (so we'd have to > abstract-out the Hotspot logic that adds X to the memory > quantity). This way, anyone modifying the Hotspot logic won't risk > rendering the Bits logic wrong again. I don't follow that. > > That's only one way to fix the accuracy problem here though. Suggestions > welcome. You are throwing two effects together: - As mentioned above, I consider the align-up of the user requested size to be at least questionable. It shows up as user size in NMT which should not be. I also fail to see a compelling reason for it, but maybe someone else can enlighten me. - But anything else - NMT headers, overwriter guards, etc added by the VM I consider in the same class as any other overhead incurred e.g. by the CRT or the OS when calling malloc (e.g. malloc allocator bucket size). Basically, rss will go up by more than size requested by malloc. Something maybe worth noting, but IMHO not as part of the numbers returned by java.nio.Bits. Just my 2 cents. Best Regards, Thomas > > Best Regards > > Adam Farley > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From martinrb at google.com Tue Jun 5 16:40:14 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 5 Jun 2018 09:40:14 -0700 Subject: RFR 8203768 : Avoid reallocation in java.base/unix/classes/java/lang/ProcessImpl.java In-Reply-To: References: <09e6208b-52ab-461e-fae6-5cc1c6f671b7@oracle.com> Message-ID: On Mon, Jun 4, 2018 at 10:31 PM, Ivan Gerasimov wrote: > But suppose available() incorrectly claims that a million bytes are > available, but you only read 10. Then there's a tradeoff - the cost of > reallocation versus the risk that the mostly empty backing array will be > retained for a long time if the Process object is not garbage collected. > > On the other hand, if available() claims a million bytes, and then only > 999999990 bytes were read, there will be mostly unnecessary allocation and > copying. > > Yup. > I'm leaning towards the status quo. > > Okay, let's leave it as is :) > It was meant mostly for cleaning up the code, so if there is a doubt, then > it's better keep what we have. > If your version were the status quo, it would not be worth changing. When in doubt, status quo wins. From naoto.sato at oracle.com Tue Jun 5 17:14:10 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Tue, 5 Jun 2018 10:14:10 -0700 Subject: RFR JDK-8203872: Upgrading JDK with latest available LSR data from IANA In-Reply-To: <6aa57b43-c15e-cfee-cb32-922e4eec8489@oracle.com> References: <6aa57b43-c15e-cfee-cb32-922e4eec8489@oracle.com> Message-ID: <623bc32c-5a6e-b4a4-985b-d832e17af203@oracle.com> Looks good. Naoto On 6/4/18 10:13 PM, Nishit Jain wrote: > Hi, > > Please review the fix for JDK-8203872. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203872 > Webrev: http://cr.openjdk.java.net/~nishjain/8203872/webrev.01/ > > Fix: Updated the LSR data to the version 2017-08-15 > > Regards, > Nishit Jain From mandy.chung at oracle.com Tue Jun 5 17:16:51 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 5 Jun 2018 10:16:51 -0700 Subject: RFR: JDK-8203891: Upgrade JOpt Simple to 5.0.4 In-Reply-To: <5B163BBD.9060001@oracle.com> References: <5B0FBC37.2090103@oracle.com> <799bd973-3d11-b4cc-2f16-f4dc6383b4b2@oracle.com> <5B163BBD.9060001@oracle.com> Message-ID: <308ce43a-d585-dc92-9bba-5873780ee555@oracle.com> On 6/5/18 12:29 AM, Jan Lahoda wrote: > > Yes, that would work as well, but there are already invocations like > this in the test. So I opted for using > "--libs=" + ... > and similar, so that both variants are covered by the test. That is fine to keep it as is. The change looks good. Mandy From roger.riggs at oracle.com Tue Jun 5 21:54:45 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Tue, 5 Jun 2018 17:54:45 -0400 Subject: RFR: 8201608 fix broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html In-Reply-To: References: Message-ID: <54e74af0-c8de-b410-30fd-a63cbc171d00@oracle.com> Hi Lance, Can the name change be done using? hg rename to preserve the continuity? Also, while you are there, how about converting ``` to {@code...} etc. Thanks, Roger On 6/4/18 7:22 AM, Lance Andersen wrote: > Hi, > > Bug 8201608 highlights a few broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html > > As part of this fix, I took the liberty to move from package.html to package-info.java > > The webrev can be found at http://cr.openjdk.java.net/~lancea/8201608/webrev.00/ > > I have also attached a diff of the changes as it is less obvious of the webrev prior to the migration to package-info.java > > Best > Lance > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > From martinrb at google.com Tue Jun 5 22:41:45 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 5 Jun 2018 15:41:45 -0700 Subject: Durations in existing JDK APIs In-Reply-To: References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: JDK-8204375: Add TimeUnit#convert(Duration) From stuart.marks at oracle.com Wed Jun 6 00:05:33 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 5 Jun 2018 17:05:33 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: [adding serviceability-dev] Hi serviceability folks, I'm in the process of removing Thread.destroy() and Thread.stop(Throwable) from the Java SE API. Alan and David have pointed out that there are some cross-references to Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in the JDI ThreadReference API. I've adjusted the relevant files. See please review this updated webrev: http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ For context, please see the full review thread on core-libs-dev: http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html On 6/4/18 11:11 PM, Alan Bateman wrote: > The source file that is used to generate the JDWP protocol code and the spec is > in make/data/jdwp/jdwp.spec. The JVM TI spec is at > src/hotspot/share/prims/jvmti.xml. It should be okay to skip those for this > change-set, assuming it is followed up quickly with another change-set to update > those specs. OK. I took a look at those other files and they seem simple enough to update as part of this changeset. If there aren't any objections from anyone, might as well get this all done at once. Thanks, s'marks From zgu at redhat.com Wed Jun 6 00:58:18 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 5 Jun 2018 20:58:18 -0400 Subject: RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers In-Reply-To: References: Message-ID: <9c1af95d-2c0f-b96d-76a7-5b40228996de@redhat.com> On 06/05/2018 12:10 PM, Thomas St?fe wrote:> On Tue, Jun 5, 2018 at 3:46 PM, Adam Farley8 wrote: >> Hi All, >> >> Native memory allocation for DBBs is tracked in java.nio.Bits, but that >> only includes what the user thinks they are allocating. >> > > Which is exactly what I would expect as a user... > I agree with Thomas, there is no point for a user to aware of tracking overhead, and the overhead only incurs when native memory tracking is on. As a matter of fact, it can really confuse user that values can be varied, depending on whether native memory tracking is on. Thanks, -Zhengyu >> When the VM adds extra memory to the allocation amount this extra bit is >> not represented in the Bits total. A cursory glance >> shows, minimum, that we round the requested memory quantity up to the heap >> word size in the Unsafe.allocateMemory code > > which I do not understand either - why do we do this? After all, > normal allocations from inside hotspot do not get aligned up in size, > and the java doc to Unsafe allocateMemory does not state anything > about the size being aligned. > > In addition to questioning the align up of the user requested size, I > would be in favor of adding a new NMT tag for these, maybe "mtUnsafe"? > That would be an easy fix. > >> , and >> something to do with nmt_header_size in os:malloc() (os.cpp) too. > > That is mighty unspecific and also wrong. The align-up mentioned above > goes into the size reported by Bits; the nmt header size does not. > >> >> On its own, and in small quantities, align_up(sz, HeapWordSize) isn't that >> big of an issue. But when you allocate a lot of DBBs, >> and coupled with the nmt_header_size business, it makes the Bits values >> wrong. The more DBB allocations, the more inaccurate those >> numbers will be. > > To be annoyingly precise, it will never be more wrong than 1:7 on > 64bit machines :) - if all memory requested via Unsafe.allocateMemory > would be of size 1 byte. > >> >> To get the "+X", it seems to me that the best option would be to introduce >> an native method in Bits that fetches "X" directly >> from Hotspot, using the same code that Hotspot uses (so we'd have to >> abstract-out the Hotspot logic that adds X to the memory >> quantity). This way, anyone modifying the Hotspot logic won't risk >> rendering the Bits logic wrong again. > > I don't follow that. > >> >> That's only one way to fix the accuracy problem here though. Suggestions >> welcome. > > You are throwing two effects together: > > - As mentioned above, I consider the align-up of the user requested > size to be at least questionable. It shows up as user size in NMT > which should not be. I also fail to see a compelling reason for it, > but maybe someone else can enlighten me. > > - But anything else - NMT headers, overwriter guards, etc added by the > VM I consider in the same class as any other overhead incurred e.g. by > the CRT or the OS when calling malloc (e.g. malloc allocator bucket > size). Basically, rss will go up by more than size requested by > malloc. Something maybe worth noting, but IMHO not as part of the > numbers returned by java.nio.Bits. > > Just my 2 cents. > > Best Regards, Thomas > >> >> Best Regards >> >> Adam Farley >> Unless stated otherwise above: >> IBM United Kingdom Limited - Registered in England and Wales with number >> 741598. >> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From david.holmes at oracle.com Wed Jun 6 03:48:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 6 Jun 2018 13:48:30 +1000 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: <33e2e693-1b5e-5e4f-cfd0-a8d5140b0eba@oracle.com> Hi Stuart, This all looks fine to me. One minor nit in threadPrimitiveDeprecation.html is whether the new statements you reference a particular Java version? It reads a little oddly to go through all the explanation and then say "Thread.x has been removed". In the spirit of @since you might say "has been removed as of JDK 11". Or arguably you could remove discussion of the removed methods altogether. Thanks, David On 6/06/2018 10:05 AM, Stuart Marks wrote: > [adding serviceability-dev] > > Hi serviceability folks, > > I'm in the process of removing Thread.destroy() and > Thread.stop(Throwable) from the Java SE API. Alan and David have pointed > out that there are some cross-references to Thread.stop(Throwable) in > the JDWP and JVMTI specs, as well as in the JDI ThreadReference API. > I've adjusted the relevant files. > > See please review this updated webrev: > > ??? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ > > For context, please see the full review thread on core-libs-dev: > > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html > > > > On 6/4/18 11:11 PM, Alan Bateman wrote: >> The source file that is used to generate the JDWP protocol code and >> the spec is in make/data/jdwp/jdwp.spec. The JVM TI spec is at >> src/hotspot/share/prims/jvmti.xml. It should be okay to skip those for >> this change-set, assuming it is followed up quickly with another >> change-set to update those specs. > > OK. I took a look at those other files and they seem simple enough to > update as part of this changeset. If there aren't any objections from > anyone, might as well get this all done at once. > > Thanks, > > s'marks From turbanoff at gmail.com Tue Jun 5 20:17:35 2018 From: turbanoff at gmail.com (=?UTF-8?B?0JDQvdC00YDQtdC5INCi0YPRgNCx0LDQvdC+0LI=?=) Date: Tue, 5 Jun 2018 23:17:35 +0300 Subject: Implementations of PrimitiveIterator.OfInt#forEachRemaining(IntConsumer) doesn't check IntConsumer for null Message-ID: Hello. I observe that implementations of `forEachRemaining` method inside java.lang.CharSequence doesn't always compare IntConsumer parameter with null. Is there any reason why there is no nonNull check? Performance? Andrey Turbanov. From deepak.kejriwal at oracle.com Wed Jun 6 07:07:09 2018 From: deepak.kejriwal at oracle.com (Deepak Kejriwal) Date: Wed, 6 Jun 2018 00:07:09 -0700 (PDT) Subject: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries In-Reply-To: References: Message-ID: <1b0a8bde-4f32-4045-afbd-18897fbb572b@default> Hi Martin, ? Backporting entire tck directory would be over kill as most of files under tck were checked in as part of ?https://bugs.openjdk.java.net/browse/JDK-8146467 and are not really related to JDK-8186171. May be down the line if needed we can invest time to address it. ? The scope Bug8186171Test.java was to test the fix for scenario mentioned in JDK-8186171. Regards, Deepak ? From: Martin Buchholz Sent: Tuesday, June 5, 2018 5:35 AM To: Deepak Kejriwal ; Doug Lea ```
``` Cc: core-libs-dev Subject: Re: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries ? Hej Deepak, ? This looks alright, but you really need to add that trailing newline on the test file (a Martin pet peeve). I wonder if instead we invest a little more work, but backport the entire tck directory. All tests should pass except for those that test features not yet backported. ? On Mon, Jun 4, 2018 at 4:46 AM, Deepak Kejriwal wrote: Hi all, Please review the fix for JDK8u Backport of https://bugs.openjdk.java.net/browse/JDK-8186171 Webrev: http://cr.openjdk.java.net/~rpatil/8186171/webrev.00/ Summary(also added to backport bug description): The back port for test files is not clean back port as all tests files are extending JSR166TestCase.java which was added to JDK 9 as part of HYPERLINK "https://bugs.openjdk.java.net/browse/JDK-8146467"JDK-8146467: Integrate JSR 166 jck tests into JDK repo. This is not present in JDK8 version. Therefore, I have extracted test are relevant to the fix done JDK-8186171 and created a new test file Bug8186171Test.java that contains below two test methods: .? ? ? ? ?testBug8186171NonDeterministic : This method is copy of "testBug8186171" present in "MapTest.java" of original changeset http://hg.openjdk.java.net/jdk10/master/rev/3f5f9bc0bdc2. As it is based on randomization and runs 1000 times, the method name is suffixed with "NonDeterministic". .? ? ? ? ?testBug8186171Deterministic : This is a new method that runs single time as it produces the exact scenario mentioned in JDK-8186171. Therefore, it is not needed to run multiple times to produce the scenario mentioned in bug. Hence the method name is suffixed with "Deterministic" Regards, Deepak ? From Alan.Bateman at oracle.com Wed Jun 6 10:40:24 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 6 Jun 2018 11:40:24 +0100 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: <7f96f3ae-246e-8960-93fc-0e93aad5dd04@oracle.com> On 06/06/2018 01:05, Stuart Marks wrote: > [adding serviceability-dev] > > Hi serviceability folks, > > I'm in the process of removing Thread.destroy() and > Thread.stop(Throwable) from the Java SE API. Alan and David have > pointed out that there are some cross-references to > Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in the > JDI ThreadReference API. I've adjusted the relevant files. > > See please review this updated webrev: > > ??? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ Have you considered removing the "What about Thread.stop(Throwable)" section completely from the threadPrimitiveDeprecation doc? A new developer reading the javadoc in 2019 shouldn't need to read about a removed method. Otherwise I think this looks good, including the updates to the JDWP and JVM TI specs. -Alan From Alan.Bateman at oracle.com Wed Jun 6 13:37:56 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 6 Jun 2018 14:37:56 +0100 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> Message-ID: <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> On 30/05/2018 22:16, Peter Levart wrote: > I thought there would be some hint from Alan about which of the two > paths we should take for more refinement. > > The Tony's: > > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ > > Or the Alan's with my mods: > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ > Finally getting back to this one. I don't think the two approaches are all that different now. Tony's point on the number of hooks vs. number of locals was an important point but with Peter's update to use a thread local registry then I think we have easy to maintain solution in the DBBCache_Cleanup/webrev.02 patch. So I think I prefer that approach. We need to better name for "JdkThreadLocal", something to indicate that it holds resources or it notified when a thread terminates. Also along the way, we touched on exposing getIfPresent and we should look at that again. If it is expose then it fits well with the second approach too. -Alan From Alan.Bateman at oracle.com Wed Jun 6 13:57:06 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 6 Jun 2018 14:57:06 +0100 Subject: RFR 8204310 : Simpler RandomAccessFile.setLength() on Windows In-Reply-To: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> References: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> Message-ID: <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> On 05/06/2018 07:23, Ivan Gerasimov wrote: > Hello! > > When a file size is modified via RandomAccessFile.setLength(int) on > Windows, it is currently done in three steps: > > SetFilePointer(), SetEndOfFile(), SetFilePointerEx(). > > First two can be combined in one call of SetFileInformationByHandle(), > which will make the code shorter and more aligned with Unix variant > (ftruncate64 is used on Unix systems, which behaves close to > SetFileInformationByHandle(_,FileEndOfFileInfo,_)). > > Would you please help review this trivial fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8204310 > WEBREV: http://cr.openjdk.java.net/~igerasim/8204310/00/webrev/ > > All the existing tests pass Ok. > I think this should be okay but the Windows implementation has a long history of biting the fingers of anyone that dares touch it. Sometimes unexpected behavior changes only come to light long after the change. The recent mails here about Kafka and sparse files is a good example of that. So I think it's important to check the test coverage before pushing this change. Specifically I think we need to check that we have tests that 1. Exercise shrinking, extending, and not change the file length 2. Check getFilePointer when used with setLength. 3. Check how FileChannel behaves when created from a RandomAccessFile but the file length is changed after the FileChannel is obtained. I realize this is more work than what you might have wanted to put into this but we have to extremely careful here. -Alan From jason_mehrens at hotmail.com Wed Jun 6 14:34:04 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Wed, 6 Jun 2018 14:34:04 +0000 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: <5f6cb9eb-f95b-309b-433e-63dde8614948@oracle.com>, Message-ID: Roger, Looks like StaticProperty.initProperty is never called. I assume that was suppose to be called on lines 40-43? Jason ________________________________________ From: core-libs-dev on behalf of Roger Riggs Sent: Tuesday, June 5, 2018 9:18 AM To: Stuart Marks Cc: Core-Libs-Dev Subject: Re: RFR JDK-8066709 Make some JDK system properties read only Hi Stuart, On 6/4/2018 9:52 PM, Stuart Marks wrote: > > > On 6/4/18 6:32 AM, Roger Riggs wrote: >> Please review a change to make the values of java.home, user.home, >> user.dir, and user.name >> effectively read-only for internal use. The values are cached during >> initialization and the >> cached values are used. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8066709 >> >> CSR: >> https://bugs.openjdk.java.net/browse/JDK-8204235 > > Hi Roger, > > Overall I think the intent of the change is a good one, as is the > reimplementation of internal uses of system properties to use an > internal API instead. > > I think it would be good to have an explicit initialization of the > SystemProperty class (or whatever it ends up being called) instead of > implicitly initializing via a static initializer. If the class is > initialized too early, for some reason, things would go wrong. > That would be a no-op; I want the values of the properties to be final statics which means they have to be initialized by the static initializer. > Should there be an error, a warning, or assertion checking to make > sure that none of the cached values are null? They should all be > non-null, right? Yes, null will be a fatal InternalError. Thanks, Roger > > s'marks From tprintezis at twitter.com Wed Jun 6 14:37:56 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 6 Jun 2018 07:37:56 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> Message-ID: Alan, Thanks for your thoughts! Peter, Any chance of taking the two suggestions I made in an earlier e-mail into account? a) It?d be nice if users can override initialValue(), like when using the standard ThreadLocal class, instead of calculateInitialValue(). (I can?t come up with a clean solution on how to do that, though. I?ll think about it for a bit.) b) It?d be very helpful to pass T value to threadTerminated(), as I cannot imagine many use-cases where the value will not be needed. Re: renaming JdkThreadLocal: ThreadLocalWithExitHooks? Re: exposing getIfPresent() : Yes! Pretty please! :-) This will be very helpful and can avoid completely unnecessary allocations. Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 6, 2018 at 9:38:05 AM, Alan Bateman (alan.bateman at oracle.com) wrote: On 30/05/2018 22:16, Peter Levart wrote: > I thought there would be some hint from Alan about which of the two > paths we should take for more refinement. > > The Tony's: > > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ > > Or the Alan's with my mods: > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ > Finally getting back to this one. I don't think the two approaches are all that different now. Tony's point on the number of hooks vs. number of locals was an important point but with Peter's update to use a thread local registry then I think we have easy to maintain solution in the DBBCache_Cleanup/webrev.02 patch. So I think I prefer that approach. We need to better name for "JdkThreadLocal", something to indicate that it holds resources or it notified when a thread terminates. Also along the way, we touched on exposing getIfPresent and we should look at that again. If it is expose then it fits well with the second approach too. -Alan From lance.andersen at oracle.com Wed Jun 6 14:56:47 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 6 Jun 2018 10:56:47 -0400 Subject: RFR: 8201608 fix broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html In-Reply-To: <54e74af0-c8de-b410-30fd-a63cbc171d00@oracle.com> References: <54e74af0-c8de-b410-30fd-a63cbc171d00@oracle.com> Message-ID: <8BEFA400-F64C-4A49-8F2D-CE8BCF9582B6@oracle.com> Hi Roger > On Jun 5, 2018, at 5:54 PM, Roger Riggs wrote: > > Hi Lance, > > Can the name change be done using hg rename to preserve the continuity? I did do an hg rename, and just did it again in a different workspace: hg rename package.html package-info.java ljanders-mac:rowset ljanders\$ hg status -mar M test/jdk/tools/jmod/hashes/HashesTest.java M test/jdk/tools/launcher/modules/addexports/AddExportsTest.java A src/java.sql.rowset/share/classes/javax/sql/rowset/package-info.java R src/java.sql.rowset/share/classes/javax/sql/rowset/package.html > > Also, while you are there, how about converting to {@code...} etc. I do plan to do this, but thought I would keep things minimal for this updateand do that in a follow-on due to the renaming so it is easier to follow in the webrev. Best Lance > > Thanks, Roger > > > On 6/4/18 7:22 AM, Lance Andersen wrote: >> Hi, >> >> Bug 8201608 highlights a few broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html >> >> As part of this fix, I took the liberty to move from package.html to package-info.java >> >> The webrev can be found at http://cr.openjdk.java.net/~lancea/8201608/webrev.00/ >> >> I have also attached a diff of the changes as it is less obvious of the webrev prior to the migration to package-info.java >> >> Best >> Lance >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From peter.levart at gmail.com Wed Jun 6 15:12:42 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 6 Jun 2018 17:12:42 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> Message-ID: <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> Hi Tony, Alan, On 06/06/2018 04:37 PM, Tony Printezis wrote: > Alan, > > Thanks for your thoughts! > > Peter, > > Any chance of taking the two suggestions I made in an earlier e-mail > into account? Right, I'll prepare new webrev with that shortly. > > a) It?d be nice if users can override initialValue(), like when using > the standard ThreadLocal class, instead of calculateInitialValue(). (I > can?t come up with a clean solution on how to do that, though. I?ll > think about it for a bit.) Your concern was that users might accidentally override initialValue() instead of calculateInitialValue(), if I understood you correctly. If that was the concern, they can't, because it is final. If the concern was that users would want to override initialValue() because they are used to do so, then perhaps a javadoc on final initialiValue() pointing to calculateInitialValue() might help them. Do you agree? This is currently internal API and consistent naming is not a big concern. > b) It?d be very helpful to pass T value to threadTerminated(), as I > cannot imagine many use-cases where the value will not be needed. Agree. Will include that in new webrev. > > Re: renaming JdkThreadLocal: ThreadLocalWithExitHooks? Hm. Exit Hooks are already something that is used in JVM (Threads started when VM is about to exit), so this might be confusing for someone. - DisposableThreadLocal - ThreadLocalWithCleanup And my favorite: - TerminatingThreadLocal > > Re: exposing getIfPresent() : Yes! Pretty please! :-) This will be > very helpful and can avoid completely unnecessary allocations. I agree that this would be generally useful functionality. Might be good to ask for opinion on concurrency-interest mailing list first. Will do that. Regards, Peter > > Tony > > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > > On June 6, 2018 at 9:38:05 AM, Alan Bateman (alan.bateman at oracle.com > ) wrote: > >> >> >> On 30/05/2018 22:16, Peter Levart wrote: >> > I thought there would be some hint from Alan about which of the two >> > paths we should take for more refinement. >> > >> > The Tony's: >> > >> > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ >> >> > >> > Or the Alan's with my mods: >> > >> > >> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ >> >> >> > >> Finally getting back to this one. >> >> I don't think the two approaches are all that different now. Tony's >> point on the number of hooks vs. number of locals was an important point >> but with Peter's update to use a thread local registry then I think we >> have easy to maintain solution in the DBBCache_Cleanup/webrev.02 patch. >> So I think I prefer that approach. We need to better name for >> "JdkThreadLocal", something to indicate that it holds resources or it >> notified when a thread terminates. >> >> Also along the way, we touched on exposing getIfPresent and we should >> look at that again. If it is expose then it fits well with the second >> approach too. >> >> -Alan From roger.riggs at oracle.com Wed Jun 6 15:26:51 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Wed, 6 Jun 2018 11:26:51 -0400 Subject: RFR: 8201608 fix broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html In-Reply-To: <8BEFA400-F64C-4A49-8F2D-CE8BCF9582B6@oracle.com> References: <54e74af0-c8de-b410-30fd-a63cbc171d00@oracle.com> <8BEFA400-F64C-4A49-8F2D-CE8BCF9582B6@oracle.com> Message-ID: <3f52b00b-bc67-dcc9-b7da-abab78d08624@oracle.com> Hi Lance, That's fine, the conversion from .html to .java made the diff extensive; hiding the original link fix. The changes are fine by me. Roger On 6/6/18 10:56 AM, Lance Andersen wrote: > Hi Roger >> On Jun 5, 2018, at 5:54 PM, Roger Riggs > > wrote: >> >> Hi Lance, >> >> Can the name change be done using? hg rename to preserve the continuity? > > I did do an hg rename, and just did it again in a different workspace: > > hg rename package.html package-info.java > ljanders-mac:rowset ljanders\$ hg status -mar > M test/jdk/tools/jmod/hashes/HashesTest.java > M test/jdk/tools/launcher/modules/addexports/AddExportsTest.java > A src/java.sql.rowset/share/classes/javax/sql/rowset/package-info.java > R src/java.sql.rowset/share/classes/javax/sql/rowset/package.html >> >> Also, while you are there, how about converting to {@code...} etc. > > I do plan to do this, but thought I would keep things minimal ?for > this updateand do that in a follow-on due to the renaming so it is > easier to follow in the webrev. > > > Best > Lance >> >> Thanks, Roger >> >> >> On 6/4/18 7:22 AM, Lance Andersen wrote: >>> Hi, >>> >>> Bug 8201608 highlights a few broken links in >>> javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html >>> >>> As part of this fix, I took the liberty to move from package.html to >>> package-info.java >>> >>> The webrev can be found at >>> http://cr.openjdk.java.net/~lancea/8201608/webrev.00/ >>> >>> >> > >>> >>> I have also attached a diff of the changes as it is less obvious of >>> the webrev prior to the migration to package-info.java >>> >>> Best >>> Lance >>> >>> ? >>> ? >>> >>> ?Lance >>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> >>> >>> >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From Roger.Riggs at Oracle.com Wed Jun 6 15:31:34 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 6 Jun 2018 11:31:34 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: <5f6cb9eb-f95b-309b-433e-63dde8614948@oracle.com> Message-ID: <0d86047d-285a-a5ae-1fd3-8c3484d55fdc@Oracle.com> Thanks Jason,? 1/2 a fix is not enough. Thanks, Roger p.s. I don't think the comments and suggestions are all in yet. On 6/6/2018 10:34 AM, Jason Mehrens wrote: > Roger, > > Looks like StaticProperty.initProperty is never called. I assume that was suppose to be called on lines 40-43? > > Jason > ________________________________________ > From: core-libs-dev on behalf of Roger Riggs > Sent: Tuesday, June 5, 2018 9:18 AM > To: Stuart Marks > Cc: Core-Libs-Dev > Subject: Re: RFR JDK-8066709 Make some JDK system properties read only > > Hi Stuart, > > On 6/4/2018 9:52 PM, Stuart Marks wrote: >> >> On 6/4/18 6:32 AM, Roger Riggs wrote: >>> Please review a change to make the values of java.home, user.home, >>> user.dir, and user.name >>> effectively read-only for internal use. The values are cached during >>> initialization and the >>> cached values are used. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>> >>> CSR: >>> https://bugs.openjdk.java.net/browse/JDK-8204235 >> Hi Roger, >> >> Overall I think the intent of the change is a good one, as is the >> reimplementation of internal uses of system properties to use an >> internal API instead. >> >> I think it would be good to have an explicit initialization of the >> SystemProperty class (or whatever it ends up being called) instead of >> implicitly initializing via a static initializer. If the class is >> initialized too early, for some reason, things would go wrong. >> > That would be a no-op; I want the values of the properties to be final > statics which means they have to > be initialized by the static initializer. >> Should there be an error, a warning, or assertion checking to make >> sure that none of the cached values are null? They should all be >> non-null, right? > Yes, null will be a fatal InternalError. > > Thanks, Roger > >> s'marks From tprintezis at twitter.com Wed Jun 6 15:41:20 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 6 Jun 2018 08:41:20 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> Message-ID: Peter, You?re totally right re: JdkThreadLocal::initialValue being final and cannot be overridden (I had completely missed the final keyword when I first looked at the webrev). But it?d be nice to have the same API in both versions of ThreadLocal. I assume you didn?t want to override get() since you only wanted to update the REGISTRY the first time get() is called by a thread. If getIfPresent() is exposed, would something like the following work? @Override public T get() { final T value = getIfPresent(); if (value != null) { return value; } REGISTRY.get().add(this); return super.get(); } One more question re: getIfPresent() (and maybe I?m overthinking this): It returns null to indicate that a value is not present. Isn?t null a valid ThreadLocal value? Would using an Optional here be more appropriate? Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 6, 2018 at 11:12:44 AM, Peter Levart (peter.levart at gmail.com) wrote: Hi Tony, Alan, On 06/06/2018 04:37 PM, Tony Printezis wrote: Alan, Thanks for your thoughts! Peter, Any chance of taking the two suggestions I made in an earlier e-mail into account? Right, I'll prepare new webrev with that shortly. a) It?d be nice if users can override initialValue(), like when using the standard ThreadLocal class, instead of calculateInitialValue(). (I can?t come up with a clean solution on how to do that, though. I?ll think about it for a bit.) Your concern was that users might accidentally override initialValue() instead of calculateInitialValue(), if I understood you correctly. If that was the concern, they can't, because it is final. If the concern was that users would want to override initialValue() because they are used to do so, then perhaps a javadoc on final initialiValue() pointing to calculateInitialValue() might help them. Do you agree? This is currently internal API and consistent naming is not a big concern. b) It?d be very helpful to pass T value to threadTerminated(), as I cannot imagine many use-cases where the value will not be needed. Agree. Will include that in new webrev. Re: renaming JdkThreadLocal: ThreadLocalWithExitHooks? Hm. Exit Hooks are already something that is used in JVM (Threads started when VM is about to exit), so this might be confusing for someone. - DisposableThreadLocal - ThreadLocalWithCleanup And my favorite: - TerminatingThreadLocal Re: exposing getIfPresent() : Yes! Pretty please! :-) This will be very helpful and can avoid completely unnecessary allocations. I agree that this would be generally useful functionality. Might be good to ask for opinion on concurrency-interest mailing list first. Will do that. Regards, Peter Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 6, 2018 at 9:38:05 AM, Alan Bateman (alan.bateman at oracle.com) wrote: On 30/05/2018 22:16, Peter Levart wrote: > I thought there would be some hint from Alan about which of the two > paths we should take for more refinement. > > The Tony's: > > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ > > Or the Alan's with my mods: > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ > Finally getting back to this one. I don't think the two approaches are all that different now. Tony's point on the number of hooks vs. number of locals was an important point but with Peter's update to use a thread local registry then I think we have easy to maintain solution in the DBBCache_Cleanup/webrev.02 patch. So I think I prefer that approach. We need to better name for "JdkThreadLocal", something to indicate that it holds resources or it notified when a thread terminates. Also along the way, we touched on exposing getIfPresent and we should look at that again. If it is expose then it fits well with the second approach too. -Alan From lance.andersen at oracle.com Wed Jun 6 15:46:11 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 6 Jun 2018 11:46:11 -0400 Subject: RFR: 8201608 fix broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html In-Reply-To: <3f52b00b-bc67-dcc9-b7da-abab78d08624@oracle.com> References: <54e74af0-c8de-b410-30fd-a63cbc171d00@oracle.com> <8BEFA400-F64C-4A49-8F2D-CE8BCF9582B6@oracle.com> <3f52b00b-bc67-dcc9-b7da-abab78d08624@oracle.com> Message-ID: <2B8887F4-9529-4909-B5BB-F9FD815858D1@oracle.com> Hi Roger > On Jun 6, 2018, at 11:26 AM, Roger Riggs wrote: > > Hi Lance, > > That's fine, the conversion from .html to .java made the diff extensive; hiding the original link fix. Yes, I had attached them originally to the email with the diff against package.html and then forgot it would be stripped by the mail server so I attached it to the bug per Paul?s suggestion. > The changes are fine by me. Thank you Best Lance > > Roger > > > On 6/6/18 10:56 AM, Lance Andersen wrote: >> Hi Roger >>> On Jun 5, 2018, at 5:54 PM, Roger Riggs > wrote: >>> >>> Hi Lance, >>> >>> Can the name change be done using hg rename to preserve the continuity? >> >> I did do an hg rename, and just did it again in a different workspace: >> >> hg rename package.html package-info.java >> ljanders-mac:rowset ljanders\$ hg status -mar >> M test/jdk/tools/jmod/hashes/HashesTest.java >> M test/jdk/tools/launcher/modules/addexports/AddExportsTest.java >> A src/java.sql.rowset/share/classes/javax/sql/rowset/package-info.java >> R src/java.sql.rowset/share/classes/javax/sql/rowset/package.html >>> >>> Also, while you are there, how about converting to {@code...} etc. >> >> I do plan to do this, but thought I would keep things minimal for this updateand do that in a follow-on due to the renaming so it is easier to follow in the webrev. >> >> >> Best >> Lance >>> >>> Thanks, Roger >>> >>> >>> On 6/4/18 7:22 AM, Lance Andersen wrote: >>>> Hi, >>>> >>>> Bug 8201608 highlights a few broken links in javax/sql/rowset/package.html and javax/sql/rowset/spi/package.html >>>> >>>> As part of this fix, I took the liberty to move from package.html to package-info.java >>>> >>>> The webrev can be found at http://cr.openjdk.java.net/~lancea/8201608/webrev.00/ > >>>> >>>> I have also attached a diff of the changes as it is less obvious of the webrev prior to the migration to package-info.java >>>> >>>> Best >>>> Lance >>>> >>>> > >>>> > > >>>> >Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>> Oracle Java Engineering >>>> 1 Network Drive >>>> Burlington, MA 01803 >>>> Lance.Andersen at oracle.com > >>>> >>>> >>> >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From paul.sandoz at oracle.com Wed Jun 6 15:53:17 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 6 Jun 2018 08:53:17 -0700 Subject: Implementations of PrimitiveIterator.OfInt#forEachRemaining(IntConsumer) doesn't check IntConsumer for null In-Reply-To: References: Message-ID: Performance should not be a concern, null checking is highly optimized by the JIT and is anyway required when the consumer is accessed in the loop. It?s probably an oversight but FWIW these internal iterators are never exposed directly and are wrapped in spliterators and those wrappers will perform dominating null checks. Paul. > On Jun 5, 2018, at 1:17 PM, ?????? ???????? wrote: > > Hello. > > I observe that implementations of `forEachRemaining` method inside > java.lang.CharSequence doesn't always compare IntConsumer parameter with > null. > Is there any reason why there is no nonNull check? Performance? > > Andrey Turbanov. From iris.clark at oracle.com Wed Jun 6 16:28:15 2018 From: iris.clark at oracle.com (Iris Clark) Date: Wed, 6 Jun 2018 09:28:15 -0700 (PDT) Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <7f96f3ae-246e-8960-93fc-0e93aad5dd04@oracle.com> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <7f96f3ae-246e-8960-93fc-0e93aad5dd04@oracle.com> Message-ID: <9d09f2b9-775d-4c95-b54b-37c426c62d3d@default> Hi, Stuart. I think you need to make changes to this file too: http://hg.openjdk.java.net/jdk/jdk/file/tip/src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html Thanks, iris -----Original Message----- From: Alan Bateman Sent: Wednesday, June 6, 2018 3:40 AM To: Stuart Marks ; serviceability-dev Cc: core-libs-dev Subject: Re: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) On 06/06/2018 01:05, Stuart Marks wrote: > [adding serviceability-dev] > > Hi serviceability folks, > > I'm in the process of removing Thread.destroy() and > Thread.stop(Throwable) from the Java SE API. Alan and David have > pointed out that there are some cross-references to > Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in the > JDI ThreadReference API. I've adjusted the relevant files. > > See please review this updated webrev: > > ??? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ Have you considered removing the "What about Thread.stop(Throwable)" section completely from the threadPrimitiveDeprecation doc? A new developer reading the javadoc in 2019 shouldn't need to read about a removed method. Otherwise I think this looks good, including the updates to the JDWP and JVM TI specs. -Alan From peter.levart at gmail.com Wed Jun 6 16:58:33 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 6 Jun 2018 18:58:33 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> Message-ID: <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> On 06/06/18 17:41, Tony Printezis wrote: > Peter, > > You?re totally right re: JdkThreadLocal::initialValue being final and > cannot be overridden (I had completely missed the final keyword when I > first looked at the webrev). But it?d be nice to have the same API in > both versions of ThreadLocal. I assume you didn?t want to override > get() since you only wanted to update the REGISTRY the first time > get() is called by a thread. If getIfPresent() is exposed, would > something like the following work? > > @Override > public T get() { > ? final T value = getIfPresent(); > ? if (value != null) { > ? ? ? return value; > ? } > ? REGISTRY.get().add(this); > ? return super.get(); > } This would work, but if mapped value was 'null', it would keep calling REGISTRY.get().add(this) for each get(). Logically correct, but with useless overhead. ?Let me try to do something that might satisfy you... (in next webrev). > > One more question re: getIfPresent() (and maybe I?m overthinking > this): It returns null to indicate that a value is not present. Isn?t > null a valid ThreadLocal value? Would using an Optional here be more > appropriate? The problem with Optional is that it does not provide an additional value. Optional.empty() is a replacement for null. There's no Optional.of(null). So what would getIfPresent() return when there was a mapping present, but that mapping was null? Regards, Peter > > Tony > > ????? > Tony Printezis | @TonyPrintezis | tprintezis at twitter.com > > > > On June 6, 2018 at 11:12:44 AM, Peter Levart (peter.levart at gmail.com > ) wrote: > >> Hi Tony, Alan, >> >> On 06/06/2018 04:37 PM, Tony Printezis wrote: >>> Alan, >>> >>> Thanks for your thoughts! >>> >>> Peter, >>> >>> Any chance of taking the two suggestions I made in an earlier e-mail >>> into account? >> >> Right, I'll prepare new webrev with that shortly. >> >>> >>> a) It?d be nice if users can override initialValue(), like when >>> using the standard ThreadLocal class, instead of >>> calculateInitialValue(). (I can?t come up with a clean solution on >>> how to do that, though. I?ll think about it for a bit.) >> >> Your concern was that users might accidentally override >> initialValue() instead of calculateInitialValue(), if I understood >> you correctly. If that was the concern, they can't, because it is >> final. If the concern was that users would want to override >> initialValue() because they are used to do so, then perhaps a javadoc >> on final initialiValue() pointing to calculateInitialValue() might >> help them. Do you agree? This is currently internal API and >> consistent naming is not a big concern. >> >>> b) It?d be very helpful to pass T value to threadTerminated(), as I >>> cannot imagine many use-cases where the value will not be needed. >> >> Agree. Will include that in new webrev. >> >>> >>> Re: renaming JdkThreadLocal: ThreadLocalWithExitHooks? >> >> Hm. Exit Hooks are already something that is used in JVM (Threads >> started when VM is about to exit), so this might be confusing for >> someone. >> >> - DisposableThreadLocal >> - ThreadLocalWithCleanup >> >> And my favorite: >> >> - TerminatingThreadLocal >> >> >>> >>> Re: exposing getIfPresent() : Yes! Pretty please! :-) This will be >>> very helpful and can avoid completely unnecessary allocations. >> >> I agree that this would be generally useful functionality. Might be >> good to ask for opinion on concurrency-interest mailing list first. >> Will do that. >> >> Regards, Peter >> >>> >>> Tony >>> >>> >>> ????? >>> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >>> >>> >>> >>> On June 6, 2018 at 9:38:05 AM, Alan Bateman (alan.bateman at oracle.com >>> ) wrote: >>> >>>> >>>> >>>> On 30/05/2018 22:16, Peter Levart wrote: >>>> > I thought there would be some hint from Alan about which of the two >>>> > paths we should take for more refinement. >>>> > >>>> > The Tony's: >>>> > >>>> > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ >>>> >>>> > >>>> > Or the Alan's with my mods: >>>> > >>>> > >>>> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ >>>> >>>> > >>>> Finally getting back to this one. >>>> >>>> I don't think the two approaches are all that different now. Tony's >>>> point on the number of hooks vs. number of locals was an important >>>> point >>>> but with Peter's update to use a thread local registry then I think we >>>> have easy to maintain solution in the DBBCache_Cleanup/webrev.02 patch. >>>> So I think I prefer that approach. We need to better name for >>>> "JdkThreadLocal", something to indicate that it holds resources or it >>>> notified when a thread terminates. >>>> >>>> Also along the way, we touched on exposing getIfPresent and we should >>>> look at that again. If it is expose then it fits well with the second >>>> approach too. >>>> >>>> -Alan >> From martinrb at google.com Wed Jun 6 17:03:09 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 6 Jun 2018 10:03:09 -0700 Subject: Durations in existing JDK APIs In-Reply-To: References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: OK, here is a RFR for low-hanging changes (some of these changes have already been reviewed by some of you): 8204375: Add TimeUnit#convert(Duration) http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/convertDuration/index.html https://bugs.openjdk.java.net/browse/JDK-8204375 8204444: java.time cleanup http://cr.openjdk.java.net/~martin/webrevs/jdk/time-lint/ https://bugs.openjdk.java.net/browse/JDK-8204444 8204377: Rename Object#wait parameter name from "timeout" to "timeoutMillis" http://cr.openjdk.java.net/~martin/webrevs/jdk/Object-wait-timeout/ https://bugs.openjdk.java.net/browse/JDK-8204377 From mandy.chung at oracle.com Wed Jun 6 17:13:49 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 6 Jun 2018 10:13:49 -0700 Subject: Draft JEP proposal: JDK-8200758: Packaging Tool In-Reply-To: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> References: <945981dc-3af0-5452-524a-14279d8fb1c9@oracle.com> Message-ID: On 5/30/18 5:10 PM, Kevin Rushforth wrote: > I would like to propose the following Draft JEP [1] for discussion. > > JDK-8200758: Packaging Tool > > This is intended as a JDK-scope replacement for the existing > javapackager tool that ships with Oracle JDK 10 (and earlier Oracle JDK > releases), and was delivered as part of the JavaFX build. The > javapackager tool has been removed from Oracle JDK 11 along with the > removal of JavaFX. > > Comments on this JEP are welcome. It is currently not targeted for a > specific release, but we are aiming for JDK 12. > > -- Kevin > > [1] https://bugs.openjdk.java.net/browse/JDK-820075 Good to see this moving along. A couple of comments: I think the JEP may need to mention the layout of an application image and define what's supported and unsupported to make it clear for developers what should or should not depend on, for example, the location of the application launcher and user-editable configuration files etcs. javapackager strips the tool launchers from the run-time image and enables compression. Does jpackager do something similar? As the development moves along, it'd be useful to include some command-line examples and describe the content of the run-time image produced. As part of this JEP, we need to look at the difference of the native launcher created by jpackager and by jlink. jlink creates a native launcher for modular application and possibly share same mechanism in creating native launchers. Mandy From martinrb at google.com Wed Jun 6 18:01:39 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 6 Jun 2018 11:01:39 -0700 Subject: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries In-Reply-To: <1b0a8bde-4f32-4045-afbd-18897fbb572b@default> References: <1b0a8bde-4f32-4045-afbd-18897fbb572b@default> Message-ID: Alright, I leave it to you. Your current change is fine. On Wed, Jun 6, 2018 at 12:07 AM, Deepak Kejriwal wrote: > Hi Martin, > > > > Backporting entire tck directory would be over kill as most of files under > tck were checked in as part of https://bugs.openjdk.java. > net/browse/JDK-8146467 and are not really related to JDK-8186171. May be > down the line if needed we can invest time to address it. > > > > The scope Bug8186171Test.java was to test the fix for scenario mentioned > in JDK-8186171. > > Regards, > > Deepak > > > > *From:* Martin Buchholz > *Sent:* Tuesday, June 5, 2018 5:35 AM > *To:* Deepak Kejriwal ; Doug Lea < > dl at cs.oswego.edu> > *Cc:* core-libs-dev > *Subject:* Re: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue > may not work after Iterator.remove() called for previous entries > > > > Hej Deepak, > > > > This looks alright, but you really need to add that trailing newline on > the test file (a Martin pet peeve). > > I wonder if instead we invest a little more work, but backport the entire > tck directory. > > All tests should pass except for those that test features not yet > backported. > > > > On Mon, Jun 4, 2018 at 4:46 AM, Deepak Kejriwal < > deepak.kejriwal at oracle.com> wrote: > > Hi all, > > > > Please review the fix for JDK8u Backport of https://bugs.openjdk.java.net/ > browse/JDK-8186171 > > Webrev: http://cr.openjdk.java.net/~rpatil/8186171/webrev.00/ > > > > Summary(also added to backport bug description): > > > > The back port for test files is not clean back port as all tests files are > extending JSR166TestCase.java which was added to JDK 9 as part of HYPERLINK > "https://bugs.openjdk.java.net/browse/JDK-8146467"JDK-8146467: Integrate > JSR 166 jck tests into JDK repo. This is not present in JDK8 version. > > Therefore, I have extracted test are relevant to the fix done JDK-8186171 > and created a new test file Bug8186171Test.java that contains below two > test methods: > > > > . testBug8186171NonDeterministic : This method is copy of > "testBug8186171" present in "MapTest.java" of original changeset > http://hg.openjdk.java.net/jdk10/master/rev/3f5f9bc0bdc2. As it is based > on randomization and runs 1000 times, the method name is suffixed with > "NonDeterministic". > > . testBug8186171Deterministic : This is a new method that runs > single time as it produces the exact scenario mentioned in JDK-8186171. > Therefore, it is not needed to run multiple times to produce the scenario > mentioned in bug. Hence the method name is suffixed with "Deterministic" > > > > Regards, > > Deepak > > > From martinrb at google.com Wed Jun 6 18:03:17 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 6 Jun 2018 11:03:17 -0700 Subject: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries In-Reply-To: References: <1b0a8bde-4f32-4045-afbd-18897fbb572b@default> Message-ID: On Wed, Jun 6, 2018 at 11:01 AM, Martin Buchholz wrote: > Alright, I leave it to you. Your current change is fine. > Except, add that trailing newline to the test! From peter.levart at gmail.com Wed Jun 6 18:55:49 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 6 Jun 2018 20:55:49 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> Message-ID: <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Ok, here's next webrev: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.03/ The changes from webrev.02 are: - renamed JdkThreadLocal -> TerminatingThreadLocal - instead of overriding initialValue(), ThreadLocal.setInitialValue() calls TerminatingThreadLocal.register() at the right moment - TerminatingThreadLocal.threadTerminated() now takes a (T value) parameter - TerminatingThreadLocal.REGISTRY uses IdentityHashSet instead of HashSet (if .equals()/.hashCode() are overriden in some TerminatingThreadLocal subclass) David Lloyd has commented on concurrency-interest about ThreadLocal.getIfPresent(). There is a concern that such new method would be inconsistent with what ThreadLocal.get() returns from existing ThreadLocal subclasses that override .get() and possibly transform the value returned from super.get() on the way. An alternative to "T getIfPresent()" is a "boolean isPresent()" method. Even if it remains package-private, with such method TerminatingThreadLocal could be implemented as: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ If TreadLocal.isPresent() was made public, the code could be further simplified. Regards, Peter On 06/06/18 18:58, Peter Levart wrote: > > > On 06/06/18 17:41, Tony Printezis wrote: >> Peter, >> >> You?re totally right re: JdkThreadLocal::initialValue being final and >> cannot be overridden (I had completely missed the final keyword when >> I first looked at the webrev). But it?d be nice to have the same API >> in both versions of ThreadLocal. I assume you didn?t want to override >> get() since you only wanted to update the REGISTRY the first time >> get() is called by a thread. If getIfPresent() is exposed, would >> something like the following work? >> >> @Override >> public T get() { >> ? final T value = getIfPresent(); >> ? if (value != null) { >> ? ? ? return value; >> ? } >> ? REGISTRY.get().add(this); >> ? return super.get(); >> } > > This would work, but if mapped value was 'null', it would keep calling > REGISTRY.get().add(this) for each get(). Logically correct, but with > useless overhead. > > ?Let me try to do something that might satisfy you... (in next webrev). > >> >> One more question re: getIfPresent() (and maybe I?m overthinking >> this): It returns null to indicate that a value is not present. Isn?t >> null a valid ThreadLocal value? Would using an Optional here be more >> appropriate? > > The problem with Optional is that it does not provide an additional > value. Optional.empty() is a replacement for null. There's no > Optional.of(null). So what would getIfPresent() return when there was > a mapping present, but that mapping was null? > > Regards, Peter > >> >> Tony >> >> ????? >> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >> >> >> >> On June 6, 2018 at 11:12:44 AM, Peter Levart (peter.levart at gmail.com >> ) wrote: >> >>> Hi Tony, Alan, >>> >>> On 06/06/2018 04:37 PM, Tony Printezis wrote: >>>> Alan, >>>> >>>> Thanks for your thoughts! >>>> >>>> Peter, >>>> >>>> Any chance of taking the two suggestions I made in an earlier >>>> e-mail into account? >>> >>> Right, I'll prepare new webrev with that shortly. >>> >>>> >>>> a) It?d be nice if users can override initialValue(), like when >>>> using the standard ThreadLocal class, instead of >>>> calculateInitialValue(). (I can?t come up with a clean solution on >>>> how to do that, though. I?ll think about it for a bit.) >>> >>> Your concern was that users might accidentally override >>> initialValue() instead of calculateInitialValue(), if I understood >>> you correctly. If that was the concern, they can't, because it is >>> final. If the concern was that users would want to override >>> initialValue() because they are used to do so, then perhaps a >>> javadoc on final initialiValue() pointing to calculateInitialValue() >>> might help them. Do you agree? This is currently internal API and >>> consistent naming is not a big concern. >>> >>>> b) It?d be very helpful to pass T value to threadTerminated(), as I >>>> cannot imagine many use-cases where the value will not be needed. >>> >>> Agree. Will include that in new webrev. >>> >>>> >>>> Re: renaming JdkThreadLocal: ThreadLocalWithExitHooks? >>> >>> Hm. Exit Hooks are already something that is used in JVM (Threads >>> started when VM is about to exit), so this might be confusing for >>> someone. >>> >>> - DisposableThreadLocal >>> - ThreadLocalWithCleanup >>> >>> And my favorite: >>> >>> - TerminatingThreadLocal >>> >>> >>>> >>>> Re: exposing getIfPresent() : Yes! Pretty please! :-) This will be >>>> very helpful and can avoid completely unnecessary allocations. >>> >>> I agree that this would be generally useful functionality. Might be >>> good to ask for opinion on concurrency-interest mailing list first. >>> Will do that. >>> >>> Regards, Peter >>> >>>> >>>> Tony >>>> >>>> >>>> ????? >>>> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >>>> >>>> >>>> >>>> On June 6, 2018 at 9:38:05 AM, Alan Bateman >>>> (alan.bateman at oracle.com ) wrote: >>>> >>>>> >>>>> >>>>> On 30/05/2018 22:16, Peter Levart wrote: >>>>> > I thought there would be some hint from Alan about which of the two >>>>> > paths we should take for more refinement. >>>>> > >>>>> > The Tony's: >>>>> > >>>>> > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ >>>>> >>>>> > >>>>> > Or the Alan's with my mods: >>>>> > >>>>> > >>>>> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ >>>>> >>>>> > >>>>> Finally getting back to this one. >>>>> >>>>> I don't think the two approaches are all that different now. Tony's >>>>> point on the number of hooks vs. number of locals was an important >>>>> point >>>>> but with Peter's update to use a thread local registry then I think we >>>>> have easy to maintain solution in the DBBCache_Cleanup/webrev.02 >>>>> patch. >>>>> So I think I prefer that approach. We need to better name for >>>>> "JdkThreadLocal", something to indicate that it holds resources or it >>>>> notified when a thread terminates. >>>>> >>>>> Also along the way, we touched on exposing getIfPresent and we should >>>>> look at that again. If it is expose then it fits well with the second >>>>> approach too. >>>>> >>>>> -Alan >>> > From xueming.shen at oracle.com Wed Jun 6 19:39:02 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 06 Jun 2018 12:39:02 -0700 Subject: RFR https://bugs.openjdk.java.net/browse/JDK-8204494 Message-ID: <5B183856.60108@oracle.com> Hi Please help review the change for JDK-8204494, a regression caused by the fix for JDK-8200530 [1]. It appears that fix fails to deal with the corner case that a pair of \r\n is at the boundary of the internal buf of Manifest.FastInputtStream, which is 8192 for the default. In that scenario, the previous change fails to consume the trailing \n, then triggers the Attributes parser fails to work on the "continuation line" defined by the jar spec. issue: https://bugs.openjdk.java.net/browse/JDK-8204494 webrev: http://cr.openjdk.java.net/~sherman/8204494/webrev/ Fix has been verified by the regression reported in INTJDK-7627771. Thanks, Sherman [1] issue: https://bugs.openjdk.java.net/browse/JDK-8200530 webrev: http://cr.openjdk.java.net/~sherman/8200530/webrev From roger.riggs at oracle.com Wed Jun 6 20:02:43 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Wed, 6 Jun 2018 16:02:43 -0400 Subject: Durations in existing JDK APIs In-Reply-To: References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: Hi Martin, All three look fine to me.? +3 Roger On 6/6/18 1:03 PM, Martin Buchholz wrote: > OK, here is a RFR for low-hanging changes (some of these changes have > already been reviewed by some of you): > > 8204375: Add TimeUnit#convert(Duration) > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/convertDuration/index.html > > https://bugs.openjdk.java.net/browse/JDK-8204375 > > 8204444: java.time cleanup > http://cr.openjdk.java.net/~martin/webrevs/jdk/time-lint/ > > https://bugs.openjdk.java.net/browse/JDK-8204444 > > 8204377: Rename Object#wait parameter name from "timeout" to > "timeoutMillis" > http://cr.openjdk.java.net/~martin/webrevs/jdk/Object-wait-timeout/ > > https://bugs.openjdk.java.net/browse/JDK-8204377 > From tprintezis at twitter.com Wed Jun 6 20:38:06 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Wed, 6 Jun 2018 13:38:06 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: Hi Peter, Thanks for the updated webrev! Please see inline. ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 6, 2018 at 2:55:51 PM, Peter Levart (peter.levart at gmail.com) wrote: Ok, here's next webrev: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.03/ The changes from webrev.02 are: - renamed JdkThreadLocal -> TerminatingThreadLocal #ThumbsUp - instead of overriding initialValue(), ThreadLocal.setInitialValue() calls TerminatingThreadLocal.register() at the right moment Thanks. I much prefer not having to introduce calculateInitialValue(). One extra suggestion: Given you introduced a call to TerminatingThreadLocal from ThreadLocal, would it make sense to maybe do the same for set() and remove() (you?d have to add an appropriate check in unregister) and not override them in TerminatingTreadLocal? I totally get why you might not want to do that (an extra check when a ThreadLocal is not the Terminating one). I?m OK either way. - TerminatingThreadLocal.threadTerminated() now takes a (T value) parameter #ThumbsUp - TerminatingThreadLocal.REGISTRY uses IdentityHashSet instead of HashSet (if .equals()/.hashCode() are overriden in some TerminatingThreadLocal subclass) #ThumbsUp David Lloyd has commented on concurrency-interest about ThreadLocal.getIfPresent(). There is a concern that such new method would be inconsistent with what ThreadLocal.get() returns from existing ThreadLocal subclasses that override .get() and possibly transform the value returned from super.get() on the way. That?s a very good point. An alternative to "T getIfPresent()" is a "boolean isPresent()" method. Even if it remains package-private, with such method TerminatingThreadLocal could be implemented as: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ If TreadLocal.isPresent() was made public, the code could be further simplified. Something like: if (tl.isPresent()) { T val = t.get(); ?. } will do two look-ups if the value exists. However, that?s better than allocating unnecessarily. So, I?ll take isPresent() over not having a way to check whether a value exists. Thumbs up from me. Tony Regards, Peter On 06/06/18 18:58, Peter Levart wrote: On 06/06/18 17:41, Tony Printezis wrote: Peter, You?re totally right re: JdkThreadLocal::initialValue being final and cannot be overridden (I had completely missed the final keyword when I first looked at the webrev). But it?d be nice to have the same API in both versions of ThreadLocal. I assume you didn?t want to override get() since you only wanted to update the REGISTRY the first time get() is called by a thread. If getIfPresent() is exposed, would something like the following work? @Override public T get() { final T value = getIfPresent(); if (value != null) { return value; } REGISTRY.get().add(this); return super.get(); } This would work, but if mapped value was 'null', it would keep calling REGISTRY.get().add(this) for each get(). Logically correct, but with useless overhead. Let me try to do something that might satisfy you... (in next webrev). One more question re: getIfPresent() (and maybe I?m overthinking this): It returns null to indicate that a value is not present. Isn?t null a valid ThreadLocal value? Would using an Optional here be more appropriate? The problem with Optional is that it does not provide an additional value. Optional.empty() is a replacement for null. There's no Optional.of(null). So what would getIfPresent() return when there was a mapping present, but that mapping was null? Regards, Peter Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 6, 2018 at 11:12:44 AM, Peter Levart (peter.levart at gmail.com) wrote: Hi Tony, Alan, On 06/06/2018 04:37 PM, Tony Printezis wrote: Alan, Thanks for your thoughts! Peter, Any chance of taking the two suggestions I made in an earlier e-mail into account? Right, I'll prepare new webrev with that shortly. a) It?d be nice if users can override initialValue(), like when using the standard ThreadLocal class, instead of calculateInitialValue(). (I can?t come up with a clean solution on how to do that, though. I?ll think about it for a bit.) Your concern was that users might accidentally override initialValue() instead of calculateInitialValue(), if I understood you correctly. If that was the concern, they can't, because it is final. If the concern was that users would want to override initialValue() because they are used to do so, then perhaps a javadoc on final initialiValue() pointing to calculateInitialValue() might help them. Do you agree? This is currently internal API and consistent naming is not a big concern. b) It?d be very helpful to pass T value to threadTerminated(), as I cannot imagine many use-cases where the value will not be needed. Agree. Will include that in new webrev. Re: renaming JdkThreadLocal: ThreadLocalWithExitHooks? Hm. Exit Hooks are already something that is used in JVM (Threads started when VM is about to exit), so this might be confusing for someone. - DisposableThreadLocal - ThreadLocalWithCleanup And my favorite: - TerminatingThreadLocal Re: exposing getIfPresent() : Yes! Pretty please! :-) This will be very helpful and can avoid completely unnecessary allocations. I agree that this would be generally useful functionality. Might be good to ask for opinion on concurrency-interest mailing list first. Will do that. Regards, Peter Tony ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 6, 2018 at 9:38:05 AM, Alan Bateman (alan.bateman at oracle.com) wrote: On 30/05/2018 22:16, Peter Levart wrote: > I thought there would be some hint from Alan about which of the two > paths we should take for more refinement. > > The Tony's: > > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ > > Or the Alan's with my mods: > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ > Finally getting back to this one. I don't think the two approaches are all that different now. Tony's point on the number of hooks vs. number of locals was an important point but with Peter's update to use a thread local registry then I think we have easy to maintain solution in the DBBCache_Cleanup/webrev.02 patch. So I think I prefer that approach. We need to better name for "JdkThreadLocal", something to indicate that it holds resources or it notified when a thread terminates. Also along the way, we touched on exposing getIfPresent and we should look at that again. If it is expose then it fits well with the second approach too. -Alan From stuart.marks at oracle.com Wed Jun 6 21:02:26 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 6 Jun 2018 14:02:26 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> Message-ID: <34074732-06a4-cc54-d935-835edc69a558@oracle.com> On 6/6/18 10:58 AM, serguei.spitsyn at oracle.com wrote: > But the fix below is not clear to me: > > http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/src/hotspot/share/prims/jvmti.xml.udiff.html > > Stop Thread > > - Send the specified asynchronous exception to the specified thread > - (similar to java.lang.Thread.stop). > + Send the specified asynchronous exception to the specified thread. > Normally, this function is used to kill the specified thread with an > - instance of the exception ThreadDeath. > + instance of the exception ThreadDeath, similar to > + java.lang.Thread.stop. > > A reference to the java.lang.Thread.stop has been removed in one place and > added in another. > I thought, we wanted to get rid of these references in the spec. > Do I miss anything? Could you, explain this a little bit? Sure. The current state (through JDK 10) is that there two APIs: ??? 1. Thread.stop(Throwable) ??? 2. Thread.stop() // no-arg They are both deprecated, but only (1) is deprecated for removal, and it's the one being removed by this changeset. Method (2) will remain in the platform for the forseeable future. Method (1) causes the target thread asynchronously to throw the argument, which can be an instance of any subtype of Throwable. Method (2) causes the target thread to throw ThreadDeath (a subtype of Error) asynchronously. The wording in the JVMTI doc isn't terribly explicit. The original wording actually means "similar to Thread.stop(Throwable)" that is method (1). My rewording places the mention of Thread.stop into the context of throwing ThreadDeath, implying method (2). Since that will be the only Thread.stop method remaining, there's no ambiguity. So yes, the wording change looks odd, but there is a subtle shift in the meaning, and I think the meaning is clear after (1) has been removed. But I can remove the "similar to Thread.stop" bit if you prefer. s'marks > > Thanks, > Serguei > > On 6/5/18 17:05, Stuart Marks wrote: >> [adding serviceability-dev] >> >> Hi serviceability folks, >> >> I'm in the process of removing Thread.destroy() and Thread.stop(Throwable) >> from the Java SE API. Alan and David have pointed out that there are some >> cross-references to Thread.stop(Throwable) in the JDWP and JVMTI specs, as >> well as in the JDI ThreadReference API. I've adjusted the relevant files. >> >> See please review this updated webrev: >> >> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ >> >> For context, please see the full review thread on core-libs-dev: >> >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html >> >> >> >> On 6/4/18 11:11 PM, Alan Bateman wrote: >>> The source file that is used to generate the JDWP protocol code and the spec >>> is in make/data/jdwp/jdwp.spec. The JVM TI spec is at >>> src/hotspot/share/prims/jvmti.xml. It should be okay to skip those for this >>> change-set, assuming it is followed up quickly with another change-set to >>> update those specs. >> >> OK. I took a look at those other files and they seem simple enough to update >> as part of this changeset. If there aren't any objections from anyone, might >> as well get this all done at once. >> >> Thanks, >> >> s'marks > From roger.riggs at oracle.com Wed Jun 6 21:13:57 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Wed, 6 Jun 2018 17:13:57 -0400 Subject: RFR https://bugs.openjdk.java.net/browse/JDK-8204494 In-Reply-To: <5B183856.60108@oracle.com> References: <5B183856.60108@oracle.com> Message-ID: <8ae3eb45-3d64-e4c2-8de9-21e6df04445f@oracle.com> Hi Sherman, Looks fine Roger On 6/6/18 3:39 PM, Xueming Shen wrote: > Hi > > Please help review the change for JDK-8204494, a regression caused by > the fix for > JDK-8200530 [1]. It appears that fix fails to deal with the corner > case that a pair of > \r\n is at the boundary of the internal buf of > Manifest.FastInputtStream, which is > 8192 for the default. In that scenario, the previous change fails to > consume the trailing > \n, then triggers the Attributes parser fails to work on the > "continuation line" defined > by the jar spec. > > issue: https://bugs.openjdk.java.net/browse/JDK-8204494 > webrev: http://cr.openjdk.java.net/~sherman/8204494/webrev/ > > Fix has been verified by the regression reported in INTJDK-7627771. > > Thanks, > Sherman > > [1] issue: https://bugs.openjdk.java.net/browse/JDK-8200530 > ???? webrev: http://cr.openjdk.java.net/~sherman/8200530/webrev > From stuart.marks at oracle.com Wed Jun 6 21:29:58 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 6 Jun 2018 14:29:58 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <9d09f2b9-775d-4c95-b54b-37c426c62d3d@default> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <7f96f3ae-246e-8960-93fc-0e93aad5dd04@oracle.com> <9d09f2b9-775d-4c95-b54b-37c426c62d3d@default> Message-ID: <8e780019-dc81-9f6b-5bbc-06d2ffe234d1@oracle.com> Yeah, maybe it's better simply to remove the mentions of the methods that are being removed. I'll do so. I note that there are many other updates that could be done (the examples use applets!) but I think that's a task for another time. s'marks On 6/6/18 9:28 AM, Iris Clark wrote: > Hi, Stuart. > > I think you need to make changes to this file too: > > http://hg.openjdk.java.net/jdk/jdk/file/tip/src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html > > Thanks, > iris > > -----Original Message----- > From: Alan Bateman > Sent: Wednesday, June 6, 2018 3:40 AM > To: Stuart Marks ; serviceability-dev > Cc: core-libs-dev > Subject: Re: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) > > On 06/06/2018 01:05, Stuart Marks wrote: >> [adding serviceability-dev] >> >> Hi serviceability folks, >> >> I'm in the process of removing Thread.destroy() and >> Thread.stop(Throwable) from the Java SE API. Alan and David have >> pointed out that there are some cross-references to >> Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in the >> JDI ThreadReference API. I've adjusted the relevant files. >> >> See please review this updated webrev: >> >> ??? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ > Have you considered removing the "What about Thread.stop(Throwable)" > section completely from the threadPrimitiveDeprecation doc? A new developer reading the javadoc in 2019 shouldn't need to read about a removed method. > > Otherwise I think this looks good, including the updates to the JDWP and JVM TI specs. > > -Alan > From stuart.marks at oracle.com Wed Jun 6 21:58:13 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 6 Jun 2018 14:58:13 -0700 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: <3c10d04f-180f-a945-cf5e-f6e9089bb58e@oracle.com> References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <34074732-06a4-cc54-d935-835edc69a558@oracle.com> <3c10d04f-180f-a945-cf5e-f6e9089bb58e@oracle.com> Message-ID: Serguei, great! Thanks. All, I've updated the webrev to with changes to threadPrimitiveDeprecation.html to remove the sections that mention the removed methods. http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.2/ s'marks On 6/6/18 2:37 PM, serguei.spitsyn at oracle.com wrote: > Stuart, > > Thank you for explanation! > I'm Okay with the fix it is now. > > Thanks, > Serguei > > > On 6/6/18 14:02, Stuart Marks wrote: >> On 6/6/18 10:58 AM, serguei.spitsyn at oracle.com wrote: >>> But the fix below is not clear to me: >>> >>> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/src/hotspot/share/prims/jvmti.xml.udiff.html >>> >>> Stop Thread >>> >>> - Send the specified asynchronous exception to the specified thread >>> - (similar to java.lang.Thread.stop). >>> + Send the specified asynchronous exception to the specified thread. >>> Normally, this function is used to kill the specified thread with an >>> - instance of the exception ThreadDeath. >>> + instance of the exception ThreadDeath, similar to >>> + java.lang.Thread.stop. >>> >>> A reference to the java.lang.Thread.stop has been removed in one place and >>> added in another. >>> I thought, we wanted to get rid of these references in the spec. >>> Do I miss anything? Could you, explain this a little bit? >> Sure. The current state (through JDK 10) is that there two APIs: >> >> ??? 1. Thread.stop(Throwable) >> ??? 2. Thread.stop() // no-arg >> >> They are both deprecated, but only (1) is deprecated for removal, and it's >> the one being removed by this changeset. Method (2) will remain in the >> platform for the forseeable future. >> >> Method (1) causes the target thread asynchronously to throw the argument, >> which can be an instance of any subtype of Throwable. Method (2) causes the >> target thread to throw ThreadDeath (a subtype of Error) asynchronously. >> >> The wording in the JVMTI doc isn't terribly explicit. The original wording >> actually means "similar to Thread.stop(Throwable)" that is method (1). My >> rewording places the mention of Thread.stop into the context of throwing >> ThreadDeath, implying method (2). Since that will be the only Thread.stop >> method remaining, there's no ambiguity. So yes, the wording change looks odd, >> but there is a subtle shift in the meaning, and I think the meaning is clear >> after (1) has been removed. >> >> But I can remove the "similar to Thread.stop" bit if you prefer. >> >> s'marks >>> >>> Thanks, >>> Serguei >>> >>> On 6/5/18 17:05, Stuart Marks wrote: >>>> [adding serviceability-dev] >>>> >>>> Hi serviceability folks, >>>> >>>> I'm in the process of removing Thread.destroy() and Thread.stop(Throwable) >>>> from the Java SE API. Alan and David have pointed out that there are some >>>> cross-references to Thread.stop(Throwable) in the JDWP and JVMTI specs, as >>>> well as in the JDI ThreadReference API. I've adjusted the relevant files. >>>> >>>> See please review this updated webrev: >>>> >>>> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ >>>> >>>> For context, please see the full review thread on core-libs-dev: >>>> >>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html >>>> >>>> >>>> >>>> On 6/4/18 11:11 PM, Alan Bateman wrote: >>>>> The source file that is used to generate the JDWP protocol code and the >>>>> spec is in make/data/jdwp/jdwp.spec. The JVM TI spec is at >>>>> src/hotspot/share/prims/jvmti.xml. It should be okay to skip those for >>>>> this change-set, assuming it is followed up quickly with another >>>>> change-set to update those specs. >>>> >>>> OK. I took a look at those other files and they seem simple enough to >>>> update as part of this changeset. If there aren't any objections from >>>> anyone, might as well get this all done at once. >>>> >>>> Thanks, >>>> >>>> s'marks >>> >> > From iris.clark at oracle.com Wed Jun 6 22:18:02 2018 From: iris.clark at oracle.com (Iris Clark) Date: Wed, 6 Jun 2018 15:18:02 -0700 (PDT) Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <34074732-06a4-cc54-d935-835edc69a558@oracle.com> <3c10d04f-180f-a945-cf5e-f6e9089bb58e@oracle.com> Message-ID: <4a93f51f-01b9-489f-9e53-cb5bba47506c@default> Hi, Stuart. ? http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.2/ ? The simple update to remove references to the removed methods looks good. ? Thanks, iris ? From: Stuart Marks Sent: Wednesday, June 6, 2018 2:58 PM To: Serguei Spitsyn ; David Holmes ; Alan Bateman ; Iris Clark Cc: serviceability-dev ; core-libs-dev Subject: Re: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) ? Serguei, great! Thanks. All, I've updated the webrev to with changes to threadPrimitiveDeprecation.html to remove the sections that mention the removed methods. http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.2/ s'marks ? From david.holmes at oracle.com Thu Jun 7 02:13:32 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 7 Jun 2018 12:13:32 +1000 Subject: RFR(s): 8204243: remove Thread.destroy() and Thread.stop(Throwable) In-Reply-To: References: <66f6ea53-cebc-209a-b2da-d6786bf08b20@oracle.com> <3f353a92-cd58-eb4d-8898-07a4d7bc5836@oracle.com> <8cf772c8-4cef-ed90-8b85-402426ea340c@oracle.com> <77b8b7ec-270a-e311-9a95-9acde478ff85@oracle.com> <34074732-06a4-cc54-d935-835edc69a558@oracle.com> <3c10d04f-180f-a945-cf5e-f6e9089bb58e@oracle.com> Message-ID: On 7/06/2018 7:58 AM, Stuart Marks wrote: > Serguei, great! Thanks. > > All, I've updated the webrev to with changes to > threadPrimitiveDeprecation.html to remove the sections that mention the > removed methods. That reads much better! Thanks, David > http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.2/ > > s'marks > > > On 6/6/18 2:37 PM, serguei.spitsyn at oracle.com wrote: >> Stuart, >> >> Thank you for explanation! >> I'm Okay with the fix it is now. >> >> Thanks, >> Serguei >> >> >> On 6/6/18 14:02, Stuart Marks wrote: >>> On 6/6/18 10:58 AM, serguei.spitsyn at oracle.com wrote: >>>> But the fix below is not clear to me: >>>> >>>> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/src/hotspot/share/prims/jvmti.xml.udiff.html >>>> >>>> Stop Thread >>>> >>>> - Send the specified asynchronous exception to the specified thread >>>> - (similar to java.lang.Thread.stop). >>>> + Send the specified asynchronous exception to the specified thread. >>>> Normally, this function is used to kill the specified thread with an >>>> - instance of the exception ThreadDeath. >>>> + instance of the exception ThreadDeath, similar to >>>> + java.lang.Thread.stop. >>>> >>>> A reference to the java.lang.Thread.stop has been removed in one >>>> place and added in another. >>>> I thought, we wanted to get rid of these references in the spec. >>>> Do I miss anything? Could you, explain this a little bit? >>> Sure. The current state (through JDK 10) is that there two APIs: >>> >>> ??? 1. Thread.stop(Throwable) >>> ??? 2. Thread.stop() // no-arg >>> >>> They are both deprecated, but only (1) is deprecated for removal, and >>> it's the one being removed by this changeset. Method (2) will remain >>> in the platform for the forseeable future. >>> >>> Method (1) causes the target thread asynchronously to throw the >>> argument, which can be an instance of any subtype of Throwable. >>> Method (2) causes the target thread to throw ThreadDeath (a subtype >>> of Error) asynchronously. >>> >>> The wording in the JVMTI doc isn't terribly explicit. The original >>> wording actually means "similar to Thread.stop(Throwable)" that is >>> method (1). My rewording places the mention of Thread.stop into the >>> context of throwing ThreadDeath, implying method (2). Since that will >>> be the only Thread.stop method remaining, there's no ambiguity. So >>> yes, the wording change looks odd, but there is a subtle shift in the >>> meaning, and I think the meaning is clear after (1) has been removed. >>> >>> But I can remove the "similar to Thread.stop" bit if you prefer. >>> >>> s'marks >>>> >>>> Thanks, >>>> Serguei >>>> >>>> On 6/5/18 17:05, Stuart Marks wrote: >>>>> [adding serviceability-dev] >>>>> >>>>> Hi serviceability folks, >>>>> >>>>> I'm in the process of removing Thread.destroy() and >>>>> Thread.stop(Throwable) from the Java SE API. Alan and David have >>>>> pointed out that there are some cross-references to >>>>> Thread.stop(Throwable) in the JDWP and JVMTI specs, as well as in >>>>> the JDI ThreadReference API. I've adjusted the relevant files. >>>>> >>>>> See please review this updated webrev: >>>>> >>>>> http://cr.openjdk.java.net/~smarks/reviews/8204243/webrev.1/ >>>>> >>>>> For context, please see the full review thread on core-libs-dev: >>>>> >>>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html >>>>> >>>>> >>>>> >>>>> On 6/4/18 11:11 PM, Alan Bateman wrote: >>>>>> The source file that is used to generate the JDWP protocol code >>>>>> and the spec is in make/data/jdwp/jdwp.spec. The JVM TI spec is at >>>>>> src/hotspot/share/prims/jvmti.xml. It should be okay to skip those >>>>>> for this change-set, assuming it is followed up quickly with >>>>>> another change-set to update those specs. >>>>> >>>>> OK. I took a look at those other files and they seem simple enough >>>>> to update as part of this changeset. If there aren't any objections >>>>> from anyone, might as well get this all done at once. >>>>> >>>>> Thanks, >>>>> >>>>> s'marks >>>> >>> >> > From ivan.gerasimov at oracle.com Thu Jun 7 08:12:54 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 7 Jun 2018 01:12:54 -0700 Subject: RFR 8204310 : Simpler RandomAccessFile.setLength() on Windows In-Reply-To: <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> References: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> Message-ID: Hi Alan! On 6/6/18 6:57 AM, Alan Bateman wrote: > I think this should be okay but the Windows implementation has a long > history of biting the fingers of anyone that dares touch it. Sometimes > unexpected behavior changes only come to light long after the change. > The recent mails here about Kafka and sparse files is a good example > of that. So I think it's important to check the test coverage before > pushing this change. Specifically I think we need to check that we > have tests that > > 1. Exercise shrinking, extending, and not change the file length > > 2. Check getFilePointer when used with setLength. > > 3. Check how FileChannel behaves when created from a RandomAccessFile > but the file length is changed after the FileChannel is obtained. > > I realize this is more work than what you might have wanted to put > into this but we have to extremely careful here. > > -Alan > I agree it's a good idea to extend coverage in this area. I'll check existing tests and will add whatever is not yet there. With kind regards, Ivan From xu.y.yin at oracle.com Thu Jun 7 08:32:08 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Thu, 7 Jun 2018 16:32:08 +0800 Subject: [JDK 11] RFR 8201528: Add new test to check for package versioning information in OpenJDK Message-ID: Please review below new added test to check for package versioning information which customized in jar(s) manifest, many thanks bug: https://bugs.openjdk.java.net/browse/JDK-8201528 webrev: http://cr.openjdk.java.net/~xyin/8201528/webrev.00/ Regards, Chris From peter.levart at gmail.com Thu Jun 7 09:21:41 2018 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 7 Jun 2018 11:21:41 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: <160715cd-9d39-0c3f-21f7-f029ef4139c6@gmail.com> Hi Tony, Thanks for taking a look. Just a couple of comments inline... On 06/06/18 22:38, Tony Printezis wrote: >> >> - instead of overriding initialValue(), ThreadLocal.setInitialValue() >> calls TerminatingThreadLocal.register() at the right moment > > > Thanks. I much prefer not having to introduce calculateInitialValue(). > > One extra suggestion: Given you introduced a call to > TerminatingThreadLocal from ThreadLocal, would it make sense to maybe > do the same for set() and remove() (you?d have to add an appropriate > check in unregister) and not override them in TerminatingTreadLocal? I > totally get why you might not want to do that (an extra check when a > ThreadLocal is not the Terminating one). I?m OK either way. > Yes, precisely. My 1st version did just that, but since there are usages of ThreadLocal out there that are very performance sensitive, I opted for an approach where there is zero performance regression for non-TerminatingThreadLocal(s) when calling set() or remove(). A call-back from setInitialValue is not so problematic as it usually occurs just once per thread, but I can imagine a scenario where calls to get() and set() occur very rapidly. >> >> An alternative to "T getIfPresent()" is a "boolean isPresent()" >> method. Even if it remains package-private, with such method >> TerminatingThreadLocal could be implemented as: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ >> >> If TreadLocal.isPresent() was made public, the code could be further >> simplified. > > > Something like: > > if (tl.isPresent()) { > > ? ?T val = t.get(); > > ? ??. > > } > > will do two look-ups if the value exists. However, that?s better than > allocating unnecessarily. So, I?ll take isPresent() over not having a > way to check whether a value exists. > Right, and in our scenario, it is just isPresent() that is called for every termination of every thread (REGISTRY.isPresent()). .get() is then called only when there's actually something to do. > > Thumbs up from me. > Let's just wait for a day or two to see whether the discussion on concurrency-interest gives us any additional ideas. Currently I'm thinking of proposing the addition of isPresent() API. As far as this RFR is concerned, I'm consequently promoting the latest webrev.04. So any comments on that one? Alan? Regards, Peter From deepak.kejriwal at oracle.com Thu Jun 7 10:08:35 2018 From: deepak.kejriwal at oracle.com (Deepak Kejriwal) Date: Thu, 7 Jun 2018 03:08:35 -0700 (PDT) Subject: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries In-Reply-To: References: <1b0a8bde-4f32-4045-afbd-18897fbb572b@default> Message-ID: <163052d6-7682-40f9-9839-02b6324a2369@default> Hi Martin, Thanks for input. I have added the missing trailing newline to the test. Please find below updated webrev details:- Webrev: http://cr.openjdk.java.net/~rpatil/8186171/webrev.01/ Regards, Deepak From: Martin Buchholz Sent: Wednesday, June 6, 2018 11:33 PM To: Deepak Kejriwal Cc: Doug Lea ; core-libs-dev Subject: Re: RFR: JDK8U Backport of 8186171: HashMap: Entry.setValue may not work after Iterator.remove() called for previous entries On Wed, Jun 6, 2018 at 11:01 AM, Martin Buchholz wrote: Alright, I leave it to you.? Your current change is fine.?? Except, add that trailing newline to the test!? From matthias.baesken at sap.com Thu Jun 7 11:35:24 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 7 Jun 2018 11:35:24 +0000 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Message-ID: Hi, could you please review this small change that improves the error messages in matchJavaTZ . A reason of the failure is added to the message , and also the offset where the error happened . Bug : https://bugs.openjdk.java.net/browse/JDK-8204539 Webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ Thanks, Matthias From christoph.langer at sap.com Thu Jun 7 11:43:54 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 7 Jun 2018 11:43:54 +0000 Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 In-Reply-To: References: <0c73caec8810b9844a463343bdaec064@linux.vnet.ibm.com> <81453d0cbe05477ea558917de263ada2@sap.com> Message-ID: <91ad0c1765a44b90bed87ae0fb61ae09@sap.com> Hi Ichiroh, your proposal seems to make sense. I have created a bug for this: https://bugs.openjdk.java.net/browse/JDK-8204541 Can you please generate a webrev (referencing this bug, -c option of webrev.ksh) and mail it over to me. Then I'll upload it and you can post an official RFR mail. Best regards Christoph > -----Original Message----- > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > Sent: Dienstag, 5. Juni 2018 08:59 > To: Baesken, Matthias > Cc: Langer, Christoph ; 'build- > dev at openjdk.java.net' ; ppc-aix-port- > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > Goetz > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 > > Hello Matthias and Christoph. > Thank you for your explanations. > > I did not have enough knowledge about "visibility". > > I created following patches. > > ================================ > diff -r 02934b0d661b > src/java.base/share/native/libjimage/NativeImageBuffer.cpp > --- a/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Wed > May > 30 14:46:28 2018 +0200 > +++ b/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Tue Jun > 05 12:10:41 2018 +0900 > @@ -39,7 +39,9 @@ > #include "imageFile.hpp" > #include "inttypes.hpp" > #include "jimage.hpp" > +#if !defined(_AIX) > #include "osSupport.hpp" > +#endif > > #include "jdk_internal_jimage_NativeImageBuffer.h" > > diff -r 02934b0d661b src/java.base/unix/native/include/jni_md.h > --- a/src/java.base/unix/native/include/jni_md.h Wed May 30 14:46:28 > 2018 +0200 > +++ b/src/java.base/unix/native/include/jni_md.h Tue Jun 05 12:10:41 > 2018 +0900 > @@ -29,7 +29,8 @@ > #ifndef __has_attribute > #define __has_attribute(x) 0 > #endif > -#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) > +#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) \ > + || (defined(_AIX) && (defined(__xlC__) && (__xlC__ >= 0xD01))) > #ifdef ARM > #define JNIEXPORT > __attribute__((externally_visible,visibility("default"))) > #define JNIIMPORT > __attribute__((externally_visible,visibility("default"))) > ================================ > > If "osSupport.hpp" was included, XLC++ compiler complained. > I could not understand, which part was invalid... > I'm not sure, "osSupport.hpp" is really required on > NativeImageBuffer.cpp or not... > > I added __xlC__ macro to determine XLC/C++'s version# into jni_md.h. [1] > 0xD01 means 13.1 by hexadecimal. > > I checked symbol table by "dump -Tv -X64". [2] > It seemed it was fine, some symbols were hidden. > > Does someone review my code? > > [1] > https://www.ibm.com/support/knowledgecenter/en/SSGH2K_13.1.3/com.i > bm.xlc1313.aix.doc/compiler_ref/xlmacros.html > [2] > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > visibility/index.html > > On 2018-06-01 21:12, Baesken, Matthias wrote: > > Hi , my change 8202322 just handled the fact that the > > visibility - flags are not supported with xlc 12.1 , so setting > > them generated a TON of compile - time warnings . > > > > The introduction of the "-qvisibility=hidden" came with the > > mapfile removal changes : > > > > 8200358: Remove mapfiles for JDK executables > > http://hg.openjdk.java.net/jdk/jdk/rev/210cf224b690 > > > > 8200178: Remove mapfiles for JDK native libraries > > http://hg.openjdk.java.net/jdk/jdk/rev/396ea30afbd5 > > > > I guess it might need further testing+adjustments to make the > > "visibility hiding" work nicely with XLC13 , but currently we > > build only with XLC12 . > > > > As a workaround you might want to remove the "-qvisibility=hidden" > > setting for XLC 13 as well , like I did for XLC12 with the change > > 8202322 . > > > > > > Best regards, Matthias > > > > > > > > > >> -----Original Message----- > >> From: Langer, Christoph > >> Sent: Freitag, 1. Juni 2018 10:57 > >> To: Ichiroh Takiguchi > >> Cc: Baesken, Matthias ; 'build- > >> dev at openjdk.java.net' ; ppc-aix-port- > >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > >> Goetz > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > >> on xlc 12.1 > >> > >> Hi Ichiroh, > >> > >> we do not use the XLC 13 compiler on AIX yet here at SAP and I believe > >> nobody of my colleagues has played with it yet. So you are on a new > >> playground here ?? > >> > >> However, I believe the idea in OpenJDK with the abolition of map files > >> is that > >> symbols should be invisible externally unless they are declared > >> exported, > >> e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the > >> correct > >> default and whatever JNIEXPORT expands to should contain the right > >> attributes to get that symbol visible. > >> > >> Can you check if either my assumption is completely wrong, JNIEXPORT > >> does > >> not expand to the right thing, XLC 13 has a bug or maybe just sume > >> specific > >> required symbols are not declared correctly? > >> > >> Best regards > >> Christoph > >> > >> > -----Original Message----- > >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > >> > Sent: Donnerstag, 31. Mai 2018 09:55 > >> > To: Langer, Christoph > >> > Cc: Baesken, Matthias ; 'build- > >> > dev at openjdk.java.net' ; ppc-aix-port- > >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > >> > Goetz > >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc > 12.1 > >> > > >> > Hello. > >> > 8202322 was integrated into jdk-11+15. > >> > I'm using XLC 13.1.3 on AIX 7.1.4. > >> > Build was failed because of "-qvisibility=hidden" on > >> > make/lib/LibCommon.gmk. > >> > According to "XL C/C++ for AIX 13.1.3" documentation [1], > >> > "-qvisibility=hidden" cannot create shared libraries entry points. > >> > For example, libverify.so was there, but entry points were not resolved > >> > by "-lverify" option. > >> > I think it should be "-qvisibility=default" (I tried, it worked) > >> > or "-qvisibility=protected" (I had not tried) ? > >> > I'm not familiar with -qvisibility option, but I'd like to find out > >> > right way. > >> > > >> > [1] > >> > > >> > https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm. > >> > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html > >> > > >> > On 2018-05-16 16:08, Langer, Christoph wrote: > >> > > Hi Matthias, > >> > > > >> > > yes, reviewed. > >> > > > >> > > Best regards > >> > > Christoph > >> > > > >> > > From: Baesken, Matthias > >> > > Sent: Mittwoch, 16. Mai 2018 09:06 > >> > > To: Langer, Christoph ; > >> > > 'build-dev at openjdk.java.net' ; > >> > > ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net > >> > > Cc: Lindenmaier, Goetz > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > >> > > xlc 12.1 > >> > > > >> > > Hi Christoph can I add you as second reviewer (other reviewer was > >> > > Erik Joelsson) can push the change ? > >> > > > >> > > Best regards, Matthias > >> > > > >> > > > >> > > > >> > > From: Langer, Christoph > >> > > Sent: Donnerstag, 26. April 2018 16:38 > >> > > To: Baesken, Matthias > >> > > >; > >> > > 'build-dev at openjdk.java.net' > >> > > dev at openjdk.java.net>>; > >> > > ppc-aix-port-dev at openjdk.java.net >> > dev at openjdk.java.net>; > >> > > core-libs-dev at openjdk.java.net >> dev at openjdk.java.net> > >> > > Cc: Simonis, Volker > >> > > > > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > >> > > xlc 12.1 > >> > > > >> > > Hi Matthias, > >> > > > >> > > to me the change in principal looks good. > >> > > > >> > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. > >> > > extract major number before the first dot, then compare numerically) > - > >> > > but maybe it is too complicated and the current single version > compare > >> > > suits our needs ? > >> > > > >> > > Best regards > >> > > Christoph > >> > > > >> > > From: Baesken, Matthias > >> > > Sent: Donnerstag, 26. April 2018 16:14 > >> > > To: 'build-dev at openjdk.java.net' > >> > > dev at openjdk.java.net>>; > >> > > ppc-aix-port-dev at openjdk.java.net >> > dev at openjdk.java.net>; > >> > > core-libs-dev at openjdk.java.net >> dev at openjdk.java.net> > >> > > Cc: Langer, Christoph > >> > > >; > >> Simonis, > >> > > Volker > > >> > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc > >> > > 12.1 > >> > > > >> > > Hello , could you please review this small adjustment to the symbol > >> > > visibility compilation settings on AIX ? > >> > > Currently we use XLC 12.1 to compile JDK on AIX . > >> > > > >> > > However XLC 12.1 does not support the "-qvisibility=hidden" > >> > > setting currently set on AIX. > >> > > It was introduced with XLC 13.1 . Christoph found some info about it > >> > > here : > >> > > > >> > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > >> > visibility-part2/index.html > >> > > > >> > > Setting it only generates hundreds of warnings in the build log , > >> > > warnings look like this : > >> > > XlC12.1 > >> > > > >> > > bash-4.4\$ xlC -qversion > >> > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > >> > > Version: 12.01.0000.0019 > >> > > > >> > > bash-4.4\$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list > >> > > of valid options. > >> > > > >> > > Compare to XLC13.1 > >> > > > >> > > bash-3.00\$ xlC -qversion > >> > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > >> > > Version: 13.01.0000.0008 > >> > > bash-3.00\$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > >> > > bash-3.00\$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> > > > >> > > > >> > > So it is better to avoid setting these flags when using xlc12.1 . > >> > > Please review : > >> > > > >> > > Bug : > >> > > > >> > > https://bugs.openjdk.java.net/browse/JDK-8202322 > >> > > > >> > > Change : > >> > > > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > >> > > > >> > > > >> > > Best regards, Matthias From christoph.langer at sap.com Thu Jun 7 11:51:36 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 7 Jun 2018 11:51:36 +0000 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: References: Message-ID: <67fa98e2ed454a208af23a7a56e4627b@sap.com> Hi Matthias, in line 527, where the actual output is done, I think you would need to replace variable 'message' with 'outputMessage', otherwise I guess it won't compile?? Also, mapFileName can't be used at this place, because it has already been free'ed there. But in general the additions make sense and will make it easier to find issues in the tzmappings file. Best regards Christoph From: Baesken, Matthias Sent: Donnerstag, 7. Juni 2018 13:35 To: core-libs-dev at openjdk.java.net Cc: Langer, Christoph ; Lindenmaier, Goetz Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi, could you please review this small change that improves the error messages in matchJavaTZ . A reason of the failure is added to the message , and also the offset where the error happened . Bug : https://bugs.openjdk.java.net/browse/JDK-8204539 Webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ Thanks, Matthias From matthias.baesken at sap.com Thu Jun 7 12:03:01 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 7 Jun 2018 12:03:01 +0000 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: <67fa98e2ed454a208af23a7a56e4627b@sap.com> References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> Message-ID: Hi Christoph you are correct I compiled it on the wrong platforms , stupid mistake ! Will send an updated web rev . * Also, mapFileName can't be used at this place, because it has already been free'ed there. Yes, makes sense. Will move the free-calls . Best regards, Matthias From: Langer, Christoph Sent: Donnerstag, 7. Juni 2018 13:52 To: Baesken, Matthias ; core-libs-dev at openjdk.java.net Cc: Lindenmaier, Goetz Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi Matthias, in line 527, where the actual output is done, I think you would need to replace variable 'message' with 'outputMessage', otherwise I guess it won't compile?? Also, mapFileName can't be used at this place, because it has already been free'ed there. But in general the additions make sense and will make it easier to find issues in the tzmappings file. Best regards Christoph From: Baesken, Matthias Sent: Donnerstag, 7. Juni 2018 13:35 To: core-libs-dev at openjdk.java.net Cc: Langer, Christoph >; Lindenmaier, Goetz > Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi, could you please review this small change that improves the error messages in matchJavaTZ . A reason of the failure is added to the message , and also the offset where the error happened . Bug : https://bugs.openjdk.java.net/browse/JDK-8204539 Webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ Thanks, Matthias From sean.coffey at oracle.com Thu Jun 7 12:30:10 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Thu, 7 Jun 2018 13:30:10 +0100 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> Message-ID: <00ad2878-3f6c-4f5e-5529-baf76cfc275a@oracle.com> On 07/06/2018 13:03, Baesken, Matthias wrote: > Hi Christoph you are correct I compiled it on the wrong platforms , stupid mistake ! > Will send an updated web rev . > > > * Also, mapFileName can't be used at this place, because it has already been free'ed there. > > Yes, makes sense. Will move the free-calls . Please make sure you run some manual tests also to ensure the message looks correct. Not sure if you really need mapFileName to be printed - it's generally a static file within java.home regards, Sean. > > Best regards, Matthias > > From: Langer, Christoph > Sent: Donnerstag, 7. Juni 2018 13:52 > To: Baesken, Matthias ; core-libs-dev at openjdk.java.net > Cc: Lindenmaier, Goetz > Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ [windows] > > Hi Matthias, > > in line 527, where the actual output is done, I think you would need to replace variable 'message' with 'outputMessage', otherwise I guess it won't compile?? > > Also, mapFileName can't be used at this place, because it has already been free'ed there. > > But in general the additions make sense and will make it easier to find issues in the tzmappings file. > > Best regards > Christoph > > From: Baesken, Matthias > Sent: Donnerstag, 7. Juni 2018 13:35 > To: core-libs-dev at openjdk.java.net > Cc: Langer, Christoph >; Lindenmaier, Goetz > > Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] > > Hi, could you please review this small change that improves the error messages in matchJavaTZ . > A reason of the failure is added to the message , and also the offset where the error happened . > > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8204539 > > > Webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ > > > Thanks, Matthias From takiguc at linux.vnet.ibm.com Thu Jun 7 12:53:06 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Thu, 07 Jun 2018 21:53:06 +0900 Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags Message-ID: <5928b735dc946ef140d3ccdf3852dc1f@linux.vnet.ibm.com> Hello. Could you review it ? Bug: https://bugs.openjdk.java.net/browse/JDK-8204541 Change: http://cr.openjdk.java.net/~clanger/webrevs/8204541.0/ Thanks, Ichiroh Takiguchi IBM Japan, Ltd. -------- Original Message -------- Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc 12.1 Date: 2018-06-07 20:43 From: "Langer, Christoph" To: Ichiroh Takiguchi Cc: "'build-dev at openjdk.java.net'" , "ppc-aix-port-dev at openjdk.java.net" , "core-libs-dev at openjdk.java.net" , "Lindenmaier, Goetz" , "Baesken, Matthias" Hi Ichiroh, your proposal seems to make sense. I have created a bug for this: https://bugs.openjdk.java.net/browse/JDK-8204541 Can you please generate a webrev (referencing this bug, -c option of webrev.ksh) and mail it over to me. Then I'll upload it and you can post an official RFR mail. Best regards Christoph > -----Original Message----- > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > Sent: Dienstag, 5. Juni 2018 08:59 > To: Baesken, Matthias > Cc: Langer, Christoph ; 'build- > dev at openjdk.java.net' ; ppc-aix-port- > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > Goetz > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > xlc 12.1 > > Hello Matthias and Christoph. > Thank you for your explanations. > > I did not have enough knowledge about "visibility". > > I created following patches. > > ================================ > diff -r 02934b0d661b > src/java.base/share/native/libjimage/NativeImageBuffer.cpp > --- a/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Wed > May > 30 14:46:28 2018 +0200 > +++ b/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Tue > Jun > 05 12:10:41 2018 +0900 > @@ -39,7 +39,9 @@ > #include "imageFile.hpp" > #include "inttypes.hpp" > #include "jimage.hpp" > +#if !defined(_AIX) > #include "osSupport.hpp" > +#endif > > #include "jdk_internal_jimage_NativeImageBuffer.h" > > diff -r 02934b0d661b src/java.base/unix/native/include/jni_md.h > --- a/src/java.base/unix/native/include/jni_md.h Wed May 30 14:46:28 > 2018 +0200 > +++ b/src/java.base/unix/native/include/jni_md.h Tue Jun 05 12:10:41 > 2018 +0900 > @@ -29,7 +29,8 @@ > #ifndef __has_attribute > #define __has_attribute(x) 0 > #endif > -#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) > +#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) \ > + || (defined(_AIX) && (defined(__xlC__) && (__xlC__ >= 0xD01))) > #ifdef ARM > #define JNIEXPORT > __attribute__((externally_visible,visibility("default"))) > #define JNIIMPORT > __attribute__((externally_visible,visibility("default"))) > ================================ > > If "osSupport.hpp" was included, XLC++ compiler complained. > I could not understand, which part was invalid... > I'm not sure, "osSupport.hpp" is really required on > NativeImageBuffer.cpp or not... > > I added __xlC__ macro to determine XLC/C++'s version# into jni_md.h. > [1] > 0xD01 means 13.1 by hexadecimal. > > I checked symbol table by "dump -Tv -X64". [2] > It seemed it was fine, some symbols were hidden. > > Does someone review my code? > > [1] > https://www.ibm.com/support/knowledgecenter/en/SSGH2K_13.1.3/com.i > bm.xlc1313.aix.doc/compiler_ref/xlmacros.html > [2] > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > visibility/index.html > > On 2018-06-01 21:12, Baesken, Matthias wrote: > > Hi , my change 8202322 just handled the fact that the > > visibility - flags are not supported with xlc 12.1 , so setting > > them generated a TON of compile - time warnings . > > > > The introduction of the "-qvisibility=hidden" came with the > > mapfile removal changes : > > > > 8200358: Remove mapfiles for JDK executables > > http://hg.openjdk.java.net/jdk/jdk/rev/210cf224b690 > > > > 8200178: Remove mapfiles for JDK native libraries > > http://hg.openjdk.java.net/jdk/jdk/rev/396ea30afbd5 > > > > I guess it might need further testing+adjustments to make the > > "visibility hiding" work nicely with XLC13 , but currently we > > build only with XLC12 . > > > > As a workaround you might want to remove the "-qvisibility=hidden" > > setting for XLC 13 as well , like I did for XLC12 with the change > > 8202322 . > > > > > > Best regards, Matthias > > > > > > > > > >> -----Original Message----- > >> From: Langer, Christoph > >> Sent: Freitag, 1. Juni 2018 10:57 > >> To: Ichiroh Takiguchi > >> Cc: Baesken, Matthias ; 'build- > >> dev at openjdk.java.net' ; ppc-aix-port- > >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > >> Goetz > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > >> on xlc 12.1 > >> > >> Hi Ichiroh, > >> > >> we do not use the XLC 13 compiler on AIX yet here at SAP and I believe > >> nobody of my colleagues has played with it yet. So you are on a new > >> playground here ?? > >> > >> However, I believe the idea in OpenJDK with the abolition of map files > >> is that > >> symbols should be invisible externally unless they are declared > >> exported, > >> e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the > >> correct > >> default and whatever JNIEXPORT expands to should contain the right > >> attributes to get that symbol visible. > >> > >> Can you check if either my assumption is completely wrong, JNIEXPORT > >> does > >> not expand to the right thing, XLC 13 has a bug or maybe just sume > >> specific > >> required symbols are not declared correctly? > >> > >> Best regards > >> Christoph > >> > >> > -----Original Message----- > >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > >> > Sent: Donnerstag, 31. Mai 2018 09:55 > >> > To: Langer, Christoph > >> > Cc: Baesken, Matthias ; 'build- > >> > dev at openjdk.java.net' ; ppc-aix-port- > >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > >> > Goetz > >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on xlc > 12.1 > >> > > >> > Hello. > >> > 8202322 was integrated into jdk-11+15. > >> > I'm using XLC 13.1.3 on AIX 7.1.4. > >> > Build was failed because of "-qvisibility=hidden" on > >> > make/lib/LibCommon.gmk. > >> > According to "XL C/C++ for AIX 13.1.3" documentation [1], > >> > "-qvisibility=hidden" cannot create shared libraries entry points. > >> > For example, libverify.so was there, but entry points were not resolved > >> > by "-lverify" option. > >> > I think it should be "-qvisibility=default" (I tried, it worked) > >> > or "-qvisibility=protected" (I had not tried) ? > >> > I'm not familiar with -qvisibility option, but I'd like to find out > >> > right way. > >> > > >> > [1] > >> > > >> > https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm. > >> > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html > >> > > >> > On 2018-05-16 16:08, Langer, Christoph wrote: > >> > > Hi Matthias, > >> > > > >> > > yes, reviewed. > >> > > > >> > > Best regards > >> > > Christoph > >> > > > >> > > From: Baesken, Matthias > >> > > Sent: Mittwoch, 16. Mai 2018 09:06 > >> > > To: Langer, Christoph ; > >> > > 'build-dev at openjdk.java.net' ; > >> > > ppc-aix-port-dev at openjdk.java.net; core-libs-dev at openjdk.java.net > >> > > Cc: Lindenmaier, Goetz > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > >> > > xlc 12.1 > >> > > > >> > > Hi Christoph can I add you as second reviewer (other reviewer was > >> > > Erik Joelsson) can push the change ? > >> > > > >> > > Best regards, Matthias > >> > > > >> > > > >> > > > >> > > From: Langer, Christoph > >> > > Sent: Donnerstag, 26. April 2018 16:38 > >> > > To: Baesken, Matthias > >> > > >; > >> > > 'build-dev at openjdk.java.net' > >> > > dev at openjdk.java.net>>; > >> > > ppc-aix-port-dev at openjdk.java.net >> > dev at openjdk.java.net>; > >> > > core-libs-dev at openjdk.java.net >> dev at openjdk.java.net> > >> > > Cc: Simonis, Volker > >> > > > > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > >> > > xlc 12.1 > >> > > > >> > > Hi Matthias, > >> > > > >> > > to me the change in principal looks good. > >> > > > >> > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. > >> > > extract major number before the first dot, then compare numerically) > - > >> > > but maybe it is too complicated and the current single version > compare > >> > > suits our needs ? > >> > > > >> > > Best regards > >> > > Christoph > >> > > > >> > > From: Baesken, Matthias > >> > > Sent: Donnerstag, 26. April 2018 16:14 > >> > > To: 'build-dev at openjdk.java.net' > >> > > dev at openjdk.java.net>>; > >> > > ppc-aix-port-dev at openjdk.java.net >> > dev at openjdk.java.net>; > >> > > core-libs-dev at openjdk.java.net >> dev at openjdk.java.net> > >> > > Cc: Langer, Christoph > >> > > >; > >> Simonis, > >> > > Volker > > >> > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc > >> > > 12.1 > >> > > > >> > > Hello , could you please review this small adjustment to the symbol > >> > > visibility compilation settings on AIX ? > >> > > Currently we use XLC 12.1 to compile JDK on AIX . > >> > > > >> > > However XLC 12.1 does not support the "-qvisibility=hidden" > >> > > setting currently set on AIX. > >> > > It was introduced with XLC 13.1 . Christoph found some info about it > >> > > here : > >> > > > >> > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > >> > visibility-part2/index.html > >> > > > >> > > Setting it only generates hundreds of warnings in the build log , > >> > > warnings look like this : > >> > > XlC12.1 > >> > > > >> > > bash-4.4\$ xlC -qversion > >> > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > >> > > Version: 12.01.0000.0019 > >> > > > >> > > bash-4.4\$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list > >> > > of valid options. > >> > > > >> > > Compare to XLC13.1 > >> > > > >> > > bash-3.00\$ xlC -qversion > >> > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > >> > > Version: 13.01.0000.0008 > >> > > bash-3.00\$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > >> > > bash-3.00\$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> > > > >> > > > >> > > So it is better to avoid setting these flags when using xlc12.1 . > >> > > Please review : > >> > > > >> > > Bug : > >> > > > >> > > https://bugs.openjdk.java.net/browse/JDK-8202322 > >> > > > >> > > Change : > >> > > > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > >> > > > >> > > > >> > > Best regards, Matthias From christoph.langer at sap.com Thu Jun 7 13:06:42 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 7 Jun 2018 13:06:42 +0000 Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags In-Reply-To: <5928b735dc946ef140d3ccdf3852dc1f@linux.vnet.ibm.com> References: <5928b735dc946ef140d3ccdf3852dc1f@linux.vnet.ibm.com> Message-ID: Hi Ichiroh, what's the exact error message if you #include "osSupport.hpp"? (I have no xlC 13 at hand to try myself...) Best regards Christoph > -----Original Message----- > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > Sent: Donnerstag, 7. Juni 2018 14:53 > To: build-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; core- > libs-dev at openjdk.java.net > Cc: Lindenmaier, Goetz ; Baesken, Matthias > ; Langer, Christoph > > Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags > > Hello. > > Could you review it ? > Bug: https://bugs.openjdk.java.net/browse/JDK-8204541 > Change: http://cr.openjdk.java.net/~clanger/webrevs/8204541.0/ > > Thanks, > Ichiroh Takiguchi > IBM Japan, Ltd. > > -------- Original Message -------- > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > xlc 12.1 > Date: 2018-06-07 20:43 > From: "Langer, Christoph" > To: Ichiroh Takiguchi > Cc: "'build-dev at openjdk.java.net'" , > "ppc-aix-port-dev at openjdk.java.net" dev at openjdk.java.net>, > "core-libs-dev at openjdk.java.net" , > "Lindenmaier, Goetz" , "Baesken, Matthias" > > > Hi Ichiroh, > > your proposal seems to make sense. I have created a bug for this: > https://bugs.openjdk.java.net/browse/JDK-8204541 > > Can you please generate a webrev (referencing this bug, -c option of > webrev.ksh) and mail it over to me. Then I'll upload it and you can post > an official RFR mail. > > Best regards > Christoph > > > -----Original Message----- > > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > > Sent: Dienstag, 5. Juni 2018 08:59 > > To: Baesken, Matthias > > Cc: Langer, Christoph ; 'build- > > dev at openjdk.java.net' ; ppc-aix-port- > > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > > Goetz > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > > xlc 12.1 > > > > Hello Matthias and Christoph. > > Thank you for your explanations. > > > > I did not have enough knowledge about "visibility". > > > > I created following patches. > > > > ================================ > > diff -r 02934b0d661b > > src/java.base/share/native/libjimage/NativeImageBuffer.cpp > > --- a/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Wed > > May > > 30 14:46:28 2018 +0200 > > +++ b/src/java.base/share/native/libjimage/NativeImageBuffer.cpp > Tue > > Jun > > 05 12:10:41 2018 +0900 > > @@ -39,7 +39,9 @@ > > #include "imageFile.hpp" > > #include "inttypes.hpp" > > #include "jimage.hpp" > > +#if !defined(_AIX) > > #include "osSupport.hpp" > > +#endif > > > > #include "jdk_internal_jimage_NativeImageBuffer.h" > > > > diff -r 02934b0d661b src/java.base/unix/native/include/jni_md.h > > --- a/src/java.base/unix/native/include/jni_md.h Wed May 30 14:46:28 > > 2018 +0200 > > +++ b/src/java.base/unix/native/include/jni_md.h Tue Jun 05 12:10:41 > > 2018 +0900 > > @@ -29,7 +29,8 @@ > > #ifndef __has_attribute > > #define __has_attribute(x) 0 > > #endif > > -#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && > > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) > > +#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && > > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) \ > > + || (defined(_AIX) && (defined(__xlC__) && (__xlC__ >= 0xD01))) > > #ifdef ARM > > #define JNIEXPORT > > __attribute__((externally_visible,visibility("default"))) > > #define JNIIMPORT > > __attribute__((externally_visible,visibility("default"))) > > ================================ > > > > If "osSupport.hpp" was included, XLC++ compiler complained. > > I could not understand, which part was invalid... > > I'm not sure, "osSupport.hpp" is really required on > > NativeImageBuffer.cpp or not... > > > > I added __xlC__ macro to determine XLC/C++'s version# into jni_md.h. > > [1] > > 0xD01 means 13.1 by hexadecimal. > > > > I checked symbol table by "dump -Tv -X64". [2] > > It seemed it was fine, some symbols were hidden. > > > > Does someone review my code? > > > > [1] > > > https://www.ibm.com/support/knowledgecenter/en/SSGH2K_13.1.3/com.i > > bm.xlc1313.aix.doc/compiler_ref/xlmacros.html > > [2] > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > > visibility/index.html > > > > On 2018-06-01 21:12, Baesken, Matthias wrote: > > > Hi , my change 8202322 just handled the fact that the > > > visibility - flags are not supported with xlc 12.1 , so setting > > > them generated a TON of compile - time warnings . > > > > > > The introduction of the "-qvisibility=hidden" came with the > > > mapfile removal changes : > > > > > > 8200358: Remove mapfiles for JDK executables > > > http://hg.openjdk.java.net/jdk/jdk/rev/210cf224b690 > > > > > > 8200178: Remove mapfiles for JDK native libraries > > > http://hg.openjdk.java.net/jdk/jdk/rev/396ea30afbd5 > > > > > > I guess it might need further testing+adjustments to make the > > > "visibility hiding" work nicely with XLC13 , but currently we > > > build only with XLC12 . > > > > > > As a workaround you might want to remove the "-qvisibility=hidden" > > > setting for XLC 13 as well , like I did for XLC12 with the change > > > 8202322 . > > > > > > > > > Best regards, Matthias > > > > > > > > > > > > > > >> -----Original Message----- > > >> From: Langer, Christoph > > >> Sent: Freitag, 1. Juni 2018 10:57 > > >> To: Ichiroh Takiguchi > > >> Cc: Baesken, Matthias ; 'build- > > >> dev at openjdk.java.net' ; ppc-aix-port- > > >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > > >> Goetz > > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > > >> on xlc 12.1 > > >> > > >> Hi Ichiroh, > > >> > > >> we do not use the XLC 13 compiler on AIX yet here at SAP and I believe > > >> nobody of my colleagues has played with it yet. So you are on a new > > >> playground here ?? > > >> > > >> However, I believe the idea in OpenJDK with the abolition of map files > > >> is that > > >> symbols should be invisible externally unless they are declared > > >> exported, > > >> e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the > > >> correct > > >> default and whatever JNIEXPORT expands to should contain the right > > >> attributes to get that symbol visible. > > >> > > >> Can you check if either my assumption is completely wrong, JNIEXPORT > > >> does > > >> not expand to the right thing, XLC 13 has a bug or maybe just sume > > >> specific > > >> required symbols are not declared correctly? > > >> > > >> Best regards > > >> Christoph > > >> > > >> > -----Original Message----- > > >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > > >> > Sent: Donnerstag, 31. Mai 2018 09:55 > > >> > To: Langer, Christoph > > >> > Cc: Baesken, Matthias ; 'build- > > >> > dev at openjdk.java.net' ; ppc-aix-port- > > >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; > Lindenmaier, > > >> > Goetz > > >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > xlc > > 12.1 > > >> > > > >> > Hello. > > >> > 8202322 was integrated into jdk-11+15. > > >> > I'm using XLC 13.1.3 on AIX 7.1.4. > > >> > Build was failed because of "-qvisibility=hidden" on > > >> > make/lib/LibCommon.gmk. > > >> > According to "XL C/C++ for AIX 13.1.3" documentation [1], > > >> > "-qvisibility=hidden" cannot create shared libraries entry points. > > >> > For example, libverify.so was there, but entry points were not > resolved > > >> > by "-lverify" option. > > >> > I think it should be "-qvisibility=default" (I tried, it worked) > > >> > or "-qvisibility=protected" (I had not tried) ? > > >> > I'm not familiar with -qvisibility option, but I'd like to find out > > >> > right way. > > >> > > > >> > [1] > > >> > > > >> > > > https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm. > > >> > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html > > >> > > > >> > On 2018-05-16 16:08, Langer, Christoph wrote: > > >> > > Hi Matthias, > > >> > > > > >> > > yes, reviewed. > > >> > > > > >> > > Best regards > > >> > > Christoph > > >> > > > > >> > > From: Baesken, Matthias > > >> > > Sent: Mittwoch, 16. Mai 2018 09:06 > > >> > > To: Langer, Christoph ; > > >> > > 'build-dev at openjdk.java.net' ; > > >> > > ppc-aix-port-dev at openjdk.java.net; core-libs- > dev at openjdk.java.net > > >> > > Cc: Lindenmaier, Goetz > > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > > >> > > xlc 12.1 > > >> > > > > >> > > Hi Christoph can I add you as second reviewer (other reviewer was > > >> > > Erik Joelsson) can push the change ? > > >> > > > > >> > > Best regards, Matthias > > >> > > > > >> > > > > >> > > > > >> > > From: Langer, Christoph > > >> > > Sent: Donnerstag, 26. April 2018 16:38 > > >> > > To: Baesken, Matthias > > >> > > > >; > > >> > > 'build-dev at openjdk.java.net' > > >> > > > dev at openjdk.java.net>>; > > >> > > ppc-aix-port-dev at openjdk.java.net > >> > dev at openjdk.java.net>; > > >> > > core-libs-dev at openjdk.java.net > >> dev at openjdk.java.net> > > >> > > Cc: Simonis, Volker > > >> > > > > > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > > >> > > xlc 12.1 > > >> > > > > >> > > Hi Matthias, > > >> > > > > >> > > to me the change in principal looks good. > > >> > > > > >> > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. > > >> > > extract major number before the first dot, then compare > numerically) > > - > > >> > > but maybe it is too complicated and the current single version > > compare > > >> > > suits our needs ? > > >> > > > > >> > > Best regards > > >> > > Christoph > > >> > > > > >> > > From: Baesken, Matthias > > >> > > Sent: Donnerstag, 26. April 2018 16:14 > > >> > > To: 'build-dev at openjdk.java.net' > > >> > > > dev at openjdk.java.net>>; > > >> > > ppc-aix-port-dev at openjdk.java.net > >> > dev at openjdk.java.net>; > > >> > > core-libs-dev at openjdk.java.net > >> dev at openjdk.java.net> > > >> > > Cc: Langer, Christoph > > >> > > >; > > >> Simonis, > > >> > > Volker > > > > >> > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc > > >> > > 12.1 > > >> > > > > >> > > Hello , could you please review this small adjustment to the symbol > > >> > > visibility compilation settings on AIX ? > > >> > > Currently we use XLC 12.1 to compile JDK on AIX . > > >> > > > > >> > > However XLC 12.1 does not support the "-qvisibility=hidden" > > >> > > setting currently set on AIX. > > >> > > It was introduced with XLC 13.1 . Christoph found some info about it > > >> > > here : > > >> > > > > >> > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > > >> > visibility-part2/index.html > > >> > > > > >> > > Setting it only generates hundreds of warnings in the build log , > > >> > > warnings look like this : > > >> > > XlC12.1 > > >> > > > > >> > > bash-4.4\$ xlC -qversion > > >> > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > > >> > > Version: 12.01.0000.0019 > > >> > > > > >> > > bash-4.4\$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > > >> > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list > > >> > > of valid options. > > >> > > > > >> > > Compare to XLC13.1 > > >> > > > > >> > > bash-3.00\$ xlC -qversion > > >> > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > > >> > > Version: 13.01.0000.0008 > > >> > > bash-3.00\$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > > >> > > bash-3.00\$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > > >> > > > > >> > > > > >> > > So it is better to avoid setting these flags when using xlc12.1 . > > >> > > Please review : > > >> > > > > >> > > Bug : > > >> > > > > >> > > https://bugs.openjdk.java.net/browse/JDK-8202322 > > >> > > > > >> > > Change : > > >> > > > > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > > >> > > > > >> > > > > >> > > Best regards, Matthias From srinivas.dama at oracle.com Thu Jun 7 14:01:38 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Thu, 7 Jun 2018 07:01:38 -0700 (PDT) Subject: RFR: 8196990 :Resolve disabled warnings for libjli Message-ID: Hi, Please review http://cr.openjdk.java.net/~sdama/8196990/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8196990 Regards, Srinivas From tprintezis at twitter.com Thu Jun 7 14:07:25 2018 From: tprintezis at twitter.com (Tony Printezis) Date: Thu, 7 Jun 2018 07:07:25 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <160715cd-9d39-0c3f-21f7-f029ef4139c6@gmail.com> References: <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> <160715cd-9d39-0c3f-21f7-f029ef4139c6@gmail.com> Message-ID: Peter, Inline. ????? Tony Printezis | @TonyPrintezis | tprintezis at twitter.com On June 7, 2018 at 5:21:43 AM, Peter Levart (peter.levart at gmail.com) wrote: Hi Tony, Thanks for taking a look. Just a couple of comments inline... On 06/06/18 22:38, Tony Printezis wrote: - instead of overriding initialValue(), ThreadLocal.setInitialValue() calls TerminatingThreadLocal.register() at the right moment Thanks. I much prefer not having to introduce calculateInitialValue(). One extra suggestion: Given you introduced a call to TerminatingThreadLocal from ThreadLocal, would it make sense to maybe do the same for set() and remove() (you?d have to add an appropriate check in unregister) and not override them in TerminatingTreadLocal? I totally get why you might not want to do that (an extra check when a ThreadLocal is not the Terminating one). I?m OK either way. Yes, precisely. My 1st version did just that, but since there are usages of ThreadLocal out there that are very performance sensitive, I opted for an approach where there is zero performance regression for non-TerminatingThreadLocal(s) when calling set() or remove(). A call-back from setInitialValue is not so problematic as it usually occurs just once per thread, but I can imagine a scenario where calls to get() and set() occur very rapidly. Yeah, I agree that this is a good trade-off given how the code is currently structured. An alternative to "T getIfPresent()" is a "boolean isPresent()" method. Even if it remains package-private, with such method TerminatingThreadLocal could be implemented as: http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ If TreadLocal.isPresent() was made public, the code could be further simplified. Something like: if (tl.isPresent()) { T val = t.get(); ?. } will do two look-ups if the value exists. However, that?s better than allocating unnecessarily. So, I?ll take isPresent() over not having a way to check whether a value exists. Right, and in our scenario, it is just isPresent() that is called for every termination of every thread (REGISTRY.isPresent()). .get() is then called only when there's actually something to do. Yes. I?m not worried about that at all. Thumbs up from me. Let's just wait for a day or two to see whether the discussion on concurrency-interest gives us any additional ideas. Currently I'm thinking of proposing the addition of isPresent() API. As far as this RFR is concerned, I'm consequently promoting the latest webrev.04. Again, #ThumbsUp from me. Thanks, Tony So any comments on that one? Alan? Regards, Peter From james.laskey at oracle.com Thu Jun 7 14:07:46 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 7 Jun 2018 11:07:46 -0300 Subject: RFR: 8196990 :Resolve disabled warnings for libjli In-Reply-To: References: Message-ID: <830AE99F-314D-4965-A6D3-1A36DCFA0479@oracle.com> traditionally there is a space after // simplify comment to something like (rest is redundant) // initialize to avoid -Werror=maybe-uninitialized issues from gcc 7.3 onwards. > On Jun 7, 2018, at 11:01 AM, Srinivas Dama wrote: > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196990/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8196990 > > Regards, > Srinivas From srinivas.dama at oracle.com Thu Jun 7 15:14:55 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Thu, 7 Jun 2018 08:14:55 -0700 (PDT) Subject: RFR: 8196990 :Resolve disabled warnings for libjli Message-ID: Ok. Thank you Jim. Please find revised webrev at http://cr.openjdk.java.net/~sdama/8196990/webrev.01/. Regards, Srinivas ----- Original Message ----- From: james.laskey at oracle.com To: srinivas.dama at oracle.com, core-libs-dev at openjdk.java.net Sent: Thursday, 7 June, 2018 7:37:48 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: RFR: 8196990 :Resolve disabled warnings for libjli traditionally there is a space after // simplify comment to something like (rest is redundant) // initialize to avoid -Werror=maybe-uninitialized issues from gcc 7.3 onwards. > On Jun 7, 2018, at 11:01 AM, Srinivas Dama wrote: > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196990/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8196990 > > Regards, > Srinivas From james.laskey at oracle.com Thu Jun 7 15:16:16 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 7 Jun 2018 12:16:16 -0300 Subject: RFR: 8196990 :Resolve disabled warnings for libjli In-Reply-To: References: Message-ID: <218D2C33-C208-407C-987B-1E9D40420687@oracle.com> +1 > On Jun 7, 2018, at 12:14 PM, Srinivas Dama wrote: > > Ok. Thank you Jim. > Please find revised webrev at http://cr.openjdk.java.net/~sdama/8196990/webrev.01/. > > Regards, > Srinivas > > ----- Original Message ----- > From: james.laskey at oracle.com > To: srinivas.dama at oracle.com, core-libs-dev at openjdk.java.net > Sent: Thursday, 7 June, 2018 7:37:48 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi > Subject: Re: RFR: 8196990 :Resolve disabled warnings for libjli > > traditionally there is a space after // > > simplify comment to something like (rest is redundant) > > // initialize to avoid -Werror=maybe-uninitialized issues from gcc 7.3 onwards. > > > >> On Jun 7, 2018, at 11:01 AM, Srinivas Dama wrote: >> >> Hi, >> >> Please review http://cr.openjdk.java.net/~sdama/8196990/webrev.00/ >> for https://bugs.openjdk.java.net/browse/JDK-8196990 >> >> Regards, >> Srinivas > From takiguc at linux.vnet.ibm.com Thu Jun 7 16:29:09 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Fri, 08 Jun 2018 01:29:09 +0900 Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags In-Reply-To: References: <5928b735dc946ef140d3ccdf3852dc1f@linux.vnet.ibm.com> Message-ID: <29f9a3494ea8ebb6f5db05e4fb06fa30@linux.vnet.ibm.com> Hello Christoph According to build log, if <#include "osSupport.hpp"> was there: "/home/jdktest/sandbox/jdk/build/aix-ppc64-normal-server-release/support/headers/java.base/jdk_internal_jimage_NativeImageBuffer.h", line 15.27: 1540-0040 (S) The text "Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap" is unexpected. "visibility" may be undeclared or ambiguous. make[3]: *** [/home/jdktest/sandbox/jdk/build/aix-ppc64-normal-server-release/support/native/java.base/libjimage/NativeImageBuffer.o] Error 1 make[3]: Leaving directory `/home/jdktest/sandbox/jdk/make' make[2]: *** [java.base-libs] Error 2 make[2]: *** Waiting for unfinished jobs.... On 2018-06-07 22:06, Langer, Christoph wrote: > Hi Ichiroh, > > what's the exact error message if you #include "osSupport.hpp"? (I > have no xlC 13 at hand to try myself...) > > Best regards > Christoph > >> -----Original Message----- >> From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] >> Sent: Donnerstag, 7. Juni 2018 14:53 >> To: build-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; >> core- >> libs-dev at openjdk.java.net >> Cc: Lindenmaier, Goetz ; Baesken, Matthias >> ; Langer, Christoph >> >> Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol >> visibility flags >> >> Hello. >> >> Could you review it ? >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204541 >> Change: http://cr.openjdk.java.net/~clanger/webrevs/8204541.0/ >> >> Thanks, >> Ichiroh Takiguchi >> IBM Japan, Ltd. >> >> -------- Original Message -------- >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support >> on >> xlc 12.1 >> Date: 2018-06-07 20:43 >> From: "Langer, Christoph" >> To: Ichiroh Takiguchi >> Cc: "'build-dev at openjdk.java.net'" , >> "ppc-aix-port-dev at openjdk.java.net" > dev at openjdk.java.net>, >> "core-libs-dev at openjdk.java.net" , >> "Lindenmaier, Goetz" , "Baesken, Matthias" >> >> >> Hi Ichiroh, >> >> your proposal seems to make sense. I have created a bug for this: >> https://bugs.openjdk.java.net/browse/JDK-8204541 >> >> Can you please generate a webrev (referencing this bug, -c option of >> webrev.ksh) and mail it over to me. Then I'll upload it and you can >> post >> an official RFR mail. >> >> Best regards >> Christoph >> >> > -----Original Message----- >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] >> > Sent: Dienstag, 5. Juni 2018 08:59 >> > To: Baesken, Matthias >> > Cc: Langer, Christoph ; 'build- >> > dev at openjdk.java.net' ; ppc-aix-port- >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, >> > Goetz >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on >> > xlc 12.1 >> > >> > Hello Matthias and Christoph. >> > Thank you for your explanations. >> > >> > I did not have enough knowledge about "visibility". >> > >> > I created following patches. >> > >> > ================================ >> > diff -r 02934b0d661b >> > src/java.base/share/native/libjimage/NativeImageBuffer.cpp >> > --- a/src/java.base/share/native/libjimage/NativeImageBuffer.cpp Wed >> > May >> > 30 14:46:28 2018 +0200 >> > +++ b/src/java.base/share/native/libjimage/NativeImageBuffer.cpp >> Tue >> > Jun >> > 05 12:10:41 2018 +0900 >> > @@ -39,7 +39,9 @@ >> > #include "imageFile.hpp" >> > #include "inttypes.hpp" >> > #include "jimage.hpp" >> > +#if !defined(_AIX) >> > #include "osSupport.hpp" >> > +#endif >> > >> > #include "jdk_internal_jimage_NativeImageBuffer.h" >> > >> > diff -r 02934b0d661b src/java.base/unix/native/include/jni_md.h >> > --- a/src/java.base/unix/native/include/jni_md.h Wed May 30 14:46:28 >> > 2018 +0200 >> > +++ b/src/java.base/unix/native/include/jni_md.h Tue Jun 05 12:10:41 >> > 2018 +0900 >> > @@ -29,7 +29,8 @@ >> > #ifndef __has_attribute >> > #define __has_attribute(x) 0 >> > #endif >> > -#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && >> > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) >> > +#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) && >> > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) \ >> > + || (defined(_AIX) && (defined(__xlC__) && (__xlC__ >= 0xD01))) >> > #ifdef ARM >> > #define JNIEXPORT >> > __attribute__((externally_visible,visibility("default"))) >> > #define JNIIMPORT >> > __attribute__((externally_visible,visibility("default"))) >> > ================================ >> > >> > If "osSupport.hpp" was included, XLC++ compiler complained. >> > I could not understand, which part was invalid... >> > I'm not sure, "osSupport.hpp" is really required on >> > NativeImageBuffer.cpp or not... >> > >> > I added __xlC__ macro to determine XLC/C++'s version# into jni_md.h. >> > [1] >> > 0xD01 means 13.1 by hexadecimal. >> > >> > I checked symbol table by "dump -Tv -X64". [2] >> > It seemed it was fine, some symbols were hidden. >> > >> > Does someone review my code? >> > >> > [1] >> > >> https://www.ibm.com/support/knowledgecenter/en/SSGH2K_13.1.3/com.i >> > bm.xlc1313.aix.doc/compiler_ref/xlmacros.html >> > [2] >> > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- >> > visibility/index.html >> > >> > On 2018-06-01 21:12, Baesken, Matthias wrote: >> > > Hi , my change 8202322 just handled the fact that the >> > > visibility - flags are not supported with xlc 12.1 , so setting >> > > them generated a TON of compile - time warnings . >> > > >> > > The introduction of the "-qvisibility=hidden" came with the >> > > mapfile removal changes : >> > > >> > > 8200358: Remove mapfiles for JDK executables >> > > http://hg.openjdk.java.net/jdk/jdk/rev/210cf224b690 >> > > >> > > 8200178: Remove mapfiles for JDK native libraries >> > > http://hg.openjdk.java.net/jdk/jdk/rev/396ea30afbd5 >> > > >> > > I guess it might need further testing+adjustments to make the >> > > "visibility hiding" work nicely with XLC13 , but currently we >> > > build only with XLC12 . >> > > >> > > As a workaround you might want to remove the "-qvisibility=hidden" >> > > setting for XLC 13 as well , like I did for XLC12 with the change >> > > 8202322 . >> > > >> > > >> > > Best regards, Matthias >> > > >> > > >> > > >> > > >> > >> -----Original Message----- >> > >> From: Langer, Christoph >> > >> Sent: Freitag, 1. Juni 2018 10:57 >> > >> To: Ichiroh Takiguchi >> > >> Cc: Baesken, Matthias ; 'build- >> > >> dev at openjdk.java.net' ; ppc-aix-port- >> > >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, >> > >> Goetz >> > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support >> > >> on xlc 12.1 >> > >> >> > >> Hi Ichiroh, >> > >> >> > >> we do not use the XLC 13 compiler on AIX yet here at SAP and I believe >> > >> nobody of my colleagues has played with it yet. So you are on a new >> > >> playground here ?? >> > >> >> > >> However, I believe the idea in OpenJDK with the abolition of map files >> > >> is that >> > >> symbols should be invisible externally unless they are declared >> > >> exported, >> > >> e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the >> > >> correct >> > >> default and whatever JNIEXPORT expands to should contain the right >> > >> attributes to get that symbol visible. >> > >> >> > >> Can you check if either my assumption is completely wrong, JNIEXPORT >> > >> does >> > >> not expand to the right thing, XLC 13 has a bug or maybe just sume >> > >> specific >> > >> required symbols are not declared correctly? >> > >> >> > >> Best regards >> > >> Christoph >> > >> >> > >> > -----Original Message----- >> > >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] >> > >> > Sent: Donnerstag, 31. Mai 2018 09:55 >> > >> > To: Langer, Christoph >> > >> > Cc: Baesken, Matthias ; 'build- >> > >> > dev at openjdk.java.net' ; ppc-aix-port- >> > >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; >> Lindenmaier, >> > >> > Goetz >> > >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on >> xlc >> > 12.1 >> > >> > >> > >> > Hello. >> > >> > 8202322 was integrated into jdk-11+15. >> > >> > I'm using XLC 13.1.3 on AIX 7.1.4. >> > >> > Build was failed because of "-qvisibility=hidden" on >> > >> > make/lib/LibCommon.gmk. >> > >> > According to "XL C/C++ for AIX 13.1.3" documentation [1], >> > >> > "-qvisibility=hidden" cannot create shared libraries entry points. >> > >> > For example, libverify.so was there, but entry points were not >> resolved >> > >> > by "-lverify" option. >> > >> > I think it should be "-qvisibility=default" (I tried, it worked) >> > >> > or "-qvisibility=protected" (I had not tried) ? >> > >> > I'm not familiar with -qvisibility option, but I'd like to find out >> > >> > right way. >> > >> > >> > >> > [1] >> > >> > >> > >> >> > >> https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm. >> > >> > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html >> > >> > >> > >> > On 2018-05-16 16:08, Langer, Christoph wrote: >> > >> > > Hi Matthias, >> > >> > > >> > >> > > yes, reviewed. >> > >> > > >> > >> > > Best regards >> > >> > > Christoph >> > >> > > >> > >> > > From: Baesken, Matthias >> > >> > > Sent: Mittwoch, 16. Mai 2018 09:06 >> > >> > > To: Langer, Christoph ; >> > >> > > 'build-dev at openjdk.java.net' ; >> > >> > > ppc-aix-port-dev at openjdk.java.net; core-libs- >> dev at openjdk.java.net >> > >> > > Cc: Lindenmaier, Goetz >> > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on >> > >> > > xlc 12.1 >> > >> > > >> > >> > > Hi Christoph can I add you as second reviewer (other reviewer was >> > >> > > Erik Joelsson) can push the change ? >> > >> > > >> > >> > > Best regards, Matthias >> > >> > > >> > >> > > >> > >> > > >> > >> > > From: Langer, Christoph >> > >> > > Sent: Donnerstag, 26. April 2018 16:38 >> > >> > > To: Baesken, Matthias >> > >> > > >> >; >> > >> > > 'build-dev at openjdk.java.net' >> > >> > > > > dev at openjdk.java.net>>; >> > >> > > ppc-aix-port-dev at openjdk.java.net> > >> > dev at openjdk.java.net>; >> > >> > > core-libs-dev at openjdk.java.net> > >> dev at openjdk.java.net> >> > >> > > Cc: Simonis, Volker >> > >> > > > >> > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on >> > >> > > xlc 12.1 >> > >> > > >> > >> > > Hi Matthias, >> > >> > > >> > >> > > to me the change in principal looks good. >> > >> > > >> > >> > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. >> > >> > > extract major number before the first dot, then compare >> numerically) >> > - >> > >> > > but maybe it is too complicated and the current single version >> > compare >> > >> > > suits our needs ? >> > >> > > >> > >> > > Best regards >> > >> > > Christoph >> > >> > > >> > >> > > From: Baesken, Matthias >> > >> > > Sent: Donnerstag, 26. April 2018 16:14 >> > >> > > To: 'build-dev at openjdk.java.net' >> > >> > > > > dev at openjdk.java.net>>; >> > >> > > ppc-aix-port-dev at openjdk.java.net> > >> > dev at openjdk.java.net>; >> > >> > > core-libs-dev at openjdk.java.net> > >> dev at openjdk.java.net> >> > >> > > Cc: Langer, Christoph >> > >> > > >; >> > >> Simonis, >> > >> > > Volker >> > >> > >> > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on xlc >> > >> > > 12.1 >> > >> > > >> > >> > > Hello , could you please review this small adjustment to the symbol >> > >> > > visibility compilation settings on AIX ? >> > >> > > Currently we use XLC 12.1 to compile JDK on AIX . >> > >> > > >> > >> > > However XLC 12.1 does not support the "-qvisibility=hidden" >> > >> > > setting currently set on AIX. >> > >> > > It was introduced with XLC 13.1 . Christoph found some info about it >> > >> > > here : >> > >> > > >> > >> > > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- >> > >> > visibility-part2/index.html >> > >> > > >> > >> > > Setting it only generates hundreds of warnings in the build log , >> > >> > > warnings look like this : >> > >> > > XlC12.1 >> > >> > > >> > >> > > bash-4.4\$ xlC -qversion >> > >> > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) >> > >> > > Version: 12.01.0000.0019 >> > >> > > >> > >> > > bash-4.4\$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> > >> > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list >> > >> > > of valid options. >> > >> > > >> > >> > > Compare to XLC13.1 >> > >> > > >> > >> > > bash-3.00\$ xlC -qversion >> > >> > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) >> > >> > > Version: 13.01.0000.0008 >> > >> > > bash-3.00\$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc >> > >> > > bash-3.00\$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc >> > >> > > >> > >> > > >> > >> > > So it is better to avoid setting these flags when using xlc12.1 . >> > >> > > Please review : >> > >> > > >> > >> > > Bug : >> > >> > > >> > >> > > https://bugs.openjdk.java.net/browse/JDK-8202322 >> > >> > > >> > >> > > Change : >> > >> > > >> > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ >> > >> > > >> > >> > > >> > >> > > Best regards, Matthias From bob.vandette at oracle.com Thu Jun 7 17:43:23 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 7 Jun 2018 13:43:23 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: Can I get one more reviewer for this RFE so I can integrate it? > http://cr.openjdk.java.net/~bobv/8203357/webrev.01 Mandy Chung has reviewed this change. I?ve run Mach5 hotspot and core lib tests. I?ve reviewed the tests which were written by Harsha Wardhana I filed a CSR for the command line change and it?s now approved and closed. Thanks, Bob. > On May 30, 2018, at 3:45 PM, Bob Vandette wrote: > > Please review the following RFE which adds an internal API, along with jtreg tests that provide > access to Docker container configuration data and metrics. In addition to the API which we hope to > take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional > option to -XshowSettings:system than dumps out the container or host cgroup confguration > information. See the sample output below: > > RFE: Container Metrics > > https://bugs.openjdk.java.net/browse/JDK-8203357 > > WEBREV: > > http://cr.openjdk.java.net/~bobv/8203357/webrev.01 > > > This commit will also include a fix for the following bug. > > BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails > > https://bugs.openjdk.java.net/browse/JDK-8203691 > > WEBREV: > > http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html > > SAMPLE USAGE and OUTPUT: > > docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash > ./java -XshowSettings:system > Operating System Metrics: > Provider: cgroupv1 > Effective CPU Count: 4 > CPU Period: 100000 > CPU Quota: -1 > CPU Shares: -1 > List of Processors, 4 total: > 4 5 6 7 > List of Effective Processors, 4 total: > 4 5 6 7 > List of Memory Nodes, 2 total: > 0 1 > List of Available Memory Nodes, 2 total: > 0 1 > CPUSet Memory Pressure Enabled: false > Memory Limit: 256.00M > Memory Soft Limit: Unlimited > Memory & Swap Limit: 512.00M > Kernel Memory Limit: Unlimited > TCP Memory Limit: Unlimited > Out Of Memory Killer Enabled: true > > TEST RESULTS: > > testing runtime container APIs > Directory "JTwork" not found: creating > Passed: runtime/containers/cgroup/PlainRead.java > Passed: runtime/containers/docker/DockerBasicTest.java > Passed: runtime/containers/docker/TestCPUAwareness.java > Passed: runtime/containers/docker/TestCPUSets.java > Passed: runtime/containers/docker/TestMemoryAwareness.java > Passed: runtime/containers/docker/TestMisc.java > Test results: passed: 6 > Results written to /export/users/bobv/jdk11/build/jtreg/JTwork > > testing jdk.internal.platform APIs > Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java > Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java > Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java > Passed: jdk/internal/platform/docker/TestSystemMetrics.java > Test results: passed: 4 > Results written to /export/users/bobv/jdk11/build/jtreg/JTwork > > testing -XshowSettings:system launcher option > Passed: tools/launcher/Settings.java > Test results: passed: 1 > > > Bob. > > From brent.christian at oracle.com Thu Jun 7 19:13:46 2018 From: brent.christian at oracle.com (Brent Christian) Date: Thu, 7 Jun 2018 12:13:46 -0700 Subject: RFR 8204565 : (spec) Document java.{vm.}?specification.version system properties' relation to \$FEATURE Message-ID: <7ff4c005-bcb3-7715-554c-22bc5570b03a@oracle.com> Hi, Please review this doc-only change. From the bug report: 'With the integration of JEP 322 "Time-Based Release Versioning" into JDK 10, VERSION_FEATURE is used to set the value of the system properties "java.specification.version" [1] and "java.vm.specification.version" [2] (though the term "major" is still used in VM code, see JDK-8193719). We can update the System.getProperties() javadoc to be more specific about the value reported in these system properties.' Issue: https://bugs.openjdk.java.net/browse/JDK-8204565 Webrev: http://cr.openjdk.java.net/~bchristi/8204565/webrev/ Thanks, -Brent 1. http://hg.openjdk.java.net/jdk/jdk/rev/d2a837cf9ff1#l6.23 2. http://hg.openjdk.java.net/jdk/jdk/rev/d2a837cf9ff1#l15.17 From magnus.ihse.bursie at oracle.com Thu Jun 7 20:22:48 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 7 Jun 2018 22:22:48 +0200 Subject: RFR (XL): JDK-8204572 SetupJdkLibrary should setup SRC and -I flags automatically Message-ID: This change needs some background information. I've been working on simplifying and streamlining the compilation of native libraries of the JDK. Previously, I introduced the SetupJdkLibrary function, and started to get a better control of compiler flags. This patch continues on both paths. The original intent of this change, which I naively thought was going to be much simpler than it turned out :-) was to separate the -I flags (location of header files) from other flags, and to generate these automatically, wherever possible. Since we have a standard way of locating native code, and most libraries just want to have an -I path to their own source base and the generated Java header, we should be able to provide this in SetupJdkLibrary. This turned out to be closely related to SetupJdkLibrary being able to discover the location of the SRC directories itself, using "convention over configuration" and assuming that the library "libfoo" for "java.module" would be located in java.module/*/native/libfoo. While this sounds simple in theory, when actually trying to implement this I of course ran into all the places where some special handling was indeed needed. So even if like 90% of all libraries were simple to get to build using automated discovery of source and header directories, the 10% that did not caused me much more headaches than I had anticipated. On the other hand, now that I've sorted out all those places, the few remaining odd solutions is clearly documented and not just something that "just happens" due to strange configurations. One file deserves mentioning specifically: Awt2dLibraries.gmk. The java.desktop libraries are unfortunately quite entangled with each other, and do not readily follow the patterns that are used elsewhere in the code base. So it might just look like the file has just gone from one state of messiness, to another, which would hardly be an improvement. :-( I would still argue that the new messiness is better: It is now much clearer in what ways the libraries diverge from our standard assumption, and what course of action needs to be taken to minimize these differences. (Which is something I believe should be done -- these issues are not just cosmetic but is the root of most of the issues we always see for these libraries, when upgrading compilers, etc.) During this change, I noticed that not all native libraries include the proper generated header file. This is a dangerous coding practice, since a change in the Java part of the interface might not get picked up properly in the native part. I've added the missing includes that I've detected, and due to these changes, I'm also including the component teams in what is really only a build change. As can be seen for jdk.crypto.mscapi; there had indeed been changes that needed correcting. Since this is (basically) a pure build change, my gold standard here has been the build compare script. In essence, the build output prior to my change and with this change are 100% identical. In truth, this is a bit too strong a claim. A few changes has occurred, but none of them should matter. Here's a breakdown of the compare.sh results: * Windows-x64: No differences at all. * Solaris: Two libraries are reported to differ: libsaproc.so and libfontmanager.so, both with a disass diff on ~700 bytes. Analyzing this, I found that the object files used to link these two libraries has no disass differences. They have a slight binary difference and a difference in size, due to the include paths being different (and this is stored in the .o file, which makes it different). Somehow this apparently triggers the linker to generate a slightly different code in a few places, using a different register or so. (Weird...) * MacOS: Two libraries are reported to differ: libjava.dylib and libmlib_image.dylib, both of them just reported as a binary diff (no symbol, disass or fulldump differences). This is not really unsuspected, but I analyzed it anyway. I found that for libjava.dylib, a single .o file was different. This one was actually picked up from closed sources, and are not really relevant for OpenJDK. Anyway, the reason for the difference was the same as for the Solaris libs; the include paths had changes, which caused a binary diff. For libmlib_image.dylib, the link order had changed causing the noted binary difference. * Linux: On linux, the compare script noted differences for libextnet, libjava, libmlib_image, libprefs, libsaproc, libsplashscreen and libsunec. The differences for libextnet, libprefs, libsplashscreen and libsunec turned out to be caused by the added #include of the generated Java headers. This caused binary differences (reasonably), and for some odd reason also a symbol difference in java_awt_SplashScreen.o (clazz.10057 and mid.10058 were replaced by clazz.10015 and mid.10016). I can't claim to understand this, but I'm assuming it's some kind of generated code. libsaproc and libjava changes was caused by closed source changes, and is therefore not relevant to OpenJDK. For libmlib_image.dylib, the link order had changed causing the noted binary difference, as on MacOS. Bug: https://bugs.openjdk.java.net/browse/JDK-8204572 WebRev: http://cr.openjdk.java.net/~ihse/JDK-8204572-autodetect-SRC-and-headers-dirs/webrev.01 /Magnus From mandy.chung at oracle.com Thu Jun 7 20:24:49 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 7 Jun 2018 13:24:49 -0700 Subject: RFR 8204565 : (spec) Document java.{vm.}?specification.version system properties' relation to \$FEATURE In-Reply-To: <7ff4c005-bcb3-7715-554c-22bc5570b03a@oracle.com> References: <7ff4c005-bcb3-7715-554c-22bc5570b03a@oracle.com> Message-ID: <05f55a3a-a7d5-4b5d-899d-2f2fbfe1d2d8@oracle.com> Hi Brent, On 6/7/18 12:13 PM, Brent Christian wrote: > Hi, > > Please review this doc-only change.? From the bug report: > > 'With the integration of JEP 322 "Time-Based Release Versioning" > into JDK 10, VERSION_FEATURE is used to set the value of the system > properties "java.specification.version" [1] and > "java.vm.specification.version" [2] (though the term "major" is still > used in VM code, see JDK-8193719). > > We can update the System.getProperties() javadoc to be more specific > about the value reported in these system properties.' > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8204565 > > Webrev: > http://cr.openjdk.java.net/~bchristi/8204565/webrev/ Is there an existing test validating this? If not, we should add a test for this change? Mandy From jonathan.gibbons at oracle.com Thu Jun 7 21:19:23 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 07 Jun 2018 14:19:23 -0700 Subject: Locale languageTag anomaly Message-ID: <5B19A15B.3000702@oracle.com> Regarding Locale.toLanguageTag, Locale.forLanguageTag Should I be surprised (i.e. is it a bug) that out of 736 installed locales in a standard build of JDK, exactly 1 locale fails the following round-trip test: locale.equals(Locale.forLanguageTag(locale.toLanguageTag())) The locale "no_NO_NY" comes back as "nn_NO". Attached is a test program. Here is the output from the test program: \$ /opt/jdk/1.9.0/bin/java -cp play/locales/classes Locales test locale: no_NO_NY [no,NO,NY] nn-NO forLanguageTag: nn_NO Tested 736 locales Mismatch 1 locales -- Jon From erik.joelsson at oracle.com Thu Jun 7 21:20:52 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 7 Jun 2018 14:20:52 -0700 Subject: RFR (XL): JDK-8204572 SetupJdkLibrary should setup SRC and -I flags automatically In-Reply-To: References: Message-ID: Hello Magnus, Very nice refactoring! JdkNativeCompilation.gmk line 126-127 looks a bit long. There is an extra space on 126. Also, why not addprefix for adding -I instead of clunky foreach? Not that I care greatly, but I usually prefer that construct. Otherwise looks good. /Erik On 2018-06-07 13:22, Magnus Ihse Bursie wrote: > This change needs some background information. > > I've been working on simplifying and streamlining the compilation of > native libraries of the JDK. Previously, I introduced the > SetupJdkLibrary function, and started to get a better control of > compiler flags. This patch continues on both paths. > > The original intent of this change, which I naively thought was going > to be much simpler than it turned out :-) was to separate the -I flags > (location of header files) from other flags, and to generate these > automatically, wherever possible. Since we have a standard way of > locating native code, and most libraries just want to have an -I path > to their own source base and the generated Java header, we should be > able to provide this in SetupJdkLibrary. > > This turned out to be closely related to SetupJdkLibrary being able to > discover the location of the SRC directories itself, using "convention > over configuration" and assuming that the library "libfoo" for > "java.module" would be located in java.module/*/native/libfoo. > > While this sounds simple in theory, when actually trying to implement > this I of course ran into all the places where some special handling > was indeed needed. So even if like 90% of all libraries were simple to > get to build using automated discovery of source and header > directories, the 10% that did not caused me much more headaches than I > had anticipated. On the other hand, now that I've sorted out all those > places, the few remaining odd solutions is clearly documented and not > just something that "just happens" due to strange configurations. > > One file deserves mentioning specifically: Awt2dLibraries.gmk. The > java.desktop libraries are unfortunately quite entangled with each > other, and do not readily follow the patterns that are used elsewhere > in the code base. So it might just look like the file has just gone > from one state of messiness, to another, which would hardly be an > improvement. :-( I would still argue that the new messiness is better: > It is now much clearer in what ways the libraries diverge from our > standard assumption, and what course of action needs to be taken to > minimize these differences. (Which is something I believe should be > done -- these issues are not just cosmetic but is the root of most of > the issues we always see for these libraries, when upgrading > compilers, etc.) > > During this change, I noticed that not all native libraries include > the proper generated header file. This is a dangerous coding practice, > since a change in the Java part of the interface might not get picked > up properly in the native part. I've added the missing includes that > I've detected, and due to these changes, I'm also including the > component teams in what is really only a build change. As can be seen > for jdk.crypto.mscapi; there had indeed been changes that needed > correcting. > > Since this is (basically) a pure build change, my gold standard here > has been the build compare script. In essence, the build output prior > to my change and with this change are 100% identical. In truth, this > is a bit too strong a claim. A few changes has occurred, but none of > them should matter. Here's a breakdown of the compare.sh results: > > * Windows-x64: > > No differences at all. > > * Solaris: > > Two libraries are reported to differ: libsaproc.so and > libfontmanager.so, both with a disass diff on ~700 bytes. Analyzing > this, I found that the object files used to link these two libraries > has no disass differences. They have a slight binary difference and a > difference in size, due to the include paths being different (and this > is stored in the .o file, which makes it different). Somehow this > apparently triggers the linker to generate a slightly different code > in a few places, using a different register or so. (Weird...) > > * MacOS: > > Two libraries are reported to differ: libjava.dylib and > libmlib_image.dylib, both of them just reported as a binary diff (no > symbol, disass or fulldump differences). This is not really > unsuspected, but I analyzed it anyway. > > I found that for libjava.dylib, a single .o file was different. This > one was actually picked up from closed sources, and are not really > relevant for OpenJDK. Anyway, the reason for the difference was the > same as for the Solaris libs; the include paths had changes, which > caused a binary diff. > > For libmlib_image.dylib, the link order had changed causing the noted > binary difference. > > * Linux: > > On linux, the compare script noted differences for libextnet, libjava, > libmlib_image, libprefs, libsaproc, libsplashscreen and libsunec. > > The differences for libextnet, libprefs, libsplashscreen and libsunec > turned out to be caused by the added #include of the generated Java > headers. This caused binary differences (reasonably), and for some odd > reason also a symbol difference in java_awt_SplashScreen.o > (clazz.10057 and mid.10058 were replaced by clazz.10015 and > mid.10016). I can't claim to understand this, but I'm assuming it's > some kind of generated code. > > libsaproc and libjava changes was caused by closed source changes, and > is therefore not relevant to OpenJDK. > > For libmlib_image.dylib, the link order had changed causing the noted > binary difference, as on MacOS. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204572 > WebRev: > http://cr.openjdk.java.net/~ihse/JDK-8204572-autodetect-SRC-and-headers-dirs/webrev.01 > > /Magnus > From mikhailo.seledtsov at oracle.com Thu Jun 7 21:30:24 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Thu, 07 Jun 2018 14:30:24 -0700 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: <5B19A3F0.1010804@oracle.com> Hi Bob, I looked at the tests. In general they look good. I am a bit concerned about the use of ERROR_MARGIN in one of the tests. We need to make sure that the tests are stable, and do not produce intermittent failures. Thank you, Misha On 6/7/18, 10:43 AM, Bob Vandette wrote: > Can I get one more reviewer for this RFE so I can integrate it? > >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 > Mandy Chung has reviewed this change. > > I?ve run Mach5 hotspot and core lib tests. > > I?ve reviewed the tests which were written by Harsha Wardhana > > I filed a CSR for the command line change and it?s now approved and closed. > > Thanks, > Bob. > > >> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >> >> Please review the following RFE which adds an internal API, along with jtreg tests that provide >> access to Docker container configuration data and metrics. In addition to the API which we hope to >> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >> option to -XshowSettings:system than dumps out the container or host cgroup confguration >> information. See the sample output below: >> >> RFE: Container Metrics >> >> https://bugs.openjdk.java.net/browse/JDK-8203357 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> >> >> This commit will also include a fix for the following bug. >> >> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >> >> https://bugs.openjdk.java.net/browse/JDK-8203691 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >> >> SAMPLE USAGE and OUTPUT: >> >> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >> ./java -XshowSettings:system >> Operating System Metrics: >> Provider: cgroupv1 >> Effective CPU Count: 4 >> CPU Period: 100000 >> CPU Quota: -1 >> CPU Shares: -1 >> List of Processors, 4 total: >> 4 5 6 7 >> List of Effective Processors, 4 total: >> 4 5 6 7 >> List of Memory Nodes, 2 total: >> 0 1 >> List of Available Memory Nodes, 2 total: >> 0 1 >> CPUSet Memory Pressure Enabled: false >> Memory Limit: 256.00M >> Memory Soft Limit: Unlimited >> Memory& Swap Limit: 512.00M >> Kernel Memory Limit: Unlimited >> TCP Memory Limit: Unlimited >> Out Of Memory Killer Enabled: true >> >> TEST RESULTS: >> >> testing runtime container APIs >> Directory "JTwork" not found: creating >> Passed: runtime/containers/cgroup/PlainRead.java >> Passed: runtime/containers/docker/DockerBasicTest.java >> Passed: runtime/containers/docker/TestCPUAwareness.java >> Passed: runtime/containers/docker/TestCPUSets.java >> Passed: runtime/containers/docker/TestMemoryAwareness.java >> Passed: runtime/containers/docker/TestMisc.java >> Test results: passed: 6 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing jdk.internal.platform APIs >> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >> Test results: passed: 4 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing -XshowSettings:system launcher option >> Passed: tools/launcher/Settings.java >> Test results: passed: 1 >> >> >> Bob. >> >> From naoto.sato at oracle.com Thu Jun 7 21:42:26 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 7 Jun 2018 14:42:26 -0700 Subject: Locale languageTag anomaly In-Reply-To: <5B19A15B.3000702@oracle.com> References: <5B19A15B.3000702@oracle.com> Message-ID: Hi Jon, JDK historically represents Norwegian Nynorsk language with no_NO_NY (JDK unique "NY" variant), because it predates the ISO 639-1:2002 standard which introduced "nn" as the language code for Nynorsk. This legacy JDK locale is converted to BCP47 compliant language tag using "nn" language code. Locale.toLanguageTag() has the following sentence for this: --- A locale with language "no", country "NO", and variant "NY", representing Norwegian Nynorsk (Norway), is converted to a language tag "nn-NO". --- So in short, that is the expected behavior. Naoto On 6/7/18 2:19 PM, Jonathan Gibbons wrote: > Regarding Locale.toLanguageTag, Locale.forLanguageTag > > Should I be surprised (i.e. is it a bug) that out of 736 installed > locales in a standard build of JDK, exactly 1 locale fails the following > round-trip test: > > ??? locale.equals(Locale.forLanguageTag(locale.toLanguageTag())) > > The locale "no_NO_NY" comes back as "nn_NO". > > Attached is a test program.? Here is the output from the test program: > > \$ /opt/jdk/1.9.0/bin/java -cp play/locales/classes Locales > test locale: no_NO_NY [no,NO,NY] nn-NO > ???? forLanguageTag: nn_NO > Tested 736 locales > Mismatch 1 locales > > -- Jon From mandy.chung at oracle.com Thu Jun 7 22:38:16 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 7 Jun 2018 15:38:16 -0700 Subject: [JDK 11] RFR 8201528: Add new test to check for package versioning information in OpenJDK In-Reply-To: References: Message-ID: <08fb7059-2e3f-d77b-bd60-69bff4517edc@oracle.com> Hi Chris, On 6/7/18 1:32 AM, Chris Yin wrote: > Please review below new added test to check for package versioning > information which customized in jar(s) manifest, many thanks > > bug: https://bugs.openjdk.java.net/browse/JDK-8201528 webrev: > http://cr.openjdk.java.net/~xyin/8201528/webrev.00/ Thanks for adding the new test cases that we are missing. Can you describe all these test cases? In the class javadoc is fine. 32 * @run main PackageFromManifest runJar single 33 * @run main/othervm PackageFromManifest runUrlLoader single 34 * @run main PackageFromManifest runJar 1 35 * @run main PackageFromManifest runJar 2 36 * @run main/othervm PackageFromManifest runUrlLoader 1 37 * @run main/othervm PackageFromManifest runUrlLoader 2 74 if (runType.equalsIgnoreCase("runTest")) { The test accepts "runTest" but such test case is not listed above. It'd be good to refactor the main method into separate cases and a switch/if statement on different runtType will help the readability. 123 runTest(cl.loadClass( 124 PACKAGE_NAME + "." + TEST_CLASS_NAME1) 125 .getPackage(), option); Can you use Class::forName here? 200 cmds = new String[] { "-cp", 201 TEST_JAR_FILE1 + File.pathSeparator + TEST_JAR_FILE2, 202 "PackageFromManifest", "runTest", option }; Thanks for adding the split package test. It'd be good to test 1) loading TEST_CLASS_NAME1 first and TEST_CLASS_NAME2 second 2) loading TEST_CLASS_NAME2 first and TEST_CLASS_NAME1 second Package should use the manifest attribute from the JAR file from which the first class of package foo is loaded. Looks like it creates the JAR files each time the non-runTest case is run. You could do the setup as the first @run and subsequent @run are executing individual test cases (or make it testng test if you prefer). Mandy From jonathan.gibbons at oracle.com Thu Jun 7 22:38:46 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 07 Jun 2018 15:38:46 -0700 Subject: Locale languageTag anomaly In-Reply-To: References: <5B19A15B.3000702@oracle.com> Message-ID: <5B19B3F6.6020501@oracle.com> Naoto, Thanks, and I guess I didn't think to check out the special case rules. Sorry about that. -- Jon On 06/07/2018 02:42 PM, Naoto Sato wrote: > Hi Jon, > > JDK historically represents Norwegian Nynorsk language with no_NO_NY > (JDK unique "NY" variant), because it predates the ISO 639-1:2002 > standard which introduced "nn" as the language code for Nynorsk. This > legacy JDK locale is converted to BCP47 compliant language tag using > "nn" language code. > > Locale.toLanguageTag() has the following sentence for this: > > --- > A locale with language "no", country "NO", and variant "NY", > representing Norwegian Nynorsk (Norway), is converted to a language > tag "nn-NO". > --- > > So in short, that is the expected behavior. > > Naoto > > On 6/7/18 2:19 PM, Jonathan Gibbons wrote: >> Regarding Locale.toLanguageTag, Locale.forLanguageTag >> >> Should I be surprised (i.e. is it a bug) that out of 736 installed >> locales in a standard build of JDK, exactly 1 locale fails the >> following round-trip test: >> >> locale.equals(Locale.forLanguageTag(locale.toLanguageTag())) >> >> The locale "no_NO_NY" comes back as "nn_NO". >> >> Attached is a test program. Here is the output from the test program: >> >> \$ /opt/jdk/1.9.0/bin/java -cp play/locales/classes Locales >> test locale: no_NO_NY [no,NO,NY] nn-NO >> forLanguageTag: nn_NO >> Tested 736 locales >> Mismatch 1 locales >> >> -- Jon From mandy.chung at oracle.com Thu Jun 7 23:06:15 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 7 Jun 2018 16:06:15 -0700 Subject: (XS) Review Request JDK-8204584: jdeps generates illegal dot file containing ranksep=0,600000 Message-ID: The dot files are generated and used as module graph in the docs build. The format of double should use no localization; otherwise an illegal ranksep attribute. Mandy diff --git a/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java b/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java --- a/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java +++ b/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java @@ -44,6 +44,7 @@ import java.util.Deque; import java.util.HashSet; import java.util.List; +import java.util.Locale; import java.util.Map; import java.util.Objects; import java.util.Set; @@ -329,7 +330,7 @@ out.format("digraph \"%s\" {%n", name); out.format(" nodesep=.5;%n"); - out.format(" ranksep=%f;%n", attributes.rankSep()); + out.format((Locale)null, " ranksep=%f;%n", attributes.rankSep()); out.format(" pencolor=transparent;%n"); out.format(" node [shape=plaintext, fontcolor=\"%s\", fontname=\"%s\"," + " fontsize=%d, margin=\".2,.2\"];%n", From jonathan.gibbons at oracle.com Thu Jun 7 23:10:36 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 07 Jun 2018 16:10:36 -0700 Subject: (XS) Review Request JDK-8204584: jdeps generates illegal dot file containing ranksep=0,600000 In-Reply-To: References: Message-ID: <5B19BB6C.6080505@oracle.com> Looks OK to me. -- Jon On 06/07/2018 04:06 PM, mandy chung wrote: > The dot files are generated and used as module graph in the docs build. > The format of double should use no localization; otherwise an illegal > ranksep attribute. > > Mandy > > > diff --git > a/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java > b/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java > --- a/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java > +++ b/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/ModuleDotGraph.java > @@ -44,6 +44,7 @@ > import java.util.Deque; > import java.util.HashSet; > import java.util.List; > +import java.util.Locale; > import java.util.Map; > import java.util.Objects; > import java.util.Set; > @@ -329,7 +330,7 @@ > > out.format("digraph \"%s\" {%n", name); > out.format(" nodesep=.5;%n"); > - out.format(" ranksep=%f;%n", attributes.rankSep()); > + out.format((Locale)null, " ranksep=%f;%n", > attributes.rankSep()); > out.format(" pencolor=transparent;%n"); > out.format(" node [shape=plaintext, > fontcolor=\"%s\", fontname=\"%s\"," > + " fontsize=%d, margin=\".2,.2\"];%n", From paul.sandoz at oracle.com Fri Jun 8 00:42:04 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 7 Jun 2018 17:42:04 -0700 Subject: RFC: Experiment in accessing/managing persistent memory from Java In-Reply-To: <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE74C14@FMSMSX126.amr.corp.intel.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE74C14@FMSMSX126.amr.corp.intel.com> Message-ID: Hi Andrew, Jonathan, Sandhya gave an overview to a few of us Oracle folks. I agree with what Sandhya says regarding the API, a small surface, and on pursuing an unsafe intrinsic. I like it and would encourage the writing of a draft JEP, especially to give this visibility. I expect this will be beneficial for experimentation with the Panama foreign API where we can use a Pointer to reference into a byte buffer and scribble on it. Further, i hope this work may also benefit the persistent collections effort (PCJ). It intersects with https://bugs.openjdk.java.net/browse/JDK-8153111 ((bf) Allocating ByteBuffer on heterogeneous memory), which is attempting to be more generic. We might also need to increase the velocity on https://bugs.openjdk.java.net/browse/JDK-8180628 (retrofit direct buffer support for size beyond gigabyte scales), and i would be very interested your views on this, how you might be currently working around such size limitations, and what buffer enhancements would work for you. Thanks, Paul. > On May 30, 2018, at 10:21 PM, Viswanathan, Sandhya wrote: > > Hi Andrew/Jonathan, > > Thanks a lot for sharing this work. Copying hotspot-compiler-dev to get their feedback as well. > > Couple of thoughts/observations below: > * Supporting ByteBuffer on persistent memory using existing FileChannel and MappedByteBuffer mechanism sounds like a very good idea. > > * Extending FileChannel.map to take additional parameter to indicate that the ByteBuffer is backed by persistent memory is a small API change. > > * Adding MappedByteBuffer.force(int from, int to) method on smaller range is very useful in addition to the force() on entire ByteBuffer. > > * The underlying force0_mapsync() could be implemented in terms of new unsafe APIs which in turn could be intrinsified. > The advantage of this is that the unsafe APIs could then be used for other future persistent memory APIs in the JRE. > Specifically the following two unsafe APIs would be useful: > a) public native void flush(long address, long size); > b) public native void storeFence(); > storeFence() exists today but doesn?t generate any instruction for x86. > Wondering if we could have additional boolean parameter to force the sfence generation. > * DEFAULT_CACHE_LINE_SIZE is 128 in src/hotspot/cpu/x86/globalDefinitions_x86.hpp whereas actual cache line on the hardware is 64 bytes. > This could be the cause for some of performance that you saw with compiler intrinsic vs pure C native. > > Best Regards, > Sandhya > > > > RFC: Experiment in accessing/managing persistent memory from Java > Andrew Dinn adinn at redhat.com > Mon May 21 09:47:46 UTC 2018 > > I have been helping one of my Red Hat colleagues, Jonathan Halliday, to > investigate provision of a Java equivalent to Intel's libpmem suite of C > libraries [1]. This approach avoids the significant cost of using the > Intel libraries from Java via JNI (or, worse, as a virtual driver for a > persistent memory device). > > Jonathan has modified the JVM/JDK to allow a MappedByteBuffer to be > mapped over persistent memory, providing equivalent function to libpmem > itself. > > On top of this he implemented a Java journaled log class providing > equivalent functionality to one of the Intel client libs, libpmemlog, > built over libpmem. > > The modified MappedByteBuffer can be configured to use either i) a > registered native method or ii) a JIT intrinsic to perform the critical > task of cache line writeback i.e. the persistence step (the intrinsic is > my contribution). > > Jonathan's tests compare use of JNI, registered native and intrinsic > with an equivalent C program to write a large swathe of records to a > journaled log file stored in persistent memory. Performance is worse > than C when relying on JNI and significantly better with JVM/JDK > support. Indeed, as one might reasonably expect, use of the JIT > intrinsic almost completely eliminates writeback costs. > > The journaled log code, jdk dev tree patch, build instructions, test > code plus C equivalent and test results are all available from > Jonathan's git repo [2]. > > For those who do not want to look at the actual code, the README file > [3] provides background to use of persistent memory, an overview of the > design, and summary details of the test process and results. > > [1] https://pmem.io/pmdk/ > [2] https://github.com/jhalliday/pmem > [3] http://github.com://jhalliday/pmem/README.md > > n.b. Jonathan has experimented with using this same prototype to replace > the journaled log used in the Red Hat Narayana transaction manager. It > provides a significant improvement on the current disk file based log, > both for throughput and latency (the code is not yet available as > getting it to work involved some horrible hacking of the build to > migrate up to jdk11). > > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From david.holmes at oracle.com Fri Jun 8 02:20:39 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 8 Jun 2018 12:20:39 +1000 Subject: (trivial) RFR: 8204589: ProblemList failing launcher tests Message-ID: <33d65120-5ae0-0dfb-6450-b3475d87a61e@oracle.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8204589 webrev: http://cr.openjdk.java.net/~dholmes/8204589/webrev/ After the push for: JDK-8201274 Launch Single-File Source-Code Programs there are a few test failures in our CI testing, so they are being ProblemListed to keep tiers 1 and 2 clean. Thanks, David From joe.darcy at oracle.com Fri Jun 8 02:22:22 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Thu, 07 Jun 2018 19:22:22 -0700 Subject: (trivial) RFR: 8204589: ProblemList failing launcher tests In-Reply-To: <33d65120-5ae0-0dfb-6450-b3475d87a61e@oracle.com> References: <33d65120-5ae0-0dfb-6450-b3475d87a61e@oracle.com> Message-ID: <5B19E85E.40805@oracle.com> +1 -Joe On 6/7/2018 7:20 PM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8204589 > webrev: http://cr.openjdk.java.net/~dholmes/8204589/webrev/ > > After the push for: > > JDK-8201274 Launch Single-File Source-Code Programs > > there are a few test failures in our CI testing, so they are being > ProblemListed to keep tiers 1 and 2 clean. > > Thanks, > David From david.holmes at oracle.com Fri Jun 8 02:30:43 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 8 Jun 2018 12:30:43 +1000 Subject: (trivial) RFR: 8204589: ProblemList failing launcher tests In-Reply-To: <5B19E85E.40805@oracle.com> References: <33d65120-5ae0-0dfb-6450-b3475d87a61e@oracle.com> <5B19E85E.40805@oracle.com> Message-ID: Thanks Joe! Pushed. Should be in CI 753. David On 8/06/2018 12:22 PM, Joseph D. Darcy wrote: > +1 > > -Joe > > On 6/7/2018 7:20 PM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204589 >> webrev: http://cr.openjdk.java.net/~dholmes/8204589/webrev/ >> >> After the push for: >> >> ?JDK-8201274 Launch Single-File Source-Code Programs >> >> there are a few test failures in our CI testing, so they are being >> ProblemListed to keep tiers 1 and 2 clean. >> >> Thanks, >> David > From xueming.shen at oracle.com Fri Jun 8 02:33:11 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 07 Jun 2018 19:33:11 -0700 Subject: RFR JDK-8204229: Formatter and String.format ignore the width with the percent modifier (%5%) Message-ID: <5B19EAE7.9030509@oracle.com> Hi, Please help review the change for JDK-8204229. It appears to be a overlook in the implementation. We do have a method print(String, Locale) that adjust the "padding spaces" issue: https://bugs.openjdk.java.net/browse/JDK-8204229 webrev: http://cr.openjdk.java.net/~sherman/8204229/webrev Thanks, Sherman From harsha.wardhana.b at oracle.com Fri Jun 8 04:30:39 2018 From: harsha.wardhana.b at oracle.com (Harsha Wardhana B) Date: Fri, 8 Jun 2018 10:00:39 +0530 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <5B19A3F0.1010804@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <5B19A3F0.1010804@oracle.com> Message-ID: <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> [Replying to all mailing-lists] Hi Misha, The ERROR_MARGIN in tests was introduced to make the tests stable. There are times where metric values (specifically CPU usage) can change drastically in between two reads. The metrics value got from the API and the cgroup file can be different and 0.1 ERROR_MARGIN should take care of that, though at times even that may not be enough. Hence the CPU usage related tests only print a warning if ERROR_MARGIN is exceeded. Thanks Harsha On Friday 08 June 2018 03:00 AM, Mikhailo Seledtsov wrote: > Hi Bob, > > ? I looked at the tests. In general they look good. I am a bit > concerned about the use of ERROR_MARGIN in one of the tests. We need > to make sure that the tests are stable, and do not produce > intermittent failures. > > > Thank you, > Misha > > On 6/7/18, 10:43 AM, Bob Vandette wrote: >> Can I get one more reviewer for this RFE so I can integrate it? >> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> Mandy Chung has reviewed this change. >> >> I?ve run Mach5 hotspot and core lib tests. >> >> I?ve reviewed the tests which were written by Harsha Wardhana >> >> I filed a CSR for the command line change and it?s now approved and >> closed. >> >> Thanks, >> Bob. >> >> >>> On May 30, 2018, at 3:45 PM, Bob Vandette? >>> wrote: >>> >>> Please review the following RFE which adds an internal API, along >>> with jtreg tests that provide >>> access to Docker container configuration data and metrics.? In >>> addition to the API which we hope to >>> take advantage of in the future with Java Flight Recorder and a JMX >>> Mbean, I?ve added an additional >>> option to -XshowSettings:system than dumps out the container or host >>> cgroup confguration >>> information.? See the sample output below: >>> >>> RFE: Container Metrics >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> >>> >>> This commit will also include a fix for the following bug. >>> >>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>> >>> >>> SAMPLE USAGE and OUTPUT: >>> >>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>> ./java -XshowSettings:system >>> Operating System Metrics: >>> ??? Provider: cgroupv1 >>> ??? Effective CPU Count: 4 >>> ??? CPU Period: 100000 >>> ??? CPU Quota: -1 >>> ??? CPU Shares: -1 >>> ??? List of Processors, 4 total: >>> ??? 4 5 6 7 >>> ??? List of Effective Processors, 4 total: >>> ??? 4 5 6 7 >>> ??? List of Memory Nodes, 2 total: >>> ??? 0 1 >>> ??? List of Available Memory Nodes, 2 total: >>> ??? 0 1 >>> ??? CPUSet Memory Pressure Enabled: false >>> ??? Memory Limit: 256.00M >>> ??? Memory Soft Limit: Unlimited >>> ??? Memory&? Swap Limit: 512.00M >>> ??? Kernel Memory Limit: Unlimited >>> ??? TCP Memory Limit: Unlimited >>> ??? Out Of Memory Killer Enabled: true >>> >>> TEST RESULTS: >>> >>> testing runtime container APIs >>> Directory "JTwork" not found: creating >>> Passed: runtime/containers/cgroup/PlainRead.java >>> Passed: runtime/containers/docker/DockerBasicTest.java >>> Passed: runtime/containers/docker/TestCPUAwareness.java >>> Passed: runtime/containers/docker/TestCPUSets.java >>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>> Passed: runtime/containers/docker/TestMisc.java >>> Test results: passed: 6 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing jdk.internal.platform APIs >>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>> Test results: passed: 4 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing -XshowSettings:system launcher option >>> Passed: tools/launcher/Settings.java >>> Test results: passed: 1 >>> >>> >>> Bob. >>> >>> From srinivas.dama at oracle.com Fri Jun 8 06:17:10 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Thu, 7 Jun 2018 23:17:10 -0700 (PDT) Subject: RFR: 8196988 (Resolve disabled warnings for libjimage) Message-ID: <7477c842-56d3-4980-b51b-66ad9d8a05c0@default> Hi, Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.01/ for https://bugs.openjdk.java.net/browse/JDK-8196988 Note: Modified earlier fix to handle warning messages specific to libjimage only. Regards, Srinivas ----- Original Message ----- From: srinivas.dama at oracle.com To: core-libs-dev at openjdk.java.net Sent: Monday, 4 June, 2018 11:08:08 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: RFR: 8196988 (Resolve disabled warnings for implicit-fallthrough gcc option) Hi, Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8196988 Note: gcc 7.3 compiler emits warnings for implicit fall-through switch cases,but these warnings are disabled earlier.I have enabled these warnings and fixed in sources. Regards, Srinivas From matthias.baesken at sap.com Fri Jun 8 06:51:18 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 8 Jun 2018 06:51:18 +0000 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: <67fa98e2ed454a208af23a7a56e4627b@sap.com> References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> Message-ID: <091861e7d9c94e27af146eed3e57d3eb@sap.com> Hi, looks like I uploaded a wrong version of the patch. Updated webrev is here : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.1/ I followed the advice of Sean and removed the mapFileName from the output because it is usually a static name . Updated webrev was going through our internal build/tests . Best regards, Matthias From: Langer, Christoph Sent: Donnerstag, 7. Juni 2018 13:52 To: Baesken, Matthias ; core-libs-dev at openjdk.java.net Cc: Lindenmaier, Goetz Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi Matthias, in line 527, where the actual output is done, I think you would need to replace variable 'message' with 'outputMessage', otherwise I guess it won't compile?? Also, mapFileName can't be used at this place, because it has already been free'ed there. But in general the additions make sense and will make it easier to find issues in the tzmappings file. Best regards Christoph From: Baesken, Matthias Sent: Donnerstag, 7. Juni 2018 13:35 To: core-libs-dev at openjdk.java.net Cc: Langer, Christoph >; Lindenmaier, Goetz > Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi, could you please review this small change that improves the error messages in matchJavaTZ . A reason of the failure is added to the message , and also the offset where the error happened . Bug : https://bugs.openjdk.java.net/browse/JDK-8204539 Webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ Thanks, Matthias From xu.y.yin at oracle.com Fri Jun 8 07:07:29 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Fri, 8 Jun 2018 15:07:29 +0800 Subject: [JDK 11] RFR 8201528: Add new test to check for package versioning information in OpenJDK In-Reply-To: <08fb7059-2e3f-d77b-bd60-69bff4517edc@oracle.com> References: <08fb7059-2e3f-d77b-bd60-69bff4517edc@oracle.com> Message-ID: <7D1544F3-4BD6-48FA-AB7B-BA3941EEE04E@oracle.com> Hi, Mandy Many thanks for your detailed review and comments, updates new webrev as below, and comment inline, thanks webrev: http://cr.openjdk.java.net/~xyin/8201528/webrev.01/ > On 8 Jun 2018, at 6:38 AM, mandy chung wrote: > > Hi Chris, > > On 6/7/18 1:32 AM, Chris Yin wrote: >> Please review below new added test to check for package versioning >> information which customized in jar(s) manifest, many thanks >> bug: https://bugs.openjdk.java.net/browse/JDK-8201528 webrev: >> http://cr.openjdk.java.net/~xyin/8201528/webrev.00/ > > Thanks for adding the new test cases that we are missing. > > Can you describe all these test cases? In the class javadoc is fine. Sure, just added class javadoc to explain input arguments and each @run usage > > 32 * @run main PackageFromManifest runJar single > 33 * @run main/othervm PackageFromManifest runUrlLoader single > 34 * @run main PackageFromManifest runJar 1 > 35 * @run main PackageFromManifest runJar 2 > 36 * @run main/othervm PackageFromManifest runUrlLoader 1 > 37 * @run main/othervm PackageFromManifest runUrlLoader 2 > > 74 if (runType.equalsIgnoreCase("runTest")) { > > The test accepts "runTest" but such test case is not listed above. Thanks for checking this, ?runTest? run type will be used in test logic runJar only (for example, called from runJar method, "java -cp testpack1.jar PackageFromManifest runTest single") > > It'd be good to refactor the main method into separate cases and > a switch/if statement on different runtType will help the readability. Thanks, refactored in new webrev as you suggested, it looks more clear now > > 123 runTest(cl.loadClass( > 124 PACKAGE_NAME + "." + TEST_CLASS_NAME1) > 125 .getPackage(), option); > > Can you use Class::forName here? Sure, fixed in new webrev > > 200 cmds = new String[] { "-cp", > 201 TEST_JAR_FILE1 + File.pathSeparator + TEST_JAR_FILE2, > 202 "PackageFromManifest", "runTest", option }; > > Thanks for adding the split package test. It'd be good to test > 1) loading TEST_CLASS_NAME1 first and TEST_CLASS_NAME2 second > 2) loading TEST_CLASS_NAME2 first and TEST_CLASS_NAME1 second > > Package should use the manifest attribute from the JAR file from > which the first class of package foo is loaded. Yep, to make it clear, in new webrev, the first loaded class name will be printed (combine load and print) before runTest, such as System.out.println("Load " + Class.forName(PACKAGE_NAME + ?.? + TEST_CLASS_PREFIX + option) + " first"); > > Looks like it creates the JAR files each time the non-runTest case > is run. You could do the setup as the first @run and subsequent > @run are executing individual test cases (or make it testng test > if you prefer). Used setup at the first @run in new webrev as you suggested, thanks Regards, Chris > > Mandy From christoph.langer at sap.com Fri Jun 8 07:14:16 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 8 Jun 2018 07:14:16 +0000 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: <091861e7d9c94e27af146eed3e57d3eb@sap.com> References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> <091861e7d9c94e27af146eed3e57d3eb@sap.com> Message-ID: Hi Matthias, this looks good. Reviewed from my end. Best regards Christoph From: Baesken, Matthias Sent: Freitag, 8. Juni 2018 08:51 To: Langer, Christoph ; core-libs-dev at openjdk.java.net; 'sean.coffey at oracle.com' Cc: Lindenmaier, Goetz Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi, looks like I uploaded a wrong version of the patch. Updated webrev is here : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.1/ I followed the advice of Sean and removed the mapFileName from the output because it is usually a static name . Updated webrev was going through our internal build/tests . Best regards, Matthias From: Langer, Christoph Sent: Donnerstag, 7. Juni 2018 13:52 To: Baesken, Matthias >; core-libs-dev at openjdk.java.net Cc: Lindenmaier, Goetz > Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi Matthias, in line 527, where the actual output is done, I think you would need to replace variable 'message' with 'outputMessage', otherwise I guess it won't compile?? Also, mapFileName can't be used at this place, because it has already been free'ed there. But in general the additions make sense and will make it easier to find issues in the tzmappings file. Best regards Christoph From: Baesken, Matthias Sent: Donnerstag, 7. Juni 2018 13:35 To: core-libs-dev at openjdk.java.net Cc: Langer, Christoph >; Lindenmaier, Goetz > Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] Hi, could you please review this small change that improves the error messages in matchJavaTZ . A reason of the failure is added to the message , and also the offset where the error happened . Bug : https://bugs.openjdk.java.net/browse/JDK-8204539 Webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ Thanks, Matthias From goetz.lindenmaier at sap.com Fri Jun 8 07:33:45 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 8 Jun 2018 07:33:45 +0000 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: <091861e7d9c94e27af146eed3e57d3eb@sap.com> References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> <091861e7d9c94e27af146eed3e57d3eb@sap.com> Message-ID: <3922fa39aa4749c1919b54e467541f7c@sap.com> Hi Matthias, thanks for adding this information, looks good! Best regards, Goety. > -----Original Message----- > From: Baesken, Matthias > Sent: Freitag, 8. Juni 2018 08:51 > To: Langer, Christoph ; core-libs- > dev at openjdk.java.net; 'sean.coffey at oracle.com' > > Cc: Lindenmaier, Goetz > Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ > [windows] > > Hi, looks like I uploaded a wrong version of the patch. > > Updated webrev is here : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.1/ > > > > I followed the advice of Sean and removed the mapFileName from the > output because it is usually a static name . > > Updated webrev was going through our internal build/tests . > > > > > > Best regards, Matthias > > > > > > From: Langer, Christoph > Sent: Donnerstag, 7. Juni 2018 13:52 > To: Baesken, Matthias ; core-libs- > dev at openjdk.java.net > Cc: Lindenmaier, Goetz > Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ > [windows] > > > > Hi Matthias, > > > > in line 527, where the actual output is done, I think you would need to > replace variable 'message' with 'outputMessage', otherwise I guess it won't > compile?? > > > > Also, mapFileName can't be used at this place, because it has already been > free'ed there. > > > > But in general the additions make sense and will make it easier to find issues > in the tzmappings file. > > > > Best regards > > Christoph > > > > From: Baesken, Matthias > Sent: Donnerstag, 7. Juni 2018 13:35 > To: core-libs-dev at openjdk.java.net dev at openjdk.java.net> > Cc: Langer, Christoph >; Lindenmaier, Goetz > > > Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] > > > > Hi, could you please review this small change that improves the error > messages in matchJavaTZ . > > A reason of the failure is added to the message , and also the offset where > the error happened . > > > > > > Bug : > > > > https://bugs.openjdk.java.net/browse/JDK-8204539 > > > > > > Webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ > > > > > > Thanks, Matthias From christoph.langer at sap.com Fri Jun 8 07:35:48 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 8 Jun 2018 07:35:48 +0000 Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags In-Reply-To: <29f9a3494ea8ebb6f5db05e4fb06fa30@linux.vnet.ibm.com> References: <5928b735dc946ef140d3ccdf3852dc1f@linux.vnet.ibm.com> <29f9a3494ea8ebb6f5db05e4fb06fa30@linux.vnet.ibm.com> Message-ID: Hi Ichiroh, Ok, so as per the output, via the include of osSupport.hpp, something must happen which undeclares "visibility" or makes it ambiguous. Looking at osSupport.hpp, I can't see anything special. It would just include pthread.h and declare some c++ classes. You could try to get and analyze the preprocessed output of xlC by specifying the option -P or -E to the compile call. You will get the original xlC command line by calling 'cat support/native/java.base/libjimage/NativeImageBuffer.o.cmdline' in your build directory. I think we should really understand what's happening there and fix it correctly instead of just excluding osSupport.hpp. Best regards Christoph > -----Original Message----- > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > Sent: Donnerstag, 7. Juni 2018 18:29 > To: Langer, Christoph > Cc: build-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; core- > libs-dev at openjdk.java.net; Lindenmaier, Goetz > ; Baesken, Matthias > > Subject: RE: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags > > Hello Christoph > > According to build log, if <#include "osSupport.hpp"> was there: > "/home/jdktest/sandbox/jdk/build/aix-ppc64-normal-server- > release/support/headers/java.base/jdk_internal_jimage_NativeImageBuffe > r.h", > line 15.27: 1540-0040 (S) The text > "Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap" is > unexpected. "visibility" may be undeclared or ambiguous. > make[3]: *** > [/home/jdktest/sandbox/jdk/build/aix-ppc64-normal-server- > release/support/native/java.base/libjimage/NativeImageBuffer.o] > Error 1 > make[3]: Leaving directory `/home/jdktest/sandbox/jdk/make' > make[2]: *** [java.base-libs] Error 2 > make[2]: *** Waiting for unfinished jobs.... > > On 2018-06-07 22:06, Langer, Christoph wrote: > > Hi Ichiroh, > > > > what's the exact error message if you #include "osSupport.hpp"? (I > > have no xlC 13 at hand to try myself...) > > > > Best regards > > Christoph > > > >> -----Original Message----- > >> From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > >> Sent: Donnerstag, 7. Juni 2018 14:53 > >> To: build-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; > >> core- > >> libs-dev at openjdk.java.net > >> Cc: Lindenmaier, Goetz ; Baesken, > Matthias > >> ; Langer, Christoph > >> > >> Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol > >> visibility flags > >> > >> Hello. > >> > >> Could you review it ? > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204541 > >> Change: http://cr.openjdk.java.net/~clanger/webrevs/8204541.0/ > >> > >> Thanks, > >> Ichiroh Takiguchi > >> IBM Japan, Ltd. > >> > >> -------- Original Message -------- > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > >> on > >> xlc 12.1 > >> Date: 2018-06-07 20:43 > >> From: "Langer, Christoph" > >> To: Ichiroh Takiguchi > >> Cc: "'build-dev at openjdk.java.net'" , > >> "ppc-aix-port-dev at openjdk.java.net" >> dev at openjdk.java.net>, > >> "core-libs-dev at openjdk.java.net" , > >> "Lindenmaier, Goetz" , "Baesken, > Matthias" > >> > >> > >> Hi Ichiroh, > >> > >> your proposal seems to make sense. I have created a bug for this: > >> https://bugs.openjdk.java.net/browse/JDK-8204541 > >> > >> Can you please generate a webrev (referencing this bug, -c option of > >> webrev.ksh) and mail it over to me. Then I'll upload it and you can > >> post > >> an official RFR mail. > >> > >> Best regards > >> Christoph > >> > >> > -----Original Message----- > >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > >> > Sent: Dienstag, 5. Juni 2018 08:59 > >> > To: Baesken, Matthias > >> > Cc: Langer, Christoph ; 'build- > >> > dev at openjdk.java.net' ; ppc-aix-port- > >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Lindenmaier, > >> > Goetz > >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > >> > xlc 12.1 > >> > > >> > Hello Matthias and Christoph. > >> > Thank you for your explanations. > >> > > >> > I did not have enough knowledge about "visibility". > >> > > >> > I created following patches. > >> > > >> > ================================ > >> > diff -r 02934b0d661b > >> > src/java.base/share/native/libjimage/NativeImageBuffer.cpp > >> > --- a/src/java.base/share/native/libjimage/NativeImageBuffer.cpp > Wed > >> > May > >> > 30 14:46:28 2018 +0200 > >> > +++ b/src/java.base/share/native/libjimage/NativeImageBuffer.cpp > >> Tue > >> > Jun > >> > 05 12:10:41 2018 +0900 > >> > @@ -39,7 +39,9 @@ > >> > #include "imageFile.hpp" > >> > #include "inttypes.hpp" > >> > #include "jimage.hpp" > >> > +#if !defined(_AIX) > >> > #include "osSupport.hpp" > >> > +#endif > >> > > >> > #include "jdk_internal_jimage_NativeImageBuffer.h" > >> > > >> > diff -r 02934b0d661b src/java.base/unix/native/include/jni_md.h > >> > --- a/src/java.base/unix/native/include/jni_md.h Wed May 30 > 14:46:28 > >> > 2018 +0200 > >> > +++ b/src/java.base/unix/native/include/jni_md.h Tue Jun 05 > 12:10:41 > >> > 2018 +0900 > >> > @@ -29,7 +29,8 @@ > >> > #ifndef __has_attribute > >> > #define __has_attribute(x) 0 > >> > #endif > >> > -#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) > && > >> > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) > >> > +#if (defined(__GNUC__) && ((__GNUC__ > 4) || (__GNUC__ == 4) > && > >> > (__GNUC_MINOR__ > 2))) || __has_attribute(visibility) \ > >> > + || (defined(_AIX) && (defined(__xlC__) && (__xlC__ >= 0xD01))) > >> > #ifdef ARM > >> > #define JNIEXPORT > >> > __attribute__((externally_visible,visibility("default"))) > >> > #define JNIIMPORT > >> > __attribute__((externally_visible,visibility("default"))) > >> > ================================ > >> > > >> > If "osSupport.hpp" was included, XLC++ compiler complained. > >> > I could not understand, which part was invalid... > >> > I'm not sure, "osSupport.hpp" is really required on > >> > NativeImageBuffer.cpp or not... > >> > > >> > I added __xlC__ macro to determine XLC/C++'s version# into jni_md.h. > >> > [1] > >> > 0xD01 means 13.1 by hexadecimal. > >> > > >> > I checked symbol table by "dump -Tv -X64". [2] > >> > It seemed it was fine, some symbols were hidden. > >> > > >> > Does someone review my code? > >> > > >> > [1] > >> > > >> > https://www.ibm.com/support/knowledgecenter/en/SSGH2K_13.1.3/com.i > >> > bm.xlc1313.aix.doc/compiler_ref/xlmacros.html > >> > [2] > >> > https://www.ibm.com/developerworks/aix/library/au-aix-symbol- > >> > visibility/index.html > >> > > >> > On 2018-06-01 21:12, Baesken, Matthias wrote: > >> > > Hi , my change 8202322 just handled the fact that the > >> > > visibility - flags are not supported with xlc 12.1 , so setting > >> > > them generated a TON of compile - time warnings . > >> > > > >> > > The introduction of the "-qvisibility=hidden" came with the > >> > > mapfile removal changes : > >> > > > >> > > 8200358: Remove mapfiles for JDK executables > >> > > http://hg.openjdk.java.net/jdk/jdk/rev/210cf224b690 > >> > > > >> > > 8200178: Remove mapfiles for JDK native libraries > >> > > http://hg.openjdk.java.net/jdk/jdk/rev/396ea30afbd5 > >> > > > >> > > I guess it might need further testing+adjustments to make the > >> > > "visibility hiding" work nicely with XLC13 , but currently we > >> > > build only with XLC12 . > >> > > > >> > > As a workaround you might want to remove the "- > qvisibility=hidden" > >> > > setting for XLC 13 as well , like I did for XLC12 with the change > >> > > 8202322 . > >> > > > >> > > > >> > > Best regards, Matthias > >> > > > >> > > > >> > > > >> > > > >> > >> -----Original Message----- > >> > >> From: Langer, Christoph > >> > >> Sent: Freitag, 1. Juni 2018 10:57 > >> > >> To: Ichiroh Takiguchi > >> > >> Cc: Baesken, Matthias ; 'build- > >> > >> dev at openjdk.java.net' ; ppc-aix- > port- > >> > >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; > Lindenmaier, > >> > >> Goetz > >> > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > >> > >> on xlc 12.1 > >> > >> > >> > >> Hi Ichiroh, > >> > >> > >> > >> we do not use the XLC 13 compiler on AIX yet here at SAP and I > believe > >> > >> nobody of my colleagues has played with it yet. So you are on a new > >> > >> playground here ?? > >> > >> > >> > >> However, I believe the idea in OpenJDK with the abolition of map > files > >> > >> is that > >> > >> symbols should be invisible externally unless they are declared > >> > >> exported, > >> > >> e.g. JNIEXPORT. So I would think "-qvisibility=hidden" should be the > >> > >> correct > >> > >> default and whatever JNIEXPORT expands to should contain the right > >> > >> attributes to get that symbol visible. > >> > >> > >> > >> Can you check if either my assumption is completely wrong, > JNIEXPORT > >> > >> does > >> > >> not expand to the right thing, XLC 13 has a bug or maybe just sume > >> > >> specific > >> > >> required symbols are not declared correctly? > >> > >> > >> > >> Best regards > >> > >> Christoph > >> > >> > >> > >> > -----Original Message----- > >> > >> > From: Ichiroh Takiguchi [mailto:takiguc at linux.vnet.ibm.com] > >> > >> > Sent: Donnerstag, 31. Mai 2018 09:55 > >> > >> > To: Langer, Christoph > >> > >> > Cc: Baesken, Matthias ; 'build- > >> > >> > dev at openjdk.java.net' ; ppc-aix- > port- > >> > >> > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; > >> Lindenmaier, > >> > >> > Goetz > >> > >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > on > >> xlc > >> > 12.1 > >> > >> > > >> > >> > Hello. > >> > >> > 8202322 was integrated into jdk-11+15. > >> > >> > I'm using XLC 13.1.3 on AIX 7.1.4. > >> > >> > Build was failed because of "-qvisibility=hidden" on > >> > >> > make/lib/LibCommon.gmk. > >> > >> > According to "XL C/C++ for AIX 13.1.3" documentation [1], > >> > >> > "-qvisibility=hidden" cannot create shared libraries entry points. > >> > >> > For example, libverify.so was there, but entry points were not > >> resolved > >> > >> > by "-lverify" option. > >> > >> > I think it should be "-qvisibility=default" (I tried, it worked) > >> > >> > or "-qvisibility=protected" (I had not tried) ? > >> > >> > I'm not familiar with -qvisibility option, but I'd like to find out > >> > >> > right way. > >> > >> > > >> > >> > [1] > >> > >> > > >> > >> > >> > > >> > https://www.ibm.com/support/knowledgecenter/SSGH3R_13.1.3/com.ibm. > >> > >> > xlcpp1313.aix.doc/compiler_ref/opt_visibility.html > >> > >> > > >> > >> > On 2018-05-16 16:08, Langer, Christoph wrote: > >> > >> > > Hi Matthias, > >> > >> > > > >> > >> > > yes, reviewed. > >> > >> > > > >> > >> > > Best regards > >> > >> > > Christoph > >> > >> > > > >> > >> > > From: Baesken, Matthias > >> > >> > > Sent: Mittwoch, 16. Mai 2018 09:06 > >> > >> > > To: Langer, Christoph ; > >> > >> > > 'build-dev at openjdk.java.net' ; > >> > >> > > ppc-aix-port-dev at openjdk.java.net; core-libs- > >> dev at openjdk.java.net > >> > >> > > Cc: Lindenmaier, Goetz > >> > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > on > >> > >> > > xlc 12.1 > >> > >> > > > >> > >> > > Hi Christoph can I add you as second reviewer (other reviewer > was > >> > >> > > Erik Joelsson) can push the change ? > >> > >> > > > >> > >> > > Best regards, Matthias > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > From: Langer, Christoph > >> > >> > > Sent: Donnerstag, 26. April 2018 16:38 > >> > >> > > To: Baesken, Matthias > >> > >> > > > >> >; > >> > >> > > 'build-dev at openjdk.java.net' > >> > >> > > >> > dev at openjdk.java.net>>; > >> > >> > > ppc-aix-port-dev at openjdk.java.net >> > >> > dev at openjdk.java.net>; > >> > >> > > core-libs-dev at openjdk.java.net >> > >> dev at openjdk.java.net> > >> > >> > > Cc: Simonis, Volker > >> > >> > > > > >> > >> > > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > on > >> > >> > > xlc 12.1 > >> > >> > > > >> > >> > > Hi Matthias, > >> > >> > > > >> > >> > > to me the change in principal looks good. > >> > >> > > > >> > >> > > I'm wondering if it is possible to do a comparison like xlc < 13 (e.g. > >> > >> > > extract major number before the first dot, then compare > >> numerically) > >> > - > >> > >> > > but maybe it is too complicated and the current single version > >> > compare > >> > >> > > suits our needs ? > >> > >> > > > >> > >> > > Best regards > >> > >> > > Christoph > >> > >> > > > >> > >> > > From: Baesken, Matthias > >> > >> > > Sent: Donnerstag, 26. April 2018 16:14 > >> > >> > > To: 'build-dev at openjdk.java.net' > >> > >> > > >> > dev at openjdk.java.net>>; > >> > >> > > ppc-aix-port-dev at openjdk.java.net >> > >> > dev at openjdk.java.net>; > >> > >> > > core-libs-dev at openjdk.java.net >> > >> dev at openjdk.java.net> > >> > >> > > Cc: Langer, Christoph > >> > >> > > > >; > >> > >> Simonis, > >> > >> > > Volker > >> > > >> > >> > > Subject: RFR : 8202322: AIX: symbol visibility flags not support on > xlc > >> > >> > > 12.1 > >> > >> > > > >> > >> > > Hello , could you please review this small adjustment to the > symbol > >> > >> > > visibility compilation settings on AIX ? > >> > >> > > Currently we use XLC 12.1 to compile JDK on AIX . > >> > >> > > > >> > >> > > However XLC 12.1 does not support the "-qvisibility=hidden" > >> > >> > > setting currently set on AIX. > >> > >> > > It was introduced with XLC 13.1 . Christoph found some info > about it > >> > >> > > here : > >> > >> > > > >> > >> > > https://www.ibm.com/developerworks/aix/library/au-aix- > symbol- > >> > >> > visibility-part2/index.html > >> > >> > > > >> > >> > > Setting it only generates hundreds of warnings in the build log , > >> > >> > > warnings look like this : > >> > >> > > XlC12.1 > >> > >> > > > >> > >> > > bash-4.4\$ xlC -qversion > >> > >> > > IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72) > >> > >> > > Version: 12.01.0000.0019 > >> > >> > > > >> > >> > > bash-4.4\$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> > >> > > 1506-173 (W) Option visibility=hidden is not valid. Enter xlC for list > >> > >> > > of valid options. > >> > >> > > > >> > >> > > Compare to XLC13.1 > >> > >> > > > >> > >> > > bash-3.00\$ xlC -qversion > >> > >> > > IBM XL C/C++ for AIX, V13.1 (5725-C72, 5765-J07) > >> > >> > > Version: 13.01.0000.0008 > >> > >> > > bash-3.00\$ xlC -qvisibility=default sizeof.c -o sizeof_aixxlc > >> > >> > > bash-3.00\$ xlC -qvisibility=hidden sizeof.c -o sizeof_aixxlc > >> > >> > > > >> > >> > > > >> > >> > > So it is better to avoid setting these flags when using xlc12.1 . > >> > >> > > Please review : > >> > >> > > > >> > >> > > Bug : > >> > >> > > > >> > >> > > https://bugs.openjdk.java.net/browse/JDK-8202322 > >> > >> > > > >> > >> > > Change : > >> > >> > > > >> > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202322/ > >> > >> > > > >> > >> > > > >> > >> > > Best regards, Matthias From sean.coffey at oracle.com Fri Jun 8 07:36:30 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 8 Jun 2018 08:36:30 +0100 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> <091861e7d9c94e27af146eed3e57d3eb@sap.com> Message-ID: Not sure if this has been tested. Don't you need to escape the printing of the \0 character ? errorMessage = "illegal character \0 found"; regards, Sean. On 08/06/2018 08:14, Langer, Christoph wrote: > > Hi Matthias, > > this looks good. Reviewed from my end. > > Best regards > > Christoph > > *From:*Baesken, Matthias > *Sent:* Freitag, 8. Juni 2018 08:51 > *To:* Langer, Christoph ; > core-libs-dev at openjdk.java.net; 'sean.coffey at oracle.com' > > *Cc:* Lindenmaier, Goetz > *Subject:* RE: [RFR] 8204539: improve error messages in matchJavaTZ > [windows] > > Hi,? looks like I uploaded a wrong version of the patch. > > Updated webrev is here? : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.1/ > > > I followed the advice of Sean and? removed? the? mapFileName?? from > the output because it is usually a static name . > > Updated webrev was going through our internal build/tests . > > Best regards, Matthias > > *From:*Langer, Christoph > *Sent:* Donnerstag, 7. Juni 2018 13:52 > *To:* Baesken, Matthias >; core-libs-dev at openjdk.java.net > > *Cc:* Lindenmaier, Goetz > > *Subject:* RE: [RFR] 8204539: improve error messages in matchJavaTZ > [windows] > > Hi Matthias, > > in line 527, where the actual output is done, I think you would need > to replace variable ?message? with ?outputMessage?, otherwise I guess > it won?t compile?? > > Also, mapFileName can?t be used at this place, because it has already > been free?ed there. > > But in general the additions make sense and will make it easier to > find issues in the tzmappings file. > > Best regards > > Christoph > > *From:*Baesken, Matthias > *Sent:* Donnerstag, 7. Juni 2018 13:35 > *To:* core-libs-dev at openjdk.java.net > > *Cc:* Langer, Christoph >; Lindenmaier, Goetz > > > *Subject:* [RFR] 8204539: improve error messages in matchJavaTZ [windows] > > Hi, could you please? review this small? change that improves? the > error messages in matchJavaTZ . > > A reason of the failure is added to the message , and also? the offset > where the error happened . > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8204539 > > Webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ > > > Thanks, Matthias > From magnus.ihse.bursie at oracle.com Fri Jun 8 08:50:37 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 8 Jun 2018 10:50:37 +0200 Subject: RFR (XL): JDK-8204572 SetupJdkLibrary should setup SRC and -I flags automatically In-Reply-To: References: Message-ID: <4ff74b09-ec9f-0a80-31d5-49039d5e1e8c@oracle.com> On 2018-06-07 23:20, Erik Joelsson wrote: > Hello Magnus, > > Very nice refactoring! Thanks! > > JdkNativeCompilation.gmk > line 126-127 looks a bit long. There is an extra space on 126. Also, > why not addprefix for adding -I instead of clunky foreach? Not that I > care greatly, but I usually prefer that construct. Yeah, I agree, addprefix is better. I just forgot about it; foreach is a nice allround tool. Updated webrev: http://cr.openjdk.java.net/~ihse/JDK-8204572-autodetect-SRC-and-headers-dirs/webrev.02/ (Only changes is in JdkNativeCompilation.gmk, as per above comments). /Magnus > > Otherwise looks good. > > /Erik > > > On 2018-06-07 13:22, Magnus Ihse Bursie wrote: >> This change needs some background information. >> >> I've been working on simplifying and streamlining the compilation of >> native libraries of the JDK. Previously, I introduced the >> SetupJdkLibrary function, and started to get a better control of >> compiler flags. This patch continues on both paths. >> >> The original intent of this change, which I naively thought was going >> to be much simpler than it turned out :-) was to separate the -I >> flags (location of header files) from other flags, and to generate >> these automatically, wherever possible. Since we have a standard way >> of locating native code, and most libraries just want to have an -I >> path to their own source base and the generated Java header, we >> should be able to provide this in SetupJdkLibrary. >> >> This turned out to be closely related to SetupJdkLibrary being able >> to discover the location of the SRC directories itself, using >> "convention over configuration" and assuming that the library >> "libfoo" for "java.module" would be located in >> java.module/*/native/libfoo. >> >> While this sounds simple in theory, when actually trying to implement >> this I of course ran into all the places where some special handling >> was indeed needed. So even if like 90% of all libraries were simple >> to get to build using automated discovery of source and header >> directories, the 10% that did not caused me much more headaches than >> I had anticipated. On the other hand, now that I've sorted out all >> those places, the few remaining odd solutions is clearly documented >> and not just something that "just happens" due to strange >> configurations. >> >> One file deserves mentioning specifically: Awt2dLibraries.gmk. The >> java.desktop libraries are unfortunately quite entangled with each >> other, and do not readily follow the patterns that are used elsewhere >> in the code base. So it might just look like the file has just gone >> from one state of messiness, to another, which would hardly be an >> improvement. :-( I would still argue that the new messiness is >> better: It is now much clearer in what ways the libraries diverge >> from our standard assumption, and what course of action needs to be >> taken to minimize these differences. (Which is something I believe >> should be done -- these issues are not just cosmetic but is the root >> of most of the issues we always see for these libraries, when >> upgrading compilers, etc.) >> >> During this change, I noticed that not all native libraries include >> the proper generated header file. This is a dangerous coding >> practice, since a change in the Java part of the interface might not >> get picked up properly in the native part. I've added the missing >> includes that I've detected, and due to these changes, I'm also >> including the component teams in what is really only a build change. >> As can be seen for jdk.crypto.mscapi; there had indeed been changes >> that needed correcting. >> >> Since this is (basically) a pure build change, my gold standard here >> has been the build compare script. In essence, the build output prior >> to my change and with this change are 100% identical. In truth, this >> is a bit too strong a claim. A few changes has occurred, but none of >> them should matter. Here's a breakdown of the compare.sh results: >> >> * Windows-x64: >> >> No differences at all. >> >> * Solaris: >> >> Two libraries are reported to differ: libsaproc.so and >> libfontmanager.so, both with a disass diff on ~700 bytes. Analyzing >> this, I found that the object files used to link these two libraries >> has no disass differences. They have a slight binary difference and a >> difference in size, due to the include paths being different (and >> this is stored in the .o file, which makes it different). Somehow >> this apparently triggers the linker to generate a slightly different >> code in a few places, using a different register or so. (Weird...) >> >> * MacOS: >> >> Two libraries are reported to differ: libjava.dylib and >> libmlib_image.dylib, both of them just reported as a binary diff (no >> symbol, disass or fulldump differences). This is not really >> unsuspected, but I analyzed it anyway. >> >> I found that for libjava.dylib, a single .o file was different. This >> one was actually picked up from closed sources, and are not really >> relevant for OpenJDK. Anyway, the reason for the difference was the >> same as for the Solaris libs; the include paths had changes, which >> caused a binary diff. >> >> For libmlib_image.dylib, the link order had changed causing the noted >> binary difference. >> >> * Linux: >> >> On linux, the compare script noted differences for libextnet, >> libjava, libmlib_image, libprefs, libsaproc, libsplashscreen and >> libsunec. >> >> The differences for libextnet, libprefs, libsplashscreen and libsunec >> turned out to be caused by the added #include of the generated Java >> headers. This caused binary differences (reasonably), and for some >> odd reason also a symbol difference in java_awt_SplashScreen.o >> (clazz.10057 and mid.10058 were replaced by clazz.10015 and >> mid.10016). I can't claim to understand this, but I'm assuming it's >> some kind of generated code. >> >> libsaproc and libjava changes was caused by closed source changes, >> and is therefore not relevant to OpenJDK. >> >> For libmlib_image.dylib, the link order had changed causing the noted >> binary difference, as on MacOS. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204572 >> WebRev: >> http://cr.openjdk.java.net/~ihse/JDK-8204572-autodetect-SRC-and-headers-dirs/webrev.01 >> >> /Magnus >> > From matthias.baesken at sap.com Fri Jun 8 11:28:12 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 8 Jun 2018 11:28:12 +0000 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: <3922fa39aa4749c1919b54e467541f7c@sap.com> References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> <091861e7d9c94e27af146eed3e57d3eb@sap.com> <3922fa39aa4749c1919b54e467541f7c@sap.com> Message-ID: <403aead86f1346a2bfb9df8a98be436d@sap.com> Hi Goetz/Christoph, thanks for the reviews . However of course Sean is absolutely correct about the null character message output. Updated the null character related message output : http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.2/ Best regards, Matthias > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Freitag, 8. Juni 2018 09:34 > To: Baesken, Matthias ; Langer, Christoph > ; core-libs-dev at openjdk.java.net; > 'sean.coffey at oracle.com' > Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ > [windows] > > Hi Matthias, > > thanks for adding this information, looks good! > > Best regards, > Goety. > > > -----Original Message----- > > From: Baesken, Matthias > > Sent: Freitag, 8. Juni 2018 08:51 > > To: Langer, Christoph ; core-libs- > > dev at openjdk.java.net; 'sean.coffey at oracle.com' > > > > Cc: Lindenmaier, Goetz > > Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ > > [windows] > > > > Hi, looks like I uploaded a wrong version of the patch. > > > > Updated webrev is here : > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.1/ > > > > > > > > I followed the advice of Sean and removed the mapFileName from the > > output because it is usually a static name . > > > > Updated webrev was going through our internal build/tests . > > > > > > > > > > > > Best regards, Matthias > > > > > > > > > > > > From: Langer, Christoph > > Sent: Donnerstag, 7. Juni 2018 13:52 > > To: Baesken, Matthias ; core-libs- > > dev at openjdk.java.net > > Cc: Lindenmaier, Goetz > > Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ > > [windows] > > > > > > > > Hi Matthias, > > > > > > > > in line 527, where the actual output is done, I think you would need to > > replace variable 'message' with 'outputMessage', otherwise I guess it > won't > > compile?? > > > > > > > > Also, mapFileName can't be used at this place, because it has already been > > free'ed there. > > > > > > > > But in general the additions make sense and will make it easier to find > issues > > in the tzmappings file. > > > > > > > > Best regards > > > > Christoph > > > > > > > > From: Baesken, Matthias > > Sent: Donnerstag, 7. Juni 2018 13:35 > > To: core-libs-dev at openjdk.java.net > dev at openjdk.java.net> > > Cc: Langer, Christoph > >; Lindenmaier, Goetz > > > > > Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] > > > > > > > > Hi, could you please review this small change that improves the error > > messages in matchJavaTZ . > > > > A reason of the failure is added to the message , and also the offset where > > the error happened . > > > > > > > > > > > > Bug : > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8204539 > > > > > > > > > > > > Webrev : > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ > > > > > > > > > > > > Thanks, Matthias From sean.coffey at oracle.com Fri Jun 8 12:05:13 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 8 Jun 2018 13:05:13 +0100 Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] In-Reply-To: <403aead86f1346a2bfb9df8a98be436d@sap.com> References: <67fa98e2ed454a208af23a7a56e4627b@sap.com> <091861e7d9c94e27af146eed3e57d3eb@sap.com> <3922fa39aa4749c1919b54e467541f7c@sap.com> <403aead86f1346a2bfb9df8a98be436d@sap.com> Message-ID: Looks good. Thanks for adding this improvement. Regards, Sean. On 08/06/18 12:28, Baesken, Matthias wrote: > Hi Goetz/Christoph, thanks for the reviews . > However of course Sean is absolutely correct about the null character message output. > Updated the null character related message output : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.2/ > > Best regards, Matthias > > >> -----Original Message----- >> From: Lindenmaier, Goetz >> Sent: Freitag, 8. Juni 2018 09:34 >> To: Baesken, Matthias ; Langer, Christoph >> ; core-libs-dev at openjdk.java.net; >> 'sean.coffey at oracle.com' >> Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ >> [windows] >> >> Hi Matthias, >> >> thanks for adding this information, looks good! >> >> Best regards, >> Goety. >> >>> -----Original Message----- >>> From: Baesken, Matthias >>> Sent: Freitag, 8. Juni 2018 08:51 >>> To: Langer, Christoph ; core-libs- >>> dev at openjdk.java.net; 'sean.coffey at oracle.com' >>> >>> Cc: Lindenmaier, Goetz >>> Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ >>> [windows] >>> >>> Hi, looks like I uploaded a wrong version of the patch. >>> >>> Updated webrev is here : >>> >>> >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8204539.1/ >>> >>> >>> >>> I followed the advice of Sean and removed the mapFileName from the >>> output because it is usually a static name . >>> >>> Updated webrev was going through our internal build/tests . >>> >>> >>> >>> >>> >>> Best regards, Matthias >>> >>> >>> >>> >>> >>> From: Langer, Christoph >>> Sent: Donnerstag, 7. Juni 2018 13:52 >>> To: Baesken, Matthias ; core-libs- >>> dev at openjdk.java.net >>> Cc: Lindenmaier, Goetz >>> Subject: RE: [RFR] 8204539: improve error messages in matchJavaTZ >>> [windows] >>> >>> >>> >>> Hi Matthias, >>> >>> >>> >>> in line 527, where the actual output is done, I think you would need to >>> replace variable 'message' with 'outputMessage', otherwise I guess it >> won't >>> compile?? >>> >>> >>> >>> Also, mapFileName can't be used at this place, because it has already been >>> free'ed there. >>> >>> >>> >>> But in general the additions make sense and will make it easier to find >> issues >>> in the tzmappings file. >>> >>> >>> >>> Best regards >>> >>> Christoph >>> >>> >>> >>> From: Baesken, Matthias >>> Sent: Donnerstag, 7. Juni 2018 13:35 >>> To: core-libs-dev at openjdk.java.net >> dev at openjdk.java.net> >>> Cc: Langer, Christoph >> >; Lindenmaier, Goetz >>> > >>> Subject: [RFR] 8204539: improve error messages in matchJavaTZ [windows] >>> >>> >>> >>> Hi, could you please review this small change that improves the error >>> messages in matchJavaTZ . >>> >>> A reason of the failure is added to the message , and also the offset where >>> the error happened . >>> >>> >>> >>> >>> >>> Bug : >>> >>> >>> >>> https://bugs.openjdk.java.net/browse/JDK-8204539 >>> >>> >>> >>> >>> >>> Webrev : >>> >>> >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8204539/ >>> >>> >>> >>> >>> >>> Thanks, Matthias From james.laskey at oracle.com Fri Jun 8 12:10:50 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 8 Jun 2018 09:10:50 -0300 Subject: RFR JDK-8204229: Formatter and String.format ignore the width with the percent modifier (%5%) In-Reply-To: <5B19EAE7.9030509@oracle.com> References: <5B19EAE7.9030509@oracle.com> Message-ID: +1 > On Jun 7, 2018, at 11:33 PM, Xueming Shen wrote: > > Hi, > > Please help review the change for JDK-8204229. It appears to be a overlook > in the implementation. We do have a method print(String, Locale) that adjust > the "padding spaces" > > issue: https://bugs.openjdk.java.net/browse/JDK-8204229 > webrev: http://cr.openjdk.java.net/~sherman/8204229/webrev > > Thanks, > Sherman From srinivas.dama at oracle.com Fri Jun 8 13:02:58 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Fri, 8 Jun 2018 06:02:58 -0700 (PDT) Subject: RFR: 8196993: Resolve disabled warnings for libunpack Message-ID: Hi, Please review http://cr.openjdk.java.net/~sdama/8196993/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8196993 Regards, Srinivas From roger.riggs at oracle.com Fri Jun 8 14:04:26 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Fri, 8 Jun 2018 10:04:26 -0400 Subject: [11] RFR: 8202088: Japanese new era implementation In-Reply-To: <2bf7e4b1-28de-5dcb-5027-bfdb64573d5d@oracle.com> References: <079f3232-1cc0-1860-4dc4-671f71827f34@oracle.com> <76102e2c-f2ad-f98c-8acd-9709551c2dd5@oracle.com> <2bf7e4b1-28de-5dcb-5027-bfdb64573d5d@oracle.com> Message-ID: Hi Naoto, test/jdk/java/util/Calendar/SupplementalJapaneseEraTest.java: 125 missing spaces around "+". Please rename the test to have functional name. (test/jdk/java/util/Calendar/Bug8202088.java) We're trying to get away for uninformative bug numbers for tests. No further review from me needed with those changes. Thanks, Roger On 5/25/18 4:13 PM, naoto.sato at oracle.com wrote: > Thanks for the review, Roger and Max. I modified the webrev according > to your suggestions. Also, from an internal comment, I added a test > case (Bug8202088.java) for the modification below. Here is the updated > webrev: > > http://cr.openjdk.java.net/~naoto/8202088/webrev.04/ > > Max, > > --- > BTW, why must 8202088 be pushed before Apr 2019? If someone try to > show the Japanese calendar date of a day in 2020 now, do you really > expect them to see "NewEra year 2"? (or something like it, I am not > sure). > --- > > I think so, given that they understand "NewEra" is a placeholder. > Actually it was requested by customers that we implement the era way > before the actual transition so that their applications should have > been tested accordingly with the new era ahead of time (let alone we > need to backport it to jdk8u/etc. lines) > > Naoto > > On 5/24/18 8:45 AM, Naoto Sato wrote: >> Found an issue on retrieving the localized era name for >> java.util.Calendar. The reason is that even we provide the l10n in >> our own resource bundles, the current CLDR does not provide the name >> (duh!). Added a fallback to COMPAT provider in such a case. >> >> Here is the updated webrev: >> >> http://cr.openjdk.java.net/~naoto/8202088/webrev.03/ >> >> Naoto >> >> On 5/18/18 3:26 PM, naoto.sato at oracle.com wrote: >>> Hi Roger, thank you for the comments. >>> >>> On 5/18/18 11:11 AM, Roger Riggs wrote: >>>> Hi Naoto, >>>> >>>> Is there a reference to the official description or anticipation of >>>> the new Era? >>> >>> AFAIK, the most recent information was for Japanese Govt. to >>> announce the new era name one month prior to the ascension. This is >>> indeed the reason I decided to introduce the placeholder. >>> >>>> >>>> JapaneseImperialCalendar: 134 NEWERA = 5;?? (The real name can also >>>> be defined later; but still might be more unique as ERA_MAY_1_2019.) >>> >>> I wanted to keep the name "NEWERA" for the convenience when they are >>> to be replaced with the real name. I changed the access modifier to >>> "private", though. >>> >>>> >>>> Syntax style: >>>> >>>> ??- TCKJapaneseChronology:692: align the? columns of decimal values. >>>> >>>> ??- TestJapaneseChronology:61-62: space before the '}' brackets >>>> ???? :89: extra space before '}'? //? inconsistent within the file >>>> but local consistency is good >>>> >>>> TestUmmAlQuraChronology: there might be test dates that would not >>>> require more changes later when the era name changes. >>> >>> Fixed as suggested. The updated webrev is here: >>> >>> http://cr.openjdk.java.net/~naoto/8202088/webrev.02/ >>> >>> Naoto >>> >>>> >>>> Regards, Roger >>>> >>>> >>>> On 5/17/18 4:31 PM, Naoto Sato wrote: >>>>> Hi, >>>>> >>>>> Please review the changes to the subject issue: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8202088 >>>>> >>>>> The proposed change is located at: >>>>> >>>>> http://cr.openjdk.java.net/~naoto/8202088/webrev.01/ >>>>> >>>>> This is the implementation part of the upcoming Japanese new era, >>>>> starting from May 1st, 2019. Current anticipation is that the new >>>>> name won't be known till one month prior to the beginning of the >>>>> era. So it's not possible to make changes to the JDK with the >>>>> proper name. Instead, here we are going to implement the new era >>>>> with a place holder name which will be immediately replaced with >>>>> the proper name once it's known. The CSR is currently under review: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8202336 >>>>> >>>>> Naoto >>>> From bob.vandette at oracle.com Fri Jun 8 15:03:23 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 8 Jun 2018 11:03:23 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <5B19A3F0.1010804@oracle.com> <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> Message-ID: <4B4CBD97-1081-4DB6-871F-5BF292BF4DD0@oracle.com> I didn?t actually have any ERROR_MARGIN problems during testing. I had issues with the testCpuConsumption test in http://cr.openjdk.java.net/~bobv/8203357/webrev.01/test/lib/jdk/test/lib/containers/cgroup/MetricsTester.java.html I had to initialize the cpu usage values during setup rather than inside the test to ensure that sufficient cpu usage had occurred by the time the test was run. The original code executed and received the same values after attempting to exec a linux utility. My change uses the time taken to run several tests instead. This seems to have eliminated any intermittent failures. Bob. > On Jun 8, 2018, at 12:30 AM, Harsha Wardhana B wrote: > > [Replying to all mailing-lists] > Hi Misha, > > The ERROR_MARGIN in tests was introduced to make the tests stable. There are times where metric values (specifically CPU usage) can change drastically in between two reads. The metrics value got from the API and the cgroup file can be different and 0.1 ERROR_MARGIN should take care of that, though at times even that may not be enough. Hence the CPU usage related tests only print a warning if ERROR_MARGIN is exceeded. > > Thanks > Harsha > > On Friday 08 June 2018 03:00 AM, Mikhailo Seledtsov wrote: >> Hi Bob, >> >> I looked at the tests. In general they look good. I am a bit concerned about the use of ERROR_MARGIN in one of the tests. We need to make sure that the tests are stable, and do not produce intermittent failures. >> >> >> Thank you, >> Misha >> >> On 6/7/18, 10:43 AM, Bob Vandette wrote: >>> Can I get one more reviewer for this RFE so I can integrate it? >>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> Mandy Chung has reviewed this change. >>> >>> I?ve run Mach5 hotspot and core lib tests. >>> >>> I?ve reviewed the tests which were written by Harsha Wardhana >>> >>> I filed a CSR for the command line change and it?s now approved and closed. >>> >>> Thanks, >>> Bob. >>> >>> >>>> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >>>> >>>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>>> information. See the sample output below: >>>> >>>> RFE: Container Metrics >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>> >>>> >>>> This commit will also include a fix for the following bug. >>>> >>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>> >>>> SAMPLE USAGE and OUTPUT: >>>> >>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>> ./java -XshowSettings:system >>>> Operating System Metrics: >>>> Provider: cgroupv1 >>>> Effective CPU Count: 4 >>>> CPU Period: 100000 >>>> CPU Quota: -1 >>>> CPU Shares: -1 >>>> List of Processors, 4 total: >>>> 4 5 6 7 >>>> List of Effective Processors, 4 total: >>>> 4 5 6 7 >>>> List of Memory Nodes, 2 total: >>>> 0 1 >>>> List of Available Memory Nodes, 2 total: >>>> 0 1 >>>> CPUSet Memory Pressure Enabled: false >>>> Memory Limit: 256.00M >>>> Memory Soft Limit: Unlimited >>>> Memory& Swap Limit: 512.00M >>>> Kernel Memory Limit: Unlimited >>>> TCP Memory Limit: Unlimited >>>> Out Of Memory Killer Enabled: true >>>> >>>> TEST RESULTS: >>>> >>>> testing runtime container APIs >>>> Directory "JTwork" not found: creating >>>> Passed: runtime/containers/cgroup/PlainRead.java >>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>> Passed: runtime/containers/docker/TestCPUSets.java >>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>> Passed: runtime/containers/docker/TestMisc.java >>>> Test results: passed: 6 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing jdk.internal.platform APIs >>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>> Test results: passed: 4 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing -XshowSettings:system launcher option >>>> Passed: tools/launcher/Settings.java >>>> Test results: passed: 1 >>>> >>>> >>>> Bob. >>>> >>>> > From mikhailo.seledtsov at oracle.com Fri Jun 8 15:31:52 2018 From: mikhailo.seledtsov at oracle.com (mikhailo) Date: Fri, 8 Jun 2018 08:31:52 -0700 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <5B19A3F0.1010804@oracle.com> <78ad68bd-0020-836f-7511-2b7a49dc6d73@oracle.com> Message-ID: <4f9ac7ba-5a81-c9e4-feaa-42023d9f518f@oracle.com> Hi Harsha, ? Thank you for the explanation, makes sense to me. Please be aware, if a specific test turns out to be unstable in CI testing, it should be problem listed until solution is found to make it more stable. If the test is highly intermittent (fails intermittently but rarely) then it should be tagged with intermittent keyword. Overall tests look good to me, Thank you, Misha On 06/07/2018 09:30 PM, Harsha Wardhana B wrote: > [Replying to all mailing-lists] > Hi Misha, > > The ERROR_MARGIN in tests was introduced to make the tests stable. > There are times where metric values (specifically CPU usage) can > change drastically in between two reads. The metrics value got from > the API and the cgroup file can be different and 0.1 ERROR_MARGIN > should take care of that, though at times even that may not be enough. > Hence the CPU usage related tests only print a warning if ERROR_MARGIN > is exceeded. > > Thanks > Harsha > > On Friday 08 June 2018 03:00 AM, Mikhailo Seledtsov wrote: >> Hi Bob, >> >> ? I looked at the tests. In general they look good. I am a bit >> concerned about the use of ERROR_MARGIN in one of the tests. We need >> to make sure that the tests are stable, and do not produce >> intermittent failures. >> >> >> Thank you, >> Misha >> >> On 6/7/18, 10:43 AM, Bob Vandette wrote: >>> Can I get one more reviewer for this RFE so I can integrate it? >>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> Mandy Chung has reviewed this change. >>> >>> I?ve run Mach5 hotspot and core lib tests. >>> >>> I?ve reviewed the tests which were written by Harsha Wardhana >>> >>> I filed a CSR for the command line change and it?s now approved and >>> closed. >>> >>> Thanks, >>> Bob. >>> >>> >>>> On May 30, 2018, at 3:45 PM, Bob Vandette? >>>> wrote: >>>> >>>> Please review the following RFE which adds an internal API, along >>>> with jtreg tests that provide >>>> access to Docker container configuration data and metrics. In >>>> addition to the API which we hope to >>>> take advantage of in the future with Java Flight Recorder and a JMX >>>> Mbean, I?ve added an additional >>>> option to -XshowSettings:system than dumps out the container or >>>> host cgroup confguration >>>> information.? See the sample output below: >>>> >>>> RFE: Container Metrics >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>> >>>> >>>> This commit will also include a fix for the following bug. >>>> >>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>> >>>> >>>> SAMPLE USAGE and OUTPUT: >>>> >>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>> ./java -XshowSettings:system >>>> Operating System Metrics: >>>> ??? Provider: cgroupv1 >>>> ??? Effective CPU Count: 4 >>>> ??? CPU Period: 100000 >>>> ??? CPU Quota: -1 >>>> ??? CPU Shares: -1 >>>> ??? List of Processors, 4 total: >>>> ??? 4 5 6 7 >>>> ??? List of Effective Processors, 4 total: >>>> ??? 4 5 6 7 >>>> ??? List of Memory Nodes, 2 total: >>>> ??? 0 1 >>>> ??? List of Available Memory Nodes, 2 total: >>>> ??? 0 1 >>>> ??? CPUSet Memory Pressure Enabled: false >>>> ??? Memory Limit: 256.00M >>>> ??? Memory Soft Limit: Unlimited >>>> ??? Memory&? Swap Limit: 512.00M >>>> ??? Kernel Memory Limit: Unlimited >>>> ??? TCP Memory Limit: Unlimited >>>> ??? Out Of Memory Killer Enabled: true >>>> >>>> TEST RESULTS: >>>> >>>> testing runtime container APIs >>>> Directory "JTwork" not found: creating >>>> Passed: runtime/containers/cgroup/PlainRead.java >>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>> Passed: runtime/containers/docker/TestCPUSets.java >>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>> Passed: runtime/containers/docker/TestMisc.java >>>> Test results: passed: 6 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing jdk.internal.platform APIs >>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>> Test results: passed: 4 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing -XshowSettings:system launcher option >>>> Passed: tools/launcher/Settings.java >>>> Test results: passed: 1 >>>> >>>> >>>> Bob. >>>> >>>> > From erik.joelsson at oracle.com Fri Jun 8 15:37:10 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 8 Jun 2018 08:37:10 -0700 Subject: RFR (XL): JDK-8204572 SetupJdkLibrary should setup SRC and -I flags automatically In-Reply-To: <4ff74b09-ec9f-0a80-31d5-49039d5e1e8c@oracle.com> References: <4ff74b09-ec9f-0a80-31d5-49039d5e1e8c@oracle.com> Message-ID: <04d6b9dc-e091-4d63-1816-15846a7b34d3@oracle.com> Looks good. /Erik On 2018-06-08 01:50, Magnus Ihse Bursie wrote: > On 2018-06-07 23:20, Erik Joelsson wrote: >> Hello Magnus, >> >> Very nice refactoring! > Thanks! > >> >> JdkNativeCompilation.gmk >> line 126-127 looks a bit long. There is an extra space on 126. Also, >> why not addprefix for adding -I instead of clunky foreach? Not that I >> care greatly, but I usually prefer that construct. > > Yeah, I agree, addprefix is better. I just forgot about it; foreach is > a nice allround tool. > > Updated webrev: > http://cr.openjdk.java.net/~ihse/JDK-8204572-autodetect-SRC-and-headers-dirs/webrev.02/ > (Only changes is in JdkNativeCompilation.gmk, as per above comments). > > /Magnus > >> >> Otherwise looks good. >> >> /Erik >> >> >> On 2018-06-07 13:22, Magnus Ihse Bursie wrote: >>> This change needs some background information. >>> >>> I've been working on simplifying and streamlining the compilation of >>> native libraries of the JDK. Previously, I introduced the >>> SetupJdkLibrary function, and started to get a better control of >>> compiler flags. This patch continues on both paths. >>> >>> The original intent of this change, which I naively thought was >>> going to be much simpler than it turned out :-) was to separate the >>> -I flags (location of header files) from other flags, and to >>> generate these automatically, wherever possible. Since we have a >>> standard way of locating native code, and most libraries just want >>> to have an -I path to their own source base and the generated Java >>> header, we should be able to provide this in SetupJdkLibrary. >>> >>> This turned out to be closely related to SetupJdkLibrary being able >>> to discover the location of the SRC directories itself, using >>> "convention over configuration" and assuming that the library >>> "libfoo" for "java.module" would be located in >>> java.module/*/native/libfoo. >>> >>> While this sounds simple in theory, when actually trying to >>> implement this I of course ran into all the places where some >>> special handling was indeed needed. So even if like 90% of all >>> libraries were simple to get to build using automated discovery of >>> source and header directories, the 10% that did not caused me much >>> more headaches than I had anticipated. On the other hand, now that >>> I've sorted out all those places, the few remaining odd solutions is >>> clearly documented and not just something that "just happens" due to >>> strange configurations. >>> >>> One file deserves mentioning specifically: Awt2dLibraries.gmk. The >>> java.desktop libraries are unfortunately quite entangled with each >>> other, and do not readily follow the patterns that are used >>> elsewhere in the code base. So it might just look like the file has >>> just gone from one state of messiness, to another, which would >>> hardly be an improvement. :-( I would still argue that the new >>> messiness is better: It is now much clearer in what ways the >>> libraries diverge from our standard assumption, and what course of >>> action needs to be taken to minimize these differences. (Which is >>> something I believe should be done -- these issues are not just >>> cosmetic but is the root of most of the issues we always see for >>> these libraries, when upgrading compilers, etc.) >>> >>> During this change, I noticed that not all native libraries include >>> the proper generated header file. This is a dangerous coding >>> practice, since a change in the Java part of the interface might not >>> get picked up properly in the native part. I've added the missing >>> includes that I've detected, and due to these changes, I'm also >>> including the component teams in what is really only a build change. >>> As can be seen for jdk.crypto.mscapi; there had indeed been changes >>> that needed correcting. >>> >>> Since this is (basically) a pure build change, my gold standard here >>> has been the build compare script. In essence, the build output >>> prior to my change and with this change are 100% identical. In >>> truth, this is a bit too strong a claim. A few changes has occurred, >>> but none of them should matter. Here's a breakdown of the compare.sh >>> results: >>> >>> * Windows-x64: >>> >>> No differences at all. >>> >>> * Solaris: >>> >>> Two libraries are reported to differ: libsaproc.so and >>> libfontmanager.so, both with a disass diff on ~700 bytes. Analyzing >>> this, I found that the object files used to link these two libraries >>> has no disass differences. They have a slight binary difference and >>> a difference in size, due to the include paths being different (and >>> this is stored in the .o file, which makes it different). Somehow >>> this apparently triggers the linker to generate a slightly different >>> code in a few places, using a different register or so. (Weird...) >>> >>> * MacOS: >>> >>> Two libraries are reported to differ: libjava.dylib and >>> libmlib_image.dylib, both of them just reported as a binary diff (no >>> symbol, disass or fulldump differences). This is not really >>> unsuspected, but I analyzed it anyway. >>> >>> I found that for libjava.dylib, a single .o file was different. This >>> one was actually picked up from closed sources, and are not really >>> relevant for OpenJDK. Anyway, the reason for the difference was the >>> same as for the Solaris libs; the include paths had changes, which >>> caused a binary diff. >>> >>> For libmlib_image.dylib, the link order had changed causing the >>> noted binary difference. >>> >>> * Linux: >>> >>> On linux, the compare script noted differences for libextnet, >>> libjava, libmlib_image, libprefs, libsaproc, libsplashscreen and >>> libsunec. >>> >>> The differences for libextnet, libprefs, libsplashscreen and >>> libsunec turned out to be caused by the added #include of the >>> generated Java headers. This caused binary differences (reasonably), >>> and for some odd reason also a symbol difference in >>> java_awt_SplashScreen.o (clazz.10057 and mid.10058 were replaced by >>> clazz.10015 and mid.10016). I can't claim to understand this, but >>> I'm assuming it's some kind of generated code. >>> >>> libsaproc and libjava changes was caused by closed source changes, >>> and is therefore not relevant to OpenJDK. >>> >>> For libmlib_image.dylib, the link order had changed causing the >>> noted binary difference, as on MacOS. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8204572 >>> WebRev: >>> http://cr.openjdk.java.net/~ihse/JDK-8204572-autodetect-SRC-and-headers-dirs/webrev.01 >>> >>> /Magnus >>> >> > From jonathan.halliday at redhat.com Fri Jun 8 10:59:58 2018 From: jonathan.halliday at redhat.com (Jonathan Halliday) Date: Fri, 8 Jun 2018 11:59:58 +0100 Subject: RFC: Experiment in accessing/managing persistent memory from Java In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE74C14@FMSMSX126.amr.corp.intel.com> Message-ID: Hi Paul Looks like we're all on the same page regarding the basic approach of using a small API and making the critical bits intrinsic. We perhaps have some way to go on exactly what that API looks like in terms of the classes and methods, but iterating on it by discussion of a JEP seems like the best way forward. The important thing from my perspective is that so far nobody has come forward with a use case that is not covered by the proposed primitives. So it's a small API, but not too small. As far a tweaks go, we have considered making the low level primitive method / intrinsic just a flushCacheline(base_address), since the arithmetic and loop for writing flush(from,to) in terms of that low level op is something that can be optimized fine by the JIT already. Though that does mean exposing the cache line size to the Java layer, whereas currently it's only visible in the C code. My own background and focus is transactions systems, so I'm more about the speed and fault tolerance than the capacity, but I can see long vs. int being of interest to our Infinispan data grid team and likewise for e.g. Oracle Coherence or databases like Cassandra. OTOH it's not uncommon to prefer moderately sized files and shard over them, which sidesteps the issue. Utility code to assist with fine-grained memory management within the buffer/file may be more useful than support for really large buffers, since they tend to be used with some form of internal block/heap structure anyhow, rather than to hold very large objects. Providing that may be the role of a 3rd party pure Java library like PCJ though, rather than something we want in the JDK itself at this early stage. The researcher in me is kinda interested in how much of the memory allocation and GC code can be re-purposed here though... What's the intended timeline on long buffer indexing at present? My feeling is a pmem API JEP is probably targeting around JDK 13, but we're flexible on that. We may also want to look at related enhancements like unmapping buffers. I think those pieces are sufficient decoupled that they won't be dependencies for the pmem API though, unlike other factors such as the availability of test hardware. Regards Jonathan. On 08/06/18 01:42, Paul Sandoz wrote: > Hi Andrew, Jonathan, > > Sandhya gave an overview to a few of us Oracle folks. I agree with what Sandhya says regarding the API, a small surface, and on pursuing an unsafe intrinsic. I like it and would encourage the writing of a draft JEP, especially to give this visibility. > > I expect this will be beneficial for experimentation with the Panama foreign API where we can use a Pointer to reference into a byte buffer and scribble on it. Further, i hope this work may also benefit the persistent collections effort (PCJ). > > > It intersects with https://bugs.openjdk.java.net/browse/JDK-8153111 ((bf) Allocating ByteBuffer on heterogeneous memory), which is attempting to be more generic. > > We might also need to increase the velocity on https://bugs.openjdk.java.net/browse/JDK-8180628 (retrofit direct buffer support for size beyond gigabyte scales), and i would be very interested your views on this, how you might be currently working around such size limitations, and what buffer enhancements would work for you. > > Thanks, > Paul. > > >> On May 30, 2018, at 10:21 PM, Viswanathan, Sandhya wrote: >> >> Hi Andrew/Jonathan, >> >> Thanks a lot for sharing this work. Copying hotspot-compiler-dev to get their feedback as well. >> >> Couple of thoughts/observations below: >> * Supporting ByteBuffer on persistent memory using existing FileChannel and MappedByteBuffer mechanism sounds like a very good idea. >> >> * Extending FileChannel.map to take additional parameter to indicate that the ByteBuffer is backed by persistent memory is a small API change. >> >> * Adding MappedByteBuffer.force(int from, int to) method on smaller range is very useful in addition to the force() on entire ByteBuffer. >> >> * The underlying force0_mapsync() could be implemented in terms of new unsafe APIs which in turn could be intrinsified. >> The advantage of this is that the unsafe APIs could then be used for other future persistent memory APIs in the JRE. >> Specifically the following two unsafe APIs would be useful: >> a) public native void flush(long address, long size); >> b) public native void storeFence(); >> storeFence() exists today but doesn?t generate any instruction for x86. >> Wondering if we could have additional boolean parameter to force the sfence generation. >> * DEFAULT_CACHE_LINE_SIZE is 128 in src/hotspot/cpu/x86/globalDefinitions_x86.hpp whereas actual cache line on the hardware is 64 bytes. >> This could be the cause for some of performance that you saw with compiler intrinsic vs pure C native. >> >> Best Regards, >> Sandhya >> >> >> >> RFC: Experiment in accessing/managing persistent memory from Java >> Andrew Dinn adinn at redhat.com >> Mon May 21 09:47:46 UTC 2018 >> >> I have been helping one of my Red Hat colleagues, Jonathan Halliday, to >> investigate provision of a Java equivalent to Intel's libpmem suite of C >> libraries [1]. This approach avoids the significant cost of using the >> Intel libraries from Java via JNI (or, worse, as a virtual driver for a >> persistent memory device). >> >> Jonathan has modified the JVM/JDK to allow a MappedByteBuffer to be >> mapped over persistent memory, providing equivalent function to libpmem >> itself. >> >> On top of this he implemented a Java journaled log class providing >> equivalent functionality to one of the Intel client libs, libpmemlog, >> built over libpmem. >> >> The modified MappedByteBuffer can be configured to use either i) a >> registered native method or ii) a JIT intrinsic to perform the critical >> task of cache line writeback i.e. the persistence step (the intrinsic is >> my contribution). >> >> Jonathan's tests compare use of JNI, registered native and intrinsic >> with an equivalent C program to write a large swathe of records to a >> journaled log file stored in persistent memory. Performance is worse >> than C when relying on JNI and significantly better with JVM/JDK >> support. Indeed, as one might reasonably expect, use of the JIT >> intrinsic almost completely eliminates writeback costs. >> >> The journaled log code, jdk dev tree patch, build instructions, test >> code plus C equivalent and test results are all available from >> Jonathan's git repo [2]. >> >> For those who do not want to look at the actual code, the README file >> [3] provides background to use of persistent memory, an overview of the >> design, and summary details of the test process and results. >> >> [1] https://pmem.io/pmdk/ >> [2] https://github.com/jhalliday/pmem >> [3] http://github.com://jhalliday/pmem/README.md >> >> n.b. Jonathan has experimented with using this same prototype to replace >> the journaled log used in the Red Hat Narayana transaction manager. It >> provides a significant improvement on the current disk file based log, >> both for throughput and latency (the code is not yet available as >> getting it to work involved some horrible hacking of the build to >> migrate up to jdk11). >> >> >> regards, >> >> >> Andrew Dinn >> ----------- >> Senior Principal Software Engineer >> Red Hat UK Ltd >> Registered in England and Wales under Company Registration No. 03798903 >> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > -- Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From brent.christian at oracle.com Fri Jun 8 19:11:49 2018 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 8 Jun 2018 12:11:49 -0700 Subject: RFR 8204565 : (spec) Document java.{vm.}?specification.version system properties' relation to \$FEATURE In-Reply-To: <05f55a3a-a7d5-4b5d-899d-2f2fbfe1d2d8@oracle.com> References: <7ff4c005-bcb3-7715-554c-22bc5570b03a@oracle.com> <05f55a3a-a7d5-4b5d-899d-2f2fbfe1d2d8@oracle.com> Message-ID: <2786fbb3-4cff-427c-2f72-2d0dfbf37031@oracle.com> On 6/7/18 1:24 PM, mandy chung wrote: >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8204565 >> >> Webrev: >> http://cr.openjdk.java.net/~bchristi/8204565/webrev/ > > Is there an existing test validating this? Looks like there is (kind of), for libs and for hotspot. I've rev'ed the webrev in place with how the tests might be updated. (or I can leave the hotspot test to be updated along with 8193719 if that's preferred). Thanks, -Brent From mandy.chung at oracle.com Fri Jun 8 19:27:57 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 8 Jun 2018 12:27:57 -0700 Subject: RFR 8204565 : (spec) Document java.{vm.}?specification.version system properties' relation to \$FEATURE In-Reply-To: <2786fbb3-4cff-427c-2f72-2d0dfbf37031@oracle.com> References: <7ff4c005-bcb3-7715-554c-22bc5570b03a@oracle.com> <05f55a3a-a7d5-4b5d-899d-2f2fbfe1d2d8@oracle.com> <2786fbb3-4cff-427c-2f72-2d0dfbf37031@oracle.com> Message-ID: On 6/8/18 12:11 PM, Brent Christian wrote: > On 6/7/18 1:24 PM, mandy chung wrote: >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8204565 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~bchristi/8204565/webrev/ >> >> Is there an existing test validating this? > > Looks like there is (kind of), for libs and for hotspot.? I've rev'ed > the webrev in place with how the tests might be updated. (or I can leave > the hotspot test to be updated along with 8193719 if that's preferred). test/jdk/java/lang/System/Versions.java it can also verify java.vm.specification.version. The hotspot test looks to me that it should expect the test be run with OpenJDK build and the vendor verification should consider that. That's a separate issue unrelated to this change. I suggest to remove the comment at line 36. Otherwise, looks good. Mandy From mandy.chung at oracle.com Fri Jun 8 20:22:22 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 8 Jun 2018 13:22:22 -0700 Subject: [JDK 11] RFR 8201528: Add new test to check for package versioning information in OpenJDK In-Reply-To: <7D1544F3-4BD6-48FA-AB7B-BA3941EEE04E@oracle.com> References: <08fb7059-2e3f-d77b-bd60-69bff4517edc@oracle.com> <7D1544F3-4BD6-48FA-AB7B-BA3941EEE04E@oracle.com> Message-ID: <8c813a78-e769-f453-0c37-61a2166105f1@oracle.com> On 6/8/18 12:07 AM, Chris Yin wrote: > Hi, Mandy > > Many thanks for your detailed review and comments, updates new webrev > as below, and comment inline, thanks > > webrev: http://cr.openjdk.java.net/~xyin/8201528/webrev.01/ Thanks for adding the test description, that is very helpful. This is looking pretty good. I have some suggestion on the test arguments that one can translate to the command-line easily. 69 * PackageFromManifest runJar single 73 * PackageFromManifest runUrlLoader single single means running with "testpack1.jar". What about putting the JAR file name as the option which makes it very clear what is being tested. Nit: "testpack1" could be confused with pack200 and maybe simply test1.jar. How about @run main PackageFromManifest runJar test1.jar 77 * PackageFromManifest runJar 1 81 * PackageFromManifest runJar 2 These tests load two JAR files and load foo.Foo. What do you think to express: PackageFromManifest runJar test1.jar test2.jar foo.Foo1 PackageFromManifest runJar test1.jar test2.jar foo.Foo2 85 * PackageFromManifest runUrlLoader 1 89 * PackageFromManifest runUrlLoader 2 PackageFromManifest runUrlLoader test1.jar test2.jar foo.Foo1 PackageFromManifest runUrlLoader test1.jar test2.jar foo.Foo2 Nit: I suggest to group the runType together. i.e. move line 34 to above line 37. "" not strictly needed as we won't generate the javadoc. You can consider leaving it a blank line. Mandy From brent.christian at oracle.com Fri Jun 8 21:39:55 2018 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 8 Jun 2018 14:39:55 -0700 Subject: RFR 8204565 : (spec) Document java.{vm.}?specification.version system properties' relation to \$FEATURE In-Reply-To: References: <7ff4c005-bcb3-7715-554c-22bc5570b03a@oracle.com> <05f55a3a-a7d5-4b5d-899d-2f2fbfe1d2d8@oracle.com> <2786fbb3-4cff-427c-2f72-2d0dfbf37031@oracle.com> Message-ID: <3977f83a-72d8-bf3c-663f-2f469505362d@oracle.com> On 6/8/18 12:27 PM, mandy chung wrote: >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~bchristi/8204565/webrev/ >>> > > test/jdk/java/lang/System/Versions.java > ? it can also verify java.vm.specification.version. > > The hotspot test looks to me that it should expect the test be run > with OpenJDK build and the vendor verification should consider that. > That's a separate issue unrelated to this change.? I suggest to > remove the comment at line 36. > > Otherwise, looks good. OK, thanks - done. (Also updated the @bug, copyright year, and removed unused 'VMversion'. -Brent From jonathan.gibbons at oracle.com Fri Jun 8 22:06:26 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 8 Jun 2018 15:06:26 -0700 Subject: RFR: JDK-8204588: Test failures after "Launch Single-File Source-Code Programs" Message-ID: <7c0d8d00-bbe4-558f-68f5-66114e59ce6b@oracle.com> Please review two test fixes related to the source launcher feature. In one test, the fix is to use File.separator to construct "golden output" for comparison. In the other test, the failure was caused by excessively long paths to the Java launcher in some test execution environments, causing the shebang line to overflow the underlying system limit of 128 characters.? The fix is to skip test cases when that occurs, as well as to augment the test with more tracing information to help diagnose failures on remote build and test systems. The tests are removed from the corresponding ProblemList files. JBS: https://bugs.openjdk.java.net/browse/JDK-8204588 Webrev: http://cr.openjdk.java.net/~jjg/8204588/webrev.00/index.html -- Jon From mandy.chung at oracle.com Fri Jun 8 22:20:01 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 8 Jun 2018 15:20:01 -0700 Subject: RFR: JDK-8204588: Test failures after "Launch Single-File Source-Code Programs" In-Reply-To: <7c0d8d00-bbe4-558f-68f5-66114e59ce6b@oracle.com> References: <7c0d8d00-bbe4-558f-68f5-66114e59ce6b@oracle.com> Message-ID: <7efdde87-741b-3a22-5321-d68bba4e3b4f@oracle.com> On 6/8/18 3:06 PM, Jonathan Gibbons wrote: > Please review two test fixes related to the source launcher feature. > > In one test, the fix is to use File.separator to construct "golden > output" for comparison. > > In the other test, the failure was caused by excessively long paths to > the Java launcher in some test execution environments, causing the > shebang line to overflow the underlying system limit of 128 characters. > The fix is to skip test cases when that occurs, as well as to augment > the test with more tracing information to help diagnose failures on > remote build and test systems. > > The tests are removed from the corresponding ProblemList files. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8204588 > Webrev: http://cr.openjdk.java.net/~jjg/8204588/webrev.00/index.html The change looks good to me. Skipping the test when it exceeds the system limit which is reasonable. Mandy From mandy.chung at oracle.com Sat Jun 9 04:29:53 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 8 Jun 2018 21:29:53 -0700 Subject: Review Request: 8204648: test/jdk/tools/launchers/SourceMode.java fails with long shebang line Message-ID: <8b28dd7a-1de3-7ee0-da73-40834f139e2c@oracle.com> JDK-8204588 [1] fixed the test failure caused by long paths to the Java launcher in some test execution environments, causing the shebang line to overflow the underlying system limit of 128 characters. The test needs a small tweak to the max javaCmd length to reduce from 100 to 90 since the arguments passed to java command are more than 28 characters. This is a quick fix for the test failure. A better fix would be to compute the length of the entire shebang line in each test case and determine if it should be skipped. That can be done as a follow up fix. diff --git a/test/jdk/tools/launcher/SourceMode.java b/test/jdk/tools/launcher/SourceMode.java --- a/test/jdk/tools/launcher/SourceMode.java +++ b/test/jdk/tools/launcher/SourceMode.java @@ -71,7 +71,7 @@ // limit of 120 characters for a shebang line. Path p = cwd.relativize(cmd); shortJavaCmd = (p.toString().length() < cmd.toString().length()) ? p : cmd; - skipShebangTest = shortJavaCmd.toString().length() > 100; + skipShebangTest = shortJavaCmd.toString().length() > 90; } log = System.err; thanks Mandy [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053700.html From mandy.chung at oracle.com Sat Jun 9 04:57:16 2018 From: mandy.chung at oracle.com (mandy chung) Date: Fri, 8 Jun 2018 21:57:16 -0700 Subject: Review Request: 8204648: test/jdk/tools/launchers/SourceMode.java fails with long shebang line In-Reply-To: <8b28dd7a-1de3-7ee0-da73-40834f139e2c@oracle.com> References: <8b28dd7a-1de3-7ee0-da73-40834f139e2c@oracle.com> Message-ID: <428b1358-bc26-0b9a-81c6-b6070d10cb98@oracle.com> I run into some issue with shebang tests. Since Jon is on vacation, I revise the patch to skip the shebang test temporarily until he returns. Mandy diff --git a/test/jdk/tools/launcher/SourceMode.java b/test/jdk/tools/launcher/SourceMode.java --- a/test/jdk/tools/launcher/SourceMode.java +++ b/test/jdk/tools/launcher/SourceMode.java @@ -71,7 +71,8 @@ // limit of 120 characters for a shebang line. Path p = cwd.relativize(cmd); shortJavaCmd = (p.toString().length() < cmd.toString().length()) ? p : cmd; - skipShebangTest = shortJavaCmd.toString().length() > 100; + // skipShebangTest = shortJavaCmd.toString().length() > 90; + skipShebangTest = true; } log = System.err; On 6/8/18 9:29 PM, mandy chung wrote: > JDK-8204588 [1] fixed the test failure caused by long paths to the Java > launcher in some test execution environments, causing the shebang line > to overflow the underlying system limit of 128 characters. > > The test needs a small tweak to the max javaCmd length to reduce from > 100 to 90 since the arguments passed to java command are more than 28 > characters.? This is a quick fix for the test failure.? A better fix > would be to compute the length of the entire shebang line in each test > case and determine if it should be skipped.? That can be done as a > follow up fix. > > diff --git a/test/jdk/tools/launcher/SourceMode.java > b/test/jdk/tools/launcher/SourceMode.java > --- a/test/jdk/tools/launcher/SourceMode.java > +++ b/test/jdk/tools/launcher/SourceMode.java > @@ -71,7 +71,7 @@ > ???????????? // limit of 120 characters for a shebang line. > ???????????? Path p = cwd.relativize(cmd); > ???????????? shortJavaCmd = (p.toString().length() < > cmd.toString().length()) ? p : cmd; > -??????????? skipShebangTest = shortJavaCmd.toString().length() > 100; > +??????????? skipShebangTest = shortJavaCmd.toString().length() > 90; > ???????? } > > ???????? log = System.err; > > thanks > Mandy > [1] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053700.html From fw at deneb.enyo.de Sat Jun 9 10:27:55 2018 From: fw at deneb.enyo.de (Florian Weimer) Date: Sat, 09 Jun 2018 12:27:55 +0200 Subject: The store for byte strings Message-ID: <87y3fo8das.fsf@mid.deneb.enyo.de> Lately I've been thinking about string representation. The world turned out not to be UCS-2 or UTF-16, after all, and we often have to deal with strings generally encoded as ASCII or UTF-8, but we aren't always encoded this way (and there might not even be a charset declaration, see the ELF spec). (a) byte[] with defensive copies. Internal storage is byte[], copy is made before returning it to the caller. Quite common across the JDK. (b) byte[] without defensive copies. Internal storage is byte[], and a reference is returned. In the past, this could be a security bug, and usually, it was adjusted to (a) when noticed. Without security requirements, this can be quite efficient, but there is ample potential for API misuse. (c) java.lang.String with ISO-8859-1 decoding/encoding. Sometimes done by reconfiguring the entire JVM to run with ISO-8859-1, usually so that it is possible to process malformed UTF-8. The advantage is that there is rich API support, including regular expressions, and good optimization. There is also language support for string literals. (d) java.lang.String with UTF-8 decoding/encoding and replacement. This seems to be very common, but is not completely accurate and can lead to subtle bugs (or completely non-processible data). Otherwise has the same advantages as (c). (e) Various variants of ByteBuffer. Have not seen this much in practice (outside binary file format parsers). In the past, it needed deep defensive copies on input for security (because there isn't an immutably backed ByteBuffer), and shallow copies for access. The ByteBuffer objects themselves are also quite heavy when they can't be optimized away. For that reason, probably most useful on interfaces, and not for storage. (f) Custom, immutable ByteString class. Quite common, but has cross-library interoperability issues, and a full complement of support (matching java.lang.String) is quite hard. (g) Something based on VarHandle. Haven't seen this yet. Probably not useful for storage. Anything that I have missed? Considering these choices, what is the expected direction on the JDK side for new code? Option (d) for things generally ASCII/UTF-8, and (b) for things of a more binary nature? What to do if the choice is difficult? From henry.jen at oracle.com Sat Jun 9 16:16:31 2018 From: henry.jen at oracle.com (Henry Jen) Date: Sat, 9 Jun 2018 09:16:31 -0700 Subject: RFR: 8199871: Deprecate pack200 and unpack200 tools In-Reply-To: <78256921-225B-4AEF-8313-42E135BDB886@oracle.com> References: <78256921-225B-4AEF-8313-42E135BDB886@oracle.com> Message-ID: <84F9D20B-201E-4E16-B5F1-7A885343E29D@oracle.com> I revised the webrev to have warning only executable name for unpack200 instead of full path on Windows, which I believe is what was intended all along, the test is also revised to take unpack200.exe on the Windows platform. The updated version is at http://cr.openjdk.java.net/~henryjen/jdk11/8199871/1/webrev/ Cheers, Henry > On Jun 8, 2018, at 6:56 PM, Henry Jen wrote: > > Hi, > > Please review this webrev[1] in which we mark Pack200 related API and tools deprecate, and print a warning about tools to be remove in a future JDK release. To avoid interrupt existing tools depends on the output, an option is provided to suppress the warning message. > > The option name is following the convention of javah, not the GNU style as we should. The rationale is that this is a temporary option and an existing convention would make it more consistent for users. Anyhow, if we have a strong feeling this should be changed, we can do that. > > For jar tool, the normalize option use Pack200 is also deprecated, the suppress option only available to the new GNU style option mode. As if user need to provide the option, the command line change is necessary, so without compatible mode option should not be an issue. > > Cheers, > Henry > > [1] http://cr.openjdk.java.net/~henryjen/jdk11/8199871/0/webrev/ From joe.darcy at oracle.com Sat Jun 9 16:39:21 2018 From: joe.darcy at oracle.com (joe darcy) Date: Sat, 9 Jun 2018 09:39:21 -0700 Subject: Review Request: 8204648: test/jdk/tools/launchers/SourceMode.java fails with long shebang line In-Reply-To: <428b1358-bc26-0b9a-81c6-b6070d10cb98@oracle.com> References: <8b28dd7a-1de3-7ee0-da73-40834f139e2c@oracle.com> <428b1358-bc26-0b9a-81c6-b6070d10cb98@oracle.com> Message-ID: Skipping the shebang tests is fine a workaround Mandy; thanks, -Joe On 6/8/2018 9:57 PM, mandy chung wrote: > I run into some issue with shebang tests.? Since Jon is on vacation, > I revise the patch to skip the shebang test temporarily until he returns. > > Mandy > > diff --git a/test/jdk/tools/launcher/SourceMode.java > b/test/jdk/tools/launcher/SourceMode.java > --- a/test/jdk/tools/launcher/SourceMode.java > +++ b/test/jdk/tools/launcher/SourceMode.java > @@ -71,7 +71,8 @@ > ???????????? // limit of 120 characters for a shebang line. > ???????????? Path p = cwd.relativize(cmd); > ???????????? shortJavaCmd = (p.toString().length() < > cmd.toString().length()) ? p : cmd; > -??????????? skipShebangTest = shortJavaCmd.toString().length() > 100; > +??????????? // skipShebangTest = shortJavaCmd.toString().length() > 90; > +??????????? skipShebangTest = true; > ???????? } > > ???????? log = System.err; > > > On 6/8/18 9:29 PM, mandy chung wrote: >> JDK-8204588 [1] fixed the test failure caused by long paths to the >> Java launcher in some test execution environments, causing the >> shebang line to overflow the underlying system limit of 128 characters. >> >> The test needs a small tweak to the max javaCmd length to reduce from >> 100 to 90 since the arguments passed to java command are more than 28 >> characters.? This is a quick fix for the test failure.? A better fix >> would be to compute the length of the entire shebang line in each >> test case and determine if it should be skipped.? That can be done as >> a follow up fix. >> >> diff --git a/test/jdk/tools/launcher/SourceMode.java >> b/test/jdk/tools/launcher/SourceMode.java >> --- a/test/jdk/tools/launcher/SourceMode.java >> +++ b/test/jdk/tools/launcher/SourceMode.java >> @@ -71,7 +71,7 @@ >> ????????????? // limit of 120 characters for a shebang line. >> ????????????? Path p = cwd.relativize(cmd); >> ????????????? shortJavaCmd = (p.toString().length() < >> cmd.toString().length()) ? p : cmd; >> -??????????? skipShebangTest = shortJavaCmd.toString().length() > 100; >> +??????????? skipShebangTest = shortJavaCmd.toString().length() > 90; >> ????????? } >> >> ????????? log = System.err; >> >> thanks >> Mandy >> [1] >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053700.html From xueming.shen at oracle.com Sat Jun 9 19:18:05 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Sat, 09 Jun 2018 12:18:05 -0700 Subject: The store for byte strings In-Reply-To: <87y3fo8das.fsf@mid.deneb.enyo.de> References: <87y3fo8das.fsf@mid.deneb.enyo.de> Message-ID: <5B1C27ED.2020404@oracle.com> On 6/9/18, 3:27 AM, Florian Weimer wrote: > Lately I've been thinking about string representation. The world > turned out not to be UCS-2 or UTF-16, after all, and we often have to > deal with strings generally encoded as ASCII or UTF-8, but we aren't > always encoded this way (and there might not even be a charset > declaration, see the ELF spec). > > (a) byte[] with defensive copies. > Internal storage is byte[], copy is made before returning it to > the caller. Quite common across the JDK. > > (b) byte[] without defensive copies. > Internal storage is byte[], and a reference is returned. In the > past, this could be a security bug, and usually, it was adjusted > to (a) when noticed. Without security requirements, this can be > quite efficient, but there is ample potential for API misuse. > > (c) java.lang.String with ISO-8859-1 decoding/encoding. > Sometimes done by reconfiguring the entire JVM to run with > ISO-8859-1, usually so that it is possible to process malformed > UTF-8. The advantage is that there is rich API support, including > regular expressions, and good optimization. There is also > language support for string literals. > > (d) java.lang.String with UTF-8 decoding/encoding and replacement. > This seems to be very common, but is not completely accurate > and can lead to subtle bugs (or completely non-processible > data). Otherwise has the same advantages as (c). > > (e) Various variants of ByteBuffer. > Have not seen this much in practice (outside binary file format > parsers). In the past, it needed deep defensive copies on input > for security (because there isn't an immutably backed ByteBuffer), > and shallow copies for access. The ByteBuffer objects themselves > are also quite heavy when they can't be optimized away. For that > reason, probably most useful on interfaces, and not for storage. > > (f) Custom, immutable ByteString class. > Quite common, but has cross-library interoperability issues, > and a full complement of support (matching java.lang.String) > is quite hard. > > (g) Something based on VarHandle. > Haven't seen this yet. Probably not useful for storage. > > Anything that I have missed? > > Considering these choices, what is the expected direction on the JDK > side for new code? Option (d) for things generally ASCII/UTF-8, and > (b) for things of a more binary nature? What to do if the choice is > difficult? Hi Florian, Some comments about the j.l.String storage. Ideally I would assume we would want to have a utf-8 internal storage for String, even in theory utf8 is supposed to be used externally and utf16 to be the internal one. I did have a byte[]/utf-8 prototype implementation when we did the compact string for jdk9 but that was finally dropped because of the potential performance regression for index base access, such as the basic String.charAt(int), as you have to count from the beginning to locate the target character each every time. But I think we might want to try it again later, especially for use scenario that index base access performance is not that important/critical and the throughput operation of the String, means input from /output to the external utf-8/byte[] world, is more desired. Given we are heading utf-8 as the default encoding for jvm [1], I think we might want to at least provide some alternative that you can "optionally" do that for String object. The idea might go further (wild, just an idea, not necessary something thing we really want to do :-) for Java String) to other charsets, so you can simply store the byte[] (verified no malformed/unmappable) + charsetId directly when creating a String object. This might be useful and efficient in use scenario that the String object is simply a vehicle to carry a sequence of characters back and forth between a front end server and back end server, the jvm is simply passing them around/through. Defensive copy when getting byte[] in & out of String object seems still inevitable for now, before we can have something like "read-only" byte[], given the nature of its immutability commitment. Regards, Sherman [1] https://bugs.openjdk.java.net/browse/JDK-8187041 From john.r.rose at oracle.com Sat Jun 9 23:52:25 2018 From: john.r.rose at oracle.com (John Rose) Date: Sat, 9 Jun 2018 16:52:25 -0700 Subject: The store for byte strings In-Reply-To: <87y3fo8das.fsf@mid.deneb.enyo.de> References: <87y3fo8das.fsf@mid.deneb.enyo.de> Message-ID: <82A561D3-1A00-47E4-9A9D-C180F2B8AA8E@oracle.com> I'm glad to see you are thinking about this, Florian. You appear to be aiming at a way to compactly store and manipulate series of octets (in an arbitrary encoding) with an emphasis on using those octets to represent strings, in the usual sense of character sequences. Would you agree that this design problem factors well into a generic problem of storing and manipulating octet sequences, plus a detachable upper layer that allows strings (in various encodings) to be extracted from those sequences? I think the sweet spot here is to introduce a "stringy but char-free" API which commits to dealing with chunks of memory (viewed as octet sequences), regardless of how those chunks will be interpreted. In https://bugs.openjdk.java.net/browse/JDK-8161256 I discuss this nascent API under the name "ByteSequence", which is analogous to CharSequence, but doesn't mention the types 'char' or 'String'. By "stringy" I mean that there are natural ways to index or partition an existing sequence of octets, or concatenate multiple sequences into one. I also mean that immutability plays a strong role, enabling algorithms to work without defensive copies. Making it an interface like CharSequence means we can use backing stores like ByteBuffer or byte[], or more exotic things like Panama native memory, interoperably. Here are some uses for an immutable octet sequence type: - manipulation of non-UTF16 character data (which you mention) - zero copy views (slices, modifiable or not) into existing types (ByteBuffer, byte[], etc.) - zero copy views into file images (N.B. requires a 'long' size property, not 'int') - zero copy views to intra-classfile resources (CONSTANT_Bytes) - backing stores for Panama data structures and smart pointers - copy-reduced scatter and gather nodes associated with packet processing - octet-level cursors for parsers, scanners, packet decoders, etc. If the ByteSequence views are value instances, they can be created at a very high rate with little or no GC impact. Generic algorithms would still operate on them A mutable octet sequence class, analogous to StringBuilder, would allow immutable sequences to be built with fewer intermediate copies, just like with StringBuilder. If the API is properly defined it can be inserted directly into existing types like ByteBuffer. Doing this will probably require us to polish ByteBuffer a little, adding immutability as an option and lifting the 32-bit limits. It should be possible to "freeze" a ByteBuffer or array and use it as a backing store that is reliably immutable, so it can be handed to zero-copy algorithms that work with ByteSequences. For some of this see https://bugs.openjdk.java.net/browse/JDK-8180628 . Independently, I want to eventually add frozen arrays, including frozen byte[] arrays, to the JVM, but that doesn't cover zero-copy use cases; it has to be an interface like CharSequence. So the option I prefer is not on your list; it would be: (h) ByteSequence interface with retrofits to ByteBuffer, byte[], etc. This is more flexible than (f) the concrete ByteString class. I think the ByteString you are thinking of would appear as a non-public class created by a ByteSequence factory, analogous to List::of. ? John On Jun 9, 2018, at 3:27 AM, Florian Weimer wrote: > > Lately I've been thinking about string representation. The world > turned out not to be UCS-2 or UTF-16, after all, and we often have to > deal with strings generally encoded as ASCII or UTF-8, but we aren't > always encoded this way (and there might not even be a charset > declaration, see the ELF spec). > > (a) byte[] with defensive copies. > Internal storage is byte[], copy is made before returning it to > the caller. Quite common across the JDK. > > (b) byte[] without defensive copies. > Internal storage is byte[], and a reference is returned. In the > past, this could be a security bug, and usually, it was adjusted > to (a) when noticed. Without security requirements, this can be > quite efficient, but there is ample potential for API misuse. > > (c) java.lang.String with ISO-8859-1 decoding/encoding. > Sometimes done by reconfiguring the entire JVM to run with > ISO-8859-1, usually so that it is possible to process malformed > UTF-8. The advantage is that there is rich API support, including > regular expressions, and good optimization. There is also > language support for string literals. > > (d) java.lang.String with UTF-8 decoding/encoding and replacement. > This seems to be very common, but is not completely accurate > and can lead to subtle bugs (or completely non-processible > data). Otherwise has the same advantages as (c). > > (e) Various variants of ByteBuffer. > Have not seen this much in practice (outside binary file format > parsers). In the past, it needed deep defensive copies on input > for security (because there isn't an immutably backed ByteBuffer), > and shallow copies for access. The ByteBuffer objects themselves > are also quite heavy when they can't be optimized away. For that > reason, probably most useful on interfaces, and not for storage. > > (f) Custom, immutable ByteString class. > Quite common, but has cross-library interoperability issues, > and a full complement of support (matching java.lang.String) > is quite hard. > > (g) Something based on VarHandle. > Haven't seen this yet. Probably not useful for storage. > > Anything that I have missed? > > Considering these choices, what is the expected direction on the JDK > side for new code? Option (d) for things generally ASCII/UTF-8, and > (b) for things of a more binary nature? What to do if the choice is > difficult? From john.r.rose at oracle.com Sun Jun 10 01:48:59 2018 From: john.r.rose at oracle.com (John Rose) Date: Sat, 9 Jun 2018 18:48:59 -0700 Subject: The store for byte strings In-Reply-To: <5B1C27ED.2020404@oracle.com> References: <87y3fo8das.fsf@mid.deneb.enyo.de> <5B1C27ED.2020404@oracle.com> Message-ID: On Jun 9, 2018, at 12:18 PM, Xueming Shen wrote: > > Ideally I would assume we would want to have a utf-8 internal storage for > String, even in theory utf8 is supposed to be used externally and utf16 > to be the internal one. Separately from my point about ByteSequence, I agree that "doubling down" on Utf8 as a standard format for packed strings is a good idea. A reasonable way to prototype right now would be an implementation of CharSequence that is backed by a byte[] (eventually ByteSequence) and has some sort of fast access (probably streaming) to Utf16 code points. To make it pay for itself the Utf8 encoding should be applicable as an overlay in as many places as possible, including slices of byte[] and ByteBuffer objects, and later ByteSequences. > Defensive copy when getting byte[] in & out of String object seems still > inevitable for now, before we can have something like "read-only" byte[], > given the nature of its immutability commitment. We didn't need frozen char[] arrays to avoid defensive copying of String objects, only an immutability invariant on the class. We could pull a similar trick with Utf8 by supplying a ByteSequence view of a String's underlying bytes. If the String has underlying chars (Utf16) a view is also possible, although it is more difficult to get right (as you described). ? John From fw at deneb.enyo.de Sun Jun 10 09:47:11 2018 From: fw at deneb.enyo.de (Florian Weimer) Date: Sun, 10 Jun 2018 11:47:11 +0200 Subject: The store for byte strings In-Reply-To: <82A561D3-1A00-47E4-9A9D-C180F2B8AA8E@oracle.com> (John Rose's message of "Sat, 9 Jun 2018 16:52:25 -0700") References: <87y3fo8das.fsf@mid.deneb.enyo.de> <82A561D3-1A00-47E4-9A9D-C180F2B8AA8E@oracle.com> Message-ID: <877en73rds.fsf@mid.deneb.enyo.de> * John Rose: > In https://bugs.openjdk.java.net/browse/JDK-8161256 I discuss > this nascent API under the name "ByteSequence", which is analogous > to CharSequence, but doesn't mention the types 'char' or 'String'. Very interesting. What's the specification for toString() and hashCode()? One problem of retrofitting a custom ByteString into a CharSequence is that CharSequence reuses toString() in a fairly central fashion, and it's hard to reconcile this with byte-based length() and charAt() methods unless ISO-8859-1 encoding is used. If this feature is supposed to land before JEP 218? If not, how does ByteSequence differ from List? > If the ByteSequence views are value instances, they can be created > at a very high rate with little or no GC impact. Generic algorithms > would still operate on them I'm not up-to-date with those upcoming changes. Would the nature as value instances be expressed as part of the ByteSequence interface type? > If the API is properly defined it can be inserted directly into > existing types like ByteBuffer. Doing this will probably require us > to polish ByteBuffer a little, adding immutability as an option and > lifting the 32-bit limits. It should be possible to "freeze" a > ByteBuffer or array and use it as a backing store that is reliably > immutable, so it can be handed to zero-copy algorithms that work > with ByteSequences. Such freezing is incompatible with mapped byte buffers, right? Even if the implementation prevents changes at the VM/process level, changes on the file system could well become visible. Do you expect to make freezing an optional operation (probably not a good idea), or copy the contents of the mapping to the heap (which is probably not too bad, considering that a shared byte[] could also result in arbitrarily large copies). > Independently, I want to eventually add frozen arrays, including > frozen byte[] arrays, to the JVM, but that doesn't cover zero-copy use > cases; it has to be an interface like CharSequence. Well, there is already the VarHandle approach for that. But it's not a particularly rich interface and very far away from strings. > So the option I prefer is not on your list; it would be: > > (h) ByteSequence interface with retrofits to ByteBuffer, byte[], etc. > > This is more flexible than (f) the concrete ByteString class. I think > the ByteString you are thinking of would appear as a non-public class > created by a ByteSequence factory, analogous to List::of. Yes, this seems reasonable. It's a bit of a drawback that the immutable nature of a value cannot be expressed in the type system (so that you have to remember to use ByteSequence::of to get a view to an immutable object), but at least it's consistent with collections. From xu.y.yin at oracle.com Mon Jun 11 05:12:07 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Mon, 11 Jun 2018 13:12:07 +0800 Subject: [JDK 11] RFR 8201528: Add new test to check for package versioning information in OpenJDK In-Reply-To: <8c813a78-e769-f453-0c37-61a2166105f1@oracle.com> References: <08fb7059-2e3f-d77b-bd60-69bff4517edc@oracle.com> <7D1544F3-4BD6-48FA-AB7B-BA3941EEE04E@oracle.com> <8c813a78-e769-f453-0c37-61a2166105f1@oracle.com> Message-ID: <3C0AFC7B-AA3C-4DE1-AF9A-0DB556A1AEAB@oracle.com> Hi, Mandy Thanks lot for your suggestions, update webrev as below and comment inline, thanks http://cr.openjdk.java.net/~xyin/8201528/webrev.02/ > On 9 Jun 2018, at 4:22 AM, mandy chung wrote: > > > > On 6/8/18 12:07 AM, Chris Yin wrote: >> Hi, Mandy >> Many thanks for your detailed review and comments, updates new webrev >> as below, and comment inline, thanks >> webrev: http://cr.openjdk.java.net/~xyin/8201528/webrev.01/ > > > Thanks for adding the test description, that is very helpful. > This is looking pretty good. I have some suggestion on the test > arguments that one can translate to the command-line easily. > > 69 * PackageFromManifest runJar single > 73 * PackageFromManifest runUrlLoader single > > single means running with "testpack1.jar". What about putting > the JAR file name as the option which makes it very clear what > is being tested. Nit: "testpack1" could be confused with pack200 > and maybe simply test1.jar. How about Fixed, thanks > > @run main PackageFromManifest runJar test1.jar > > 77 * PackageFromManifest runJar 1 > 81 * PackageFromManifest runJar 2 > > These tests load two JAR files and load foo.Foo. What do you > think to express: > > PackageFromManifest runJar test1.jar test2.jar foo.Foo1 > PackageFromManifest runJar test1.jar test2.jar foo.Foo2 > > 85 * PackageFromManifest runUrlLoader 1 > 89 * PackageFromManifest runUrlLoader 2 > > PackageFromManifest runUrlLoader test1.jar test2.jar foo.Foo1 > PackageFromManifest runUrlLoader test1.jar test2.jar foo.Foo2 Used new style as you suggested, that looks exactly match with the description :) > > Nit: I suggest to group the runType together. i.e. move line 34 > to above line 37. "" not strictly needed as we won't generate > the javadoc. You can consider leaving it a blank line. Sure, grouped runType and removed (oh, it?s added by IDE during code reformat) Thanks & Regards, Chris > > Mandy > From ivan.gerasimov at oracle.com Mon Jun 11 06:15:17 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Sun, 10 Jun 2018 23:15:17 -0700 Subject: RFR 8204310 : Simpler RandomAccessFile.setLength() on Windows In-Reply-To: <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> References: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> Message-ID: <70a97527-931a-7a01-8f46-32e6eb85a44b@oracle.com> Hi Alan! On 6/6/18 6:57 AM, Alan Bateman wrote: > I think this should be okay but the Windows implementation has a long > history of biting the fingers of anyone that dares touch it. Sometimes > unexpected behavior changes only come to light long after the change. > The recent mails here about Kafka and sparse files is a good example > of that. So I think it's important to check the test coverage before > pushing this change. Specifically I think we need to check that we > have tests that > > 1. Exercise shrinking, extending, and not change the file length > > 2. Check getFilePointer when used with setLength. > > 3. Check how FileChannel behaves when created from a RandomAccessFile > but the file length is changed after the FileChannel is obtained. > I extended the existing reg. test java/io/RandomAccessFile/SetLength.java to cover more cases that involve changing the file size. Also I added another regression test, as you Alan suggested, to check that RandomAccessFile and its FileChannel behave consistently in various scenarios. All the tests, including the new ones, pass on all supported platforms. BUGURL: https://bugs.openjdk.java.net/browse/JDK-8204310 WEBREV: http://cr.openjdk.java.net/~igerasim/8204310/01/webrev/ Would you please help review the fix? -- With kind regards, Ivan Gerasimov From ecki at zusammenkunft.net Mon Jun 11 07:55:50 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Mon, 11 Jun 2018 07:55:50 +0000 (UTC) Subject: RFR 8204310 : Simpler RandomAccessFile.setLength() on Windows In-Reply-To: <70a97527-931a-7a01-8f46-32e6eb85a44b@oracle.com> References: <343b55c8-c1d7-e229-4380-be3501f374ff@oracle.com> <259aa1ca-6e17-6624-4f21-46f468c6c8ea@oracle.com> <70a97527-931a-7a01-8f46-32e6eb85a44b@oracle.com> Message-ID: <8C274EA3A72EE413.A6ABA911-B6E7-442F-973A-999E61EE9EDD@mail.outlook.com> We had BTW problems (on Windows) with SMB hosted files where appending causes sometimes the filepointer to not increase. We fixed that by closing the file in such conditions. It does however look like a smb Cache and not Java Code problem. But just in case anyone has seen encountered that... Not sure how easy it is to replicate, but will let you know. Bernd -- https://Bernd.eckenfels.net On Mon, Jun 11, 2018 at 9:45 AM +0200, "Ivan Gerasimov" wrote: Hi Alan! On 6/6/18 6:57 AM, Alan Bateman wrote: > I think this should be okay but the Windows implementation has a long > history of biting the fingers of anyone that dares touch it. Sometimes > unexpected behavior changes only come to light long after the change. > The recent mails here about Kafka and sparse files is a good example > of that. So I think it's important to check the test coverage before > pushing this change. Specifically I think we need to check that we > have tests that > > 1. Exercise shrinking, extending, and not change the file length > > 2. Check getFilePointer when used with setLength. > > 3. Check how FileChannel behaves when created from a RandomAccessFile > but the file length is changed after the FileChannel is obtained. > I extended the existing reg. test java/io/RandomAccessFile/SetLength.java to cover more cases that involve changing the file size. Also I added another regression test, as you Alan suggested, to check that RandomAccessFile and its FileChannel behave consistently in various scenarios. All the tests, including the new ones, pass on all supported platforms. BUGURL: https://bugs.openjdk.java.net/browse/JDK-8204310 WEBREV: http://cr.openjdk.java.net/~igerasim/8204310/01/webrev/ Would you please help review the fix? -- With kind regards, Ivan Gerasimov From robbin.ehn at oracle.com Mon Jun 11 08:07:07 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 11 Jun 2018 10:07:07 +0200 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: Hi Bob, On 06/07/2018 07:43 PM, Bob Vandette wrote: > Can I get one more reviewer for this RFE so I can integrate it? > >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 Seems okay. Metrics.java "Returns the length of the operating system time slice" Note that is is only true if you are using a batch scheduler. Otherwise this period may be split on multiple 'time slices'. In printSystemMetrics there is no units, maybe intentional? Do we have support now in mach5 for docker jtreg, or do we still run these separate? You can ship it. Thanks for fixing, and super thanks for fixing the bug in PlainRead also! /Robbin > > Mandy Chung has reviewed this change. > > I?ve run Mach5 hotspot and core lib tests. > > I?ve reviewed the tests which were written by Harsha Wardhana > > I filed a CSR for the command line change and it?s now approved and closed. > > Thanks, > Bob. > > >> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >> >> Please review the following RFE which adds an internal API, along with jtreg tests that provide >> access to Docker container configuration data and metrics. In addition to the API which we hope to >> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >> option to -XshowSettings:system than dumps out the container or host cgroup confguration >> information. See the sample output below: >> >> RFE: Container Metrics >> >> https://bugs.openjdk.java.net/browse/JDK-8203357 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> >> >> This commit will also include a fix for the following bug. >> >> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >> >> https://bugs.openjdk.java.net/browse/JDK-8203691 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >> >> SAMPLE USAGE and OUTPUT: >> >> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >> ./java -XshowSettings:system >> Operating System Metrics: >> Provider: cgroupv1 >> Effective CPU Count: 4 >> CPU Period: 100000 >> CPU Quota: -1 >> CPU Shares: -1 >> List of Processors, 4 total: >> 4 5 6 7 >> List of Effective Processors, 4 total: >> 4 5 6 7 >> List of Memory Nodes, 2 total: >> 0 1 >> List of Available Memory Nodes, 2 total: >> 0 1 >> CPUSet Memory Pressure Enabled: false >> Memory Limit: 256.00M >> Memory Soft Limit: Unlimited >> Memory & Swap Limit: 512.00M >> Kernel Memory Limit: Unlimited >> TCP Memory Limit: Unlimited >> Out Of Memory Killer Enabled: true >> >> TEST RESULTS: >> >> testing runtime container APIs >> Directory "JTwork" not found: creating >> Passed: runtime/containers/cgroup/PlainRead.java >> Passed: runtime/containers/docker/DockerBasicTest.java >> Passed: runtime/containers/docker/TestCPUAwareness.java >> Passed: runtime/containers/docker/TestCPUSets.java >> Passed: runtime/containers/docker/TestMemoryAwareness.java >> Passed: runtime/containers/docker/TestMisc.java >> Test results: passed: 6 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing jdk.internal.platform APIs >> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >> Test results: passed: 4 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing -XshowSettings:system launcher option >> Passed: tools/launcher/Settings.java >> Test results: passed: 1 >> >> >> Bob. >> >> > From david.holmes at oracle.com Mon Jun 11 08:32:00 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 11 Jun 2018 18:32:00 +1000 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> Sorry Bob I haven't had a chance to look at this detail. For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? David On 8/06/2018 3:43 AM, Bob Vandette wrote: > Can I get one more reviewer for this RFE so I can integrate it? > >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 > > Mandy Chung has reviewed this change. > > I?ve run Mach5 hotspot and core lib tests. > > I?ve reviewed the tests which were written by Harsha Wardhana > > I filed a CSR for the command line change and it?s now approved and closed. > > Thanks, > Bob. > > >> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >> >> Please review the following RFE which adds an internal API, along with jtreg tests that provide >> access to Docker container configuration data and metrics. In addition to the API which we hope to >> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >> option to -XshowSettings:system than dumps out the container or host cgroup confguration >> information. See the sample output below: >> >> RFE: Container Metrics >> >> https://bugs.openjdk.java.net/browse/JDK-8203357 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> >> >> This commit will also include a fix for the following bug. >> >> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >> >> https://bugs.openjdk.java.net/browse/JDK-8203691 >> >> WEBREV: >> >> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >> >> SAMPLE USAGE and OUTPUT: >> >> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >> ./java -XshowSettings:system >> Operating System Metrics: >> Provider: cgroupv1 >> Effective CPU Count: 4 >> CPU Period: 100000 >> CPU Quota: -1 >> CPU Shares: -1 >> List of Processors, 4 total: >> 4 5 6 7 >> List of Effective Processors, 4 total: >> 4 5 6 7 >> List of Memory Nodes, 2 total: >> 0 1 >> List of Available Memory Nodes, 2 total: >> 0 1 >> CPUSet Memory Pressure Enabled: false >> Memory Limit: 256.00M >> Memory Soft Limit: Unlimited >> Memory & Swap Limit: 512.00M >> Kernel Memory Limit: Unlimited >> TCP Memory Limit: Unlimited >> Out Of Memory Killer Enabled: true >> >> TEST RESULTS: >> >> testing runtime container APIs >> Directory "JTwork" not found: creating >> Passed: runtime/containers/cgroup/PlainRead.java >> Passed: runtime/containers/docker/DockerBasicTest.java >> Passed: runtime/containers/docker/TestCPUAwareness.java >> Passed: runtime/containers/docker/TestCPUSets.java >> Passed: runtime/containers/docker/TestMemoryAwareness.java >> Passed: runtime/containers/docker/TestMisc.java >> Test results: passed: 6 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing jdk.internal.platform APIs >> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >> Test results: passed: 4 >> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >> >> testing -XshowSettings:system launcher option >> Passed: tools/launcher/Settings.java >> Test results: passed: 1 >> >> >> Bob. >> >> > From scolebourne at joda.org Mon Jun 11 09:22:50 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 11 Jun 2018 10:22:50 +0100 Subject: Durations in existing JDK APIs In-Reply-To: References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: Finally got to check the logic in convert(Duration), and it looks fine to me. All three look good thanks Stephen (not an OpenJDK reviewer). On 6 June 2018 at 18:03, Martin Buchholz wrote: > OK, here is a RFR for low-hanging changes (some of these changes have > already been reviewed by some of you): > > 8204375: Add TimeUnit#convert(Duration) > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/convertDuration/index.html > https://bugs.openjdk.java.net/browse/JDK-8204375 > > 8204444: java.time cleanup > http://cr.openjdk.java.net/~martin/webrevs/jdk/time-lint/ > https://bugs.openjdk.java.net/browse/JDK-8204444 > > 8204377: Rename Object#wait parameter name from "timeout" to "timeoutMillis" > http://cr.openjdk.java.net/~martin/webrevs/jdk/Object-wait-timeout/ > https://bugs.openjdk.java.net/browse/JDK-8204377 > From peter.levart at gmail.com Mon Jun 11 10:57:57 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 11 Jun 2018 12:57:57 +0200 Subject: Proposal: optimization of Map.copyOf and Collectors.toUnmodifiableMap Message-ID: Hi, Those two methods were added in JDK 10 and they are not very optimal. Map.copyOf(Map map) 1st dumps the source map into an array of Map.Entry(s) (map.entrySet().toArray(new Entry[0])), which typically creates new Map.Entry objects, then passes the array to Map.ofEntries(Map.Entry[] entries) factory method that iterates the array and constructs a key/value interleaved array from it which is passed to ImmutableCollections.MapN constructor, which then constructs a linear-probing hash table from it. So each key and value is copied 3 times, while several temporary objects are created in the process. One copying step could be eliminated and construction of temporary Map.Entry objects too: http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.01/ Collecting stream(s) using Collectors.toUnmodifiableMap() 1st collects key/value pairs into a HashMap, then it performs equivalent operations as Map.copyOf(hashMap) at the end. Using Map.copyOf() directly benefits this collection operation too. The following benchmark: http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapBench.java Shows up to 30% improvement of .copyOf operation with this patch applied: Original: Benchmark?????????????????????????????? (size)? Mode Cnt????? Score???? Error? Units UnmodifiableMapBench.copyOf???????????????? 10? avgt 10??? 403.633 ??? 2.640? ns/op UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 3489.623 ?? 44.590? ns/op UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 40030.572 ? 277.075? ns/op UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10??? 831.221 ??? 3.816? ns/op UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9783.519 ?? 43.097? ns/op UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 96524.536 ? 670.818? ns/op Patched: Benchmark?????????????????????????????? (size)? Mode Cnt????? Score????? Error? Units UnmodifiableMapBench.copyOf???????????????? 10? avgt 10??? 264.172 ???? 1.882? ns/op UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 2318.974 ??? 15.877? ns/op UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 29291.782 ? 3139.737? ns/op UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10??? 771.221 ??? 65.432? ns/op UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9275.016 ?? 725.722? ns/op UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 82204.342 ?? 851.741? ns/op Production of garbage is also reduced, since no Map.Entry temporary objects are constructed: Original: Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10??? 416.001 ????? 0.002??? B/op UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10?? 2936.005 ????? 0.019??? B/op UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10? 28136.059 ????? 0.199??? B/op UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10? avgt?? 10?? 1368.001 ????? 0.004??? B/op UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100? avgt?? 10? 10208.139 ????? 0.045??? B/op UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000? avgt?? 10? 93025.923 ????? 0.573??? B/op Patched: Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10??? 304.000 ????? 0.001??? B/op UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10?? 2464.004 ????? 0.012??? B/op UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10? 24064.040 ????? 0.137??? B/op UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10? avgt?? 10?? 1256.001 ????? 0.003??? B/op UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100? avgt?? 10?? 9720.153 ????? 0.055??? B/op UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000? avgt?? 10? 88905.688 ????? 0.574??? B/op So what do you think? Is this an improvement? Regards, Peter From peter.levart at gmail.com Mon Jun 11 12:39:38 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 11 Jun 2018 14:39:38 +0200 Subject: BiCollector Message-ID: Hi, Have you ever wanted to perform a collection of the same Stream into two different targets using two Collectors? Say you wanted to collect Map.Entry elements into two parallel lists, each of them containing keys and values respectively. Or you wanted to collect elements into? groups by some key, but also count them at the same time? Currently this is not possible to do with a single Stream. You have to create two identical streams, so you end up passing Supplier to other methods instead of bare Stream. I created a little utility Collector implementation that serves the purpose quite well: /** ?* A {@link Collector} implementation taking two delegate Collector(s) and producing result composed ?* of two results produced by delegating collectors, wrapped in {@link Map.Entry} object. ?* ?* @param the type of elements collected ?* @param the type of 1st delegate collector collected result ?* @param tye type of 2nd delegate collector collected result ?*/ public class BiCollector implements Collector, Map.Entry> { ??? private final Collector keyCollector; ??? private final Collector valCollector; ??? @SuppressWarnings("unchecked") ??? public BiCollector(Collector keyCollector, Collector valCollector) { ??????? this.keyCollector = (Collector) Objects.requireNonNull(keyCollector); ??????? this.valCollector = (Collector) Objects.requireNonNull(valCollector); ??? } ??? @Override ??? public Supplier> supplier() { ??????? Supplier keySupplier = keyCollector.supplier(); ??????? Supplier valSupplier = valCollector.supplier(); ??????? return () -> new AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); ??? } ??? @Override ??? public BiConsumer, T> accumulator() { ??????? BiConsumer keyAccumulator = keyCollector.accumulator(); ??????? BiConsumer valAccumulator = valCollector.accumulator(); ??????? return (accumulation, t) -> { ??????????? keyAccumulator.accept(accumulation.getKey(), t); ??????????? valAccumulator.accept(accumulation.getValue(), t); ??????? }; ??? } ??? @Override ??? public BinaryOperator> combiner() { ??????? BinaryOperator keyCombiner = keyCollector.combiner(); ??????? BinaryOperator valCombiner = valCollector.combiner(); ??????? return (accumulation1, accumulation2) -> new AbstractMap.SimpleImmutableEntry<>( ??????????? keyCombiner.apply(accumulation1.getKey(), accumulation2.getKey()), ??????????? valCombiner.apply(accumulation1.getValue(), accumulation2.getValue()) ??????? ); ??? } ??? @Override ??? public Function, Map.Entry> finisher() { ??????? Function keyFinisher = keyCollector.finisher(); ??????? Function valFinisher = valCollector.finisher(); ??????? return accumulation -> new AbstractMap.SimpleImmutableEntry<>( ??????????? keyFinisher.apply(accumulation.getKey()), ??????????? valFinisher.apply(accumulation.getValue()) ??????? ); ??? } ??? @Override ??? public Set characteristics() { ??????? EnumSet intersection = EnumSet.copyOf(keyCollector.characteristics()); ??????? intersection.retainAll(valCollector.characteristics()); ??????? return intersection; ??? } } Do you think this class is general enough to be part of standard Collectors repertoire? For example, accessed via factory method Collectors.toBoth(Collector coll1, Collector coll2), bi-collection could then be coded simply as: ??????? Map map = ... ??????? Map.Entry, List> keys_values = ??????????? map.entrySet() ?????????????? .stream() ?????????????? .collect( ?????????????????? toBoth( ?????????????????????? mapping(Map.Entry::getKey, toList()), ?????????????????????? mapping(Map.Entry::getValue, toList()) ?????????????????? ) ?????????????? ); ??????? Map.Entry, Long> histogram_count = ??????????? ThreadLocalRandom ??????????????? .current() ??????????????? .ints(100, 0, 10) ??????????????? .boxed() ??????????????? .collect( ??????????????????? toBoth( ??????????????????????? groupingBy(Function.identity(), counting()), ??????????????????????? counting() ??????????????????? ) ??????????????? ); Regards, Peter From thomas.stuefe at gmail.com Mon Jun 11 13:13:23 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 11 Jun 2018 15:13:23 +0200 Subject: RFR(xs): 8204663: clean up remaining native parts after JDK-8187631 Message-ID: Hi, may I please have reviews for this small cleanup. Bug: https://bugs.openjdk.java.net/browse/JDK-8204663 Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/JDK-8204663-clean-up-after-JDK-8187631/webrev.00/webrev/ This change removes some native functions from FOS/FIS which are orphaned and unused since "JDK-8187631: Refactor FileDescriptor close implementation". Tests on jdk-submit came back clean save one strange test error on windows which I cannot reproduce locally. I am investigating that one. Thank you, Thomas From bob.vandette at oracle.com Mon Jun 11 14:12:29 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 11 Jun 2018 10:12:29 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> Message-ID: > On Jun 11, 2018, at 4:32 AM, David Holmes wrote: > > Sorry Bob I haven't had a chance to look at this detail. > > For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. > > For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? The comments reads: * Returns the length of the operating system time slice, in * milliseconds, for processes within the Isolation Group. The comment does infer that it?s process and cgroup (Isolation group) specific and not the generic os timeslice. Isn?t this sufficient? Thanks, Bob. > > David > > On 8/06/2018 3:43 AM, Bob Vandette wrote: >> Can I get one more reviewer for this RFE so I can integrate it? >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >> Mandy Chung has reviewed this change. >> I?ve run Mach5 hotspot and core lib tests. >> I?ve reviewed the tests which were written by Harsha Wardhana >> I filed a CSR for the command line change and it?s now approved and closed. >> Thanks, >> Bob. >>> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >>> >>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>> information. See the sample output below: >>> >>> RFE: Container Metrics >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> >>> >>> This commit will also include a fix for the following bug. >>> >>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>> >>> SAMPLE USAGE and OUTPUT: >>> >>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>> ./java -XshowSettings:system >>> Operating System Metrics: >>> Provider: cgroupv1 >>> Effective CPU Count: 4 >>> CPU Period: 100000 >>> CPU Quota: -1 >>> CPU Shares: -1 >>> List of Processors, 4 total: >>> 4 5 6 7 >>> List of Effective Processors, 4 total: >>> 4 5 6 7 >>> List of Memory Nodes, 2 total: >>> 0 1 >>> List of Available Memory Nodes, 2 total: >>> 0 1 >>> CPUSet Memory Pressure Enabled: false >>> Memory Limit: 256.00M >>> Memory Soft Limit: Unlimited >>> Memory & Swap Limit: 512.00M >>> Kernel Memory Limit: Unlimited >>> TCP Memory Limit: Unlimited >>> Out Of Memory Killer Enabled: true >>> >>> TEST RESULTS: >>> >>> testing runtime container APIs >>> Directory "JTwork" not found: creating >>> Passed: runtime/containers/cgroup/PlainRead.java >>> Passed: runtime/containers/docker/DockerBasicTest.java >>> Passed: runtime/containers/docker/TestCPUAwareness.java >>> Passed: runtime/containers/docker/TestCPUSets.java >>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>> Passed: runtime/containers/docker/TestMisc.java >>> Test results: passed: 6 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing jdk.internal.platform APIs >>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>> Test results: passed: 4 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing -XshowSettings:system launcher option >>> Passed: tools/launcher/Settings.java >>> Test results: passed: 1 >>> >>> >>> Bob. >>> >>> From roger.riggs at oracle.com Mon Jun 11 15:55:32 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Mon, 11 Jun 2018 11:55:32 -0400 Subject: RFR(xs): 8204663: clean up remaining native parts after JDK-8187631 In-Reply-To: References: Message-ID: <4528b331-12d0-931d-ed09-6a89ecc95055@oracle.com> On 6/11/18 9:13 AM, Thomas St?fe wrote: > Hi, > > may I please have reviews for this small cleanup. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204663 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/JDK-8204663-clean-up-after-JDK-8187631/webrev.00/webrev/ > > This change removes some native functions from FOS/FIS which are > orphaned and unused since "JDK-8187631: Refactor FileDescriptor close > implementation". > > Tests on jdk-submit came back clean save one strange test error on > windows which I cannot reproduce locally. I am investigating that one. > > Thank you, Thomas From roger.riggs at oracle.com Mon Jun 11 15:56:18 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Mon, 11 Jun 2018 11:56:18 -0400 Subject: RFR(xs): 8204663: clean up remaining native parts after JDK-8187631 In-Reply-To: References: Message-ID: <7f4ab1a7-2fbd-7731-dd14-78227b82e207@oracle.com> Hi Thomas, Looks fine.? (And also passed the tests I ran). Thanks, Roger On 6/11/18 9:13 AM, Thomas St?fe wrote: > Hi, > > may I please have reviews for this small cleanup. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204663 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/JDK-8204663-clean-up-after-JDK-8187631/webrev.00/webrev/ > > This change removes some native functions from FOS/FIS which are > orphaned and unused since "JDK-8187631: Refactor FileDescriptor close > implementation". > > Tests on jdk-submit came back clean save one strange test error on > windows which I cannot reproduce locally. I am investigating that one. > > Thank you, Thomas From thomas.stuefe at gmail.com Mon Jun 11 16:00:19 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 11 Jun 2018 18:00:19 +0200 Subject: RFR(xs): 8204663: clean up remaining native parts after JDK-8187631 In-Reply-To: <7f4ab1a7-2fbd-7731-dd14-78227b82e207@oracle.com> References: <7f4ab1a7-2fbd-7731-dd14-78227b82e207@oracle.com> Message-ID: Thank you Roger! I re-ran submission tests and all went well this time. Must have been a windows specific fluke. ..Thomas On Mon, Jun 11, 2018 at 5:56 PM, Roger Riggs wrote: > Hi Thomas, > > Looks fine. (And also passed the tests I ran). > > Thanks, Roger > > > On 6/11/18 9:13 AM, Thomas St?fe wrote: >> >> Hi, >> >> may I please have reviews for this small cleanup. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204663 >> Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/JDK-8204663-clean-up-after-JDK-8187631/webrev.00/webrev/ >> >> This change removes some native functions from FOS/FIS which are >> orphaned and unused since "JDK-8187631: Refactor FileDescriptor close >> implementation". >> >> Tests on jdk-submit came back clean save one strange test error on >> windows which I cannot reproduce locally. I am investigating that one. >> >> Thank you, Thomas > > From paul.sandoz at oracle.com Mon Jun 11 17:10:53 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 11 Jun 2018 10:10:53 -0700 Subject: BiCollector In-Reply-To: References: Message-ID: Hi Peter, I like it and can see it being useful, thanks for sharing. I am hesitating a little about it being in the JDK because there is the larger abstraction of a BiStream, where a similar form of collection would naturally fit (but perhaps without the intersection constraints for the characteristics?). We experimented a few times with BiStream and got quite far but decided pull back due to the lack of value types and specialized generics. So i dunno how this might turn out in the future and if your BiCollector fits nicely into such a future model. What are you thoughts on this? FWIW i would call it a ?splitting? or ?bisecting" collector e.g. ?s.collect(bisecting(?))? Paul. > On Jun 11, 2018, at 5:39 AM, Peter Levart wrote: > > Hi, > > Have you ever wanted to perform a collection of the same Stream into two different targets using two Collectors? Say you wanted to collect Map.Entry elements into two parallel lists, each of them containing keys and values respectively. Or you wanted to collect elements into groups by some key, but also count them at the same time? Currently this is not possible to do with a single Stream. You have to create two identical streams, so you end up passing Supplier to other methods instead of bare Stream. > > I created a little utility Collector implementation that serves the purpose quite well: > > /** > * A {@link Collector} implementation taking two delegate Collector(s) and producing result composed > * of two results produced by delegating collectors, wrapped in {@link Map.Entry} object. > * > * @param the type of elements collected > * @param the type of 1st delegate collector collected result > * @param tye type of 2nd delegate collector collected result > */ > public class BiCollector implements Collector, Map.Entry> { > private final Collector keyCollector; > private final Collector valCollector; > > @SuppressWarnings("unchecked") > public BiCollector(Collector keyCollector, Collector valCollector) { > this.keyCollector = (Collector) Objects.requireNonNull(keyCollector); > this.valCollector = (Collector) Objects.requireNonNull(valCollector); > } > > @Override > public Supplier> supplier() { > Supplier keySupplier = keyCollector.supplier(); > Supplier valSupplier = valCollector.supplier(); > return () -> new AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); > } > > @Override > public BiConsumer, T> accumulator() { > BiConsumer keyAccumulator = keyCollector.accumulator(); > BiConsumer valAccumulator = valCollector.accumulator(); > return (accumulation, t) -> { > keyAccumulator.accept(accumulation.getKey(), t); > valAccumulator.accept(accumulation.getValue(), t); > }; > } > > @Override > public BinaryOperator> combiner() { > BinaryOperator keyCombiner = keyCollector.combiner(); > BinaryOperator valCombiner = valCollector.combiner(); > return (accumulation1, accumulation2) -> new AbstractMap.SimpleImmutableEntry<>( > keyCombiner.apply(accumulation1.getKey(), accumulation2.getKey()), > valCombiner.apply(accumulation1.getValue(), accumulation2.getValue()) > ); > } > > @Override > public Function, Map.Entry> finisher() { > Function keyFinisher = keyCollector.finisher(); > Function valFinisher = valCollector.finisher(); > return accumulation -> new AbstractMap.SimpleImmutableEntry<>( > keyFinisher.apply(accumulation.getKey()), > valFinisher.apply(accumulation.getValue()) > ); > } > > @Override > public Set characteristics() { > EnumSet intersection = EnumSet.copyOf(keyCollector.characteristics()); > intersection.retainAll(valCollector.characteristics()); > return intersection; > } > } > > > Do you think this class is general enough to be part of standard Collectors repertoire? > > For example, accessed via factory method Collectors.toBoth(Collector coll1, Collector coll2), bi-collection could then be coded simply as: > > Map map = ... > > Map.Entry, List> keys_values = > map.entrySet() > .stream() > .collect( > toBoth( > mapping(Map.Entry::getKey, toList()), > mapping(Map.Entry::getValue, toList()) > ) > ); > > > Map.Entry, Long> histogram_count = > ThreadLocalRandom > .current() > .ints(100, 0, 10) > .boxed() > .collect( > toBoth( > groupingBy(Function.identity(), counting()), > counting() > ) > ); > > > Regards, Peter > From maurizio.cimadamore at oracle.com Mon Jun 11 17:32:46 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 11 Jun 2018 18:32:46 +0100 Subject: BiCollector In-Reply-To: References: Message-ID: Note also that this has some overlappings with Collectors.partitioningBy - which currently wraps results into a Map, where O is the desired collector output type. Without commenting on the feasibility of its inclusion in the JDK (Paul rules here :-)), I note that BiStream would obviously allow this functionality to be exposed in a more user friendly way. Cheers Maurizio On 11/06/18 13:39, Peter Levart wrote: > Hi, > > Have you ever wanted to perform a collection of the same Stream into > two different targets using two Collectors? Say you wanted to collect > Map.Entry elements into two parallel lists, each of them containing > keys and values respectively. Or you wanted to collect elements into? > groups by some key, but also count them at the same time? Currently > this is not possible to do with a single Stream. You have to create > two identical streams, so you end up passing Supplier to other > methods instead of bare Stream. > > I created a little utility Collector implementation that serves the > purpose quite well: > > /** > ?* A {@link Collector} implementation taking two delegate Collector(s) > and producing result composed > ?* of two results produced by delegating collectors, wrapped in {@link > Map.Entry} object. > ?* > ?* @param the type of elements collected > ?* @param the type of 1st delegate collector collected result > ?* @param tye type of 2nd delegate collector collected result > ?*/ > public class BiCollector implements Collector Map.Entry, Map.Entry> { > ??? private final Collector keyCollector; > ??? private final Collector valCollector; > > ??? @SuppressWarnings("unchecked") > ??? public BiCollector(Collector keyCollector, Collector ?, V> valCollector) { > ??????? this.keyCollector = (Collector) > Objects.requireNonNull(keyCollector); > ??????? this.valCollector = (Collector) > Objects.requireNonNull(valCollector); > ??? } > > ??? @Override > ??? public Supplier> supplier() { > ??????? Supplier keySupplier = keyCollector.supplier(); > ??????? Supplier valSupplier = valCollector.supplier(); > ??????? return () -> new > AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); > ??? } > > ??? @Override > ??? public BiConsumer, T> accumulator() { > ??????? BiConsumer keyAccumulator = > keyCollector.accumulator(); > ??????? BiConsumer valAccumulator = > valCollector.accumulator(); > ??????? return (accumulation, t) -> { > ??????????? keyAccumulator.accept(accumulation.getKey(), t); > ??????????? valAccumulator.accept(accumulation.getValue(), t); > ??????? }; > ??? } > > ??? @Override > ??? public BinaryOperator> combiner() { > ??????? BinaryOperator keyCombiner = keyCollector.combiner(); > ??????? BinaryOperator valCombiner = valCollector.combiner(); > ??????? return (accumulation1, accumulation2) -> new > AbstractMap.SimpleImmutableEntry<>( > ??????????? keyCombiner.apply(accumulation1.getKey(), > accumulation2.getKey()), > ??????????? valCombiner.apply(accumulation1.getValue(), > accumulation2.getValue()) > ??????? ); > ??? } > > ??? @Override > ??? public Function, Map.Entry> > finisher() { > ??????? Function keyFinisher = keyCollector.finisher(); > ??????? Function valFinisher = valCollector.finisher(); > ??????? return accumulation -> new AbstractMap.SimpleImmutableEntry<>( > ??????????? keyFinisher.apply(accumulation.getKey()), > ??????????? valFinisher.apply(accumulation.getValue()) > ??????? ); > ??? } > > ??? @Override > ??? public Set characteristics() { > ??????? EnumSet intersection = > EnumSet.copyOf(keyCollector.characteristics()); > ??????? intersection.retainAll(valCollector.characteristics()); > ??????? return intersection; > ??? } > } > > > Do you think this class is general enough to be part of standard > Collectors repertoire? > > For example, accessed via factory method Collectors.toBoth(Collector > coll1, Collector coll2), bi-collection could then be coded simply as: > > ??????? Map map = ... > > ??????? Map.Entry, List> keys_values = > ??????????? map.entrySet() > ?????????????? .stream() > ?????????????? .collect( > ?????????????????? toBoth( > ?????????????????????? mapping(Map.Entry::getKey, toList()), > ?????????????????????? mapping(Map.Entry::getValue, toList()) > ?????????????????? ) > ?????????????? ); > > > ??????? Map.Entry, Long> histogram_count = > ??????????? ThreadLocalRandom > ??????????????? .current() > ??????????????? .ints(100, 0, 10) > ??????????????? .boxed() > ??????????????? .collect( > ??????????????????? toBoth( > ??????????????????????? groupingBy(Function.identity(), counting()), > ??????????????????????? counting() > ??????????????????? ) > ??????????????? ); > > > Regards, Peter > From kirk.pepperdine at gmail.com Mon Jun 11 17:45:33 2018 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Mon, 11 Jun 2018 20:45:33 +0300 Subject: BiCollector In-Reply-To: References: Message-ID: <5E1659A3-9309-4C27-A4F6-37686BA62DF1@gmail.com> Hi, I?ve created one of my own and I?d happily toss it for a standard implementation. ? Kirk > On Jun 11, 2018, at 8:10 PM, Paul Sandoz wrote: > > Hi Peter, > > I like it and can see it being useful, thanks for sharing. > > I am hesitating a little about it being in the JDK because there is the larger abstraction of a BiStream, where a similar form of collection would naturally fit (but perhaps without the intersection constraints for the characteristics?). We experimented a few times with BiStream and got quite far but decided pull back due to the lack of value types and specialized generics. So i dunno how this might turn out in the future and if your BiCollector fits nicely into such a future model. > > What are you thoughts on this? > > FWIW i would call it a ?splitting? or ?bisecting" collector e.g. ?s.collect(bisecting(?))? > > Paul. > > > > >> On Jun 11, 2018, at 5:39 AM, Peter Levart wrote: >> >> Hi, >> >> Have you ever wanted to perform a collection of the same Stream into two different targets using two Collectors? Say you wanted to collect Map.Entry elements into two parallel lists, each of them containing keys and values respectively. Or you wanted to collect elements into groups by some key, but also count them at the same time? Currently this is not possible to do with a single Stream. You have to create two identical streams, so you end up passing Supplier to other methods instead of bare Stream. >> >> I created a little utility Collector implementation that serves the purpose quite well: >> >> /** >> * A {@link Collector} implementation taking two delegate Collector(s) and producing result composed >> * of two results produced by delegating collectors, wrapped in {@link Map.Entry} object. >> * >> * @param the type of elements collected >> * @param the type of 1st delegate collector collected result >> * @param tye type of 2nd delegate collector collected result >> */ >> public class BiCollector implements Collector, Map.Entry> { >> private final Collector keyCollector; >> private final Collector valCollector; >> >> @SuppressWarnings("unchecked") >> public BiCollector(Collector keyCollector, Collector valCollector) { >> this.keyCollector = (Collector) Objects.requireNonNull(keyCollector); >> this.valCollector = (Collector) Objects.requireNonNull(valCollector); >> } >> >> @Override >> public Supplier> supplier() { >> Supplier keySupplier = keyCollector.supplier(); >> Supplier valSupplier = valCollector.supplier(); >> return () -> new AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); >> } >> >> @Override >> public BiConsumer, T> accumulator() { >> BiConsumer keyAccumulator = keyCollector.accumulator(); >> BiConsumer valAccumulator = valCollector.accumulator(); >> return (accumulation, t) -> { >> keyAccumulator.accept(accumulation.getKey(), t); >> valAccumulator.accept(accumulation.getValue(), t); >> }; >> } >> >> @Override >> public BinaryOperator> combiner() { >> BinaryOperator keyCombiner = keyCollector.combiner(); >> BinaryOperator valCombiner = valCollector.combiner(); >> return (accumulation1, accumulation2) -> new AbstractMap.SimpleImmutableEntry<>( >> keyCombiner.apply(accumulation1.getKey(), accumulation2.getKey()), >> valCombiner.apply(accumulation1.getValue(), accumulation2.getValue()) >> ); >> } >> >> @Override >> public Function, Map.Entry> finisher() { >> Function keyFinisher = keyCollector.finisher(); >> Function valFinisher = valCollector.finisher(); >> return accumulation -> new AbstractMap.SimpleImmutableEntry<>( >> keyFinisher.apply(accumulation.getKey()), >> valFinisher.apply(accumulation.getValue()) >> ); >> } >> >> @Override >> public Set characteristics() { >> EnumSet intersection = EnumSet.copyOf(keyCollector.characteristics()); >> intersection.retainAll(valCollector.characteristics()); >> return intersection; >> } >> } >> >> >> Do you think this class is general enough to be part of standard Collectors repertoire? >> >> For example, accessed via factory method Collectors.toBoth(Collector coll1, Collector coll2), bi-collection could then be coded simply as: >> >> Map map = ... >> >> Map.Entry, List> keys_values = >> map.entrySet() >> .stream() >> .collect( >> toBoth( >> mapping(Map.Entry::getKey, toList()), >> mapping(Map.Entry::getValue, toList()) >> ) >> ); >> >> >> Map.Entry, Long> histogram_count = >> ThreadLocalRandom >> .current() >> .ints(100, 0, 10) >> .boxed() >> .collect( >> toBoth( >> groupingBy(Function.identity(), counting()), >> counting() >> ) >> ); >> >> >> Regards, Peter >> > From mandy.chung at oracle.com Mon Jun 11 17:48:21 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Jun 2018 10:48:21 -0700 Subject: [JDK 11] RFR 8201528: Add new test to check for package versioning information in OpenJDK In-Reply-To: <3C0AFC7B-AA3C-4DE1-AF9A-0DB556A1AEAB@oracle.com> References: <08fb7059-2e3f-d77b-bd60-69bff4517edc@oracle.com> <7D1544F3-4BD6-48FA-AB7B-BA3941EEE04E@oracle.com> <8c813a78-e769-f453-0c37-61a2166105f1@oracle.com> <3C0AFC7B-AA3C-4DE1-AF9A-0DB556A1AEAB@oracle.com> Message-ID: <9711e185-356e-9e87-b991-b34f6ccbbc12@oracle.com> On 6/10/18 10:12 PM, Chris Yin wrote: > Hi, Mandy > > Thanks lot for your suggestions, update webrev as below and comment > inline, thanks > > http://cr.openjdk.java.net/~xyin/8201528/webrev.02/ Looks good. Thanks for the update and this is much easier to understand the test cases. Mandy From bob.vandette at oracle.com Mon Jun 11 18:28:05 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 11 Jun 2018 14:28:05 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Message-ID: <54435D5C-5E6B-4174-8344-984CDDF5D46A@oracle.com> > On Jun 11, 2018, at 4:07 AM, Robbin Ehn wrote: > > Hi Bob, > > On 06/07/2018 07:43 PM, Bob Vandette wrote: >> Can I get one more reviewer for this RFE so I can integrate it? >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 > > Seems okay. > > Metrics.java > "Returns the length of the operating system time slice" > > Note that is is only true if you are using a batch scheduler. > Otherwise this period may be split on multiple 'time slices?. This is a cgroup metric which uses CFS not the OS time slice. 136 /** 137 * Returns the length of the operating system time slice, in 138 * milliseconds, for processes within the Isolation Group. > > In printSystemMetrics there is no units, maybe intentional? I?ll add ms for the quote/period output. The memory metrics do have units. > > Do we have support now in mach5 for docker jtreg, or do we still run these separate? > > You can ship it. Thanks! Bob. > > Thanks for fixing, and super thanks for fixing the bug in PlainRead also! > > /Robbin > >> Mandy Chung has reviewed this change. >> I?ve run Mach5 hotspot and core lib tests. >> I?ve reviewed the tests which were written by Harsha Wardhana >> I filed a CSR for the command line change and it?s now approved and closed. >> Thanks, >> Bob. >>> On May 30, 2018, at 3:45 PM, Bob Vandette wrote: >>> >>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>> information. See the sample output below: >>> >>> RFE: Container Metrics >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> >>> >>> This commit will also include a fix for the following bug. >>> >>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>> >>> WEBREV: >>> >>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>> >>> SAMPLE USAGE and OUTPUT: >>> >>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>> ./java -XshowSettings:system >>> Operating System Metrics: >>> Provider: cgroupv1 >>> Effective CPU Count: 4 >>> CPU Period: 100000 >>> CPU Quota: -1 >>> CPU Shares: -1 >>> List of Processors, 4 total: >>> 4 5 6 7 >>> List of Effective Processors, 4 total: >>> 4 5 6 7 >>> List of Memory Nodes, 2 total: >>> 0 1 >>> List of Available Memory Nodes, 2 total: >>> 0 1 >>> CPUSet Memory Pressure Enabled: false >>> Memory Limit: 256.00M >>> Memory Soft Limit: Unlimited >>> Memory & Swap Limit: 512.00M >>> Kernel Memory Limit: Unlimited >>> TCP Memory Limit: Unlimited >>> Out Of Memory Killer Enabled: true >>> >>> TEST RESULTS: >>> >>> testing runtime container APIs >>> Directory "JTwork" not found: creating >>> Passed: runtime/containers/cgroup/PlainRead.java >>> Passed: runtime/containers/docker/DockerBasicTest.java >>> Passed: runtime/containers/docker/TestCPUAwareness.java >>> Passed: runtime/containers/docker/TestCPUSets.java >>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>> Passed: runtime/containers/docker/TestMisc.java >>> Test results: passed: 6 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing jdk.internal.platform APIs >>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>> Test results: passed: 4 >>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>> >>> testing -XshowSettings:system launcher option >>> Passed: tools/launcher/Settings.java >>> Test results: passed: 1 >>> >>> >>> Bob. >>> >>> From peter.levart at gmail.com Mon Jun 11 18:28:36 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 11 Jun 2018 20:28:36 +0200 Subject: BiCollector In-Reply-To: References: Message-ID: Hi Paul, Can you point me to some BiStream code (if it is available publicly)? Thanks, Peter On 06/11/18 19:10, Paul Sandoz wrote: > Hi Peter, > > I like it and can see it being useful, thanks for sharing. > > I am hesitating a little about it being in the JDK because there is the larger abstraction of a BiStream, where a similar form of collection would naturally fit (but perhaps without the intersection constraints for the characteristics?). We experimented a few times with BiStream and got quite far but decided pull back due to the lack of value types and specialized generics. So i dunno how this might turn out in the future and if your BiCollector fits nicely into such a future model. > > What are you thoughts on this? > > FWIW i would call it a ?splitting? or ?bisecting" collector e.g. ?s.collect(bisecting(?))? > > Paul. > > > > >> On Jun 11, 2018, at 5:39 AM, Peter Levart wrote: >> >> Hi, >> >> Have you ever wanted to perform a collection of the same Stream into two different targets using two Collectors? Say you wanted to collect Map.Entry elements into two parallel lists, each of them containing keys and values respectively. Or you wanted to collect elements into groups by some key, but also count them at the same time? Currently this is not possible to do with a single Stream. You have to create two identical streams, so you end up passing Supplier to other methods instead of bare Stream. >> >> I created a little utility Collector implementation that serves the purpose quite well: >> >> /** >> * A {@link Collector} implementation taking two delegate Collector(s) and producing result composed >> * of two results produced by delegating collectors, wrapped in {@link Map.Entry} object. >> * >> * @param the type of elements collected >> * @param the type of 1st delegate collector collected result >> * @param tye type of 2nd delegate collector collected result >> */ >> public class BiCollector implements Collector, Map.Entry> { >> private final Collector keyCollector; >> private final Collector valCollector; >> >> @SuppressWarnings("unchecked") >> public BiCollector(Collector keyCollector, Collector valCollector) { >> this.keyCollector = (Collector) Objects.requireNonNull(keyCollector); >> this.valCollector = (Collector) Objects.requireNonNull(valCollector); >> } >> >> @Override >> public Supplier> supplier() { >> Supplier keySupplier = keyCollector.supplier(); >> Supplier valSupplier = valCollector.supplier(); >> return () -> new AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); >> } >> >> @Override >> public BiConsumer, T> accumulator() { >> BiConsumer keyAccumulator = keyCollector.accumulator(); >> BiConsumer valAccumulator = valCollector.accumulator(); >> return (accumulation, t) -> { >> keyAccumulator.accept(accumulation.getKey(), t); >> valAccumulator.accept(accumulation.getValue(), t); >> }; >> } >> >> @Override >> public BinaryOperator> combiner() { >> BinaryOperator keyCombiner = keyCollector.combiner(); >> BinaryOperator valCombiner = valCollector.combiner(); >> return (accumulation1, accumulation2) -> new AbstractMap.SimpleImmutableEntry<>( >> keyCombiner.apply(accumulation1.getKey(), accumulation2.getKey()), >> valCombiner.apply(accumulation1.getValue(), accumulation2.getValue()) >> ); >> } >> >> @Override >> public Function, Map.Entry> finisher() { >> Function keyFinisher = keyCollector.finisher(); >> Function valFinisher = valCollector.finisher(); >> return accumulation -> new AbstractMap.SimpleImmutableEntry<>( >> keyFinisher.apply(accumulation.getKey()), >> valFinisher.apply(accumulation.getValue()) >> ); >> } >> >> @Override >> public Set characteristics() { >> EnumSet intersection = EnumSet.copyOf(keyCollector.characteristics()); >> intersection.retainAll(valCollector.characteristics()); >> return intersection; >> } >> } >> >> >> Do you think this class is general enough to be part of standard Collectors repertoire? >> >> For example, accessed via factory method Collectors.toBoth(Collector coll1, Collector coll2), bi-collection could then be coded simply as: >> >> Map map = ... >> >> Map.Entry, List> keys_values = >> map.entrySet() >> .stream() >> .collect( >> toBoth( >> mapping(Map.Entry::getKey, toList()), >> mapping(Map.Entry::getValue, toList()) >> ) >> ); >> >> >> Map.Entry, Long> histogram_count = >> ThreadLocalRandom >> .current() >> .ints(100, 0, 10) >> .boxed() >> .collect( >> toBoth( >> groupingBy(Function.identity(), counting()), >> counting() >> ) >> ); >> >> >> Regards, Peter >> From forax at univ-mlv.fr Mon Jun 11 18:40:33 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Mon, 11 Jun 2018 20:40:33 +0200 (CEST) Subject: BiCollector In-Reply-To: References: Message-ID: <1111894492.377996.1528742433755.JavaMail.zimbra@u-pem.fr> Hi Peter, You may find something on lambda-dev, i remember that we discussed about BiStream, and as Paul said we decide to not include it because the performance were not great and it adds another axis to the API making it even harder to retrofitted it when we will introduce specialization. regards, R?mi ----- Mail original ----- > De: "Peter Levart" > ?: "Paul Sandoz" > Cc: "core-libs-dev" > Envoy?: Lundi 11 Juin 2018 20:28:36 > Objet: Re: BiCollector > Hi Paul, > > Can you point me to some BiStream code (if it is available publicly)? > > Thanks, Peter > > On 06/11/18 19:10, Paul Sandoz wrote: >> Hi Peter, >> >> I like it and can see it being useful, thanks for sharing. >> >> I am hesitating a little about it being in the JDK because there is the larger >> abstraction of a BiStream, where a similar form of collection would naturally >> fit (but perhaps without the intersection constraints for the >> characteristics?). We experimented a few times with BiStream and got quite far >> but decided pull back due to the lack of value types and specialized generics. >> So i dunno how this might turn out in the future and if your BiCollector fits >> nicely into such a future model. >> >> What are you thoughts on this? >> >> FWIW i would call it a ?splitting? or ?bisecting" collector e.g. >> ?s.collect(bisecting(?))? >> >> Paul. >> >> >> >> >>> On Jun 11, 2018, at 5:39 AM, Peter Levart wrote: >>> >>> Hi, >>> >>> Have you ever wanted to perform a collection of the same Stream into two >>> different targets using two Collectors? Say you wanted to collect Map.Entry >>> elements into two parallel lists, each of them containing keys and values >>> respectively. Or you wanted to collect elements into groups by some key, but >>> also count them at the same time? Currently this is not possible to do with a >>> single Stream. You have to create two identical streams, so you end up passing >>> Supplier to other methods instead of bare Stream. >>> >>> I created a little utility Collector implementation that serves the purpose >>> quite well: >>> >>> /** >>> * A {@link Collector} implementation taking two delegate Collector(s) and >>> producing result composed >>> * of two results produced by delegating collectors, wrapped in {@link Map.Entry} >>> object. >>> * >>> * @param the type of elements collected >>> * @param the type of 1st delegate collector collected result >>> * @param tye type of 2nd delegate collector collected result >>> */ >>> public class BiCollector implements Collector>> Object>, Map.Entry> { >>> private final Collector keyCollector; >>> private final Collector valCollector; >>> >>> @SuppressWarnings("unchecked") >>> public BiCollector(Collector keyCollector, Collector >>> valCollector) { >>> this.keyCollector = (Collector) Objects.requireNonNull(keyCollector); >>> this.valCollector = (Collector) Objects.requireNonNull(valCollector); >>> } >>> >>> @Override >>> public Supplier> supplier() { >>> Supplier keySupplier = keyCollector.supplier(); >>> Supplier valSupplier = valCollector.supplier(); >>> return () -> new AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), >>> valSupplier.get()); >>> } >>> >>> @Override >>> public BiConsumer, T> accumulator() { >>> BiConsumer keyAccumulator = keyCollector.accumulator(); >>> BiConsumer valAccumulator = valCollector.accumulator(); >>> return (accumulation, t) -> { >>> keyAccumulator.accept(accumulation.getKey(), t); >>> valAccumulator.accept(accumulation.getValue(), t); >>> }; >>> } >>> >>> @Override >>> public BinaryOperator> combiner() { >>> BinaryOperator keyCombiner = keyCollector.combiner(); >>> BinaryOperator valCombiner = valCollector.combiner(); >>> return (accumulation1, accumulation2) -> new AbstractMap.SimpleImmutableEntry<>( >>> keyCombiner.apply(accumulation1.getKey(), accumulation2.getKey()), >>> valCombiner.apply(accumulation1.getValue(), accumulation2.getValue()) >>> ); >>> } >>> >>> @Override >>> public Function, Map.Entry> finisher() { >>> Function keyFinisher = keyCollector.finisher(); >>> Function valFinisher = valCollector.finisher(); >>> return accumulation -> new AbstractMap.SimpleImmutableEntry<>( >>> keyFinisher.apply(accumulation.getKey()), >>> valFinisher.apply(accumulation.getValue()) >>> ); >>> } >>> >>> @Override >>> public Set characteristics() { >>> EnumSet intersection = >>> EnumSet.copyOf(keyCollector.characteristics()); >>> intersection.retainAll(valCollector.characteristics()); >>> return intersection; >>> } >>> } >>> >>> >>> Do you think this class is general enough to be part of standard Collectors >>> repertoire? >>> >>> For example, accessed via factory method Collectors.toBoth(Collector coll1, >>> Collector coll2), bi-collection could then be coded simply as: >>> >>> Map map = ... >>> >>> Map.Entry, List> keys_values = >>> map.entrySet() >>> .stream() >>> .collect( >>> toBoth( >>> mapping(Map.Entry::getKey, toList()), >>> mapping(Map.Entry::getValue, toList()) >>> ) >>> ); >>> >>> >>> Map.Entry, Long> histogram_count = >>> ThreadLocalRandom >>> .current() >>> .ints(100, 0, 10) >>> .boxed() >>> .collect( >>> toBoth( >>> groupingBy(Function.identity(), counting()), >>> counting() >>> ) >>> ); >>> >>> >>> Regards, Peter From david.holmes at oracle.com Mon Jun 11 21:21:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 Jun 2018 07:21:23 +1000 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> Message-ID: <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> On 12/06/2018 12:12 AM, Bob Vandette wrote: > >> On Jun 11, 2018, at 4:32 AM, David Holmes > > wrote: >> >> Sorry Bob I haven't had a chance to look at this detail. >> >> For the Java code ... methods that return arrays should return >> zero-length arrays when something is not available rather than null. > > All methods do return zero length arrays except I missed the > getPerCpuUsage. ?I?ll fix that one and correct the javadoc. There are a few more too: 231 * @return An array of available CPUs or null if metric is not available. 232 * 233 */ 234 public int[] getCpuSetCpus(); 242 * @return An array of available and online CPUs or null if the metric 243 * is not available. 244 * 245 */ 246 public int[] getEffectiveCpuSetCpus(); 256 * @return An array of available memory nodes or null if metric is not available. 257 * 258 */ 259 public int[] getCpuSetMems(); 267 * @return An array of available and online nodes or null if the metric 268 * is not available. 269 * 270 */ 271 public int[] getEffectiveCpuSetMems(); > >> >> For getCpuPeriod() the term "operating system time slice" can be >> misconstrued as being related to the scheduler timeslice that may, or >> may not, exist, depending on the scheduler and scheduling policy etc. >> This "timeslice" is something specific to cgroups - no? > > The comments reads: > > * Returns the length of the operating system time slice, in > * milliseconds, for processes within the Isolation Group. > > The comment does infer that it?s process and cgroup (Isolation group) > specific and not the generic os timeslice. > Isn?t this sufficient? The phrase "operating system" makes this sound like some kind of global timeslice notion - which it isn't. And I don't like to think of cpu periods/shares/quotas in terms of "time slice" anyway. I don't see the Docker or Cgroup documentation using "time slice" either. It suffices IMHO to just say for period: * Returns the length of the scheduling period, in * milliseconds, for processes within the Isolation Group. then for quota: * Returns the total available run-time allowed, in milliseconds, * during each scheduling period for all tasks in the Isolation Group. Thanks, David > > Thanks, > Bob. >> >> David >> >> On 8/06/2018 3:43 AM, Bob Vandette wrote: >>> Can I get one more reviewer for this RFE so I can integrate it? >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>> Mandy Chung has reviewed this change. >>> I?ve run Mach5 hotspot and core lib tests. >>> I?ve reviewed the tests which were written by Harsha Wardhana >>> I filed a CSR for the command line change and it?s now approved and >>> closed. >>> Thanks, >>> Bob. >>>> On May 30, 2018, at 3:45 PM, Bob Vandette >>> > wrote: >>>> >>>> Please review the following RFE which adds an internal API, along >>>> with jtreg tests that provide >>>> access to Docker container configuration data and metrics. ?In >>>> addition to the API which we hope to >>>> take advantage of in the future with Java Flight Recorder and a JMX >>>> Mbean, I?ve added an additional >>>> option to -XshowSettings:system than dumps out the container or host >>>> cgroup confguration >>>> information. ?See the sample output below: >>>> >>>> RFE: Container Metrics >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>> >>>> >>>> This commit will also include a fix for the following bug. >>>> >>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>> >>>> WEBREV: >>>> >>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>> >>>> SAMPLE USAGE and OUTPUT: >>>> >>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>> ./java -XshowSettings:system >>>> Operating System Metrics: >>>> ???Provider: cgroupv1 >>>> ???Effective CPU Count: 4 >>>> ???CPU Period: 100000 >>>> ???CPU Quota: -1 >>>> ???CPU Shares: -1 >>>> ???List of Processors, 4 total: >>>> ???4 5 6 7 >>>> ???List of Effective Processors, 4 total: >>>> ???4 5 6 7 >>>> ???List of Memory Nodes, 2 total: >>>> ???0 1 >>>> ???List of Available Memory Nodes, 2 total: >>>> ???0 1 >>>> ???CPUSet Memory Pressure Enabled: false >>>> ???Memory Limit: 256.00M >>>> ???Memory Soft Limit: Unlimited >>>> ???Memory & Swap Limit: 512.00M >>>> ???Kernel Memory Limit: Unlimited >>>> ???TCP Memory Limit: Unlimited >>>> ???Out Of Memory Killer Enabled: true >>>> >>>> TEST RESULTS: >>>> >>>> testing runtime container APIs >>>> Directory "JTwork" not found: creating >>>> Passed: runtime/containers/cgroup/PlainRead.java >>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>> Passed: runtime/containers/docker/TestCPUSets.java >>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>> Passed: runtime/containers/docker/TestMisc.java >>>> Test results: passed: 6 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing jdk.internal.platform APIs >>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>> Test results: passed: 4 >>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>> >>>> testing -XshowSettings:system launcher option >>>> Passed: tools/launcher/Settings.java >>>> Test results: passed: 1 >>>> >>>> >>>> Bob. >>>> >>>> > From bob.vandette at oracle.com Mon Jun 11 23:30:04 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 11 Jun 2018 19:30:04 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> Message-ID: > On Jun 11, 2018, at 5:21 PM, David Holmes wrote: > > On 12/06/2018 12:12 AM, Bob Vandette wrote: >>> On Jun 11, 2018, at 4:32 AM, David Holmes > wrote: >>> >>> Sorry Bob I haven't had a chance to look at this detail. >>> >>> For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. >> All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. > > There are a few more too: > Those are covered by the function that converts the string range. > 231 * @return An array of available CPUs or null if metric is not available. > 232 * > 233 */ > 234 public int[] getCpuSetCpus(); > > 242 * @return An array of available and online CPUs or null if the metric > 243 * is not available. > 244 * > 245 */ > 246 public int[] getEffectiveCpuSetCpus(); > > 256 * @return An array of available memory nodes or null if metric is not available. > 257 * > 258 */ > 259 public int[] getCpuSetMems(); > > 267 * @return An array of available and online nodes or null if the metric > 268 * is not available. > 269 * > 270 */ > 271 public int[] getEffectiveCpuSetMems(); >>> >>> For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? >> The comments reads: >> * Returns the length of the operating system time slice, in >> * milliseconds, for processes within the Isolation Group. >> The comment does infer that it?s process and cgroup (Isolation group) specific and not the generic os timeslice. >> Isn?t this sufficient? > > The phrase "operating system" makes this sound like some kind of global timeslice notion - which it isn't. And I don't like to think of cpu periods/shares/quotas in terms of "time slice" anyway. I don't see the Docker or Cgroup documentation using "time slice" either. It suffices IMHO to just say for period: > > * Returns the length of the scheduling period, in > * milliseconds, for processes within the Isolation Group. > > then for quota: > > * Returns the total available run-time allowed, in milliseconds, > * during each scheduling period for all tasks in the Isolation Group. > Ok. I?ll update the docs. Bob > Thanks, > David > >> Thanks, >> Bob. >>> >>> David >>> >>>> On 8/06/2018 3:43 AM, Bob Vandette wrote: >>>> Can I get one more reviewer for this RFE so I can integrate it? >>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>> Mandy Chung has reviewed this change. >>>> I?ve run Mach5 hotspot and core lib tests. >>>> I?ve reviewed the tests which were written by Harsha Wardhana >>>> I filed a CSR for the command line change and it?s now approved and closed. >>>> Thanks, >>>> Bob. >>>>> On May 30, 2018, at 3:45 PM, Bob Vandette > wrote: >>>>> >>>>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>>>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>>>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>>>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>>>> information. See the sample output below: >>>>> >>>>> RFE: Container Metrics >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>>> >>>>> WEBREV: >>>>> >>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>> >>>>> >>>>> This commit will also include a fix for the following bug. >>>>> >>>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>>> >>>>> WEBREV: >>>>> >>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>>> >>>>> SAMPLE USAGE and OUTPUT: >>>>> >>>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>>> ./java -XshowSettings:system >>>>> Operating System Metrics: >>>>> Provider: cgroupv1 >>>>> Effective CPU Count: 4 >>>>> CPU Period: 100000 >>>>> CPU Quota: -1 >>>>> CPU Shares: -1 >>>>> List of Processors, 4 total: >>>>> 4 5 6 7 >>>>> List of Effective Processors, 4 total: >>>>> 4 5 6 7 >>>>> List of Memory Nodes, 2 total: >>>>> 0 1 >>>>> List of Available Memory Nodes, 2 total: >>>>> 0 1 >>>>> CPUSet Memory Pressure Enabled: false >>>>> Memory Limit: 256.00M >>>>> Memory Soft Limit: Unlimited >>>>> Memory & Swap Limit: 512.00M >>>>> Kernel Memory Limit: Unlimited >>>>> TCP Memory Limit: Unlimited >>>>> Out Of Memory Killer Enabled: true >>>>> >>>>> TEST RESULTS: >>>>> >>>>> testing runtime container APIs >>>>> Directory "JTwork" not found: creating >>>>> Passed: runtime/containers/cgroup/PlainRead.java >>>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>>> Passed: runtime/containers/docker/TestCPUSets.java >>>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>>> Passed: runtime/containers/docker/TestMisc.java >>>>> Test results: passed: 6 >>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>> >>>>> testing jdk.internal.platform APIs >>>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>>> Test results: passed: 4 >>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>> >>>>> testing -XshowSettings:system launcher option >>>>> Passed: tools/launcher/Settings.java >>>>> Test results: passed: 1 >>>>> >>>>> >>>>> Bob. >>>>> >>>>> From xu.y.yin at oracle.com Tue Jun 12 00:53:33 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Tue, 12 Jun 2018 08:53:33 +0800 Subject: [JDK 11] RFR 8201528: Add new test to check for package versioning information in OpenJDK In-Reply-To: <9711e185-356e-9e87-b991-b34f6ccbbc12@oracle.com> References: <08fb7059-2e3f-d77b-bd60-69bff4517edc@oracle.com> <7D1544F3-4BD6-48FA-AB7B-BA3941EEE04E@oracle.com> <8c813a78-e769-f453-0c37-61a2166105f1@oracle.com> <3C0AFC7B-AA3C-4DE1-AF9A-0DB556A1AEAB@oracle.com> <9711e185-356e-9e87-b991-b34f6ccbbc12@oracle.com> Message-ID: <6552B418-CF87-4DFE-A82A-75103B84316C@oracle.com> Thank you, Mandy Regards, Chris > On 12 Jun 2018, at 1:48 AM, mandy chung wrote: > > > > On 6/10/18 10:12 PM, Chris Yin wrote: >> Hi, Mandy >> Thanks lot for your suggestions, update webrev as below and comment inline, thanks >> http://cr.openjdk.java.net/~xyin/8201528/webrev.02/ > > > Looks good. Thanks for the update and this is much easier to understand the test cases. > > Mandy From david.holmes at oracle.com Tue Jun 12 05:12:54 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 Jun 2018 15:12:54 +1000 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> Message-ID: <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> On 12/06/2018 9:30 AM, Bob Vandette wrote: > > >> On Jun 11, 2018, at 5:21 PM, David Holmes wrote: >> >> On 12/06/2018 12:12 AM, Bob Vandette wrote: >>>> On Jun 11, 2018, at 4:32 AM, David Holmes > wrote: >>>> >>>> Sorry Bob I haven't had a chance to look at this detail. >>>> >>>> For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. >>> All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. >> >> There are a few more too: >> > > Those are covered by the function that converts the string range. ??? I have no idea what you mean. Java API design style is to return zero-length arrays rather than null. [Ref: Effective Java First Edition, Item 27]. Cheers, David ----- >> 231 * @return An array of available CPUs or null if metric is not available. >> 232 * >> 233 */ >> 234 public int[] getCpuSetCpus(); >> >> 242 * @return An array of available and online CPUs or null if the metric >> 243 * is not available. >> 244 * >> 245 */ >> 246 public int[] getEffectiveCpuSetCpus(); >> >> 256 * @return An array of available memory nodes or null if metric is not available. >> 257 * >> 258 */ >> 259 public int[] getCpuSetMems(); >> >> 267 * @return An array of available and online nodes or null if the metric >> 268 * is not available. >> 269 * >> 270 */ >> 271 public int[] getEffectiveCpuSetMems(); >>>> >>>> For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? >>> The comments reads: >>> * Returns the length of the operating system time slice, in >>> * milliseconds, for processes within the Isolation Group. >>> The comment does infer that it?s process and cgroup (Isolation group) specific and not the generic os timeslice. >>> Isn?t this sufficient? >> >> The phrase "operating system" makes this sound like some kind of global timeslice notion - which it isn't. And I don't like to think of cpu periods/shares/quotas in terms of "time slice" anyway. I don't see the Docker or Cgroup documentation using "time slice" either. It suffices IMHO to just say for period: >> >> * Returns the length of the scheduling period, in >> * milliseconds, for processes within the Isolation Group. >> >> then for quota: >> >> * Returns the total available run-time allowed, in milliseconds, >> * during each scheduling period for all tasks in the Isolation Group. >> > > Ok. I?ll update the docs. > Bob > >> Thanks, >> David >> >>> Thanks, >>> Bob. >>>> >>>> David >>>> >>>>> On 8/06/2018 3:43 AM, Bob Vandette wrote: >>>>> Can I get one more reviewer for this RFE so I can integrate it? >>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>> Mandy Chung has reviewed this change. >>>>> I?ve run Mach5 hotspot and core lib tests. >>>>> I?ve reviewed the tests which were written by Harsha Wardhana >>>>> I filed a CSR for the command line change and it?s now approved and closed. >>>>> Thanks, >>>>> Bob. >>>>>> On May 30, 2018, at 3:45 PM, Bob Vandette > wrote: >>>>>> >>>>>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>>>>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>>>>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>>>>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>>>>> information. See the sample output below: >>>>>> >>>>>> RFE: Container Metrics >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>>>> >>>>>> WEBREV: >>>>>> >>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>>> >>>>>> >>>>>> This commit will also include a fix for the following bug. >>>>>> >>>>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>>>> >>>>>> WEBREV: >>>>>> >>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>>>> >>>>>> SAMPLE USAGE and OUTPUT: >>>>>> >>>>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>>>> ./java -XshowSettings:system >>>>>> Operating System Metrics: >>>>>> Provider: cgroupv1 >>>>>> Effective CPU Count: 4 >>>>>> CPU Period: 100000 >>>>>> CPU Quota: -1 >>>>>> CPU Shares: -1 >>>>>> List of Processors, 4 total: >>>>>> 4 5 6 7 >>>>>> List of Effective Processors, 4 total: >>>>>> 4 5 6 7 >>>>>> List of Memory Nodes, 2 total: >>>>>> 0 1 >>>>>> List of Available Memory Nodes, 2 total: >>>>>> 0 1 >>>>>> CPUSet Memory Pressure Enabled: false >>>>>> Memory Limit: 256.00M >>>>>> Memory Soft Limit: Unlimited >>>>>> Memory & Swap Limit: 512.00M >>>>>> Kernel Memory Limit: Unlimited >>>>>> TCP Memory Limit: Unlimited >>>>>> Out Of Memory Killer Enabled: true >>>>>> >>>>>> TEST RESULTS: >>>>>> >>>>>> testing runtime container APIs >>>>>> Directory "JTwork" not found: creating >>>>>> Passed: runtime/containers/cgroup/PlainRead.java >>>>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>>>> Passed: runtime/containers/docker/TestCPUSets.java >>>>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>>>> Passed: runtime/containers/docker/TestMisc.java >>>>>> Test results: passed: 6 >>>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>>> >>>>>> testing jdk.internal.platform APIs >>>>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>>>> Test results: passed: 4 >>>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>>> >>>>>> testing -XshowSettings:system launcher option >>>>>> Passed: tools/launcher/Settings.java >>>>>> Test results: passed: 1 >>>>>> >>>>>> >>>>>> Bob. >>>>>> >>>>>> > From david.holmes at oracle.com Tue Jun 12 05:16:13 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 Jun 2018 15:16:13 +1000 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> Message-ID: Here is one further minor update from the CSR discussions: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v5-incr/src/java.base/share/classes/java/lang/Class.java.cdiff.html Thanks, David On 25/05/2018 3:52 PM, David Holmes wrote: > Here are the further minor updates so far in response to all the review > comments. > > Incremental corelibs webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3-incr/ > > Full corelibs webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3/ > > Change summary: > > src/java.base/share/classes/jdk/internal/reflect/Reflection.java > - remove inaccurate pseudo-assertion comment > > test/jdk/java/lang/reflect/Nestmates/SampleNest.java > - code cleanup: <> operator > > test/jdk/java/lang/reflect/Nestmates/TestReflectionAPI.java > - code cleanup: streamify duplicate removals > > test/jdk/java/lang/invoke/PrivateInterfaceCall.java > - use consistent @bug number > > Thanks, > David > > On 22/05/2018 8:15 PM, David Holmes wrote: >> Here are the updates so far in response to all the review comments. >> >> Incremental webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v2-incr/ >> >> >> Full webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v2/ >> >> Change summary: >> >> src/java.base/share/classes/java/lang/Class.java >> - getNesthost: >> ?? - change "any error" -> "any linkage error" as runtime errors will >> propagate. [This needs ratifying by EG] >> ?? - add clarification that primitive and array classes are not >> explicitly members of any nest and so form singleton nests >> ?? - add clarification that all nestmates are in the same package >> ?? - re-word @return text to exclude the "royal 'we'" >> - fix javadoc cross references >> >> --- >> >> Moved reflection API tests from >> test/hotspot/jtreg/runtime/Nestmates/reflectionAPI/ to >> test/jdk/java/lang/reflect/Nestmates/ >> >> --- >> >> java/lang/reflect/Nestmates/TestReflectionAPI.java >> >> Run tests twice to show that failure reasons remain the same. >> >> --- >> >> test/jdk/jdk/lambda/vm/InterfaceAccessFlagsTest.java >> >> Disable test via annotation rather than commenting out. >> >> --- >> >> src/java.base/share/classes/jdk/internal/reflect/Reflection.java >> >> - Fix indent for nestmate access check. >> - Remove unnecessary local variable >> >> --- >> >> src/java.base/share/classes/sun/invoke/util/VerifyAccess.java >> >> - Replace myassert with proper assert >> >> --- >> >> Thanks, >> David >> >> On 15/05/2018 10:52 AM, David Holmes wrote: >>> This review is being spread across four groups: langtools, core-libs, >>> hotspot and serviceability. This is the specific review thread for >>> core-libs - webrev: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v1/ >>> >>> See below for full details - including annotated full webrev guiding >>> the review. >>> >>> The intent is to have JEP-181 targeted and integrated by the end of >>> this month. >>> >>> Thanks, >>> David >>> ----- >>> >>> The nestmates project (JEP-181) introduces new classfile attributes >>> to identify classes and interfaces in the same nest, so that the VM >>> can perform access control based on those attributes and so allow >>> direct private access between nestmates without requiring javac to >>> generate synthetic accessor methods. These access control changes >>> also extend to core reflection and the MethodHandle.Lookup contexts. >>> >>> Direct private calls between nestmates requires a more general >>> calling context than is permitted by invokespecial, and so the JVMS >>> is updated to allow, and javac updated to use, invokevirtual and >>> invokeinterface for private class and interface method calls >>> respectively. These changed semantics also extend to MethodHandle >>> findXXX operations. >>> >>> At this time we are only concerned with static nest definitions, >>> which map to a top-level class/interface as the nest-host and all its >>> nested types as nest-members. >>> >>> Please see the JEP for further details. >>> >>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>> >>> All of the specification changes have been previously been worked out >>> by the Valhalla Project Expert Group, and the implementation reviewed >>> by the various contributors and discussed on the valhalla-dev mailing >>> list. >>> >>> Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, >>> Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, >>> Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan >>> >>> Master webrev of all changes: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>> >>> Annotated master webrev index: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>> >>> Performance: this is expected to be performance neutral in a general >>> sense. Benchmarking and performance runs are about to start. >>> >>> Testing Discussion: >>> ------------------ >>> >>> The testing for nestmates can be broken into four main groups: >>> >>> -? New tests specifically related to nestmates and currently in the >>> runtime/Nestmates directory >>> >>> - New tests to complement existing tests by adding in testcases not >>> previously expressible. >>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>> use of invokespecial for private interface methods and performing >>> receiver typechecks, so we add >>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>> invokeinterface. >>> >>> -? New JVM TI tests to verify the spec changes related to nest >>> attributes. >>> >>> -? Existing tests significantly affected by the nestmates changes, >>> primarily: >>> ??? -? runtime/SelectionResolution >>> >>> ??? In most cases the nestmate changes makes certain invocations that >>> were illegal, legal (e.g. not requiring invokespecial to invoke >>> private interface methods; allowing access to private members via >>> reflection/Methodhandles that were previously not allowed). >>> >>> - Existing tests incidentally affected by the nestmate changes >>> >>> ?? This includes tests of things utilising class >>> redefinition/retransformation to alter nested types but which >>> unintentionally alter nest relationships (which is not permitted). >>> >>> There are still a number of tests problem-listed with issues filed >>> against them to have them adapted to work with nestmates. Some of >>> these are intended to be addressed in the short-term, while some >>> (such as the runtime/SelectionResolution test changes) may not >>> eventuate. >>> >>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>> >>> There is also further test work still to be completed (the JNI and >>> JDI invocation tests): >>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>> which will continue in parallel with the main RFR. >>> >>> Pre-integration Testing: >>> ??- General: >>> ???? - Mach5: hs/jdk tier1,2 >>> ???? - Mach5: hs-nightly (tiers 1 -3) >>> ??- Targetted >>> ??? - nashorn (for asm changes) >>> ??? - hotspot: runtime/* >>> ?????????????? serviceability/* >>> ?????????????? compiler/* >>> ?????????????? vmTestbase/* >>> ??? - jdk: java/lang/invoke/* >>> ?????????? java/lang/reflect/* >>> ?????????? java/lang/instrument/* >>> ?????????? java/lang/Class/* >>> ?????????? java/lang/management/* >>> ?? - langtools: tools/javac >>> ??????????????? tools/javap >>> From srinivas.dama at oracle.com Tue Jun 12 05:21:47 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Mon, 11 Jun 2018 22:21:47 -0700 (PDT) Subject: RFR: 8196993: Resolve disabled warnings for libunpack In-Reply-To: References: Message-ID: Gentle reminder . May I have review for the below changeset. Regards, Srinivas -----Original Message----- From: Srinivas Dama Sent: Friday, June 08, 2018 6:33 PM To: Core-Libs-Dev Subject: RFR: 8196993: Resolve disabled warnings for libunpack Hi, Please review http://cr.openjdk.java.net/~sdama/8196993/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8196993 Regards, Srinivas From srinivas.dama at oracle.com Tue Jun 12 05:23:14 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Mon, 11 Jun 2018 22:23:14 -0700 (PDT) Subject: RFR: 8196988 (Resolve disabled warnings for libjimage) In-Reply-To: <7477c842-56d3-4980-b51b-66ad9d8a05c0@default> References: <7477c842-56d3-4980-b51b-66ad9d8a05c0@default> Message-ID: <6b5d7d2f-d735-4b62-b1c6-8dd09f22e542@default> Gentle reminder. May I have review for the below changeset. Regards, Srinivas -----Original Message----- From: Srinivas Dama Sent: Friday, June 08, 2018 11:47 AM To: core-libs-dev at openjdk.java.net Subject: Re: RFR: 8196988 (Resolve disabled warnings for libjimage) Hi, Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.01/ for https://bugs.openjdk.java.net/browse/JDK-8196988 Note: Modified earlier fix to handle warning messages specific to libjimage only. Regards, Srinivas ----- Original Message ----- From: srinivas.dama at oracle.com To: core-libs-dev at openjdk.java.net Sent: Monday, 4 June, 2018 11:08:08 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: RFR: 8196988 (Resolve disabled warnings for implicit-fallthrough gcc option) Hi, Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8196988 Note: gcc 7.3 compiler emits warnings for implicit fall-through switch cases,but these warnings are disabled earlier.I have enabled these warnings and fixed in sources. Regards, Srinivas From mandy.chung at oracle.com Tue Jun 12 05:31:05 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Jun 2018 22:31:05 -0700 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> Message-ID: On 6/11/18 10:12 PM, David Holmes wrote: >>>>> >>>>> For the Java code ... methods that return arrays should return >>>>> zero-length arrays when something is not available rather than null. >>>> All methods do return zero length arrays except I missed the >>>> getPerCpuUsage.? I?ll fix that one and correct the javadoc. >>> >>> There are a few more too: >>> >> >> Those are covered by the function that converts the string range. > > ??? I have no idea what you mean. I think the methods returning an array calls Subsystem::StringRangeToIntArray which returns an empty array. 171 public static int[] StringRangeToIntArray(String range) { 172 int[] ints = new int[0]; 173 174 if (range == null) return ints; Mandy From mandy.chung at oracle.com Tue Jun 12 05:31:59 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Jun 2018 22:31:59 -0700 Subject: RFR: 8196993: Resolve disabled warnings for libunpack In-Reply-To: References: Message-ID: <3188419b-a919-eea6-eb54-186d660db025@oracle.com> Looks fine. Mandy On 6/11/18 10:21 PM, Srinivas Dama wrote: > Gentle reminder . > > May I have review for the below changeset. > > Regards, > Srinivas > -----Original Message----- > From: Srinivas Dama > Sent: Friday, June 08, 2018 6:33 PM > To: Core-Libs-Dev > Subject: RFR: 8196993: Resolve disabled warnings for libunpack > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196993/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8196993 > > Regards, > Srinivas > From mandy.chung at oracle.com Tue Jun 12 05:38:10 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Jun 2018 22:38:10 -0700 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> Message-ID: <82fdd320-f5fb-2c34-5bb8-b72f6718fef6@oracle.com> On 6/11/18 10:16 PM, David Holmes wrote: > Here is one further minor update from the CSR discussions: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v5-incr/src/java.base/share/classes/java/lang/Class.java.cdiff.html "This implementation" is fine, as used in many @implNote. Any reason why it has to be changed to "reference implementation"? Mandy From david.holmes at oracle.com Tue Jun 12 05:43:13 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 12 Jun 2018 15:43:13 +1000 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> Message-ID: <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> On 12/06/2018 3:31 PM, mandy chung wrote: > On 6/11/18 10:12 PM, David Holmes wrote: >>>>>> >>>>>> For the Java code ... methods that return arrays should return >>>>>> zero-length arrays when something is not available rather than null. >>>>> All methods do return zero length arrays except I missed the >>>>> getPerCpuUsage.? I?ll fix that one and correct the javadoc. >>>> >>>> There are a few more too: >>>> >>> >>> Those are covered by the function that converts the string range. >> >> ??? I have no idea what you mean. > > > I think the methods returning an array calls > Subsystem::StringRangeToIntArray which returns an empty array. > > ?171???? public static int[] StringRangeToIntArray(String range) { > ?172???????? int[] ints = new int[0]; > ?173 > ?174???????? if (range == null) return ints; I'm commenting on the specification of the Metrics interface: http://cr.openjdk.java.net/~bobv/8203357/webrev.01/src/java.base/share/classes/jdk/internal/platform/Metrics.java.html not any implementation. Cheers, David > > Mandy From mandy.chung at oracle.com Tue Jun 12 06:00:57 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 11 Jun 2018 23:00:57 -0700 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> Message-ID: <413aa29c-1a7c-8cd2-ca27-201dc6b53432@oracle.com> On 6/11/18 10:43 PM, David Holmes wrote: > On 12/06/2018 3:31 PM, mandy chung wrote: >> On 6/11/18 10:12 PM, David Holmes wrote: >>>>>>> >>>>>>> For the Java code ... methods that return arrays should return >>>>>>> zero-length arrays when something is not available rather than null. >>>>>> All methods do return zero length arrays except I missed the >>>>>> getPerCpuUsage.? I?ll fix that one and correct the javadoc. >>>>> >>>>> There are a few more too: >>>>> >>>> >>>> Those are covered by the function that converts the string range. >>> >>> ??? I have no idea what you mean. >> >> >> I think the methods returning an array calls >> Subsystem::StringRangeToIntArray which returns an empty array. >> >> ??171???? public static int[] StringRangeToIntArray(String range) { >> ??172???????? int[] ints = new int[0]; >> ??173 >> ??174???????? if (range == null) return ints; > > I'm commenting on the specification of the Metrics interface: > > http://cr.openjdk.java.net/~bobv/8203357/webrev.01/src/java.base/share/classes/jdk/internal/platform/Metrics.java.html > not any implementation. The implementation returns empty array, which is good. Yes the javadoc should be updated. Mandy From bhamaram at in.ibm.com Tue Jun 12 09:00:55 2018 From: bhamaram at in.ibm.com (Bhaktavatsal R Maram) Date: Tue, 12 Jun 2018 09:00:55 +0000 Subject: [AIX] Add charset IBM-964 (default charset for zh_TW.IBM-eucTW) to stdcs In-Reply-To: <5AF5C0FC.5020504@oracle.com> References: <5AF5C0FC.5020504@oracle.com>, <9cecb38d51627bfb0e11afceb0e0ecb4@linux.vnet.ibm.com>, Message-ID: Thank you for review. This is not new charset. This codepage already exist in openJDK and current fix is to move it to standard charset list (only for AIX) as it is default charset for locale zh_TW.IBM-eucTW. This enabling jdk11 on locale zh_TW.IBM-eucTW. May I request someone to create JIRA bug for this issue and sponsor the fix? - Bhaktavatsal Reddy. -----"core-libs-dev" wrote: ----- To: core-libs-dev at openjdk.java.net From: Xueming Shen Sent by: "core-libs-dev" Date: 05/11/2018 09:43PM Subject: Re: [AIX] Add charset IBM-964 (default charset for zh_TW.IBM-eucTW) to stdcs SimpleEUCEncoder is something leftover when I reimplemented the charsets to be map-based-built-during-build-time. I would recommend to leave them as Bhaktavatsal suggested for now. Ideally any new charsets added to the platform needs to be based on the new model (open/make/data/charetmapping), instead of hard-coding the map into the source code. Thanks! Sherman On 5/10/18, 8:20 PM, Bhaktavatsal R Maram wrote: > Hi Ichiroh, > > Yes, moving SimpleEUCEncoder.java to sun.nio.cs package would leave build tools untouched. But, I think that for platforms other than AIX, SimpleEUCEncoder.java in java.base module would not add any value. > > - Bhaktavatsal > > -----Ichiroh Takiguchi wrote: ----- > To: Bhaktavatsal R Maram/India/IBM at IBMIN > From: Ichiroh Takiguchi > Date: 05/11/2018 06:31AM > Cc: Java Core Libs > Subject: Re: [AIX] Add charset IBM-964 (default charset for zh_TW.IBM-eucTW) to stdcs > > Hello Bhaktavatsal. > > I think we should move > from > src/jdk.charsets/share/classes/sun/nio/cs/ext/SimpleEUCEncoder.java > to > src/java.base/share/classes/sun/nio/cs/SimpleEUCEncoder.java > instead of moving to > > src/jdk.charsets/share/classes/sun/nio/cs/ext/SimpleEUCEncoder.java.template > > Then we don't need to touch following files > make/jdk/src/classes/build/tools/charsetmapping/Main.java > make/jdk/src/classes/build/tools/charsetmapping/SRC.java > > I think SimpleEUCEncoder.java is not large file, > so it can be moved to java.base module. > > What do you think ? > > On 2018-05-10 22:35, Bhaktavatsal R Maram wrote: >> Hi All, >> >> In bug 8201540 (Extend the set of supported charsets in java.base on >> AIX)[1] we moved default charsets that are available in OpenJDK to >> java.base module except to IBM964. Charset IBM964 is bit tricky. Its >> Encoder is extended from sun.nio.cs.ext.SimpleEUCEncoder. So, >> SimpleEUCEncoder also should be moved along with IBM964 to sun.nio.cs. >> However, there is another charset IBM33722 which should always in >> sun.nio.cs.ext whose encoder also extends SimpleEUCEncoder. >> >> Hence, the challenge is that sun.nio.cs.ext.IBM33722 should import >> SimpleEUCEncoder from either sun.nio.cs or sun.nio.cs.ext based on the >> location of IBM964. >> >> On AIX, following should be the package locations >> sun.nio.cs.IBM964 >> sun.nio.cs.SimpleEUCEncoder >> sun.nio.cs.ext.IBM33722 imports sun.nio.cs.SimpleEUCEncoder >> >> On Linux (and other platforms), following should be the package >> locations >> sun.nio.cs.ext.IBM964 >> sun.nio.cs.ext.SimpleEUCEncoder >> sun.nio.cs.ext.IBM33722 imports sun.nio.cs.ext.SimpleEUCEncoder >> >> To achieve this, I've changed all source files involved to templates. >> And, I also modified build tools to check if IBM964 is moved to stdcs >> or not and handled import statement of SimpleEUCEncoder in >> sun.nio.cs.ext.IBM33722 accordingly. >> >> Patch can be found here [2]. Please review. After jira bug is opened, >> I'll host it again with bug ID. >> >> With this change, java -version is successful on locale >> zh_TW.IBM-eucTW. >> >> -bash-4.4\$ LANG=zh_TW.IBM-eucTW >> ~/openjdk/jdk11/build/aix-ppc64-normal-server-release/jdk/bin/java >> -version >> openjdk version "11-internal" 2018-09-25 >> OpenJDK Runtime Environment (build 11-internal+0-adhoc.bhamaram.jdk11) >> OpenJDK 64-Bit Server VM (build 11-internal+0-adhoc.bhamaram.jdk11, >> mixed mode) >> >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8201540 >> [2] http://cr.openjdk.java.net/~aleonard/AIXIBM964/webrev.01/ >> >> Thanks, >> Bhaktavatsal Reddy From vivek.theeyarath at oracle.com Tue Jun 12 09:34:22 2018 From: vivek.theeyarath at oracle.com (Vivek Theeyarath) Date: Tue, 12 Jun 2018 02:34:22 -0700 (PDT) Subject: RFR: 8202216: (bf) Add Buffer mismatch() Message-ID: <38b94492-28bf-4034-8a91-ff44124e5ea7@default> Hi All, Please review fix for https://bugs.openjdk.java.net/browse/JDK-8202216 Webrev: http://cr.openjdk.java.net/~vtheeyarath/8202216/webrev.00/ CSR : https://bugs.openjdk.java.net/browse/JDK-8204852 Regards Vivek From peter.levart at gmail.com Tue Jun 12 10:20:52 2018 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 12 Jun 2018 12:20:52 +0200 Subject: RFR: 8202216: (bf) Add Buffer mismatch() In-Reply-To: <38b94492-28bf-4034-8a91-ff44124e5ea7@default> References: <38b94492-28bf-4034-8a91-ff44124e5ea7@default> Message-ID: <18401a8c-8315-97f3-6a22-619976c534ce@gmail.com> Hi, On 06/12/2018 11:34 AM, Vivek Theeyarath wrote: > Hi All, > > Please review fix for https://bugs.openjdk.java.net/browse/JDK-8202216 > > > > Webrev: http://cr.openjdk.java.net/~vtheeyarath/8202216/webrev.00/ > > CSR : https://bugs.openjdk.java.net/browse/JDK-8204852 > > > > Regards > > Vivek > > This looks good as is, but would it make sense for this new method to be defined as abstract method on Buffer? Like for example: public abstract class Buffer { ??? public abstract int mismatch(B that); ... public abstract class ByteBuffer ??? extends Buffer ??? implements Comparable { ??? @Override ??? public int mismatch(ByteBuffer that) { ... public abstract class CharBuffer ??? extends Buffer ??? implements Comparable { ??? @Override ??? public int mismatch(CharBuffer that) { ... Regards, Peter From vivek.theeyarath at oracle.com Tue Jun 12 10:45:05 2018 From: vivek.theeyarath at oracle.com (Vivek Theeyarath) Date: Tue, 12 Jun 2018 03:45:05 -0700 (PDT) Subject: RFR: 8204342: CLONE - methods in java.time's TCKZoneRules OpenJDK test miss @Test annotation Message-ID: <4327b7fd-46f3-40d6-8930-03aee44f3707@default> Hi All, Please review fix for https://bugs.openjdk.java.net/browse/JDK-8204342 . The jtreg test runs fine with the changes. http://cr.openjdk.java.net/~vtheeyarath/8204342/webrev.00/ Regards Vivek From bob.vandette at oracle.com Tue Jun 12 10:56:33 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 12 Jun 2018 06:56:33 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> Message-ID: > On Jun 12, 2018, at 1:12 AM, David Holmes wrote: > > On 12/06/2018 9:30 AM, Bob Vandette wrote: >>> On Jun 11, 2018, at 5:21 PM, David Holmes wrote: >>> >>> On 12/06/2018 12:12 AM, Bob Vandette wrote: >>>>> On Jun 11, 2018, at 4:32 AM, David Holmes > wrote: >>>>> >>>>> Sorry Bob I haven't had a chance to look at this detail. >>>>> >>>>> For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. >>>> All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. >>> >>> There are a few more too: >>> >> Those are covered by the function that converts the string range. > > ??? I have no idea what you mean. > > Java API design style is to return zero-length arrays rather than null. [Ref: Effective Java First Edition, Item 27]. Look at line 174 in this file. http://cr.openjdk.java.net/~bobv/8203357/webrev.01/src/java.base/linux/classes/jdk/internal/platform/cgroupv1/SubSystem.java.html Bob > > Cheers, > David > ----- > >>> 231 * @return An array of available CPUs or null if metric is not available. >>> 232 * >>> 233 */ >>> 234 public int[] getCpuSetCpus(); >>> >>> 242 * @return An array of available and online CPUs or null if the metric >>> 243 * is not available. >>> 244 * >>> 245 */ >>> 246 public int[] getEffectiveCpuSetCpus(); >>> >>> 256 * @return An array of available memory nodes or null if metric is not available. >>> 257 * >>> 258 */ >>> 259 public int[] getCpuSetMems(); >>> >>> 267 * @return An array of available and online nodes or null if the metric >>> 268 * is not available. >>> 269 * >>> 270 */ >>> 271 public int[] getEffectiveCpuSetMems(); >>>>> >>>>> For getCpuPeriod() the term "operating system time slice" can be misconstrued as being related to the scheduler timeslice that may, or may not, exist, depending on the scheduler and scheduling policy etc. This "timeslice" is something specific to cgroups - no? >>>> The comments reads: >>>> * Returns the length of the operating system time slice, in >>>> * milliseconds, for processes within the Isolation Group. >>>> The comment does infer that it?s process and cgroup (Isolation group) specific and not the generic os timeslice. >>>> Isn?t this sufficient? >>> >>> The phrase "operating system" makes this sound like some kind of global timeslice notion - which it isn't. And I don't like to think of cpu periods/shares/quotas in terms of "time slice" anyway. I don't see the Docker or Cgroup documentation using "time slice" either. It suffices IMHO to just say for period: >>> >>> * Returns the length of the scheduling period, in >>> * milliseconds, for processes within the Isolation Group. >>> >>> then for quota: >>> >>> * Returns the total available run-time allowed, in milliseconds, >>> * during each scheduling period for all tasks in the Isolation Group. >>> >> Ok. I?ll update the docs. >> Bob >>> Thanks, >>> David >>> >>>> Thanks, >>>> Bob. >>>>> >>>>> David >>>>> >>>>>>> On 8/06/2018 3:43 AM, Bob Vandette wrote: >>>>>>> Can I get one more reviewer for this RFE so I can integrate it? >>>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>>> Mandy Chung has reviewed this change. >>>>>> I?ve run Mach5 hotspot and core lib tests. >>>>>> I?ve reviewed the tests which were written by Harsha Wardhana >>>>>> I filed a CSR for the command line change and it?s now approved and closed. >>>>>> Thanks, >>>>>> Bob. >>>>>>> On May 30, 2018, at 3:45 PM, Bob Vandette > wrote: >>>>>>> >>>>>>> Please review the following RFE which adds an internal API, along with jtreg tests that provide >>>>>>> access to Docker container configuration data and metrics. In addition to the API which we hope to >>>>>>> take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional >>>>>>> option to -XshowSettings:system than dumps out the container or host cgroup confguration >>>>>>> information. See the sample output below: >>>>>>> >>>>>>> RFE: Container Metrics >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8203357 >>>>>>> >>>>>>> WEBREV: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.01 >>>>>>> >>>>>>> >>>>>>> This commit will also include a fix for the following bug. >>>>>>> >>>>>>> BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8203691 >>>>>>> >>>>>>> WEBREV: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html >>>>>>> >>>>>>> SAMPLE USAGE and OUTPUT: >>>>>>> >>>>>>> docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash >>>>>>> ./java -XshowSettings:system >>>>>>> Operating System Metrics: >>>>>>> Provider: cgroupv1 >>>>>>> Effective CPU Count: 4 >>>>>>> CPU Period: 100000 >>>>>>> CPU Quota: -1 >>>>>>> CPU Shares: -1 >>>>>>> List of Processors, 4 total: >>>>>>> 4 5 6 7 >>>>>>> List of Effective Processors, 4 total: >>>>>>> 4 5 6 7 >>>>>>> List of Memory Nodes, 2 total: >>>>>>> 0 1 >>>>>>> List of Available Memory Nodes, 2 total: >>>>>>> 0 1 >>>>>>> CPUSet Memory Pressure Enabled: false >>>>>>> Memory Limit: 256.00M >>>>>>> Memory Soft Limit: Unlimited >>>>>>> Memory & Swap Limit: 512.00M >>>>>>> Kernel Memory Limit: Unlimited >>>>>>> TCP Memory Limit: Unlimited >>>>>>> Out Of Memory Killer Enabled: true >>>>>>> >>>>>>> TEST RESULTS: >>>>>>> >>>>>>> testing runtime container APIs >>>>>>> Directory "JTwork" not found: creating >>>>>>> Passed: runtime/containers/cgroup/PlainRead.java >>>>>>> Passed: runtime/containers/docker/DockerBasicTest.java >>>>>>> Passed: runtime/containers/docker/TestCPUAwareness.java >>>>>>> Passed: runtime/containers/docker/TestCPUSets.java >>>>>>> Passed: runtime/containers/docker/TestMemoryAwareness.java >>>>>>> Passed: runtime/containers/docker/TestMisc.java >>>>>>> Test results: passed: 6 >>>>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>>>> >>>>>>> testing jdk.internal.platform APIs >>>>>>> Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java >>>>>>> Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java >>>>>>> Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java >>>>>>> Passed: jdk/internal/platform/docker/TestSystemMetrics.java >>>>>>> Test results: passed: 4 >>>>>>> Results written to /export/users/bobv/jdk11/build/jtreg/JTwork >>>>>>> >>>>>>> testing -XshowSettings:system launcher option >>>>>>> Passed: tools/launcher/Settings.java >>>>>>> Test results: passed: 1 >>>>>>> >>>>>>> >>>>>>> Bob. >>>>>>> >>>>>>> From bob.vandette at oracle.com Tue Jun 12 10:59:23 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 12 Jun 2018 06:59:23 -0400 Subject: Ping!! Re: RFR: 8203357 Container Metrics In-Reply-To: <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> References: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> <1c57d8ee-1178-2e9a-4b7d-aafbe885473c@oracle.com> <469a46b5-aa9c-c6fd-b270-6a4230b4e08a@oracle.com> <2b705a94-afe0-c9bb-4377-accabf73696e@oracle.com> <73dd06ec-544f-3cc9-00d5-c5c8c8ffdacc@oracle.com> Message-ID: > On Jun 12, 2018, at 1:43 AM, David Holmes wrote: > >> On 12/06/2018 3:31 PM, mandy chung wrote: >> On 6/11/18 10:12 PM, David Holmes wrote: >>>>>>> >>>>>>> For the Java code ... methods that return arrays should return zero-length arrays when something is not available rather than null. >>>>>> All methods do return zero length arrays except I missed the getPerCpuUsage. I?ll fix that one and correct the javadoc. >>>>> >>>>> There are a few more too: >>>>> >>>> >>>> Those are covered by the function that converts the string range. >>> >>> ??? I have no idea what you mean. >> I think the methods returning an array calls Subsystem::StringRangeToIntArray which returns an empty array. >> 171 public static int[] StringRangeToIntArray(String range) { >> 172 int[] ints = new int[0]; >> 173 >> 174 if (range == null) return ints; > > I'm commenting on the specification of the Metrics interface: > > http://cr.openjdk.java.net/~bobv/8203357/webrev.01/src/java.base/share/classes/jdk/internal/platform/Metrics.java.html > > not any implementation. Oh. I previously mentioned that I needed to correct the javadoc comments. I had corrected the implementation but hadn?t fixed the spec. Bob. > > Cheers, > David > >> Mandy From srinivas.dama at oracle.com Tue Jun 12 12:46:51 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Tue, 12 Jun 2018 05:46:51 -0700 (PDT) Subject: RFR: 8204861 : fix for 8196993 has broken the build on linux Message-ID: <71aad77c-0a6a-43f8-95af-5d777d451197@default> Hi, My earlier fix for https://bugs.openjdk.java.net/browse/JDK-8196993 has broken the build on linux. So I am reverting my fix for now. Bug : https://bugs.openjdk.java.net/browse/JDK-8204861 webrev: http://cr.openjdk.java.net/~sdama/8204861/webrev.00/ Regards, Srinivas From stefan.karlsson at oracle.com Tue Jun 12 12:52:21 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 12 Jun 2018 14:52:21 +0200 Subject: RFR: 8204861 : fix for 8196993 has broken the build on linux In-Reply-To: <71aad77c-0a6a-43f8-95af-5d777d451197@default> References: <71aad77c-0a6a-43f8-95af-5d777d451197@default> Message-ID: Looks good to me. Thanks for fixing this. StefanK On 2018-06-12 14:46, Srinivas Dama wrote: > Hi, > > My earlier fix for https://bugs.openjdk.java.net/browse/JDK-8196993 has broken the build on linux. > So I am reverting my fix for now. > > Bug : https://bugs.openjdk.java.net/browse/JDK-8204861 > webrev: http://cr.openjdk.java.net/~sdama/8204861/webrev.00/ > > Regards, > Srinivas > From shade at redhat.com Tue Jun 12 12:56:34 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 12 Jun 2018 14:56:34 +0200 Subject: RFR: 8204861 : fix for 8196993 has broken the build on linux In-Reply-To: <71aad77c-0a6a-43f8-95af-5d777d451197@default> References: <71aad77c-0a6a-43f8-95af-5d777d451197@default> Message-ID: On 06/12/2018 02:46 PM, Srinivas Dama wrote: > Hi, > > My earlier fix for https://bugs.openjdk.java.net/browse/JDK-8196993 has broken the build on linux. > So I am reverting my fix for now. > > Bug : https://bugs.openjdk.java.net/browse/JDK-8204861 > webrev: http://cr.openjdk.java.net/~sdama/8204861/webrev.00/ Please do it. Reversal looks good. Thanks, -Aleksey From roger.riggs at oracle.com Tue Jun 12 13:24:05 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Tue, 12 Jun 2018 09:24:05 -0400 Subject: RFR: 8204342: CLONE - methods in java.time's TCKZoneRules OpenJDK test miss @Test annotation In-Reply-To: <4327b7fd-46f3-40d6-8930-03aee44f3707@default> References: <4327b7fd-46f3-40d6-8930-03aee44f3707@default> Message-ID: Hi Vivek, Look fine, Roger On 6/12/18 6:45 AM, Vivek Theeyarath wrote: > Hi All, > > Please review fix for https://bugs.openjdk.java.net/browse/JDK-8204342 . The jtreg test runs fine with the changes. > > > > http://cr.openjdk.java.net/~vtheeyarath/8204342/webrev.00/ > > > > Regards > > Vivek From Roger.Riggs at Oracle.com Tue Jun 12 14:17:17 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 12 Jun 2018 10:17:17 -0400 Subject: RFR 8197930: JNI exception pending in initializeEncoding of jni_util.c Message-ID: <5acb2cd8-e35b-f7ff-a020-508958187ada@Oracle.com> Please review a cleanup to not ignore errors from GetMethodID and GetFieldID while initializing the system encoding. Webrev: ?? http://cr.openjdk.java.net/~rriggs/webrev-jnicheck-8197930/ issue: ?? https://bugs.openjdk.java.net/browse/JDK-8197930 Thanks, Roger From naoto.sato at oracle.com Tue Jun 12 15:06:52 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Tue, 12 Jun 2018 08:06:52 -0700 Subject: RFR: 8204342: CLONE - methods in java.time's TCKZoneRules OpenJDK test miss @Test annotation In-Reply-To: <4327b7fd-46f3-40d6-8930-03aee44f3707@default> References: <4327b7fd-46f3-40d6-8930-03aee44f3707@default> Message-ID: <9f7af9b8-93d1-9144-fb80-e7e20da05320@oracle.com> +1 Naoto On 6/12/18 3:45 AM, Vivek Theeyarath wrote: > Hi All, > > Please review fix for https://bugs.openjdk.java.net/browse/JDK-8204342 . The jtreg test runs fine with the changes. > > > > http://cr.openjdk.java.net/~vtheeyarath/8204342/webrev.00/ > > > > Regards > > Vivek > From paul.sandoz at oracle.com Tue Jun 12 15:17:32 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 12 Jun 2018 08:17:32 -0700 Subject: BiCollector In-Reply-To: References: Message-ID: <055EBC54-C8D2-40A5-98B7-B9341F7403B9@oracle.com> I don?t have anything publicly available. I?ll see if i can dig something out and send it to you directly. Paul. > On Jun 11, 2018, at 11:28 AM, Peter Levart wrote: > > Hi Paul, > > Can you point me to some BiStream code (if it is available publicly)? > > Thanks, Peter > > On 06/11/18 19:10, Paul Sandoz wrote: >> Hi Peter, >> >> I like it and can see it being useful, thanks for sharing. >> >> I am hesitating a little about it being in the JDK because there is the larger abstraction of a BiStream, where a similar form of collection would naturally fit (but perhaps without the intersection constraints for the characteristics?). We experimented a few times with BiStream and got quite far but decided pull back due to the lack of value types and specialized generics. So i dunno how this might turn out in the future and if your BiCollector fits nicely into such a future model. >> >> What are you thoughts on this? >> >> FWIW i would call it a ?splitting? or ?bisecting" collector e.g. ?s.collect(bisecting(?))? >> >> Paul. From paul.sandoz at oracle.com Tue Jun 12 15:39:33 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 12 Jun 2018 08:39:33 -0700 Subject: RFR: 8202216: (bf) Add Buffer mismatch() In-Reply-To: <18401a8c-8315-97f3-6a22-619976c534ce@gmail.com> References: <38b94492-28bf-4034-8a91-ff44124e5ea7@default> <18401a8c-8315-97f3-6a22-619976c534ce@gmail.com> Message-ID: <29E31126-00CA-431C-92B5-B089971E9998@oracle.com> Hi Peter, Buffer is not currently a generic class and making it so likely would cause some source compatibility issues (e.g. lots of raw type warnings/errors). Paul. > On Jun 12, 2018, at 3:20 AM, Peter Levart wrote: > > Hi, > > On 06/12/2018 11:34 AM, Vivek Theeyarath wrote: >> Hi All, >> >> Please review fix for https://bugs.openjdk.java.net/browse/JDK-8202216 >> >> >> Webrev: http://cr.openjdk.java.net/~vtheeyarath/8202216/webrev.00/ >> >> CSR : https://bugs.openjdk.java.net/browse/JDK-8204852 >> >> >> Regards >> >> Vivek >> >> > > This looks good as is, but would it make sense for this new method to be defined as abstract method on Buffer? Like for example: > > public abstract class Buffer { > public abstract int mismatch(B that); > ... > > > public abstract class ByteBuffer > extends Buffer > implements Comparable > { > @Override > public int mismatch(ByteBuffer that) { > ... > > > public abstract class CharBuffer > extends Buffer > implements Comparable > { > @Override > public int mismatch(CharBuffer that) { > ... > > > > Regards, Peter > From thomas.stuefe at gmail.com Tue Jun 12 15:47:13 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 12 Jun 2018 17:47:13 +0200 Subject: RFR 8197930: JNI exception pending in initializeEncoding of jni_util.c In-Reply-To: <5acb2cd8-e35b-f7ff-a020-508958187ada@Oracle.com> References: <5acb2cd8-e35b-f7ff-a020-508958187ada@Oracle.com> Message-ID: Looks good. Thanks, Thomas On Tue, Jun 12, 2018 at 4:17 PM, Roger Riggs wrote: > Please review a cleanup to not ignore errors from GetMethodID and GetFieldID > while initializing the system encoding. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-jnicheck-8197930/ > issue: > https://bugs.openjdk.java.net/browse/JDK-8197930 > > Thanks, Roger > From paul.sandoz at oracle.com Tue Jun 12 16:11:21 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 12 Jun 2018 09:11:21 -0700 Subject: RFR: 8202216: (bf) Add Buffer mismatch() In-Reply-To: <38b94492-28bf-4034-8a91-ff44124e5ea7@default> References: <38b94492-28bf-4034-8a91-ff44124e5ea7@default> Message-ID: <5DB14AEA-D496-4BC6-9327-B03E7FD1F163@oracle.com> +1 Paul. > On Jun 12, 2018, at 2:34 AM, Vivek Theeyarath wrote: > > Hi All, > > Please review fix for https://bugs.openjdk.java.net/browse/JDK-8202216 > > > > Webrev: http://cr.openjdk.java.net/~vtheeyarath/8202216/webrev.00/ > > CSR : https://bugs.openjdk.java.net/browse/JDK-8204852 > > > > Regards > > Vivek > > From mandy.chung at oracle.com Tue Jun 12 16:30:59 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 12 Jun 2018 09:30:59 -0700 Subject: RFR 8197930: JNI exception pending in initializeEncoding of jni_util.c In-Reply-To: <5acb2cd8-e35b-f7ff-a020-508958187ada@Oracle.com> References: <5acb2cd8-e35b-f7ff-a020-508958187ada@Oracle.com> Message-ID: <4bf29fcb-d39c-3dc6-a762-62bb11736447@oracle.com> Looks good. Mandy On 6/12/18 7:17 AM, Roger Riggs wrote: > Please review a cleanup to not ignore errors from GetMethodID and > GetFieldID while initializing the system encoding. > > Webrev: > ?? http://cr.openjdk.java.net/~rriggs/webrev-jnicheck-8197930/ > issue: > ?? https://bugs.openjdk.java.net/browse/JDK-8197930 > > Thanks, Roger > From paul.sandoz at oracle.com Tue Jun 12 16:38:51 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 12 Jun 2018 09:38:51 -0700 Subject: RFR: 8196988 (Resolve disabled warnings for libjimage) In-Reply-To: <7477c842-56d3-4980-b51b-66ad9d8a05c0@default> References: <7477c842-56d3-4980-b51b-66ad9d8a05c0@default> Message-ID: <3AB1E03A-F47E-48E6-8C9B-DBC698ABA2A1@oracle.com> Looks ok. Including Jim in case he is not listening in? I am guessing the comment selectively disables the warning in this code (rather than using a diagnostic pragma). Thanks, Paul. > On Jun 7, 2018, at 11:17 PM, Srinivas Dama wrote: > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.01/ > for https://bugs.openjdk.java.net/browse/JDK-8196988 > > > Note: Modified earlier fix to handle warning messages specific to > libjimage only. > > Regards, > Srinivas > > ----- Original Message ----- > From: srinivas.dama at oracle.com > To: core-libs-dev at openjdk.java.net > Sent: Monday, 4 June, 2018 11:08:08 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi > Subject: RFR: 8196988 (Resolve disabled warnings for implicit-fallthrough gcc option) > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8196988 > > Note: gcc 7.3 compiler emits warnings for implicit fall-through switch cases,but > these warnings are disabled earlier.I have enabled these warnings and fixed in sources. > > Regards, > Srinivas From mandy.chung at oracle.com Tue Jun 12 16:44:21 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 12 Jun 2018 09:44:21 -0700 Subject: RFR: 8196988 (Resolve disabled warnings for libjimage) In-Reply-To: <6b5d7d2f-d735-4b62-b1c6-8dd09f22e542@default> References: <7477c842-56d3-4980-b51b-66ad9d8a05c0@default> <6b5d7d2f-d735-4b62-b1c6-8dd09f22e542@default> Message-ID: <818bdf09-1c95-3672-3381-b47fe5ea44fc@oracle.com> Have you verified the build on all platforms (release and debug)? Mandy On 6/11/18 10:23 PM, Srinivas Dama wrote: > Gentle reminder. > > May I have review for the below changeset. > > Regards, > Srinivas > -----Original Message----- > From: Srinivas Dama > Sent: Friday, June 08, 2018 11:47 AM > To: core-libs-dev at openjdk.java.net > Subject: Re: RFR: 8196988 (Resolve disabled warnings for libjimage) > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.01/ > for https://bugs.openjdk.java.net/browse/JDK-8196988 > > > Note: Modified earlier fix to handle warning messages specific to > libjimage only. > > Regards, > Srinivas > > ----- Original Message ----- > From: srinivas.dama at oracle.com > To: core-libs-dev at openjdk.java.net > Sent: Monday, 4 June, 2018 11:08:08 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi > Subject: RFR: 8196988 (Resolve disabled warnings for implicit-fallthrough gcc option) > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8196988 > > Note: gcc 7.3 compiler emits warnings for implicit fall-through switch cases,but > these warnings are disabled earlier.I have enabled these warnings and fixed in sources. > > Regards, > Srinivas > From huizhe.wang at oracle.com Tue Jun 12 16:52:09 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 12 Jun 2018 09:52:09 -0700 Subject: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> Message-ID: <5B1FFA39.9000704@oracle.com> Hi all, It's been a while since the 1st round of reviews. Thank you all for the valuable comments and suggestions! Note that Roger's last response went to nio-dev, but not core-libs-dev, I've therefore attached it below. CSR [2]: the CSR is now approved. Note the write method has been changed to writeString. Impl [3]: for performance reason and the different requirement from byte-string conversion for malformed/unmappable character handing, the implementation uses a specific method separate from the existing ones. Both Sherman and Alan preferred specific method than adding parameters to the APIs that might be error prone. Thanks Sherman for the code! JMH benchmark tests showed the new read method performs better than an operation that reads the bytes and convert to string. The gains start to be significant even at quite reasonable sizes (< 10K). [1] JBS: https://bugs.openjdk.java.net/browse/JDK-8201276 [2] CSR: https://bugs.openjdk.java.net/browse/JDK-8202055 [3] webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8201276/webrev/ Please review. Thanks, Joe On 5/15/18, 7:51 AM, Roger Riggs wrote: > Hi Joe, et. al. > > My \$.02 on line separators: > > - We should avoid clever tricks trying to solve problems that are > infrequent such as cross OS file line operations. > Most files will be read/written on a single system so the line > separators will be as expected. > > - Have/use APIs that split lines consistently accepting both line > separators so developer code can be agnostic to line separators. > aka BufferedReader.readLine for developers that are processing the > contents *as lines*. > Those other methods already exist; if there are any gaps in line > oriented processing that's a separate task. > > - These new file methods are defined to handle Charset > encoding/decoding and buffering. > Since there are other methods to deal with files as lines these > methods should not look for or break to lines. > > - Performance: adding code to look for line characters will slow it > down and in most cases would have no impact > since the line endings will match the system. > > - The strings passed to writeString (CharSequence) should have been > constructed using the proper line separators. > There are any number of methods that insert the os specific line > terminator and its easy to write File.separator as needed. > > As for the method naming: > > I'd prefer "writeString" as a familiar term that isn't stretched too > far by making the argument be a CharSequence. > its fine to call a CharSequence a string (with a lower case s). > > \$.02, Roger > > > On 4/27/18 6:33 PM, Joe Wang wrote: >> >> >> On 4/27/2018 12:50 PM, forax at univ-mlv.fr wrote: >>> ----- Mail original ----- >>>> De: "Joe Wang" >>>> ?: "Remi Forax" , "Alan Bateman" >>>> >>>> Cc: "nio-dev" , "core-libs-dev" >>>> >>>> Envoy?: Vendredi 27 Avril 2018 21:21:01 >>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>> reading/writing a string from/to a file >>>> Hi R?mi, Alan, >>> Hi Joe, >>> >>>> I'm not sure we'd want to replace system dependent line separators >>>> with >>>> '\n'. There are cases as you described where the replacement makes >>>> sense. However, there are other cases too. For example, the >>>> purpose is >>>> to read, edit a text file and then write it back. If this is on >>>> Windows, >>>> and if line separators are replaced with '\n', that would cause the >>>> resulting file not formatted properly in for example Notepad. There >>>> are >>>> cases where users write text files on Windows using Java, and only >>>> found >>>> the lines were not separated in Notepad. >>> I agree that why the counterpart of readString() that write the >>> string should inserts the platform's line separator. >>> BTW, i just found that those methods are not named writeString, or >>> writeCharSequence, i think they should. >> >> While readString() does not modify the original content (e.g. by >> replacing the platform's line separator with '\n'), write(String) >> won't either, by adding extra characters such as the line separator. >> >> I would think interfaces shall read along with the parameters. >> readString(Path) == read as a String from the specified Path (one >> could argue for readToString, readAsString, but we generally omit the >> preps) >> write(Path, CharSequence) == write the CharSequence to the file, >> since CharSequence is already in the method signature as a parameter, >> we probably don't want to add that to the name, otherwise it would >> read like repeating the word CharSequence. >> >> It is in a similar situation as write(Path, Iterable) where it was >> defined as writeLines(Path, Iterable). >> >>> >>>> Files.write(Path, Iterable) is also specified to terminate each line >>>> with the platform's line separator. If readString does the >>>> replacement, >>>> it would be inconsistent. >>> >>> Anyway, if we look for consistency the methods writeCharSequence >>> should transform the \n in the CharSequence to the platform's line >>> separator. >>> >>> Files.write(Path, Iterable) is is not a counterpart of readString(), >>> it's consistent with Files.lines() or Files.readLines() (or >>> BufferedReader.readLine()) that all suppress the line separators. >>> Anyway, Files.write(path, readString(path)::line) will be consistent >>> if you replace the line separators or not because String.line() >>> suppresses the line separators. >> >> readString pairs with write(String), therefore it's more like >> Files.write(path, readString(path)) than readString(path)::line. The >> use case: >> >> String s = Files.read(path); >> Files.write(path, s.replace("/config/location", "/new/location")); >> >> would then work as expected. >> >> These two methods are one-off (open/read/write/close file) operation. >> write(String) therefore is not intended for adding/appending multiple >> Strings from other operations such as String.line(). If an app needs >> to put the result of String.line() or any other processes into a file >> using this method, it needs to take care of adding the line separator >> itself when necessary. "when necessary" would be a judgement call by >> the developer. >> >> That said, if there's a consensus on the idea of terminating the >> string with a line separator, then readString method would need to >> strip it off to go with the write method. >> >>> >>>> I would think readString behaves like readAllBytes, that would >>>> simply read all content faithfully. >>> readString does an interpolation due to the Charset so it's not the >>> real content, again, the idea is that developers should not have to >>> care about the platform's line separators (they have more important >>> things to think). >>> >>> The other solution is to just not introduce those new methods, after >>> all Files.lines().collect(Collectors.joining("\n")) already does the >>> job, no ? >> >> While there are many ways to do it, none is as straight-forward. >> "Read (entire) file to String"/"Write String to file" are popular >> requests from users. Read to string -> do some String manipulation -> >> write it back is such a simple use case, it really needs to be very >> easy to do as illustrated in the above code snippet. >> >> Cheers, >> Joe >> >>> >>>> Best, >>>> Joe >>> regards, >>> R?mi >>> >>>> On 4/27/2018 4:43 AM, forax at univ-mlv.fr wrote: >>>>> Hi Alan, >>>>> >>>>> People do not read the documentation. >>>>> So adding something in the documentation about when a method >>>>> should be used or >>>>> not is never a solution. >>>>> >>>>> Here the user want a String and provides a charset so you have no >>>>> way but to >>>>> decode the content to substitute the line separator. >>>>> >>>>> cheers, >>>>> R?mi >>>>> >>>>> ----- Mail original ----- >>>>>> De: "Alan Bateman" >>>>>> ?: "Remi Forax" , "Joe Wang" >>>>>> >>>>>> Cc: "nio-dev" , "core-libs-dev" >>>>>> >>>>>> Envoy?: Vendredi 27 Avril 2018 13:34:12 >>>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>>> reading/writing a string from/to a file >>>>>> On 27/04/2018 12:29, Remi Forax wrote: >>>>>>> I think that having a readString that includes OS dependent line >>>>>>> separators is a >>>>>>> mistake, >>>>>>> Java does a great job to try to shield the developer from the >>>>>>> kind of things >>>>>>> that makes programs behave differently on different platforms. >>>>>>> >>>>>>> readString should subtitute (\r)?\n to \n otherwise either >>>>>>> people will do a call >>>>>>> replace() which is less efficient or will learn the lesson the >>>>>>> hard way. >>>>>>> >>>>>>> raw string literal does the same substitution for the same reason. >>>>>>> >>>>>> Yes, there are several discussion points around this and somewhat >>>>>> timely >>>>>> with multi-string support. >>>>>> >>>>>> One thing that I think Joe will need in this API is some note to >>>>>> make it >>>>>> clearer what the intended usage is (as I think the intention is >>>>>> simple >>>>>> cases with mostly single lines of text). >>>>>> >>>>>> -Alan. >> > From paul.sandoz at oracle.com Tue Jun 12 17:12:39 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 12 Jun 2018 10:12:39 -0700 Subject: RFC: Experiment in accessing/managing persistent memory from Java In-Reply-To: References: <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE74C14@FMSMSX126.amr.corp.intel.com> Message-ID: <382F6C6B-D097-4DF2-A8AB-FBCB60C5D3F5@oracle.com> Hi Jonathan, > On Jun 8, 2018, at 3:59 AM, Jonathan Halliday wrote: > > > Hi Paul > > Looks like we're all on the same page regarding the basic approach of using a small API and making the critical bits intrinsic. We perhaps have some way to go on exactly what that API looks like in terms of the classes and methods, but iterating on it by discussion of a JEP seems like the best way forward. The important thing from my perspective is that so far nobody has come forward with a use case that is not covered by the proposed primitives. So it's a small API, but not too small. Yes, a smallish API we can iterate on. > > As far a tweaks go, we have considered making the low level primitive method / intrinsic just a flushCacheline(base_address), since the arithmetic and loop for writing flush(from,to) in terms of that low level op is something that can be optimized fine by the JIT already. Though that does mean exposing the cache line size to the Java layer, whereas currently it's only visible in the C code. > That?s ok. The simpler the intrinsics while relying on Java code + JIT would be my generally preferred pattern. > > My own background and focus is transactions systems, so I'm more about the speed and fault tolerance than the capacity, but I can see long vs. int being of interest to our Infinispan data grid team and likewise for e.g. Oracle Coherence or databases like Cassandra. OTOH it's not uncommon to prefer moderately sized files and shard over them, which sidesteps the issue. > Ok, which is conveniently how developers currently work around the issue of mapping large files :-) > Utility code to assist with fine-grained memory management within the buffer/file may be more useful than support for really large buffers, since they tend to be used with some form of internal block/heap structure anyhow, rather than to hold very large objects. Providing that may be the role of a 3rd party pure Java library like PCJ though, rather than something we want in the JDK itself at this early stage. The researcher in me is kinda interested in how much of the memory allocation and GC code can be re-purposed here though... > > What's the intended timeline on long buffer indexing at present? Unsure, but it's probably something we want to solve soonish. > My feeling is a pmem API JEP is probably targeting around JDK 13, but we're flexible on that. Note that the JEP process can be started before then and JEPs are not targeted to a release until ready, if its ready sooner great! otherwise later. Keeping such a JEP focused on the mapping/flushing of BBs for NVM would be my recommendation rather than expanding its scope. > We may also want to look at related enhancements like unmapping buffers. I think those pieces are sufficient decoupled that they won't be dependencies for the pmem API though, unlike other factors such as the availability of test hardware. > That?s tricky! We have been through many discussions over the years on how to achieve this without much resolution. Andrew Haley came up with an interesting solution which IIRC requires the deallocating/unmapping thread to effectively reach safe point and wait for all other threads to pass through a check point. Project Panama is looking at the explicit scoping of resources, perhaps also resources that are thread confined or owned. My sense is Project Panama will eventually push strongly on this area and that?s where we should focus our efforts. Thanks, Paul. From Roger.Riggs at Oracle.com Tue Jun 12 17:37:56 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 12 Jun 2018 13:37:56 -0400 Subject: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B1FFA39.9000704@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> Message-ID: <69fc785c-9d19-e972-783e-e23cd9daf13a@Oracle.com> Hi Joe, Looks good to me. Typo: StringCoding.java:1026 "unmappble"? (no new webrev needed) Regards, Roger On 6/12/2018 12:52 PM, Joe Wang wrote: > Hi all, > > It's been a while since the 1st round of reviews. Thank you all for > the valuable comments and suggestions!? Note that Roger's last > response went to nio-dev, but not core-libs-dev, I've therefore > attached it below. > > CSR [2]: the CSR is now approved. Note the write method has been > changed to writeString. > > Impl [3]: for performance reason and the different requirement from > byte-string conversion for malformed/unmappable character handing, the > implementation uses a specific method separate from the existing ones. > Both Sherman and Alan preferred specific method than adding parameters > to the APIs that might be error prone. Thanks Sherman for the code! > > JMH benchmark tests showed the new read method performs better than an > operation that reads the bytes and convert to string. The gains start > to be significant even at quite reasonable sizes (< 10K). > > > [1] JBS: https://bugs.openjdk.java.net/browse/JDK-8201276 > [2] CSR: https://bugs.openjdk.java.net/browse/JDK-8202055 > [3] webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8201276/webrev/ > > Please review. > > Thanks, > Joe > > On 5/15/18, 7:51 AM, Roger Riggs wrote: >> Hi Joe, et. al. >> >> My \$.02 on line separators: >> >> ?- We should avoid clever tricks trying to solve problems that are >> infrequent such as cross OS file line operations. >> ??? Most files will be read/written on a single system so the line >> separators will be as expected. >> >> ?- Have/use APIs that split lines consistently accepting both line >> separators so developer code can be agnostic to line separators. >> ??? aka BufferedReader.readLine for developers that are processing >> the contents *as lines*. >> ?? Those other methods already exist; if there are any gaps in line >> oriented processing that's a separate task. >> >> ?- These new file methods are defined to handle Charset >> encoding/decoding and buffering. >> ??? Since there are other methods to deal with files as lines these >> methods should not look for or break to lines. >> >> ?- Performance: adding code to look for line characters will slow it >> down and in most cases would have no impact >> ??? since the line endings will match the system. >> >> ?- The strings passed to writeString (CharSequence) should have been >> constructed using the proper line separators. >> ??? There are any number of methods that insert the os specific line >> terminator and its easy to write File.separator as needed. >> >> As for the method naming: >> >> I'd prefer "writeString" as a familiar term that isn't stretched too >> far by making the argument be a CharSequence. >> its fine to call a CharSequence a string (with a lower case s). >> >> \$.02, Roger >> >> >> On 4/27/18 6:33 PM, Joe Wang wrote: >>> >>> >>> On 4/27/2018 12:50 PM, forax at univ-mlv.fr wrote: >>>> ----- Mail original ----- >>>>> De: "Joe Wang" >>>>> ?: "Remi Forax" , "Alan Bateman" >>>>> >>>>> Cc: "nio-dev" , "core-libs-dev" >>>>> >>>>> Envoy?: Vendredi 27 Avril 2018 21:21:01 >>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>> reading/writing a string from/to a file >>>>> Hi R?mi, Alan, >>>> Hi Joe, >>>> >>>>> I'm not sure we'd want to replace system dependent line separators >>>>> with >>>>> '\n'. There are cases as you described where the replacement makes >>>>> sense. However,? there are other cases too. For example, the >>>>> purpose is >>>>> to read, edit a text file and then write it back. If this is on >>>>> Windows, >>>>> and if line separators are replaced with '\n', that would cause the >>>>> resulting file not formatted properly in for example Notepad. >>>>> There are >>>>> cases where users write text files on Windows using Java, and only >>>>> found >>>>> the lines were not separated in Notepad. >>>> I agree that why the counterpart of readString() that write the >>>> string should inserts the platform's line separator. >>>> BTW, i just found that those methods are not named writeString, or >>>> writeCharSequence, i think they should. >>> >>> While readString() does not modify the original content (e.g. by >>> replacing the platform's line separator with '\n'), write(String) >>> won't either, by adding extra characters such as the line separator. >>> >>> I would think interfaces shall read along with the parameters. >>> ??? readString(Path) == read as a String from the specified Path >>> (one could argue for readToString, readAsString, but we generally >>> omit the preps) >>> ??? write(Path, CharSequence) == write the CharSequence to the file, >>> since CharSequence is already in the method signature as a >>> parameter, we probably don't want to add that to the name, otherwise >>> it would read like repeating the word CharSequence. >>> >>> It is in a similar situation as write(Path, Iterable) where it was >>> defined as writeLines(Path, Iterable). >>> >>>> >>>>> Files.write(Path, Iterable) is also specified to terminate each line >>>>> with the platform's line separator. If readString does the >>>>> replacement, >>>>> it would be inconsistent. >>>> >>>> Anyway, if we look for consistency the methods writeCharSequence >>>> should transform the \n in the CharSequence to the platform's line >>>> separator. >>>> >>>> Files.write(Path, Iterable) is is not a counterpart of >>>> readString(), it's consistent with Files.lines() or >>>> Files.readLines() (or BufferedReader.readLine()) that all suppress >>>> the line separators. Anyway, Files.write(path, >>>> readString(path)::line) will be consistent if you replace the line >>>> separators or not because String.line() suppresses the line >>>> separators. >>> >>> readString pairs with write(String), therefore it's more like >>> Files.write(path, readString(path)) than readString(path)::line. The >>> use case: >>> >>> ??? String s = Files.read(path); >>> ??? Files.write(path, s.replace("/config/location", "/new/location")); >>> >>> would then work as expected. >>> >>> These two methods are one-off (open/read/write/close file) >>> operation. write(String) therefore is not intended for >>> adding/appending multiple Strings from other operations such as >>> String.line(). If an app needs to put the result of String.line() or >>> any other processes into a file using this method, it needs to take >>> care of adding the line separator itself when necessary. "when >>> necessary" would be a judgement call by the developer. >>> >>> That said, if there's a consensus on the idea of terminating the >>> string with a line separator, then readString method would need to >>> strip it off to go with the write method. >>> >>>> >>>>> I would think readString behaves like readAllBytes, that would >>>>> simply read all content faithfully. >>>> readString does an interpolation due to the Charset so it's not the >>>> real content, again, the idea is that developers should not have to >>>> care about the platform's line separators (they have more important >>>> things to think). >>>> >>>> The other solution is to just not introduce those new methods, >>>> after all Files.lines().collect(Collectors.joining("\n")) already >>>> does the job, no ? >>> >>> While there are many ways to do it, none is as straight-forward. >>> "Read (entire) file to String"/"Write String to file" are popular >>> requests from users. Read to string -> do some String manipulation >>> -> write it back is such a simple use case, it really needs to be >>> very easy to do as illustrated in the above code snippet. >>> >>> Cheers, >>> Joe >>> >>>> >>>>> Best, >>>>> Joe >>>> regards, >>>> R?mi >>>> >>>>> On 4/27/2018 4:43 AM, forax at univ-mlv.fr wrote: >>>>>> Hi Alan, >>>>>> >>>>>> People do not read the documentation. >>>>>> So adding something in the documentation about when a method >>>>>> should be used or >>>>>> not is never a solution. >>>>>> >>>>>> Here the user want a String and provides a charset so you have no >>>>>> way but to >>>>>> decode the content to substitute the line separator. >>>>>> >>>>>> cheers, >>>>>> R?mi >>>>>> >>>>>> ----- Mail original ----- >>>>>>> De: "Alan Bateman" >>>>>>> ?: "Remi Forax" , "Joe Wang" >>>>>>> >>>>>>> Cc: "nio-dev" , "core-libs-dev" >>>>>>> >>>>>>> Envoy?: Vendredi 27 Avril 2018 13:34:12 >>>>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>>>> reading/writing a string from/to a file >>>>>>> On 27/04/2018 12:29, Remi Forax wrote: >>>>>>>> I think that having a readString that includes OS dependent >>>>>>>> line separators is a >>>>>>>> mistake, >>>>>>>> Java does a great job to try to shield the developer from the >>>>>>>> kind of things >>>>>>>> that makes programs behave differently on different platforms. >>>>>>>> >>>>>>>> readString should subtitute (\r)?\n to \n otherwise either >>>>>>>> people will do a call >>>>>>>> replace() which is less efficient or will learn the lesson the >>>>>>>> hard way. >>>>>>>> >>>>>>>> raw string literal does the same substitution for the same reason. >>>>>>>> >>>>>>> Yes, there are several discussion points around this and >>>>>>> somewhat timely >>>>>>> with multi-string support. >>>>>>> >>>>>>> One thing that I think Joe will need in this API is some note to >>>>>>> make it >>>>>>> clearer what the intended usage is (as I think the intention is >>>>>>> simple >>>>>>> cases with mostly single lines of text). >>>>>>> >>>>>>> -Alan. >>> >> From aph at redhat.com Tue Jun 12 17:58:00 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 12 Jun 2018 18:58:00 +0100 Subject: RFC: Experiment in accessing/managing persistent memory from Java In-Reply-To: <382F6C6B-D097-4DF2-A8AB-FBCB60C5D3F5@oracle.com> References: <02FCFB8477C4EF43A2AD8E0C60F3DA2B9EE74C14@FMSMSX126.amr.corp.intel.com> <382F6C6B-D097-4DF2-A8AB-FBCB60C5D3F5@oracle.com> Message-ID: <7defb48e-c77a-4ca5-119e-bacd7cf9aa93@redhat.com> On 06/12/2018 06:12 PM, Paul Sandoz wrote: > >> We may also want to look at related enhancements like unmapping >> buffers. I think those pieces are sufficient decoupled that they >> won't be dependencies for the pmem API though, unlike other factors >> such as the availability of test hardware. >> > That?s tricky! We have been through many discussions over the years > on how to achieve this without much resolution. Andrew Haley came up > with an interesting solution which IIRC requires the > deallocating/unmapping thread to effectively reach safe point and > wait for all other threads to pass through a check point. Project > Panama is looking at the explicit scoping of resources, perhaps also > resources that are thread confined or owned. My sense is Project > Panama will eventually push strongly on this area and that?s where > we should focus our efforts. Yeah, perhaps so. I've been waiting to come up for air to have enough time to handle the ByteBuffer.unmap() bug. I can see the advantage of handling it at a static language level, but the solutions aren't necessarily exclusive. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From joe.darcy at oracle.com Wed Jun 13 00:49:13 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Tue, 12 Jun 2018 17:49:13 -0700 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <82fdd320-f5fb-2c34-5bb8-b72f6718fef6@oracle.com> References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <82fdd320-f5fb-2c34-5bb8-b72f6718fef6@oracle.com> Message-ID: <5B206A09.5000404@oracle.com> On 6/11/2018 10:38 PM, mandy chung wrote: > > > On 6/11/18 10:16 PM, David Holmes wrote: >> Here is one further minor update from the CSR discussions: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v5-incr/src/java.base/share/classes/java/lang/Class.java.cdiff.html > > > "This implementation" is fine, as used in many @implNote. Any reason > why it has to be changed to "reference implementation"? > To summarize the concern there, the phrase "This implementation..." when used elsewhere has a different meaning. Often the phrasing "This implementation..." or the more commonly used "The default implementation..." is used for text that is part of the contract of a method that can be overridden; that is, used to separate out the contract that is independent of which class or interface provides the implementation from the contract of a particular implementation. One example from an API I work on occurs for the method javax.lang.model.element.ElementVisitor.visitModule. The default method defined in an interface states "The default implementation visits a ModuleElement by calling visitUnknown..." and then various visitor classes define their own behavior for this method while still being able to @inheritDoc the general "visit a module element" contract of the visitModule method. However, java.lang.Class is final so for a particular JDK version there is only one implementation of the method in question. In that context "This implementation" doesn't mean "the implementation in this particular class or interface as opposed to the implementation in an a more specific subtype" it means "the implemetnation for the final method used in a particular JDK release." I think using the term "This implementation" in the latter context is misleading so I suggested the alternative wording David sent out for review. HTH, -Joe From weijun.wang at oracle.com Wed Jun 13 01:59:49 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Wed, 13 Jun 2018 09:59:49 +0800 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: References: Message-ID: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> Hi Roger 1. Should all occurrences of reading of these system properties be updated? For example, the following one is not touched http://hg.openjdk.java.net/jdk/jdk/file/4d2e3f5abb48/src/java.base/share/classes/sun/security/tools/keytool/Main.java#l842 2. I assume that with this change not only there is no use calling System.setProperty() in the application but also setting it on the command line is now useless. Is this right? Do we need to make this clear in the CSR? Thanks Max > On Jun 4, 2018, at 9:32 PM, Roger Riggs wrote: > > Please review a change to make the values of java.home, user.home, user.dir, and user.name > effectively read-only for internal use. The values are cached during initialization and the > cached values are used. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8066709 > > CSR: > https://bugs.openjdk.java.net/browse/JDK-8204235 > > Thanks, Roger > > From weijun.wang at oracle.com Wed Jun 13 02:10:57 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Wed, 13 Jun 2018 10:10:57 +0800 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> Message-ID: <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> In fact, why is setting user.name and user.home always evil? If I only want to set them on the command line so that a special "user environment" is used, why is it a problem? In fact, we have a test setting user.home to an empty directory to avoid unexpected result because we cannot control the test runner's home directory. Thanks Max > On Jun 13, 2018, at 9:59 AM, Weijun Wang wrote: > > Hi Roger > > 1. Should all occurrences of reading of these system properties be updated? For example, the following one is not touched > http://hg.openjdk.java.net/jdk/jdk/file/4d2e3f5abb48/src/java.base/share/classes/sun/security/tools/keytool/Main.java#l842 > > 2. I assume that with this change not only there is no use calling System.setProperty() in the application but also setting it on the command line is now useless. Is this right? Do we need to make this clear in the CSR? > > Thanks > Max > > >> On Jun 4, 2018, at 9:32 PM, Roger Riggs wrote: >> >> Please review a change to make the values of java.home, user.home, user.dir, and user.name >> effectively read-only for internal use. The values are cached during initialization and the >> cached values are used. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8066709 >> >> CSR: >> https://bugs.openjdk.java.net/browse/JDK-8204235 >> >> Thanks, Roger >> >> > From rachna.goel at oracle.com Wed Jun 13 05:33:49 2018 From: rachna.goel at oracle.com (Rachna Goel) Date: Wed, 13 Jun 2018 11:03:49 +0530 Subject: RFR: [11] 8202537: CLDR 33 Message-ID: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> Hi, Kindly review fix for JDK-8202537. Fix is to upgrade Unicode consortium's CLDR data into JDK from current version 29 to 33. For more info : http://cldr.unicode.org/index/downloads/cldr-33 Bug : https://bugs.openjdk.java.net/browse/JDK-8202537 Patch: http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.06/index.html -- Thanks, Rachna From joe.darcy at oracle.com Wed Jun 13 06:08:55 2018 From: joe.darcy at oracle.com (joe darcy) Date: Tue, 12 Jun 2018 23:08:55 -0700 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> Message-ID: <9cfdde9b-5929-a5b0-ea26-e79a6d4f56e6@oracle.com> Hi David, On 5/24/2018 10:52 PM, David Holmes wrote: > Here are the further minor updates so far in response to all the > review comments. > > Incremental corelibs webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3-incr/ > > > Full corelibs webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3/ > In Class.java, 3990???????? Class[] members = getNestMembers0(); 3991???????? // Can't actually enable this due to bootstrapping issues 3992???????? // assert(members.length != 1 || members[0] == this); // expected invariant from VM can these checks be enabled unconditionally without using the assert facility, throwing AssertionError or some other kind of error? Thanks, -Joe From amy.lu at oracle.com Wed Jun 13 09:31:23 2018 From: amy.lu at oracle.com (Amy Lu) Date: Wed, 13 Jun 2018 17:31:23 +0800 Subject: [JDK 11] RFR 8204944: Remove java/util/Map/InPlaceOpsCollisions.java from ProblemList.txt Message-ID: <2ed84e62-71db-22fc-e892-f48405985a97@oracle.com> Please review this patch to remove java/util/Map/InPlaceOpsCollisions.java from ProblemList.txt, related bug JDK-8203387 (JDK-8203425) has been fixed. Thanks, Amy --- old/test/jdk/ProblemList.txt??? 2018-06-13 17:23:33.000000000 +0800 +++ new/test/jdk/ProblemList.txt??? 2018-06-13 17:23:32.000000000 +0800 @@ -853,8 +853,6 @@ ?# jdk_util -java/util/Map/InPlaceOpsCollisions.java 8203387 generic-all - ?############################################################################ ?# jdk_instrument From thomas.stuefe at gmail.com Wed Jun 13 09:56:43 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 13 Jun 2018 11:56:43 +0200 Subject: RFR(xs): 8204663: clean up remaining native parts after JDK-8187631 In-Reply-To: References: Message-ID: May I have a second review please. Thanks, Thomas On Mon, Jun 11, 2018, 15:13 Thomas St?fe wrote: > Hi, > > may I please have reviews for this small cleanup. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204663 > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/JDK-8204663-clean-up-after-JDK-8187631/webrev.00/webrev/ > > This change removes some native functions from FOS/FIS which are > orphaned and unused since "JDK-8187631: Refactor FileDescriptor close > implementation". > > Tests on jdk-submit came back clean save one strange test error on > windows which I cannot reproduce locally. I am investigating that one. > > Thank you, Thomas > From Roger.Riggs at Oracle.com Wed Jun 13 14:10:02 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 13 Jun 2018 10:10:02 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> Message-ID: <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> Hi Joe, The CSR text is still in draft and got out of sync with the open review and comments. The updated apiNote clarifies that changes made through the System.*property*? methods may not affect the value cached during initialization. The implementation changes affect a very short list of properties. The note on the methods is to raise awareness that individual properties may have different behavior and may be unpredictable. On 6/12/2018 10:10 PM, Weijun Wang wrote: > In fact, why is setting user.name and user.home always evil? If I only want to set them on the command line so that a special "user environment" is used, why is it a problem? There is no change to the ability to set properties on the command line. The concern is with the predictability of (some) properties changing dynamically while the application is running. The property user.home is used for mime-types and mailcap files and user.name is used in some networking cases where a username is needed. > > In fact, we have a test setting user.home to an empty directory to avoid unexpected result because we cannot control the test runner's home directory. Right, there is no problem with that Regards, Roger > > Thanks > Max > >> On Jun 13, 2018, at 9:59 AM, Weijun Wang wrote: >> >> Hi Roger >> >> 1. Should all occurrences of reading of these system properties be updated? For example, the following one is not touched >> http://hg.openjdk.java.net/jdk/jdk/file/4d2e3f5abb48/src/java.base/share/classes/sun/security/tools/keytool/Main.java#l842 That usage was in a tool and I did not modify any tools. >> >> 2. I assume that with this change not only there is no use calling System.setProperty() in the application but also setting it on the command line is now useless. Is this right? Do we need to make this clear in the CSR? No change to command line setting of properties. Roger >> >> Thanks >> Max >> >> >>> On Jun 4, 2018, at 9:32 PM, Roger Riggs wrote: >>> >>> Please review a change to make the values of java.home, user.home, user.dir, and user.name >>> effectively read-only for internal use. The values are cached during initialization and the >>> cached values are used. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>> >>> CSR: >>> https://bugs.openjdk.java.net/browse/JDK-8204235 >>> >>> Thanks, Roger >>> >>> From Roger.Riggs at Oracle.com Wed Jun 13 14:36:52 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 13 Jun 2018 10:36:52 -0400 Subject: RFR: [11] 8202537: CLDR 33 In-Reply-To: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> References: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> Message-ID: <045baa5b-ce2a-962d-f480-d95ec9e9b63e@Oracle.com> Hi Rachna, How much of the updates are done mechanically between the CLDR data and the .xml files? Manually reviewing that many files is likely to be error prone. Thanks, Roger On 6/13/2018 1:33 AM, Rachna Goel wrote: > Hi, > > Kindly review fix for JDK-8202537. Fix is to upgrade Unicode > consortium's CLDR data into JDK from current version 29 to 33. > > For more info : http://cldr.unicode.org/index/downloads/cldr-33 > > Bug : https://bugs.openjdk.java.net/browse/JDK-8202537 > > Patch: http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.06/index.html > From rachna.goel at oracle.com Wed Jun 13 15:07:30 2018 From: rachna.goel at oracle.com (Rachna Goel) Date: Wed, 13 Jun 2018 20:37:30 +0530 Subject: RFR: [11] 8202537: CLDR 33 In-Reply-To: <045baa5b-ce2a-962d-f480-d95ec9e9b63e@Oracle.com> References: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> <045baa5b-ce2a-962d-f480-d95ec9e9b63e@Oracle.com> Message-ID: <10896dc3-76df-3abd-e706-1814466badda@oracle.com> Hi Roger, No update is done mechanically between CLDR data and .xml files. CLDR's data in .xml files are generated into resourcebundles at build time by cldrconverter tool. Already existing regression test "test/jdk/sun/text/resources/LocaleDataTest.java" verify that same data is retrieved by APIs. Thanks, Rachna* * On 6/13/18 8:06 PM, Roger Riggs wrote: > Hi Rachna, > > How much of the updates are done mechanically between the CLDR data > and the .xml files? > Manually reviewing that many files is likely to be error prone. > > Thanks, Roger > > > On 6/13/2018 1:33 AM, Rachna Goel wrote: >> Hi, >> >> Kindly review fix for JDK-8202537. Fix is to upgrade Unicode >> consortium's CLDR data into JDK from current version 29 to 33. >> >> For more info : http://cldr.unicode.org/index/downloads/cldr-33 >> >> Bug : https://bugs.openjdk.java.net/browse/JDK-8202537 >> >> Patch: >> http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.06/index.html >> > -- Thanks, Rachna From paul.sandoz at oracle.com Wed Jun 13 16:19:39 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 13 Jun 2018 09:19:39 -0700 Subject: RFR(xs): 8204663: clean up remaining native parts after JDK-8187631 In-Reply-To: References: Message-ID: <35C6B224-6AE8-4DD8-A4B2-36889AB7EB0E@oracle.com> +1 Paul. > On Jun 13, 2018, at 2:56 AM, Thomas St?fe wrote: > > May I have a second review please. > > Thanks, Thomas > > On Mon, Jun 11, 2018, 15:13 Thomas St?fe wrote: > >> Hi, >> >> may I please have reviews for this small cleanup. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204663 >> Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/JDK-8204663-clean-up-after-JDK-8187631/webrev.00/webrev/ >> >> This change removes some native functions from FOS/FIS which are >> orphaned and unused since "JDK-8187631: Refactor FileDescriptor close >> implementation". >> >> Tests on jdk-submit came back clean save one strange test error on >> windows which I cannot reproduce locally. I am investigating that one. >> >> Thank you, Thomas >> From paul.sandoz at oracle.com Wed Jun 13 16:21:17 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 13 Jun 2018 09:21:17 -0700 Subject: [JDK 11] RFR 8204944: Remove java/util/Map/InPlaceOpsCollisions.java from ProblemList.txt In-Reply-To: <2ed84e62-71db-22fc-e892-f48405985a97@oracle.com> References: <2ed84e62-71db-22fc-e892-f48405985a97@oracle.com> Message-ID: +1 Paul. > On Jun 13, 2018, at 2:31 AM, Amy Lu wrote: > > Please review this patch to remove java/util/Map/InPlaceOpsCollisions.java from ProblemList.txt, > related bug JDK-8203387 (JDK-8203425) has been fixed. > > Thanks, > Amy > > --- old/test/jdk/ProblemList.txt 2018-06-13 17:23:33.000000000 +0800 > +++ new/test/jdk/ProblemList.txt 2018-06-13 17:23:32.000000000 +0800 > @@ -853,8 +853,6 @@ > > # jdk_util > > -java/util/Map/InPlaceOpsCollisions.java 8203387 generic-all > - > ############################################################################ > > # jdk_instrument > > From srinivas.dama at oracle.com Wed Jun 13 16:57:32 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Wed, 13 Jun 2018 09:57:32 -0700 (PDT) Subject: RFR: 8204967: Resolve disabled warnings for libunpack Message-ID: <4cd2a4ef-7dde-469a-8218-230693ba7e68@default> Hi, Please review http://cr.openjdk.java.net/~sdama/8204967/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8204967 Regards, Srinivas From naoto.sato at oracle.com Wed Jun 13 17:03:23 2018 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Wed, 13 Jun 2018 10:03:23 -0700 Subject: RFR: [11] 8202537: CLDR 33 In-Reply-To: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> References: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> Message-ID: Hi Rachna, A couple of comments: - NumberingSystemsParseHandler.java Since the code substitutes latin digits for supplementary digits, it can skip line 68-79. - A test should be written for the above substitution. Naoto On 6/12/18 10:33 PM, Rachna Goel wrote: > Hi, > > Kindly review fix for JDK-8202537. Fix is to upgrade Unicode > consortium's CLDR data into JDK from current version 29 to 33. > > For more info : http://cldr.unicode.org/index/downloads/cldr-33 > > Bug : https://bugs.openjdk.java.net/browse/JDK-8202537 > > Patch: http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.06/index.html > From james.laskey at oracle.com Wed Jun 13 17:07:26 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Wed, 13 Jun 2018 14:07:26 -0300 Subject: RFR: 8204967: Resolve disabled warnings for libunpack In-Reply-To: <4cd2a4ef-7dde-469a-8218-230693ba7e68@default> References: <4cd2a4ef-7dde-469a-8218-230693ba7e68@default> Message-ID: Does -Wimplicit-fallthrough=3 work for gnu c and clang? unpack.cpp redundant comment 3713 // else fall through: +1 ? Jim > On Jun 13, 2018, at 1:57 PM, Srinivas Dama wrote: > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8204967/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8204967 > > Regards, > Srinivas From thomas.stuefe at gmail.com Wed Jun 13 17:20:06 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 13 Jun 2018 19:20:06 +0200 Subject: RFR(xs): 8204663: clean up remaining native parts after JDK-8187631 In-Reply-To: <35C6B224-6AE8-4DD8-A4B2-36889AB7EB0E@oracle.com> References: <35C6B224-6AE8-4DD8-A4B2-36889AB7EB0E@oracle.com> Message-ID: Thank you Paul! On Wed, Jun 13, 2018 at 6:19 PM, Paul Sandoz wrote: > +1 > Paul. > >> On Jun 13, 2018, at 2:56 AM, Thomas St?fe wrote: >> >> May I have a second review please. >> >> Thanks, Thomas >> >> On Mon, Jun 11, 2018, 15:13 Thomas St?fe wrote: >> >>> Hi, >>> >>> may I please have reviews for this small cleanup. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8204663 >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/JDK-8204663-clean-up-after-JDK-8187631/webrev.00/webrev/ >>> >>> This change removes some native functions from FOS/FIS which are >>> orphaned and unused since "JDK-8187631: Refactor FileDescriptor close >>> implementation". >>> >>> Tests on jdk-submit came back clean save one strange test error on >>> windows which I cannot reproduce locally. I am investigating that one. >>> >>> Thank you, Thomas >>> > From james.laskey at oracle.com Wed Jun 13 19:36:18 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Wed, 13 Jun 2018 16:36:18 -0300 Subject: RFR: JDK-8204172 Predicate::not should explicitly mention "NullPointerException - if target is null" Message-ID: <05556A68-691F-44F7-8163-D46507B00F95@oracle.com> CSR: https://bugs.openjdk.java.net/browse/JDK-8204959 webrev: http://cr.openjdk.java.net/~jlaskey/8204172/webrev/index.html Thank you, ? Jim From erik.joelsson at oracle.com Wed Jun 13 19:47:52 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 13 Jun 2018 12:47:52 -0700 Subject: RFR: JDK-8204973: Add build support for filtering translations Message-ID: Hello, Oracle will reduce the number of languages that it maintains translations of JDK resources for. The current translations will remain in the source for now, but we need a way to filter out a set of translations at build time so that we only include the ones we support. This patch adds such a configuration option. It also changes how Oracle builds by using the option to exclude all translations except English, Japanese, Simplified Chinese and Traditional Chinese. Anyone else building OpenJDK will by default include all translations present in the source, just as before. I added a test that verifies this for builds with the "IMPLEMENTOR" field in the release file set to "Oracle Corporation". The test will not be run for other OpenJDK builds. I had to modify an existing test for java.logging which used various translations to verify localized log messages to only use translations that Oracle chooses to include. Since this is the second test that specifically verifies build behavior, I moved the previous such test together with this new test into a common top level test directory "build", under the jdk test root. I put these tests in the jdk tier3 test group. I have run all tier1, 2 and 3 tests in Mach 5 as well as specifically looked for tests that use the java.util.Locale class and ran them locally. Webrev: http://cr.openjdk.java.net/~erikj/8204973/webrev.01/index.html Bug: https://bugs.openjdk.java.net/browse/JDK-8204973 /Erik From huizhe.wang at oracle.com Wed Jun 13 19:52:30 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Wed, 13 Jun 2018 12:52:30 -0700 Subject: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <69fc785c-9d19-e972-783e-e23cd9daf13a@Oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> <69fc785c-9d19-e972-783e-e23cd9daf13a@Oracle.com> Message-ID: <5B2175FE.8040500@oracle.com> Thanks Roger! I pushed the changeset after s/unmappble/unmappable, it found three of them :-) Best, Joe On 6/12/18, 10:37 AM, Roger Riggs wrote: > Hi Joe, > > Looks good to me. > > Typo: StringCoding.java:1026 "unmappble" (no new webrev needed) > > Regards, Roger > > > On 6/12/2018 12:52 PM, Joe Wang wrote: >> Hi all, >> >> It's been a while since the 1st round of reviews. Thank you all for >> the valuable comments and suggestions! Note that Roger's last >> response went to nio-dev, but not core-libs-dev, I've therefore >> attached it below. >> >> CSR [2]: the CSR is now approved. Note the write method has been >> changed to writeString. >> >> Impl [3]: for performance reason and the different requirement from >> byte-string conversion for malformed/unmappable character handing, >> the implementation uses a specific method separate from the existing >> ones. Both Sherman and Alan preferred specific method than adding >> parameters to the APIs that might be error prone. Thanks Sherman for >> the code! >> >> JMH benchmark tests showed the new read method performs better than >> an operation that reads the bytes and convert to string. The gains >> start to be significant even at quite reasonable sizes (< 10K). >> >> >> [1] JBS: https://bugs.openjdk.java.net/browse/JDK-8201276 >> [2] CSR: https://bugs.openjdk.java.net/browse/JDK-8202055 >> [3] webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8201276/webrev/ >> >> Please review. >> >> Thanks, >> Joe >> >> On 5/15/18, 7:51 AM, Roger Riggs wrote: >>> Hi Joe, et. al. >>> >>> My \$.02 on line separators: >>> >>> - We should avoid clever tricks trying to solve problems that are >>> infrequent such as cross OS file line operations. >>> Most files will be read/written on a single system so the line >>> separators will be as expected. >>> >>> - Have/use APIs that split lines consistently accepting both line >>> separators so developer code can be agnostic to line separators. >>> aka BufferedReader.readLine for developers that are processing >>> the contents *as lines*. >>> Those other methods already exist; if there are any gaps in line >>> oriented processing that's a separate task. >>> >>> - These new file methods are defined to handle Charset >>> encoding/decoding and buffering. >>> Since there are other methods to deal with files as lines these >>> methods should not look for or break to lines. >>> >>> - Performance: adding code to look for line characters will slow it >>> down and in most cases would have no impact >>> since the line endings will match the system. >>> >>> - The strings passed to writeString (CharSequence) should have been >>> constructed using the proper line separators. >>> There are any number of methods that insert the os specific line >>> terminator and its easy to write File.separator as needed. >>> >>> As for the method naming: >>> >>> I'd prefer "writeString" as a familiar term that isn't stretched too >>> far by making the argument be a CharSequence. >>> its fine to call a CharSequence a string (with a lower case s). >>> >>> \$.02, Roger >>> >>> >>> On 4/27/18 6:33 PM, Joe Wang wrote: >>>> >>>> >>>> On 4/27/2018 12:50 PM, forax at univ-mlv.fr wrote: >>>>> ----- Mail original ----- >>>>>> De: "Joe Wang" >>>>>> ?: "Remi Forax" , "Alan Bateman" >>>>>> >>>>>> Cc: "nio-dev" , "core-libs-dev" >>>>>> >>>>>> Envoy?: Vendredi 27 Avril 2018 21:21:01 >>>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>>> reading/writing a string from/to a file >>>>>> Hi R?mi, Alan, >>>>> Hi Joe, >>>>> >>>>>> I'm not sure we'd want to replace system dependent line >>>>>> separators with >>>>>> '\n'. There are cases as you described where the replacement makes >>>>>> sense. However, there are other cases too. For example, the >>>>>> purpose is >>>>>> to read, edit a text file and then write it back. If this is on >>>>>> Windows, >>>>>> and if line separators are replaced with '\n', that would cause the >>>>>> resulting file not formatted properly in for example Notepad. >>>>>> There are >>>>>> cases where users write text files on Windows using Java, and >>>>>> only found >>>>>> the lines were not separated in Notepad. >>>>> I agree that why the counterpart of readString() that write the >>>>> string should inserts the platform's line separator. >>>>> BTW, i just found that those methods are not named writeString, or >>>>> writeCharSequence, i think they should. >>>> >>>> While readString() does not modify the original content (e.g. by >>>> replacing the platform's line separator with '\n'), write(String) >>>> won't either, by adding extra characters such as the line separator. >>>> >>>> I would think interfaces shall read along with the parameters. >>>> readString(Path) == read as a String from the specified Path >>>> (one could argue for readToString, readAsString, but we generally >>>> omit the preps) >>>> write(Path, CharSequence) == write the CharSequence to the >>>> file, since CharSequence is already in the method signature as a >>>> parameter, we probably don't want to add that to the name, >>>> otherwise it would read like repeating the word CharSequence. >>>> >>>> It is in a similar situation as write(Path, Iterable) where it was >>>> defined as writeLines(Path, Iterable). >>>> >>>>> >>>>>> Files.write(Path, Iterable) is also specified to terminate each line >>>>>> with the platform's line separator. If readString does the >>>>>> replacement, >>>>>> it would be inconsistent. >>>>> >>>>> Anyway, if we look for consistency the methods writeCharSequence >>>>> should transform the \n in the CharSequence to the platform's line >>>>> separator. >>>>> >>>>> Files.write(Path, Iterable) is is not a counterpart of >>>>> readString(), it's consistent with Files.lines() or >>>>> Files.readLines() (or BufferedReader.readLine()) that all suppress >>>>> the line separators. Anyway, Files.write(path, >>>>> readString(path)::line) will be consistent if you replace the line >>>>> separators or not because String.line() suppresses the line >>>>> separators. >>>> >>>> readString pairs with write(String), therefore it's more like >>>> Files.write(path, readString(path)) than readString(path)::line. >>>> The use case: >>>> >>>> String s = Files.read(path); >>>> Files.write(path, s.replace("/config/location", "/new/location")); >>>> >>>> would then work as expected. >>>> >>>> These two methods are one-off (open/read/write/close file) >>>> operation. write(String) therefore is not intended for >>>> adding/appending multiple Strings from other operations such as >>>> String.line(). If an app needs to put the result of String.line() >>>> or any other processes into a file using this method, it needs to >>>> take care of adding the line separator itself when necessary. "when >>>> necessary" would be a judgement call by the developer. >>>> >>>> That said, if there's a consensus on the idea of terminating the >>>> string with a line separator, then readString method would need to >>>> strip it off to go with the write method. >>>> >>>>> >>>>>> I would think readString behaves like readAllBytes, that would >>>>>> simply read all content faithfully. >>>>> readString does an interpolation due to the Charset so it's not >>>>> the real content, again, the idea is that developers should not >>>>> have to care about the platform's line separators (they have more >>>>> important things to think). >>>>> >>>>> The other solution is to just not introduce those new methods, >>>>> after all Files.lines().collect(Collectors.joining("\n")) already >>>>> does the job, no ? >>>> >>>> While there are many ways to do it, none is as straight-forward. >>>> "Read (entire) file to String"/"Write String to file" are popular >>>> requests from users. Read to string -> do some String manipulation >>>> -> write it back is such a simple use case, it really needs to be >>>> very easy to do as illustrated in the above code snippet. >>>> >>>> Cheers, >>>> Joe >>>> >>>>> >>>>>> Best, >>>>>> Joe >>>>> regards, >>>>> R?mi >>>>> >>>>>> On 4/27/2018 4:43 AM, forax at univ-mlv.fr wrote: >>>>>>> Hi Alan, >>>>>>> >>>>>>> People do not read the documentation. >>>>>>> So adding something in the documentation about when a method >>>>>>> should be used or >>>>>>> not is never a solution. >>>>>>> >>>>>>> Here the user want a String and provides a charset so you have >>>>>>> no way but to >>>>>>> decode the content to substitute the line separator. >>>>>>> >>>>>>> cheers, >>>>>>> R?mi >>>>>>> >>>>>>> ----- Mail original ----- >>>>>>>> De: "Alan Bateman" >>>>>>>> ?: "Remi Forax" , "Joe Wang" >>>>>>>> >>>>>>>> Cc: "nio-dev" , "core-libs-dev" >>>>>>>> >>>>>>>> Envoy?: Vendredi 27 Avril 2018 13:34:12 >>>>>>>> Objet: Re: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for >>>>>>>> reading/writing a string from/to a file >>>>>>>> On 27/04/2018 12:29, Remi Forax wrote: >>>>>>>>> I think that having a readString that includes OS dependent >>>>>>>>> line separators is a >>>>>>>>> mistake, >>>>>>>>> Java does a great job to try to shield the developer from the >>>>>>>>> kind of things >>>>>>>>> that makes programs behave differently on different platforms. >>>>>>>>> >>>>>>>>> readString should subtitute (\r)?\n to \n otherwise either >>>>>>>>> people will do a call >>>>>>>>> replace() which is less efficient or will learn the lesson the >>>>>>>>> hard way. >>>>>>>>> >>>>>>>>> raw string literal does the same substitution for the same >>>>>>>>> reason. >>>>>>>>> >>>>>>>> Yes, there are several discussion points around this and >>>>>>>> somewhat timely >>>>>>>> with multi-string support. >>>>>>>> >>>>>>>> One thing that I think Joe will need in this API is some note >>>>>>>> to make it >>>>>>>> clearer what the intended usage is (as I think the intention is >>>>>>>> simple >>>>>>>> cases with mostly single lines of text). >>>>>>>> >>>>>>>> -Alan. >>>> >>> > From daniel.fuchs at oracle.com Wed Jun 13 19:58:28 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 13 Jun 2018 20:58:28 +0100 Subject: RFR: JDK-8204973: Add build support for filtering translations In-Reply-To: References: Message-ID: <8eab54d8-d225-18d3-7472-aa663faba527@oracle.com> Hi Erik, The modifications to the logging test look good to me. Caveat: I don't speak chinese nor japanese ;-) cheers, -- daniel On 13/06/18 20:47, Erik Joelsson wrote: > Hello, > > Oracle will reduce the number of languages that it maintains > translations of JDK resources for. The current translations will remain > in the source for now, but we need a way to filter out a set of > translations at build time so that we only include the ones we support. > This patch adds such a configuration option. It also changes how Oracle > builds by using the option to exclude all translations except English, > Japanese, Simplified Chinese and Traditional Chinese. Anyone else > building OpenJDK will by default include all translations present in the > source, just as before. > > I added a test that verifies this for builds with the "IMPLEMENTOR" > field in the release file set to "Oracle Corporation". The test will not > be run for other OpenJDK builds. > > I had to modify an existing test for java.logging which used various > translations to verify localized log messages to only use translations > that Oracle chooses to include. > > Since this is the second test that specifically verifies build behavior, > I moved the previous such test together with this new test into a common > top level test directory "build", under the jdk test root. I put these > tests in the jdk tier3 test group. > > I have run all tier1, 2 and 3 tests in Mach 5 as well as specifically > looked for tests that use the java.util.Locale class and ran them locally. > > Webrev: http://cr.openjdk.java.net/~erikj/8204973/webrev.01/index.html > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204973 > > /Erik > From daniel.fuchs at oracle.com Wed Jun 13 20:01:25 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 13 Jun 2018 21:01:25 +0100 Subject: RFR: JDK-8204172 Predicate::not should explicitly mention "NullPointerException - if target is null" In-Reply-To: <05556A68-691F-44F7-8163-D46507B00F95@oracle.com> References: <05556A68-691F-44F7-8163-D46507B00F95@oracle.com> Message-ID: Hi Jim, Looks good to me. best regards, -- daniel On 13/06/18 20:36, Jim Laskey wrote: > CSR: https://bugs.openjdk.java.net/browse/JDK-8204959 > webrev: http://cr.openjdk.java.net/~jlaskey/8204172/webrev/index.html > > Thank you, > > ? Jim > From patrick at reini.net Wed Jun 13 20:22:26 2018 From: patrick at reini.net (Patrick Reinhart) Date: Wed, 13 Jun 2018 22:22:26 +0200 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior Message-ID: Hi everybody, While looking into the current Reader Spec and the failing methods I found that the ready() method actually does behave wrong and I fixed this. For the case of mark() I would like to revise the specification to align with the Reader?s default behavior that states for the mark method: IOException - If the stream does not support mark(), or if some other I/O error occurs The new spec for those methods would then read like: 70 * The {@code markSupported()} method returns {@code false}. The 71 * {@code mark()} and {@code reset()} methods throw an {@code IOException}. Here the link to the webrev: http://cr.openjdk.java.net/~reinhapa/reviews/8204930/webrev -Patrick From brian.burkhalter at oracle.com Wed Jun 13 20:56:36 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 13 Jun 2018 13:56:36 -0700 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: References: Message-ID: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> Hi Patrick, Not part of your change, but I noticed that at line 66 of Reader.java there is an extra parenthesis after ready(). In the test, the bug ID at line 39 could simply be appended to line 38. Otherwise looks good although I suppose given the specification update you?ll need an approved CSR before checking it in. Thanks, Brian On Jun 13, 2018, at 1:22 PM, Patrick Reinhart wrote: > While looking into the current Reader Spec and the failing methods I found that the ready() method actually does behave wrong and I fixed this. > > For the case of mark() I would like to revise the specification to align with the Reader?s default behavior that states for the mark method: > > IOException - If the stream does not support mark(), or if some other I/O error occurs > > The new spec for those methods would then read like: > > 70 * The {@code markSupported()} method returns {@code false}. The > 71 * {@code mark()} and {@code reset()} methods throw an {@code IOException}. > > > Here the link to the webrev: > > http://cr.openjdk.java.net/~reinhapa/reviews/8204930/webrev From roger.riggs at oracle.com Wed Jun 13 21:06:08 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Wed, 13 Jun 2018 17:06:08 -0400 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> References: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> Message-ID: Hi Patrick, Yes, looks good to me too. For the CSR, the easiest path to clarify the spec is to withdraw the CSR, update the spec, and add a note on the revised behavior. Then finalize the CSR again.? That's enough to get it reviewed and approved. (Using a new CSR would just spread the behavior over two CSRs). Thanks, Roger On 6/13/18 4:56 PM, Brian Burkhalter wrote: > Hi Patrick, > > Not part of your change, but I noticed that at line 66 of Reader.java there is an extra parenthesis after ready(). > > In the test, the bug ID at line 39 could simply be appended to line 38. > > Otherwise looks good although I suppose given the specification update you?ll need an approved CSR before checking it in. > > Thanks, > > Brian > > On Jun 13, 2018, at 1:22 PM, Patrick Reinhart wrote: > >> While looking into the current Reader Spec and the failing methods I found that the ready() method actually does behave wrong and I fixed this. >> >> For the case of mark() I would like to revise the specification to align with the Reader?s default behavior that states for the mark method: >> >> IOException - If the stream does not support mark(), or if some other I/O error occurs >> >> The new spec for those methods would then read like: >> >> 70 * The {@code markSupported()} method returns {@code false}. The >> 71 * {@code mark()} and {@code reset()} methods throw an {@code IOException}. >> >> >> Here the link to the webrev: >> >> http://cr.openjdk.java.net/~reinhapa/reviews/8204930/webrev From mandy.chung at oracle.com Wed Jun 13 21:10:03 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 13 Jun 2018 14:10:03 -0700 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <5B206A09.5000404@oracle.com> References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <82fdd320-f5fb-2c34-5bb8-b72f6718fef6@oracle.com> <5B206A09.5000404@oracle.com> Message-ID: Thanks for the explanation, Joe. I see the subtle difference on these terminologies. I'm okay with this patch. I like the other option better to remove @apiNote since the spec states that the duplicates may not be removed. Mandy On 6/12/18 5:49 PM, Joseph D. Darcy wrote: > On 6/11/2018 10:38 PM, mandy chung wrote: >> >> >> On 6/11/18 10:16 PM, David Holmes wrote: >>> Here is one further minor update from the CSR discussions: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v5-incr/src/java.base/share/classes/java/lang/Class.java.cdiff.html >> >> >> >> "This implementation" is fine, as used in many @implNote.? Any reason >> why it has to be changed to "reference implementation"? >> > > To summarize the concern there, the phrase "This implementation..." when > used elsewhere has a different meaning. > > Often the phrasing "This implementation..." or the more commonly used > "The default implementation..." is used for text that is part of the > contract of a method that can be overridden; that is, used to separate > out the contract that is independent of which class or interface > provides the implementation from the contract of a particular > implementation. > > One example from an API I work on occurs for the method > javax.lang.model.element.ElementVisitor.visitModule. The default method > defined in an interface states "The default implementation visits a > ModuleElement by calling visitUnknown..." and then various visitor > classes define their own behavior for this method while still being able > to @inheritDoc the general "visit a module element" contract of the > visitModule method. > > However, java.lang.Class is final so for a particular JDK version there > is only one implementation of the method in question. In that context > "This implementation" doesn't mean "the implementation in this > particular class or interface as opposed to the implementation in an a > more specific subtype" it means "the implemetnation for the final method > used in a particular JDK release." > > I think using the term "This implementation" in the latter context is > misleading so I suggested the alternative wording David sent out for > review. > > HTH, > > -Joe From brian.burkhalter at oracle.com Wed Jun 13 21:16:17 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 13 Jun 2018 14:16:17 -0700 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: References: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> Message-ID: <3A88FBD3-6C6E-4F38-B11E-CE2D6A251D52@oracle.com> Hi Roger, Thanks for pointing this out: simpler and cleaner. Brian On Jun 13, 2018, at 2:06 PM, Roger Riggs wrote: > For the CSR, the easiest path to clarify the spec is to withdraw the CSR, update the spec, > and add a note on the revised behavior. > > Then finalize the CSR again. That's enough to get it reviewed and approved. > > (Using a new CSR would just spread the behavior over two CSRs). From brian.goetz at oracle.com Wed Jun 13 21:40:58 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 13 Jun 2018 17:40:58 -0400 Subject: BiCollector In-Reply-To: References: Message-ID: <616ff86f-b4ea-df23-6707-6771fc475d68@oracle.com> I really like how BiCollector can be private, so all we'd have to expose is toBoth(), and the arguments of toBoth() are just ordinary collectors.? So no new types or abstractions; just a multiplexing. The use of Map.Entry as a pair surrogate is unfortunate -- its definitely not an Entry -- though I understand why you did this. I'm not sure if this is fatal or not for a JDK method, but it's pretty bad*.? (You could generalize to n-ary, returning a List or array (you can take a varargs of Collector), at the loss of sharp types for the results.) *Yes, I'm sure one can find precedent of this being done; this has no effect on whether it's bad. On 6/11/2018 8:39 AM, Peter Levart wrote: > Hi, > > Have you ever wanted to perform a collection of the same Stream into > two different targets using two Collectors? Say you wanted to collect > Map.Entry elements into two parallel lists, each of them containing > keys and values respectively. Or you wanted to collect elements into? > groups by some key, but also count them at the same time? Currently > this is not possible to do with a single Stream. You have to create > two identical streams, so you end up passing Supplier to other > methods instead of bare Stream. > > I created a little utility Collector implementation that serves the > purpose quite well: > > /** > ?* A {@link Collector} implementation taking two delegate Collector(s) > and producing result composed > ?* of two results produced by delegating collectors, wrapped in {@link > Map.Entry} object. > ?* > ?* @param the type of elements collected > ?* @param the type of 1st delegate collector collected result > ?* @param tye type of 2nd delegate collector collected result > ?*/ > public class BiCollector implements Collector Map.Entry, Map.Entry> { > ??? private final Collector keyCollector; > ??? private final Collector valCollector; > > ??? @SuppressWarnings("unchecked") > ??? public BiCollector(Collector keyCollector, Collector ?, V> valCollector) { > ??????? this.keyCollector = (Collector) > Objects.requireNonNull(keyCollector); > ??????? this.valCollector = (Collector) > Objects.requireNonNull(valCollector); > ??? } > > ??? @Override > ??? public Supplier> supplier() { > ??????? Supplier keySupplier = keyCollector.supplier(); > ??????? Supplier valSupplier = valCollector.supplier(); > ??????? return () -> new > AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); > ??? } > > ??? @Override > ??? public BiConsumer, T> accumulator() { > ??????? BiConsumer keyAccumulator = > keyCollector.accumulator(); > ??????? BiConsumer valAccumulator = > valCollector.accumulator(); > ??????? return (accumulation, t) -> { > ??????????? keyAccumulator.accept(accumulation.getKey(), t); > ??????????? valAccumulator.accept(accumulation.getValue(), t); > ??????? }; > ??? } > > ??? @Override > ??? public BinaryOperator> combiner() { > ??????? BinaryOperator keyCombiner = keyCollector.combiner(); > ??????? BinaryOperator valCombiner = valCollector.combiner(); > ??????? return (accumulation1, accumulation2) -> new > AbstractMap.SimpleImmutableEntry<>( > ??????????? keyCombiner.apply(accumulation1.getKey(), > accumulation2.getKey()), > ??????????? valCombiner.apply(accumulation1.getValue(), > accumulation2.getValue()) > ??????? ); > ??? } > > ??? @Override > ??? public Function, Map.Entry> > finisher() { > ??????? Function keyFinisher = keyCollector.finisher(); > ??????? Function valFinisher = valCollector.finisher(); > ??????? return accumulation -> new AbstractMap.SimpleImmutableEntry<>( > ??????????? keyFinisher.apply(accumulation.getKey()), > ??????????? valFinisher.apply(accumulation.getValue()) > ??????? ); > ??? } > > ??? @Override > ??? public Set characteristics() { > ??????? EnumSet intersection = > EnumSet.copyOf(keyCollector.characteristics()); > ??????? intersection.retainAll(valCollector.characteristics()); > ??????? return intersection; > ??? } > } > > > Do you think this class is general enough to be part of standard > Collectors repertoire? > > For example, accessed via factory method Collectors.toBoth(Collector > coll1, Collector coll2), bi-collection could then be coded simply as: > > ??????? Map map = ... > > ??????? Map.Entry, List> keys_values = > ??????????? map.entrySet() > ?????????????? .stream() > ?????????????? .collect( > ?????????????????? toBoth( > ?????????????????????? mapping(Map.Entry::getKey, toList()), > ?????????????????????? mapping(Map.Entry::getValue, toList()) > ?????????????????? ) > ?????????????? ); > > > ??????? Map.Entry, Long> histogram_count = > ??????????? ThreadLocalRandom > ??????????????? .current() > ??????????????? .ints(100, 0, 10) > ??????????????? .boxed() > ??????????????? .collect( > ??????????????????? toBoth( > ??????????????????????? groupingBy(Function.identity(), counting()), > ??????????????????????? counting() > ??????????????????? ) > ??????????????? ); > > > Regards, Peter > From joe.darcy at oracle.com Thu Jun 14 00:51:13 2018 From: joe.darcy at oracle.com (joe darcy) Date: Wed, 13 Jun 2018 17:51:13 -0700 Subject: JDK 11 RFR of JDK-8205003: Replace selected link tags with linkplain in java.lang.Class Message-ID: <0222299d-9540-4e99-501d-70d7e552aacf@oracle.com> Hello, Please review the small patch below to address ??? JDK-8205003: Replace selected link tags with linkplain in java.lang.Class Thanks, -Joe diff -r 0742a087710e src/java.base/share/classes/java/lang/Class.java --- a/src/java.base/share/classes/java/lang/Class.java??? Wed Jun 13 13:12:50 2018 -0700 +++ b/src/java.base/share/classes/java/lang/Class.java??? Wed Jun 13 17:50:12 2018 -0700 @@ -820,7 +820,7 @@ ????? * primitive type or void, then the {@code Module} object for the ????? * {@code java.base} module is returned. ????? * -???? * If this class is in an unnamed module then the {@link +???? * If this class is in an unnamed module then the {@linkplain ????? * ClassLoader#getUnnamedModule() unnamed} {@code Module} of the class ????? * loader for this class is returned. ????? * @@ -953,14 +953,14 @@ ????? * empty string if the class is in an unnamed package. ????? * ????? * If this class is a member class, then this method is equivalent to -???? * invoking {@code getPackageName()} on the {@link #getEnclosingClass +???? * invoking {@code getPackageName()} on the {@linkplain #getEnclosingClass ????? * enclosing class}. ????? * -???? * If this class is a {@link #isLocalClass local class} or an {@link +???? * If this class is a {@linkplain #isLocalClass local class} or an {@linkplain ????? * #isAnonymousClass() anonymous class}, then this method is equivalent to -???? * invoking {@code getPackageName()} on the {@link #getDeclaringClass -???? * declaring class} of the {@link #getEnclosingMethod enclosing method} or -???? * {@link #getEnclosingConstructor enclosing constructor}. +???? * invoking {@code getPackageName()} on the {@linkplain #getDeclaringClass +???? * declaring class} of the {@linkplain #getEnclosingMethod enclosing method} or +???? * {@linkplain #getEnclosingConstructor enclosing constructor}. ????? * ????? * If this class represents an array type then this method returns the ????? * package name of the element type. If this class represents a primitive @@ -2576,7 +2576,7 @@ ????? * @param? name name of the desired resource ????? * @return? A {@link java.io.InputStream} object; {@code null} if no ????? *????????? resource with this name is found, the resource is in a package -???? *????????? that is not {@link Module#isOpen(String, Module) open} to at +???? *????????? that is not {@linkplain Module#isOpen(String, Module) open} to at ????? *????????? least the caller module, or access to the resource is denied ????? *????????? by the security manager. ????? * @throws? NullPointerException If {@code name} is {@code null} @@ -2675,7 +2675,7 @@ ????? * @return A {@link java.net.URL} object; {@code null} if no resource with ????? *???????? this name is found, the resource cannot be located by a URL, the ????? *???????? resource is in a package that is not -???? *???????? {@link Module#isOpen(String, Module) open} to at least the caller +???? *???????? {@linkplain Module#isOpen(String, Module) open} to at least the caller ????? *???????? module, or access to the resource is denied by the security ????? *???????? manager. ????? * @throws NullPointerException If {@code name} is {@code null} From brian.burkhalter at oracle.com Thu Jun 14 00:56:25 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 13 Jun 2018 17:56:25 -0700 Subject: JDK 11 RFR of JDK-8205003: Replace selected link tags with linkplain in java.lang.Class In-Reply-To: <0222299d-9540-4e99-501d-70d7e552aacf@oracle.com> References: <0222299d-9540-4e99-501d-70d7e552aacf@oracle.com> Message-ID: Hi Joe, +1 Brian On Jun 13, 2018, at 5:51 PM, joe darcy wrote: > Please review the small patch below to address > > JDK-8205003: Replace selected link tags with linkplain in java.lang.Class From mandy.chung at oracle.com Thu Jun 14 01:02:23 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 13 Jun 2018 18:02:23 -0700 Subject: JDK 11 RFR of JDK-8205003: Replace selected link tags with linkplain in java.lang.Class In-Reply-To: <0222299d-9540-4e99-501d-70d7e552aacf@oracle.com> References: <0222299d-9540-4e99-501d-70d7e552aacf@oracle.com> Message-ID: +1 Mandy On 6/13/18 5:51 PM, joe darcy wrote: > Hello, > > Please review the small patch below to address > > ??? JDK-8205003: Replace selected link tags with linkplain in > java.lang.Class > > Thanks, > > -Joe > > diff -r 0742a087710e src/java.base/share/classes/java/lang/Class.java > --- a/src/java.base/share/classes/java/lang/Class.java??? Wed Jun 13 > 13:12:50 2018 -0700 > +++ b/src/java.base/share/classes/java/lang/Class.java??? Wed Jun 13 > 17:50:12 2018 -0700 > @@ -820,7 +820,7 @@ > ????? * primitive type or void, then the {@code Module} object for the > ????? * {@code java.base} module is returned. > ????? * > -???? * If this class is in an unnamed module then the {@link > +???? * If this class is in an unnamed module then the {@linkplain > ????? * ClassLoader#getUnnamedModule() unnamed} {@code Module} of the > class > ????? * loader for this class is returned. > ????? * > @@ -953,14 +953,14 @@ > ????? * empty string if the class is in an unnamed package. > ????? * > ????? * If this class is a member class, then this method is > equivalent to > -???? * invoking {@code getPackageName()} on the {@link #getEnclosingClass > +???? * invoking {@code getPackageName()} on the {@linkplain > #getEnclosingClass > ????? * enclosing class}. > ????? * > -???? * If this class is a {@link #isLocalClass local class} or an > {@link > +???? * If this class is a {@linkplain #isLocalClass local class} or > an {@linkplain > ????? * #isAnonymousClass() anonymous class}, then this method is > equivalent to > -???? * invoking {@code getPackageName()} on the {@link #getDeclaringClass > -???? * declaring class} of the {@link #getEnclosingMethod enclosing > method} or > -???? * {@link #getEnclosingConstructor enclosing constructor}. > +???? * invoking {@code getPackageName()} on the {@linkplain > #getDeclaringClass > +???? * declaring class} of the {@linkplain #getEnclosingMethod > enclosing method} or > +???? * {@linkplain #getEnclosingConstructor enclosing constructor}. > ????? * > ????? * If this class represents an array type then this method > returns the > ????? * package name of the element type. If this class represents a > primitive > @@ -2576,7 +2576,7 @@ > ????? * @param? name name of the desired resource > ????? * @return? A {@link java.io.InputStream} object; {@code null} if no > ????? *????????? resource with this name is found, the resource is in a > package > -???? *????????? that is not {@link Module#isOpen(String, Module) open} > to at > +???? *????????? that is not {@linkplain Module#isOpen(String, Module) > open} to at > ????? *????????? least the caller module, or access to the resource is > denied > ????? *????????? by the security manager. > ????? * @throws? NullPointerException If {@code name} is {@code null} > @@ -2675,7 +2675,7 @@ > ????? * @return A {@link java.net.URL} object; {@code null} if no > resource with > ????? *???????? this name is found, the resource cannot be located by a > URL, the > ????? *???????? resource is in a package that is not > -???? *???????? {@link Module#isOpen(String, Module) open} to at least > the caller > +???? *???????? {@linkplain Module#isOpen(String, Module) open} to at > least the caller > ????? *???????? module, or access to the resource is denied by the > security > ????? *???????? manager. > ????? * @throws NullPointerException If {@code name} is {@code null} > From Lance.Andersen at oracle.com Thu Jun 14 01:45:32 2018 From: Lance.Andersen at oracle.com (Lance Andersen) Date: Wed, 13 Jun 2018 21:45:32 -0400 Subject: JDK 11 RFR of JDK-8205003: Replace selected link tags with linkplain in java.lang.Class In-Reply-To: <0222299d-9540-4e99-501d-70d7e552aacf@oracle.com> References: <0222299d-9540-4e99-501d-70d7e552aacf@oracle.com> Message-ID: +1 -- Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPhone > On Jun 13, 2018, at 8:51 PM, joe darcy wrote: > > Hello, > > Please review the small patch below to address > > JDK-8205003: Replace selected link tags with linkplain in java.lang.Class > > Thanks, > > -Joe > > diff -r 0742a087710e src/java.base/share/classes/java/lang/Class.java > --- a/src/java.base/share/classes/java/lang/Class.java Wed Jun 13 13:12:50 2018 -0700 > +++ b/src/java.base/share/classes/java/lang/Class.java Wed Jun 13 17:50:12 2018 -0700 > @@ -820,7 +820,7 @@ > * primitive type or void, then the {@code Module} object for the > * {@code java.base} module is returned. > * > - * If this class is in an unnamed module then the {@link > + * If this class is in an unnamed module then the {@linkplain > * ClassLoader#getUnnamedModule() unnamed} {@code Module} of the class > * loader for this class is returned. > * > @@ -953,14 +953,14 @@ > * empty string if the class is in an unnamed package. > * > * If this class is a member class, then this method is equivalent to > - * invoking {@code getPackageName()} on the {@link #getEnclosingClass > + * invoking {@code getPackageName()} on the {@linkplain #getEnclosingClass > * enclosing class}. > * > - * If this class is a {@link #isLocalClass local class} or an {@link > + * If this class is a {@linkplain #isLocalClass local class} or an {@linkplain > * #isAnonymousClass() anonymous class}, then this method is equivalent to > - * invoking {@code getPackageName()} on the {@link #getDeclaringClass > - * declaring class} of the {@link #getEnclosingMethod enclosing method} or > - * {@link #getEnclosingConstructor enclosing constructor}. > + * invoking {@code getPackageName()} on the {@linkplain #getDeclaringClass > + * declaring class} of the {@linkplain #getEnclosingMethod enclosing method} or > + * {@linkplain #getEnclosingConstructor enclosing constructor}. > * > * If this class represents an array type then this method returns the > * package name of the element type. If this class represents a primitive > @@ -2576,7 +2576,7 @@ > * @param name name of the desired resource > * @return A {@link java.io.InputStream} object; {@code null} if no > * resource with this name is found, the resource is in a package > - * that is not {@link Module#isOpen(String, Module) open} to at > + * that is not {@linkplain Module#isOpen(String, Module) open} to at > * least the caller module, or access to the resource is denied > * by the security manager. > * @throws NullPointerException If {@code name} is {@code null} > @@ -2675,7 +2675,7 @@ > * @return A {@link java.net.URL} object; {@code null} if no resource with > * this name is found, the resource cannot be located by a URL, the > * resource is in a package that is not > - * {@link Module#isOpen(String, Module) open} to at least the caller > + * {@linkplain Module#isOpen(String, Module) open} to at least the caller > * module, or access to the resource is denied by the security > * manager. > * @throws NullPointerException If {@code name} is {@code null} > From weijun.wang at oracle.com Thu Jun 14 01:48:33 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Thu, 14 Jun 2018 09:48:33 +0800 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> Message-ID: <3733DE65-8E74-465E-9BE2-CCB1D236CC96@oracle.com> Thanks a lot for the explanation. Everything looks OK to me now. --Max > On Jun 13, 2018, at 10:10 PM, Roger Riggs wrote: > > Hi Joe, > > The CSR text is still in draft and got out of sync with the open review and comments. > > The updated apiNote clarifies that changes made through the System.*property* methods > may not affect the value cached during initialization. > The implementation changes affect a very short list of properties. > > The note on the methods is to raise awareness that individual properties may have > different behavior and may be unpredictable. > > On 6/12/2018 10:10 PM, Weijun Wang wrote: >> In fact, why is setting user.name and user.home always evil? If I only want to set them on the command line so that a special "user environment" is used, why is it a problem? > There is no change to the ability to set properties on the command line. > The concern is with the predictability of (some) properties changing dynamically while the application > is running. > > The property user.home is used for mime-types and mailcap files and user.name is used in some > networking cases where a username is needed. >> >> In fact, we have a test setting user.home to an empty directory to avoid unexpected result because we cannot control the test runner's home directory. >> > Right, there is no problem with that > > Regards, Roger >> >> Thanks >> Max >> >> >>> On Jun 13, 2018, at 9:59 AM, Weijun Wang >>> wrote: >>> >>> Hi Roger >>> >>> 1. Should all occurrences of reading of these system properties be updated? For example, the following one is not touched >>> >>> http://hg.openjdk.java.net/jdk/jdk/file/4d2e3f5abb48/src/java.base/share/classes/sun/security/tools/keytool/Main.java#l842 > That usage was in a tool and I did not modify any tools. >>> >>> 2. I assume that with this change not only there is no use calling System.setProperty() in the application but also setting it on the command line is now useless. Is this right? Do we need to make this clear in the CSR? >>> > No change to command line setting of properties. > > Roger > >>> >>> Thanks >>> Max >>> >>> >>> >>>> On Jun 4, 2018, at 9:32 PM, Roger Riggs >>>> wrote: >>>> >>>> Please review a change to make the values of java.home, user.home, user.dir, and user.name >>>> effectively read-only for internal use. The values are cached during initialization and the >>>> cached values are used. >>>> >>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ >>>> >>>> >>>> Issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8066709 >>>> >>>> >>>> CSR: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8204235 >>>> >>>> >>>> Thanks, Roger >>>> >>>> >>>> > From bhamaram at in.ibm.com Thu Jun 14 03:23:44 2018 From: bhamaram at in.ibm.com (Bhaktavatsal R Maram) Date: Thu, 14 Jun 2018 03:23:44 +0000 Subject: RFR(XS): 8202329 [AIX] Fix codepage mappings for IBM-943 and Big5 In-Reply-To: References: , , <5331ad7ba3c63a066895d41a91bd9593@linux.vnet.ibm.com> Message-ID: Hi Thomas, I've added testcase along with fix. Please find webrev at http://cr.openjdk.java.net/~aleonard/8202329/webrev.01/ Thanks, Bhaktavatsal Reddy -----"core-libs-dev" wrote: ----- To: "Thomas St?fe" From: "Bhaktavatsal R Maram" Sent by: "core-libs-dev" Date: 06/01/2018 01:34PM Cc: Java Core Libs Subject: Re: RFR(XS): 8202329 [AIX] Fix codepage mappings for IBM-943 and Big5 Hi Thomas, Sorry, I was on vacation. I will submit webrev with jtreg testcase. Thanks, Bhaktavatsal -----"Thomas St?fe" wrote: ----- To: Bhaktavatsal R Maram From: "Thomas St?fe" Date: 05/23/2018 12:32AM Cc: Ichiroh Takiguchi , Volker Simonis , Java Core Libs Subject: Re: RFR(XS): 8202329 [AIX] Fix codepage mappings for IBM-943 and Big5 Hi Bhaktavatsal, the fix is fine. Reviewed. Thanks a lot for your work! Could you please (its okay in a future patch) add a regression test for IBM943 and IBM943C, in the form of a jtreg test? I would like to test conversions for both code pages (at least for the crucial tilde/backslash code points). Additionally, that when started on AIX with Ja_JP.IBM-943, we correctly default to IBM943C. As an example, here is an old test I wrote years ago. You won't be able to use it verbatim, so it is just a suggestion. Feel free to do it differently. http://cr.openjdk.java.net/~stuefe/other/IBM943MappingTest.java If you have not written jtreg tests before: http://openjdk.java.net/jtreg/ Also, under /test are many many examples. Best Regards, Thomas On Mon, May 21, 2018 at 9:47 AM, Bhaktavatsal R Maram wrote: > Hi Thomas, > > Is this fix ready to merge? > > Thanks, > Bhaktavatsal > > > > > -----"Thomas St?fe" wrote: ----- > To: Ichiroh Takiguchi , Bhaktavatsal R Maram > From: "Thomas St?fe" > Date: 05/11/2018 11:49AM > Cc: Volker Simonis , Java Core Libs > Subject: Re: RFR(XS): 8202329 [AIX] Fix codepage mappings for IBM-943 and Big5 > > Hi, > > I'll test and review next week. We also have some in-house tests which > I'd like to run. > > You IBM folks should really apply for authorship so that this > contribution process gets streamlined. After all, if something breaks > in this code, you want to be able to fix it, yes? So even if you do > not contribute much else, more patches may be forthcoming. > > Of course I hope these are not your last contributions :) > > Best, Thomas > > > > On Fri, May 11, 2018 at 7:57 AM, Ichiroh Takiguchi > wrote: >> Hi. >> >> I tested this fix on AIX. >> >> I got following results. >> \$ LANG=Ja_JP ~/jdk/bin/java PrintDefaultCharset >> Ja_JP x-IBM943C IBM-943C IBM-943C >> \$ LANG=Ja_JP.IBM-943 ~/jdk/bin/java PrintDefaultCharset >> Ja_JP.IBM-943 x-IBM943C IBM-943C IBM-943C >> \$ LANG=Zh_TW ~/jdk/bin/java PrintDefaultCharset >> Zh_TW x-IBM950 IBM-950 IBM-950 >> \$ LANG=Zh_TW.big5 ~/jdk/bin/java PrintDefaultCharset >> Zh_TW.big5 x-IBM950 IBM-950 IBM-950 >> >> Also I reviewed source code, it's fine >> >> Since this testing requires locale installation for Ja_JP and Zh_TW, >> so it's not easy to test it... >> (At least, I think bos.loc.pc.Ja_JP and bos.loc.iso.Zh_TW filesets are >> required) >> >> >> On 2018-05-02 18:32, Volker Simonis wrote: >>> >>> Hi Bhaktavatsal Reddy, >>> >>> your change looks good. I can sponsor it. >>> >>> Just waiting for a second review... >>> >>> Thank you and best regards, >>> Volker >>> >>> >>> On Mon, Apr 30, 2018 at 11:29 AM, Bhaktavatsal R Maram >>> wrote: >>>> >>>> Hi All, >>>> >>>> Please review the fix. >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8202329 >>>> webrev: http://cr.openjdk.java.net/~aleonard/8202329/webrev.00/ >>>> >>>> Thanks, >>>> Bhaktavatsal Reddy >>>> >>>> -----"core-libs-dev" wrote: >>>> ----- >>>> To: Volker Simonis >>>> From: "Bhaktavatsal R Maram" >>>> Sent by: "core-libs-dev" >>>> Date: 04/26/2018 09:31PM >>>> Cc: Java Core Libs >>>> Subject: Re: [AIX] Fix codepage mappings in Java for IBM-943 and Big5 >>>> >>>> Hi Volker, >>>> >>>> Thank you. I will address your review comments and send webrev for >>>> review. >>>> >>>> - Bhaktavatsal Reddy >>>> >>>> >>>> >>>> -----Volker Simonis wrote: ----- >>>> To: Bhaktavatsal R Maram >>>> From: Volker Simonis >>>> Date: 04/26/2018 09:12PM >>>> Cc: Java Core Libs >>>> Subject: Re: [AIX] Fix codepage mappings in Java for IBM-943 and Big5 >>>> >>>> Hi Bhaktavatsal Reddy, >>>> >>>> I've opened the following issue for this problem: >>>> >>>> 8202329: [AIX] Fix codepage mappings for IBM-943 and Big5 >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8202329 >>>> >>>> Looking at you fix, can you please replace the "#elif AIX" by "#ifdef >>>> AIX" and the original "#else" by "#ifdef __solaris__". The original >>>> else branch contains Solaris-only code anyway and it is an historical >>>> omission that there are still a lot of places in the code where "not >>>> Linux" implicitly means "Solaris", but that's often wrong. >>>> >>>> Regards, >>>> Volker >>>> >>>> >>>> On Thu, Apr 26, 2018 at 4:02 PM, Bhaktavatsal R Maram >>>> wrote: >>>>> >>>>> Oops! Looks like there is problem with attachment (might be because I >>>>> attached .class file as well). I'm pasting the fix and test program here in >>>>> mail. >>>>> >>>>> Test Program: >>>>> >>>>> import java.nio.charset.*; >>>>> class PrintDefaultCharset { >>>>> public static void main(String[] args) { >>>>> System.out.println("LANG = "+System.getenv("LANG")); >>>>> System.out.println("Default charset = >>>>> "+Charset.defaultCharset().name()); >>>>> System.out.println("file.encoding = >>>>> "+System.getProperty("file.encoding")); >>>>> System.out.println("sun.jnu.encoding = >>>>> "+System.getProperty("sun.jnu.encoding")); >>>>> } >>>>> } >>>>> >>>>> >>>>> Fix: >>>>> >>>>> diff --git a/src/java.base/unix/native/libjava/java_props_md.c >>>>> b/src/java.base/unix/native/libjava/java_props_md.c >>>>> --- a/src/java.base/unix/native/libjava/java_props_md.c >>>>> +++ b/src/java.base/unix/native/libjava/java_props_md.c >>>>> @@ -1,5 +1,5 @@ >>>>> /* >>>>> - * Copyright (c) 1998, 2016, Oracle and/or its affiliates. All rights >>>>> reserved. >>>>> + * Copyright (c) 1998, 2018, Oracle and/or its affiliates. All rights >>>>> reserved. >>>>> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>>>> * >>>>> * This code is free software; you can redistribute it and/or modify it >>>>> @@ -297,6 +297,18 @@ >>>>> if (strcmp(p, "EUC-JP") == 0) { >>>>> *std_encoding = "EUC-JP-LINUX"; >>>>> } >>>>> +#elif defined _AIX >>>>> + if (strcmp(p, "big5") == 0) { >>>>> + /* On AIX Traditional Chinese Big5 codeset is mapped to >>>>> IBM-950 */ >>>>> + *std_encoding = "IBM-950"; >>>>> + } else if (strcmp(p, "IBM-943") == 0) { >>>>> + /* >>>>> + * On AIX, IBM-943 is mapped to IBM-943C in which symbol >>>>> 'yen' and >>>>> + * 'overline' are replaced with 'backslash' and 'tilde' >>>>> from ASCII >>>>> + * making first 96 code points same as ASCII. >>>>> + */ >>>>> + *std_encoding = "IBM-943C"; >>>>> + } >>>>> #else >>>>> if (strcmp(p,"eucJP") == 0) { >>>>> /* For Solaris use customized vendor defined character >>>>> >>>>> >>>>> Thanks, >>>>> Bhaktavatsal Reddy >>>>> >>>>> >>>>> -----"core-libs-dev" wrote: >>>>> ----- >>>>> To: "Java Core Libs" >>>>> From: "Bhaktavatsal R Maram" >>>>> Sent by: "core-libs-dev" >>>>> Date: 04/26/2018 07:26PM >>>>> Subject: [AIX] Fix codepage mappings in Java for IBM-943 and Big5 >>>>> >>>>> Hi All, >>>>> >>>>> This issue is continuation to bug 8201540 (Extend the set of supported >>>>> charsets in java.base on AIX) in which we have moved default charsets of >>>>> most of the locales supported by Operating System to java.base module thus >>>>> enabling OpenJDK on those locales for AIX platform. >>>>> >>>>> As part of that, charsets for locales Ja_JP (IBM-943) and Zh_TW (big5) >>>>> also have been moved. However, corresponding charsets mapped in Java is not >>>>> correct for them on AIX. Following are the details: >>>>> >>>>> 1. IBM-943 [1] for locale Ja_JP should be mapped to IBM-943C [2] >>>>> >>>>> Fundamental difference between IBM-943 and IBM-943C is that IBM-943C is >>>>> ASCII compatible which means code points 'yen' and 'overline' of IBM-943 is >>>>> replaced with 'backslash' and 'tilde' from ASCII character set. >>>>> >>>>> >>>>> 2. Big5 for locale Zh_TW should be mapped to IBM-950 [3] >>>>> >>>>> I've attached simple test program to print the default charset along >>>>> with fix for this issue. When run test program (PrintDefaultCharset) with >>>>> IBM JDK 8 (on AIX) for locales Ja_JP & Zh_TW, following is output. >>>>> >>>>> -bash-4.4\$ LANG=Ja_JP ~/JDKs/IBM/80/ON/sdk/jre/bin/java >>>>> PrintDefaultCharset >>>>> LANG = Ja_JP >>>>> Default charset = x-IBM943C >>>>> file.encoding = IBM-943C >>>>> sun.jnu.encoding = IBM-943C >>>>> >>>>> -bash-4.4\$ LANG=Zh_TW ~/JDKs/IBM/80/ON/sdk/jre/bin/java >>>>> PrintDefaultCharset >>>>> LANG = Zh_TW >>>>> Default charset = x-IBM950 >>>>> file.encoding = IBM-950 >>>>> sun.jnu.encoding = IBM-950 >>>>> >>>>> >>>>> Same test run with openJDK 11 gives following output >>>>> >>>>> -bash-4.4\$ LANG=Ja_JP ~/jdk/bin/java PrintDefaultCharset >>>>> LANG = Ja_JP >>>>> Default charset = x-IBM943 >>>>> file.encoding = IBM-943 >>>>> sun.jnu.encoding = IBM-943 >>>>> >>>>> -bash-4.4\$ LANG=Zh_TW ~/jdk/bin/java PrintDefaultCharset >>>>> LANG = Zh_TW >>>>> Default charset = Big5 >>>>> file.encoding = big5 >>>>> sun.jnu.encoding = big5 >>>>> >>>>> I will get webrev hosted in >>>>> http://cr.openjdk.java.net >>>>> for this change and send it for review once JIRA bug is created. >>>>> >>>>> [1] >>>>> http://demo.icu-project.org/icu-bin/convexp?conv=ibm-943_P130-1999&s=JAVA >>>>> [2] >>>>> http://demo.icu-project.org/icu-bin/convexp?conv=ibm-943_P15A-2003&s=ALL >>>>> [3] >>>>> https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm.aix.nlsgdrf/big5.htm >>>>> >>>>> >>>>> Thanks, >>>>> Bhaktavatsal Reddy >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >> > > From david.holmes at oracle.com Thu Jun 14 03:29:27 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Jun 2018 13:29:27 +1000 Subject: [core-libs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <9cfdde9b-5929-a5b0-ea26-e79a6d4f56e6@oracle.com> References: <9db804f6-de0a-62bb-30f2-cbc3fbfda289@oracle.com> <10226439-2f83-c1d8-f27f-353e739e062c@oracle.com> <9cfdde9b-5929-a5b0-ea26-e79a6d4f56e6@oracle.com> Message-ID: On 13/06/2018 4:08 PM, joe darcy wrote: > Hi David, > > On 5/24/2018 10:52 PM, David Holmes wrote: >> Here are the further minor updates so far in response to all the >> review comments. >> >> Incremental corelibs webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3-incr/ >> >> >> Full corelibs webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.corelibs.v3/ >> > > In Class.java, > > 3990???????? Class[] members = getNestMembers0(); > 3991???????? // Can't actually enable this due to bootstrapping issues > 3992???????? // assert(members.length != 1 || members[0] == this); // > expected invariant from VM > > can these checks be enabled unconditionally without using the assert > facility, throwing AssertionError or some other kind of error? Of course - but we don't want to pay the price of always checking something that would indicate an error on the VM side. There's an equivalent assertion on the VM side. Thanks, David > Thanks, > > -Joe From amaembo at gmail.com Thu Jun 14 04:56:25 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Thu, 14 Jun 2018 11:56:25 +0700 Subject: BiCollector In-Reply-To: <616ff86f-b4ea-df23-6707-6771fc475d68@oracle.com> References: <616ff86f-b4ea-df23-6707-6771fc475d68@oracle.com> Message-ID: Hello! In my StreamEx library I created a "pairing" collector which does similar job, but allows user to decide how to combine the results of two collectors. This adds more flexibility. The signature is like this: public static Collector pairing( Collector c1, Collector c2, BiFunction finisher) Having such collector, The proposed `toBoth(c1, c2)` can be implemented as simple as `pairing(c1, c2, Map::entry)`. OTOH if somebody wants to use their own Pair class, it would be `pairing(c1, c2, Pair::new)`. Sometimes you don't need a pair, but can create compound result object right here. E.g.: Collector summing = Collectors.reducing(BigDecimal.ZERO, BigDecimal::add); Collector averaging = pairing(summing, Collectors.counting(), (sum, count) -> sum.divide(BigDecimal.valueOf(count), RoundingMode.HALF_EVEN)); So locking to Map.Entry is entirely unnecessary. With best regards, Tagir Valeev. On Thu, Jun 14, 2018 at 6:11 AM Brian Goetz wrote: > I really like how BiCollector can be private, so all we'd have to expose > is toBoth(), and the arguments of toBoth() are just ordinary > collectors. So no new types or abstractions; just a multiplexing. > > The use of Map.Entry as a pair surrogate is unfortunate -- its > definitely not an Entry -- though I understand why you did this. I'm not > sure if this is fatal or not for a JDK method, but it's pretty bad*. > (You could generalize to n-ary, returning a List or array (you can take > a varargs of Collector), at the loss of sharp types for the results.) > > > *Yes, I'm sure one can find precedent of this being done; this has no > effect on whether it's bad. > > On 6/11/2018 8:39 AM, Peter Levart wrote: > > Hi, > > > > Have you ever wanted to perform a collection of the same Stream into > > two different targets using two Collectors? Say you wanted to collect > > Map.Entry elements into two parallel lists, each of them containing > > keys and values respectively. Or you wanted to collect elements into > > groups by some key, but also count them at the same time? Currently > > this is not possible to do with a single Stream. You have to create > > two identical streams, so you end up passing Supplier to other > > methods instead of bare Stream. > > > > I created a little utility Collector implementation that serves the > > purpose quite well: > > > > /** > > * A {@link Collector} implementation taking two delegate Collector(s) > > and producing result composed > > * of two results produced by delegating collectors, wrapped in {@link > > Map.Entry} object. > > * > > * @param the type of elements collected > > * @param the type of 1st delegate collector collected result > > * @param tye type of 2nd delegate collector collected result > > */ > > public class BiCollector implements Collector > Map.Entry, Map.Entry> { > > private final Collector keyCollector; > > private final Collector valCollector; > > > > @SuppressWarnings("unchecked") > > public BiCollector(Collector keyCollector, Collector > ?, V> valCollector) { > > this.keyCollector = (Collector) > > Objects.requireNonNull(keyCollector); > > this.valCollector = (Collector) > > Objects.requireNonNull(valCollector); > > } > > > > @Override > > public Supplier> supplier() { > > Supplier keySupplier = keyCollector.supplier(); > > Supplier valSupplier = valCollector.supplier(); > > return () -> new > > AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); > > } > > > > @Override > > public BiConsumer, T> accumulator() { > > BiConsumer keyAccumulator = > > keyCollector.accumulator(); > > BiConsumer valAccumulator = > > valCollector.accumulator(); > > return (accumulation, t) -> { > > keyAccumulator.accept(accumulation.getKey(), t); > > valAccumulator.accept(accumulation.getValue(), t); > > }; > > } > > > > @Override > > public BinaryOperator> combiner() { > > BinaryOperator keyCombiner = keyCollector.combiner(); > > BinaryOperator valCombiner = valCollector.combiner(); > > return (accumulation1, accumulation2) -> new > > AbstractMap.SimpleImmutableEntry<>( > > keyCombiner.apply(accumulation1.getKey(), > > accumulation2.getKey()), > > valCombiner.apply(accumulation1.getValue(), > > accumulation2.getValue()) > > ); > > } > > > > @Override > > public Function, Map.Entry> > > finisher() { > > Function keyFinisher = keyCollector.finisher(); > > Function valFinisher = valCollector.finisher(); > > return accumulation -> new AbstractMap.SimpleImmutableEntry<>( > > keyFinisher.apply(accumulation.getKey()), > > valFinisher.apply(accumulation.getValue()) > > ); > > } > > > > @Override > > public Set characteristics() { > > EnumSet intersection = > > EnumSet.copyOf(keyCollector.characteristics()); > > intersection.retainAll(valCollector.characteristics()); > > return intersection; > > } > > } > > > > > > Do you think this class is general enough to be part of standard > > Collectors repertoire? > > > > For example, accessed via factory method Collectors.toBoth(Collector > > coll1, Collector coll2), bi-collection could then be coded simply as: > > > > Map map = ... > > > > Map.Entry, List> keys_values = > > map.entrySet() > > .stream() > > .collect( > > toBoth( > > mapping(Map.Entry::getKey, toList()), > > mapping(Map.Entry::getValue, toList()) > > ) > > ); > > > > > > Map.Entry, Long> histogram_count = > > ThreadLocalRandom > > .current() > > .ints(100, 0, 10) > > .boxed() > > .collect( > > toBoth( > > groupingBy(Function.identity(), counting()), > > counting() > > ) > > ); > > > > > > Regards, Peter > > > > From peter.levart at gmail.com Thu Jun 14 06:53:50 2018 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 14 Jun 2018 08:53:50 +0200 Subject: BiCollector In-Reply-To: References: <616ff86f-b4ea-df23-6707-6771fc475d68@oracle.com> Message-ID: <6890a641-6440-6cac-3c4d-80e0348b6cf5@gmail.com> Hi Tagir, Great! I've been thinking about the same trick - just add another function and you don't have to hard code the decision about the choice of imperfect tuple type. But are we just talking of aestetics here? In the absence of value types, every choice is imperfect. Ok, sometimes you may have luck to be able to further reduce the result into a singular type, but how often does that occur? I'm not so concerned about the choice of tuple type but about another aspect. The characteristics of a combined collector can only be an intersection of characteristics of individual collectors (minus IDENTITY_FINISH, when you provide the finisher/combiner of the results). If one of the collectors is CONCURRENT, but the other is not, then both will be used in non-CONCURRENT collection mode, which might be sub-optimal. But such is the nature of "single pass". You can't travel to the destination via two distinct paths at the same time (if you're not doing quantum superposition :-). I was positively surprised that such composition is even possible in such a simple way, which speaks of the well thought out initial design of the API. So I guess that although a very useful tool in practice, pairing/bi-collection will remain in the domain of custom Stream extensions. Or maybe not? Regards, Peter On 06/14/18 06:56, Tagir Valeev wrote: > Hello! > > In my StreamEx library I created a "pairing" collector which does > similar job, but allows user to decide how to combine the results of > two collectors. This adds more flexibility. The signature is like this: > > public static Collector pairing( > ? ? Collector c1, > ? ? Collector c2, > ? ? BiFunction finisher) > > Having such collector, The proposed `toBoth(c1, c2)` can be > implemented as simple as `pairing(c1, c2, Map::entry)`. OTOH if > somebody wants to use their own Pair class, it would be `pairing(c1, > c2, Pair::new)`. > Sometimes you don't need a pair, but can create compound result object > right here. E.g.: > > Collector summing = > Collectors.reducing(BigDecimal.ZERO, BigDecimal::add); > Collector averaging = > ? ? ? ? ? ? pairing(summing, Collectors.counting(), (sum, count) -> > ? ? ? ? ? ? ? ? ? ? sum.divide(BigDecimal.valueOf(count), > RoundingMode.HALF_EVEN)); > > So locking to Map.Entry is entirely unnecessary. > > With best regards, > Tagir Valeev. > > > > On Thu, Jun 14, 2018 at 6:11 AM Brian Goetz > wrote: > > I really like how BiCollector can be private, so all we'd have to > expose > is toBoth(), and the arguments of toBoth() are just ordinary > collectors.? So no new types or abstractions; just a multiplexing. > > The use of Map.Entry as a pair surrogate is unfortunate -- its > definitely not an Entry -- though I understand why you did this. > I'm not > sure if this is fatal or not for a JDK method, but it's pretty bad*. > (You could generalize to n-ary, returning a List or array (you can > take > a varargs of Collector), at the loss of sharp types for the results.) > > > *Yes, I'm sure one can find precedent of this being done; this has no > effect on whether it's bad. > > On 6/11/2018 8:39 AM, Peter Levart wrote: > > Hi, > > > > Have you ever wanted to perform a collection of the same Stream > into > > two different targets using two Collectors? Say you wanted to > collect > > Map.Entry elements into two parallel lists, each of them containing > > keys and values respectively. Or you wanted to collect elements > into > > groups by some key, but also count them at the same time? Currently > > this is not possible to do with a single Stream. You have to create > > two identical streams, so you end up passing Supplier to > other > > methods instead of bare Stream. > > > > I created a little utility Collector implementation that serves the > > purpose quite well: > > > > /** > > ?* A {@link Collector} implementation taking two delegate > Collector(s) > > and producing result composed > > ?* of two results produced by delegating collectors, wrapped in > {@link > > Map.Entry} object. > > ?* > > ?* @param the type of elements collected > > ?* @param the type of 1st delegate collector collected result > > ?* @param tye type of 2nd delegate collector collected result > > ?*/ > > public class BiCollector implements Collector > Map.Entry, Map.Entry> { > > ??? private final Collector keyCollector; > > ??? private final Collector valCollector; > > > > ??? @SuppressWarnings("unchecked") > > ??? public BiCollector(Collector keyCollector, > Collector > ?, V> valCollector) { > > ??????? this.keyCollector = (Collector) > > Objects.requireNonNull(keyCollector); > > ??????? this.valCollector = (Collector) > > Objects.requireNonNull(valCollector); > > ??? } > > > > ??? @Override > > ??? public Supplier> supplier() { > > ??????? Supplier keySupplier = keyCollector.supplier(); > > ??????? Supplier valSupplier = valCollector.supplier(); > > ??????? return () -> new > > AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), > valSupplier.get()); > > ??? } > > > > ??? @Override > > ??? public BiConsumer, T> accumulator() { > > ??????? BiConsumer keyAccumulator = > > keyCollector.accumulator(); > > ??????? BiConsumer valAccumulator = > > valCollector.accumulator(); > > ??????? return (accumulation, t) -> { > > ??????????? keyAccumulator.accept(accumulation.getKey(), t); > > valAccumulator.accept(accumulation.getValue(), t); > > ??????? }; > > ??? } > > > > ??? @Override > > ??? public BinaryOperator> combiner() { > > ??????? BinaryOperator keyCombiner = > keyCollector.combiner(); > > ??????? BinaryOperator valCombiner = > valCollector.combiner(); > > ??????? return (accumulation1, accumulation2) -> new > > AbstractMap.SimpleImmutableEntry<>( > > ??????????? keyCombiner.apply(accumulation1.getKey(), > > accumulation2.getKey()), > > ??????????? valCombiner.apply(accumulation1.getValue(), > > accumulation2.getValue()) > > ??????? ); > > ??? } > > > > ??? @Override > > ??? public Function, Map.Entry> > > finisher() { > > ??????? Function keyFinisher = keyCollector.finisher(); > > ??????? Function valFinisher = valCollector.finisher(); > > ??????? return accumulation -> new > AbstractMap.SimpleImmutableEntry<>( > > ??????????? keyFinisher.apply(accumulation.getKey()), > > ??????????? valFinisher.apply(accumulation.getValue()) > > ??????? ); > > ??? } > > > > ??? @Override > > ??? public Set characteristics() { > > ??????? EnumSet intersection = > > EnumSet.copyOf(keyCollector.characteristics()); > > intersection.retainAll(valCollector.characteristics()); > > ??????? return intersection; > > ??? } > > } > > > > > > Do you think this class is general enough to be part of standard > > Collectors repertoire? > > > > For example, accessed via factory method > Collectors.toBoth(Collector > > coll1, Collector coll2), bi-collection could then be coded > simply as: > > > > ??????? Map map = ... > > > > ??????? Map.Entry, List> keys_values = > > ??????????? map.entrySet() > > ?????????????? .stream() > > ?????????????? .collect( > > ?????????????????? toBoth( > > ?????????????????????? mapping(Map.Entry::getKey, toList()), > > ?????????????????????? mapping(Map.Entry::getValue, toList()) > > ?????????????????? ) > > ?????????????? ); > > > > > > ??????? Map.Entry, Long> histogram_count = > > ??????????? ThreadLocalRandom > > ??????????????? .current() > > ??????????????? .ints(100, 0, 10) > > ??????????????? .boxed() > > ??????????????? .collect( > > ??????????????????? toBoth( > > ??????????????????????? groupingBy(Function.identity(), counting()), > > ??????????????????????? counting() > > ??????????????????? ) > > ??????????????? ); > > > > > > Regards, Peter > > > From peter.levart at gmail.com Thu Jun 14 07:29:16 2018 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 14 Jun 2018 09:29:16 +0200 Subject: BiCollector In-Reply-To: References: Message-ID: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> Hi Paul, On 06/11/18 19:10, Paul Sandoz wrote: > Hi Peter, > > I like it and can see it being useful, thanks for sharing. > > I am hesitating a little about it being in the JDK because there is the larger abstraction of a BiStream, where a similar form of collection would naturally fit (but perhaps without the intersection constraints for the characteristics?). We experimented a few times with BiStream and got quite far but decided pull back due to the lack of value types and specialized generics. So i dunno how this might turn out in the future and if your BiCollector fits nicely into such a future model. > > What are you thoughts on this? Well, I don't see the need to pack the two results into a Map.Entry (or any similar) container as a drawback. It's not a performance drawback for sure, because this is not happening on the stream-element scale, but on the final result or intermediate accumulation results scale (the later only in parallel non-CONCURRENT scenario). In non-parallel scenario, only a single (or two for non-IDENTITY_FINISH) Map.Entry objects are created. I also don't see a larger abstraction like BiStream as a natural fit for a similar thing. As I understand BiStream attempts (maybe I haven't seen the right ones?), they are more about passing pairs of elements down the pipeline. BiCollector OTOH is "splitting" the single element pipeline at the final collection stage, with the purpose of constructing two independent collections from a single pass of the original single-element Stream. This is more about single pass than anything else. And single pass, I think, is inevitably such that it can only execute a single collection strategy (CONCURRENT vs. not CONCURRENT), regardless of the type of the stream (Stream vs. BiStream). Or have you prototyped a combined strategy in BiStream? Regards, Peter > FWIW i would call it a ?splitting? or ?bisecting" collector e.g. ?s.collect(bisecting(?))? > > Paul. > > > > >> On Jun 11, 2018, at 5:39 AM, Peter Levart wrote: >> >> Hi, >> >> Have you ever wanted to perform a collection of the same Stream into two different targets using two Collectors? Say you wanted to collect Map.Entry elements into two parallel lists, each of them containing keys and values respectively. Or you wanted to collect elements into groups by some key, but also count them at the same time? Currently this is not possible to do with a single Stream. You have to create two identical streams, so you end up passing Supplier to other methods instead of bare Stream. >> >> I created a little utility Collector implementation that serves the purpose quite well: >> >> /** >> * A {@link Collector} implementation taking two delegate Collector(s) and producing result composed >> * of two results produced by delegating collectors, wrapped in {@link Map.Entry} object. >> * >> * @param the type of elements collected >> * @param the type of 1st delegate collector collected result >> * @param tye type of 2nd delegate collector collected result >> */ >> public class BiCollector implements Collector, Map.Entry> { >> private final Collector keyCollector; >> private final Collector valCollector; >> >> @SuppressWarnings("unchecked") >> public BiCollector(Collector keyCollector, Collector valCollector) { >> this.keyCollector = (Collector) Objects.requireNonNull(keyCollector); >> this.valCollector = (Collector) Objects.requireNonNull(valCollector); >> } >> >> @Override >> public Supplier> supplier() { >> Supplier keySupplier = keyCollector.supplier(); >> Supplier valSupplier = valCollector.supplier(); >> return () -> new AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); >> } >> >> @Override >> public BiConsumer, T> accumulator() { >> BiConsumer keyAccumulator = keyCollector.accumulator(); >> BiConsumer valAccumulator = valCollector.accumulator(); >> return (accumulation, t) -> { >> keyAccumulator.accept(accumulation.getKey(), t); >> valAccumulator.accept(accumulation.getValue(), t); >> }; >> } >> >> @Override >> public BinaryOperator> combiner() { >> BinaryOperator keyCombiner = keyCollector.combiner(); >> BinaryOperator valCombiner = valCollector.combiner(); >> return (accumulation1, accumulation2) -> new AbstractMap.SimpleImmutableEntry<>( >> keyCombiner.apply(accumulation1.getKey(), accumulation2.getKey()), >> valCombiner.apply(accumulation1.getValue(), accumulation2.getValue()) >> ); >> } >> >> @Override >> public Function, Map.Entry> finisher() { >> Function keyFinisher = keyCollector.finisher(); >> Function valFinisher = valCollector.finisher(); >> return accumulation -> new AbstractMap.SimpleImmutableEntry<>( >> keyFinisher.apply(accumulation.getKey()), >> valFinisher.apply(accumulation.getValue()) >> ); >> } >> >> @Override >> public Set characteristics() { >> EnumSet intersection = EnumSet.copyOf(keyCollector.characteristics()); >> intersection.retainAll(valCollector.characteristics()); >> return intersection; >> } >> } >> >> >> Do you think this class is general enough to be part of standard Collectors repertoire? >> >> For example, accessed via factory method Collectors.toBoth(Collector coll1, Collector coll2), bi-collection could then be coded simply as: >> >> Map map = ... >> >> Map.Entry, List> keys_values = >> map.entrySet() >> .stream() >> .collect( >> toBoth( >> mapping(Map.Entry::getKey, toList()), >> mapping(Map.Entry::getValue, toList()) >> ) >> ); >> >> >> Map.Entry, Long> histogram_count = >> ThreadLocalRandom >> .current() >> .ints(100, 0, 10) >> .boxed() >> .collect( >> toBoth( >> groupingBy(Function.identity(), counting()), >> counting() >> ) >> ); >> >> >> Regards, Peter >> From peter.levart at gmail.com Thu Jun 14 07:46:31 2018 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 14 Jun 2018 09:46:31 +0200 Subject: BiCollector In-Reply-To: References: Message-ID: <496d36b1-9b05-d299-91fc-457f1d5a6481@gmail.com> Hi Maurizio, Collectors.partitioningBy is only superficially similar. It partitions the stream into two disjunct sub-streams of elements and collects each of them using the same collector. BiCollector OTOH collects all the elemets of the original Stream "two times" into two independent collections (in general using two different collectors). So result of partitioningBy and toBoth are usually totally different. The features of toBoth allow constructing an equivalent case of partitioningBy: Stream stream = ...; Predicate predicate = ...; Collector downstreamCollector = ...; stream.collect( ??? partitioningBy(predicate, downstreamCollector) ); gives equivalent results as: stream.collect( ??? toBoth( ??? ??? filtering(predicate, downstreamCollector), ??? ??? filtering(predicate.negate, downstreamCollector) ??? ) } But there's no way to use partitioningBy to simulate all the variants of toBoth... Regards, Peter On 06/11/18 19:32, Maurizio Cimadamore wrote: > Note also that this has some overlappings with > Collectors.partitioningBy - which currently wraps results into a > Map, where O is the desired collector output type. Without > commenting on the feasibility of its inclusion in the JDK (Paul rules > here :-)), I note that BiStream would obviously allow this > functionality to be exposed in a more user friendly way. > > Cheers > Maurizio > > > On 11/06/18 13:39, Peter Levart wrote: >> Hi, >> >> Have you ever wanted to perform a collection of the same Stream into >> two different targets using two Collectors? Say you wanted to collect >> Map.Entry elements into two parallel lists, each of them containing >> keys and values respectively. Or you wanted to collect elements into? >> groups by some key, but also count them at the same time? Currently >> this is not possible to do with a single Stream. You have to create >> two identical streams, so you end up passing Supplier to >> other methods instead of bare Stream. >> >> I created a little utility Collector implementation that serves the >> purpose quite well: >> >> /** >> ?* A {@link Collector} implementation taking two delegate >> Collector(s) and producing result composed >> ?* of two results produced by delegating collectors, wrapped in >> {@link Map.Entry} object. >> ?* >> ?* @param the type of elements collected >> ?* @param the type of 1st delegate collector collected result >> ?* @param tye type of 2nd delegate collector collected result >> ?*/ >> public class BiCollector implements Collector> Map.Entry, Map.Entry> { >> ??? private final Collector keyCollector; >> ??? private final Collector valCollector; >> >> ??? @SuppressWarnings("unchecked") >> ??? public BiCollector(Collector keyCollector, Collector> ?, V> valCollector) { >> ??????? this.keyCollector = (Collector) >> Objects.requireNonNull(keyCollector); >> ??????? this.valCollector = (Collector) >> Objects.requireNonNull(valCollector); >> ??? } >> >> ??? @Override >> ??? public Supplier> supplier() { >> ??????? Supplier keySupplier = keyCollector.supplier(); >> ??????? Supplier valSupplier = valCollector.supplier(); >> ??????? return () -> new >> AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), >> valSupplier.get()); >> ??? } >> >> ??? @Override >> ??? public BiConsumer, T> accumulator() { >> ??????? BiConsumer keyAccumulator = >> keyCollector.accumulator(); >> ??????? BiConsumer valAccumulator = >> valCollector.accumulator(); >> ??????? return (accumulation, t) -> { >> ??????????? keyAccumulator.accept(accumulation.getKey(), t); >> ??????????? valAccumulator.accept(accumulation.getValue(), t); >> ??????? }; >> ??? } >> >> ??? @Override >> ??? public BinaryOperator> combiner() { >> ??????? BinaryOperator keyCombiner = keyCollector.combiner(); >> ??????? BinaryOperator valCombiner = valCollector.combiner(); >> ??????? return (accumulation1, accumulation2) -> new >> AbstractMap.SimpleImmutableEntry<>( >> ??????????? keyCombiner.apply(accumulation1.getKey(), >> accumulation2.getKey()), >> ??????????? valCombiner.apply(accumulation1.getValue(), >> accumulation2.getValue()) >> ??????? ); >> ??? } >> >> ??? @Override >> ??? public Function, Map.Entry> >> finisher() { >> ??????? Function keyFinisher = keyCollector.finisher(); >> ??????? Function valFinisher = valCollector.finisher(); >> ??????? return accumulation -> new AbstractMap.SimpleImmutableEntry<>( >> ??????????? keyFinisher.apply(accumulation.getKey()), >> ??????????? valFinisher.apply(accumulation.getValue()) >> ??????? ); >> ??? } >> >> ??? @Override >> ??? public Set characteristics() { >> ??????? EnumSet intersection = >> EnumSet.copyOf(keyCollector.characteristics()); >> ??????? intersection.retainAll(valCollector.characteristics()); >> ??????? return intersection; >> ??? } >> } >> >> >> Do you think this class is general enough to be part of standard >> Collectors repertoire? >> >> For example, accessed via factory method Collectors.toBoth(Collector >> coll1, Collector coll2), bi-collection could then be coded simply as: >> >> ??????? Map map = ... >> >> ??????? Map.Entry, List> keys_values = >> ??????????? map.entrySet() >> ?????????????? .stream() >> ?????????????? .collect( >> ?????????????????? toBoth( >> ?????????????????????? mapping(Map.Entry::getKey, toList()), >> ?????????????????????? mapping(Map.Entry::getValue, toList()) >> ?????????????????? ) >> ?????????????? ); >> >> >> ??????? Map.Entry, Long> histogram_count = >> ??????????? ThreadLocalRandom >> ??????????????? .current() >> ??????????????? .ints(100, 0, 10) >> ??????????????? .boxed() >> ??????????????? .collect( >> ??????????????????? toBoth( >> ??????????????????????? groupingBy(Function.identity(), counting()), >> ??????????????????????? counting() >> ??????????????????? ) >> ??????????????? ); >> >> >> Regards, Peter >> > From magnus.ihse.bursie at oracle.com Thu Jun 14 09:53:21 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 14 Jun 2018 11:53:21 +0200 Subject: RFR: 8204967: Resolve disabled warnings for libunpack In-Reply-To: <4cd2a4ef-7dde-469a-8218-230693ba7e68@default> References: <4cd2a4ef-7dde-469a-8218-230693ba7e68@default> Message-ID: <677ede96-2e3f-a6a0-19f9-ef8d18246a3c@oracle.com> On 2018-06-13 18:57, Srinivas Dama wrote: > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8204967/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8204967 Hi Srinivas, In src/jdk.pack/share/native/common-unpack/zip.cpp, you just read and discard the return value from fread to hide the warning. But the purpose of the warning is to point out that this kind of code is incorrect. The proper fix would be to check the return code from fread to make sure that the read did not fail. Otherwise you're just making it impossible for the compiler to point out the badly written code, and if you're not going to fix this properly, it's better to keep the warning (that way we know the code is fishy). /Magnus > > Regards, > Srinivas From patrick at reini.net Thu Jun 14 10:50:34 2018 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 14 Jun 2018 12:50:34 +0200 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: <3A88FBD3-6C6E-4F38-B11E-CE2D6A251D52@oracle.com> References: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> <3A88FBD3-6C6E-4F38-B11E-CE2D6A251D52@oracle.com> Message-ID: <315f23c1dddb0724908fd22134021316@reini.net> Hi everybody, I re-proposed the CSR with the changed API. Somehow I'm not able to add the comment explaning the change. (Tried both in proposed and draft state) :-( I also updated the webrev to represent the two small changes as proposed from Brian. -Patrick On 2018-06-13 23:16, Brian Burkhalter wrote: > Hi Roger, > > Thanks for pointing this out: simpler and cleaner. > > Brian > > On Jun 13, 2018, at 2:06 PM, Roger Riggs > wrote: > >> For the CSR, the easiest path to clarify the spec is to withdraw the >> CSR, update the spec, >> and add a note on the revised behavior. >> >> Then finalize the CSR again. That's enough to get it reviewed and >> approved. >> >> (Using a new CSR would just spread the behavior over two CSRs). From Alan.Bateman at oracle.com Thu Jun 14 11:03:04 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Jun 2018 12:03:04 +0100 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: <315f23c1dddb0724908fd22134021316@reini.net> References: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> <3A88FBD3-6C6E-4F38-B11E-CE2D6A251D52@oracle.com> <315f23c1dddb0724908fd22134021316@reini.net> Message-ID: <97e913d2-c79f-2e1a-c2be-cb45815b78f2@oracle.com> On 14/06/2018 11:50, Patrick Reinhart wrote: > Hi everybody, > > I re-proposed the CSR with the changed API. The CSR is associated with the original change-set that was pushed a few months so the simplest is to just create a follow-up CSR that is linked to the new bug. > Somehow I'm not able to add the comment explaning the change. (Tried > both in proposed and draft state) :-( Yeah, it's annoying, it seems have come with a recent JIRA update. > > I also updated the webrev to represent the two small changes as > proposed from Brian. From patrick at reini.net Thu Jun 14 11:08:20 2018 From: patrick at reini.net (Patrick Reinhart) Date: Thu, 14 Jun 2018 13:08:20 +0200 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: <97e913d2-c79f-2e1a-c2be-cb45815b78f2@oracle.com> References: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> <3A88FBD3-6C6E-4F38-B11E-CE2D6A251D52@oracle.com> <315f23c1dddb0724908fd22134021316@reini.net> <97e913d2-c79f-2e1a-c2be-cb45815b78f2@oracle.com> Message-ID: <67efb7a80aa9c47aa0b7c7c8b1cd4f5e@reini.net> Sorry, everybody it seems, that I completely messed things up :-(( @Alan can you help me getting that back in order again? -Patrick On 2018-06-14 13:03, Alan Bateman wrote: > On 14/06/2018 11:50, Patrick Reinhart wrote: >> Hi everybody, >> >> I re-proposed the CSR with the changed API. > The CSR is associated with the original change-set that was pushed a > few months so the simplest is to just create a follow-up CSR that is > linked to the new bug. > > >> Somehow I'm not able to add the comment explaning the change. (Tried >> both in proposed and draft state) :-( > Yeah, it's annoying, it seems have come with a recent JIRA update. > > >> >> I also updated the webrev to represent the two small changes as >> proposed from Brian. From Alan.Bateman at oracle.com Thu Jun 14 11:30:29 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Jun 2018 12:30:29 +0100 Subject: RFR (JDK11/NIO) 8201276: (fs) Add methods to Files for reading/writing a string from/to a file In-Reply-To: <5B1FFA39.9000704@oracle.com> References: <5AE2AC03.9000705@oracle.com> <1297536212.1989108.1524828579265.JavaMail.zimbra@u-pem.fr> <515d4180-e2b7-48f3-5856-f472cf7892a4@oracle.com> <106042075.1993206.1524829404390.JavaMail.zimbra@u-pem.fr> <499f6815-ae0d-73e6-e606-5f3352c89ba9@oracle.com> <2100883236.2133173.1524858613899.JavaMail.zimbra@u-pem.fr> <7ffaa0de-cbee-0fc7-57b2-44caa39dad47@oracle.com> <5B1FFA39.9000704@oracle.com> Message-ID: <7dfb89ef-6625-8699-87c5-3cf31d76ae57@oracle.com> On 12/06/2018 17:52, Joe Wang wrote: > Hi all, > > It's been a while since the 1st round of reviews. Thank you all for > the valuable comments and suggestions!? Note that Roger's last > response went to nio-dev, but not core-libs-dev, I've therefore > attached it below. > > CSR [2]: the CSR is now approved. Note the write method has been > changed to writeString. > > Impl [3]: for performance reason and the different requirement from > byte-string conversion for malformed/unmappable character handing, the > implementation uses a specific method separate from the existing ones. > Both Sherman and Alan preferred specific method than adding parameters > to the APIs that might be error prone. Thanks Sherman for the code! > > JMH benchmark tests showed the new read method performs better than an > operation that reads the bytes and convert to string. The gains start > to be significant even at quite reasonable sizes (< 10K). > > > [1] JBS: https://bugs.openjdk.java.net/browse/JDK-8201276 > [2] CSR: https://bugs.openjdk.java.net/browse/JDK-8202055 > [3] webrevs: http://cr.openjdk.java.net/~joehw/jdk11/8201276/webrev/ > > Please review. The javadoc looks good. One part of the implementation that could be cleaner is the exception thrown for the malformed or unmappable cases. There are sub-classes of CharacterCodingException defined for this that would be clearer than an IOException with an IllegalArgumentException as cause. A minor nit in Files is that you can import jdk.internal.misc.JavaLangAccess rather than repeating the fully qualified class name. You can also move the definition of JLA up to the top. There's also a few inconsistencies with the existing code that would be goof to fix too (indenting and line length issues mostly). The test looks reasonable. In getData() then then "shouldn't happen" case should throw an exception as a NPE here might be tricky to diagnose there. Another nit is the sb field - can that be removed. -Alan From Alan.Bateman at oracle.com Thu Jun 14 12:39:24 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Jun 2018 13:39:24 +0100 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: <67efb7a80aa9c47aa0b7c7c8b1cd4f5e@reini.net> References: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> <3A88FBD3-6C6E-4F38-B11E-CE2D6A251D52@oracle.com> <315f23c1dddb0724908fd22134021316@reini.net> <97e913d2-c79f-2e1a-c2be-cb45815b78f2@oracle.com> <67efb7a80aa9c47aa0b7c7c8b1cd4f5e@reini.net> Message-ID: <0f336fa8-9cde-1943-58b7-d751234c453d@oracle.com> On 14/06/2018 12:08, Patrick Reinhart wrote: > > Sorry, everybody it seems, that I completely messed things up :-(( > > @Alan can you help me getting that back in order again? Sure. BTW: I should have mentioned that the main workaround that people are using is to edit the CSR as that allows you to add a comment, even if you don't touch other fields. Hopefully the JIRA bug will get fixed soon. From scolebourne at joda.org Thu Jun 14 12:56:14 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 14 Jun 2018 13:56:14 +0100 Subject: BiCollector In-Reply-To: References: <616ff86f-b4ea-df23-6707-6771fc475d68@oracle.com> Message-ID: In this form (not locked to Map.Entry) it seems like a candidate for the JDK. The implementation is private, so it would be one public method that provides a new way to process streams. Stephen On 14 June 2018 at 05:56, Tagir Valeev wrote: > Hello! > > In my StreamEx library I created a "pairing" collector which does similar > job, but allows user to decide how to combine the results of two > collectors. This adds more flexibility. The signature is like this: > > public static Collector pairing( > Collector c1, > Collector c2, > BiFunction finisher) > > Having such collector, The proposed `toBoth(c1, c2)` can be implemented as > simple as `pairing(c1, c2, Map::entry)`. OTOH if somebody wants to use > their own Pair class, it would be `pairing(c1, c2, Pair::new)`. > Sometimes you don't need a pair, but can create compound result object > right here. E.g.: > > Collector summing = > Collectors.reducing(BigDecimal.ZERO, BigDecimal::add); > Collector averaging = > pairing(summing, Collectors.counting(), (sum, count) -> > sum.divide(BigDecimal.valueOf(count), > RoundingMode.HALF_EVEN)); > > So locking to Map.Entry is entirely unnecessary. > > With best regards, > Tagir Valeev. > > > > On Thu, Jun 14, 2018 at 6:11 AM Brian Goetz wrote: > >> I really like how BiCollector can be private, so all we'd have to expose >> is toBoth(), and the arguments of toBoth() are just ordinary >> collectors. So no new types or abstractions; just a multiplexing. >> >> The use of Map.Entry as a pair surrogate is unfortunate -- its >> definitely not an Entry -- though I understand why you did this. I'm not >> sure if this is fatal or not for a JDK method, but it's pretty bad*. >> (You could generalize to n-ary, returning a List or array (you can take >> a varargs of Collector), at the loss of sharp types for the results.) >> >> >> *Yes, I'm sure one can find precedent of this being done; this has no >> effect on whether it's bad. >> >> On 6/11/2018 8:39 AM, Peter Levart wrote: >> > Hi, >> > >> > Have you ever wanted to perform a collection of the same Stream into >> > two different targets using two Collectors? Say you wanted to collect >> > Map.Entry elements into two parallel lists, each of them containing >> > keys and values respectively. Or you wanted to collect elements into >> > groups by some key, but also count them at the same time? Currently >> > this is not possible to do with a single Stream. You have to create >> > two identical streams, so you end up passing Supplier to other >> > methods instead of bare Stream. >> > >> > I created a little utility Collector implementation that serves the >> > purpose quite well: >> > >> > /** >> > * A {@link Collector} implementation taking two delegate Collector(s) >> > and producing result composed >> > * of two results produced by delegating collectors, wrapped in {@link >> > Map.Entry} object. >> > * >> > * @param the type of elements collected >> > * @param the type of 1st delegate collector collected result >> > * @param tye type of 2nd delegate collector collected result >> > */ >> > public class BiCollector implements Collector> > Map.Entry, Map.Entry> { >> > private final Collector keyCollector; >> > private final Collector valCollector; >> > >> > @SuppressWarnings("unchecked") >> > public BiCollector(Collector keyCollector, Collector> > ?, V> valCollector) { >> > this.keyCollector = (Collector) >> > Objects.requireNonNull(keyCollector); >> > this.valCollector = (Collector) >> > Objects.requireNonNull(valCollector); >> > } >> > >> > @Override >> > public Supplier> supplier() { >> > Supplier keySupplier = keyCollector.supplier(); >> > Supplier valSupplier = valCollector.supplier(); >> > return () -> new >> > AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); >> > } >> > >> > @Override >> > public BiConsumer, T> accumulator() { >> > BiConsumer keyAccumulator = >> > keyCollector.accumulator(); >> > BiConsumer valAccumulator = >> > valCollector.accumulator(); >> > return (accumulation, t) -> { >> > keyAccumulator.accept(accumulation.getKey(), t); >> > valAccumulator.accept(accumulation.getValue(), t); >> > }; >> > } >> > >> > @Override >> > public BinaryOperator> combiner() { >> > BinaryOperator keyCombiner = keyCollector.combiner(); >> > BinaryOperator valCombiner = valCollector.combiner(); >> > return (accumulation1, accumulation2) -> new >> > AbstractMap.SimpleImmutableEntry<>( >> > keyCombiner.apply(accumulation1.getKey(), >> > accumulation2.getKey()), >> > valCombiner.apply(accumulation1.getValue(), >> > accumulation2.getValue()) >> > ); >> > } >> > >> > @Override >> > public Function, Map.Entry> >> > finisher() { >> > Function keyFinisher = keyCollector.finisher(); >> > Function valFinisher = valCollector.finisher(); >> > return accumulation -> new AbstractMap.SimpleImmutableEntry<>( >> > keyFinisher.apply(accumulation.getKey()), >> > valFinisher.apply(accumulation.getValue()) >> > ); >> > } >> > >> > @Override >> > public Set characteristics() { >> > EnumSet intersection = >> > EnumSet.copyOf(keyCollector.characteristics()); >> > intersection.retainAll(valCollector.characteristics()); >> > return intersection; >> > } >> > } >> > >> > >> > Do you think this class is general enough to be part of standard >> > Collectors repertoire? >> > >> > For example, accessed via factory method Collectors.toBoth(Collector >> > coll1, Collector coll2), bi-collection could then be coded simply as: >> > >> > Map map = ... >> > >> > Map.Entry, List> keys_values = >> > map.entrySet() >> > .stream() >> > .collect( >> > toBoth( >> > mapping(Map.Entry::getKey, toList()), >> > mapping(Map.Entry::getValue, toList()) >> > ) >> > ); >> > >> > >> > Map.Entry, Long> histogram_count = >> > ThreadLocalRandom >> > .current() >> > .ints(100, 0, 10) >> > .boxed() >> > .collect( >> > toBoth( >> > groupingBy(Function.identity(), counting()), >> > counting() >> > ) >> > ); >> > >> > >> > Regards, Peter >> > >> >> From rachna.goel at oracle.com Thu Jun 14 14:01:55 2018 From: rachna.goel at oracle.com (Rachna Goel) Date: Thu, 14 Jun 2018 19:31:55 +0530 Subject: RFR: [11] 8202537: CLDR 33 In-Reply-To: References: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> Message-ID: <2cc825ca-35e3-de95-e233-4444158e32c2@oracle.com> Hi Naoto, Thanks a lot for the review. I have made suggested changes, Kindly have a look at : http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.07/ - Updated NumberingSystemsParseHandler.java - Updated LocaleData.cldr for new test case. Thanks, Rachna On 6/13/18 10:33 PM, naoto.sato at oracle.com wrote: > Hi Rachna, > > A couple of comments: > > - NumberingSystemsParseHandler.java > > Since the code substitutes latin digits for supplementary digits, it > can skip line 68-79. > > - A test should be written for the above substitution. > > Naoto > > On 6/12/18 10:33 PM, Rachna Goel wrote: >> Hi, >> >> Kindly review fix for JDK-8202537. Fix is to upgrade Unicode >> consortium's CLDR data into JDK from current version 29 to 33. >> >> For more info : http://cldr.unicode.org/index/downloads/cldr-33 >> >> Bug : https://bugs.openjdk.java.net/browse/JDK-8202537 >> >> Patch: >> http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.06/index.html >> -- Thanks, Rachna From paul.sandoz at oracle.com Thu Jun 14 15:36:16 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 14 Jun 2018 08:36:16 -0700 Subject: BiCollector In-Reply-To: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> Message-ID: <1EF1B940-0C42-4075-8B92-596C181F6295@oracle.com> Hi Peter, I am not concerned about performance of Map.Entry. I find its use awkward for similar reasons as Brian outlined. Tagir?s approach using a finisher nicely side steps this at the expense of another function. If in the future we have an officially blessed pair/2-tuple class we can overload the collector factory method. Regarding BitStream. A Stream could be bisected into a BiStream, then merged back into a Stream or bi-collect (something which i did not get time to look at). Having gone back and looked at the prototype work I tend to see a bisecting collector as complementary in this respect since we have similar operations for collection as we do on Stream. Paul. > On Jun 14, 2018, at 12:29 AM, Peter Levart wrote: > > Hi Paul, > > On 06/11/18 19:10, Paul Sandoz wrote: >> Hi Peter, >> >> I like it and can see it being useful, thanks for sharing. >> >> I am hesitating a little about it being in the JDK because there is the larger abstraction of a BiStream, where a similar form of collection would naturally fit (but perhaps without the intersection constraints for the characteristics?). We experimented a few times with BiStream and got quite far but decided pull back due to the lack of value types and specialized generics. So i dunno how this might turn out in the future and if your BiCollector fits nicely into such a future model. >> >> What are you thoughts on this? > > Well, I don't see the need to pack the two results into a Map.Entry (or any similar) container as a drawback. It's not a performance drawback for sure, because this is not happening on the stream-element scale, but on the final result or intermediate accumulation results scale (the later only in parallel non-CONCURRENT scenario). In non-parallel scenario, only a single (or two for non-IDENTITY_FINISH) Map.Entry objects are created. > > I also don't see a larger abstraction like BiStream as a natural fit for a similar thing. As I understand BiStream attempts (maybe I haven't seen the right ones?), they are more about passing pairs of elements down the pipeline. BiCollector OTOH is "splitting" the single element pipeline at the final collection stage, with the purpose of constructing two independent collections from a single pass of the original single-element Stream. This is more about single pass than anything else. And single pass, I think, is inevitably such that it can only execute a single collection strategy (CONCURRENT vs. not CONCURRENT), regardless of the type of the stream (Stream vs. BiStream). Or have you prototyped a combined strategy in BiStream? > > Regards, Peter > >> FWIW i would call it a ?splitting? or ?bisecting" collector e.g. ?s.collect(bisecting(?))? >> >> Paul. >> >> >> >> >>> On Jun 11, 2018, at 5:39 AM, Peter Levart wrote: >>> >>> Hi, >>> >>> Have you ever wanted to perform a collection of the same Stream into two different targets using two Collectors? Say you wanted to collect Map.Entry elements into two parallel lists, each of them containing keys and values respectively. Or you wanted to collect elements into groups by some key, but also count them at the same time? Currently this is not possible to do with a single Stream. You have to create two identical streams, so you end up passing Supplier to other methods instead of bare Stream. >>> >>> I created a little utility Collector implementation that serves the purpose quite well: >>> >>> /** >>> * A {@link Collector} implementation taking two delegate Collector(s) and producing result composed >>> * of two results produced by delegating collectors, wrapped in {@link Map.Entry} object. >>> * >>> * @param the type of elements collected >>> * @param the type of 1st delegate collector collected result >>> * @param tye type of 2nd delegate collector collected result >>> */ >>> public class BiCollector implements Collector, Map.Entry> { >>> private final Collector keyCollector; >>> private final Collector valCollector; >>> >>> @SuppressWarnings("unchecked") >>> public BiCollector(Collector keyCollector, Collector valCollector) { >>> this.keyCollector = (Collector) Objects.requireNonNull(keyCollector); >>> this.valCollector = (Collector) Objects.requireNonNull(valCollector); >>> } >>> >>> @Override >>> public Supplier> supplier() { >>> Supplier keySupplier = keyCollector.supplier(); >>> Supplier valSupplier = valCollector.supplier(); >>> return () -> new AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); >>> } >>> >>> @Override >>> public BiConsumer, T> accumulator() { >>> BiConsumer keyAccumulator = keyCollector.accumulator(); >>> BiConsumer valAccumulator = valCollector.accumulator(); >>> return (accumulation, t) -> { >>> keyAccumulator.accept(accumulation.getKey(), t); >>> valAccumulator.accept(accumulation.getValue(), t); >>> }; >>> } >>> >>> @Override >>> public BinaryOperator> combiner() { >>> BinaryOperator keyCombiner = keyCollector.combiner(); >>> BinaryOperator valCombiner = valCollector.combiner(); >>> return (accumulation1, accumulation2) -> new AbstractMap.SimpleImmutableEntry<>( >>> keyCombiner.apply(accumulation1.getKey(), accumulation2.getKey()), >>> valCombiner.apply(accumulation1.getValue(), accumulation2.getValue()) >>> ); >>> } >>> >>> @Override >>> public Function, Map.Entry> finisher() { >>> Function keyFinisher = keyCollector.finisher(); >>> Function valFinisher = valCollector.finisher(); >>> return accumulation -> new AbstractMap.SimpleImmutableEntry<>( >>> keyFinisher.apply(accumulation.getKey()), >>> valFinisher.apply(accumulation.getValue()) >>> ); >>> } >>> >>> @Override >>> public Set characteristics() { >>> EnumSet intersection = EnumSet.copyOf(keyCollector.characteristics()); >>> intersection.retainAll(valCollector.characteristics()); >>> return intersection; >>> } >>> } >>> >>> >>> Do you think this class is general enough to be part of standard Collectors repertoire? >>> >>> For example, accessed via factory method Collectors.toBoth(Collector coll1, Collector coll2), bi-collection could then be coded simply as: >>> >>> Map map = ... >>> >>> Map.Entry, List> keys_values = >>> map.entrySet() >>> .stream() >>> .collect( >>> toBoth( >>> mapping(Map.Entry::getKey, toList()), >>> mapping(Map.Entry::getValue, toList()) >>> ) >>> ); >>> >>> >>> Map.Entry, Long> histogram_count = >>> ThreadLocalRandom >>> .current() >>> .ints(100, 0, 10) >>> .boxed() >>> .collect( >>> toBoth( >>> groupingBy(Function.identity(), counting()), >>> counting() >>> ) >>> ); >>> >>> >>> Regards, Peter >>> > From brian.goetz at oracle.com Thu Jun 14 17:04:18 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 14 Jun 2018 13:04:18 -0400 Subject: BiCollector In-Reply-To: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> Message-ID: <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> > Well, I don't see the need to pack the two results into a Map.Entry > (or any similar) container as a drawback. From an "integrity of the JDK APIs" perspective, it is unquestionably a drawback.? These items are not a Key and an associated Value in a Map; it's merely pretending that Map.Entry really means "Pair".? There's a reason we don't have a Pair class in the JDK (and no, let's not reopen that now); using something else as a Pair proxy that is supposed to have specific semantics is worse. (It's fine to do this in your own code, but not in the JDK. Different standards for code that has different audiences.) Tagir's proposed sidestepping is nice, and it will also play nicely with records, because then you can say: ???? record NameAndCount(String name, int count); ???? stream.collect(pairing(collectName, collectCount, NameAndCount::new)); and get a more properly abstract result out.? And its more in the spirit of existing Collectors.? If you want to use Map.Entry as an _implementation detail_, that's fine. I can support this form. > I also don't see a larger abstraction like BiStream as a natural fit > for a similar thing. I think the BiStream connection is mostly tangential.? We tried hard to support streams of (K,V) pairs when we did streams, as Paul can attest, but it was a huge complexity-inflater and dropping this out paid an enormous simplification payoff. With records, having streams of tuples will be simpler to represent, but no more performant; it will take until we get to value types and specialized generics to get the performance we want out of this. From maurizio.cimadamore at oracle.com Thu Jun 14 17:13:26 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 14 Jun 2018 18:13:26 +0100 Subject: BiCollector In-Reply-To: <496d36b1-9b05-d299-91fc-457f1d5a6481@gmail.com> References: <496d36b1-9b05-d299-91fc-457f1d5a6481@gmail.com> Message-ID: On 14/06/18 08:46, Peter Levart wrote: > But there's no way to use partitioningBy to simulate all the variants > of toBoth... I wasn't suggesting that - more the other way around, as you illustrated Maurizio From vicente.romero at oracle.com Thu Jun 14 17:31:06 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Thu, 14 Jun 2018 13:31:06 -0400 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: References: <1f87dd21-701b-975c-a1c1-f1d9ed2f1eaf@oracle.com> <5A90B204.4070005@oracle.com> <5A960359.4070507@oracle.com> Message-ID: I'm not an expert in the area, but the patch looks good to me, Vicente On 05/31/2018 08:39 PM, Liam Miller-Cushon wrote: > Hi, > > Are there any concerns with the patch that I can address? I was hoping to > get this in to JDK 11. > > I confirmed that it still builds and passes all tests at head. > > Thanks, > Liam > > On Mon, May 14, 2018 at 10:38 AM Liam Miller-Cushon > wrote: > >> Ping >> >> On Thu, Apr 5, 2018 at 10:57 AM Liam Miller-Cushon >> wrote: >> >>> Hi, >>> >>> Is there any more feedback on the patch? >>> >>> On Wed, Feb 28, 2018 at 4:26 PM Liam Miller-Cushon >>> wrote: >>> >>>> On Wed, Feb 28, 2018 at 4:13 PM, Martin Buchholz >>>> wrote: >>>> >>>>> Follow their lead and rename your method to assertArrayEquals >>>>> >>>> Done: http://cr.openjdk.java.net/~cushon/7183985/webrev.04/ >>>> From paul.sandoz at oracle.com Thu Jun 14 17:41:20 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 14 Jun 2018 10:41:20 -0700 Subject: RFR 8202922 Method reference identity is broken by serialization Message-ID: <89462B7C-BE80-42FA-8D49-DC5737948A01@oracle.com> Hi, Please review an update to the specifications of LambdaMetafactory and SerializedLambda to clarify that the identity of function objects is unpredictable. A similar (informational) clarification was made to the language specification and similar wording is used. Amusingly a quirk of (or issue with) lambda deserialization (we count our blessings it works as it does!) motivated this clarification. http://cr.openjdk.java.net/~psandoz/jdk/JDK-8202922-function-object-identity/webrev/ CSR: https://bugs.openjdk.java.net/browse/JDK-8205060 Thanks, Paul. From erik.joelsson at oracle.com Thu Jun 14 18:52:02 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 14 Jun 2018 11:52:02 -0700 Subject: RFR: JDK-8204973: Add build support for filtering translations In-Reply-To: References: Message-ID: <33cc25be-838d-bb88-1332-b8f9178fc742@oracle.com> Hello, Here is a new version of the patch: http://cr.openjdk.java.net/~erikj/8204973/webrev.02/ Changes from last time: * Made the regexp for finding locales more correct. It still does not try to match 3 letter language strings because doing so triggers a large amount of false positives in our souce tree. * Added another accepted locale (en_US_POSIX) that is now matched by the improved regexp. * Added more locales to the exclude list as they were now discovered by the improved regexp. /Erik On 2018-06-13 12:47, Erik Joelsson wrote: > Hello, > > Oracle will reduce the number of languages that it maintains > translations of JDK resources for. The current translations will > remain in the source for now, but we need a way to filter out a set of > translations at build time so that we only include the ones we > support. This patch adds such a configuration option. It also changes > how Oracle builds by using the option to exclude all translations > except English, Japanese, Simplified Chinese and Traditional Chinese. > Anyone else building OpenJDK will by default include all translations > present in the source, just as before. > > I added a test that verifies this for builds with the "IMPLEMENTOR" > field in the release file set to "Oracle Corporation". The test will > not be run for other OpenJDK builds. > > I had to modify an existing test for java.logging which used various > translations to verify localized log messages to only use translations > that Oracle chooses to include. > > Since this is the second test that specifically verifies build > behavior, I moved the previous such test together with this new test > into a common top level test directory "build", under the jdk test > root. I put these tests in the jdk tier3 test group. > > I have run all tier1, 2 and 3 tests in Mach 5 as well as specifically > looked for tests that use the java.util.Locale class and ran them > locally. > > Webrev: http://cr.openjdk.java.net/~erikj/8204973/webrev.01/index.html > > Bug: https://bugs.openjdk.java.net/browse/JDK-8204973 > > /Erik > From naoto.sato at oracle.com Thu Jun 14 19:42:54 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 14 Jun 2018 12:42:54 -0700 Subject: RFR: [11] 8202537: CLDR 33 In-Reply-To: <2cc825ca-35e3-de95-e233-4444158e32c2@oracle.com> References: <1ba7ab6d-d1d7-7c06-1b3e-1f9665cbe61a@oracle.com> <2cc825ca-35e3-de95-e233-4444158e32c2@oracle.com> Message-ID: <7b670a7b-654a-273c-6ef8-50a087006c3b@oracle.com> Looks good to me. Naoto On 6/14/18 7:01 AM, Rachna Goel wrote: > Hi Naoto, > > Thanks a lot for the review. > > I have made suggested changes, Kindly have a look at : > http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.07/ > > - Updated NumberingSystemsParseHandler.java > > - Updated LocaleData.cldr for new test case. > > Thanks, > > Rachna > > > On 6/13/18 10:33 PM, naoto.sato at oracle.com wrote: >> Hi Rachna, >> >> A couple of comments: >> >> - NumberingSystemsParseHandler.java >> >> Since the code substitutes latin digits for supplementary digits, it >> can skip line 68-79. >> >> - A test should be written for the above substitution. >> >> Naoto >> >> On 6/12/18 10:33 PM, Rachna Goel wrote: >>> Hi, >>> >>> Kindly review fix for JDK-8202537. Fix is to upgrade Unicode >>> consortium's CLDR data into JDK from current version 29 to 33. >>> >>> For more info : http://cldr.unicode.org/index/downloads/cldr-33 >>> >>> Bug : https://bugs.openjdk.java.net/browse/JDK-8202537 >>> >>> Patch: >>> http://cr.openjdk.java.net/~rgoel/JDK-8202537/webrev.06/index.html >>> > > -- > Thanks, > Rachna > From magnus.ihse.bursie at oracle.com Thu Jun 14 19:49:07 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 14 Jun 2018 21:49:07 +0200 Subject: RFR: JDK-8204973: Add build support for filtering translations In-Reply-To: <33cc25be-838d-bb88-1332-b8f9178fc742@oracle.com> References: <33cc25be-838d-bb88-1332-b8f9178fc742@oracle.com> Message-ID: Would it have been to problematic to reverse the the logic, i.e. having a "include translation" rather than an exclude? Feels more brittle; if someone adds a translation the exclude list needs to be updated. Also, a include mechanism could possibly be used, much simpler, by someone who only wants to build e.g. a chinese jdk. But I realize the contraints by the somewhat odd request from the bug report, to deliver just a subset of the available translations. So I'm okay with the patch. /Magnus On 2018-06-14 20:52, Erik Joelsson wrote: > Hello, > > Here is a new version of the patch: > > http://cr.openjdk.java.net/~erikj/8204973/webrev.02/ > > Changes from last time: > > * Made the regexp for finding locales more correct. It still does not > try to match 3 letter language strings because doing so triggers a > large amount of false positives in our souce tree. > > * Added another accepted locale (en_US_POSIX) that is now matched by > the improved regexp. > > * Added more locales to the exclude list as they were now discovered > by the improved regexp. > > /Erik > > > On 2018-06-13 12:47, Erik Joelsson wrote: >> Hello, >> >> Oracle will reduce the number of languages that it maintains >> translations of JDK resources for. The current translations will >> remain in the source for now, but we need a way to filter out a set >> of translations at build time so that we only include the ones we >> support. This patch adds such a configuration option. It also changes >> how Oracle builds by using the option to exclude all translations >> except English, Japanese, Simplified Chinese and Traditional Chinese. >> Anyone else building OpenJDK will by default include all translations >> present in the source, just as before. >> >> I added a test that verifies this for builds with the "IMPLEMENTOR" >> field in the release file set to "Oracle Corporation". The test will >> not be run for other OpenJDK builds. >> >> I had to modify an existing test for java.logging which used various >> translations to verify localized log messages to only use >> translations that Oracle chooses to include. >> >> Since this is the second test that specifically verifies build >> behavior, I moved the previous such test together with this new test >> into a common top level test directory "build", under the jdk test >> root. I put these tests in the jdk tier3 test group. >> >> I have run all tier1, 2 and 3 tests in Mach 5 as well as specifically >> looked for tests that use the java.util.Locale class and ran them >> locally. >> >> Webrev: http://cr.openjdk.java.net/~erikj/8204973/webrev.01/index.html >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204973 >> >> /Erik >> > From naoto.sato at oracle.com Thu Jun 14 20:05:15 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 14 Jun 2018 13:05:15 -0700 Subject: RFR: JDK-8204973: Add build support for filtering translations In-Reply-To: <33cc25be-838d-bb88-1332-b8f9178fc742@oracle.com> References: <33cc25be-838d-bb88-1332-b8f9178fc742@oracle.com> Message-ID: Looks good to me. Naoto On 6/14/18 11:52 AM, Erik Joelsson wrote: > Hello, > > Here is a new version of the patch: > > http://cr.openjdk.java.net/~erikj/8204973/webrev.02/ > > Changes from last time: > > * Made the regexp for finding locales more correct. It still does not > try to match 3 letter language strings because doing so triggers a large > amount of false positives in our souce tree. > > * Added another accepted locale (en_US_POSIX) that is now matched by the > improved regexp. > > * Added more locales to the exclude list as they were now discovered by > the improved regexp. > > /Erik > > > On 2018-06-13 12:47, Erik Joelsson wrote: >> Hello, >> >> Oracle will reduce the number of languages that it maintains >> translations of JDK resources for. The current translations will >> remain in the source for now, but we need a way to filter out a set of >> translations at build time so that we only include the ones we >> support. This patch adds such a configuration option. It also changes >> how Oracle builds by using the option to exclude all translations >> except English, Japanese, Simplified Chinese and Traditional Chinese. >> Anyone else building OpenJDK will by default include all translations >> present in the source, just as before. >> >> I added a test that verifies this for builds with the "IMPLEMENTOR" >> field in the release file set to "Oracle Corporation". The test will >> not be run for other OpenJDK builds. >> >> I had to modify an existing test for java.logging which used various >> translations to verify localized log messages to only use translations >> that Oracle chooses to include. >> >> Since this is the second test that specifically verifies build >> behavior, I moved the previous such test together with this new test >> into a common top level test directory "build", under the jdk test >> root. I put these tests in the jdk tier3 test group. >> >> I have run all tier1, 2 and 3 tests in Mach 5 as well as specifically >> looked for tests that use the java.util.Locale class and ran them >> locally. >> >> Webrev: http://cr.openjdk.java.net/~erikj/8204973/webrev.01/index.html >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204973 >> >> /Erik >> > From erik.joelsson at oracle.com Thu Jun 14 20:08:55 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 14 Jun 2018 13:08:55 -0700 Subject: RFR: JDK-8204973: Add build support for filtering translations In-Reply-To: References: <33cc25be-838d-bb88-1332-b8f9178fc742@oracle.com> Message-ID: <84ffd038-6c14-ef39-6103-686d1e8bb538@oracle.com> On 2018-06-14 12:49, Magnus Ihse Bursie wrote: > Would it have been to problematic to reverse the the logic, i.e. > having a "include translation" rather than an exclude? Feels more > brittle; if someone adds a translation the exclude list needs to be > updated. Also, a include mechanism could possibly be used, much > simpler, by someone who only wants to build e.g. a chinese jdk. > I agree that include would have been easier to use, and I was initially trying to implement it that way, but couldn't come up with an effective solution in make. You need to identify all source files that have locale suffixes and only filter among them. With the exclude, at least the build logic is reasonably straight forward since the user defines the locale strings that define a translation that warrants action. This is why I added a test that uses include logic, hoping that the combination would prove resilient enough to catch such changes. The test is still a bit brittle since it can easily catch false positives. > But I realize the contraints by the somewhat odd request from the bug > report, to deliver just a subset of the available translations. So I'm > okay with the patch. > Thanks! /Erik > /Magnus > > On 2018-06-14 20:52, Erik Joelsson wrote: >> Hello, >> >> Here is a new version of the patch: >> >> http://cr.openjdk.java.net/~erikj/8204973/webrev.02/ >> >> Changes from last time: >> >> * Made the regexp for finding locales more correct. It still does not >> try to match 3 letter language strings because doing so triggers a >> large amount of false positives in our souce tree. >> >> * Added another accepted locale (en_US_POSIX) that is now matched by >> the improved regexp. >> >> * Added more locales to the exclude list as they were now discovered >> by the improved regexp. >> >> /Erik >> >> >> On 2018-06-13 12:47, Erik Joelsson wrote: >>> Hello, >>> >>> Oracle will reduce the number of languages that it maintains >>> translations of JDK resources for. The current translations will >>> remain in the source for now, but we need a way to filter out a set >>> of translations at build time so that we only include the ones we >>> support. This patch adds such a configuration option. It also >>> changes how Oracle builds by using the option to exclude all >>> translations except English, Japanese, Simplified Chinese and >>> Traditional Chinese. Anyone else building OpenJDK will by default >>> include all translations present in the source, just as before. >>> >>> I added a test that verifies this for builds with the "IMPLEMENTOR" >>> field in the release file set to "Oracle Corporation". The test will >>> not be run for other OpenJDK builds. >>> >>> I had to modify an existing test for java.logging which used various >>> translations to verify localized log messages to only use >>> translations that Oracle chooses to include. >>> >>> Since this is the second test that specifically verifies build >>> behavior, I moved the previous such test together with this new test >>> into a common top level test directory "build", under the jdk test >>> root. I put these tests in the jdk tier3 test group. >>> >>> I have run all tier1, 2 and 3 tests in Mach 5 as well as >>> specifically looked for tests that use the java.util.Locale class >>> and ran them locally. >>> >>> Webrev: http://cr.openjdk.java.net/~erikj/8204973/webrev.01/index.html >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8204973 >>> >>> /Erik >>> >> > From paul.sandoz at oracle.com Thu Jun 14 21:35:49 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 14 Jun 2018 14:35:49 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal Message-ID: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> Hi, Please review this enhancement to improve the performance of BitSet traversal when using a stream. http://cr.openjdk.java.net/~psandoz/jdk/JDK-8170159-bitset-traverse/webrev/ The associated issue started out life referring to the BitSet?s spliterator reporting SIZED, and to report that the cardinality has to be computed. This has some cost but performance analysis (see issue for attached benchmark) has shown that the cost is small compared to the cost of traversal. It is recognized that the cost is higher for smaller BitSets but not unduly so. Thus it was concluded that it was reasonable for the spliterator to report SIZED. The issue was adjusted to focus on improving the performance of the BitSet?s spliterator forEachRemaining method. For large randomized BitSets a 1.6x speedup can be observed, and for smaller sizes a more modest speed up. The prior conclusion about reporting SIZED still holds. Thanks, Paul. From naokikishida at gmail.com Thu Jun 14 12:13:25 2018 From: naokikishida at gmail.com (kishida naoki) Date: Thu, 14 Jun 2018 21:13:25 +0900 Subject: Japanese New Era supporting does not accept Heisei 32 Message-ID: I'm trying to use the Japanese new era supporting and have found that it does not accept Heisei 32. There are many document that include Heisei 32 or later at present. For example, the expiration date of my driver license is Heisei 32 March. JDK10 can accept Heisei 32. I think the change of the behavior make some trouble. Additionally, Japanese government says some system use Heisei after 5/1/2019 for a while. The JDK version that include the new era support can not be used on such system. https://digital.asahi.com/articles/DA3S13491490.html There are 3 ways to support the new era. 1. H32 is not accepted and S90 is not accepted. (current new era support behavior) 2. H32 is accepted and S90 is not accepted. (current JDK behavior) 3. H32 is accepted and S90 is accepted. (to conformity) I prefer 2 not to change the current JDK behavior. With taking 2 or 3, we might have a new API like JapaneseDate.from(JapaneseEra, TemporalAccessor) to specify an era on making JapaneseDate. -- Naoki Kishdia From naoto.sato at oracle.com Thu Jun 14 23:00:45 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 14 Jun 2018 16:00:45 -0700 Subject: Japanese New Era supporting does not accept Heisei 32 In-Reply-To: References: Message-ID: <33afc91d-8a8d-e3fc-0105-9ba4ecd901e8@oracle.com> Hi, Yes, that's the expected behavior with the implementation assuming "Heisei 32" is non-existent. I see some discussion as you mentioned, whether to allow to use Heisei for a while, for transitional purpose, but at the moment there's very little & no official information available. We may consider adding an additional API to convert those dates leniently. Naoto On 6/14/18 5:13 AM, kishida naoki wrote: > I'm trying to use the Japanese new era supporting and have found that it > does not accept Heisei 32. > There are many document that include Heisei 32 or later at present. For > example, the expiration date of my driver license is Heisei 32 March. > JDK10 can accept Heisei 32. > I think the change of the behavior make some trouble. > > Additionally, Japanese government says some system use Heisei after > 5/1/2019 for a while. The JDK version that include the new era support can > not be used on such system. > https://digital.asahi.com/articles/DA3S13491490.html > > There are 3 ways to support the new era. > 1. H32 is not accepted and S90 is not accepted. (current new era support > behavior) > 2. H32 is accepted and S90 is not accepted. (current JDK behavior) > 3. H32 is accepted and S90 is accepted. (to conformity) > > I prefer 2 not to change the current JDK behavior. > > With taking 2 or 3, we might have a new API like > JapaneseDate.from(JapaneseEra, TemporalAccessor) to specify an era on > making JapaneseDate. > From stuart.marks at oracle.com Fri Jun 15 01:02:04 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 14 Jun 2018 18:02:04 -0700 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) Message-ID: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> Hi all, I wanted to restart review of the changeset for bug JDK-8060192 [1]. The latest webrev is here: [2]. The previous review was from December 2017: [3]. The only difference between the current changeset and the previous one is the removal of the overriding implementations (see explanation below). Much of the discussion in the previous review thread was around performance and the shape of the API. # Performance I previously had done some benchmarking on this and reported the results: [4]. (I've recently re-done the benchmarks and conclusions are essentially the same.) The upshot is that implementations that use Arrays.copyOf() are the fastest, probably because it's an intrinsic. It can avoid zero-filling the freshly allocated array because it knows the entire array contents will be overwritten. This is true regardless of what the public API looks like. The default implementation calls generator.apply(0) to get a zero-length array and then simply calls toArray(T[]). This goes through the Arrays.copyOf() fast path, so it's essentially the same speed as toArray(new T[0]). The overrides I had previously provided in specific implementation classes like ArrayList actually are slower, because the allocation of the array is done separately from filling it. This necessitates the extra zero-filling step. Thus, I've removed the overrides. # API Shape There was some discussion about whether it would be preferable to have a Class parameter as a type token for the array component type or the array type itself, instead of using an IntFunction generator. Most of this boils down to what people are comfortable with. However, there are a few points that make the generator function approach preferable. One point in favor of using a generator is that Stream already has a similar toArray(generator) method. Comparing this to the type token alternatives, for simple tasks like converting Collection to String[], things are about equal: toArray(String[]::new) toArray(String.class) toArray(String[].class) However, things are hairier if the element type of the collection is generic, such as Collection>. The result type is a generic array List[]. Using class literals or array constructor references will necessitate using an unchecked cast, because none of these can be used to represent a generic type. However, it's possible to use a method reference to provide a properly generic IntFunction. This would enable the toArray(generator) method to be used without any unchecked casts. (The method it refers to might need have an unchecked cast within it, though.) Such a method could also be reused for the corresponding Stream.toArray(generator) method. For these reasons I'd like to proceed with adding toArray(generator) API. Thanks, s'marks [1] https://bugs.openjdk.java.net/browse/JDK-8060192 [2] http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.3/ [3] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050325.html [4] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050585.html From srinivas.dama at oracle.com Fri Jun 15 04:45:52 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Thu, 14 Jun 2018 21:45:52 -0700 (PDT) Subject: RFR: 8204967: Resolve disabled warnings for libunpack Message-ID: <0df0b775-3ff8-49fa-81dd-0cfc3b16641e@default> Hi Magnus/Jim, Thank you for the comments. Here is the latest webrev with suggested changes: webrev: http://cr.openjdk.java.net/~sdama/8204967/webrev.01/ Bug: https://bugs.openjdk.java.net/browse/JDK-8204967 Jim, gnu c supports -Wimplicit-fallthrough=3. clang takes -Wimplicit-fallthrough without any parameters and ignores it without any effect. clang++ works with -Wimplicit-fallthrough along with -std=c++11(it is cpp specific option) Note: My earlier fix http://cr.openjdk.java.net/~sdama/8204967/webrev.00/ broke linux build because some switch case statements which are having implicit-fallthroughs will be enabled only in debug build(src/jdk.pack/share/native/common-unpack/unpack.cpp) Earlier,I missed to fix those so debug build failed in linux. Regards, Srinivas ----- Original Message ----- From: magnus.ihse.bursie at oracle.com To: srinivas.dama at oracle.com, core-libs-dev at openjdk.java.net Sent: Thursday, 14 June, 2018 3:23:23 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: RFR: 8204967: Resolve disabled warnings for libunpack On 2018-06-13 18:57, Srinivas Dama wrote: > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8204967/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8204967 Hi Srinivas, In src/jdk.pack/share/native/common-unpack/zip.cpp, you just read and discard the return value from fread to hide the warning. But the purpose of the warning is to point out that this kind of code is incorrect. The proper fix would be to check the return code from fread to make sure that the read did not fail. Otherwise you're just making it impossible for the compiler to point out the badly written code, and if you're not going to fix this properly, it's better to keep the warning (that way we know the code is fishy). /Magnus > > Regards, > Srinivas From patrick at reini.net Fri Jun 15 06:33:17 2018 From: patrick at reini.net (Patrick Reinhart) Date: Fri, 15 Jun 2018 08:33:17 +0200 Subject: RFR JDK-8204930 Reader:nullReader() spec does not match the behavior In-Reply-To: <97e913d2-c79f-2e1a-c2be-cb45815b78f2@oracle.com> References: <1AED6005-82BA-499E-8523-38D552DF6E4B@oracle.com> <3A88FBD3-6C6E-4F38-B11E-CE2D6A251D52@oracle.com> <315f23c1dddb0724908fd22134021316@reini.net> <97e913d2-c79f-2e1a-c2be-cb45815b78f2@oracle.com> Message-ID: <207c050f941fa12992cfe2fb57d7e98e@reini.net> On 2018-06-14 13:03, Alan Bateman wrote: > On 14/06/2018 11:50, Patrick Reinhart wrote: >> Hi everybody, >> >> I re-proposed the CSR with the changed API. > The CSR is associated with the original change-set that was pushed a > few months so the simplest is to just create a follow-up CSR that is > linked to the new bug. I created a follow-up CSR with relation to the existing one now. In the meantime I got a comment on the initial issue according the specification of the [ ready(), skip(), transferTo() ] where it seen to be unclear what is the expected behavior "if end of stream reached" exactly means. Any suggestions on that part? Should it state that ready() will always return false, skip() and transferTo() always return zero? -Patrick From forax at univ-mlv.fr Fri Jun 15 07:26:59 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 15 Jun 2018 09:26:59 +0200 (CEST) Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> References: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> Message-ID: <1255272034.1809068.1529047619362.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Stuart Marks" > ?: "core-libs-dev" > Envoy?: Vendredi 15 Juin 2018 03:02:04 > Objet: RFR(s): 8060192: Add default method Collection.toArray(generator) > Hi all, Hi Stuart, > > I wanted to restart review of the changeset for bug JDK-8060192 [1]. The latest > webrev is here: [2]. > > The previous review was from December 2017: [3]. The only difference between the > current changeset and the previous one is the removal of the overriding > implementations (see explanation below). Much of the discussion in the previous > review thread was around performance and the shape of the API. > > # Performance > > I previously had done some benchmarking on this and reported the results: [4]. > (I've recently re-done the benchmarks and conclusions are essentially the same.) > > The upshot is that implementations that use Arrays.copyOf() are the fastest, > probably because it's an intrinsic. It can avoid zero-filling the freshly > allocated array because it knows the entire array contents will be overwritten. > This is true regardless of what the public API looks like. The default > implementation calls generator.apply(0) to get a zero-length array and then > simply calls toArray(T[]). This goes through the Arrays.copyOf() fast path, so > it's essentially the same speed as toArray(new T[0]). > > The overrides I had previously provided in specific implementation classes like > ArrayList actually are slower, because the allocation of the array is done > separately from filling it. This necessitates the extra zero-filling step. Thus, > I've removed the overrides. for ArrayList, you may use a code like this, T[] toArray(IntFunction generator) { return (T[]) Arrays.copyOf(elementData, size, generator.apply(0).getClass()); } so you win only one comparison (yeah !), which can be perfectly predicted, so you should not see any perf difference :) > > # API Shape > > There was some discussion about whether it would be preferable to have a Class > parameter as a type token for the array component type or the array type itself, > instead of using an IntFunction generator. Most of this boils down to what > people are comfortable with. However, there are a few points that make the > generator function approach preferable. > > One point in favor of using a generator is that Stream already has a similar > toArray(generator) method. > > Comparing this to the type token alternatives, for simple tasks like converting > Collection to String[], things are about equal: > > toArray(String[]::new) > toArray(String.class) > toArray(String[].class) > > However, things are hairier if the element type of the collection is generic, > such as Collection>. The result type is a generic array > List[]. Using class literals or array constructor references will > necessitate using an unchecked cast, because none of these can be used to > represent a generic type. > > However, it's possible to use a method reference to provide a properly generic > IntFunction. This would enable the toArray(generator) method to be used without > any unchecked casts. (The method it refers to might need have an unchecked cast > within it, though.) Such a method could also be reused for the corresponding > Stream.toArray(generator) method. List.class or List[].class do not work either. > > For these reasons I'd like to proceed with adding toArray(generator) API. > so thumb up for me ! > Thanks, > > s'marks R?mi > > > [1] https://bugs.openjdk.java.net/browse/JDK-8060192 > > [2] http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.3/ > > [3] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050325.html > > [4] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050585.html From takiguc at linux.vnet.ibm.com Fri Jun 15 07:33:59 2018 From: takiguc at linux.vnet.ibm.com (Ichiroh Takiguchi) Date: Fri, 15 Jun 2018 16:33:59 +0900 Subject: Add x-IBM-1129 charset In-Reply-To: References: Message-ID: <1ebd2f6516ee598a3a0be2f1eee99b0e@linux.vnet.ibm.com> Hello. I tested IBM-1129 charset on AIX platform. It worked fine for default encoding, \$ LANG=Vi_VN ~/jdk/bin/java PrintDefaultCharset Vi_VN x-IBM1129 IBM-1129 IBM-1129 \$ And also code table is fine. We would appreciate to open a bug/RFE for this change request. Ichiroh Takiguchi IBM Japan, Ltd. On 2018-05-30 10:41, Nasser Ebrahim wrote: > Hello, > > I just realized that I missed to provide the details of the test I have > conducted for the new charset IBM-1129. Please note that I have done > the > following jtreg tests to make sure the new charset is working properly. > > sun.nio.cs.TestCharsetMapping > java.nio.charset.Charset.RegisteredCharsets > sun.nio.cs.CheckHistoricalNamesTest > java.nio.charset.RemovingSunIO.SunioAlias > > Please let me know if I have to run any additional tests. > > Am aware of the suggestion from Alan and Thomas in the mail thread > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/053316.html > to move the IBM platform specific charsets to a separate module instead > of > adding to jdk.charsets. We will open another thread to discuss that > topic > as we would like to contribute many more extended charsets. However, I > believe we can still consider integrating this charset without waiting > for > the conclusion of that discussion as this charset is part of default > charset for AIX and needs to be part of the java.base. If we decided to > move the IBM charsets out of jdk.charsets, we can move this charsets as > well for non-AIX platforms. > > Kindly request you to open a bug/RFE for this change request, review > the > fix and provide your feedback. > > Thank you, > Nasser Ebrahim > > > From: Nasser Ebrahim/India/IBM > To: core-libs-dev at openjdk.java.net > Date: 05/19/2018 12:51 PM > Subject: Add x-IBM-1129 charset > > > > Hello, > > With the following three bugs, all the default locale charsets except > two > (Vi_VN.IBM-1129 & ja_JP.IBM-eucJP) are fixed for AIX platform. > > - JDK-8201540: [AIX] Extend the set of supported charsets in java.base > - JDK-8202329: Codepage mappings for IBM-943 and Big5 (aix) > - > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-May/053050.html > : [AIX] Add charset IBM-964 (default charset for zh_TW.IBM-eucTW) to > stdcs > [bug not yet opened]. > > For those fixed charsets, the charsets were existing in the extended > charsets (jdk.charsets) and they were not working with default locale > charset as it did not exist in the standard charset (java.base). The > charsets correspond to the two pending locale (Vi_VN.IBM-1129 & > ja_JP.IBM-eucJP) does not exist in the jdk. They need to be added to > the > extended charsets before adding to stdcs on AIX platform. > > Here, am including the patch to fix the charset IBM-1129 for the > locale > Vi_VN.IBM-1129. We are working on the other missing charset (for > ja_JP.IBM-eucJP) which will be contributed in some time. > > The webrev of the fix is available at > http://cr.openjdk.java.net/~aleonard/IBM1129/webrev.00/ > > Kindly request you to open a bug and review the fix. Please let me know > if > you have any questions. > > Thank you, > Nasser Ebrahim From TOSHIONA at jp.ibm.com Fri Jun 15 09:10:14 2018 From: TOSHIONA at jp.ibm.com (Toshio 5 Nakamura) Date: Fri, 15 Jun 2018 18:10:14 +0900 Subject: JDK-8042131: Proposal of Mapped-values formatter for non-IsoChronology Message-ID: Hello core-libs and i18n folks, We'd like to request to reconsider JDK-8042131, "DateTimeFormatterBuilder Mapped-values do not work for JapaneseDate". The report was posted by our team long time ago, and was closed as not an issue. At that time, the feature was for only IsoChronology. Now, I found the attached patch can activate the mapped-values formatter for non-IsoChronology. Additionally, the Japanese new era will be expected in May 2019. The first year of each era has a special expression in Japanese, "U+5143 U+5e74" (Gannen). java.util.JapaneseImperialCalendar uses this expression. We'd like to use the expression with JapaneseChronology by mapped-values formatter as an option. Can we have a sponsor of this proposal? --sample usage of Japanese new era-- DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); Map yearMap = new HashMap<>(); yearMap.put(1L, "\u5143"); builder.appendText(ChronoField.ERA, TextStyle.FULL) .appendText(ChronoField.YEAR_OF_ERA, yearMap) .appendLiteral("\u5e74"); --------------- Report: https://bugs.openjdk.java.net/browse/JDK-8042131 Patch: --- old/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java 2018-06-15 17:39:11.489303979 +0900 +++ new/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java 2018-06-15 17:39:11.157303972 +0900 @@ -793,6 +793,11 @@ final LocaleStore store = new LocaleStore(map); DateTimeTextProvider provider = new DateTimeTextProvider() { @Override + public String getText(Chronology chrono, TemporalField field, + long value, TextStyle style, Locale locale) { + return store.getText(value, style); + } + @Override public String getText(TemporalField field, long value, TextStyle style, Locale locale) { return store.getText(value, style); } --- old/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java 2018-06-15 17:39:12.664304007 +0900 +++ new/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java 2018-06-15 17:39:12.298303999 +0900 @@ -67,14 +67,21 @@ import java.time.chrono.Chronology; import java.time.chrono.IsoChronology; import java.time.chrono.JapaneseChronology; +import java.time.chrono.JapaneseEra; import java.time.chrono.MinguoChronology; +import java.time.chrono.ThaiBuddhistChronology; +import java.time.chrono.ChronoLocalDate; import java.time.format.DateTimeFormatter; import java.time.format.DateTimeFormatterBuilder; import java.time.format.FormatStyle; import java.time.LocalDate; import java.time.temporal.Temporal; +import java.time.temporal.ChronoField; +import static java.time.temporal.ChronoUnit.YEARS; import java.util.Locale; +import java.util.Map; +import java.util.HashMap; import static org.testng.Assert.assertEquals; @@ -115,6 +122,31 @@ } //----------------------------------------------------------------------- + @DataProvider(name="mappedPatterns") + Object[][] localizedMappedPatterns() { + return new Object[][] { + {IsoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, + {JapaneseChronology.INSTANCE.date(JapaneseEra.HEISEI, + 1,1,8), Locale.ENGLISH}, + {MinguoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, + {ThaiBuddhistChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, + }; + } + + @Test(dataProvider="mappedPatterns") + public void test_getLocalizedMappedPattern(ChronoLocalDate date, Locale locale) { + final String new1st = "1st"; + Map yearMap = new HashMap<>(); + yearMap.put(1L, new1st); + DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); + builder.appendText(ChronoField.YEAR_OF_ERA, yearMap); + + String actual = date.format(builder.toFormatter(locale)); + assertEquals(actual, new1st); + } + + + //----------------------------------------------------------------------- @DataProvider(name="localePatterns") Object[][] localizedDateTimePatterns() { return new Object[][] { --- Best regards, Toshio Nakamura, IBM Japan From scolebourne at joda.org Fri Jun 15 09:30:44 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 15 Jun 2018 10:30:44 +0100 Subject: JDK-8042131: Proposal of Mapped-values formatter for non-IsoChronology In-Reply-To: References: Message-ID: >From the looks of it, this is a perfectly reasonable enhancement. Sadly I can't sponsor it as I'm not a committer. Stephen On 15 June 2018 at 10:10, Toshio 5 Nakamura wrote: > > Hello core-libs and i18n folks, > > We'd like to request to reconsider JDK-8042131, > "DateTimeFormatterBuilder Mapped-values do not work for JapaneseDate". > The report was posted by our team long time ago, and was closed as not an > issue. > At that time, the feature was for only IsoChronology. > > Now, I found the attached patch can activate the mapped-values formatter > for > non-IsoChronology. > > Additionally, the Japanese new era will be expected in May 2019. The first > year of > each era has a special expression in Japanese, "U+5143 U+5e74" (Gannen). > java.util.JapaneseImperialCalendar uses this expression. > We'd like to use the expression with JapaneseChronology by mapped-values > formatter as an option. > > Can we have a sponsor of this proposal? > > --sample usage of Japanese new era-- > DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); > Map yearMap = new HashMap<>(); > yearMap.put(1L, "\u5143"); > builder.appendText(ChronoField.ERA, TextStyle.FULL) > .appendText(ChronoField.YEAR_OF_ERA, yearMap) > .appendLiteral("\u5e74"); > --------------- > > Report: > https://bugs.openjdk.java.net/browse/JDK-8042131 > > Patch: > --- old/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java 2018-06-15 17:39:11.489303979 +0900 > +++ new/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java 2018-06-15 17:39:11.157303972 +0900 > @@ -793,6 +793,11 @@ > final LocaleStore store = new LocaleStore(map); > DateTimeTextProvider provider = new DateTimeTextProvider() { > @Override > + public String getText(Chronology chrono, TemporalField field, > + long value, TextStyle style, Locale locale) { > + return store.getText(value, style); > + } > + @Override > public String getText(TemporalField field, long value, TextStyle style, Locale locale) { > return store.getText(value, style); > } > --- old/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java 2018-06-15 17:39:12.664304007 +0900 > +++ new/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java 2018-06-15 17:39:12.298303999 +0900 > @@ -67,14 +67,21 @@ > import java.time.chrono.Chronology; > import java.time.chrono.IsoChronology; > import java.time.chrono.JapaneseChronology; > +import java.time.chrono.JapaneseEra; > import java.time.chrono.MinguoChronology; > +import java.time.chrono.ThaiBuddhistChronology; > +import java.time.chrono.ChronoLocalDate; > import java.time.format.DateTimeFormatter; > import java.time.format.DateTimeFormatterBuilder; > import java.time.format.FormatStyle; > import java.time.LocalDate; > import java.time.temporal.Temporal; > +import java.time.temporal.ChronoField; > +import static java.time.temporal.ChronoUnit.YEARS; > > import java.util.Locale; > +import java.util.Map; > +import java.util.HashMap; > > import static org.testng.Assert.assertEquals; > > @@ -115,6 +122,31 @@ > } > > //----------------------------------------------------------------------- > + @DataProvider(name="mappedPatterns") > + Object[][] localizedMappedPatterns() { > + return new Object[][] { > + {IsoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, > + {JapaneseChronology.INSTANCE.date(JapaneseEra.HEISEI, > + 1,1,8), Locale.ENGLISH}, > + {MinguoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, > + {ThaiBuddhistChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, > + }; > + } > + > + @Test(dataProvider="mappedPatterns") > + public void test_getLocalizedMappedPattern(ChronoLocalDate date, Locale locale) { > + final String new1st = "1st"; > + Map yearMap = new HashMap<>(); > + yearMap.put(1L, new1st); > + DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); > + builder.appendText(ChronoField.YEAR_OF_ERA, yearMap); > + > + String actual = date.format(builder.toFormatter(locale)); > + assertEquals(actual, new1st); > + } > + > + > + //----------------------------------------------------------------------- > @DataProvider(name="localePatterns") > Object[][] localizedDateTimePatterns() { > return new Object[][] { > > --- > Best regards, > Toshio Nakamura, IBM Japan From amaembo at gmail.com Fri Jun 15 11:26:15 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Fri, 15 Jun 2018 18:26:15 +0700 Subject: BiCollector In-Reply-To: <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> Message-ID: Hello! I created a preliminary webrev of my own implementation (no testcases yet): http://cr.openjdk.java.net/~tvaleev/patches/pairing/webrev/ If anybody wants to sponsor my implementation, I will happily log an issue and write tests. The name "pairing" was invented by me, but as I'm not a native English speaker I cannot judge whether it's optimal, so better ideas are welcome. Also I decided to remove accumulator types from public type variables. They do not add anything to type signature, only clutter it increasing the number of type parameters from 4 to 6. I think it was a mistake to expose the accumulator type parameter in other cascading collectors like filtering(), collectingAndThen(), groupingBy(), etc. I'm not insisting though, if you feel that conformance to existing collectors is more important than simplicity. With best regards, Tagir Valeev. On Fri, Jun 15, 2018 at 5:05 AM Brian Goetz wrote: > > > Well, I don't see the need to pack the two results into a Map.Entry > > (or any similar) container as a drawback. > > From an "integrity of the JDK APIs" perspective, it is unquestionably a > drawback. These items are not a Key and an associated Value in a Map; > it's merely pretending that Map.Entry really means "Pair". There's a > reason we don't have a Pair class in the JDK (and no, let's not reopen > that now); using something else as a Pair proxy that is supposed to have > specific semantics is worse. (It's fine to do this in your own code, but > not in the JDK. Different standards for code that has different audiences.) > > Tagir's proposed sidestepping is nice, and it will also play nicely with > records, because then you can say: > > record NameAndCount(String name, int count); > > stream.collect(pairing(collectName, collectCount, > NameAndCount::new)); > > and get a more properly abstract result out. And its more in the spirit > of existing Collectors. If you want to use Map.Entry as an > _implementation detail_, that's fine. > > I can support this form. > > > I also don't see a larger abstraction like BiStream as a natural fit > > for a similar thing. > > I think the BiStream connection is mostly tangential. We tried hard to > support streams of (K,V) pairs when we did streams, as Paul can attest, > but it was a huge complexity-inflater and dropping this out paid an > enormous simplification payoff. > > With records, having streams of tuples will be simpler to represent, but > no more performant; it will take until we get to value types and > specialized generics to get the performance we want out of this. > > > From amaembo at gmail.com Fri Jun 15 11:31:51 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Fri, 15 Jun 2018 18:31:51 +0700 Subject: BiCollector In-Reply-To: References: Message-ID: Peter, > EnumSet intersection = EnumSet.copyOf(keyCollector.characteristics()); > intersection.retainAll(valCollector.characteristics()); > return intersection; Please note that `copyOf` should either receive an EnumSet or non-empty Collection to obtain the enum class. This is not guaranteed by `characteristics()` implementation, thus failure is possible. Testcase: new BiCollector<>(Collectors.counting(), Collectors.toSet()).characteristics(); Fails with Exception in thread "main" java.lang.IllegalArgumentException: Collection is empty at java.base/java.util.EnumSet.copyOf(EnumSet.java:173) at BiCollector.characteristics(BiCollector.java:77) at Main.main(Main.java:21) With best regards, Tagir Valeev. On Mon, Jun 11, 2018 at 7:40 PM Peter Levart wrote: > Hi, > > Have you ever wanted to perform a collection of the same Stream into two > different targets using two Collectors? Say you wanted to collect > Map.Entry elements into two parallel lists, each of them containing keys > and values respectively. Or you wanted to collect elements into groups > by some key, but also count them at the same time? Currently this is not > possible to do with a single Stream. You have to create two identical > streams, so you end up passing Supplier to other methods instead > of bare Stream. > > I created a little utility Collector implementation that serves the > purpose quite well: > > /** > * A {@link Collector} implementation taking two delegate Collector(s) > and producing result composed > * of two results produced by delegating collectors, wrapped in {@link > Map.Entry} object. > * > * @param the type of elements collected > * @param the type of 1st delegate collector collected result > * @param tye type of 2nd delegate collector collected result > */ > public class BiCollector implements Collector Map.Entry, Map.Entry> { > private final Collector keyCollector; > private final Collector valCollector; > > @SuppressWarnings("unchecked") > public BiCollector(Collector keyCollector, Collector V> valCollector) { > this.keyCollector = (Collector) > Objects.requireNonNull(keyCollector); > this.valCollector = (Collector) > Objects.requireNonNull(valCollector); > } > > @Override > public Supplier> supplier() { > Supplier keySupplier = keyCollector.supplier(); > Supplier valSupplier = valCollector.supplier(); > return () -> new > AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), valSupplier.get()); > } > > @Override > public BiConsumer, T> accumulator() { > BiConsumer keyAccumulator = keyCollector.accumulator(); > BiConsumer valAccumulator = valCollector.accumulator(); > return (accumulation, t) -> { > keyAccumulator.accept(accumulation.getKey(), t); > valAccumulator.accept(accumulation.getValue(), t); > }; > } > > @Override > public BinaryOperator> combiner() { > BinaryOperator keyCombiner = keyCollector.combiner(); > BinaryOperator valCombiner = valCollector.combiner(); > return (accumulation1, accumulation2) -> new > AbstractMap.SimpleImmutableEntry<>( > keyCombiner.apply(accumulation1.getKey(), > accumulation2.getKey()), > valCombiner.apply(accumulation1.getValue(), > accumulation2.getValue()) > ); > } > > @Override > public Function, Map.Entry> > finisher() { > Function keyFinisher = keyCollector.finisher(); > Function valFinisher = valCollector.finisher(); > return accumulation -> new AbstractMap.SimpleImmutableEntry<>( > keyFinisher.apply(accumulation.getKey()), > valFinisher.apply(accumulation.getValue()) > ); > } > > @Override > public Set characteristics() { > EnumSet intersection = > EnumSet.copyOf(keyCollector.characteristics()); > intersection.retainAll(valCollector.characteristics()); > return intersection; > } > } > > > Do you think this class is general enough to be part of standard > Collectors repertoire? > > For example, accessed via factory method Collectors.toBoth(Collector > coll1, Collector coll2), bi-collection could then be coded simply as: > > Map map = ... > > Map.Entry, List> keys_values = > map.entrySet() > .stream() > .collect( > toBoth( > mapping(Map.Entry::getKey, toList()), > mapping(Map.Entry::getValue, toList()) > ) > ); > > > Map.Entry, Long> histogram_count = > ThreadLocalRandom > .current() > .ints(100, 0, 10) > .boxed() > .collect( > toBoth( > groupingBy(Function.identity(), counting()), > counting() > ) > ); > > > Regards, Peter > > From james.laskey at oracle.com Fri Jun 15 11:56:54 2018 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 15 Jun 2018 08:56:54 -0300 Subject: RFR: 8204967: Resolve disabled warnings for libunpack In-Reply-To: <0df0b775-3ff8-49fa-81dd-0cfc3b16641e@default> References: <0df0b775-3ff8-49fa-81dd-0cfc3b16641e@default> Message-ID: <5CA9827A-76CF-4F8F-8292-058BA4F3E443@oracle.com> +1 > On Jun 15, 2018, at 1:45 AM, Srinivas Dama wrote: > > Hi Magnus/Jim, > > Thank you for the comments. > Here is the latest webrev with suggested changes: > webrev: http://cr.openjdk.java.net/~sdama/8204967/webrev.01/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8204967 > > Jim, > gnu c supports -Wimplicit-fallthrough=3. > clang takes -Wimplicit-fallthrough without any parameters and ignores it without any effect. > clang++ works with -Wimplicit-fallthrough along with -std=c++11(it is cpp specific option) > > Note: My earlier fix http://cr.openjdk.java.net/~sdama/8204967/webrev.00/ broke linux > build because some switch case statements which are having implicit-fallthroughs will be enabled > only in debug build(src/jdk.pack/share/native/common-unpack/unpack.cpp) > Earlier,I missed to fix those so debug build failed in linux. > > Regards, > Srinivas > > ----- Original Message ----- > From: magnus.ihse.bursie at oracle.com > To: srinivas.dama at oracle.com, core-libs-dev at openjdk.java.net > Sent: Thursday, 14 June, 2018 3:23:23 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi > Subject: Re: RFR: 8204967: Resolve disabled warnings for libunpack > > On 2018-06-13 18:57, Srinivas Dama wrote: >> Hi, >> >> Please review http://cr.openjdk.java.net/~sdama/8204967/webrev.00/ >> for https://bugs.openjdk.java.net/browse/JDK-8204967 > > Hi Srinivas, > > In src/jdk.pack/share/native/common-unpack/zip.cpp, you just read and > discard the return value from fread to hide the warning. But the purpose > of the warning is to point out that this kind of code is incorrect. The > proper fix would be to check the return code from fread to make sure > that the read did not fail. Otherwise you're just making it impossible > for the compiler to point out the badly written code, and if you're not > going to fix this properly, it's better to keep the warning (that way we > know the code is fishy). > > /Magnus > >> >> Regards, >> Srinivas > From roger.riggs at oracle.com Fri Jun 15 14:38:01 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Fri, 15 Jun 2018 10:38:01 -0400 Subject: JDK-8042131: Proposal of Mapped-values formatter for non-IsoChronology In-Reply-To: References: Message-ID: Hi, A good suggestion to reconsider, I re-opened the issue. I have not looked closely to see if/where the spec needs to be changed. Thanks, Roger On 6/15/18 5:30 AM, Stephen Colebourne wrote: > From the looks of it, this is a perfectly reasonable enhancement. > Sadly I can't sponsor it as I'm not a committer. > > Stephen > > > On 15 June 2018 at 10:10, Toshio 5 Nakamura wrote: >> Hello core-libs and i18n folks, >> >> We'd like to request to reconsider JDK-8042131, >> "DateTimeFormatterBuilder Mapped-values do not work for JapaneseDate". >> The report was posted by our team long time ago, and was closed as not an >> issue. >> At that time, the feature was for only IsoChronology. >> >> Now, I found the attached patch can activate the mapped-values formatter >> for >> non-IsoChronology. >> >> Additionally, the Japanese new era will be expected in May 2019. The first >> year of >> each era has a special expression in Japanese, "U+5143 U+5e74" (Gannen). >> java.util.JapaneseImperialCalendar uses this expression. >> We'd like to use the expression with JapaneseChronology by mapped-values >> formatter as an option. >> >> Can we have a sponsor of this proposal? >> >> --sample usage of Japanese new era-- >> DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); >> Map yearMap = new HashMap<>(); >> yearMap.put(1L, "\u5143"); >> builder.appendText(ChronoField.ERA, TextStyle.FULL) >> .appendText(ChronoField.YEAR_OF_ERA, yearMap) >> .appendLiteral("\u5e74"); >> --------------- >> >> Report: >> https://bugs.openjdk.java.net/browse/JDK-8042131 >> >> Patch: >> --- old/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java 2018-06-15 17:39:11.489303979 +0900 >> +++ new/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java 2018-06-15 17:39:11.157303972 +0900 >> @@ -793,6 +793,11 @@ >> final LocaleStore store = new LocaleStore(map); >> DateTimeTextProvider provider = new DateTimeTextProvider() { >> @Override >> + public String getText(Chronology chrono, TemporalField field, >> + long value, TextStyle style, Locale locale) { >> + return store.getText(value, style); >> + } >> + @Override >> public String getText(TemporalField field, long value, TextStyle style, Locale locale) { >> return store.getText(value, style); >> } >> --- old/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java 2018-06-15 17:39:12.664304007 +0900 >> +++ new/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java 2018-06-15 17:39:12.298303999 +0900 >> @@ -67,14 +67,21 @@ >> import java.time.chrono.Chronology; >> import java.time.chrono.IsoChronology; >> import java.time.chrono.JapaneseChronology; >> +import java.time.chrono.JapaneseEra; >> import java.time.chrono.MinguoChronology; >> +import java.time.chrono.ThaiBuddhistChronology; >> +import java.time.chrono.ChronoLocalDate; >> import java.time.format.DateTimeFormatter; >> import java.time.format.DateTimeFormatterBuilder; >> import java.time.format.FormatStyle; >> import java.time.LocalDate; >> import java.time.temporal.Temporal; >> +import java.time.temporal.ChronoField; >> +import static java.time.temporal.ChronoUnit.YEARS; >> >> import java.util.Locale; >> +import java.util.Map; >> +import java.util.HashMap; >> >> import static org.testng.Assert.assertEquals; >> >> @@ -115,6 +122,31 @@ >> } >> >> //----------------------------------------------------------------------- >> + @DataProvider(name="mappedPatterns") >> + Object[][] localizedMappedPatterns() { >> + return new Object[][] { >> + {IsoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, >> + {JapaneseChronology.INSTANCE.date(JapaneseEra.HEISEI, >> + 1,1,8), Locale.ENGLISH}, >> + {MinguoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, >> + {ThaiBuddhistChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, >> + }; >> + } >> + >> + @Test(dataProvider="mappedPatterns") >> + public void test_getLocalizedMappedPattern(ChronoLocalDate date, Locale locale) { >> + final String new1st = "1st"; >> + Map yearMap = new HashMap<>(); >> + yearMap.put(1L, new1st); >> + DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); >> + builder.appendText(ChronoField.YEAR_OF_ERA, yearMap); >> + >> + String actual = date.format(builder.toFormatter(locale)); >> + assertEquals(actual, new1st); >> + } >> + >> + >> + //----------------------------------------------------------------------- >> @DataProvider(name="localePatterns") >> Object[][] localizedDateTimePatterns() { >> return new Object[][] { >> >> --- >> Best regards, >> Toshio Nakamura, IBM Japan From martinrb at google.com Fri Jun 15 14:57:04 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 15 Jun 2018 07:57:04 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal In-Reply-To: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> References: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> Message-ID: Code looks correct to me, but as usual there are some things I would try to do differently, 1292 while (word != 0 && tz < 63) { I'd try to make this loop simply while (word != 0) --- Probably neither of us is fond of the bug magnet special case for Integer.MAX_VALUE. Maybe just store fence as a long or as a wordindex which can't overflow (while giving up on intra-word splitting?) Here's a fun program demonstrating cardinality overflow: import java.util.BitSet; public class BitSetCardinality { public static void main(String[] args) throws Throwable { BitSet s = new BitSet(); s.flip(0, Integer.MAX_VALUE); System.out.println(s.cardinality()); s.flip(Integer.MAX_VALUE); System.out.println(s.cardinality()); } } ==> java -Xmx20g -esa -ea BitSetCardinality 2147483647 -2147483648 On Thu, Jun 14, 2018 at 2:35 PM, Paul Sandoz wrote: > Hi, > > Please review this enhancement to improve the performance of BitSet > traversal when using a stream. > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8170159- > bitset-traverse/webrev/ psandoz/jdk/JDK-8170159-bitset-traverse/webrev/> > > The associated issue started out life referring to the BitSet?s > spliterator reporting SIZED, and to report that the cardinality has to be > computed. This has some cost but performance analysis (see issue for > attached benchmark) has shown that the cost is small compared to the cost > of traversal. It is recognized that the cost is higher for smaller BitSets > but not unduly so. Thus it was concluded that it was reasonable for the > spliterator to report SIZED. > > The issue was adjusted to focus on improving the performance of the > BitSet?s spliterator forEachRemaining method. For large randomized BitSets > a 1.6x speedup can be observed, and for smaller sizes a more modest speed > up. The prior conclusion about reporting SIZED still holds. > > Thanks, > Paul. From brent.christian at oracle.com Fri Jun 15 15:44:31 2018 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 15 Jun 2018 08:44:31 -0700 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map Message-ID: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> Hi, In JDK 9, 8029891[1] refactored java.util.Properties to store its values in an internal ConcurrentHashMap, and removed synchronization from "reader" methods in order to avoid potential hangs/deadlocks during classloading. Claes has noticed that there is the possibility of the new 'map' field being observed with its default value (null), before being set. After looking at the JSR 133 FAQ[2], I agree with Claes that we should make 'map' a field final. Please review my change to do this: Webrev: http://cr.openjdk.java.net/~bchristi/8199435/webrev/ Issue: https://bugs.openjdk.java.net/browse/JDK-8199435 Thanks, -Brent 1. https://bugs.openjdk.java.net/browse/JDK-8029891 2. https://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html#finalRight From naoto.sato at oracle.com Fri Jun 15 16:04:13 2018 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 15 Jun 2018 09:04:13 -0700 Subject: JDK-8042131: Proposal of Mapped-values formatter for non-IsoChronology In-Reply-To: References: Message-ID: It does seem to be a good change. I will take a look and sponsor it. Naoto On 6/15/18 7:38 AM, Roger Riggs wrote: > Hi, > > A good suggestion to reconsider, I re-opened the issue. > > I have not looked closely to see if/where the spec needs to be changed. > > Thanks, Roger > > > On 6/15/18 5:30 AM, Stephen Colebourne wrote: >> ?From the looks of it, this is a perfectly reasonable enhancement. >> Sadly I can't sponsor it as I'm not a committer. >> >> Stephen >> >> >> On 15 June 2018 at 10:10, Toshio 5 Nakamura wrote: >>> Hello core-libs and i18n folks, >>> >>> We'd like to request to reconsider JDK-8042131, >>> "DateTimeFormatterBuilder Mapped-values do not work for JapaneseDate". >>> The report was posted by our team long time ago, and was closed as >>> not an >>> issue. >>> At that time, the feature was for only IsoChronology. >>> >>> Now, I found the attached patch can activate the mapped-values formatter >>> for >>> non-IsoChronology. >>> >>> Additionally, the Japanese new era will be expected in May 2019. The >>> first >>> year of >>> each era has a special expression in Japanese, "U+5143 U+5e74" (Gannen). >>> java.util.JapaneseImperialCalendar uses this expression. >>> We'd like to use the expression with JapaneseChronology by mapped-values >>> formatter as an option. >>> >>> Can we have a sponsor of this proposal? >>> >>> --sample usage of Japanese new era-- >>> DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); >>> Map yearMap = new HashMap<>(); >>> yearMap.put(1L, "\u5143"); >>> builder.appendText(ChronoField.ERA, TextStyle.FULL) >>> ???????????? .appendText(ChronoField.YEAR_OF_ERA, yearMap) >>> ???????????? .appendLiteral("\u5e74"); >>> --------------- >>> >>> Report: >>> https://bugs.openjdk.java.net/browse/JDK-8042131 >>> >>> Patch: >>> --- >>> old/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java >>> 2018-06-15 17:39:11.489303979 +0900 >>> +++ >>> new/src/java.base/share/classes/java/time/format/DateTimeFormatterBuilder.java >>> 2018-06-15 17:39:11.157303972 +0900 >>> @@ -793,6 +793,11 @@ >>> ????????? final LocaleStore store = new LocaleStore(map); >>> ????????? DateTimeTextProvider provider = new DateTimeTextProvider() { >>> ????????????? @Override >>> +??????????? public String getText(Chronology chrono, TemporalField >>> field, >>> +????????????????????????????????? long value, TextStyle style, >>> Locale locale) { >>> +??????????????? return store.getText(value, style); >>> +??????????? } >>> +??????????? @Override >>> ????????????? public String getText(TemporalField field, long value, >>> TextStyle style, Locale locale) { >>> ????????????????? return store.getText(value, style); >>> ????????????? } >>> --- >>> old/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java >>> 2018-06-15 17:39:12.664304007 +0900 >>> +++ >>> new/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java >>> 2018-06-15 17:39:12.298303999 +0900 >>> @@ -67,14 +67,21 @@ >>> ? import java.time.chrono.Chronology; >>> ? import java.time.chrono.IsoChronology; >>> ? import java.time.chrono.JapaneseChronology; >>> +import java.time.chrono.JapaneseEra; >>> ? import java.time.chrono.MinguoChronology; >>> +import java.time.chrono.ThaiBuddhistChronology; >>> +import java.time.chrono.ChronoLocalDate; >>> ? import java.time.format.DateTimeFormatter; >>> ? import java.time.format.DateTimeFormatterBuilder; >>> ? import java.time.format.FormatStyle; >>> ? import java.time.LocalDate; >>> ? import java.time.temporal.Temporal; >>> +import java.time.temporal.ChronoField; >>> +import static java.time.temporal.ChronoUnit.YEARS; >>> >>> ? import java.util.Locale; >>> +import java.util.Map; >>> +import java.util.HashMap; >>> >>> ? import static org.testng.Assert.assertEquals; >>> >>> @@ -115,6 +122,31 @@ >>> ????? } >>> >>> >>> //----------------------------------------------------------------------- >>> >>> +??? @DataProvider(name="mappedPatterns") >>> +??? Object[][] localizedMappedPatterns() { >>> +??????? return new Object[][] { >>> +??????????? {IsoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, >>> +??????????? {JapaneseChronology.INSTANCE.date(JapaneseEra.HEISEI, >>> +????????????????????????????????????????????? 1,1,8), Locale.ENGLISH}, >>> +??????????? {MinguoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, >>> +??????????? {ThaiBuddhistChronology.INSTANCE.date(1,1,1), >>> Locale.ENGLISH}, >>> +??????? }; >>> +??? } >>> + >>> +??? @Test(dataProvider="mappedPatterns") >>> +??? public void test_getLocalizedMappedPattern(ChronoLocalDate date, >>> Locale locale) { >>> +??????? final String new1st = "1st"; >>> +??????? Map yearMap = new HashMap<>(); >>> +??????? yearMap.put(1L, new1st); >>> +??????? DateTimeFormatterBuilder builder = new >>> DateTimeFormatterBuilder(); >>> +??????? builder.appendText(ChronoField.YEAR_OF_ERA, yearMap); >>> + >>> +??????? String actual = date.format(builder.toFormatter(locale)); >>> +??????? assertEquals(actual, new1st); >>> +??? } >>> + >>> + >>> + >>> //----------------------------------------------------------------------- >>> >>> ????? @DataProvider(name="localePatterns") >>> ????? Object[][] localizedDateTimePatterns() { >>> ????????? return new Object[][] { >>> >>> --- >>> Best regards, >>> Toshio Nakamura, IBM Japan > From paul.sandoz at oracle.com Fri Jun 15 17:17:53 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 10:17:53 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal In-Reply-To: References: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> Message-ID: <0D15FDB3-123E-464F-A926-6A01EA24AFAD@oracle.com> Hi Martin, Thanks for reviewing. > On Jun 15, 2018, at 7:57 AM, Martin Buchholz wrote: > > Code looks correct to me, but as usual there are some things I would try to do differently, > Indeed, i tried a few different approaches before settling on this code shape. > 1292 while (word != 0 && tz < 63) { > > I'd try to make this loop simply > while (word != 0) > I started out like that but in the end preferred that both loop conditions were upfront, to avoid another break or explicitly setting word to 0, then the only break is the weird one to deal with the annoying bug magnet :-) > --- > > Probably neither of us is fond of the bug magnet special case for Integer.MAX_VALUE. Maybe just store fence as a long or as a wordindex which can't overflow (while giving up on intra-word splitting?) > > Here's a fun program demonstrating cardinality overflow: > > import java.util.BitSet; > > public class BitSetCardinality { > public static void main(String[] args) throws Throwable { > BitSet s = new BitSet(); > s.flip(0, Integer.MAX_VALUE); > System.out.println(s.cardinality()); > s.flip(Integer.MAX_VALUE); > System.out.println(s.cardinality()); > } > } > > > ==> java -Xmx20g -esa -ea BitSetCardinality > 2147483647 > -2147483648 > Oh dear! Thanks, Paul. > > On Thu, Jun 14, 2018 at 2:35 PM, Paul Sandoz > wrote: > Hi, > > Please review this enhancement to improve the performance of BitSet traversal when using a stream. > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8170159-bitset-traverse/webrev/ > > > The associated issue started out life referring to the BitSet?s spliterator reporting SIZED, and to report that the cardinality has to be computed. This has some cost but performance analysis (see issue for attached benchmark) has shown that the cost is small compared to the cost of traversal. It is recognized that the cost is higher for smaller BitSets but not unduly so. Thus it was concluded that it was reasonable for the spliterator to report SIZED. > > The issue was adjusted to focus on improving the performance of the BitSet?s spliterator forEachRemaining method. For large randomized BitSets a 1.6x speedup can be observed, and for smaller sizes a more modest speed up. The prior conclusion about reporting SIZED still holds. > > Thanks, > Paul. > From cushon at google.com Fri Jun 15 17:21:17 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Fri, 15 Jun 2018 10:21:17 -0700 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: References: <1f87dd21-701b-975c-a1c1-f1d9ed2f1eaf@oracle.com> <5A90B204.4070005@oracle.com> <5A960359.4070507@oracle.com> Message-ID: Thanks, I pushed it as: http://hg.openjdk.java.net/jdk/jdk/rev/9f4c08c444e8 On Thu, Jun 14, 2018 at 10:31 AM Vicente Romero wrote: > I'm not an expert in the area, but the patch looks good to me, > > Vicente > > On 05/31/2018 08:39 PM, Liam Miller-Cushon wrote: > > Hi, > > > > Are there any concerns with the patch that I can address? I was hoping to > > get this in to JDK 11. > > > > I confirmed that it still builds and passes all tests at head. > > > > Thanks, > > Liam > > > > On Mon, May 14, 2018 at 10:38 AM Liam Miller-Cushon > > wrote: > > > >> Ping > >> > >> On Thu, Apr 5, 2018 at 10:57 AM Liam Miller-Cushon > >> wrote: > >> > >>> Hi, > >>> > >>> Is there any more feedback on the patch? > >>> > >>> On Wed, Feb 28, 2018 at 4:26 PM Liam Miller-Cushon > >>> wrote: > >>> > >>>> On Wed, Feb 28, 2018 at 4:13 PM, Martin Buchholz > > >>>> wrote: > >>>> > >>>>> Follow their lead and rename your method to assertArrayEquals > >>>>> > >>>> Done: http://cr.openjdk.java.net/~cushon/7183985/webrev.04/ > >>>> > > From paul.sandoz at oracle.com Fri Jun 15 17:35:20 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 10:35:20 -0700 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> Message-ID: > On Jun 15, 2018, at 8:44 AM, Brent Christian wrote: > > Hi, > > In JDK 9, 8029891[1] refactored java.util.Properties to store its values in an internal ConcurrentHashMap, and removed synchronization from "reader" methods in order to avoid potential hangs/deadlocks during classloading. > > Claes has noticed that there is the possibility of the new 'map' field being observed with its default value (null), before being set. > Yes. I would also publish the map in readHashtable after you have placed in the keys/values. Pail. > After looking at the JSR 133 FAQ[2], I agree with Claes that we should make 'map' a field final. > > Please review my change to do this: > > Webrev: > http://cr.openjdk.java.net/~bchristi/8199435/webrev/ > Issue: > https://bugs.openjdk.java.net/browse/JDK-8199435 > > Thanks, > -Brent > > 1. https://bugs.openjdk.java.net/browse/JDK-8029891 > 2. https://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html#finalRight From martinrb at google.com Fri Jun 15 17:43:04 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 15 Jun 2018 10:43:04 -0700 Subject: Durations in existing JDK APIs In-Reply-To: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: On Wed, May 30, 2018 at 11:32 AM, Doug Lea wrote: > > The original rationale for designing j.u.c.TimeUnit using the Flyweight > pattern was to to reduce allocation and GC-related overhead and timing > jitter for methods that otherwise may operate on the order of > nanoseconds. But there are many cases in which this is not much of a > concern (plus JVMs can now sometimes optimize), so people should be > given a choice. It would be a lot of tedious work (and aggregate code > bulk) to retrofit every time-related j.u.c method though, and it's not > clear where to compromise. But at least adding converters should not be > controversial. > Re-reading Doug's assessment, Doug seems reluctant but open to adding at least some Duration overloads. Here's an obvious first candidate in Future (yes, we have a test that discovers get(Duration) and checks that get(null) throws UOE instead of NPE.). (It's a lot of tedious work) a/util/concurrent/CompletableFuture.java =================================================================== RCS file: /export/home/jsr166/jsr166/jsr166/src/main/java/util/concurrent/CompletableFuture.java,v retrieving revision 1.212 diff -u -r1.212 CompletableFuture.java --- src/main/java/util/concurrent/CompletableFuture.java 11 Mar 2018 18:00:05 -0000 1.212 +++ src/main/java/util/concurrent/CompletableFuture.java 15 Jun 2018 17:39:09 -0000 @@ -1983,13 +1983,22 @@ * while waiting * @throws TimeoutException if the wait timed out */ - @SuppressWarnings("unchecked") public T get(long timeout, TimeUnit unit) throws InterruptedException, ExecutionException, TimeoutException { - long nanos = unit.toNanos(timeout); + return getNanos(unit.toNanos(timeout)); + } + + public T get(java.time.Duration timeout) + throws InterruptedException, ExecutionException, TimeoutException { + return getNanos(TimeUnit.NANOSECONDS.convert(timeout)); + } + + @SuppressWarnings("unchecked") + private T getNanos(long timeoutNanos) + throws InterruptedException, ExecutionException, TimeoutException { Object r; if ((r = result) == null) - r = timedGet(nanos); + r = timedGet(timeoutNanos); return (T) reportGet(r); } @@ -2797,6 +2806,8 @@ throw new UnsupportedOperationException(); } @Override public T get(long timeout, TimeUnit unit) { throw new UnsupportedOperationException(); } + @Override public T get(java.time.Duration duration) { + throw new UnsupportedOperationException(); } @Override public T getNow(T valueIfAbsent) { throw new UnsupportedOperationException(); } @Override public T join() { Index: src/main/java/util/concurrent/Future.java =================================================================== RCS file: /export/home/jsr166/jsr166/jsr166/src/main/java/util/concurrent/Future.java,v retrieving revision 1.41 diff -u -r1.41 Future.java --- src/main/java/util/concurrent/Future.java 8 Oct 2016 18:52:37 -0000 1.41 +++ src/main/java/util/concurrent/Future.java 15 Jun 2018 17:39:09 -0000 @@ -6,6 +6,10 @@ package java.util.concurrent; +import static java.util.concurrent.TimeUnit.NANOSECONDS; + +import java.time.Duration; + /** * A {@code Future} represents the result of an asynchronous * computation. Methods are provided to check if the computation is @@ -126,7 +130,30 @@ * @throws InterruptedException if the current thread was interrupted * while waiting * @throws TimeoutException if the wait timed out + * @throws NullPointerException if {@code unit} is null */ V get(long timeout, TimeUnit unit) throws InterruptedException, ExecutionException, TimeoutException; + + /** + * Waits if necessary for at most the given time for the computation + * to complete, and then retrieves its result, if available. + * + * Equivalent to: {@code + * get(NANOSECONDS.convert(timeout), NANOSECONDS)} + * + * @param timeout the maximum time to wait + * @return the computed result + * @throws CancellationException if the computation was cancelled + * @throws ExecutionException if the computation threw an + * exception + * @throws InterruptedException if the current thread was interrupted + * while waiting + * @throws TimeoutException if the wait timed out + * @throws NullPointerException if {@code timeout} is null + */ + default V get(Duration timeout) + throws InterruptedException, ExecutionException, TimeoutException { + return get(NANOSECONDS.convert(timeout), NANOSECONDS); + } } From cushon at google.com Fri Jun 15 17:58:07 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Fri, 15 Jun 2018 10:58:07 -0700 Subject: RFR 8198669: Refactor annotation array value parsing to reduce duplication Message-ID: Hello, This change is a follow-up to JDK-7183985 which replaces the interior of parseClassArray, parseEnumArray, and parseAnnotationArray with a shared method that is passed a lambda for the type-specific parsing logic. bug: https://bugs.openjdk.java.net/browse/JDK-8198669 webrev: http://cr.openjdk.java.net/~cushon/8198669/webrev.00/ Liam From brent.christian at oracle.com Fri Jun 15 18:01:51 2018 From: brent.christian at oracle.com (Brent Christian) Date: Fri, 15 Jun 2018 11:01:51 -0700 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> Message-ID: <9fe90d21-0604-97ff-2abb-a940a0268c87@oracle.com> On 6/15/18 10:35 AM, Paul Sandoz wrote: > > I would also publish the map in readHashtable after you have placed in the keys/values. > Like this? // create CHM of appropriate capacity - map = new ConcurrentHashMap<>(elements); + ConcurrentHashMap tmpMap = new ConcurrentHashMap<>(elements); // Read all the key/value objects for (; elements > 0; elements--) { Object key = s.readObject(); Object value = s.readObject(); - map.put(key, value); + tmpMap.put(key, value); } + + UNSAFE.putObjectVolatile(this, MAP_OFFSET, tmpMap); } -B From martinrb at google.com Fri Jun 15 18:05:42 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 15 Jun 2018 11:05:42 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal In-Reply-To: <0D15FDB3-123E-464F-A926-6A01EA24AFAD@oracle.com> References: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> <0D15FDB3-123E-464F-A926-6A01EA24AFAD@oracle.com> Message-ID: It took me a while to realize why you were checking tz < 63. It's because the shift by 64 that might occur below would be a no-op! 1301 action.accept(i++);1302 word &= (WORD_MASK << i); Can we rewrite to simply flip the one bit via word &= ~(1L << i); action.accept(i++); and then we can simplify the tz handling? From paul.sandoz at oracle.com Fri Jun 15 18:37:41 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 11:37:41 -0700 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <9fe90d21-0604-97ff-2abb-a940a0268c87@oracle.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <9fe90d21-0604-97ff-2abb-a940a0268c87@oracle.com> Message-ID: Yes, like that, thanks, Paul. > On Jun 15, 2018, at 11:01 AM, Brent Christian wrote: > > On 6/15/18 10:35 AM, Paul Sandoz wrote: >> I would also publish the map in readHashtable after you have placed in the keys/values. > > Like this? > > // create CHM of appropriate capacity > - map = new ConcurrentHashMap<>(elements); > + ConcurrentHashMap tmpMap = new ConcurrentHashMap<>(elements); > > // Read all the key/value objects > for (; elements > 0; elements--) { > Object key = s.readObject(); > Object value = s.readObject(); > - map.put(key, value); > + tmpMap.put(key, value); > } > + > + UNSAFE.putObjectVolatile(this, MAP_OFFSET, tmpMap); > } > > -B From paul.sandoz at oracle.com Fri Jun 15 20:13:51 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 13:13:51 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal In-Reply-To: References: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> <0D15FDB3-123E-464F-A926-6A01EA24AFAD@oracle.com> Message-ID: <5E1E7437-5F17-4BF6-8090-615F46F413B7@oracle.com> Thanks! Doh, obvious in hindsight. > On Jun 15, 2018, at 11:05 AM, Martin Buchholz wrote: > > It took me a while to realize why you were checking tz < 63. It's because the shift by 64 that might occur below would be a no-op! Right! > 1301 action.accept(i++); > 1302 word &= (WORD_MASK << i); > Can we rewrite to simply flip the one bit via > > word &= ~(1L << i); > action.accept(i++); > > and then we can simplify the tz handling? > Yes, and there is no need to increment i: http://cr.openjdk.java.net/~psandoz/jdk/JDK-8170159-bitset-traverse/webrev/src/java.base/share/classes/java/util/BitSet.java.sdiff.html Paul. From paul.sandoz at oracle.com Fri Jun 15 20:43:55 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 13:43:55 -0700 Subject: RFR 8198669: Refactor annotation array value parsing to reduce duplication In-Reply-To: References: Message-ID: <850CD01B-E498-407A-BC10-4EC7905D6FCC@oracle.com> +1 A nice cleanup. Did you run any tiered tests just to double check that the use of lambdas causes no bootstrapping issues? (i suspect it probably does not). Paul. > On Jun 15, 2018, at 10:58 AM, Liam Miller-Cushon wrote: > > Hello, > > This change is a follow-up to JDK-7183985 which replaces the interior of > parseClassArray, parseEnumArray, and parseAnnotationArray with a shared > method that is passed a lambda for the type-specific parsing logic. > > bug: https://bugs.openjdk.java.net/browse/JDK-8198669 > webrev: http://cr.openjdk.java.net/~cushon/8198669/webrev.00/ > > Liam From stuart.marks at oracle.com Fri Jun 15 21:22:25 2018 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 15 Jun 2018 14:22:25 -0700 Subject: Proposal: optimization of Map.copyOf and Collectors.toUnmodifiableMap In-Reply-To: References: Message-ID: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> Hi Peter, Finally getting back to this. I think there's clearly room for improvement in the creation of the unmodifiable collections. Right now the creation paths (mostly) use only public APIs, which can result in unnecessary allocation and copying. Certainly Map.copyOf does this, but there are also other cases as well. For copying from an unknown map, there are a couple approaches: * accumulate keys and values in a single ArrayList, which can be presized as necessary, but which will grow if necessary; then copy elements from a subrange of ArrayList's internal array (similar to your MapN(Object[], len) overload) * accumulate keys and values into a SpinedBuffer, which doesn't require copying to grow, which is preferable if for some reason we can't pre-size accurately; and then copy the elements out of it The Collectors.toUnmodifiableMap implementations are known to create HashMap instances, so they could pass the HashMap directly to a private MapN constructor that in turn could talk directly to HashMap to get the keys and values. This avoids allocation of a full-sized buffer and one copy. Note that these techniques involve creating new interfaces, sometimes that cross package boundaries. It's a bit of an irritant to have to plumb new paths that go through SharedSecrets, but it seems likely to be worthwhile if we can avoid bulk allocation and copying steps. Given these, it doesn't seem to me that the BiBuffer approach helps very much. I think there are many other avenues that would be worthwhile to explore, and that possibly can provide bigger savings. s'marks On 6/11/18 3:57 AM, Peter Levart wrote: > Hi, > > Those two methods were added in JDK 10 and they are not very optimal. > Map.copyOf(Map map) 1st dumps the source map into an array of Map.Entry(s) > (map.entrySet().toArray(new Entry[0])), which typically creates new Map.Entry > objects, then passes the array to Map.ofEntries(Map.Entry[] entries) factory > method that iterates the array and constructs a key/value interleaved array from > it which is passed to ImmutableCollections.MapN constructor, which then > constructs a linear-probing hash table from it. So each key and value is copied > 3 times, while several temporary objects are created in the process. > > One copying step could be eliminated and construction of temporary Map.Entry > objects too: > > http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.01/ > > Collecting stream(s) using Collectors.toUnmodifiableMap() 1st collects key/value > pairs into a HashMap, then it performs equivalent operations as > Map.copyOf(hashMap) at the end. Using Map.copyOf() directly benefits this > collection operation too. > > The following benchmark: > > http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapBench.java > > > > Shows up to 30% improvement of .copyOf operation with this patch applied: > > Original: > > Benchmark?????????????????????????????? (size)? Mode Cnt????? Score Error? Units > UnmodifiableMapBench.copyOf???????????????? 10? avgt 10??? 403.633 ? 2.640? ns/op > UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 3489.623 ? 44.590? ns/op > UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 40030.572 ? 277.075 > ns/op > UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10??? 831.221 ? 3.816? ns/op > UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9783.519 ? 43.097? ns/op > UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 96524.536 ? 670.818 > ns/op > > Patched: > > Benchmark?????????????????????????????? (size)? Mode Cnt????? Score Error? Units > UnmodifiableMapBench.copyOf???????????????? 10? avgt 10??? 264.172 ? 1.882? ns/op > UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 2318.974 ? 15.877? ns/op > UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 29291.782 ? 3139.737 > ns/op > UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10??? 771.221 ? 65.432? ns/op > UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9275.016 ? 725.722? ns/op > UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 82204.342 ? 851.741 > ns/op > > > Production of garbage is also reduced, since no Map.Entry temporary objects are > constructed: > > Original: > > Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units > UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10??? 416.001 ? > 0.002??? B/op > UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10 2936.005 ? > 0.019??? B/op > UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10 28136.059 ? > 0.199??? B/op > UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10? avgt 10 > 1368.001 ????? 0.004??? B/op > UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100? avgt 10 > 10208.139 ????? 0.045??? B/op > UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000? avgt 10 > 93025.923 ????? 0.573??? B/op > > Patched: > > Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units > UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10??? 304.000 ? > 0.001??? B/op > UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10 2464.004 ? > 0.012??? B/op > UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10 24064.040 ? > 0.137??? B/op > UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10? avgt 10 > 1256.001 ????? 0.003??? B/op > UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100? avgt 10 > 9720.153 ????? 0.055??? B/op > UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000? avgt 10 > 88905.688 ????? 0.574??? B/op > > > So what do you think? Is this an improvement? > > Regards, Peter > From martinrb at google.com Fri Jun 15 21:24:19 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 15 Jun 2018 14:24:19 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal In-Reply-To: <5E1E7437-5F17-4BF6-8090-615F46F413B7@oracle.com> References: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> <0D15FDB3-123E-464F-A926-6A01EA24AFAD@oracle.com> <5E1E7437-5F17-4BF6-8090-615F46F413B7@oracle.com> Message-ID: Still looks correct, but I might keep looking for something more elegant. What bothers me a little now is 1290 long word = words[u] & (WORD_MASK << i); which is part of the initial setup and is not necessary on each iteration. On Fri, Jun 15, 2018 at 1:13 PM, Paul Sandoz wrote: > Thanks! Doh, obvious in hindsight. > > On Jun 15, 2018, at 11:05 AM, Martin Buchholz wrote: > > It took me a while to realize why you were checking tz < 63. It's because > the shift by 64 that might occur below would be a no-op! > > > Right! > > 1301 action.accept(i++);1302 word &= (WORD_MASK << i); > > Can we rewrite to simply flip the one bit via > > word &= ~(1L << i); > action.accept(i++); > > and then we can simplify the tz handling? > > > Yes, and there is no need to increment i: > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8170159- > bitset-traverse/webrev/src/java.base/share/classes/java/ > util/BitSet.java.sdiff.html > > Paul. > From cushon at google.com Fri Jun 15 21:31:10 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Fri, 15 Jun 2018 14:31:10 -0700 Subject: RFR 8198669: Refactor annotation array value parsing to reduce duplication In-Reply-To: <850CD01B-E498-407A-BC10-4EC7905D6FCC@oracle.com> References: <850CD01B-E498-407A-BC10-4EC7905D6FCC@oracle.com> Message-ID: On Fri, Jun 15, 2018 at 1:44 PM Paul Sandoz wrote: > A nice cleanup. Did you run any tiered tests just to double check that the > use of lambdas causes no bootstrapping issues? (i suspect it probably does > not). Thanks, I ran `run-test-tier1` and everything looks fine, are there other tests I should check? From paul.sandoz at oracle.com Fri Jun 15 21:52:54 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 14:52:54 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal In-Reply-To: References: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> <0D15FDB3-123E-464F-A926-6A01EA24AFAD@oracle.com> <5E1E7437-5F17-4BF6-8090-615F46F413B7@oracle.com> Message-ID: > On Jun 15, 2018, at 2:24 PM, Martin Buchholz wrote: > > Still looks correct, but I might keep looking for something more elegant. > What bothers me a little now is > > 1290 long word = words[u] & (WORD_MASK << i); > > which is part of the initial setup and is not necessary on each iteration. > I was looking at this too. From a performance perspective i am not very concerned. Aesthetically it bothers me, but at this point i am willing to declare victory and patch later if there is a better way. Paul. From paul.sandoz at oracle.com Fri Jun 15 22:01:55 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 15:01:55 -0700 Subject: RFR 8198669: Refactor annotation array value parsing to reduce duplication In-Reply-To: References: <850CD01B-E498-407A-BC10-4EC7905D6FCC@oracle.com> Message-ID: <9C96329D-192C-46BD-AA37-685B444939E0@oracle.com> > On Jun 15, 2018, at 2:31 PM, Liam Miller-Cushon wrote: > > On Fri, Jun 15, 2018 at 1:44 PM Paul Sandoz > wrote: > A nice cleanup. Did you run any tiered tests just to double check that the use of lambdas causes no bootstrapping issues? (i suspect it probably does not). > > Thanks, I ran `run-test-tier1` and everything looks fine, are there other tests I should check? > tier1 should be ok. Thanks, Paul. From martinrb at google.com Fri Jun 15 22:03:35 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 15 Jun 2018 15:03:35 -0700 Subject: RFR 8170159 Improve the performance of BitSet traversal In-Reply-To: References: <6A349279-B0D3-4EFC-A228-9B86700C9020@oracle.com> <0D15FDB3-123E-464F-A926-6A01EA24AFAD@oracle.com> <5E1E7437-5F17-4BF6-8090-615F46F413B7@oracle.com> Message-ID: +1 On Fri, Jun 15, 2018 at 2:52 PM, Paul Sandoz wrote: > > > On Jun 15, 2018, at 2:24 PM, Martin Buchholz wrote: > > Still looks correct, but I might keep looking for something more elegant. > What bothers me a little now is > > 1290 long word = words[u] & (WORD_MASK << i); > > > which is part of the initial setup and is not necessary on each iteration. > > > I was looking at this too. From a performance perspective i am not very > concerned. Aesthetically it bothers me, but at this point i am willing to > declare victory and patch later if there is a better way. > > Paul. > From naoto.sato at oracle.com Fri Jun 15 22:14:47 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Fri, 15 Jun 2018 15:14:47 -0700 Subject: [11] RFR: 8042131: DateTimeFormatterBuilder Mapped-values do not work for JapaneseDate Message-ID: <9018eeda-d891-adf8-81b7-ef1d1db08281@oracle.com> Hello, Please review the fix to the following issue: https://bugs.openjdk.java.net/browse/JDK-8042131 The proposed change is located at: http://cr.openjdk.java.net/~naoto/8042131/webrev.00/ The fix is originally contributedy by Toshio Nakamura at IBM [1]. I am sponsoring the fix, with some trivial modifications to the test (diff is attached). Naoto [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053830.html -------------- next part -------------- diff -r 9b997bfc60d5 test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java --- a/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java +++ b/test/jdk/java/time/test/java/time/format/TestDateTimeFormatterBuilderWithLocale.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 2012, 2016, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -64,24 +64,23 @@ */ package test.java.time.format; +import java.time.chrono.ChronoLocalDate; import java.time.chrono.Chronology; import java.time.chrono.IsoChronology; import java.time.chrono.JapaneseChronology; import java.time.chrono.JapaneseEra; import java.time.chrono.MinguoChronology; import java.time.chrono.ThaiBuddhistChronology; -import java.time.chrono.ChronoLocalDate; import java.time.format.DateTimeFormatter; import java.time.format.DateTimeFormatterBuilder; import java.time.format.FormatStyle; import java.time.LocalDate; +import java.time.temporal.ChronoField; import java.time.temporal.Temporal; -import java.time.temporal.ChronoField; -import static java.time.temporal.ChronoUnit.YEARS; +import java.util.HashMap; import java.util.Locale; import java.util.Map; -import java.util.HashMap; import static org.testng.Assert.assertEquals; @@ -122,23 +121,21 @@ } //----------------------------------------------------------------------- - @DataProvider(name="mappedPatterns") - Object[][] localizedMappedPatterns() { + @DataProvider(name="mapTextLookup") + Object[][] data_mapTextLookup() { return new Object[][] { {IsoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, - {JapaneseChronology.INSTANCE.date(JapaneseEra.HEISEI, - 1,1,8), Locale.ENGLISH}, + {JapaneseChronology.INSTANCE.date(JapaneseEra.HEISEI, 1, 1, 8), Locale.ENGLISH}, {MinguoChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, {ThaiBuddhistChronology.INSTANCE.date(1,1,1), Locale.ENGLISH}, }; } - @Test(dataProvider="mappedPatterns") - public void test_getLocalizedMappedPattern(ChronoLocalDate date, Locale locale) { + @Test(dataProvider="mapTextLookup") + public void test_appendText_mapTextLookup(ChronoLocalDate date, Locale locale) { final String new1st = "1st"; Map yearMap = new HashMap<>(); yearMap.put(1L, new1st); - DateTimeFormatterBuilder builder = new DateTimeFormatterBuilder(); builder.appendText(ChronoField.YEAR_OF_ERA, yearMap); String actual = date.format(builder.toFormatter(locale)); From paul.sandoz at oracle.com Fri Jun 15 23:36:29 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jun 2018 16:36:29 -0700 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> Message-ID: <81C31CF0-8F14-47D4-B9E4-BA862D4F9DA9@oracle.com> Hi Tagir, This is looking good. My current favorite name for the factory method is ?bisecting? since we are dividing the collector into two collectors, the results of which are then merged. Suggested first paragraph of the factory method: "Returns a collector that passes the input elements to two specified collectors and merges their results with the specified merge function.? Paul. > On Jun 15, 2018, at 4:26 AM, Tagir Valeev wrote: > > Hello! > > I created a preliminary webrev of my own implementation (no testcases yet): > http://cr.openjdk.java.net/~tvaleev/patches/pairing/webrev/ > If anybody wants to sponsor my implementation, I will happily log an issue and write tests. > > The name "pairing" was invented by me, but as I'm not a native English speaker I cannot judge whether it's optimal, so better ideas are welcome. > Also I decided to remove accumulator types from public type variables. They do not add anything to type signature, only clutter it > increasing the number of type parameters from 4 to 6. I think it was a mistake to expose the accumulator type parameter in other cascading collectors > like filtering(), collectingAndThen(), groupingBy(), etc. I'm not insisting though, if you feel that conformance to existing collectors is > more important than simplicity. > > With best regards, > Tagir Valeev. > > On Fri, Jun 15, 2018 at 5:05 AM Brian Goetz wrote: > > > Well, I don't see the need to pack the two results into a Map.Entry > > (or any similar) container as a drawback. > > From an "integrity of the JDK APIs" perspective, it is unquestionably a > drawback. These items are not a Key and an associated Value in a Map; > it's merely pretending that Map.Entry really means "Pair". There's a > reason we don't have a Pair class in the JDK (and no, let's not reopen > that now); using something else as a Pair proxy that is supposed to have > specific semantics is worse. (It's fine to do this in your own code, but > not in the JDK. Different standards for code that has different audiences.) > > Tagir's proposed sidestepping is nice, and it will also play nicely with > records, because then you can say: > > record NameAndCount(String name, int count); > > stream.collect(pairing(collectName, collectCount, NameAndCount::new)); > > and get a more properly abstract result out. And its more in the spirit > of existing Collectors. If you want to use Map.Entry as an > _implementation detail_, that's fine. > > I can support this form. > > > I also don't see a larger abstraction like BiStream as a natural fit > > for a similar thing. > > I think the BiStream connection is mostly tangential. We tried hard to > support streams of (K,V) pairs when we did streams, as Paul can attest, > but it was a huge complexity-inflater and dropping this out paid an > enormous simplification payoff. > > With records, having streams of tuples will be simpler to represent, but > no more performant; it will take until we get to value types and > specialized generics to get the performance we want out of this. > > From amaembo at gmail.com Sat Jun 16 04:01:35 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Sat, 16 Jun 2018 11:01:35 +0700 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> References: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> Message-ID: Hello! If you are going to create long-living array which is likely to be empty, it's good to deduplicate them to spare the heap space. This can be easily done via existing toArray method like collection.toArray(MyClass.EMPTY_ARRAY) assuming we have an empty array constant. We use this pattern very often in IntelliJ IDEA source code (e. g. think of method which returns an array of Java member annotations; often there are none). New generator methods is much less convenient for this: toArray(s -> s == 0?MyClass.EMPTY_ARRAY:new MyClass[s]). I'm actually missing a method accepting an empty array instead of generator in the Stream API. And I don't think that new collection method is useful. The existing one is perfect. With best regards, Tagir Valeev ??, 15 ???? 2018 ?., 8:03 Stuart Marks : > Hi all, > > I wanted to restart review of the changeset for bug JDK-8060192 [1]. The > latest > webrev is here: [2]. > > The previous review was from December 2017: [3]. The only difference > between the > current changeset and the previous one is the removal of the overriding > implementations (see explanation below). Much of the discussion in the > previous > review thread was around performance and the shape of the API. > > # Performance > > I previously had done some benchmarking on this and reported the results: > [4]. > (I've recently re-done the benchmarks and conclusions are essentially the > same.) > > The upshot is that implementations that use Arrays.copyOf() are the > fastest, > probably because it's an intrinsic. It can avoid zero-filling the freshly > allocated array because it knows the entire array contents will be > overwritten. > This is true regardless of what the public API looks like. The default > implementation calls generator.apply(0) to get a zero-length array and > then > simply calls toArray(T[]). This goes through the Arrays.copyOf() fast > path, so > it's essentially the same speed as toArray(new T[0]). > > The overrides I had previously provided in specific implementation classes > like > ArrayList actually are slower, because the allocation of the array is done > separately from filling it. This necessitates the extra zero-filling step. > Thus, > I've removed the overrides. > > # API Shape > > There was some discussion about whether it would be preferable to have a > Class > parameter as a type token for the array component type or the array type > itself, > instead of using an IntFunction generator. Most of this boils down to what > people are comfortable with. However, there are a few points that make the > generator function approach preferable. > > One point in favor of using a generator is that Stream already has a > similar > toArray(generator) method. > > Comparing this to the type token alternatives, for simple tasks like > converting > Collection to String[], things are about equal: > > toArray(String[]::new) > toArray(String.class) > toArray(String[].class) > > However, things are hairier if the element type of the collection is > generic, > such as Collection>. The result type is a generic array > List[]. Using class literals or array constructor references will > necessitate using an unchecked cast, because none of these can be used to > represent a generic type. > > However, it's possible to use a method reference to provide a properly > generic > IntFunction. This would enable the toArray(generator) method to be used > without > any unchecked casts. (The method it refers to might need have an unchecked > cast > within it, though.) Such a method could also be reused for the > corresponding > Stream.toArray(generator) method. > > For these reasons I'd like to proceed with adding toArray(generator) API. > > Thanks, > > s'marks > > > [1] https://bugs.openjdk.java.net/browse/JDK-8060192 > > [2] http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.3/ > > [3] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050325.html > > [4] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050585.html > > From scolebourne at joda.org Sat Jun 16 09:33:04 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Sat, 16 Jun 2018 10:33:04 +0100 Subject: Durations in existing JDK APIs In-Reply-To: References: <1236a3de-dd83-b2f6-c9e2-c80ee4603203@cs.oswego.edu> Message-ID: Without commenting on the proposed implementation below, Future seems like a good place to start. I suspect that ExecutorService and ScheduledExecutorService would also be good targets to enhance. Stephen On Fri, 15 Jun 2018, 18:43 Martin Buchholz, wrote: > > > On Wed, May 30, 2018 at 11:32 AM, Doug Lea wrote: > >> >> The original rationale for designing j.u.c.TimeUnit using the Flyweight >> pattern was to to reduce allocation and GC-related overhead and timing >> jitter for methods that otherwise may operate on the order of >> nanoseconds. But there are many cases in which this is not much of a >> concern (plus JVMs can now sometimes optimize), so people should be >> given a choice. It would be a lot of tedious work (and aggregate code >> bulk) to retrofit every time-related j.u.c method though, and it's not >> clear where to compromise. But at least adding converters should not be >> controversial. >> > > Re-reading Doug's assessment, Doug seems reluctant but open to adding at > least some Duration overloads. Here's an obvious first candidate in Future > (yes, we have a test that discovers get(Duration) and checks that > get(null) throws UOE instead of NPE.). > (It's a lot of tedious work) > > a/util/concurrent/CompletableFuture.java > =================================================================== > RCS file: > /export/home/jsr166/jsr166/jsr166/src/main/java/util/concurrent/CompletableFuture.java,v > retrieving revision 1.212 > diff -u -r1.212 CompletableFuture.java > --- src/main/java/util/concurrent/CompletableFuture.java 11 Mar 2018 > 18:00:05 -0000 1.212 > +++ src/main/java/util/concurrent/CompletableFuture.java 15 Jun 2018 > 17:39:09 -0000 > @@ -1983,13 +1983,22 @@ > * while waiting > * @throws TimeoutException if the wait timed out > */ > - @SuppressWarnings("unchecked") > public T get(long timeout, TimeUnit unit) > throws InterruptedException, ExecutionException, TimeoutException > { > - long nanos = unit.toNanos(timeout); > + return getNanos(unit.toNanos(timeout)); > + } > + > + public T get(java.time.Duration timeout) > + throws InterruptedException, ExecutionException, TimeoutException > { > + return getNanos(TimeUnit.NANOSECONDS.convert(timeout)); > + } > + > + @SuppressWarnings("unchecked") > + private T getNanos(long timeoutNanos) > + throws InterruptedException, ExecutionException, TimeoutException > { > Object r; > if ((r = result) == null) > - r = timedGet(nanos); > + r = timedGet(timeoutNanos); > return (T) reportGet(r); > } > > @@ -2797,6 +2806,8 @@ > throw new UnsupportedOperationException(); } > @Override public T get(long timeout, TimeUnit unit) { > throw new UnsupportedOperationException(); } > + @Override public T get(java.time.Duration duration) { > + throw new UnsupportedOperationException(); } > @Override public T getNow(T valueIfAbsent) { > throw new UnsupportedOperationException(); } > @Override public T join() { > Index: src/main/java/util/concurrent/Future.java > =================================================================== > RCS file: > /export/home/jsr166/jsr166/jsr166/src/main/java/util/concurrent/Future.java,v > retrieving revision 1.41 > diff -u -r1.41 Future.java > --- src/main/java/util/concurrent/Future.java 8 Oct 2016 18:52:37 -0000 > 1.41 > +++ src/main/java/util/concurrent/Future.java 15 Jun 2018 17:39:09 -0000 > @@ -6,6 +6,10 @@ > > package java.util.concurrent; > > +import static java.util.concurrent.TimeUnit.NANOSECONDS; > + > +import java.time.Duration; > + > /** > * A {@code Future} represents the result of an asynchronous > * computation. Methods are provided to check if the computation is > @@ -126,7 +130,30 @@ > * @throws InterruptedException if the current thread was interrupted > * while waiting > * @throws TimeoutException if the wait timed out > + * @throws NullPointerException if {@code unit} is null > */ > V get(long timeout, TimeUnit unit) > throws InterruptedException, ExecutionException, TimeoutException; > + > + /** > + * Waits if necessary for at most the given time for the computation > + * to complete, and then retrieves its result, if available. > + * > + * Equivalent to: {@code > + * get(NANOSECONDS.convert(timeout), NANOSECONDS)} > + * > + * @param timeout the maximum time to wait > + * @return the computed result > + * @throws CancellationException if the computation was cancelled > + * @throws ExecutionException if the computation threw an > + * exception > + * @throws InterruptedException if the current thread was interrupted > + * while waiting > + * @throws TimeoutException if the wait timed out > + * @throws NullPointerException if {@code timeout} is null > + */ > + default V get(Duration timeout) > + throws InterruptedException, ExecutionException, TimeoutException > { > + return get(NANOSECONDS.convert(timeout), NANOSECONDS); > + } > } > > From Alan.Bateman at oracle.com Sat Jun 16 12:42:52 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 16 Jun 2018 13:42:52 +0100 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> Message-ID: <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> On 13/06/2018 15:10, Roger Riggs wrote: > Hi Joe, > > The CSR text is still in draft and got out of sync with the open > review and comments. > > The updated apiNote clarifies that changes made through the > System.*property*? methods > may not affect the value cached during initialization. > The implementation changes affect a very short list of properties. > > The note on the methods is to raise awareness that individual > properties may have > different behavior and may be unpredictable. Just catching up on this. I checked the CSR and the webrev (the latest webrev was generated on June 6 so I hope that is the version we are meant to look at). The updated apiNote (CSR version) looks okay but I think the word "internal" needs to be dropped as it comes with too many questions. "may be cached during initialization or ..." should be okay. The editing has meant the line lengths are a bit inconsistent so I assume they can be fixed up before the change is pushed. I see the original patch to SocksSocketImpl has been mostly reverted. It looks correct now. The other usages look okay to me. -Alan From peter.levart at gmail.com Sun Jun 17 08:50:58 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 17 Jun 2018 10:50:58 +0200 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> References: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> Message-ID: <940f6307-34fd-eea0-88e8-c8f5134b6829@gmail.com> Hi Stuart, On 06/15/18 03:02, Stuart Marks wrote: > Comparing this to the type token alternatives, for simple tasks like > converting Collection to String[], things are about equal: > > ??? toArray(String[]::new) > ??? toArray(String.class) > ??? toArray(String[].class) > > However, things are hairier if the element type of the collection is > generic, such as Collection>. The result type is a > generic array List[]. Using class literals or array > constructor references will necessitate using an unchecked cast, > because none of these can be used to represent a generic type. > > However, it's possible to use a method reference to provide a properly > generic IntFunction. This would enable the toArray(generator) method > to be used without any unchecked casts. (The method it refers to might > need have an unchecked cast within it, though.) Such a method could > also be reused for the corresponding Stream.toArray(generator) method. > > For these reasons I'd like to proceed with adding toArray(generator) API. It's a little strange that the generator function is used to construct an empty array (at least in the default method, but overrides will likely do the same if they want to avoid pre-zeroing overhead) in order to just extract the array's type from it. Someone could reasonably expect the provided function to be called with the exact length of needed array. The Collection.toArray(T[]) at least gives user an option to actually fully use the provided array, if user provides the correct length. The argument about using (and re-using) a method so that a method reference can be passed to the method without any unchecked casts is equally true for any of the 3 alternatives shown - the method itself might need unchecked casts, but its usage not: static List[] array(int len) static Class> elementType() static Class[]> arrayType() But I can see that you want to align the API with Stream.toArray, while still providing the optimal implementation. It's just that the API doesn't fit the usecase. The function approach makes more sense in the Stream API where it is explicitly explained that it may be used to construct arrays needed to hold intermediary results of partitioned parallel execution too, but in Collection API it is usually the case to just provide a copy of the underlying data structure in the most optimal way (without pre-zeroing overhead) and for that purpose, 2nd and 3rd alternatives are a better fit. Suppose Stream took a different approach and used the 2nd or 3rd approach (which is universal). Would then Collection API also get the same method? Regards, Peter From peter.levart at gmail.com Sun Jun 17 09:08:58 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 17 Jun 2018 11:08:58 +0200 Subject: BiCollector In-Reply-To: References: Message-ID: <1140f3ba-dad8-4809-97ac-63a461f61d7b@gmail.com> On 06/15/18 13:31, Tagir Valeev wrote: > Peter, > > > ? ? ? ?EnumSet intersection = > EnumSet.copyOf(keyCollector.characteristics()); > > ???????? intersection.retainAll(valCollector.characteristics()); > > ???????? return intersection; > > Please note that `copyOf` should either receive an EnumSet or > non-empty Collection to obtain the enum class. This is not guaranteed > by `characteristics()` implementation, thus failure is possible. > > Testcase: > new BiCollector<>(Collectors.counting(), > Collectors.toSet()).characteristics(); > Fails with > Exception in thread "main" java.lang.IllegalArgumentException: > Collection is empty > at java.base/java.util.EnumSet.copyOf(EnumSet.java:173) > at BiCollector.characteristics(BiCollector.java:77) > at Main.main(Main.java:21) Yeah, it came to my mind after I posted the example implementation. A case where API design sometimes misleads even experienced programmers, because it re-uses an idiom that is usually right (for most other Set(s)), but fails for that particular case. EnumSet needs an explicit enum type, so it should have been provided as an explicit argument... Regards, Peter > > > With best regards, > Tagir Valeev. > > On Mon, Jun 11, 2018 at 7:40 PM Peter Levart > wrote: > > Hi, > > Have you ever wanted to perform a collection of the same Stream > into two > different targets using two Collectors? Say you wanted to collect > Map.Entry elements into two parallel lists, each of them > containing keys > and values respectively. Or you wanted to collect elements into? > groups > by some key, but also count them at the same time? Currently this > is not > possible to do with a single Stream. You have to create two identical > streams, so you end up passing Supplier to other methods > instead > of bare Stream. > > I created a little utility Collector implementation that serves the > purpose quite well: > > /** > ??* A {@link Collector} implementation taking two delegate > Collector(s) > and producing result composed > ??* of two results produced by delegating collectors, wrapped in > {@link > Map.Entry} object. > ??* > ??* @param the type of elements collected > ??* @param the type of 1st delegate collector collected result > ??* @param tye type of 2nd delegate collector collected result > ??*/ > public class BiCollector implements Collector Map.Entry, Map.Entry> { > ???? private final Collector keyCollector; > ???? private final Collector valCollector; > > ???? @SuppressWarnings("unchecked") > ???? public BiCollector(Collector keyCollector, > Collector V> valCollector) { > ???????? this.keyCollector = (Collector) > Objects.requireNonNull(keyCollector); > ???????? this.valCollector = (Collector) > Objects.requireNonNull(valCollector); > ???? } > > ???? @Override > ???? public Supplier> supplier() { > ???????? Supplier keySupplier = keyCollector.supplier(); > ???????? Supplier valSupplier = valCollector.supplier(); > ???????? return () -> new > AbstractMap.SimpleImmutableEntry<>(keySupplier.get(), > valSupplier.get()); > ???? } > > ???? @Override > ???? public BiConsumer, T> accumulator() { > ???????? BiConsumer keyAccumulator = > keyCollector.accumulator(); > ???????? BiConsumer valAccumulator = > valCollector.accumulator(); > ???????? return (accumulation, t) -> { > ???????????? keyAccumulator.accept(accumulation.getKey(), t); > ???????????? valAccumulator.accept(accumulation.getValue(), t); > ???????? }; > ???? } > > ???? @Override > ???? public BinaryOperator> combiner() { > ???????? BinaryOperator keyCombiner = keyCollector.combiner(); > ???????? BinaryOperator valCombiner = valCollector.combiner(); > ???????? return (accumulation1, accumulation2) -> new > AbstractMap.SimpleImmutableEntry<>( > ???????????? keyCombiner.apply(accumulation1.getKey(), > accumulation2.getKey()), > ???????????? valCombiner.apply(accumulation1.getValue(), > accumulation2.getValue()) > ???????? ); > ???? } > > ???? @Override > ???? public Function, Map.Entry> > finisher() { > ???????? Function keyFinisher = keyCollector.finisher(); > ???????? Function valFinisher = valCollector.finisher(); > ???????? return accumulation -> new > AbstractMap.SimpleImmutableEntry<>( > ???????????? keyFinisher.apply(accumulation.getKey()), > ???????????? valFinisher.apply(accumulation.getValue()) > ???????? ); > ???? } > > ???? @Override > ???? public Set characteristics() { > ???????? EnumSet intersection = > EnumSet.copyOf(keyCollector.characteristics()); > intersection.retainAll(valCollector.characteristics()); > ???????? return intersection; > ???? } > } > > > Do you think this class is general enough to be part of standard > Collectors repertoire? > > For example, accessed via factory method Collectors.toBoth(Collector > coll1, Collector coll2), bi-collection could then be coded simply as: > > ???????? Map map = ... > > ???????? Map.Entry, List> keys_values = > ???????????? map.entrySet() > ??????????????? .stream() > ??????????????? .collect( > ??????????????????? toBoth( > ??????????????????????? mapping(Map.Entry::getKey, toList()), > ??????????????????????? mapping(Map.Entry::getValue, toList()) > ??????????????????? ) > ??????????????? ); > > > ???????? Map.Entry, Long> histogram_count = > ???????????? ThreadLocalRandom > ???????????????? .current() > ???????????????? .ints(100, 0, 10) > ???????????????? .boxed() > ???????????????? .collect( > ???????????????????? toBoth( > ???????????????????????? groupingBy(Function.identity(), counting()), > ???????????????????????? counting() > ???????????????????? ) > ???????????????? ); > > > Regards, Peter > From forax at univ-mlv.fr Sun Jun 17 09:15:37 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Sun, 17 Jun 2018 11:15:37 +0200 (CEST) Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: References: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> Message-ID: <396353523.2274681.1529226937272.JavaMail.zimbra@u-pem.fr> Hi Tagir, ----- Mail original ----- > De: "Tagir Valeev" > ?: "Stuart Marks" > Cc: "core-libs-dev" > Envoy?: Samedi 16 Juin 2018 06:01:35 > Objet: Re: RFR(s): 8060192: Add default method Collection.toArray(generator) > Hello! > > If you are going to create long-living array which is likely to be empty, > it's good to deduplicate them to spare the heap space. This can be easily > done via existing toArray method like > collection.toArray(MyClass.EMPTY_ARRAY) assuming we have an empty array > constant. We use this pattern very often in IntelliJ IDEA source code (e. > g. think of method which returns an array of Java member annotations; often > there are none). New generator methods is much less convenient for this: > toArray(s -> s == 0?MyClass.EMPTY_ARRAY:new MyClass[s]). I'm actually > missing a method accepting an empty array instead of generator in the > Stream API. And I don't think that new collection method is useful. The > existing one is perfect. > > With best regards, > Tagir Valeev we can expect the VM to not allocate the array if once inlined the code is new MyClass[0].getClass() if it's not the case, i think it's a better idea to tweak the VM rather than try to change an API based on a pattern that should not be necessary. R?mi > > ??, 15 ???? 2018 ?., 8:03 Stuart Marks : > >> Hi all, >> >> I wanted to restart review of the changeset for bug JDK-8060192 [1]. The >> latest >> webrev is here: [2]. >> >> The previous review was from December 2017: [3]. The only difference >> between the >> current changeset and the previous one is the removal of the overriding >> implementations (see explanation below). Much of the discussion in the >> previous >> review thread was around performance and the shape of the API. >> >> # Performance >> >> I previously had done some benchmarking on this and reported the results: >> [4]. >> (I've recently re-done the benchmarks and conclusions are essentially the >> same.) >> >> The upshot is that implementations that use Arrays.copyOf() are the >> fastest, >> probably because it's an intrinsic. It can avoid zero-filling the freshly >> allocated array because it knows the entire array contents will be >> overwritten. >> This is true regardless of what the public API looks like. The default >> implementation calls generator.apply(0) to get a zero-length array and >> then >> simply calls toArray(T[]). This goes through the Arrays.copyOf() fast >> path, so >> it's essentially the same speed as toArray(new T[0]). >> >> The overrides I had previously provided in specific implementation classes >> like >> ArrayList actually are slower, because the allocation of the array is done >> separately from filling it. This necessitates the extra zero-filling step. >> Thus, >> I've removed the overrides. >> >> # API Shape >> >> There was some discussion about whether it would be preferable to have a >> Class >> parameter as a type token for the array component type or the array type >> itself, >> instead of using an IntFunction generator. Most of this boils down to what >> people are comfortable with. However, there are a few points that make the >> generator function approach preferable. >> >> One point in favor of using a generator is that Stream already has a >> similar >> toArray(generator) method. >> >> Comparing this to the type token alternatives, for simple tasks like >> converting >> Collection to String[], things are about equal: >> >> toArray(String[]::new) >> toArray(String.class) >> toArray(String[].class) >> >> However, things are hairier if the element type of the collection is >> generic, >> such as Collection>. The result type is a generic array >> List[]. Using class literals or array constructor references will >> necessitate using an unchecked cast, because none of these can be used to >> represent a generic type. >> >> However, it's possible to use a method reference to provide a properly >> generic >> IntFunction. This would enable the toArray(generator) method to be used >> without >> any unchecked casts. (The method it refers to might need have an unchecked >> cast >> within it, though.) Such a method could also be reused for the >> corresponding >> Stream.toArray(generator) method. >> >> For these reasons I'd like to proceed with adding toArray(generator) API. >> >> Thanks, >> >> s'marks >> >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8060192 >> >> [2] http://cr.openjdk.java.net/~smarks/reviews/8060192/webrev.3/ >> >> [3] >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050325.html >> >> [4] >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050585.html >> From peter.levart at gmail.com Sun Jun 17 09:44:10 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 17 Jun 2018 11:44:10 +0200 Subject: RFR(s): 8060192: Add default method Collection.toArray(generator) In-Reply-To: <940f6307-34fd-eea0-88e8-c8f5134b6829@gmail.com> References: <2b1603f7-7323-56b5-99c1-6591c39ca542@oracle.com> <940f6307-34fd-eea0-88e8-c8f5134b6829@gmail.com> Message-ID: <628e3775-02b0-a046-e54c-5480d59ff98c@gmail.com> On 06/17/18 10:50, Peter Levart wrote: > The argument about using (and re-using) a method so that its method > reference can be passed to the toArray method without any unchecked > casts is equally true for any of the 3 alternatives shown - the method > itself might need unchecked casts, but its usage not: > > static List[] array(int len) > static Class> elementType() > static Class[]> arrayType() For that purpose, I would rather get rid of the need to create a method at all. It might have been the case in the past when Java generics were introduced, that class literals like List.class? would just confuse users, because most aspects of such type token would be erased and there were fears that enabling them might limit options for reified generics in the future. But long years have passed and java programmers have generally become acquainted with Java generics and erasure to the point where it doesn't confuse them any more, while reifying Java generics has been explored further in Valhalla to the point where it has become obvious that erasure of reference types is here to stay. Java could enable use of class literals like List.class without fear that such literals would make users write buggy code or that such uses would limit options for Java in the future. Quite the contrary, they would enable users to write type-safer code than what they can write today. In the light of future Java? (valhalla specialized generics), where such class literals make even more sense, enabling generic class literals could be viewed as a stepping stone that has some purpose in the short term too. So what do you think? Regards, Peter From peter.levart at gmail.com Sun Jun 17 10:04:50 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 17 Jun 2018 12:04:50 +0200 Subject: BiCollector In-Reply-To: References: <4ab4c10c-893c-6a47-eb08-ce5edab1859d@gmail.com> <26170784-e0ff-f953-daac-fa1995fd29dc@oracle.com> Message-ID: Hi Tagir, I don't know if this is important, but in your approach the particular functions of the sub-collectors are retrieved eagerly even if they are later not used. This should typically not present a problem as Collector(s) should be prepared to be used in any scenario (parallel or serial). But anyway in my approach, the sub-functions of the given collectors are retrived lazily, each time the Stream implementation needs them - the dynamics of invoking sub-collector methods to retrieve the functions is therefore unchanged when a collector is used directly to collect a Stream or as a sub-collector in the bi-collection scenario. What do you think of that particular detail? Paul? Regards, Peter On 06/15/18 13:26, Tagir Valeev wrote: > Hello! > > I created a preliminary webrev of my own implementation (no testcases > yet): > http://cr.openjdk.java.net/~tvaleev/patches/pairing/webrev/ > > If anybody wants to sponsor my implementation, I will happily log an > issue and write tests. > > The name "pairing" was invented by me, but as I'm not a native English > speaker I cannot judge whether it's optimal, so better ideas are welcome. > Also I decided to remove accumulator types from public type variables. > They do not add anything to type signature, only clutter it > increasing the number of type parameters from 4 to 6. I think it was a > mistake to expose the accumulator type parameter in other cascading > collectors > like filtering(), collectingAndThen(), groupingBy(), etc. I'm not > insisting though, if you feel that conformance to existing collectors is > more important than simplicity. > > With best regards, > Tagir Valeev. > > On Fri, Jun 15, 2018 at 5:05 AM Brian Goetz > wrote: > > > > Well, I don't see the need to pack the two results into a Map.Entry > > (or any similar) container as a drawback. > > ?From an "integrity of the JDK APIs" perspective, it is > unquestionably a > drawback.? These items are not a Key and an associated Value in a > Map; > it's merely pretending that Map.Entry really means "Pair". There's a > reason we don't have a Pair class in the JDK (and no, let's not > reopen > that now); using something else as a Pair proxy that is supposed > to have > specific semantics is worse. (It's fine to do this in your own > code, but > not in the JDK. Different standards for code that has different > audiences.) > > Tagir's proposed sidestepping is nice, and it will also play > nicely with > records, because then you can say: > > ????? record NameAndCount(String name, int count); > > ????? stream.collect(pairing(collectName, collectCount, > NameAndCount::new)); > > and get a more properly abstract result out.? And its more in the > spirit > of existing Collectors.? If you want to use Map.Entry as an > _implementation detail_, that's fine. > > I can support this form. > > > I also don't see a larger abstraction like BiStream as a natural > fit > > for a similar thing. > > I think the BiStream connection is mostly tangential.? We tried > hard to > support streams of (K,V) pairs when we did streams, as Paul can > attest, > but it was a huge complexity-inflater and dropping this out paid an > enormous simplification payoff. > > With records, having streams of tuples will be simpler to > represent, but > no more performant; it will take until we get to value types and > specialized generics to get the performance we want out of this. > > From peter.levart at gmail.com Sun Jun 17 20:56:24 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 17 Jun 2018 22:56:24 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> Message-ID: <75f225b7-e22a-b31c-004b-2c3fab4cfe16@gmail.com> Hi Brent, If 'map' field could be observed as null (which I think is only possible, if Properties object is unsafely published), then so can 'defaults' field. You could make it final as well if it was not protected. So I'm thinking what kind of memory fences would make those fields behave so that they would not be observed uninitialized even when the instance is published via data race. I think that the following would be enough, for example: ??? private Properties(Properties defaults, int initialCapacity) { ??????? // use package-private constructor to ??????? // initialize unused fields with dummy values ??????? super((Void) null); ??????? map = new ConcurrentHashMap<>(initialCapacity); ??????? this.defaults = defaults; ??? ??? VarHandle.storeStoreFence(); ??? } ... ??? public String getProperty(String key) { ??? ??? VarHandle.loadLoadFence(); ??????? Object oval = map.get(key); ??????? String sval = (oval instanceof String) ? (String)oval : null; ??????? return ((sval == null) && (defaults != null)) ? defaults.getProperty(key) : sval; ??? } ...of course, you would have to put loadLoadFence() at the beggining of each method that uses any of those two fields. Accesses to 'defaults' in subclasses wouldn't be included, but at least accesses in Properties class would be covered. Regards, Peter On 06/15/18 17:44, Brent Christian wrote: > Hi, > > In JDK 9, 8029891[1] refactored java.util.Properties to store its > values in an internal ConcurrentHashMap, and removed synchronization > from "reader" methods in order to avoid potential hangs/deadlocks > during classloading. > > Claes has noticed that there is the possibility of the new 'map' field > being observed with its default value (null), before being set. > > After looking at the JSR 133 FAQ[2], I agree with Claes that we should > make 'map' a field final. > > Please review my change to do this: > > Webrev: > http://cr.openjdk.java.net/~bchristi/8199435/webrev/ > Issue: > https://bugs.openjdk.java.net/browse/JDK-8199435 > > Thanks, > -Brent > > 1. https://bugs.openjdk.java.net/browse/JDK-8029891 > 2. > https://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html#finalRight From peter.levart at gmail.com Sun Jun 17 21:20:08 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 17 Jun 2018 23:20:08 +0200 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: Update: the discussion on concurrent-interest about possible future addition of public ThreadLocal.getIfPresent() or ThreadLocal.isPresent() has died out, but it nevertheless reached a point where ThreadLocal.isPresent() was found the least problematic. So I would like to get further with this proposal using the last webrev.04 version of the patch that uses ThreadLocal.isPresent() internally - still package-private at the moment. Here's the webrev (unchanged from the day it was published): http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ Would this version be OK? Regards, Peter On 06/06/18 20:55, Peter Levart wrote: > Ok, here's next webrev: > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.03/ > > The changes from webrev.02 are: > > - renamed JdkThreadLocal -> TerminatingThreadLocal > - instead of overriding initialValue(), ThreadLocal.setInitialValue() > calls TerminatingThreadLocal.register() at the right moment > - TerminatingThreadLocal.threadTerminated() now takes a (T value) > parameter > - TerminatingThreadLocal.REGISTRY uses IdentityHashSet instead of > HashSet (if .equals()/.hashCode() are overriden in some > TerminatingThreadLocal subclass) > > David Lloyd has commented on concurrency-interest about > ThreadLocal.getIfPresent(). There is a concern that such new method > would be inconsistent with what ThreadLocal.get() returns from > existing ThreadLocal subclasses that override .get() and possibly > transform the value returned from super.get() on the way. > > An alternative to "T getIfPresent()" is a "boolean isPresent()" > method. Even if it remains package-private, with such method > TerminatingThreadLocal could be implemented as: > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ > > If TreadLocal.isPresent() was made public, the code could be further > simplified. > > > Regards, Peter > > > On 06/06/18 18:58, Peter Levart wrote: >> >> >> On 06/06/18 17:41, Tony Printezis wrote: >>> Peter, >>> >>> You?re totally right re: JdkThreadLocal::initialValue being final >>> and cannot be overridden (I had completely missed the final keyword >>> when I first looked at the webrev). But it?d be nice to have the >>> same API in both versions of ThreadLocal. I assume you didn?t want >>> to override get() since you only wanted to update the REGISTRY the >>> first time get() is called by a thread. If getIfPresent() is >>> exposed, would something like the following work? >>> >>> @Override >>> public T get() { >>> ? final T value = getIfPresent(); >>> ? if (value != null) { >>> ? ? ? return value; >>> ? } >>> ? REGISTRY.get().add(this); >>> ? return super.get(); >>> } >> >> This would work, but if mapped value was 'null', it would keep >> calling REGISTRY.get().add(this) for each get(). Logically correct, >> but with useless overhead. >> >> ?Let me try to do something that might satisfy you... (in next webrev). >> >>> >>> One more question re: getIfPresent() (and maybe I?m overthinking >>> this): It returns null to indicate that a value is not present. >>> Isn?t null a valid ThreadLocal value? Would using an Optional here >>> be more appropriate? >> >> The problem with Optional is that it does not provide an additional >> value. Optional.empty() is a replacement for null. There's no >> Optional.of(null). So what would getIfPresent() return when there was >> a mapping present, but that mapping was null? >> >> Regards, Peter >> >>> >>> Tony >>> >>> ????? >>> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >>> >>> >>> >>> On June 6, 2018 at 11:12:44 AM, Peter Levart (peter.levart at gmail.com >>> ) wrote: >>> >>>> Hi Tony, Alan, >>>> >>>> On 06/06/2018 04:37 PM, Tony Printezis wrote: >>>>> Alan, >>>>> >>>>> Thanks for your thoughts! >>>>> >>>>> Peter, >>>>> >>>>> Any chance of taking the two suggestions I made in an earlier >>>>> e-mail into account? >>>> >>>> Right, I'll prepare new webrev with that shortly. >>>> >>>>> >>>>> a) It?d be nice if users can override initialValue(), like when >>>>> using the standard ThreadLocal class, instead of >>>>> calculateInitialValue(). (I can?t come up with a clean solution on >>>>> how to do that, though. I?ll think about it for a bit.) >>>> >>>> Your concern was that users might accidentally override >>>> initialValue() instead of calculateInitialValue(), if I understood >>>> you correctly. If that was the concern, they can't, because it is >>>> final. If the concern was that users would want to override >>>> initialValue() because they are used to do so, then perhaps a >>>> javadoc on final initialiValue() pointing to >>>> calculateInitialValue() might help them. Do you agree? This is >>>> currently internal API and consistent naming is not a big concern. >>>> >>>>> b) It?d be very helpful to pass T value to threadTerminated(), as >>>>> I cannot imagine many use-cases where the value will not be needed. >>>> >>>> Agree. Will include that in new webrev. >>>> >>>>> >>>>> Re: renaming JdkThreadLocal: ThreadLocalWithExitHooks? >>>> >>>> Hm. Exit Hooks are already something that is used in JVM (Threads >>>> started when VM is about to exit), so this might be confusing for >>>> someone. >>>> >>>> - DisposableThreadLocal >>>> - ThreadLocalWithCleanup >>>> >>>> And my favorite: >>>> >>>> - TerminatingThreadLocal >>>> >>>> >>>>> >>>>> Re: exposing getIfPresent() : Yes! Pretty please! :-) This will be >>>>> very helpful and can avoid completely unnecessary allocations. >>>> >>>> I agree that this would be generally useful functionality. Might be >>>> good to ask for opinion on concurrency-interest mailing list first. >>>> Will do that. >>>> >>>> Regards, Peter >>>> >>>>> >>>>> Tony >>>>> >>>>> >>>>> ????? >>>>> Tony Printezis | @TonyPrintezis | tprintezis at twitter.com >>>>> >>>>> >>>>> >>>>> On June 6, 2018 at 9:38:05 AM, Alan Bateman >>>>> (alan.bateman at oracle.com ) wrote: >>>>> >>>>>> >>>>>> >>>>>> On 30/05/2018 22:16, Peter Levart wrote: >>>>>> > I thought there would be some hint from Alan about which of the two >>>>>> > paths we should take for more refinement. >>>>>> > >>>>>> > The Tony's: >>>>>> > >>>>>> > http://cr.openjdk.java.net/~tonyp/8202788/webrev.1/ >>>>>> >>>>>> > >>>>>> > Or the Alan's with my mods: >>>>>> > >>>>>> > >>>>>> http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.02/ >>>>>> >>>>>> > >>>>>> Finally getting back to this one. >>>>>> >>>>>> I don't think the two approaches are all that different now. Tony's >>>>>> point on the number of hooks vs. number of locals was an >>>>>> important point >>>>>> but with Peter's update to use a thread local registry then I >>>>>> think we >>>>>> have easy to maintain solution in the DBBCache_Cleanup/webrev.02 >>>>>> patch. >>>>>> So I think I prefer that approach. We need to better name for >>>>>> "JdkThreadLocal", something to indicate that it holds resources or it >>>>>> notified when a thread terminates. >>>>>> >>>>>> Also along the way, we touched on exposing getIfPresent and we should >>>>>> look at that again. If it is expose then it fits well with the second >>>>>> approach too. >>>>>> >>>>>> -Alan >>>> >> > From peter.levart at gmail.com Sun Jun 17 23:23:23 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 18 Jun 2018 01:23:23 +0200 Subject: Proposal: optimization of Map.copyOf and Collectors.toUnmodifiableMap In-Reply-To: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> References: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> Message-ID: Hi Stuart, On 06/15/18 23:22, Stuart Marks wrote: > Hi Peter, > > Finally getting back to this. > > I think there's clearly room for improvement in the creation of the > unmodifiable collections. Right now the creation paths (mostly) use > only public APIs, which can result in unnecessary allocation and > copying. Certainly Map.copyOf does this, but there are also other > cases as well. My attempt to optimize the Map.copyOf() was also using just public APIs (with the addition of an internal class). > > For copying from an unknown map, there are a couple approaches: > > * accumulate keys and values in a single ArrayList, which can be > presized as necessary, but which will grow if necessary; then copy > elements from a subrange of ArrayList's internal array (similar to > your MapN(Object[], len) overload) > > * accumulate keys and values into a SpinedBuffer, which doesn't > require copying to grow, which is preferable if for some reason we > can't pre-size accurately; and then copy the elements out of it > > The Collectors.toUnmodifiableMap implementations are known to create > HashMap instances, so they could pass the HashMap directly to a > private MapN constructor that in turn could talk directly to HashMap > to get the keys and values. This avoids allocation of a full-sized > buffer and one copy. > > Note that these techniques involve creating new interfaces, sometimes > that cross package boundaries. It's a bit of an irritant to have to > plumb new paths that go through SharedSecrets, but it seems likely to > be worthwhile if we can avoid bulk allocation and copying steps. > > Given these, it doesn't seem to me that the BiBuffer approach helps > very much. I think there are many other avenues that would be > worthwhile to explore, and that possibly can provide bigger savings. Well, it does speed-up Map.copyOf() by 30%. But I agree there are other approaches that could go even further. In particular when all intermediary copies could be avoided. Here's one approach (which still uses just public APIs) and avoids intermediary copying: http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.02/ Why choosing Map.forEach for dumping the content of the map instead of iteration over entrySet? Because in some Map implementations this is the most direct iteration without producing garbage (for example in IdentityHashMap), but in the worst case (default Map method for example) it is delegated to iteration over entrySet. I'll benchmark it tomorrow when I get back to the computer other benchmarks were run on so the results can be compared. So what do you think of this one? Regards, Peter > > > s'marks > > > > > > On 6/11/18 3:57 AM, Peter Levart wrote: >> Hi, >> >> Those two methods were added in JDK 10 and they are not very optimal. >> Map.copyOf(Map map) 1st dumps the source map into an array of >> Map.Entry(s) (map.entrySet().toArray(new Entry[0])), which typically >> creates new Map.Entry objects, then passes the array to >> Map.ofEntries(Map.Entry[] entries) factory method that iterates the >> array and constructs a key/value interleaved array from it which is >> passed to ImmutableCollections.MapN constructor, which then >> constructs a linear-probing hash table from it. So each key and value >> is copied 3 times, while several temporary objects are created in the >> process. >> >> One copying step could be eliminated and construction of temporary >> Map.Entry objects too: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.01/ >> >> >> Collecting stream(s) using Collectors.toUnmodifiableMap() 1st >> collects key/value pairs into a HashMap, then it performs equivalent >> operations as Map.copyOf(hashMap) at the end. Using Map.copyOf() >> directly benefits this collection operation too. >> >> The following benchmark: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapBench.java >> >> >> >> Shows up to 30% improvement of .copyOf operation with this patch >> applied: >> >> Original: >> >> Benchmark?????????????????????????????? (size)? Mode Cnt Score Error? >> Units >> UnmodifiableMapBench.copyOf???????????????? 10? avgt 10 403.633 ? >> 2.640? ns/op >> UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 3489.623 ? >> 44.590? ns/op >> UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 40030.572 ? >> 277.075? ns/op >> UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10 831.221 ? >> 3.816? ns/op >> UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9783.519 ? >> 43.097? ns/op >> UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 96524.536 ? >> 670.818? ns/op >> >> Patched: >> >> Benchmark?????????????????????????????? (size)? Mode Cnt Score Error? >> Units >> UnmodifiableMapBench.copyOf???????????????? 10? avgt 10 264.172 ? >> 1.882? ns/op >> UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 2318.974 ? >> 15.877? ns/op >> UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 29291.782 ? >> 3139.737? ns/op >> UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10 771.221 ? >> 65.432? ns/op >> UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9275.016 ? >> 725.722? ns/op >> UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 82204.342 ? >> 851.741? ns/op >> >> >> Production of garbage is also reduced, since no Map.Entry temporary >> objects are constructed: >> >> Original: >> >> Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10 416.001 >> ????? 0.002??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10 >> 2936.005 ????? 0.019??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10 >> 28136.059 ????? 0.199??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10 avgt >> 10?? 1368.001 ????? 0.004??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100 avgt >> 10? 10208.139 ????? 0.045??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000 avgt >> 10? 93025.923 ????? 0.573??? B/op >> >> Patched: >> >> Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10 304.000 >> ????? 0.001??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10 >> 2464.004 ????? 0.012??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10 >> 24064.040 ????? 0.137??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10 avgt >> 10?? 1256.001 ????? 0.003??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100 avgt >> 10?? 9720.153 ????? 0.055??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000 avgt >> 10? 88905.688 ????? 0.574??? B/op >> >> >> So what do you think? Is this an improvement? >> >> Regards, Peter >> From srinivas.dama at oracle.com Mon Jun 18 02:35:59 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Sun, 17 Jun 2018 19:35:59 -0700 (PDT) Subject: RFR: 8196988 (Resolve disabled warnings for libjimage) Message-ID: <0ac897d4-d7f2-47a0-b5b1-ec2421880cfe@default> Thank you Paul for the review. >I am guessing the comment selectively disables the warning in this code (rather than using a diagnostic pragma). yes I have updated the patch on latest repo changes. http://cr.openjdk.java.net/~sdama/8196988/webrev.02/ pushing the above changeset. Note: I have run mach5 tests successfully. Regards, Srinivas ----- Original Message ----- From: paul.sandoz at oracle.com To: srinivas.dama at oracle.com Cc: core-libs-dev at openjdk.java.net, james.laskey at oracle.com Sent: Tuesday, 12 June, 2018 10:08:53 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: RFR: 8196988 (Resolve disabled warnings for libjimage) Looks ok. Including Jim in case he is not listening in? I am guessing the comment selectively disables the warning in this code (rather than using a diagnostic pragma). Thanks, Paul. > On Jun 7, 2018, at 11:17 PM, Srinivas Dama wrote: > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.01/ > for https://bugs.openjdk.java.net/browse/JDK-8196988 > > > Note: Modified earlier fix to handle warning messages specific to > libjimage only. > > Regards, > Srinivas > > ----- Original Message ----- > From: srinivas.dama at oracle.com > To: core-libs-dev at openjdk.java.net > Sent: Monday, 4 June, 2018 11:08:08 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi > Subject: RFR: 8196988 (Resolve disabled warnings for implicit-fallthrough gcc option) > > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8196988/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8196988 > > Note: gcc 7.3 compiler emits warnings for implicit fall-through switch cases,but > these warnings are disabled earlier.I have enabled these warnings and fixed in sources. > > Regards, > Srinivas From david.holmes at oracle.com Mon Jun 18 03:45:19 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 18 Jun 2018 13:45:19 +1000 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> Message-ID: Hi Brent, On 16/06/2018 1:44 AM, Brent Christian wrote: > Hi, > > In JDK 9, 8029891[1] refactored java.util.Properties to store its values > in an internal ConcurrentHashMap, and removed synchronization from > "reader" methods in order to avoid potential hangs/deadlocks during > classloading. > > Claes has noticed that there is the possibility of the new 'map' field > being observed with its default value (null), before being set. > > After looking at the JSR 133 FAQ[2], I agree with Claes that we should > make 'map' a field final. Making it "final" to fix unsafe publication only works with actual publication after construction. You're making it "final" but then not setting it at construction time and instead using UNSAFE.putVolatile to get around the fact that it is final! The "final" serves no purpose in that regard. Further a putVolatile without corresponding getVolatile does not achieve safe publication under the JMM. I think making the actual field(s) volatile is the only JMM compliant way to correctly address this. David ----- > Please review my change to do this: > > Webrev: > http://cr.openjdk.java.net/~bchristi/8199435/webrev/ > Issue: > https://bugs.openjdk.java.net/browse/JDK-8199435 > > Thanks, > -Brent > > 1. https://bugs.openjdk.java.net/browse/JDK-8029891 > 2. > https://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html#finalRight From claes.redestad at oracle.com Mon Jun 18 10:02:39 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 18 Jun 2018 12:02:39 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> Message-ID: On 2018-06-18 05:45, David Holmes wrote: > Making it "final" to fix unsafe publication only works with actual > publication after construction. You're making it "final" but then not > setting it at construction time and instead using UNSAFE.putVolatile > to get around the fact that it is final! The "final" serves no purpose > in that regard. Further a putVolatile without corresponding > getVolatile does not achieve safe publication under the JMM. Adding a volatile read on every read through Properties is likely to have some performance impact, but it could still be better than before 8029891 where such operations were synchronized. Regarding Peter's comments about defaults, I think accesses to that field was implicitly guarded by the synchronized super.get calls before 8029891 and now needs to be re-examined too. > > I think making the actual field(s) volatile is the only JMM compliant > way to correctly address this. The only issue is serialization-related readHashTable method (clone could be implemented without Unsafe as clone.map.putAll(map)). Is there no sanctioned way (using e.g. VarHandles) to safely construct and publish a transient "final" field in the scope of a readObject? I've been shying away from experimenting with VarHandles in places like Properties as I'm certain it could lead to bootstrap issues, but for something like a deserialization hook it might very well be fine. /Claes From peter.levart at gmail.com Mon Jun 18 11:06:07 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 18 Jun 2018 13:06:07 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> Message-ID: <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> Hi Claes, On 06/18/2018 12:02 PM, Claes Redestad wrote: > > > On 2018-06-18 05:45, David Holmes wrote: >> Making it "final" to fix unsafe publication only works with actual >> publication after construction. You're making it "final" but then not >> setting it at construction time and instead using UNSAFE.putVolatile >> to get around the fact that it is final! The "final" serves no >> purpose in that regard. Further a putVolatile without corresponding >> getVolatile does not achieve safe publication under the JMM. > > Adding a volatile read on every read through Properties is likely to > have some performance impact, On non-Intel architectures in particular. On intel it would just inhibit some JIT optimizations like hoisting the read of field out of loop... > but it could still be better than before 8029891 where such operations > were synchronized. Regarding Peter's comments about defaults, I think > accesses to that field was implicitly guarded by the synchronized > super.get calls before 8029891 and now needs to be re-examined too. This was pre- 8029891 code of getProperty for example: ??? public String getProperty(String key) { ??????? Object oval = super.get(key); ??????? String sval = (oval instanceof String) ? (String)oval : null; ??????? return ((sval == null) && (defaults != null)) ? defaults.getProperty(key) : sval; ??? } So 'defaults' field was accessed out of synchronized super.get() method and getProperty was not synchronized. This is pre-existing issue. But 'map' field is new. The problem with 'defaults' field is also that is is protected and subclasses can access it without synchronization. If it is made volatile, getProperty() method will be penalized. So maybe using weaker memory fences in combination with plain 'property' field can make the field behave like it was final, so at least accesses to field within Properties class can be made non-racy. If this is done for 'properties' then same fences will work for 'map' too and it doesn't have to be final. > >> >> I think making the actual field(s) volatile is the only JMM compliant >> way to correctly address this. > > The only issue is serialization-related readHashTable method (clone > could be implemented without Unsafe as clone.map.putAll(map)). 1st you would have to do: ??? clone.map = new CHM(); and only then: ??? clone.map.putAll(map); or else you would be adding the elements of the original map to the original map (which would actually work with CHM), but then the cloned Properties object would share the map with original one. So you do need to assign to 'map' field in clone(). > Is there no sanctioned way (using e.g. VarHandles) to safely construct > and publish a transient "final" field in the scope of a readObject? > I've been shying away from experimenting with VarHandles in places > like Properties as I'm certain it could lead to bootstrap issues, but > for something like a deserialization hook it might very well be fine. Paul would know for sure, but I think this was deliberately prevented in VarHandle(s). Regards, Peter > > /Claes > From claes.redestad at oracle.com Mon Jun 18 13:27:06 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 18 Jun 2018 15:27:06 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> Message-ID: <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> On 2018-06-18 13:06, Peter Levart wrote: >> Adding a volatile read on every read through Properties is likely to >> have some performance impact, > > On non-Intel architectures in particular. On intel it would just > inhibit some JIT optimizations like hoisting the read of field out of > loop... Right, and coincidentally those platforms are where I'd expect the current implementation to cause bugs (I've not been able to provoke any real error on my Intel-based workstations). I'd be surprised if we'd be much slower than pre-8029891 using volatiles (volatile read vs synchronized - even with biased locking), and we'd still retain the scalability benefits of 8029891. Ignore my remarks about clone - I'm just back from vacation and have apparently forgotten how cloning works. :-) /Claes From claes.redestad at oracle.com Mon Jun 18 13:54:26 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 18 Jun 2018 15:54:26 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> Message-ID: I'd suggest something simple like this to ensure correctness, while keeping the number of volatile reads to a minimum: http://cr.openjdk.java.net/~redestad/8199435.00/ Then file a follow-up RFE to investigate if we can make these fields non-volatile, e.g. using careful fencing as suggested by Peter. /Claes On 2018-06-18 15:27, Claes Redestad wrote: > > > On 2018-06-18 13:06, Peter Levart wrote: >>> Adding a volatile read on every read through Properties is likely to >>> have some performance impact, >> >> On non-Intel architectures in particular. On intel it would just >> inhibit some JIT optimizations like hoisting the read of field out of >> loop... > > Right, and coincidentally those platforms are where I'd expect the > current implementation to cause bugs (I've not been able to provoke > any real error on my Intel-based workstations). > > I'd be surprised if we'd be much slower than pre-8029891 using > volatiles (volatile read vs synchronized - even with biased locking), > and we'd still retain the scalability benefits of 8029891. > > Ignore my remarks about clone - I'm just back from vacation and have > apparently forgotten how cloning works. :-) > > /Claes From martinrb at google.com Mon Jun 18 14:05:24 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 18 Jun 2018 07:05:24 -0700 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> Message-ID: There's a long history of cheating and setting final fields in pseudo-constructors readObject and clone, which is well motivated since Java should really support pseudo-constructors in a better way.. Cheating has used Unsafe or reflection with setAccessible (CopyOnWriteArrayList.resetLock()). I haven't had any success actually reproducing any data race failures in constructors - on modern machines it would only happen if the JIT reordered the writes. Doug - would CopyOnWriteArrayList.clone() be slightly safer if it had a releaseFence() ? From martinrb at google.com Mon Jun 18 14:13:01 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 18 Jun 2018 07:13:01 -0700 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: I'm ignoring the direct buffers problem (sorry Tony) and wearing my ReentrantReadWriteLock hat. I still like the idea of adding ThreadLocal.compute(Function remappingFunction) and perhaps other such map-inspired methods. RRWL wouldn't benefit much, since it already tries to minimize use of ThreadLocal, but it would at least allow get() and remove() to be coalesced on read-unlock. RRWL never changes from one non-null value to another, it only switches between absent and present. I'd like to avoid the two lookups due to get and remove. Here's a prototype that does not yet help RRWL, but helps callers switching between non-null values, and could probably be extended via surgery to ThreadLocalMap: public T compute( java.util.function.Function remappingFunction) { final Thread currentThread = Thread.currentThread(); final ThreadLocalMap map = getMap(currentThread); final ThreadLocalMap.Entry e = (map == null) ? null : map.getEntry(this); final T oldValue = (e == null) ? null : (T)e.value; final T newValue = remappingFunction.apply(oldValue); if (newValue == null) { if (oldValue != null) { map.remove(this); } } else if (e != null) { e.value = newValue; } else if (map != null) { map.set(this, newValue); } else { createMap(currentThread, newValue); } return newValue; } On Sun, Jun 17, 2018 at 2:20 PM, Peter Levart wrote: > Update: the discussion on concurrent-interest about possible future > addition of public ThreadLocal.getIfPresent() or ThreadLocal.isPresent() > has died out, but it nevertheless reached a point where > ThreadLocal.isPresent() was found the least problematic. So I would like to > get further with this proposal using the last webrev.04 version of the > patch that uses ThreadLocal.isPresent() internally - still package-private > at the moment. > > Here's the webrev (unchanged from the day it was published): > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ > > Would this version be OK? > > Regards, Peter > > > > On 06/06/18 20:55, Peter Levart wrote: > > From dl at cs.oswego.edu Mon Jun 18 14:21:56 2018 From: dl at cs.oswego.edu (Doug Lea) Date: Mon, 18 Jun 2018 10:21:56 -0400 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> Message-ID: <7ea10dea-13c7-b2e6-7146-d2bccf17b040@cs.oswego.edu> On 06/18/2018 10:05 AM, Martin Buchholz wrote: > There's a long history of cheating and setting final fields in > pseudo-constructors readObject and clone, which is well motivated since > Java should really support pseudo-constructors in a better way.. > Cheating has used Unsafe or reflection with setAccessible > (CopyOnWriteArrayList.resetLock()). > > Doug - would CopyOnWriteArrayList.clone() be slightly safer if it had a > releaseFence() ? > Or better, lockField.set in resetLock could instead be setRelease. Except it is using reflection, not VarHandles. So it could be preceded with VarHandle.releaseFence(), although it is hard to think of a scenario in which it could matter. -Doug From peter.levart at gmail.com Mon Jun 18 14:23:01 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 18 Jun 2018 16:23:01 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> Message-ID: <7db4b65c-d3d3-e4e3-7a71-0e468e28b313@gmail.com> Hi Claes, On 06/18/2018 03:54 PM, Claes Redestad wrote: > I'd suggest something simple like this to ensure correctness, while > keeping the number of volatile reads to a minimum: > > http://cr.openjdk.java.net/~redestad/8199435.00/ > > Then file a follow-up RFE to investigate if we can make these fields > non-volatile, e.g. using careful fencing as suggested > by Peter. OK, but the constructor will still need a releaseFence at the end to prevent a potential write that unsafely publishes Properties instance to float before the volatile writes of 'map' and 'defaults'... You might want to use Unsafe directly for that since VarHandle could cause bootstrap issues. Regards, Peter > > /Claes > > On 2018-06-18 15:27, Claes Redestad wrote: >> >> >> On 2018-06-18 13:06, Peter Levart wrote: >>>> Adding a volatile read on every read through Properties is likely >>>> to have some performance impact, >>> >>> On non-Intel architectures in particular. On intel it would just >>> inhibit some JIT optimizations like hoisting the read of field out >>> of loop... >> >> Right, and coincidentally those platforms are where I'd expect the >> current implementation to cause bugs (I've not been able to provoke >> any real error on my Intel-based workstations). >> >> I'd be surprised if we'd be much slower than pre-8029891 using >> volatiles (volatile read vs synchronized - even with biased locking), >> and we'd still retain the scalability benefits of 8029891. >> >> Ignore my remarks about clone - I'm just back from vacation and have >> apparently forgotten how cloning works. :-) >> >> /Claes > From Roger.Riggs at Oracle.com Mon Jun 18 14:25:37 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 18 Jun 2018 10:25:37 -0400 Subject: Proposal: optimization of Map.copyOf and Collectors.toUnmodifiableMap In-Reply-To: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> References: <543c7b19-37dd-b159-1509-280bdac98b8a@oracle.com> Message-ID: Hi Stuart, In regard to new SharedSecret interfaces, one option is move shared (but private) implementation classes to a jdk.internal.xx package (not exported).? This only works well if they are not tightly coupled to other package private classes. SpinedBuffer might be a good candidate, I have some IO cases in mind that could benefit from the allocation/reallocation savings.? (ByteArrayOutputStream for 1). Regards, Roger On 6/15/2018 5:22 PM, Stuart Marks wrote: > Hi Peter, > > Finally getting back to this. > > I think there's clearly room for improvement in the creation of the > unmodifiable collections. Right now the creation paths (mostly) use > only public APIs, which can result in unnecessary allocation and > copying. Certainly Map.copyOf does this, but there are also other > cases as well. > > For copying from an unknown map, there are a couple approaches: > > * accumulate keys and values in a single ArrayList, which can be > presized as necessary, but which will grow if necessary; then copy > elements from a subrange of ArrayList's internal array (similar to > your MapN(Object[], len) overload) > > * accumulate keys and values into a SpinedBuffer, which doesn't > require copying to grow, which is preferable if for some reason we > can't pre-size accurately; and then copy the elements out of it > > The Collectors.toUnmodifiableMap implementations are known to create > HashMap instances, so they could pass the HashMap directly to a > private MapN constructor that in turn could talk directly to HashMap > to get the keys and values. This avoids allocation of a full-sized > buffer and one copy. > > Note that these techniques involve creating new interfaces, sometimes > that cross package boundaries. It's a bit of an irritant to have to > plumb new paths that go through SharedSecrets, but it seems likely to > be worthwhile if we can avoid bulk allocation and copying steps. > > Given these, it doesn't seem to me that the BiBuffer approach helps > very much. I think there are many other avenues that would be > worthwhile to explore, and that possibly can provide bigger savings. > > s'marks > > > > > > On 6/11/18 3:57 AM, Peter Levart wrote: >> Hi, >> >> Those two methods were added in JDK 10 and they are not very optimal. >> Map.copyOf(Map map) 1st dumps the source map into an array of >> Map.Entry(s) (map.entrySet().toArray(new Entry[0])), which typically >> creates new Map.Entry objects, then passes the array to >> Map.ofEntries(Map.Entry[] entries) factory method that iterates the >> array and constructs a key/value interleaved array from it which is >> passed to ImmutableCollections.MapN constructor, which then >> constructs a linear-probing hash table from it. So each key and value >> is copied 3 times, while several temporary objects are created in the >> process. >> >> One copying step could be eliminated and construction of temporary >> Map.Entry objects too: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/webrev.01/ >> >> >> Collecting stream(s) using Collectors.toUnmodifiableMap() 1st >> collects key/value pairs into a HashMap, then it performs equivalent >> operations as Map.copyOf(hashMap) at the end. Using Map.copyOf() >> directly benefits this collection operation too. >> >> The following benchmark: >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/UnmodifiableMap_copyOf/UnmodifiableMapBench.java >> >> >> >> Shows up to 30% improvement of .copyOf operation with this patch >> applied: >> >> Original: >> >> Benchmark?????????????????????????????? (size)? Mode Cnt Score Error? >> Units >> UnmodifiableMapBench.copyOf???????????????? 10? avgt 10 403.633 ? >> 2.640? ns/op >> UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 3489.623 ? >> 44.590? ns/op >> UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 40030.572 ? >> 277.075? ns/op >> UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10 831.221 ? >> 3.816? ns/op >> UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9783.519 ? >> 43.097? ns/op >> UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 96524.536 ? >> 670.818? ns/op >> >> Patched: >> >> Benchmark?????????????????????????????? (size)? Mode Cnt Score Error? >> Units >> UnmodifiableMapBench.copyOf???????????????? 10? avgt 10 264.172 ? >> 1.882? ns/op >> UnmodifiableMapBench.copyOf??????????????? 100? avgt?? 10 2318.974 ? >> 15.877? ns/op >> UnmodifiableMapBench.copyOf?????????????? 1000? avgt?? 10 29291.782 ? >> 3139.737? ns/op >> UnmodifiableMapBench.toUnmodifiableMap????? 10? avgt 10 771.221 ? >> 65.432? ns/op >> UnmodifiableMapBench.toUnmodifiableMap???? 100? avgt?? 10 9275.016 ? >> 725.722? ns/op >> UnmodifiableMapBench.toUnmodifiableMap??? 1000? avgt?? 10 82204.342 ? >> 851.741? ns/op >> >> >> Production of garbage is also reduced, since no Map.Entry temporary >> objects are constructed: >> >> Original: >> >> Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10 416.001 >> ????? 0.002??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10 >> 2936.005 ????? 0.019??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10 >> 28136.059 ????? 0.199??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10 avgt >> 10?? 1368.001 ????? 0.004??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100 avgt >> 10? 10208.139 ????? 0.045??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000 avgt >> 10? 93025.923 ????? 0.573??? B/op >> >> Patched: >> >> Benchmark (size)? Mode? Cnt????? Score?????? Error?? Units >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 10? avgt?? 10 304.000 >> ????? 0.001??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 100? avgt?? 10 >> 2464.004 ????? 0.012??? B/op >> UnmodifiableMapBench.copyOf:?gc.alloc.rate.norm 1000? avgt?? 10 >> 24064.040 ????? 0.137??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 10 avgt >> 10?? 1256.001 ????? 0.003??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 100 avgt >> 10?? 9720.153 ????? 0.055??? B/op >> UnmodifiableMapBench.toUnmodifiableMap:?gc.alloc.rate.norm 1000 avgt >> 10? 88905.688 ????? 0.574??? B/op >> >> >> So what do you think? Is this an improvement? >> >> Regards, Peter >> From Roger.Riggs at Oracle.com Mon Jun 18 14:31:45 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 18 Jun 2018 10:31:45 -0400 Subject: RFR JDK-8066709 Make some JDK system properties read only In-Reply-To: <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> References: <339E35CB-84DB-49C1-9A25-657EDDE1B1DC@oracle.com> <7CA54898-E166-4578-B7E9-2C351ABBB6BA@oracle.com> <154a06be-e1d5-a014-b037-7e4e524b9e12@Oracle.com> <82fd1f1b-f6c5-692e-6076-33dc8aa6e050@oracle.com> Message-ID: <9b702ba3-269a-5807-072d-2e72b75a4cf3@Oracle.com> Hi Alan, Thanks for the review and suggestion. The CSR and Webrev are updated. webrev: ? http://cr.openjdk.java.net/~rriggs/webrev-static-property-8066709/ csr: ? https://bugs.openjdk.java.net/browse/JDK-8204235 On 6/16/2018 8:42 AM, Alan Bateman wrote: > On 13/06/2018 15:10, Roger Riggs wrote: >> Hi Joe, >> >> The CSR text is still in draft and got out of sync with the open >> review and comments. >> >> The updated apiNote clarifies that changes made through the >> System.*property*? methods >> may not affect the value cached during initialization. >> The implementation changes affect a very short list of properties. >> >> The note on the methods is to raise awareness that individual >> properties may have >> different behavior and may be unpredictable. > Just catching up on this. I checked the CSR and the webrev (the latest > webrev was generated on June 6 so I hope that is the version we are > meant to look at). > > The updated apiNote (CSR version) looks okay but I think the word > "internal" needs to be dropped as it comes with too many questions. > "may be cached during initialization or ..." should be okay. The > editing has meant the line lengths are a bit inconsistent so I assume > they can be fixed up before the change is pushed. > > I see the original patch to SocksSocketImpl has been mostly reverted. > It looks correct now. The other usages look okay to me. > > -Alan From claes.redestad at oracle.com Mon Jun 18 14:47:57 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 18 Jun 2018 16:47:57 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <7db4b65c-d3d3-e4e3-7a71-0e468e28b313@gmail.com> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> <7db4b65c-d3d3-e4e3-7a71-0e468e28b313@gmail.com> Message-ID: On 2018-06-18 16:23, Peter Levart wrote: > Hi Claes, > > On 06/18/2018 03:54 PM, Claes Redestad wrote: >> I'd suggest something simple like this to ensure correctness, while >> keeping the number of volatile reads to a minimum: >> >> http://cr.openjdk.java.net/~redestad/8199435.00/ >> >> Then file a follow-up RFE to investigate if we can make these fields >> non-volatile, e.g. using careful fencing as suggested >> by Peter. > > OK, but the constructor will still need a releaseFence at the end to > prevent a potential write that unsafely publishes Properties instance > to float before the volatile writes of 'map' and 'defaults'... > > You might want to use Unsafe directly for that since VarHandle could > cause bootstrap issues. I don't follow.. which constructor needs a fence in the suggested patch? /Claes From Roger.Riggs at Oracle.com Mon Jun 18 14:55:58 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 18 Jun 2018 10:55:58 -0400 Subject: [11] RFR: 8042131: DateTimeFormatterBuilder Mapped-values do not work for JapaneseDate In-Reply-To: <9018eeda-d891-adf8-81b7-ef1d1db08281@oracle.com> References: <9018eeda-d891-adf8-81b7-ef1d1db08281@oracle.com> Message-ID: <8223a2e8-08a6-9118-000f-b9930ac30d09@Oracle.com> Hi Naoto, Looks fine Regards, Roger On 6/15/2018 6:14 PM, Naoto Sato wrote: > Hello, > > Please review the fix to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8042131 > > The proposed change is located at: > > http://cr.openjdk.java.net/~naoto/8042131/webrev.00/ > > The fix is originally contributedy by Toshio Nakamura at IBM [1]. I am > sponsoring the fix, with some trivial modifications to the test (diff > is attached). > > Naoto > > [1] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053830.html From scolebourne at joda.org Mon Jun 18 15:04:11 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 18 Jun 2018 16:04:11 +0100 Subject: [11] RFR: 8042131: DateTimeFormatterBuilder Mapped-values do not work for JapaneseDate In-Reply-To: <8223a2e8-08a6-9118-000f-b9930ac30d09@Oracle.com> References: <9018eeda-d891-adf8-81b7-ef1d1db08281@oracle.com> <8223a2e8-08a6-9118-000f-b9930ac30d09@Oracle.com> Message-ID: +1 Stephen On 18 June 2018 at 15:55, Roger Riggs wrote: > Hi Naoto, > > Looks fine > > Regards, Roger > > > > On 6/15/2018 6:14 PM, Naoto Sato wrote: >> >> Hello, >> >> Please review the fix to the following issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8042131 >> >> The proposed change is located at: >> >> http://cr.openjdk.java.net/~naoto/8042131/webrev.00/ >> >> The fix is originally contributedy by Toshio Nakamura at IBM [1]. I am >> sponsoring the fix, with some trivial modifications to the test (diff is >> attached). >> >> Naoto >> >> [1] >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053830.html > > From martinrb at google.com Mon Jun 18 15:22:21 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 18 Jun 2018 08:22:21 -0700 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: <7ea10dea-13c7-b2e6-7146-d2bccf17b040@cs.oswego.edu> References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> <7ea10dea-13c7-b2e6-7146-d2bccf17b040@cs.oswego.edu> Message-ID: On Mon, Jun 18, 2018 at 7:21 AM, Doug Lea wrote: > On 06/18/2018 10:05 AM, Martin Buchholz wrote: > > There's a long history of cheating and setting final fields in > > pseudo-constructors readObject and clone, which is well motivated since > > Java should really support pseudo-constructors in a better way.. > > Cheating has used Unsafe or reflection with setAccessible > > (CopyOnWriteArrayList.resetLock()). > > > > > Doug - would CopyOnWriteArrayList.clone() be slightly safer if it had a > > releaseFence() ? > > > > Or better, lockField.set in resetLock could instead be setRelease. > Except it is using reflection, not VarHandles. So it could be preceded > with VarHandle.releaseFence(), although it is hard to think of a > scenario in which it could matter. > You mean followed by, not preceded by? try { lockField.set(this, new Object()); + java.lang.invoke.VarHandle.releaseFence(); } catch (IllegalAccessException e) { throw new Error(e); } From Alan.Bateman at oracle.com Mon Jun 18 15:41:09 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 18 Jun 2018 16:41:09 +0100 Subject: RFR: JDK-8202788: Explicitly reclaim cached thread-local direct buffers at thread exit In-Reply-To: References: <0b5b8760-5c63-1ca7-9371-095466c4d3fb@oracle.com> <327a3359-d38a-02ff-1e5d-6464dcd55ced@gmail.com> <180a867b-6105-e888-177f-02cb59302f62@gmail.com> <38f54355-f916-b6a5-286e-77c2f60aa061@oracle.com> <4adbf53d-e03b-5f30-ca47-1f5c589d4050@gmail.com> <649ee9fe-fdd2-6950-cf7f-8821fa05f48c@gmail.com> <1128be27-0241-b3b4-d810-ccbcbfb0d61c@gmail.com> Message-ID: On 17/06/2018 22:20, Peter Levart wrote: > Update: the discussion on concurrent-interest about possible future > addition of public ThreadLocal.getIfPresent() or > ThreadLocal.isPresent() has died out, but it nevertheless reached a > point where ThreadLocal.isPresent() was found the least problematic. > So I would like to get further with this proposal using the last > webrev.04 version of the patch that uses ThreadLocal.isPresent() > internally - still package-private at the moment. > > Here's the webrev (unchanged from the day it was published): > > http://cr.openjdk.java.net/~plevart/jdk-dev/DBBCache_Cleanup/webrev.04/ > > Would this version be OK? I think looks quite good. One small nit is that the update to ThreadLocal.setInitialValue makes it look like all locals are registered when setting the initial value. What would you think about moving the instanceof check from TerminatingThreadLocal.register so that it's a bit more obvious. -Alan From peter.levart at gmail.com Mon Jun 18 16:09:45 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 18 Jun 2018 18:09:45 +0200 Subject: RFR 8199435 : Unsafe publication of java.util.Properties.map In-Reply-To: References: <678ca5ae-7dda-d4cd-099c-e4226fec60c3@oracle.com> <984dedfa-c990-dc68-6ea8-cbbe1df613d9@gmail.com> <85b61ec9-605d-ed93-17ba-7fef6d3d763a@oracle.com> <7db4b65c-d3d3-e4e3-7a71-0e468e28b313@gmail.com> Message-ID: <8474cc34-36d7-fc76-6297-60f48557c432@gmail.com> On 06/18/2018 04:47 PM, Claes Redestad wrote: > > > On 2018-06-18 16:23, Peter Levart wrote: >> Hi Claes, >> >> On 06/18/2018 03:54 PM, Claes Redestad wrote: >>> I'd suggest something simple like this to ensure correctness, while >>> keeping the number of volatile reads to a minimum: >>> >>> http://cr.openjdk.java.net/~redestad/8199435.00/ >>> >>> Then file a ```