RFR 8193856 takeWhile produces incorrect result with elements produced by flatMap
paul.sandoz at oracle.com
Wed Dec 20 23:03:51 UTC 2017
> On 20 Dec 2017, at 14:47, Jonathan Bluett-Duncan <jbluettduncan at gmail.com> wrote:
> Hi Paul,
> It seems that some clever Googler managed to find a workaround for aggressive `flatMap` operations in the form of a so-called `MoreStreams.flatten` operation, implemented in a side project called google/mug.
Thanks. MoreStreams.flatten “lowers” a stream of streams to a spliterator of spliterators which in turn is input as a new stream source, enabling a cancellation check + tryAdvance for any following short-circuit or terminal ops.
I have a solution in the works for flatMap based on sorted operation (which has to buffer all elements, sort ‘em, then push ‘em all downstream).
> I'm sharing the javadoc <https://google.github.io/mug/apidocs/com/google/mu/util/stream/MoreStreams.html#flatten-java.util.stream.Stream-> and GitHub project homepage <https://github.com/google/mug> with you and the rest of the mailing list in the hope that they prove to be useful.
> On 20 December 2017 at 21:28, Paul Sandoz <paul.sandoz at oracle.com <mailto:paul.sandoz at oracle.com>> wrote:
> Please review this fix for a bug in the stream takeWhile operation:
> http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/ <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/> <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/ <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8193856-takeWhile-incorrect-results/webrev/>>
> The flatMap operation is currently aggressive and does not detect if a downstream operation may or has cancelled processing, and will push all of it’s elements downstream. Short-circuiting operations should be guarded against such behaviour but unfortunately takeWhile was not guarded.
> Separately i plan to figure out a way to ensure flatMap operations become less aggressive if there are short-circuiting downstream operations. This may increase efficiency and also allow support for flat map results that are infinite.
More information about the core-libs-dev