RFR 8190974 Parallel stream execution within a custom ForkJoinPool should obey the parallelism
paul.sandoz at oracle.com
Thu Nov 9 00:43:48 UTC 2017
> On 8 Nov 2017, at 15:48, Martin Buchholz <martinrb at google.com> wrote:
> You probably want a different summary.
> + * @summary Tests counting of streams containing Integer.MAX_VALUE + 1 elements
> Will this fail if common pool parallelism is odd?
> + int splitsForPHalfC = countSplits(new ForkJoinPool(ForkJoinPool.getCommonPoolParallelism() / 2));
Yes, the number of splits will be parallelism * 4 rounded up to the nearest power of 2.
I modified the common pool testing and removed the doubling of the parallelism since this might result in OOME when creating the threads.
> I would drop the word "factor" below:
> + * Default target factor of leaf tasks for parallel decomposition.
> Do you ever test with
> * @run junit/othervm/timeout=1000
> * --add-opens java.base/java.util.concurrent=ALL-UNNAMED
> * --add-opens java.base/java.lang=ALL-UNNAMED
> * -Djsr166.testImplementationDetails=true
> * -Djava.util.concurrent.ForkJoinPool.common.parallelism=0
> * JSR166TestCase
Added. I had to move the test outside of the stream test area, which is set up to run in a more restricted manner.
WebRev updated in place
> On Wed, Nov 8, 2017 at 1:01 PM, Paul Sandoz <paul.sandoz at oracle.com <mailto:paul.sandoz at oracle.com>> wrote:
> Please review this patch to ensure that a parallel stream obeys the parallelism of a custom fork join pool when it is executed within that pool:
> http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8190974-par-stream-custom-pool/webrev/ <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8190974-par-stream-custom-pool/webrev/> <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8190974-par-stream-custom-pool/webrev/ <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8190974-par-stream-custom-pool/webrev/>>
> Streams currently do not support capabilities to control the level of parallelism and therefore resources utilised (tricky API design problem to get right).
> At the moment the trick is to run stream executions within a custom pool, however the number of fork join tasks created will be in proportion to the parallelism of the common pool thus the execution will be over-provisioned. This can be especially noticeable on large machines with many cores.
More information about the core-libs-dev