parallel IO anomaly with windows REFS formatted drive

Michael Skells mike.skells1 at
Sun Oct 29 22:09:23 UTC 2017

It seems to me that some NIO based file creates don't scale work well with
REFS formatted drives
REFS is an alternative (to NTFS) drive format for Windows 8 and above

When working with NTFS on my test system, as we use more threads the
performance improves so some degree, (1000 files creates take 5-600 ms with
one thread and 2-300 ms with 4 threads)
when using REFS the single threaded times are roughly the same for 1 and 2
threads, but get much worse for 3 and 4 threads (1200 - 1600 ms)

this showdown is only when using and synchronous IO, for
the AsynchronousFileChannel it seems roughly the to scale as expected

CSV attached for the test results, and some test code. The test code is in
scala, and I can convert if needed, but it just really 3 calls, to open
write and close the files

My test system is an I7 4 core window 10 pro with Norton installed, but an
exclusion for the specified drives, and indexing disabled. All writes are
to an M.2 SSD with write-though cache

Some background

I am working on the back end code generation of the scala compiler which
generates many thousand  small .class files, and part of that is to tune
the IO. For windows where the IO cost is significant, so I had a few test
app to simulate and measure the relative performance of the different

Any ideas for a bulk file creation alternative welcome. I know we could
write to a JAR but that requires other re-tooling


More information about the core-libs-dev mailing list