more about - System.arraycopy(...) equivalents for ByteBuffer/FileChannel
Alan.Bateman at oracle.com
Fri Nov 2 03:24:24 PDT 2012
On 02/11/2012 01:29, Jeff Hain wrote:
> That's modulo the fact that MappedByteBuffers can't be
> explicitly and portably unmapped (Bug ID: 4724038), so
> you can't always use them.
Yes, a very difficult issue that many people have looked at but didn't
come up a solution that addresses all the concerns, see:
> That's also modulo the fact that, for BB to FC copies,
> if the FileChannel is not readable, you can't use MBB
> (no WRITE_ONLY mapping mode). A FileChannel.isReadable()
> method would allow to use MBB for readable (and writable)
> channels, without risk of an exception being thrown.
It would although if code is running into then it suggests that it might
not be using the API as intended.
> That's also modulo the fact that when you copy between
> a MappedByteBuffer and a FileChannel, src and dst might
> share memory, but there is no API to figure out if and how,
> thus you don't know if you have to copy forward or backward
> (to avoid erasing bytes to copy with copied bytes).
Right, there isn't anything in this API to know when buffers or mappings
> I found FileChannel.write(ByteBuffer,long) to be very slow
> on WinXP/7, by a factor 15, when the ByteBuffer is "large"
> (like 500Ko), even if it is direct (mapped or not), in which
> case it resolves to sun.nio.ch.IOUtils.writeFromNativeBuffer
> (which I supposed to always be fast)).
> On Linux there was no slow down.
This is another long standing issue, see:
It comes down to Windows not having the equivalent of a pread/pwrite
that doesn't change the global position. There are ideas on how to solve
this but it requires significant refactoring, hopefully some day.
> Intensive benches involving MBBs (namely FC to heap BB
> copies) were hanging from time to time, up to nearly a second,
> and then resumed at usual speed (slightly faster than temporary
> direct ByteBuffer approaches).
> As a result, I completely disabled MBBs usage for my copies.
Was this Windows 32-bit? We've seen a lot of issues with memory
management on Windows where it takes time to write-out dirty pages and
unmap the memory. I don't think we can do anything about this, assuming
this is what you are running into. Windows 64-bit appears to be better
although it can be a problem too when memory is over committed.
> 3) Non-performance remarks:
> When copying between direct ByteBuffers, the efficient way
> is to use ByteBuffer.put(ByteBuffer), but its spec says that
> it's about equivalent to
> "while (src.hasRemaining()) dst.put(src.get());",
> i.e. that it is not suited if memory is shared (which no API
> allows to figure out) and srcPos < dstPos (as raw memmory
> positions), since then it could erase bytes to copy with
> copied bytes.
We could potentially clarify the spec here although warning about
overlapping regions may require changes in several other places too.
> The bug we already talked about (the early return if
> size > size(), which masks the writability check, and the
> fact that size >= size() could be done instead)
> prevents reworking position whether truncation occurs or not,
> as the spec says (even if size > size(), position should be
> set to size if it is superior, but it isn't - or is it a
> spec bug?).
This is a corner case which is probably why it was never reported
before. We have a bug for it since you brought it up:
> If mode is null, and assertions enabled, "assert (imode >= 0)"
> fails (I have somewhere in my head the idea that in JDK assertions
> should only check private code - well in a sense here it does :).
If mode is passed as null then NullPointerException should be thrown.
Another 10+ year issue that hasn't been noticed. I'll create a bug for
this - thanks!
> This method can grow destination channel, but if the specified
> position (in dst) is > dst.size(), it just returns 0. It looks
> like a bug, as the spec says nothing about this surprising behavior.
This is specified: " If the given position is greater than the file's
current size then no bytes are transferred"
> The Javadoc says that position is not updated, but if the channel
> is in append mode it might be (since then we have position = size,
> and this method can grow the file).
There is a long standing bug tracking updating the spec on this issue, see:
Thanks for bug reports, you've gone in corner cases that our tests for
this area clearly haven't gone into.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the nio-dev