sun.nio.ch.Util: Don't cache an unlimited amount of memory
Alan.Bateman at oracle.com
Tue Dec 29 09:17:21 UTC 2015
On 27/12/2015 20:35, Evan Jones wrote:
> Summary: nio Util caches an unlimited amount of memory for temporary
> direct ByteBuffers, which effectively is a native memory leak.
> Applications that do large I/Os can inadvertently waste gigabytes of
> native memory or run out of memory. I suggest it should only cache a
> "small" amount of memory per-thread (e.g. 1 MB), and maybe have a flag
> to allow changing the limit for applications where this causes a
> performance regression.
> 1. Would JDK committers support this change?
> 2. If so, any suggestions for the default and/or how to override it
> with a flag?
> Tony Printezis (CCed here) added a flag to Twitter's internal JVM/JDK
> to limit the size of this cache, which we could probably use that as a
> starting point for a patch.
Limiting the size of the buffer cache might help in some scenarios, it
just means a bit more complexity and yet another tuning option.
Do you do scatter/gather I/O? The current implementation will cache up
to IOV_MAX buffers per thread but if you aren't doing scatter/gather I/O
then caching a maximum of one buffer per thread should reduce the memory
usage. It wouldn't be hard to modify the BufferCache implementation to
track the number of per-thread buffers in use so that this count is the
maximum cached rather than IOV_MAX. I'm curious if you've looked into
doing anything along those lines.
More information about the nio-dev