RFR(M): 8152995: Solaris os::available_memory() doesn't do what we think it does
erik.osterlund at oracle.com
Wed Apr 6 14:14:30 UTC 2016
Looping in runtime.
On 2016-04-06 16:09, Erik Österlund wrote:
> Bug: https://bugs.openjdk.java.net/browse/JDK-8152995
> CR: http://cr.openjdk.java.net/~eosterlund/8152995/webrev.00/
> On Solaris, the os::available_memory() function is currently
> calculated with sysconf(_SC_AVPHYS_PAGES).
> Unfortunately this does not match intended semantics. The intended
> semantics is to return how much memory can be allocated by mmap into
> physical memory. But, what _SC_AVPHYS_PAGES does is to return how many
> physical pages are available to be used by virtual memory as backing
> storage on-demand once it is touched, without any connection
> whatsoever to virtual memory.
> Even if we mmap to commit heap memory without NORESERVE, the
> _SC_AVPHYS_PAGES metric does not change its value - at least not until
> somebody actually touches the mmaped memory and it starts becoming
> backed by actual physical memory. So the JVM can in theory commit the
> whole physical memory, and _SC_AVPHYS_PAGES will still reply that all
> that memory is still available given that it has not been touched yet.
> It is likely that this is related to random swap-related test
> failures, where too many JVMs are created based on this metric. Even
> if it is not, the os::available_memory() call is still broken in its
> current state and should be fixed regardless.
> My proposed fix uses kstat to get the available memory that can be
> mmapped (which actually relates to virtual memory). It then uses
> swapctl() to find out the amount of free swap, subtracting that from
> the kstat value, to make sure we do not count swap memory as being
> available for grabbing, to mimick the current behaviour of other
> platforms. The code iterates over the potentially many swap resources
> and adds up the free swap memory.
> kstat gives us all memory that can be made available, including memory
> already used by the OS for things like file caches, and swap memory.
> When this value is 0, mmap will fail. That's why I calculate the
> amount of swap and remove that, assuming it is okay to use memory that
> isn't immediately available but can be made available, as long as it
> does not involve paging to the swap memory.
> * JPRT
> * Made my own test program that can be found in the comments of the
> BUG to report on memory values, so I could verify what is going on and
> that when the new os::available_memory() becomes 0, is indeed when
> paging to swap starts happening using vmstat.
> I need a sponsor to push this if anyone is interested.
More information about the hotspot-gc-dev