RFR(M): 8152995: Solaris os::available_memory() doesn't do what we think it does
erik.osterlund at oracle.com
Wed Apr 6 14:09:02 UTC 2016
On Solaris, the os::available_memory() function is currently calculated
Unfortunately this does not match intended semantics. The intended
semantics is to return how much memory can be allocated by mmap into
physical memory. But, what _SC_AVPHYS_PAGES does is to return how many
physical pages are available to be used by virtual memory as backing
storage on-demand once it is touched, without any connection whatsoever
to virtual memory.
Even if we mmap to commit heap memory without NORESERVE, the
_SC_AVPHYS_PAGES metric does not change its value - at least not until
somebody actually touches the mmaped memory and it starts becoming
backed by actual physical memory. So the JVM can in theory commit the
whole physical memory, and _SC_AVPHYS_PAGES will still reply that all
that memory is still available given that it has not been touched yet.
It is likely that this is related to random swap-related test failures,
where too many JVMs are created based on this metric. Even if it is not,
the os::available_memory() call is still broken in its current state and
should be fixed regardless.
My proposed fix uses kstat to get the available memory that can be
mmapped (which actually relates to virtual memory). It then uses
swapctl() to find out the amount of free swap, subtracting that from the
kstat value, to make sure we do not count swap memory as being available
for grabbing, to mimick the current behaviour of other platforms. The
code iterates over the potentially many swap resources and adds up the
free swap memory.
kstat gives us all memory that can be made available, including memory
already used by the OS for things like file caches, and swap memory.
When this value is 0, mmap will fail. That's why I calculate the amount
of swap and remove that, assuming it is okay to use memory that isn't
immediately available but can be made available, as long as it does not
involve paging to the swap memory.
* Made my own test program that can be found in the comments of the BUG
to report on memory values, so I could verify what is going on and that
when the new os::available_memory() becomes 0, is indeed when paging to
swap starts happening using vmstat.
I need a sponsor to push this if anyone is interested.
More information about the hotspot-gc-dev