RFR: JDK-8275582: Don't purge metaspace mapping lists
stuefe at openjdk.java.net
Tue Oct 26 04:56:16 UTC 2021
On Mon, 25 Oct 2021 14:53:44 GMT, Coleen Phillimore <coleenp at openjdk.org> wrote:
>> Metaspace, at its lowest level, keeps a list of memory mappings. That list is dynamically expandable, which makes Metaspace expandable as well, one of the key features of this allocator.
>> There are two mechanisms in place in Metaspace to return memory to the OS:
>> 1) purging the list of mappings
>> 2) uncommitting free chunks
>> (1) is the old way, existing since the inception of Metaspace. It works by unmapping and deleting mapping segments which are fully evacuated. Metaspace allocations are fine granular and cannot move, so Metaspace is subject to fragmentation. That fragmentation makes (1) rather ineffective since we only can unmap a mapping that had been fully free'd after class unloading. The mappings however are quite large (and we plan to make them larger still), the chance of one being completely empty is small.
>> (2) was introduced with JEP-387 and is much more effective. We return memory not on a mapping level, but on a chunk level, by uncommitting free chunks larger than a certain threshold. That works very well.
>> When working on JEP-387, I left (1) in place out of caution. But (1) is really not needed anymore. It has no effect in combination with (2) because if a whole mapping is free and eligible for unmapping using technique(1), all its chunks must be free, will have been maximally folded by the buddy allocator, and their backing memory will have been completely uncommitted. So unmapping that segment has no effect on the process' RSS.
>> Removing (1) would have the following advantages:
>> - a nice reduction in code complexity
>> - reduce the work we do during class unloading
>> - the mapping list then would only ever grow, never shrink, which would simplify lockless accesses (see JDK-8271124)
>> - (1) stands in the way of other improvements, e.g. the increase of the mapping size of individual mappings, which would have diminished the effectiveness of (1) even further.
>> Removing the ability to purge the mapping list means we retain the *uncommitted* virtual address space of those mappings after spikes in class unloading. But the consequences of that are minimal, it does not affect RSS at all.
>> This patch removes old-style metaspace node purging completely. Therefore it's almost all line deletions.
>> - manual, x64 + x86 Linux, tested all metaspace related jtreg tests
>> - manually tested that we still reclaim memory as we did before
>> - GHAs
>> - SAP nightly tests
> Looks great!
Thanks @coleenp and @lkorinth for the reviews!
More information about the hotspot-runtime-dev