RFR: 8233299: Implementation: JEP 365: ZGC on Windows
per.liden at oracle.com
Tue Nov 5 10:44:14 UTC 2019
On 10/31/19 11:18 AM, Stefan Karlsson wrote:
> Hi all,
> Please review this patch to add ZGC support on Windows.
> As mentioned in the JEP (https://openjdk.java.net/jeps/365), there were
> some preparation patches that needed to go in to pave the way for this
> 8232601: ZGC: Parameterize the ZGranuleMap table size
> 8232602: ZGC: Make ZGranuleMap ZAddress agnostic
> 8232604: ZGC: Make ZVerifyViews mapping and unmapping precise
> 8232648: ZGC: Move ATTRIBUTE_ALIGNED to the front of declarations
> 8232649: ZGC: Add callbacks to ZMemoryManager
> 8232650: ZGC: Add initialization hooks for OS specific code
> 8232651: Add implementation of os::processor_id() for Windows
> ... they have all been pushed now.
> One important key-point to this implementation is to use the new Windows
> APIs that support reservation and mapping of memory through
> "placeholders": VirtualAlloc2, VirtualFreeEx, MapViewOfFile3, and
> UnmapViewOfFile2. These functions are available starting from version
> 1803 of Windows 10 and Windows Server. ZGC will lookup these symbols to
> determine if the Windows version supports these functions.
> Correlating the text in the JEP with the code:
> * '"Support for multi-mapping memory". ZGC's use of colored pointers
> requires support for heap multi-mapping, so that the same physical
> memory can be accessed from multiple different locations in the process
> address space. On Windows, paging-file backed memory provides physical
> memory with an identity (a handle), which is unrelated to the virtual
> address where it is mapped. Using this identity allows ZGC to map the
> same physical memory into multiple locations.'
> We commit memory via paging file mappings and map views into that memory.
> The function ZMapper::create_and_commit_paging_file_mapping uses
> CreateFileMappingW with SEC_RESERVE to create this mapping,
> MapViewOfFile3 to map a temporary view into the mapping, VirtualAlloc2
> to commit the memory, and then UnmapViewOfFile2 to unmap the view.
> The reason to use SEC_RESERVE and the extra VirtualAlloc2, instead of
> SEC_COMMIT, is to ensure that the later multi-mappings of committed file
> mappings don't fail under low-memory situations. Earlier prototypes used
> SEC_COMMIT and saw these kind of OOME errors when mapping new views to
> already committed memory. The current platform-independent ZGC code
> isn't prepared to handle OOME errors when mapping views, so we chose
> this solution.
> MapViewOfFile3 is then used to multi-map into the committed memory.
> * '"Support for mapping paging-file backed memory into a reserved
> address space". The Windows memory management API is not as flexible as
> POSIX's mmap/munmap, especially when it comes to mapping file backed
> memory into a previously reserved address space region. To do this, ZGC
> will use the Windows concept of address space placeholders. The
> placeholder concept was introduced in version 1803 of Windows 10 and
> Windows Server. ZGC support for older versions of Windows will not be
> Before the placeholder APIs there was no way to first reserve a specific
> virtual memory range, and then map a view of a committed paging file
> over that range. The VirtuaAlloc function could be used to first reserve
> and then commit anonymous memory, but nothing similar existed for mapped
> views. Now with placeholders, we can create a placeholder reservation of
> memory with VirtualAlloc2, and then replace that reservation with
> MapViewOfFile3. When memory is unmapped, we can use UnmapViewOfFile2 to
> "preserve" the placeholder memory reservation.
> * '"Support for mapping and unmapping arbitrary parts of the heap".
> ZGC's heap layout in combination with its dynamic sizing (and re-sizing)
> of heap pages requires support for mapping and unmapping arbitrary heap
> granules. This requirement in combination with Windows address space
> placeholders requires special attention, since placeholders must be
> explicitly split/coalesced by the program, as opposed to being
> automatically split/coalesced by the operating system (as on Linux).'
> Half of the preparation patches were put in place to support this. When
> replacing a placeholder with a view of the backing file, we need to
> exactly match the address and size of a placeholder. Also, when
> unmapping a view, we need to exactly match the address and size of the
> view, and replace it with a placeholder.
> To make it easier to map and unmap arbitrary parts of the heap, we split
> reserved memory into ZGranuleSize-sized placeholders. So, whenever we
> perform any of these operations, we know that any given memory range
> could be dealt with as a number of granules.
> When memory is reserved, but not mapped, it is registered in the
> ZVirtualMemoryManager. It splits memory into granule-sized placholders
> when reserved memory is fetched, and coalesces placeholders when
> reserved memory is handed back.
> * '"Support for committing and uncommitting arbitrary parts of the
> heap". ZGC can commit and uncommit physical memory dynamically while the
> Java program is running. To support these operations the physical memory
> will be divided into, and backed by, multiple paging-file segments. Each
> paging-file segment corresponds to a ZGC heap granule, and can be
> committed and uncommitted independently of other segments.'
> Just like we can map and unmap in granules, we want to be able to commit
> and uncommit memory in granules. You can see how memory is committed and
> uncommitted in granules in ZBackingFile::commit_from_paging_file and
> ZBackingFile::uncommit_from_paging_file. Each committed granule is
> associated with one registered handle. When memory for a granule is
> uncommitted, the handle is closed. At this point, no views exist to the
> mapping and the memory is handed back to the OS.
> Final point about ZPhysicalMemoryBacking. We've tried to make this file
> similar on all OSes, with the hope to be able to combine them when both
> the Windows and macOS ports have been merged.
More information about the hotspot-gc-dev