RFR: 8233299: Implementation: JEP 365: ZGC on Windows

Stefan Karlsson stefan.karlsson at oracle.com
Tue Nov 5 10:50:45 UTC 2019

Thanks, Erik!


On 2019-11-05 11:40, erik.osterlund at oracle.com wrote:
> Hi Stefan,
> Awesome. Looks good!
> Thanks,
> /Erik
> On 10/31/19 11:18 AM, Stefan Karlsson wrote:
>> Hi all,
>> Please review this patch to add ZGC support on Windows.
>> https://cr.openjdk.java.net/~stefank/8233299/webrev.01/
>> https://bugs.openjdk.java.net/browse/JDK-8233299
>> As mentioned in the JEP (https://openjdk.java.net/jeps/365), there 
>> were some preparation patches that needed to go in to pave the way for 
>> this patch:
>>     8232601: ZGC: Parameterize the ZGranuleMap table size
>>     8232602: ZGC: Make ZGranuleMap ZAddress agnostic
>>     8232604: ZGC: Make ZVerifyViews mapping and unmapping precise
>>     8232648: ZGC: Move ATTRIBUTE_ALIGNED to the front of declarations
>>     8232649: ZGC: Add callbacks to ZMemoryManager
>>     8232650: ZGC: Add initialization hooks for OS specific code
>>     8232651: Add implementation of os::processor_id() for Windows
>> ... they have all been pushed now.
>> One important key-point to this implementation is to use the new 
>> Windows APIs that support reservation and mapping of memory through 
>> "placeholders":  VirtualAlloc2, VirtualFreeEx, MapViewOfFile3, and 
>> UnmapViewOfFile2. These functions are available starting from version 
>> 1803 of Windows 10 and Windows Server. ZGC will lookup these symbols 
>> to determine if the Windows version supports these functions.
>> Correlating the text in the JEP with the code:
>> * '"Support for multi-mapping memory". ZGC's use of colored pointers 
>> requires support for heap multi-mapping, so that the same physical 
>> memory can be accessed from multiple different locations in the 
>> process address space. On Windows, paging-file backed memory provides 
>> physical memory with an identity (a handle), which is unrelated to the 
>> virtual address where it is mapped. Using this identity allows ZGC to 
>> map the same physical memory into multiple locations.'
>> We commit memory via paging file mappings and map views into that memory.
>> The function ZMapper::create_and_commit_paging_file_mapping uses 
>> CreateFileMappingW with SEC_RESERVE to create this mapping, 
>> MapViewOfFile3 to map a temporary view into the mapping, VirtualAlloc2 
>> to commit the memory, and then UnmapViewOfFile2 to unmap the view.
>> The reason to use SEC_RESERVE and the extra VirtualAlloc2, instead of 
>> SEC_COMMIT, is to ensure that the later multi-mappings of committed 
>> file mappings don't fail under low-memory situations. Earlier 
>> prototypes used SEC_COMMIT and saw these kind of OOME errors when 
>> mapping new views to already committed memory. The current 
>> platform-independent ZGC code isn't prepared to handle OOME errors 
>> when mapping views, so we chose this solution.
>> MapViewOfFile3 is then used to multi-map into the committed memory.
>> * '"Support for mapping paging-file backed memory into a reserved 
>> address space". The Windows memory management API is not as flexible 
>> as POSIX's mmap/munmap, especially when it comes to mapping file 
>> backed memory into a previously reserved address space region. To do 
>> this, ZGC will use the Windows concept of address space placeholders. 
>> The placeholder concept was introduced in version 1803 of Windows 10 
>> and Windows Server. ZGC support for older versions of Windows will not 
>> be implemented.'
>> Before the placeholder APIs there was no way to first reserve a 
>> specific virtual memory range, and then map a view of a committed 
>> paging file over that range. The VirtuaAlloc function could be used to 
>> first reserve and then commit anonymous memory, but nothing similar 
>> existed for mapped views. Now with placeholders, we can create a 
>> placeholder reservation of memory with VirtualAlloc2, and then replace 
>> that reservation with MapViewOfFile3. When memory is unmapped, we can 
>> use UnmapViewOfFile2 to "preserve" the placeholder memory reservation.
>> * '"Support for mapping and unmapping arbitrary parts of the heap". 
>> ZGC's heap layout in combination with its dynamic sizing (and 
>> re-sizing) of heap pages requires support for mapping and unmapping 
>> arbitrary heap granules. This requirement in combination with Windows 
>> address space placeholders requires special attention, since 
>> placeholders must be explicitly split/coalesced by the program, as 
>> opposed to being automatically split/coalesced by the operating system 
>> (as on Linux).'
>> Half of the preparation patches were put in place to support this. 
>> When replacing a placeholder with a view of the backing file, we need 
>> to exactly match the address and size of a placeholder. Also, when 
>> unmapping a view, we need to exactly match the address and size of the 
>> view, and replace it with a placeholder.
>> To make it easier to map and unmap arbitrary parts of the heap, we 
>> split reserved memory into ZGranuleSize-sized placeholders. So, 
>> whenever we perform any of these operations, we know that any given 
>> memory range could be dealt with as a number of granules.
>> When memory is reserved, but not mapped, it is registered in the 
>> ZVirtualMemoryManager. It splits memory into granule-sized placholders 
>> when reserved memory is fetched, and coalesces placeholders when 
>> reserved memory is handed back.
>> * '"Support for committing and uncommitting arbitrary parts of the 
>> heap". ZGC can commit and uncommit physical memory dynamically while 
>> the Java program is running. To support these operations the physical 
>> memory will be divided into, and backed by, multiple paging-file 
>> segments. Each paging-file segment corresponds to a ZGC heap granule, 
>> and can be committed and uncommitted independently of other segments.'
>> Just like we can map and unmap in granules, we want to be able to 
>> commit and uncommit memory in granules. You can see how memory is 
>> committed and uncommitted in granules in 
>> ZBackingFile::commit_from_paging_file and 
>> ZBackingFile::uncommit_from_paging_file. Each committed granule is 
>> associated with one registered handle. When memory for a granule is 
>> uncommitted, the handle is closed. At this point, no views exist to 
>> the mapping and the memory is handed back to the OS.
>> Final point about ZPhysicalMemoryBacking. We've tried to make this 
>> file similar on all OSes, with the hope to be able to combine them 
>> when both the Windows and macOS ports have been merged.
>> Thanks,
>> StefanK

More information about the hotspot-gc-dev mailing list