Stack guard pages

Andrew John Hughes gnu_andrew at
Tue Feb 23 11:15:11 PST 2010

On 23 February 2010 16:16, Andrew Haley <aph at> wrote:
> I've been working on a bug in OpenOffice where using Java causes a
> later crash.
> It's a very strange bug: Java is called, does something, and then the
> thread is detached from the Java VM.  Long after, a segfault occurs.
> The reason for this crash is the way that stack guard pages work.
> When DetachCurrentThread() calls
> JavaThread::remove_stack_guard_pages(), the guard pages are not
> removed.  So, if at some point later in the process the stack grows
> beyond the point where the guard pages were mapped, the Linux kernel
> cannot expand the stack any further because the guard pages are in the
> way, and a segfault occurs.
> I've attached a reproducer for this bug to the end of this message.
> It crashes on many of the Linux systems I've tried.
> The right way to fix this is for remove_stack_guard_pages() to
> munmap() the guard pages.  However, it's essential not to split the
> stack region, so if the stack has already grown beyond the guard
> pages, we have to unmap() it.  Like so:
> bool os::create_stack_guard_pages(char* addr, size_t size) {
>  uintptr_t stack_extent, stack_base;
>  // get_stack_bounds() returns the memory extent mapped for the stack
>  // by the kernel.
>  if (get_stack_bounds(&stack_extent, &stack_base)) {
>    if (stack_extent < (uintptr_t)addr)
>      ::munmap((void*)stack_extent, (uintptr_t)addr - stack_extent);
>  }
>  return os::commit_memory(addr, size);
> }
> bool os::remove_stack_guard_pages(char* addr, size_t size) {
>  return ::munmap(addr, size) == 0;
> }
> This fixes the bug.  Unfortunately, there is no os:: interface for
> create_stack_guard_pages() and remove_stack_guard_pages(), so this
> solution doesn't fit well with the current organization of the VM.  I
> can't see any way of making this work only by touching os_linux.?pp .
> I could simply patch OpenJDK locally for the Linux distros, but I
> think we should have this discussion first.
> Andrew.

Segfaults here with the latest OpenJDK7 b84:

$ /mnt/builder/jdk7/j2sdk-image/bin/java -version
openjdk version "1.7.0-internal"
OpenJDK Runtime Environment (build 1.7.0-internal-andrew_2010_02_23_18_45-b00)
OpenJDK 64-Bit Server VM (build 17.0-b09, mixed mode)

$ LD_LIBRARY_PATH=/mnt/builder/jdk7/j2sdk-image/jre/lib/amd64/server ./invoke
Segmentation fault

I think it would it would be clearer what the suggested fix entails if
you posted a diff or webrev of the suggested change.

Andrew :-)

Free Java Software Engineer
Red Hat, Inc. (

Support Free Java!
Contribute to GNU Classpath and the OpenJDK

PGP Key: 94EFD9D8 (
Fingerprint: F8EF F1EA 401E 2E60 15FA  7927 142C 2591 94EF D9D8

More information about the hotspot-dev mailing list