RFR: 8212933: Thread-SMR: requesting a VM operation whilst holding a ThreadsListHandle can cause deadlocks
robbin.ehn at oracle.com
Fri Oct 26 14:33:18 UTC 2018
Hi, please review.
When the VM thread executes a handshake it uses different ThreadsLists during
the execution. A JavaThread that is armed for the handshake when it is already
in the exit path in VM will cancel the handshake. Even if the VM thread cannot
see this thread after the initial ThreadsList which where used for arming, the
handshake can progress when the exiting thread cancels the handshake.
But if a third thread takes a ThreadsList where the exiting JavaThread is
present and tries to execute a VM operation, hence waiting on VM thread to
finish the handshake, the JavaThread in the exit path can never reach the
handshake cancellation point. VM thread cannot finishes the handshake and the
third thread is stuck waiting on the VM thread.
To allow holding a ThreadsList when executing a VM operation we instead let the
VM thread use the same ThreadsList over the entire handshake making all armed
threads visible to the VM thread at all time. And if VM thread spots a
terminated thread it will count that thread is already done by only clearing
Passes local stress testing, t1-5 and the deadlock is no longer reproduce-able.
Added a jtreg handshake + thread suspend test as a reproducer.
More information about the serviceability-dev