RIP values like 0xffffffff94bf7f80 due to patched NMethod
alexandr.miloslavskiy at gmail.com
Tue Nov 13 23:24:51 UTC 2018
I'm currently investigating a strange JVM problem and decided to ask
here in hopes someone already saw such problem, or is able to provide clues.
One of our customers report that our java application crashes often for him.
I did quite a bit of debugging and got some facts:
1) Customer used Windows JRE x64 versions 8.0_144-b01, 10.0.1+10,
11.0.1+13-LTS. All of them crash with the same symptoms.
2) It crashes because java's NMethod executes a wild jmp, resulting in
crazy RIP values such as 0xffffffff94bf7f80.
3) Example of corrupted jmp:
00000000`042b2ac0 e9bb549490 jmp ffffffff`94bf7f80
4) I managed to understand that correct jmp should be:
00000000`042b2ac0 e9bb5494ff jmp 00000000`03bf7f80
5) Correct jmp address points to 'RuntimeStub: wrong_method_stub'
6) Just one byte of jmp instruction is corrupted with 0x90. It's always
0x90, and always the same byte is corrupted.
7) The crash occurs soon after jvm compiles a new NMethod for the same
8) The new NMethod is compiled with new optimization settings, usually
(but not always) it's 'CompLevel_full_profile' -->
9) The crash always occurs in the old NMethod.
10) The reason why 'jmp' is there is because old NMethod was
transitioned into 'non_entrant' state by 'nmethod::make_not_entrant()'
11) Customer says he doesn't have any java-specific tools installed such
as profilers, etc.
12) Customer provided around 20 crash logs and around 10 crash core
dumps. This is just a portion of his crashes. All of them exhibit the
13) Customer used Windows RAM test and it shown no errors. On the other
hand, error is too specific to be a hardware problem, I think: the last
byte of jmp gets corrupted with 0x90 when a new NMethod is compiled...
14) I have verified customer's jvm.dll and it's not corrupted.
15) I have verified (using core dump) the value of
'SharedRuntime::get_handle_wrong_method_stub()' and it's not corrupted.
16) In every core dump, only a single NMethod is corrupted.
More information about the hotspot-compiler-dev