RFR (XS): 8028159: C2: compiler stack overflow during inlining of @ForceInline methods
vladimir.x.ivanov at oracle.com
Wed Nov 13 06:22:59 PST 2013
12 lines changed: 7 ins; 3 del; 2 mod
There's a corner case when C2 inliner fails to delay inlining of
@ForceInline method (e.g. nested MethodHandle::asType(MethodType) calls
produce such pattern):
@ForceInline m1 -> MH::invokeBasic -> @FI m2 -> MH::invokeBasic ->
@FI m3 -> ...
Having method handle chain deep enough, it's possible to overflow
compiler thread stack and crash VM.
The fix is to limit maximum inlining depth for @ForceInline method. By
default, MaxForceInlineLimit (==100) is much larger than
MaxInlineLevel(=9) for ordinary methods.
Having it set to 100:
- no crash observed
- no inline bailouts due to MaxForceInlineLevel reached on octane
John wrote the following:
"In stress tests, method handles can be arbitrarily deep. In typical use
cases they are not; they are adapted a fixed number of times before
being installed in a call site. It may be that some of these extreme
cases can be solved by simply bailing out of the compile and continuing
A quick solution would be to have ForceInline fail after a larger stack
depth than normal.
Since part of the problem is the ForceInline directive, we need to ask
whether they are all necessary. If we are adjusting heuristics by
ForceInline, some of them can be removed, if we can detect the pattern
from inside C2."
Testing: failing test case, octane.
PS: I tried another approach and experimented with late inlining (delay
residual call chain inlining). Though I succeeded to delay inlining of
MH::invokeBasic, I hit the same problem there. Unless re-delay is
supported during late inlining, it's as easy to overflow compiler stack
as when inlining during parsing phase. Changing late inlining logic is
too much, IMO, for JDK8 at this phase.
More information about the hotspot-compiler-dev