RFR (S): 8011138: C2: stack overflow in compiler thread because of recursive inlining of lambda form methods
christian.thalinger at oracle.com
Wed Oct 2 16:04:01 PDT 2013
8011138: C2: stack overflow in compiler thread because of recursive inlining of lambda form methods
The fix is to walk the JVM states when inlining compiled lambda forms and try to figure out if the "receiver" of the same inlinee (static method in the compiled lambda form) is the same as the current one. If that's the case we have a recursion.
I was tempted to remove the InlineTree::_caller_jvms completely but that might be too intrusive for now.
Tested with the failing test case and Octane on Nashorn to verify that I didn't break anything.
More information about the hotspot-compiler-dev