sigsegv on porter stemmer (Lucene, but also otherwise)

Vladimir Kozlov vladimir.kozlov at
Mon Jul 25 15:06:02 PDT 2011

Thank you, Uwe

I looked on logs and most of them, it seems, related to the problem described in 
your bug 7070134.

And I looked on LUCENE-2975 which describes DataInput.readVInt() problem. It 
references Hotspot bug 5091921. The loop optimization changes I mentioned before 
were done to fix that bug so it should be fixed in jdk7. But, as we know now, 
those changes introduced several new problems, like yours 7070134. I am working 
on them.


Uwe Schindler wrote:
> Hi thanks for taking care!
> Thanks for the workaround. We already found another workaround to get this
> running in our 2-hourly Lucene builds at:
> It would be nice if you could look into the console logs of the failed
> builds on Saturday - you can see the bug in the earlier builds only (with
> always different stack traces). We drilled it down to one method (not sure
> if this information clipped out of the bug report. We then disabled
> compilation for only this affected method, its PorterStemmer.ends(...):
> -XX:CompileCommand=exclude,org/apache/lucene/analysis/en/PorterStemmer,ends
> -XX:CompileCommand=exclude,org/apache/lucene/analysis/PorterStemmer,ends
> (we have different class names in stable 3.x branch and trunk).
> We also see other random test failures not happening with Java 5 and Java 6,
> it would be nice, if you could review, too.
> One big bug in loops affected also Java 1.6.0_18 (still not fixed): our
> DataInput.readVInt method was incorrectly compiled in the case that
> MappedByteBuffer.get()/lucene.DataInput.readByte() was inlined, leading to
> simply wrong results (the method returned a decoded integer that was
> different than expected results). See the unwinded loops in
> /lucene/store/
> I hope this all helps you in finding more bad loop optimization bugs, all of
> those issues seem to be related to this special optimization in loops. The
> latest lucene builds also contains a failure in a test case only happening
> with Java 7 (not on every test run, so unreproducible). So it might be good
> for you to watch our Lucene builds also for other bugs.
> Some of the other developers already say, we should not trust any loops in
> java anymore and recommend not to use Java 7 with Apache Lucene/Solr, and
> that's bad news :(
> Thanks for the help,
> Uwe
> On Mon, Jul 25, 2011 at 7:49 PM, Vladimir Kozlov
> <vladimir.kozlov at> wrote:
>> Thank you very much, Dawid, for providing the test case. Bug was filed in
>> wrong category so we did not know about it. I will work on it since it
> could
>> be my changes in loop optimizations. Use next flag as workaround:
>> -XX:-UseLoopPredicate
>> Thanks,
>> Vladimir
>> Dawid Weiss wrote:
>>> Hello everyone,
>>> I am an Apache Lucene developer, we've been running tests with Java
>>> 1.7 and this came up:
>>> Porter stemmer is pretty widely used for shallow NLP, not only in
>>> Lucene. It'd be interesting to hear from jit gurus what's causing this
>>> (the problem does not occur in 1.6). Thanks in advance,
>>> Dawid
> -----
> Uwe Schindler
> uschindler at 
> Apache Lucene PMC Member / Committer
> Bremen, Germany

More information about the hotspot-compiler-dev mailing list