JMH does not terminate on error

Alex Averbuch alex.averbuch at
Thu Sep 8 15:41:16 UTC 2016

I have a benchmark that throws an exception. In spirit, it does this:

    @BenchmarkMode( {Mode.SampleTime} )
    public void doError( )

Via OptionsBuilder, Runner has been configured to fail as soon as it
encounters an error, like this:

  ChainedOptionsBuilder baseBuilder = new OptionsBuilder()
                .shouldFailOnError( true )
                .timeout( TimeValue.hours( 1 ) )

Note also that timeout is 1 hour <-- this is important to the behavior I am
seeing, I think.

When running the doError() benchmark it hangs.
It hangs for a very long time. I suspect it would be for one hour, but have
not had the patience to wait.
Instead, I changed the timeout to 10 seconds, which resulted in the
benchmark failing (as expected) in ~10 seconds.

Is this intended behavior?

It feels like a bug. At the least, it is non-intuitive.

Here is where we see it hanging:

More information about the jmh-dev mailing list