JMH runner should fail on error
thomas.johnson at lmax.com
Wed Feb 11 13:28:38 UTC 2015
We're using JMH benchmarks as part of our continuous integration pipeline, and we just had a situation where a benchmark was committed in a state which threw exceptions on execution. The result was that the benchmark job ran, failed early reporting errors to standard out, and returned 0. This was surprising - we had expected any failed process to return a non-zero result. Outputting to standard error is fine if you're running it manually from the command line, but not if it's part of a workflow where you want to fail on error. As a result, it didn't become apparent that there was a problem for some time.
Is there a reason why the runner (org.openjdk.jmh.Main) doesn't exit with a nonzero return value when it's detected what's clearly an error?
LMAX Exchange, Yellow Building, 1A Nicholas Road, London W11 4AN
2014 #1 Fastest Growing Tech Company in the UK - Sunday Times Tech Track 100
2014 Best Margin Sector Platform - Profit & Loss Readers' Choice Awards
2014 Best FX Trading Venue - ECN/MTF - WSL Institutional Trading Awards
2014 Best Infrastructure/Technology Initiative - WSL Institutional Trading Awards
2013 #15 Fastest Growing Tech Company in the UK - Sunday Times Tech Track 100
2013 Best Overall Testing Project - The European Software Testing Awards
2013 Best Margin Sector Platform - Profit & Loss Readers' Choice Awards
2013 Best FX Trading Platform - ECN/MTF - WSL Institutional Trading Awards
2013 Best Executing Venue - Forex Magnates Awards
FX and CFDs are leveraged products that can result in losses exceeding
your deposit. They are not suitable for everyone so please ensure you fully
understand the risks involved.
The information on this email is not directed at residents of the United States of America,
Australia (we will only deal with Australian clients who are "wholesale clients" as defined
under the Corporations Act 2001), Canada (although we may deal with Canadian residents
who meet the "Permitted Client" criteria), Singapore or any other jurisdiction where
FX trading and/or CFD trading is restricted or prohibited by local laws or regulations.
The information in this email and any attachment is confidential and is
intended only for the named recipient(s). The email may not be disclosed
or used by any person other than the addressee, nor may it be copied
in any way. If you are not the intended recipient please notify the sender
immediately and delete any copies of this message. Any unauthorised
copying, disclosure or distribution of the material in this e-mail
is strictly forbidden.
LMAX Limited is regulated by the Financial Conduct Authority under
the UK laws, which differ from Australian law. We are exempt from the
requirement to hold an Australian financial services licence under the
Corporations Act 2001 (Cth) (Act) in respect of the financial services
we offer to you. We only offer our services to Australian clients who are
"wholesale clients" as defined under the Act. We may provide services in
Canada as an exempt international advisor. Consequently we may only
provide such services to clients who meet the "Permitted Client" criteria.
We are not a dealer in Canada.
LMAX Limited operates a multilateral trading facility. LMAX Limited is
authorised and regulated by the Financial Conduct Authority (firm
registration number 509778) and is a company registered in England
and Wales (number 6505809). Our registered address is Yellow
Building, 1A Nicholas Road, London, W11 4AN.
More information about the jmh-dev