[icedtea-web] RFC: disable tests that are known to fail
omajid at redhat.com
Wed Mar 21 08:54:55 PDT 2012
Thanks for your sharing your thoughts.
On 03/21/2012 11:12 AM, Jiri Vanek wrote:
> Personally I'm against the disabling. My opinion is that once thy will
> be marked as @Ignored, they will be forgotten.
Our command line output, as well as most IDEs, should show the number of
tests that have been ignored.
> In this matter I'm
> really missing TestNg grouping, and put them to some group "should
> fail" with description "until fix xyz is done". But I'm willing to add
> such annotation rather then disabling them.
JUnit does support grouping, but the interface to it (if you are not
using maven) is rather crappy.
> Falling test is still
> representing the issue, what I consider as correct.
I agree. But if these tests always fail, then developers (I am guilty of
this too) will often ignore running the tests or not pay too much
attention to its output. For a developer, the impact of going from 0
failures to 1 failure is significant, but the impact of going from 20
failures to 21 is not so much. And as a counter-argument, these tests
have been failing for a while now and they haven't been fixed. What's
the value of these tests?
To be honest, I don't think there is a "right" answer here, but I am
trying to figure out what's the lesser evil and go with that.
More information about the distro-pkg-dev