[icedtea-web] RFC: disable tests that are known to fail
jvanek at redhat.com
Wed Mar 21 09:28:45 PDT 2012
On 03/21/2012 04:54 PM, Omair Majid wrote:
> Hi Jiri,
> Thanks for your sharing your thoughts.
> On 03/21/2012 11:12 AM, Jiri Vanek wrote:
>> Personally I'm against the disabling. My opinion is that once thy will
>> be marked as @Ignored, they will be forgotten.
> Our command line output, as well as most IDEs, should show the number of
> tests that have been ignored.
Yap, but noone will ever care any more why those tests are ignored.
>> In this matter I'm
>> really missing TestNg grouping, and put them to some group "should
>> fail" with description "until fix xyz is done". But I'm willing to add
>> such annotation rather then disabling them.
> JUnit does support grouping, but the interface to it (if you are not
> using maven) is rather crappy.
>> Falling test is still
>> representing the issue, what I consider as correct.
> I agree. But if these tests always fail, then developers (I am guilty of
> this too) will often ignore running the tests or not pay too much
> attention to its output. For a developer, the impact of going from 0
> failures to 1 failure is significant, but the impact of going from 20
> failures to 21 is not so much. And as a counter-argument, these tests
To much true!
> have been failing for a while now and they haven't been fixed. What's
> the value of these tests?
Thy are still representing the issue.
> To be honest, I don't think there is a "right" answer here, but I am
> trying to figure out what's the lesser evil and go with that.
As an intersection looks to annotate them @KnownToBeFailing, and allow to run make tests with
configured something like skipKnownToBeFailing, which will skip all KnownToBeFailing testmethods.
Spam & Advertisement:
This have close connection to @Bug annotation I have posted yesterday O:)
More information about the distro-pkg-dev