[rfc][cedtea-web] another reproducers stabilization
aazores at redhat.com
Thu Jan 16 07:00:46 PST 2014
On 01/16/2014 09:50 AM, Jiri Vanek wrote:
> On 01/16/2014 03:45 PM, Andrew Azores wrote:
>> On 01/16/2014 08:58 AM, Jiri Vanek wrote:
>>> Thsi is removing more xception checks from itw. By this patch
>>> another 300 reproducers will become
>>> stably green. Unluckily, the revert of
>>> is stil needed, and
>>> intersection of fixed tests is zero :(
>> Why are these checks now invalid? Are the exceptions no longer
>> printed for some reason, or are they
>> going to stdout rather than stderr? Or are they now catching false
>> negatives ie exceptions are
>> occurring that do not actually signify test failures?
> They were ensuring that *no* *general* exception occurred. Unluckily
> in many test runs some completely unrelated, and absolutely harmless
> exceptions are printed. The torturing of browsers/java by repeated
> kill -9 do this.
> So Although at first the tests were "pessimists" == everything have to
> be ok. Now they are "optimistic" == just the core of reproducer passed.
> Although I'm fan of "pessimistic" tests. In this case it caused the
> great instability of engine - random failures each run.
Okay, that's what I thought was going on. Seems fine to me then. I'd
prefer pessimistic testing, too, but it's not worth it if it incurs
random failures and instability.
More information about the distro-pkg-dev