jtreg testing integrated

Andrew John Hughes gnu_andrew at member.fsf.org
Mon May 19 07:56:50 PDT 2008

2008/5/19 Mark Wielaard <mark at klomp.org>:
> Hi,
> I integrated jtreg testing of all the tests included with openjdk into
> icedtea. It uses a slightly hacked/merged jtreg/jtharness version that
> can be compiled out of the box without requiring any external libraries
> (that means, it doesn't support javax.help, javax.servlet, javax.comm,
> ant or junit - none of it is necessary for jdk testing). I'll posts the
> diffs soon to the appropriate jtreg and jtharness mailinglists for
> people interested in this "frankenstein" IcedJTReg (see also the
> attached README).

Thanks for doing this.  I'm certainly interested in a version of jtreg
that doesn't
depend on half the Java universe.

> What this means in practice is that you can now just do:
> ./configure && make && make check -k
> Except that you probably want to run "make jtregcheck -k" directly
> (otherwise make check will do an "empty build" of all of openjdk first,
> which takes a pretty long time, even if nothing should need
> recompilation).
> make jtregcheck -k runs the testsuites of hotspot (4 tests, all PASS),
> langtools (1,342 PASS, 1 FAIL - the version check) and jdk (2,875 tests
> of which about 130 fail - rerunning tests now). corba, jaxp and jaxws
> don't come with any tests. This takes about 3 hours on my machine.
> Most of the failures are because the host javaweb.sfbay.sun.com cannot
> be resolved. But there are also some genuine failures in java.awt.color,
> jmx.snmp, javax.script, javax.print, ... so enough to do for
> enterprising hackers!

Well my guess would be that the changes from proprietary JDK to
OpenJDK invalidate
a lot of these tests.  The crypto failures I guess will be looking for
an algorithm that's not
in the crypto that was finally released.   Likewise, I guess the
colour profiles will have
changed, invalidating these test.  JMX SNMP and javax.script are
obvious failures because
the first is stubbed and no implementation of the second is included in OpenJDK.

In summary, in most cases it's the tests that need fixing. Certainly
the sfbay ones need
changing to an external address, preferably something that can shake
off a heavy load ;)

> The detailed results of the tests can be found under test/jdk/JTreport,
> test/langtools/JTreport and test/hotspot/JTreport.
> You can also fire up the gui viewer to explore (and rerun) tests:
> java -jar test/jtreg.jar -v1 -a -ignore:quiet \
>  -w:test/jdk/JTwork -r:test/jdk/JTreport \
>  -jdk:`pwd`/openjdk/control/build/linux-amd64/j2sdk-image -gui \
>  `pwd`/openjdk/jdk/test
> (Replace your architecture and the testsuite name appropriately. Leave
> off the -v1 -a -ignore:quiet to run more tests.)
> If possible it would be nice if distributions could integrate this into
> their package build process and make the results available just like the
> integrated mauve testing the fedora and ubuntu are now doing (would
> someone care to integrate that into the icedtea build proper?)

I'd be interested to see the results -- do all Mauve tests now pass for IcedTea?

> Also I hope that over time distributions will package the full versions
> of jtreg and jtharness so it is available, with full support,
> everywhere. It really is a nice framework to make unit tests in.

Hopefully in separate packages so the core can be installed without
the ant, etc.

> Have fun!
> Mark

Andrew :-)

Support Free Java!
Contribute to GNU Classpath and the OpenJDK

PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8

More information about the distro-pkg-dev mailing list