jtreg testing integrated
Jonathan.Gibbons at Sun.COM
Mon May 19 09:52:23 PDT 2008
jtreg is now open source, as of just before JavaOne. See http://openjdk.java.net/jtreg
There is a small new utility called "jtdiff" that comes with jtreg
that may do what
you want. It will do n-way comparison of any JavaTest-type results
(meaning jtreg, JCK, etc)
where each set of results can be given as a JavaTest work directory,
or just the summary.txt file within a report directory. The output can
be plain text or HTML.
jtdiff is within jtreg.jar, so the easiest way to invoke it is
java -cp jtreg.jar com.sun.javatest.diff.Main <args>
On May 19, 2008, at 8:30 AM, Martin Buchholz wrote:
> [+compiler-dev, jtreg-use]
> On Mon, May 19, 2008 at 7:56 AM, Andrew John Hughes
> <gnu_andrew at member.fsf.org> wrote:
>> 2008/5/19 Mark Wielaard <mark at klomp.org>:
>>> make jtregcheck -k runs the testsuites of hotspot (4 tests, all
>>> langtools (1,342 PASS, 1 FAIL - the version check) and jdk (2,875
>>> of which about 130 fail - rerunning tests now). corba, jaxp and
>>> don't come with any tests. This takes about 3 hours on my machine.
> Once upon a time, I wrote a test that made sure the hotspot
> and jdk library's idea of the current version and supported targets
> were in sync. Unfortunately, it is not a requirement on hotspot
> integrations that they pass this test, so the test starts failing
> hotspot starts supporting the class file version number for the next
> major release. At least this is a strong hint to the javac team to
> catch up soon by incrementing their supported targets, etc...
> I like a policy of "Read my lips; no new test failures" but OpenJDK
> is not quite there; we get test failure creep when changes in
> one component break another component's tests.
>>> Most of the failures are because the host javaweb.sfbay.sun.com
>>> be resolved.
> The jtreg tests were originally designed to be run only by Sun JDK
> development and test engineers. If someone can come up with a
> portable way of testing network services (like ftp clients) without
> setting up a dedicated machine with a well-known name, that
> would be good. Alternatively, making the name of this
> machine configurable when jtreg is run would also
> be an improvement, and a much simpler one. But the obvious
> idea of using environment variables doesn't work. Most environment
> variables are not passed to the running java test program.
> If it's considered acceptable for IcedTea hackers to get their
> hands dirty with not-100%-free technology, y'all could try
> running the jtreg tests against IcedTea, vanilla OpenJDK7,
> OpenJDK6, and JDK 6u6, and comparing the test failures.
> I once wrote a script to compare two jtreg test runs, diff-javatest.
> Jonathan et al, could you work (with me) on releasing that as open
>>> But there are also some genuine failures in java.awt.color,
>>> jmx.snmp, javax.script, javax.print, ... so enough to do for
>>> enterprising hackers!
More information about the compiler-dev