jtreg testing integrated
martinrb at google.com
Mon May 19 08:30:57 PDT 2008
On Mon, May 19, 2008 at 7:56 AM, Andrew John Hughes
<gnu_andrew at member.fsf.org> wrote:
> 2008/5/19 Mark Wielaard <mark at klomp.org>:
>> make jtregcheck -k runs the testsuites of hotspot (4 tests, all PASS),
>> langtools (1,342 PASS, 1 FAIL - the version check) and jdk (2,875 tests
>> of which about 130 fail - rerunning tests now). corba, jaxp and jaxws
>> don't come with any tests. This takes about 3 hours on my machine.
Once upon a time, I wrote a test that made sure the hotspot
and jdk library's idea of the current version and supported targets
were in sync. Unfortunately, it is not a requirement on hotspot
integrations that they pass this test, so the test starts failing whenever
hotspot starts supporting the class file version number for the next
major release. At least this is a strong hint to the javac team to
catch up soon by incrementing their supported targets, etc...
I like a policy of "Read my lips; no new test failures" but OpenJDK
is not quite there; we get test failure creep when changes in
one component break another component's tests.
>> Most of the failures are because the host javaweb.sfbay.sun.com cannot
>> be resolved.
The jtreg tests were originally designed to be run only by Sun JDK
development and test engineers. If someone can come up with a
portable way of testing network services (like ftp clients) without
setting up a dedicated machine with a well-known name, that
would be good. Alternatively, making the name of this
machine configurable when jtreg is run would also
be an improvement, and a much simpler one. But the obvious
idea of using environment variables doesn't work. Most environment
variables are not passed to the running java test program.
If it's considered acceptable for IcedTea hackers to get their
hands dirty with not-100%-free technology, y'all could try
running the jtreg tests against IcedTea, vanilla OpenJDK7,
OpenJDK6, and JDK 6u6, and comparing the test failures.
I once wrote a script to compare two jtreg test runs, diff-javatest.
Jonathan et al, could you work (with me) on releasing that as open source?
>> But there are also some genuine failures in java.awt.color,
>> jmx.snmp, javax.script, javax.print, ... so enough to do for
>> enterprising hackers!
More information about the compiler-dev