Need reviewers - jdk testing changes 6888927

Kelly O'Hair Kelly.Ohair at Sun.COM
Wed Nov 4 17:19:55 UTC 2009

Tim Bell wrote:
> Hi Kelly
>> I need some formal reviewers on these jdk/test/Makefile changes.
>> More polish and changes may need to be done later, but there
>> is value in what I have now, and I need lots of help to improve
>> things (and fix some of our testcases).
> This is a great start.
>> Here is the bugid and webrev:
>> 6888927: Fix jdk jtreg tests to indicate which ones need othervm, allow for use of samevm option
>> The goal was to see if we could easily run most of the regression
>> tests in the jdk/test directory, in minimum time, and without so
>> much noise (e.g. do not run unstable or problematic tests).
>> This could then be used as a benchmark to validate some quality measure
>> of the jdk7 that was built.
>> The primary changes include:
>>    * New jdk/test/Makefile targets: jdk_all jdk_lang jdk_util jdk_nio ...
>>    * Addition of a jdk/test/ProblemList file to be used by jdk/test/Makefile
>> I tried to group all the tests (by directory names) into jdk_* sets, then
>> tried to balance them as to how long each set ran and if they could be run
>> with the jtreg -samevm option. The balancing was tricky, and will need more
>> polish, along with the virtual frame buffer idea. I also had to give up
>> on the awt, rmi, and swing tests until the Xvfb issues are figured out.
> I wrote that script for you, but now I can see that I
> need to take a closer look at it.  On recent OpenSolaris installs,
> /usr/openwin is a symlink to /usr/X11, so my if test on line 20
> is dubious at best.

I had to tweek that part of it, I wanted to favor /usr/X11, then go
after /usr/openwin.

> I will take a look tomorrow and see if I can find a solution
> that works better for Solaris 10, OpenSolaris, and Linux.

I got it working, I think, it was a little tricky to get it to die,
probably need a trap line in the script to make sure that when the
script dies, it takes the display and manager with it.

But the bigger problem was that the awt tests and even swing tests
seem to be able to bring the Xvfb down easily.

>> I chose to not actually modify the tests themselves if they needed fixing,
>> it was just too overwhelming. So developers and teams may want to browse the
>> ProblemList for their favorite tests (which aren't my favorites ;^) and
>> consider what they might want to do.
>> With this new jdk/test/Makefile, anybody can:
>>    cd jdk/test
>>    gnumake [PRODUCT_HOME=${YOUR_JDK7_HOME}] [JT_HOME=${YOUR_JTREG_HOME}] jdk_all
>> And run about 3,000+ tests that SHOULD PASS in roughly 2 hours if your
>> machine isn't too slow and old. I was able to run it in 90 minutes on a
>> monster OpenSolaris AMD machine which had 16Gb RAM and 16 2.2Ghz cpus.
> For the record, that machine is a SunFire X2250 [1] which presents a total
> of 16 2267 MHz CPUs to the O/S.  Kelly, Kumar, and I have been dumping work
> on this system for almost a week.  It hasn't blinked - it grinds through
> everything we throw at it.  I am impressed by what our Sun hardware colleagues
> have been able to squeeze into one rack unit.  A few years ago, a server this
> powerful would have been as large as a microwave oven, if not a refrigerator.
> Thanks are due to the Sun lab managers who recently gave us two of these for
> JDK7 use.
>> Then I added "-j 4" to the gnumake command line and it only took 30mins!
>> So the various jdk_* make targets can be run in parallel.
> Very cool!  I have never attempted that before, with any of our test
> suites.  In the past, too many tests left dirt behind and caused
> many innocent test that might run later to fail...

Getting 3,000+ stable tests run in 30mins will be handy.

Managing the ProblemList (as Jon noted) will be a pain at times.


> Tim
> [1] Community info:
>         Sales info:

More information about the build-dev mailing list