How to test OpenJDK?
Jonathan.Gibbons at Sun.COM
Sun May 31 14:48:18 PDT 2009
There is no reason why it should not be possible to have 100% pass rate
for most of the jdk tests as well. If there are problematic tests in
test suite, we should identify those tests, and then logically
tests into two groups -- those that should always be expected to pass,
those that don't, for whatever reason. That at least would allow to to
a quality bar for any OpenJDK build of which tests should always pass.
Frankly, I've never understood the attitude that it is OK to have
fail, and which don't deserve attention, but as long as we have such
we should identify them, using any of the available jtreg mechanisms,
such as keywords, @ignore, or exclude lists.
And, all the other, unmarked tests, should always pass, all the time,
if a test starts failing after you've changed code, you should assume
broken something and it deserves attention.
If necessary, we could/should have separate Makefile rules for testing
two different groups of tests, so that we have an obvious 100% pass
the tests in the main group of good tests.
On May 31, 2009, at 2:25 PM, Kelly O'Hair wrote:
> Cudos to Jon for having a testbase that is 100% pass, that IS the
> way to go. ;^)
> So I think that means you can
> cd langtools/test
> and either run jtreg, or use the Makefile
> make PRODUCT_HOME=your_j2sdk-image-dir JT_HOME=jtreg_home TESTDIRS=.
> How you run the jtreg tests is up to you.
> See http://openjdk.java.net/jtreg/faq.html
> Jonathan Gibbons wrote:
>> There are ~1500 tests in langtools/test, and all should pass.
>> -- Jon
>> On May 31, 2009, at 11:39 AM, Kelly O'Hair wrote:
>>> See http://openjdk.java.net/jtreg/
>>> I tend to use the jdk/test/Makefile myself:
>>> cd jdk/test && make PRODUCT_HOME=your_j2sdk-image-dir
>>> JT_HOME=jtreg_home TESTDIRS=java/lang
>>> There are quite a few tests in the jdk/test directory and the above
>>> only runs the java/lang tests, which should all pass.
>>> As to which tests reliably pass and which fail, I can't tell you.
>>> Maybe someone else can refer to something that documents what tests
>>> are expected to pass in their area.
>>> lianshun Liu wrote:
>>>> When I've finished building OpenJDK, how can I make sure that it
>>>> will work well? Is there any test suites that can be used to test
>>>> the JVM?
>>>> Thanks :)
>>>> Lianshun Liu
More information about the build-dev