RFR: 8024140: [TESTBUG] Profile based regression test groups for jdk repo
david.holmes at oracle.com
Wed Sep 4 03:46:24 UTC 2013
With thanks to Alan and Chris for their reviews I will push this initial
set of group definitions.
On 3/09/2013 1:02 PM, David Holmes wrote:
> Similar to what was recently done on the hotspot side, this introduces
> profile-based test groups for the jdk regression tests.
> The primary groups are:
> - jdk
> - jre
> - compact3
> - compact2
> - compact2_minimal
> - compact1
> - compact1_minimal
> The minimal VM is only supported on compact1 and compact2.
> The groups are defined hierarchically in two forms:
> - The need_xxx groups list all the tests that have a dependency on a
> specific profile. This is either because it tests a feature in that
> profile, or the test infrastructure uses a feature in that profile.
> - The primary groups are defined in terms of the other primary groups
> combined with the needs_xxx groups (including and excluding them as
> appropriate). For example the jre can run all tests from compact3, plus
> those from needs_jre, but excluding those from need_jdk.
> The bottom group defines all the actual tests to be considered, simply
> by listing the top-level test directories, and then excluding all the
> needs_xxx groups.
> To select a group of tests you use :<group-name>
> Eg to run only those tests that can run on compact1 use:
> jtreg :compact1
> Of course you still need to point jtreg at the right kind of runtime
> image (and give it a full JDK as the compile-jdk!); and if testing the
> minimal VM you need to tell jtreg to select it using -javaoptions:-minimal
> These top-level groups are not as useful standalone as they are for
> hotspot due to the fact that there are so many regression tests that are
> problematic in one way or another. In most cases what is desired is a
> way to run sub-set of tests for a given profile eg all jdk_core tests
> that can run on compact1. At present this would require a new group to
> be defined for each permutation and that is not scalable. I hope that in
> the near future jtreg will allow you to implicitly define such ad-hoc
> groups via the command-line, simply by listing groups to include and
> groups to exclude.
> The full jtreg group facility is only available in the most recent jtreg
> builds, so you will need to grab the latest nightly build, or latest
> Note: once this is in place, anyone writing regression tests will need
> to be aware of whether that test is limited to certain profiles and
> update the group file accordingly. Sometimes it is not the item being
> tested that determines the minimum needed profile, but the test
> infrastructure eg if it uses XML.
> See TEST.groups for more information.
> Note: The initial group definitions proposed here are not complete.
> There are over 4000 tests of the "desktop" that can not be readily
> executed manually and it may be that a number of these tests will
> require a JDK rather than the full JRE. Our full test processes will
> discover this and update the lists as needed. (Or we fix the tests to
> not need full JDKs.)
More information about the core-libs-dev