Joe.Darcy at Sun.COM
Fri May 23 16:45:45 PDT 2008
Alan Bateman wrote:
> Joe Darcy wrote:
>> Recently Mark Wielaard started an effort to run the regression tests
>> on OpenJDK builds and publish the results
>> Certain tests in the nio and networking area assume Sun-internal
>> hosts like javaweb are available; the shell test
>> has a string for javaweb and
>> has the following constants used by other tests:
>> // Test hosts used by the channels tests - change these when
>> // executing in a different network.
>> public static final String HOST = "javaweb.sfbay.sun.com";
>> public static final String REFUSING_HOST = "jano.sfbay.sun.com";
>> public static final String FAR_HOST = "theclub.ireland.sun.com";
>> public static final String UNRESOLVABLE_HOST =
>> Mark observed these three hosts are needed:
>>> With daytime and echo available. The only wrinkle is that some tests
>>> expect to be able to use quick timeouts from HOST, but to need long
>>> timeouts from FAR_HOST. I think we could provide something public for
>>> this. But I might have missed some tests that need other services. Do
>>> you have a full overview?
>> First, is this all the networking configuration information that is
>> used in the regression tests?
> For the tests in the java/nio and sun/nio tree then the dependencies
> are centralized in TestUtil.java. When we moved these tests from
> closed->open we did talk about moving this configuration into a
> properties/configuration file but figured there wasn't much difference
> between editing a config file vs. a source file.
I think being able to pass in your own config file would be useful for
this use case; it is awkward to have to edit the source to the tests to
get them to pass :-) If -Dnetworking.test.config.file were defined, the
test could use those values instead of the defaults...
> For the classic networking, the dependencies are scattered. I believe
> some of the tests weren't moved from closed->open for this reason.
> Some of the tests uses an embedded HTTP or FTP server but there are
> others that depend on DNS, proxy, etc. Michael or Jessie may have more
> to say on this.
Could the classic networking tests be refactored a bit to be centralized
like the nio tests? Would it be natural for classic networking and nio
to get their host information from the same place?
> BTW: I briefly scanned the summary and most of the java/nio failures
> appear to be *HOST related. Now that folks are running these tests
> outside Sun it is time to own up to a few intermittent/timing related
> failures with 4 or 5 of these tests. I've been meaning to fix these
> test issues for a while but I haven't had the cycles. I'll dig up my
> notes on these. One other thing - I notice
> test/java/nio/channels/DatagramChannel/NotBound.java on the list -
> that test is failing because it is missing the public modifier. The
> test is bogus anyway so it might be best to just remove this test from
> jdk6 and open6. For jdk7/OpenJDK I have spec clarification in the
> works to fix the spec hole and as part of that work the test will be
> replaced (see 6621689).
Yes, fixing the intermittent failures and removing the invalid test in
OpenJDK 6 sounds good.
>> Second, it would be helpful is this host information could be
>> configured without changing the sources of the test each time. I've
>> spoken briefly to Jon about ways this kind of information can be
>> passed into a jtreg run. Environment variables can be used, as can
>> system properties. A system property could also be used to specify a
>> file from which the information was read. With a bit of jtreg
>> hacking, it is feasible portions of the jtharness interview procedure
>> could be exposed to configure information that way too.
> I would be concerned with a solution that requires environment
> variables or jtreg requiring configuration from somewhere other than
> the test suite. At least in Sun (sorry for being selfish), these tests
> are run by various teams on hundreds of machines. I don't think we'd
> want to require the javatest interview be run or have .jti files
> stored in various places with configuration information that will
> quickly go stale. If you mean that the configuration can be overridden
> via some input to jtreg that might be okay but would require the
> distributions and others to maintain that configuration. Anyway, I'm
> interested to hear the outcome of the discussion.
Even if the Sun-internal settings remain baked in, I think an easy way
to pass in a limited number of configuration settings is appropriate
and shouldn't be too hard to accommodate in the test code. Presumably
an interface like "openjtreg -Dmy.config.options=foo ..." wouldn't be
too awkward to use.
More information about the net-dev