jtreg testing integrated
Joseph D. Darcy
Joe.Darcy at Sun.COM
Wed May 21 16:04:41 PDT 2008
Andrew John Hughes wrote:
> On 21/05/2008, Joseph D. Darcy <Joe.Darcy at sun.com> wrote:
>> Hello Mark.
>> Mark Wielaard wrote:
>>> Hi Joe,
>>> On Mon, 2008-05-19 at 09:59 -0700, Joseph D. Darcy wrote:
>>>> Mark Wielaard wrote:
>>>>> On Mon, 2008-05-19 at 11:21 +0200, Mark Wielaard wrote:
>> Those may take a little while to figure out a good fix in the sense of
>> still testing the networking code outside Sun's firewall as opposed to just
>> adding "if(!insideSun()) pass" logic. I'll talk to the networking team
>> about this.
> Well I don't believe anyone was suggesting such a simplistic hack. We
> need a well-known external host with good uptime that would handle
> such requests without breaking a sweat.
Or at least information about what kinds of services are expected so
they could be replicated elsewhere.
>> On a related note, there are times where it could be useful for a test to
>> have somewhat different behavior on Sun's production JDK versus OpenJDK.
>> For example, I'm the author of the
>> java/lang/reflect/Generics/Probe.java test which fails on
>> OpenJDK since it references some production-only classes. There are a few
>> options to fix this. One way would be to remove all reference to the
>> production-only classes, which would reduce the usefulness of the test for
>> the production code; conversely, the whole test could be moved to closed,
>> reducing the test coverage of OpenJDK. The test could be split into open
>> and closed parts, complicating maintenance. So instead I changed the test
>> to only look at the production classes for a production build. I test for a
>> production build using
>> Out of the box, our OpenJDK builds have "java.runtime.name" start with
>> "OpenJDK"; is that true in your IcedTea/OpenJDK builds too?
> I assume what you mean by a 'production build' is the code being used
> to build a proprietary JDK with additional non-Free classes. If this
Yes, by "production build" I mean a build of Sun's product binaries with
the non-Free, non-open bits.
> is the case, I would hope that, in the long term, these will no longer
> be needed and the 'production build' will be just a build from the
> GPLed codebase like any other.
That would certainly make the engineering easier :-)
> I mention this because I think the
> term could be a little disparaging to outsiders, as it implies only a
> build with proprietary Sun code is considered production ready.
Yes, getting accurate, descriptive, and neutral (and concise) wording
can be tricky, as "proprietary" is also disparaging in some contexts ;-)
> I wouldn't add any code to rely on the name of a build from the GPLed
> code. It's just asking for trouble further down the line. Besides,
> by including such tests, you're effectively wrongly favouring some
> GPLed distributions over others.
Well, the tests in question come with the code, both under GPL, so if
someone wanted to have java.runtime.name=Fred for some reason, the tests
in question could be updated to expect "Fred" instead.
> Is a test for the production system (with an appropriate comment so we
> know that's what it is) not sufficient?
That is probably sufficient for the cases in question.
There are really three cases of possible interest: Sun's traditional
(proprietary) product bits, a "usual" OpenJDK build, and something else.
A "usual" OpenJDK build perhaps is something yet to come about; I'm
thinking there may be conventions we all choose to follow as good
practices that may affect how the tests are written in some way. A
"something else" build would, say, be based on the OpenJDK code, but
choose to go against usual practice in some manner.
The cases currently under consideration just need to make a decision
based on "Sun's traditional (proprietary) product" versus "not Sun's
traditional (proprietary) product."
>> and production JDK) fail else note/warning" logic so that our production
>> engine, it would get testing too.
> Wouldn't testing the result of
> ScriptEngineManager.getEngineFactories() suffice? Why do you need to
> specifically test for a production build?
engine; this is *not* required by the specification. The OpenJDK
happens to be present, it should be tested the same way, but not having
the engine shouldn't be fatal. This would preserve the same degree of
testing of the product bits, allow the tests to pass out of the box for
an OpenJDK build, and perform the same tests for an OpenJDK build if an
engine happens to be present.
More information about the distro-pkg-dev