Repository? -- How many lines of development?
Joseph D. Darcy
joe.darcy at oracle.com
Tue Nov 29 00:50:32 UTC 2016
On 11/28/2016 3:30 PM, Martin Buchholz wrote:
> On Mon, Nov 28, 2016 at 2:07 PM, Joseph D. Darcy <joe.darcy at oracle.com
> <mailto:joe.darcy at oracle.com>> wrote:
> For the combined dev/master forest, the most recent integration
> tag will have the same stability guarantees we have today so "pull
> the most recent jdk-10+XYZ tag" to get a stable snapshot.
> But ... I want better than the stability guarantees we have today!
Over the course of JDK 9 we've made marked improvements to the
regression test stability, at least on metrics we track internally on
internal systems, so unfortunately am I not at liberty to share them.
Briefly, the tier 1 regression tests in promoted builds are very stable,
the tier 2 tests somewhat less stable, and so on.
> I want to obsolete those messages to quality-discuss with "new
> failures" because it should not even be possible for master to get
> into a state where regression tests have regressed (except for flakes
> or external dependencies).
I don't know if those particular failures are actually in the promoted
build or just an artifact of how that particular batch of tests was run.
IIRC, we did not see a corresponding set of failures before the integration.
> As an aside, for JDK 10 I'd also like to see promoted builds on a
> more frequent schedule than once a week.
> People do "continuous testing and integration" these days. Set up
> your integration pipeline so that it is always running. The pipeline
> automatically promotes changesets to master when all tests pass.
> Easy! Then every master changeset is equally stable to a "promoted
One of the internal systems mentioned above is a CI system that run
regression tests after a change is pushed. The approximate integration
model is then to promote known-good states of sources as vetted by the
If problems are fixed soon after they are identified, then a post-push
system gives good results with avoiding the need more complexity on the
front end to manage a series of in-flight patches.
More information about the jdk10-dev