Questions about pre-submit testing using GitHub Actions
volker.simonis at gmail.com
Fri Oct 9 15:47:50 UTC 2020
first of all thanks a lot for adding support for pre-submit testing
using GitHub Actions. I think this is a really useful feature!
Unfortunately, I couldn't find much documentation about it except some
emails on this list and the initial PR , so I still have a few
1. All the pre-submit test that have been run for me until now (e.g.
) have failed for "Windows x64 (hs/tier1 gc)" presumably because of
a generic "out of disc space" error:
Error: There is not enough space on the disk. :
This starts at "TEST: gc/shenandoah/compiler/TestClone.java#id5" after
which all subsequent tests fail because of insufficient disc space:
error: error while writing TestClone: There is not enough space on the
2020-10-09T12:03:09.6475592Z TEST RESULT: Error. Write to temp. JTR
file failed (old JTR intact):
Is this a known problem? Is somebody working on fixing it?
2. When looking at "Linux x64 (hs/tier1 gc)" Ican see that there are
44 test failures:
2020-10-09T11:40:16.8365547Z Test results: passed: 255; failed: 44
2020-10-09T11:40:22.0973305Z Test summary
TOTAL PASS FAIL ERROR
2020-10-09T11:40:22.0974844Z >> jtreg:test/hotspot/jtreg:tier1_gc
299 255 44 0 <<
But the test group has a green check mark and is flagged as passed in
the report: "Linux x64 (hs/tier1 gc) succeeded (3 annotations)". How
can this happen? Is this a known problem?
By the way, the 44 failing test all fail on Windows and MacOS as well
because of a crash in:
2020-10-09T12:03:06.0162304Z # Internal Error
assert(Thread::current()->is_VM_thread()) failed: Must be
It looks like the framework only recognizes jtReg "Errors" but not
"Failures" as errors.
3. Would it be possible to save the hs_err files along with the .jtr
files in the case of crashes? I think this would be extremely useful
for quickly diagnosing problems.
4. I don't exactly understand the "free tier" you mention in your PR
. From the GitHub actions side as described in their "Usage limits"
documentation  it seems we're on the safe side. But that
information somehow contradicts with the information about "GitHub
Free for user accounts"  which mentions that free user accounts
only have 2000 free GitHub Actions minutes per month. In addition
"About billing for GitHub Actions"  mentions that these free
minutes are subject to a multiplier which is one for Linux, two for
Windows and ten for macOS which effectively means that after running
200 minutes on macOS our personal free minutes for a month will
already be consumed. So to me it seems that GitHub Actions "Usage
limits" documentation  only sets an upper bound for job and
workflow execution time of Actions but this has to be interpreted in
the context of the totally available, free minutes for Actions. But
maybe I'm wrong?
If you know better :) could you please explain which of these limits
apply and how many automatic tests jobs they are approximately
Thank you and best regards,
More information about the skara-dev