#4128 closed defect (invalid)
CircleCI Debian environments fail to report test outcome on test failure
| Reported by: | hacklschorsch | Owned by: | hacklschorsch |
|---|---|---|---|
| Priority: | normal | Milestone: | undecided |
| Component: | dev-infrastructure | Version: | n/a |
| Keywords: | ci | Cc: | |
| Launchpad Bug: |
Description
The `ALLOWED_FAILURE` mechanism in .circleci/run-tests.sh is faulty:
When turned off (failure not allowed) it makes the CI job exit before the test results, stored in Subunit2 format, are converted to JUnitXML format for viewing in CircleCI.
Since outputting to Subunit2 robs us of text output in stdout, CircleCI doesn't show us useful output at all when tests fail.
Change History (4)
comment:1 Changed at 2024-11-19T22:11:29Z by hacklschorsch
- Owner set to hacklschorsch
comment:2 Changed at 2024-11-19T22:15:57Z by hacklschorsch
This issue currently affects the following test suites on CircleCI:
- debian-11
- c-locale
- another-locale
- integration
- ubuntu-20-04
- ubuntu-22-04
comment:3 Changed at 2024-11-20T20:39:59Z by hacklschorsch
- Resolution set to invalid
- Status changed from new to closed
Turns out because our faulty subunitreporter makes the test suite pass no matter what this is only optics:
Twisted.trial succeeds (congratulations :)), but then subunit2junitxml exits non-zero if it finds failed tests in the report and we are good again.
comment:4 Changed at 2024-12-13T13:00:38Z by hacklschorsch
- Keywords ci added
Note: See
TracTickets for help on using
tickets.
Fixed in https://github.com/tahoe-lafs/tahoe-lafs/pull/1391.