#2894 closed enhancement (fixed)
Speed up the test suite with concurrent test runners
| Reported by: | exarkun | Owned by: | |
|---|---|---|---|
| Priority: | normal | Milestone: | undecided |
| Component: | dev-infrastructure | Version: | 1.12.1 |
| Keywords: | Cc: | ||
| Launchpad Bug: |
Description
On travis-ci a test suite run seems to take about 18 minutes now. I don't think travis-ci provides more than a single core to each job but using trial's -j feature (=2) nevertheless appears to speed up the run to around 12 minutes. I suspect this is because the test suite spends a lot of time on I/O wait. Telling trial to run the suite spread across two processes therefore results in better CPU utilization and results in this speedup.
Change History (3)
comment:1 Changed at 2017-07-26T14:28:16Z by exarkun
- Keywords needs-review added
comment:2 Changed at 2017-08-07T18:30:05Z by exarkun
- Keywords needs-review removed
- Resolution set to fixed
- Status changed from new to closed
warner reviewed and merged this. fixed in https://github.com/tahoe-lafs/tahoe-lafs/commit/9429fb8fad312c1347341618c6805b1c82e48afe
comment:3 Changed at 2019-06-21T13:21:59Z by exarkun
Note this PR didn't actually enable parallel test runs on CI because coverage collection is incompatible with trial's parallel test run feature. The PR only made it possible for users to manually invoke this feature on their local runs.
I've filed an upstream ticket, https://twistedmatrix.com/trac/ticket/9663, for implementing the behavior necessary to enable this on CI. It would be nice to do so since it will probably make the whole test suite take roughly half as long (perhaps 20 minutes down to 10 for one full CI job).
https://github.com/tahoe-lafs/tahoe-lafs/pull/431