Opened at 2020-08-07T12:38:23Z
Closed at 2020-08-14T15:55:35Z
#3372 closed defect (fixed)
Ratchet based on test module, not individual test
| Reported by: | chadwhitacre | Owned by: | GitHub <noreply@…> |
|---|---|---|---|
| Priority: | normal | Milestone: | Support Python 3 |
| Component: | dev-infrastructure | Version: | n/a |
| Keywords: | Cc: | ||
| Launchpad Bug: |
Description
The unit of work for the Python 3 port is the module. Currently we ratchet up successful tests at the individual test level, which means that randomly successful tests in unrelated modules that we don't care about yet distract us from the module currently in focus. At the very least, they are noise during dev and code review. If they start failing again, the developer either has to manually remove them from the "binary-ish" ratchet-passing, which is an annoyance, or fix them, which is a rabbit hole.
In #3369 there is a suggestion to not fail ratchet in CI if unexpected successes are discovered. But, since devs are expected to add expected successes to ratchet before pushing, this still places a burden on devs to then manually edit ratchet-passing and distinguish expected and unexpected successes.
Acceptance
- tox -e py36 only runs the tests explicitly listed in allmydata.utils._python3.PORTED_TEST_MODULES.
- There's no automated "ratcheting" per se. CI semantics are the same as for any other env, just limited in scope to the specified set of test modules, which is manually ratcheted (as it were).
Change History (4)
comment:1 Changed at 2020-08-07T15:13:01Z by GitHub <noreply@…>
- Owner set to GitHub <noreply@…>
- Resolution set to fixed
- Status changed from new to closed
comment:2 Changed at 2020-08-11T11:53:08Z by chadwhitacre
I ended up going a slightly different route. tox -e py36 by itself will fail. I added shtuff to the Travis invocation to constrain the set of tests we exercise under py36 in CI. In local dev one may want to use tox to run tests that haven't been ported yet.
comment:3 Changed at 2020-08-13T13:01:00Z by exarkun
- Resolution fixed deleted
- Status changed from closed to reopened
Re-opening. Some unrelated PR marked this as fixed. :/ I can't find the ticket that the sorted-ported-modules PR was supposed to fix ...
comment:4 Changed at 2020-08-14T15:55:35Z by GitHub <noreply@…>
- Resolution set to fixed
- Status changed from reopened to closed
In bc78797/trunk:
In fc901e96/trunk: