improve deprecation-warning builder

[warner]

We have a buildbot builder that runs tests with PYTHONWARNINGS=default::DeprecationWarning (you can do the same at home by running tox -e deprecations). But unfortunately it's not currently parsing the output for the DeprecationWarning: lines, so to find out if there are any problems, you have to look at the logfile and scan for red stderr messages.

My preferred fix would be for the tox deprecations environment to run some shell scripts, to tee the logfile into a separate file, grep for the warnings, and dump the results in a new deprecations.log file. Then the buildbot could add that as a "logfile", and flunk the step unless it's empty. Unfortunately it seems that tox really wants to encourage cross-platform compatibility, by not running the commands in a shell: I tried trial allmydata | tee -a log.out and got a complaint from trial that | wasn't a valid test module to run.

We could change this to make test-deprecations, which might provoke a tox complaint (since it doesn't remember installing make), but might still work. Or we could write some python tooling that installs a command into the virtualenv's bin/ that does the same thing.

I kind of want tox to do this, because then it's easier to run at home. I think I want it to dump the list of deprecations at the end of the test run (unless there are a lot of them, then maybe dumping the count and the name of the file would be enough). It could also exit with rc!=0 if there were any warnings.

If we can't get tox to drive this, we could find a way to change the buildbot to grep stderr (and not just stdout) for the messages. I think it used to do this, but something changed in buildbot in the last few years (maybe related to "new-style" build steps, and the emulation code that provides support for old-style ones), and now the regexp is only being given stdout lines, not stderr.

[warner]

Done, in [14c513db8], by adding a (python) helper script in misc/build_helpers/run-deprecations.py that gets run by tox -e deprecations. The script counts DeprecationWarning lines in stderr and emits a count (to stdout) at the end, along with a copy of all the warnings. If there were any warnings, tox exits with an error.

Alas the subprocess module doesn't make it possible to display stdout *and* stderr interspersed, while still capturing stderr, so in this mode, you can't tell which test is provoking the warning. Run trial without tox (and with PYTHONWARNINGS=default::DeprecationWarning) to see this, maybe also with |tee out.log 2>&1.

The buildbot has been configured to look for the _trial_temp/stderr.log that this produces, and to scan for the warning count (which is added to the buildstep status).

I also added tox -e upcoming-deprecations, which installs the latest Twisted and Foolscap (from their Github repos). This should give us some advance warning about things we're doing that have become deprecated in the upcoming releases of those projects.