HTTPConnectionPool briefly leaks reactor state in a way that makes resource cleanup more complicated

[exarkun]

twisted.web.client.HTTPConnectionPool, used with Agent, does not offer a way to wait until all of the connections it created have been closed.

It has a closedCachedConnections method which can be used to wait for all of its idle, cached connections (which I think you can already ensure will be an empty set if you set persistent=False). However, this will not wait for connections that are not idle.

The almost guaranteed series of events, then, is:

1. You make a request which opens a connection and does some network stuff
2. The response to the request comes in and your request Deferred fires
3. Your application code processes the response and when that is over, you believe you are done
4. Finally, the HTTP client implementation gets to clean itself up, either closing the connection or returning it to the pool.

After (3) you usually assume everything is done but there is still an open connection until (4). You'll most easily notice this in a test when you think the test can end after (3) and then trial complains about the open connection. Conceivably this could be an issue under real world use, though it seems less common that a single connection existing for slightly longer than expected causes issues there.