add docs/tests.rst documenting how to run tests and how to interpret the output

[davidsarah]

In ticket:145#comment:26, Zooko wrote about various ways to run the Tahoe-LAFS test suite, some of which work and some of which do not. He suggested adding a docs/tests.rst to document this.

tests.rst should also document cases in which the tests are known to fail (such as when we're not testing the right code), how to interpret skips, todos, and other sources of noise, and how to get Twisted log output from test cases.

[zooko]

See also Nathan's comments about "How To Run Tests" during a Nuts & Bolts meeting.

[warner]

I'm picky about making tests run quickly, so I've identified three ways to run them, with various levels of nuisance and performance:

• 1: python setup.py test [-s allmydata.test.test_FOO]
• 2: ./bin/tahoe debug trial [allmydata.test.test_FOO]
• 3: PYTHONPATH=support/lib/pythonX.Y/site-packages trial allmydata.test.test_FOO

The last is the fastest, but you have to have twisted/trial already installed (or add more to $PATH/$PYTHONPATH), tests which involve searching $PATH won't work (or add more to $PATH), and you have to know what minor version of python you're running (to get the support/ directory right).

2 is easier, and only slightly slower: the bin/tahoe wrapper performs a fork+exec to re-run the entry point with the right $PATH/$PYTHONPATH

1 is how the test automation does it, because it's not particularly tahoe-specific. It's even slower because it runs a second subprocess.call to invoke the second form. More importantly, it actually runs setup.py build first (because of the aliases in setup.cfg), which can spend several seconds looking at your support/ directory and deciding whether to install dependencies or not.

And if you use python setup.py test --coverage, then you get one subprocess.call to invoke bin/tahoe @coverage, a second one inside the bin/tahoe wrapper to run the real bin/tahoe, and third one in bin/tahoe to run coverage. But for large tests, that's usually small compared to the overhead induced by the coverage tracing function.

I'm hoping that our #2255/#2077 overhauls can remove these extra calls.