create a test file

Choose a name for your test file. We'll use test_fname.py:

touch src/allmydata/test/test_fname.py

./bin/tahoe debug trial --rterrors allmydata.test.test_fname

Okay, so it didn't work, because there is no file by that name. Create such a file, with these contents:

from twisted.trial import unittest

class T(unittest.TestCase):
    def test_a(self):
        pass

Now run it!

code coverage

Now install Ned Batchelder's "​coverage" tool and run your with code coverage, like this:

./bin/tahoe @coverage run --branch --include='src/allmydata/*' @tahoe debug trial --rterrors allmydata.test.test_fname

If you installed coverage from Debian or Ubuntu then you have to name it python-coverage, like this:

./bin/tahoe @python-coverage run --branch --include='src/allmydata/*' @tahoe debug trial --rterrors allmydata.test.test_fname

This does the same as running the tests without coverage -- print a list of what happened when each test was run. It also writes out a file named .coverage into the current directory. Run the following command to read that file and produce nice HTML pages:

./bin/tahoe @coverage html

That will product a directory named htmlcov. View its contents with a web browser.

using code coverage results

This is important: we do not treat code coverage numbers as a litmus test (like "aim to have 90% of lines covered"). We hardly even treat it as a scalar measurement of goodness — 91% code coverage is not necessarily better than 90% code coverage. Maybe the alternative would have been to remove some (covered) lines of code that were not necessary, which would have resulted in a worse “code coverage” metric but a better codebase. Finally, note that even if you have 100% branch-level coverage of a codebase, that doesn't mean that your tests are exercising all possible ways that the codebase could be run! There could be data-dependent bugs, such as a divide-by-zero error, or a path which sets one variable to a setting which is inconsistent with a different variable. These sorts of bugs might not be getting exercised by the test code even though every line and every branch of the code is getting tested.

So what do we use it for? It is a lens through which to view your code and your test code. You should look at the code coverage results and think about what it says about your tests. Think about “what could go wrong” in this function — where bugs could be in this function or a future version of it — and whether the current tests would catch those bugs. Both authors of patches and reviewers of patches should look at the code coverage results, and see if they indicate important holes in the tests.

Code coverage displays turn out to be very handy for showing you facts about your tests and your code that you didn't know.

further reading

​http://twistedmatrix.com/documents/current/core/howto/testing.html