Djangorecipe: easy test coverage reports¶

My previous setup¶

I use nose as a test runner. Easier test discovery was the reason at the time. Python’s unittest2 is better at that, so it might be time to re-visit my choice. Especially as I just read that nose isn’t really being maintained anymore.

A nice thing about nose is that you can have plugins. coverage.py is a standard plugin. And with django-nose you can easily use nose as a test runner instead of the standard django one. So once I put a few nose-related settings in my setup.cfg, coverage was run automatically every time I ran my tests. Including a quick coverage report.

The setup.cfg:

[nosetests]
cover-package = my_program
with-coverage = 1
cover-erase = 1
cover-html = 1
cover-html-dir = htmlcov

Running it:

$ bin/test
......................................
Name                       Stmts   Miss  Cover   Missing
--------------------------------------------------------
djangorecipe                   0      0   100%
djangorecipe.binscripts       42     16    62%   25, 37-57
djangorecipe.boilerplate       1      0   100%
djangorecipe.recipe          115      0   100%
--------------------------------------------------------
TOTAL                        158     16    90%
----------------------------------------------------------------------
Ran 38 tests in 1.308s

OK

An important point here is that laziness is key. Just running bin/test (or an equivalent command) should run your tests and print out a quick coverage summary.

Note: bin/test is the normal test script if you set up a project with buildout, but it is effectively the same as python manage.py test.

New situation¶

“New” is relative here. Starting in django 1.7, you cannot use a custom test runner (like django-nose) anymore to automatically run your tests with coverage enabled. The new app initialization mechanism already loads your models.py, for instance, before the test runner gets called. So your models.py shows up as largely untested.

There’s a bug report for this for django-nose.

There are basically two solutions. Either you run coverage separately:

$ coverage run python manage.py test
$ coverage report
$ coverage html_report

Ok. Right. You can guess what the prototypical programmer will do instead:

$ python manage.py test

That’s easier. But you don’t get coverage data.

The second alternative is to use the coverage API and modify your manage.py as shown in one of the answers. This is what I now build into buildout’s djangorecipe (version 2.2.1).

bin/test now starts coverage recording before django gets called. It also prints out a report and export xml results (for recording test results in Jenkins, for instance) and html results. The only thing you need to do is to add coverage = true to your buildout config.

The details plus setup examples are in the 2.2.1 documentation.

(An advantage: I can now more easily move from nose to another test runner, if needed).

Not using buildout?¶

Most people use pip. Those using buildout for django will, I think, really appreciate this new functionality.

Using pip? Well, the basic idea still stands. You can use the call-coverage-API-in-manage.py approach just fine in your manage.py manually. Make it easy for yourself and your colleagues to get automatic coverage summaries!