Github test commit reports for teams¶

Tags: nelenschuurmans, python, django

I recently wrote githubinfo. It is a quick and handy script that queries the github API of one or more organizations and gives you a simple report on the amount of commits with tests in the last week.

I figured it could be useful to others, so I made it available on pypi, including reasonable documentation of course: https://pypi.python.org/pypi/githubinfo . You can pip install githubinfo (or put it in your buildout).

It is just a simple single command and the output looks like this:

$ testcommitinfo
loading project neerslagradar-site
loading project ...
...

We want more and better testing. For a quick and dirty quantity
indication ('more'), here are the commits that have the string
'test' in of of the commit's touched filenames.

Period: 8 days.
Github organizations that I queried: ddsc, lizardsystem, nens

Projects sorted by amount of commits with tests
-----------------------------------------------

lizard-neerslagradar: 11 (25%)
lizard-progress: 3 (9%)
radar: 1 (33%)
...

Committers sorted by amount of commits with tests
-------------------------------------------------

Reinout van Rees: 12 (11%)
Remco Gerlich: 3 (6%)
Arjan Verkerk: 2 (8%)
...

Goal¶

I wrote it because we wanted to improve our development process at Nelen & Schuurmans. We wanted more tests. So I wrote a script:

It queries the github API for one or more organizations (or personal accounts).
It queries the projects in there for commmits in the last week (configurable).
For every commit, it simply looks if there’s a filename in the commit with test in its full path. If so, the commit counts as a “test commit”.
For every project, it counts the number of commits and the number of test commits.
The same for every committer.

At the end, you get a list of projects and committers sorted by number of commits.

Risk: you get what you measure¶

The metric is incomplete and imprecise. The same people that start grabbing their torches and pitchforks when someone mentions “code coverage” will start grabbing them now. My answer: bugger off.

You identify colleagues that never ever bother to test. You get to educate them. Can I borrow that pitchfork, please?
You identify projects that have improved in quality.
You identify projects that were obviously troubled by a deadline and that might bite you later on if you have to use them yourself.
You identify colleagues that bring quality to your project if you work with them.

There are a lot of things you don’t measure. But someone who doesn’t bother with tests also isn’t going to bother adding a whiteline somewhere in a test file to get at least some test commit credited to his name :-)

Configurable¶

The defaults are all terribly Nelen & Schuurmans centric. So there’s a way to customize it. See the pypi page for the documentation on how to do it.

Try it out! My main goal it to get everyone more aware of testing and thereby hopefully to increase the amounts of tests.

Reinout van Rees

My name is Reinout van Rees and I program in Python, I live in the Netherlands, I cycle recumbent bikes and I have a model railway.

Weblog feeds

Most of my website content is in my weblog. You can keep up to date by subscribing to the automatic feeds (for instance with Google reader):