Hudson continuous integration: technical and social

Tags: python, django, nelenschuurmans

Summary: last month I set up Hudson as our continuous integration server. And I constantly show Hudson’s front page on a laptop in a visible location, which increased interest and usage a lot!

Before…

Somewhere in February, after working here for a month, I set up a buildbot. I’m all in favour of some practical project automation. Having some machine automatically testing your code every time you make a change is one of the essentials. It also forces you to make your builds repeatable without any manual steps (so: buildout!), which is a good thing, too.

Problem: it did test everything pretty fine, but almost nobody looked at it.

I saw Hudson a couple of times already in some blog posts, but last May’s djangocon.eu in Berlin made sure I wanted to replace buildbot with Hudson, see for instance Eric Holscher’s talk.

Social effects

In August I installed Hudson and everyone started loving it. What did I do?

  • I installed Hudson on our buildbot machine and configured a couple of projects. And switched off the buildbot and redirected all URLs to Hudson.

  • I showed it to a couple of colleagues.

  • I asked for an old laptop and got it placed in a visible location in our development room with a full screen Hudson front page.

Hudson monitor

What was the effect?

  • Highly visible. So for a few days, everyone who got into the room wanted to know what it was. Instant invitation for a demo! Most people now know that our software is automatically tested and that it catches errors from time to time.

  • Nice visualization and nice graphs for code coverage, code neatness and overall health. Nice visualization really means you want to look at it.

  • Highly visible if something goes wrong. You really really want to fix your broken test. As soon as something is red (especially when demoing Hudson to a visitor) they want to know who the culprit is :-)

  • I’ve got pep8/pyflakes reports in there, too. So you see which code is neat. So you finally get a bonus for paying attention to your pep8 coding style. The effect? Many packages suddenly got miraculously neater.

  • As you see the code coverage in a graph over time, the amount of tests also miraculously increased.

  • More trust in the software. I noticed it myself when adding automatic jslint javascript checking: important, as just a single missed semicolon can throw off internet explorer in our javascript-heavy web apps. And even one missing semicolon is a flogging offense in our Hudson setup: a Red Dot… So having jslint running all the time made me feel much safer.

  • The boss also looked at it and suggested a different picture than the boring green ball. So the “successful build” icon is now his photograph with a green halo glow :-) Talk about management buy-in…

Visibility? Just when I was about to make a photograph, our sysadmin shut down the machine. “What?” “Oh, I’m just attaching a bigger monitor”. Hurray! I had not even requested that and suddenly we have a nicer bigger more visible monitor.

Technical implementation

There are a couple of how-to-integrate-hudson-with-django-or-python tutorials on-line, just google for them. I’ll just make some quick comments here:

  • There’s not only python in there! A colleague added R projects in there.

  • You’ll need to get your tests to spit out xml output. We now run our tests with the “nose” test runner by using django-nose. Next to the setup.py I added a setup.cfg with the following content, this tells nose to spit out xml:

    [nosetests]
    with-xunit=1
    
  • Cobertura plug-in: code coverage measurement. This also needs xml output. With coverage.py, you just need to call coverage xml.

  • “Violations” plug-in: switch on ‘pylint’ checking for your code. I don’t actually use pylint: I run pep8 and pyflakes over the code. You do need to redirect both tools’ output to a text file and fix the output up a bit to match what the plug-in expects. Effectively: every line that contains [E is an error, [W is a warning. I copied someone’s perl command-line for this. Pep8 is warning level, pyflakes is error level:

    pep8 dir_with_python | perl -ple 's/: [WE](\\d+)/: [W$1]/' > pep8.txt
    pyflakes dir_with_python | perl -ple 's/:\\ /: [E] /' >> pep8.txt
    
  • I included the rhino command line version of jslint. An error here means the script returns with a non-zero exit code, which means a test failure for Hudson.

All the above items are run in two or three script steps in Hudson. In the end I grouped everything together a bit in python scripts, btw.

Hurray for Hudson!

 
vanrees.org logo

Reinout van Rees

My name is Reinout van Rees and I program in Python, I live in the Netherlands, I cycle recumbent bikes and I have a model railway.

Weblog feeds

Most of my website content is in my weblog. You can keep up to date by subscribing to the automatic feeds (for instance with Google reader):