Python usergroup the Netherlands (PUN) meeting in Amsterdam¶

After the last europython the Dutch attendees agreed to re-start the regular PUN meetings to exchange ideas. This one was in Amsterdam in the beurs van Berlage (we were in the old archive room; nice building btw).

Jeroen Vloothuis - How Pareto uses buildout for deployment

Pareto has two divisions: one microsoft, the other one open source python/django/plone. They are a service company, so they have lots of smaller projects per year with smaller teams. It doesn't pay off to do custom deployment mechanisms for every site, they need a deployment mechanism that can be reused. Especially since they don't do hosting themselves.

Buildout is the tool of choice. It has a declaritive configuration system and dependency management. It can grab all dependencies, also specific versions. This way a programmer that is new on the project can get up to speed quickly without needing to download lots of files themselves. The versions section makes sure you can nail down specific versions of dependencies so you don't get surprised by newer versions.

Good point about buildout: it is extensible. You can write your own "recipe" which you can call from the buildout config with some options. Such a recipe can do virtually everything, like compiling and installing zope; downloading packages; installing a whole java server program (solr in their case). [Reinout: This extensibility means lots of people actually extend it. Which makes it popular and much-used. It is one of the reasons we at zest software switched from the home-grown (I started it) instancemanager plone-install-and-download-extensions program to buildout.]

For version control they use bazaar. It enables offline working, which you cannot do with subversion. You also don't have that many server dependencies. Just an sshd on the server is enough, you don't need any other programs on the server. Bazaar has good subversion integration and there's a mostly usable emacs client. A nice extra is that you can use ubuntu's launchpad service for your open source code. Pareto's choice is to put every project in its own repository.

I asked about svn externals, which you don't have in bazaar. A plone project, for instance, typically consists of various products. These separate subversion products are included in the buildout by svn:externals. Jeroen said they don't need externals as every buildout includes the various sources directly in the same checkout. They don't see the need to split that all up into seperate checkouts and so don't encounter that need in practice.

Python eggs are released to their own server, the directory is then served as a python egg index by apache. Some files are password protected by apache. Buildout works just fine with authentication if you add username/password in the server url. Great for private packages.

Great recipe: zc.sourcerelease which bundles all software in the buildout and spits out a tarball as end result. Installing such a tarball is easy. And you're not dependent on servers that are down or packages that have disappeared. They give the tarball to the hosting partner as they don't do their own hosting anymore. Add installation documentation and for good measure ask them to run the unittests.

Steve Alexander - How we develop Launchpad and Bazaar at Canonical

Launchpad does code hosting for bazaar projects, manages translations for open source projects etc. Check it out. They use it daily, of course, at Canonical (where launchpad originated). They try out lots of different approaches at programming in teams and see in practice which ones work and which ones don't.

A paper by Ian Clatworthy takes "Agile" and things that commonly happen on open source projects, combines it with lean thinking and results in something he called community agile. Recommended by Steve.

They looked at some agile principles and started to look at them from different angles. (a) Sitting together with a second person when coding works well. But knowledge and motivation is more important: if you can get two physically separate people with extra knowledge and extra motivation to work together with modern collaboration tools, you win. (b) Quality every release? Yes. But quality every submit is essential for collaborative work. Don't break the trunk as it holds up lots of people. So keep the trunk at release quality at all times. (c) Integrate good features it as quickly as possible. Contributions are important. Even though they may not have been forseen in your planning. Sprints are great for holding planning meetings, but if someone has a valuable contribution, integrate it anyway. (d) I didn't get the fourth point written down at the meeting, but you can read "community over in-house" in the paper.

Quality checks and small packets of work result in a smooth workload. Toyota's Lean thinking is about reducing waste. Every not-yet-integrated piece of code isn't usable yet, so it is waste. Make the units of work smaller and you've reduced waste. Smaller changes make for quicker reviews which tend to happen quicker and earlier. Removing code cleans up the codebase, so that reduces waste. Review code quickly, so that you're sure you have quality code and don't have code sitting around for a month as you cannot trust it fully. Deploy quickly, as undeployed code... You get the picture.

Cadance and velocity is important in "lean". Velocity is clear: how much progress do you make? Cadance is the frequency of the major events in your development project. If you have a regular cadance, you get used to it and it helps keep the project on speed and on track.

Features can only get integrated into the trunk only if they meet the quality review check and when they match with the roadmap. Another idea: have pre-commit hooks that enforce quality, for instance by running the unittests. Have continuous integration. Automated regression testing. Amplify learning. Community sprints. They have open peer reviews: everyone who has the right to see the software is really allowed to look at it and give feedback on it. That way you all learn.

Lightning talks

You tell me (www.youtellme.nl)is a product comparison company. Their talk is about the technologies they're using. They switched from php to python and django a while ago: they needed a good demo and build it in 1/3th of the time compared to previous php efforts. The background reason was that they needed to do lots of computations, naturally, to get a good comparison website. Lots of computations and non-web programming is not something that PHP is good in, so they already used python for that. They got a lot of benefit out of using sqlalchemy, a python sql wrapper.

Eric Casteleijn (thisfred) talks about Autoqueue. It gets tracks/artists from last.fm that are similar to the ones you play in your audio player, looks them up in your own music collection and queues them. It is smart enough not to play the same tune for a configurable time. It works pretty well in creating a consistent yet not predictable listening experience that's pretty nice. It doesn't work for every player, currently only Quod Libed and Rhythmbox. Itunes and pytone are in the works. You can also easily create more or less coherent mix CDs easily.

Remco Wendt (from maykin) talks about mock testing. If you do any sort of testing, being able to do mock testing is important: if you want to test real software and if you are bugged by networking issues, complex interactions etc. Mock objects mimick real objects or services in a predictable way. Michael Foord's Mock is what he uses. It doesn't use record/playback, which is quite different from what you normally do. Most other mocking libraries do use record/playback. Monkey patching is dangerous, but this mocking library allows to make changes to existing libraries easy, quickly and safely.