Software releases 5: isolation and repeatability with buildout¶

Isolated environment¶

Create a directory somewhere and download the so-called “bootstrap file”. I’m showing instructions for a Linux prompt, but it works just as well on windows (but you’ll have to make the translation to windows commands yourself):

$> mkdir testsite
$> cd testsite
$> wget http://python-distribute.org/bootstrap.py

Now you’ll need an almost-empty buildout to start with. Open up a text editor and save the following as buildout.cfg

[buildout]
parts =

Run the bootstrap file. This only has to be done once. It makes sure the basics (setuptools/distribute and buildout itself) are installed so that buildout can actually run. “Bootstrapping”, you say? Yes. Bootstrapping. The notorious Baron von Münchhausen once was stuck in a swamp and pulled himself out of it by pulling at the straps at the back of his boots. That’s where the name comes from.

After running the bootstrap you’ll have a bin/ directory with buildout or buildout.exe (on windows) inside it. Run bin/buildout and you’ve just run your first buildout (which didn’t do a bloody thing).

A small better example buildout.cfg:

[buildout]
parts = scripts

[scripts]
recipe = zc.recipe.egg
# This recipe installs python packages.
eggs =
    pep8

Run bin/buildout again:

$ bin/buildout
Installing scripts.
Generated script '/some/where/bin/pep8'.

Now we get to see the isolation. Look at the contents of that bin/pep8 file:

#!/usr/bin/python

import sys
sys.path[0:0] = [
  '/home/reinout/.buildout/eggs/pep8-0.5.0-py2.6.egg',
  '/home/reinout/.buildout/eggs/distribute-0.6.10-py2.6.egg',
  ]

import pep8

if __name__ == '__main__':
    pep8._main()

Most of the content in there is just the basic stuff that easy_install would place there, too. Apart from the sys.path[0:0] line. The sys.path is the path where python looks when importing a module (like pep8). And sys.path[0:0] places the two items you assign to it (the location of the pep8 and the distribute egg) right in front of everything else on that import path. So an “import pep8” is guaranteed to find the pep8 package that buildout installed for you first.

Version management¶

When you set up a website with buildout (grabbing django and a couple of add-ons, for instance) and test it all locally (works fine) and later run the same buildout on the server, you do not want it to crash because there’s a newer version of one of the add-ons that isn’t compatible with the rest. You want reliability.

The way to get that working is to add a few bits to the buildout:

[buildout]
parts = scripts
# Add the following two lines:
extensions = buildout.dumppickedversions
versions=versions

[versions]
# Nothing here yet

[scripts]
recipe = zc.recipe.egg
eggs =
    pep8

Re-run buildout:

$ bin/buildout
Updating scripts.
*************** PICKED VERSIONS ****************
[versions]
distribute = 0.6.10
pep8 = 0.5.0
zc.buildout = 1.4.3
zc.recipe.egg = 1.2.2

*************** /PICKED VERSIONS ***************

You now get a report on the versions buildout picked for you. You now copy-paste those versions into your [version] part to pin them. Buildout will use those exact versions from now on:

[buildout]
parts = scripts
extensions = buildout.dumppickedversions
versions=versions

[versions]
distribute = 0.6.10
pep8 = 0.5.0
zc.buildout = 1.4.3
zc.recipe.egg = 1.2.2

[scripts]
recipe = zc.recipe.egg
eggs =
    pep8

Versioning your buildout¶

When I deploy a buildout, I want the same rebuildability in my buildout as I want in my packaged python code. So I want to tag them with zest.releaser.

Normally, zest.releaser works with the version info from the setup.py, but buildouts often don’t have such a file. Luckily, zest.releaser also looks for a version.txt file. So put 0.1dev all by itself in a version.txt right next to the buildout and you can use zest.releaser.

Deploying a website is so much more robust when the entire deployment is simply an (svn-)tagged buildout directory with pinned package versions in it. No surprises.

Easy distribution to colleagues and others¶

You won’t believe the speed with which colleagues get used to buildouts. Once they’ve been pointed to an svn directory and got the instructions “just check that out, run python bootstrap.py and bin/buildout”… They’ll never want to get back to the 4-page instructions on how to collect all the bits and pieces by hand.

Recipes¶

Recipes are buildout’s way of extending itself. There are a lot of them. Setting up cron jobs, downloading from svn/hg/git/bzr, setting up django, etc. Browse through the huge number of recipes on pypi for a while.

default.cfg¶

One handy time saver on your development machine: create a .buildout directory in your home dir and put eggs, downloads and configs directories underneath that.

Then add a default.cfg in that .buildout directory:

[buildout]
eggs-directory = /home/reinout/.buildout/eggs
download-cache = /home/reinout/.buildout/downloads
extends-cache = /home/reinout/.buildout/configs

This tells buildout (ALL your buildouts) to store the eggs, downloads (and configs, for those extends=http://some.zope.server/version12.cfg lines) in those directories. Download once, use everywhere. Saves you a lot of time when you have lots of similar projects.