Yeah, with the previous four posts in this series you’re bound to have a couple of released python packages. And several external dependencies.
Where do you install them? System-wide just with
easy_install xyz? What
about conflicting versions? One project needs django 1.0, another 1.1. And
networkx 0.99 deprecated a method that was still there in 0.7, so suddenly
your website doesn’t work anymore. Oh, and those 264 packages in your system
python are no fun.
The solution for these version management and isolated environment problems is buildout.
Create a directory somewhere and download the so-called “bootstrap file”. I’m showing instructions for a Linux prompt, but it works just as well on windows (but you’ll have to make the translation to windows commands yourself):
$> mkdir testsite $> cd testsite $> wget http://python-distribute.org/bootstrap.py
Now you’ll need an almost-empty buildout to start with. Open up a text editor
and save the following as
[buildout] parts =
Run the bootstrap file. This only has to be done once. It makes sure the basics (setuptools/distribute and buildout itself) are installed so that buildout can actually run. “Bootstrapping”, you say? Yes. Bootstrapping. The notorious Baron von Münchhausen once was stuck in a swamp and pulled himself out of it by pulling at the straps at the back of his boots. That’s where the name comes from.
After running the bootstrap you’ll have a
bin/ directory with
buildout.exe (on windows) inside it. Run
bin/buildout and you’ve
just run your first buildout (which didn’t do a bloody thing).
A small better example buildout.cfg:
[buildout] parts = scripts [scripts] recipe = zc.recipe.egg # This recipe installs python packages. eggs = pep8
$ bin/buildout Installing scripts. Generated script '/some/where/bin/pep8'.
Now we get to see the isolation. Look at the contents of that
#!/usr/bin/python import sys sys.path[0:0] = [ '/home/reinout/.buildout/eggs/pep8-0.5.0-py2.6.egg', '/home/reinout/.buildout/eggs/distribute-0.6.10-py2.6.egg', ] import pep8 if __name__ == '__main__': pep8._main()
Most of the content in there is just the basic stuff that easy_install would
place there, too. Apart from the
sys.path[0:0] line. The sys.path is
the path where python looks when importing a module (like pep8). And
sys.path[0:0] places the two items you assign to it (the location of the
pep8 and the distribute egg) right in front of everything else on that import
path. So an “import pep8” is guaranteed to find the pep8 package that
buildout installed for you first.
When you set up a website with buildout (grabbing django and a couple of add-ons, for instance) and test it all locally (works fine) and later run the same buildout on the server, you do not want it to crash because there’s a newer version of one of the add-ons that isn’t compatible with the rest. You want reliability.
The way to get that working is to add a few bits to the buildout:
[buildout] parts = scripts # Add the following two lines: extensions = buildout.dumppickedversions versions=versions [versions] # Nothing here yet [scripts] recipe = zc.recipe.egg eggs = pep8
$ bin/buildout Updating scripts. *************** PICKED VERSIONS **************** [versions] distribute = 0.6.10 pep8 = 0.5.0 zc.buildout = 1.4.3 zc.recipe.egg = 1.2.2 *************** /PICKED VERSIONS ***************
You now get a report on the versions buildout picked for you. You now
copy-paste those versions into your
[version] part to pin them. Buildout
will use those exact versions from now on:
[buildout] parts = scripts extensions = buildout.dumppickedversions versions=versions [versions] distribute = 0.6.10 pep8 = 0.5.0 zc.buildout = 1.4.3 zc.recipe.egg = 1.2.2 [scripts] recipe = zc.recipe.egg eggs = pep8
Normally, zest.releaser works with the version info from the
buildouts often don’t have such a file. Luckily, zest.releaser also looks for
version.txt file. So put
0.1dev all by itself in a
right next to the buildout and you can use zest.releaser.
Deploying a website is so much more robust when the entire deployment is simply an (svn-)tagged buildout directory with pinned package versions in it. No surprises.
You won’t believe the speed with which colleagues get used to buildouts. Once
they’ve been pointed to an svn directory and got the instructions “just check
that out, run
python bootstrap.py and
bin/buildout”… They’ll never
want to get back to the 4-page instructions on how to collect all the bits and
pieces by hand.
Recipes are buildout’s way of extending itself. There are a lot of them. Setting up cron jobs, downloading from svn/hg/git/bzr, setting up django, etc. Browse through the huge number of recipes on pypi for a while.
One handy time saver on your development machine: create a
directory in your home dir and put
directories underneath that.
Then add a
default.cfg in that
[buildout] eggs-directory = /home/reinout/.buildout/eggs download-cache = /home/reinout/.buildout/downloads extends-cache = /home/reinout/.buildout/configs
This tells buildout (ALL your buildouts) to store the eggs, downloads (and
configs, for those
extends=http://some.zope.server/version12.cfg lines) in
those directories. Download once, use everywhere. Saves you a lot of time
when you have lots of similar projects.
My name is Reinout van Rees and I work a lot with Python (programming language) and Django (website framework). I live in The Netherlands and I'm happily married to Annie van Rees-Kooiman.
Most of my website content is in my weblog. You can keep up to date by subscribing to the automatic feeds (for instance with Google reader):