There will always be releases that you do not want to release to the public python package index. Internal company software; just-for-your-own-use software; trying-stuff-out software. What to do with that? Solution: a personal pypi is relatively easy to set up with sdistmaker, provided you use subversion.
What sdistmaker does:
A simple easy_install sdistmaker
is enough. This gives you two scripts:
make_sdist
, mainly for test purposes. Pass it an svn tag URL and a
destination dir and it will make a release.sdists_from_tags
is the main script. It searches an svn structure for
suitable tags and makes releases of them.For starters, just run sdists_from_tags
. It will create a var/private
directory and fill it with (as an example!) all zest.releaser releases.
Both scripts have a --help
option that show all available options and a
usage instruction.
Configuration is by means of a python file. Easiest way to get started is by printing sdistmaker’s own base defaults.py by doing:
$> sdists_from_tags --print-example-defaults
Save the output as a python file (suggestion: defaults.py). You can then
adapt it to your liking and use it with sdists_from_tags
--defaults-file=defaults.py
. The defaults file is documented in-line, so it
should be easy to adapt.
Of course, in the end you want to run sdists_from_tags automatically (Unix cron job, svn post-commit hook, etc.)
You can use sdistmaker in a buildout like this:
[buildout]
parts = sdists
[sdists]
recipe = zc.recipe.egg
eggs = sdistmaker
scripts = sdists_from_tags
# arguments =
# defaults_file='${buildout:directory}/defaults.py',
The defaults.py
is created in the same way as above.
A structure like generated with sdistmaker is a perfect index for easy_install and buildout if you let Apache host it. Only problem: you can only have one index (note: pip apparently supports multiple indexes). You can solve this problem by having Apache redirect you to pypi when something is not found.
Here’s an example Apache config snippet:
# Allow indexing
Options +Indexes
IndexOptions FancyIndexing VersionSort
# Start of rewriterules to use our own var/private/* packages
# when available and to redirect to pypi if not.
RewriteEngine On
# Use our robots.txt:
RewriteRule ^/robots.txt - [L]
# Use our apache's icons:
RewriteRule ^/icons/.* - [L]
# We want OUR index. Specified in a weird way as apache
# searches in a weird way for index.htm index.html index.php etc.
RewriteRule ^/index\..* - [L]
# Use our var/private/PROJECTNAME if available,
# redirect to pypi otherwise:
RewriteCond /path/on/server/var/private/$1 !-f
RewriteCond /path/on/server/var/private/$1 !-d
RewriteRule ^/([^/]+)/?$ http://pypi.python.org/pypi/$1/ [L]
# Use our var/private/PROJECTNAME/project-0.1.tar.gz if available,
# redirect to pypi otherwise:
RewriteCond /path/on/server/var/private/$1 !-d
RewriteRule ^/([^/]+)/([^/]+)$ http://pypi.python.org/pypi/$1/$2 [L]
(Correction 2010-03-05 after tip from Marc Rijken: the last two
rewriterules had [P,L]
instead of [L]
. I copied an older version
instead of the actually-works config).
You can use such a custom apache-served index in two ways. Easy_install has a
-i
option for passing along an index:
$> easy_install -i http://packages.my.server/ zest.releaser
In buildout, you can set it like this:
[buildout]
index = http://packages.my.server/
parts =
...
This post is number four of my releasing python software series.
The next post I’ll describe buildout: controlling which versions of those released packages you use. Repeatable and isolated environment. Handy extra automation tasks.
My name is Reinout van Rees and I work a lot with Python (programming language) and Django (website framework). I live in The Netherlands and I'm happily married to Annie van Rees-Kooiman.
Most of my website content is in my weblog. You can keep up to date by subscribing to the automatic feeds (for instance with Google reader):