Buildout development/production strategyΒΆ
Buildout is great for development and for deployment. How to combine the two? Jean-Paul Ladage and me did some brainstorming on this and we'll show you what we came up with. I'm trying it now on a project and I'm pretty happy with it. Some prerequisites and assumptions:
- Configuration is handled by buildout.
- It shouldn't get in the way of easy development.
- Deployment should be rigorous.
Production/preview are tags
Production and preview must be tagged buildouts. If preview is signed off on, it becomes the new production. In that case an "svn switch" to the relevant tag is done in the production buildout.
Variable bits
Production, preview and development differ in the port settings, zeo settings, debug mode, etcetera.
Production and preview do not differ in products, as a tagged buildout should progress from preview to production without changing. Production/preview and development do differ in products, naturally.
The following five buildout config files are needed to handle this:
- production.cfg with production port number, zeo setup, etc.
- preview.cfg with preview port number, zeo setup, etc.
- stable.cfg with the products and eggs and zcml used by both preview.cfg and production.cfg.
- unstable.cfg with the development products
- devel.cfg with both development port number (just 8080 normally), debug mode.
unstable.cfg and devel.cfg could be combined, but experience has shown that it is harder that way to spot what you have to move from development to production. The extra hassle just isn't worth the worry about the extra file.
This way, with some care, a single buildout directory can be used for all three cases. It makes it easier to coordinate the move from development to production.
Externals
Externals in the form of downloadable tarballs and eggs are handled like a charm by buildout. No problems there. Problem cases are:
- Products you develop yourself (svn trunk, mostly).
- Products without a proper release from an external svn.
Handling svn externals in a buildout that is used for both production, preview and development is a pain. Keeping the svn checkout information inside buildout (with the infrae.subversion recipe) is the solution here. You'll need two infrae.subversion sections in case you have both products and eggs under development.
Note that we're talking "products" here. Keep the lib/python stuff as externals in src/* as they don't interfere if you don't enable them. I couldn't get those to work with infrae.subversion (chicken/egg problem as you can't reference those eggs in [buildout] if you still have to download them from the same section).
Reinout van Rees
My name is Reinout van Rees and I program in Python, I live in the Netherlands, I cycle recumbent bikes and I have a model railway.
- Weblog
- Preken (NL)
- Over mij (NL)
- About me (EN)
- Ligfiets (NL)
- Eifelburgenbahn (model railway)
- Videos
- PhD (EN)
Weblog feeds
Most of my website content is in my weblog. You can keep up to date by subscribing to the automatic feeds (for instance with Google reader):
Good points, I've put them in my "fix up vanrees.org now that I've updated it to plone 3.0 and the new quills" project.
The date ought to be added to the template. Might be a local customization that I did where I forgot it.
The archive is waaaaaay too long as it is. I'm thinking about turning it into a google graph, btw. But the way you're asking: it seems you do like a date-based navigation (at least for the last months) in general?
What we ought to do is to use that debug.cfg for some common development items. that makes it way easier to copy it between various projects if there's a new handy debug tool making the rounds.
What I'd like: a way to make local modifications. Instancemanager (the tool we used before buildout) searched for an optional PROJECTNAME-local.py. But there doesn't seem to be a way to look for an optional config file in buildout yet.
Hope it is clear :-)
I enjoy reading your blogs very much, thanks for your effort. However, there are two suggestions I'd like to offer:
1) Would it be possible to add a date to your blog entries? This way, it would be possible for a reader to evaluate how up-to-date a blog entry is that turns up in a search result.
2) Maybe it's nicer for readers to see your Weblog Archive in the left column sorted from newest to oldest. That way we don't have to scroll so much to get an idea about your a-little-bit-less-than-very-recent-entries.
Happy New Year,
Roger
We have the following configs:
base.cfg -- the configuration common to all the configs. this also contains a plone.recipe.runscript (currently trunk as I had to add in zeoserver support) part that can create a plone site based on some arguments.
local.cfg -- the local development config with the debug products added in.
debug.cfg -- all the debug products, ipzope and zopepy combined into one part that can be added in the local.cfg
dev.cfg/maint.cfg/production.cfg -- the three stages of our release process. these configs mainly contain overrides to the port, ip-address and locations of Zope.
There was one trick that is needed to use the base.cfg the way I did. If you extend the base.cfg and you want to modify it's parts then normally you have to write them all over again. What I ended up doing (on philikon's suggestion) was to create a variable in the buildout part in the base.cfg named base-parts. Then define the parts of the base.cfg as: parts = ${buildout:base-parts}. This way you can add in the debugging part in the local config like this: parts = ${buildout:base-parts} ${debugging/parts}. You would also have to use the same trick for the products definition in the zope instance part.
I've turned this all into a paster template that we use internally (as it still has some SFU specific pieces). The last thing I would like to do is use bundleman to bump versions and tag everything.
I'm glad to see that we are heading down the same path. Maybe we should put some examples in a public repo for others to be inspired by and possibly add to the ZopeSkel project a complete buildout solution.
claytron
Just a quick question re. your svn switch comment (call me dumb :) - so do you tag the same svn location that contains the buildout files or do you have the preview/production buildout .cfgs in different svn locations?
Thanks,
Tim Knapp