Zope, archetypes, squid, apache, pfhew!¶

Last week, I spend time on making this website more responsive. Included in this struggle was an website on my university machine, based on the same zope/plone/archetypes combination. Both were slow, the university one the most so. Why?

• vanrees.org is now hosted at bytemark, who provide affordable root-access linux hosting. So you've got an entire machine to yourself and you can install everything you want. You don't really notice it, but they're using "user mode linux", which means you've got a purely virtual machine, sharing the actual piece of silicon and iron with a couple of others. The limiting factor: memory. I've taken their cheapest offer which gives me 64 MB. Linux is comfortably at home there, but zope is a bit of a memory hog. When the spamfilter fires together with an incoming webserver request... Ouch.
• The university server hosts objects.bcxml.net, a collaborative site for creating sort of an ontology for the building&construction industry. A mightily active graduate student (Noor Hellemans) is filling it like her grade depends on it, exposing real quickly that it didn't scale.

I solved half object.bcxml.net's problem by downloading Danny Bloemendaal's ATReferenceBrowserWidget. Which doth alloweth me a couple 'o utterings on the open source ecosystem I nicked it from.

Some USian company makes zope, a web application server, and open-sources it. Why? It would allow zope's pie to grow a lot and with them having a a lot of key zope people on board... that could make for a handsome profit. (Said the investor, of all people, and he was right). Part two is a set of people making plone, making it more of an out-of-the-box good-looking thing. And making it a userfriendly free content management system with all the goodies you'd normally expect (for completeness I'll mention that most of the hard content management stuff is handled by zope's CMF). Part three? To make creating new content types (special documents, object libraries, etc) easier and quicker, archetypes was created.

Archetypes was just what the doctor ordered to allow me to quickly build abovementioned objects.bcxml.net. And some other things too. I'm reading the archetypes-mailinglist for 1.5 years or so and I'm amazed at the amount of work done. New stuff being created. Collaboratively. In good humor. Competently.

Keeping up with these developments takes some time. Reading the mailinglist takes time. But it pays back in gold-plated spades. I shudder at the thought of having to effectively do all this work myself instead of building upon the collective effort of those competent developers...

Back to vanrees.org. The solution there was to lower the load on zope by putting a squid cache in front of zope. Most of the unimportant requests (non-changing images and stylesheets) are served directly by the cache and don't wake up the zope webserver. This is a much more effective use of the computer resources. Also I've let it cache the more or less static webpages and also this weblog. The weblog is checked more often for updates than the rest of the site for obvious reasons.

I must say that I've kept apache as the front-end webserver: squid sits between apache and zope. I did that to save me some reconfiguration and also a change in logfile formats... It probably gives me a small performance hit, but it's not that critical. So there.