Deliverance: lipstick on a pig (Matt Hamilton)¶

Tags: europython2009, europython, deliverance

Talk about dynamically re-skinning a legacy .NET portal with python. The look and feel wasn’t compelling. The usability wasn’t that good in certain spots. The content management was poor (as they bought the portal product of the commercial vendor and not, for lots of money, the content management product). And vendor lock-in: even small changes were very expensive.

The customer wanted a much more flexible and maintainable system. And as the current system was monolithic, they wanted to be able to choose the best programs for the various parts. Moodle, plone and some of the existing system. But they couldn’t actually touch the existing system for contractual reasons.

A cunning plan: put a new skin in front of the current website by using a transformation proxy. The core element: Deliverance as the proxy. Deliverance removes the need of skinning an application inside the application itself. Deliverance is basically “xslt for dummies” that transforms an incoming html page according to some rules.

Another component: WSGI (“web service gateway interface”) for chaining small modules together in a pipeline. There are a lot of existing components out there that you can reuse. And it is easy to make your own components to plug into the pipeline. The pipeline they’re using now, to give an example:

• Grab content from existing site.
• Clean up html. LXML is great. It is not only an xml library, it also cleans up broken html (like beautifulsoup). 70kb of old content with 120 validation errors is now 40kb with 27 errors (mostly just xhtml/html mismatches).
• Rewrite links. The common .asp commercial-softare hugely ridiculously long urls are now /news/1234. From 205 characters to just 41 in their case. lxml has a method that returns all links in a document; you can also pass it a method that rewrites all those links.
• Inject plone content programmatically.
• Apply theme via deliverance.

(Comment for myself: experiment more with WSGI, there’s a whole lot of things I want to experiment with). The performance is good. They do all this on-the-fly. It sure helps that the old system was dog-slow, so every improvement is noticeable.

They’ve got several pipelines (one for the old site, one for plone and one for moodle with more to come). And several pipeline plugins are shared between several pipelines: reuse!

Of course there were complications:

• One page with two content sources: how do you build the navigation?
• Search obviously is a problem with content spread all over the various source-sites. Sorl/xapian/google mini etc. is what they’re going to look at.
• .NET viewstate postback: asp.net uses a huge (couple of kilobytes) hidden state variable with a html form element wrapping the entire site. Horrible. Horrid. With some javascript that modifies all normal links from a standard GET into a POST that includes the hidden variable. Solved as most of the state variable data wasn’t needed in most cases, so it could be left out most of the time.

The complications were solvable and the site is about to go live.