How to deal with integration for a large-scale website. Zest software (8 persons) made the intranet website for the 6000-person research department of Philips . Some of the biggest challenges had to do with the performance of the sites, the desired reliability of the service and especially the project management.
The site has only logged-in users and a few thousand actually log in every day. This is different from most of the plone sites, where you have a large majority of anonymous users. The users are authenticated against a 200000 user LDAP database. Plone's ldap story at that time still needed some work and manual patching. We made a small product that took care of all ldap configuration and patching. Wiggy recently released a new plone ldap product that does basically the same thing, only better, nicer and with a handy UI :-)
Single-sign-on (SSO) is handled by apache (set up by Philips itself with mod_nltm). Apache passes along a variable in the request with the username in it. We created apachepas that grabs the variable and tells PAS that it is that user.
The homepage was made with compositepack, which we cleaned up somewhat for ploen 2.5. Now the customer himself can modify the homepage layout.
Important was location dependency. Philips research has 7 lab locations (Eindhoven, Aachen, etc.). Content can be restricted to one or more of those locations so that users are not overwhelmed with unneeded information that is not relevant to their work location. Daniel Nouri made pluggablecatalog for us, which took care of transparently filtering all catalog query results. In our case, the filtering was done on the present user's location setting. This meant we didn't have to customize a lot of plone templates, everything was handled transparantly by the catalog.
Performance. Performance. Performance. Caching. When switching on the new website, we had to switch it right back to the old one within 30 minutes. It just wasn't able to cope. One of the things we improved was dropping some personalization, for instance by not showing the current user's name on the homepage. This way, you can vary the page based on the roles of the user instead of individually logged in users. This helped a lot compared to the default cachefu rules. (Also we added more zeo clients, did generic cachefu work, figured out an error in the apache single-sign-on setup, provided more detailed caching of page elements, etc.)
Feedfeeder was created to read in atom feeds from an exisiting news server and to create them as news items in the plone site. For several usecases, we needed extra data in addition to the title, description and text: start date, meeting location. For that, we added the extra information in a special microformat inside the feed items. Feedfeeder is able to extract that and handle it because of some zope3 adapter work.
Other extra contenttypes (all in the collective svn) included "link of the day" and "quote of the day". Also handy for more generic usage: azlinks, a simple a..z listing of links.
A special problem was the migration of the existing 50 small intranet websites into the one new site. The key is to do this gradually. A so-called "migration folder" could be gradually filled with the content of the old website. The migration folder has a URL pointing at the old site and a checkbox "migrated yes/no". The migration folder is shown in the dropdown menu (the dropdown menu is the main way of navigating the site). As long it is not fully migrated, the menu item links to the old website; when the checkbox is set to "yes", the menu item links to the migration folder instead. Works like a charm.
Project management was important for such a big project. We're using extreme programming for it, chopping up the project in 2-3 week iterations. At the end of the iteration we would go to Eindhoven for a day and close the iteration and write the stories and tasks for the next iteration. This way, you get regular feedback and the customer can change the direction of the project at regular intervals. Internally, standup meetings are essential to coordinate the work and to make sure that nobody is stuck and that everybody has all the info he needs.
Very cool: we were allowed to release much of our work as generic open source packages (see some of the links to products in the text above). That's something that's to be recommended for all integrators. We got bug reports from other users that improved the products, for instance. Releasing such products also improves the plone ecosystem as a whole.
My name is Reinout van Rees and I work a lot with Python (programming language) and Django (website framework). I live in The Netherlands and I'm happily married to Annie van Rees-Kooiman.
Most of my website content is in my weblog. You can keep up to date by subscribing to the automatic feeds (for instance with Google reader):
