Djangocon.eu 2011 is going to be organized in… Amsterdam!
I opened the evening with a summary of last month’s djangocon.eu conference. I won’t write down a summary of the summaries that I already made of all the talks :-) Some main points:
Go to such a conference! Good experience. What to do? How to get there? What happens?
Django’s own development. The django technical panel, Jacob’s keynote, the bad pony talk, etc.
NoSQL. A big hit at the conference with 3 talks and 1 panel.
Some links I gave:
Idan Gazit’s design for developers talk. Read it or, better, watch the video. Highly recommended.
Subtitle: “an academic Jobboard with Django. Also featuring: Plone and SOLR”. Academictransfer is a spin-off from the VSNU, the combined Dutch universities. Academic transfer has a lot of academic jobs in the Netherlands and they’re expanding abroad. It is really focused on academic jobs.
Pareto, his company, uses Plone a lot. Plone is a really good content management system, but people sometimes abuse it for tasks for which it is not suited. Since a few years, they use Django for those kind of non-Plone-suited projects.
The old side was a 10-year old coldfusion site. A classic case of vendor lock-in of the company that build them the original site. Performance problems and big problems at every change request.
They originally asked Pareto for a Plone website (“it is a lot of content”). Plone is a good CMS, but this is mostly a database-driven site, so they advised Django.
They also use SOLR. SOLR is a high-end open source search engine (from the apache world). Till now, this has proven to be a good choice.
They use buildout (YEAH! See my buildout articles). It is their workhorse for setting up Django websites. With a buildout config, you can have a fresh, running django website within a minute. (Note: read Jacob Kaplan-Moss’s article about Django and buildout).
People are looking for a job, so the search functionality is the most important part of the site. SOLR works great. It is good at searching with facets. A facet can be an organisation or a keyword or an academic field for instance.
There’s login and personalization. Storing your queries; getting mails with new vacancies. Universities also can have their own page in the site. Most of the website is made with Django, but Plone is also used. An apache config divides up the requests between the both of them.
The UI is reused between Django and Plone. They did this by having Django ask plone for a base template. What plone returns is a plone browser view, but it is a Django template with Django’s tags! The base template is downloaded with a cron job every night as the Plone site doesn’t change that much. Another option would have been deliverance. Nice system.
A localization problem: Plone and Django use different methods for setting the language. Django stores it in the session and Plone uses a separate i18n cookie. The solution was to do the language switching on the Django side and get Django to set the Plone cookie, too. (An alternative could have been some django i18n middleware).
User management is Django-only. In the Plone pages, you want to show the username, too, though. The solution was to add an ajax call to the Plone interface that asks Django for the username :-)
The whole website is cached with varnish (which is basically the standard in the Plone world). For the username stuff, they had to use aggressive cache invalidation. An alternative they’re going to pursue is SSI (server side includes). A main reason is that such ajax injection goes against the webrichtlijnen (Dutch accessibility guidelines).
They want to show some Django content, like recent job entries, in the Plone homepage. The solution: load those entries via RSS/Atom feeds in Plone portlets. You have to make those feeds anyway.
Website statistics are of course important. The counters are stored in a Redis store. Redis is a very simple noSQL database that’s basically only a key/value store. Lightning fast. It uses http for connections. He had some doubts about that, but it turns out to be fast, too. Google analytics wasn’t an option as academic transfer had some custom requirements. Especially very custom “funnels”. Funnels are specific sequential steps that a customer can take through the site.
Some extra features:
Pluggable publication model so that you can submit to/from several university’s job sites.
Chinglish plugin: automatic English-Chinese translation. The quality isn’t perfect, but it suffices to get translated job vacancies into China’s main “baidu” search engine. The people that find it then can switch to English for the “real” job vacancy.
Several import/exports.
Question: when would you use Django, when Plone? Answer: for this site, they’d have gone with all-Django. Plone is only used for only 15 pages. But if your site needs real CMS functionality, use Plone. So if you need workflows, per-page and per-folder security, stuff like that. Don’t build anything like that yourself: it is already there in Plone.
When you update your site, there are a lot of manual steps you need to do. That’s not DRY (don’t repeat yourself). Some options:
Shell scripts (but you still need ssh).
Fabrik (pretty good, but not that friendly)
He proposes django-siteupdate. You can update your sites from the django admin. It keeps logs of your updates. Custom update scripts are possible, as is postprocessing. You see the output of your scripts in the django admin interface.
(I mentioned a djangocon talk about capistrano.)
Dynamic models are not defined in your python code. For instance, they’re created on the fly at runtime. You could create a flexible admin this way. Or you can dynamically prototype models without having python knowledge.
Or you can, before a syncdb, create dynamic copies of your old data models so you can still access them. Or, what he uses it for, generate additional models for localization.
How? Something like:
Person = type('Person', models.Model, attrs={..field...})
In the end, it is more involved of course. Most of django’s machinery still works just fine with those classes. You can run syncdb just fine from within your code, creating the tables in your database.
So: do you have a usecase for this? He said there was a nice article about this subject on the Django website.
Alper’s been programming Django for 5 years. Recently he’s working with google’s app engine.
Django is good for content-heavy websites and for complex websites that are composed of many apps.
App engine is made for scalability and availability.
The good things about app engine:
The database (nosql!) is great. SQL sucks. NoSQL doesn’t give you migrations, so you don’t have migration issues :-)
There’s a python script that deploys to google in no time. No configuration, only programming. It just works.
App engine comes with batteries included. Memcache and all sorts of other tools. Handy.
Google makes sure it is scalable and always available and safe.
The bad things:
No serious queries. No normalization. Partially bad, but it enables some of the good things.
There’s a 30 second deadline for everything. Something that takes to long doesn’t ever complete.
They are a bit focused on enterprisey stuff.
Some tips: use indexes, memcache and create json endpoints.
It is ideal for getting stuff up and running quickly. Instant gratification. He wouldn’t really tie his whole company to google at the moment.
They got way too much traffic to one of their sites (http://www.fashiolista.com/) as it got waaaay to popular.
A default approach is to cache pages in Django: the @cache_page
decorator.
NGINX and memcached are also a great combination.
Their problem: most of the content is user-specific and thus uncacheable.
The solution: combination of SSI (server side includes) and javascript. Memcached stores an anonymous page and the javascript knows how to turn it into a personalized page. SSI mixes in a bit of data with personalization instructions into the page. Javascript then applies those instructions. This SSI mixed-in data is just a very small Django request.
He showed result statistics that were pretty sweet.
A bigger tutorial and code will be up soon at http://mellowmorning.com
His goal was to find an object oriented way to create reusable views for his project. Django had almost the same goal for its “generic views”.
He used the __init__() method and django is starting to use the
__call__() method. The __call__() calls the actual methods that you
can override in subclasses. The url pattern is a problem. Thread-safety is
an issue, they’re still thinking about an optimal solution.
An advantage of his __init__ method is that the url pattern can stay the
same. There are some disadvantages with subclassing and returning self.
Search the django developer mailing list for “class based generic views in 1.3” to get the full discussion.
My name is Reinout van Rees and I work a lot with Python (programming language) and Django (website framework). I live in The Netherlands and I'm happily married to Annie van Rees-Kooiman.
Most of my website content is in my weblog. You can keep up to date by subscribing to the automatic feeds (for instance with Google reader):