Taking up the gauntlet: defense of Django¶

Transitioning to modern times¶

We have a big stack of Django apps (called Lizard, 50+ apps) and some 40 sites. I have a quite good description of how it all fits together in a previous post where I told about a pyramid experiment.

Summary: we use all of Django (models, templates, views, staticfiles, etc). Reusable apps. Two big core apps that everything inherits from for basic user interface and basic map handling.

To quote Gijs: … we are currently transitioning to API-based de-coupling of back-end and front-end. In other words: a client-side user interface which talks to an API for data. This web app should just be one of the api-consuming clients, not privileged in any way compared to an iPhone or Android native client.

So: Django is relegated to the dustbin of history and is mercifully allowed to serve up an API (probably until it is replaced in due course with some node.js goodness). Yes, I’m being over-the-top melodramatic here :-)

Not too quick, though: the “we are currently transitioning” quote should be qualified as being limited, at the moment, to one single full-blown all-out experiment on a big project. All our other projects still use ye olde Lizard with the whole Django stack.

“We are transitioning” is right in that there seems to be management buy-in.

My internal doubt. I’m an obsolete heathen.¶

That dustbin-of-history stack of Django apps called Lizard? The basis was made by me and another colleague. Apps as nicely releaseable python packages? With repeatable buildouts? Internal python package server? Current user interface? Documentation? Test setup? There’s a really big piece of me and my thinkwork and my preferences in there.

This would mostly be obsoleted by a html+javascript client-side user interface? An interface that takes over most of the work and that simply talks to an API? Wow. That turns the whole structure upside down. The flow of control is reversed. No more UI controlled by (and structured by) Django apps.

Gijs calls this a paradigm shift. When someone has worked on traditional websites using Django, Rails, PHP, Plone or you name it, the paradigm shift to API based application design is incredibly hard to wrap your head around.

Is it a problem for me? Am I resisting change? I got some flak at a previous job for sticking too long to a particular tool (“archgenxml” for those who know it) when I should really have moved along. Since then I’m a bit anxious to not be too conservative.

According to Gijs, programmers can develop the technological conservatism syndrome as they get older. Age-wise I’m number 4 in a company of 50 at the ripe age of 40 :-) Is it potentially a career-ending syndrome? In any case I “should be ready to learn new things” or there’s a nice place for me right next to the dustbin where Django already is in.

I’m a heathen. I don’t unquestingly believe the new gospel. The new hmtl+javascript-only party line. I dared to answer the rethorical question “who doesn’t totally believe in html+javascript+api?” with a “no”. I do not believe in that party line in that manner. At least I do not believe in it with the down-with-django mentality attached to it.

My theoretical misgivings (positive for Django)¶

With html+javascript+API, you can totally make great webpages/sites/apps that totally rock. A twitter app, a kanban board on top of github. Google’s apps.

But there are sites/apps/pages that are best done in a different way. And some sites just need more server-side power and control. Here are some of the theoretical and practical notes I’d like to give to you for consideration:

• When I look at https://github.com and look at the http traffic, I see a different picture from html+javascript+api. Github has a great API and I had great fun with a little script that queries that API. But… those pages are Plain Old HTML. Look what comes over the wire! Responsive as hell, but even the line numbers have their own span or div! Totally generated server-side. With some great javascript-based partial page loads and history rewriting (probably pjax?) to make it even quicker.

  If javascript-only webpages where the Absolute Truth, wouldn’t github have gotten the message? As they apparently have some measure of success (wow) and are obviously ridiculously fast, perhaps there’s something good to plain old HTML rendering with some javascript sprinkled on top?

• Google doesn’t index empty html pages with javascript. To get into google’s index, you need html content.

  (Ok, most of our lizard sites couldn’t care less if they’re indexed or not. At least we’re not actively aiming and designing for that. The nature of the data doesn’t require it. But it would be fun to try and make google results meaningful for our sites.)

• According to Gijs, … this web app should just be one of the api-consuming clients, not privileged in any way compared to an iPhone or Android native client.

  Wait one minute. You give all the control and quite a lot of the functionality to the client side javascript code (per the new paradigm) so that native clients have a level playing field. This means you have to duplicate the whole effort another two times to get an iphone and android native client? For a twitter client, ok, but there’s a shocking amount of functionality buried in all those lizard Django apps. I do not want to have to code that three times!

  The only economically sane way forward for us as a company is to have a good responsive multi-device layout. But then there’s no theoretical need to shift lots of code to the client side (at least not because of the equal-footing-for-iphone reasons).

Django doesn’t look too bad¶

To be honest, I cannot see a good theoretical reason why an html+javascript+api interface is inherently better in each and every case.

I do want to use API’s and javascript a lot, I just don’t want the user interface to be king. That kind of control is better left in the hands of good old reliable Python code. At least for the kinds of elaborate sites we’re making.

I want Python code to manage the URLs (unless it is more practical in the browser for specific pages and workflows). I want the templates to be very dumb and the javascript code to be reasonably dumb and the Python code pretty smart (though I’d personally love some smart backbone.js+api to clean up several dirty custom javascript plus dirty python code corners).

Some extra points I’d like to raise:

• i18n is easier to manage in Python/Django than in javascript. And internationalization is getting more and more important for us as a company.

• We haven’t tested a single line of javascript in the three years that we’re now using Lizard. Our Python/Django test coverage is fair (bad in some places, very good in a few). So… what does that tell me about the expected test coverage of our code when we’ve moved a lot of the functionality to javascript?

• The goal of Lizard is to make lots of different kinds of data sources available and combinable in one system. One of the two cores of the current Lizard is an “adapter mechanism” that ties the generic map/search/click/graph handling to arbitrary data sources. The other core is a generic user interface that you can customize and tweak, for instance by subclassing. All those various data sources all need different tweaks and customizations. Lizard allows that.

  The core of the new html+javascript+api approach sounds to me like “I want to build a modern web interface with html+javascript+api”. There isn’t much “data source” or “customizations” or “quick modification in a subclass” in the core of that approach. It sounds like it is aimed at building one single individual great website. See the next point.

• Repeatability, rebuildability, customizability.

  I do not doubt the beauty of the end product of a html+javascript+api interface. I sincerely hope that the current experiment succeeds and that the big project gets a great interface. I do doubt the applicability to all our projects and sites. Despite all thoughts about one single website for all clients, I pretty much know for sure that, next to the central generic website, there’ll always be a bunch of separate websites for specific clients. Can we manage that with a html+javascript+api codebase? (I don’t know the current state-of-the-art of javascript packaging, perhaps it is doable).

• I started reading the new Django book two scoops of Django last week. Wow, that provided me with some welcome support: Django is pretty good and useful. Really.

Proposal/wishes¶

I’d propose continuing with Django. And structuring that mess of javascript we have now with backbone.js or angular.js or something like that. Which means bringing in APIs.

Oh, and I’d love to make our Python-internal adapter mechanism a bit more explicit and turn it partially into an API. Lizard is a bigger project than just our company. There are quite a lot of partners! (See http://lizard.net/ if you speak Dutch). An API would make it easier to connect their custom data sources with the rest of Lizard (compared to needing a bunch of Python code). Up to a certain level this could work real well. And if more functionality is needed, we can do the real Python integration anyway.

Another wishlist item: a more responsive layout. The previous version worked OK on the desktop. The current version also works on an iPad. The next version should also work comfortably on an iPhone.

All those wishes? There’s nothing in Django holding them back. And there’s a lot of Django you can use to make it possible.

Closing comment¶

I’ve now made myself completely impossible. And I’ve made a fool out of myself. Publicly. I’ll just slink away quietly now to the house of demented elderly programmers.

That is, unless somebody shoots me first for being too verbose and for loving flowery language too much and for making my sentences (and this whole article) too long :-)