Django under the hood: HTTP in Django - Jakob Kaplan-Moss¶

Tags: django, djangocon

(One of my summaries of a talk at the 2015 django under the hood conference).

Jakob Kaplan-Moss talks about one of the oldest parts of Django: http. So: actually talking to the internet :-) Old old old code, so it needs explaining.

HTTP in Django means Django’s request/response cycle. Let’s start with a very simple view:

def my_view(request):
    return HttpResonse("it works!")

What happens behind the scenes? Actually a lot. Django is a framework. The difference between a framework and a library is that a framework calls your code. Instead of your program calling a library. So Django is in control.

Django works with WSGI. So you have some WSGI server that wants to run an application. The application is in your my_app.wsgi.py file. Which calls get_wsgi_application() in django.core.wsgi. Which does some django.setup() and an __init__() of a wsgi_object in the WSGIHandler/BaseHandler. In the end a wsgi_object goes back towards the WSGI server.

It is important that django.setup() is called very early. This sets up the ORM, for instance. So most of the setup has happened before your own code is called.

Jakob wants to focus a bit of time on an extension point that’s often missed: you can write your own WSGI app, including adding WSGI middleware. WSGI middleware wrapps the entire site in some behaviour. Much django middleware code could better be done as generic WSGI middleware.

Examples: wsgi-oauth2. And whitenoise for serving static files like javascript. It performs better that similar things in django as it doesn’t have so much overhead.

Ok, back to Django. After the __init__(), Django starts up the middleware. Warning: each of the middleware classes is instantiated only once. So watch out when storing things on self as it can lead to memory leaks.

Django middleware is often overused (just like regular WSGI middleware is often underused). A good example is django-secure. Some comments by Jakob on middleware:

• He invented django’s middleware, but he doesn’t like the name. Look at it as “request/response hooks”.

• WSGI middleware works well with other python frameworks. However, it is hard to interoperate with django-specific parts of your app (like the database).

• The django middleware has a simple API and is easy to use.

Back to our http request! This is the point at which the URL config is loaded. After the URLs are known, a resolve() figures out which view to use and which args and kwargs to pass on.

Django has a feature that’s almost never used: a request can have its own custom URL resolver. You could do things like multi-tenancy and internationalization, but Jakob couldn’t find any good examples in open source code.

Now we’ve got a view. Here it starts to get simple: Django just calls the view which returns a response :-) There’s only an exception for exceptions: those are used for error handling, 404s and 500s and so.

Now that we’ve got a response, first the middleware gets to have a go at modifying the response.

Note that you can have “lazy responses” that are only rendered very late in the process. You could use this for complex composed views. Basically you don’t return rendered content, but instead a renderer and a context. You can have lazy template responses, for instance. This allows you to modify the context in middleware instead of having to work on the rendered content.

Gotcha on the “response finished” signal: it isn’t reliably send. It depends on your WSGI runner whether a close() method is called… So watch out with it.

Django’s request/response cycle is pretty elegant and you can extend it in lots of places.

But…. big changes might come to it. Django channels might turn everything a bit on its head. Conceptually, you’ll still have “the internet” and “your view” and “django in the middle”. It’ll all be wired together differently, though. It will also support things like websockets. (More explanation about django channels will be in a talk tomorrow).

Image: Cycling on the former German “Ahrtalbahn” railway this summer. A big retaining wall!