Django under the hood: Channels - Andrew Godwin¶

The problem: the web is changing¶

A lot of the web is becoming async. Web sockets and so. But it is not only websockets: long-polling, webrtc, MQTT, server-sent events. This doesn’t match django’s regular webpage behaviour very well.

• Python is synchronous. Only the latest python 3 releases have async build-in.

• But even then, Django is still synchronous at the core.

Synchronous code is easier to write. Async is much harder. Only do it if you really have to. Synchronous code is much easier to reason about. Single-process async is not good enough, so you’ll have multiple processes, threads and perhaps even machines.

You’ll need a proven design pattern that isn’t too hard to reason about. This is no place to do something weird and new. Multiple people must be able to maintain it. And if you use it, you’ll need many people who are able to program for it.

Loose coupling¶

Loose coupling is a good thing. The solution should not be too tied to websockets: there is more. And it should also not be too tied to Django: the ecosystem can be bigger that way.

You’ll need well-defined, minimal interfaces. It should be easy to swap parts.

A solution could be a message bus. A single place where you point all the pieces to. Django can talk to it, an HTTP server can talk to it. A socket server can talk to it. There are trade-offs and it is not perfect, but it does work well and is well-researched.

Now, how do you talk to it? How do you send messages through it? That is defined in a new standard called ASGI. The five things you can do:

• Nonblocking send.

• Blocking recieve.

• Add to group.

• Discard from group.

• Send to group.

Basically, you send JSON-compatible dictionary-based messages onto named channels.

Concrete ideas¶

A web socket has “connect”, “accept/reject”, “receive”/”send” and “disconnect”. That’s the abstract concept of the websocket protocol. How do you map that to channels?

Everything is translated into a message that is send to a channel (on the message bus). So an “connect” event is send as a meessage to the “websocket.connect” channel. And so on. You have a websocket server that accepts websockets on one end and sends out messages to the message bus on the other end.

Any (django) worker on the bus can take the message and handle it. When sending it back, it has to be routed back to the specific socket server, as that one has the connection to the client. There is support for it (“reply channel” with a receiver indicator included after the channel name).

The system is not perfect: there are trade-offs, each with their own drawbacks. If something goes wrong, what do you do, for instance? Do you use at-most-once delivery? Then the message might be dropped. If you guarantee delivery, it might be delivered multiple times. Which one do you want? You’ll have to make those trade-offs deliberately. Some might sound fine, but they might cost you a lot of speed.

What has been done already: HTTP and websockets. Rough drafts for IRC, email and slack. Please don’t ever do this: minecraft and mainframe terminals.

A hard thing is ordering. Messages come in on the channel: what if they’re dependent on the order of them coming in? You can configure a specific channel to maintain ordering. An order key will be added on received messages and there’ll be explicit connection accceptance: this costs you some speed, though.

There are five packages that make up django channels.

• Channels. The django integration.

• asgi-redis. Redis backend.

• asgi-ipc. local memory backend.

• asciref. Shared code and libs.

• Daphne. http/websocket server. (“I took Twisted and Autobahn and hot-glued it together”). You can use Daphne only for websockets and use gunicorn or another wsgi runner for the regular http requests.

Workers in channels: consumers based on views¶

The consumers of messages look very much like django views:

@channel_session
def chat_receive(message):
    ......
    example_django_object = Message.objects.create(
        name=...
        content=message["text"]
    )

Routing is based on URLs:

routing = [
    route("websocket.recieve",
           consumers.chat_receive,
           path=r"&/chat/socket/$"),
    include....
]

How do you maintain state? You could use the database backend. You can also use sessions. In django channels, sessions hang off reply channels, not of cookies. It uses the same sessions backends. It can also access long-term cookie sessions. So you can set a username into the session and then use that throughout the entire websocket session!

A controversial thing: there is no middleware. Because of the ways consumers are written, it is virtually impossible to capture incoming and outgoing messages like regular middleware does. Decorators replace most of the functionality you’d normally need middleware for.

Important: the regular django views are still there! They can work via channels, too, if needed.

What is the future¶

He’ll hope it will allow generalised async communication. There’s also the binary MQTT “internet of things” sensor data. It is not only html. Do you want micro services? Separate things per CPU? Send certain stuff to a specific server? Synchronous python 3 stuff? Perhaps even mix python 2 and 3…

Part of this channels work is still young, especially by django standards. He wants more implementations to make sure it works well. Daphne is OK, for instance, but Django is not in the business of writing web servers.

He hopes it will make django much faster.

He also explicitly wants some more co-maintainers. Also documentation, bug fixing, bug reporting.

Channels is now an official django project and close to a 1.0 release.

Photo explanation: Switch and signal wires being channeled in the correct direction in the cellar of an old signal post of the “ZLSM” touristic railway in the Netherlands.