Djangocon: taking channels async - Andrew Godwin¶

Tags: djangocon, django

(One of my summaries of a talk at the 2018 european djangocon.)

Channels, started in 2015 as “django-onair”, had its 1.0 release in 2017. It used twisted, ran on python 2.7, and django runs synchronously.

Python 2.7 undermined it. Python 3 has asyncio support, 2.7 has not. Because of that, channels had to be too complex. The design was wrong.

Now, there’s channels 2.0. It requires python 3.5+. Native asyncio! Much simpler to deploy, also. It was quite a big rewrite: 75% was changed.

A big challenge was that django had to become partially asynchronous. The regular django ORM, views, middleware, url routing is still synchronous. Parallel to that, there’s channels (ASGI) middleware and so. Two separate worlds.

But still, there are a few contact points. Towards the ORM, for instance. So he needed two functions, sync_to_async and async_to_sync, to move between the two worlds. They took two months to write! Synchronous code has to run in threads. The ThreadPoolExecutor does most of the hard work.

Both async and sync code are useful. Channels lets you write your code as both. Async is hard, so you don’t want to have to be forced to use it. Channels’ two functions make it possible.

But: the async interface is separate from the sync interface. You just cannot provide both through one API.

He has a blog post that further explains his thoughts about handling async and sync code: https://www.aeracode.org/2018/02/19/python-async-simplified/

ASGI. WSGI, web service gateway interface, is used by all python web frameworks for handling requests/responses. There’s one problem: it is synchronous. And you cannot have something that starts synchronous, is async in between, and ends up call the synchronous ORM: it will block the whole thread/process. You have to start asynchronous.

So: WSGI, but then async. So: ASGI :-) It is intended for general use, just like WSGI. The core is an Application object with an async __call__(self, receive, send) method.

It is “turtles all the way down”: routing is an ASGI app. Middleware is an ASGI app.

What does this mean for django? Should it be in core? The main question is “how much can we make django async”. You could progressively change pieces of django to be async and call it from synchronous code. But….

The problem, he thought, was the ORM. What would an async ORM look like? Is it even a sensible endeavour? Note that it has to be done in small steps. But after recent talks, he thinks it could be done.

Another question: do we really need to replace WSGI? How much demand is there for long-polling and websockets? A new standard (ASGI) is another new standard. And an extra standard is not necessarily good.

Websockets are a niche. Long-polling is less of a niche, but still a niche.

For ASGI to become a standard, you need multiple servers that implement it (apart from “daphne”, there is now also “uvicorn”). And you need more frameworks that use it.

Another question: do we want to have everyone writing async? It is a pain to debug and hard to design. What is the balance? He would like for django to keep the existing sync interface for the majority of the cases. And if you need more power, that you then can dive deeper. Just like in many other cases.

So: if you have an opinion on where django should be going, talk to him and to other developers!

Photo explanation: constructing a viaduct module (which spans a 2m staircase) for my model railway on my attic.