Djangocon keynote: where the wild things are - Aymeric Augustin¶

Tags: djangocon, django

(One of the summaries of a talk at 2014 djangocon.eu)

Aymeric Augustin talks about app loading. App loading is a step in django’s initialization sequence: import all models and populate a cache. It is also a project that improves this step.

The trac ticket for this feature is seven years old… Many many people have worked on it and provided patches and branches.

The final implementation looks like this:

from django.apps import AppConfig

class MyConfig(AppConfig):
    name = ...
    label = ...
    verbose_name = ...

# and in the settings file:

INSTALLED_APPS = (
    'some_app',  # Old style.
    'yourapp.apps.MyConfig',  # Points to the config class.
    ...

All nice, but in practice there were crashes when using the implementation in production with gunicorn. The root cause was that django doesn’t have a proper signal that says that it is fully configured and ready.

So he started digging into the code. And found out that actually reading all the models and setting everything up occurs at various points in time, depending on how you call it. “runserver” is different from “shell” which is different from “running it from wsgi”.

At the core is the AppCache that django uses behind the scenes. This looks at all the models and imports them and sets up relations and reverse relations. This needs to be done before using the ORM. Model classes automatically register themselves with the app cache. The good thing is that this proveds introspection of apps and models. The bad thing is that this means “global state all over the place”.

AppCache is a singleton based on the Borg pattern.

“Model classes automatically register themselves”? Yes, the base class you use for all your models has a __new__() method that actually creates the class. And that caches it in the AppCache. That’s how django can get its hands on it.

So… what is new in django 1.7 (“I hope I broke all your code for a good reason”). See the django release notes for the full list. He had three main goals:

• Allow apps without a models module.

• Provide verbose_name for the admin. For instance to be able to translate it.

• Provide a reliable initialization signal.

Several other wishes were also implemented. You can for instance add a ready() method to your AppConfig class that runs some code once django is ready configuring all the models. This is for instance used by django’s own django.contrib.admin: you don’t need to put admin.autodiscover() in your urls.py anymore!

Now back to the problem that the AppCache was populated at different times depending on the way you call it. This was fixed. Population is triggered explicitly when django starts. get_wsgi_application() calls django.setup() before returning the wsgi callable. Same for management commands: this calls it just after parsing the command line. Backwards incompatible: you need to call it yourself in stand-alone script.

There are some consistency fixes. For instance that apps now must have unique labels, you cannot override another app by accident. Models must be in apps that are actually in the INSTALLED_APPS. Overriding management commands now uses the same order of INSTALLED_APPS as used by the rest of django. Somehow it used the reverse order.

Note: as it is a complete rewrite, the “app cache” is now called the “app registry”. It did gain a proper documented API as it is now public. The old app cache was really only for internal use.