Django under the hood: python templating - Armin Ronacher¶

Tags: django, djangocon, python

(One of the summaries of a talk at the 2014 django under the hood conference).

Armin Ronacher is the maker of many non-Django things such as Flask and Jinja2.

His initial thought was “why are we even discussing templates in 2014”? In 2011 everyone started making single-page applications. But a year later it turned out not to be too good an idea: the cloud is much faster than your mobile phone. So server-side rendering is hip again.

Django in 2005 had one of the few available template languages that were any good. It got imitated a lot, for instance in Jinja2. Which is fast.

He doesn’t really like looking at his own Jinja2 code. For him, it is pretty early Python code and it could be so much nicer. Feature-wise Jinja2 hasn’t changed a lot in the recent years. There are a couple of problems, but they’re not big enough to warrant a fix, as any fix will invariably break someone’s template.

The template language is mostly the same, but Jinja2’s and Django’s internal design differs greatly. Django has an AST (“abstract syntax tree”) interpreter with made-up semantics. Jinja2 compiles into python code (“transpiles” he calls it).

Rendering is mostly the same .You have a context object with all data for the template, you run it through the AST or the code and you get output.

The parsing of the template is widely different. Jinja2 supports nested expressions, for instance, Django not. Django parses in two stages. With a regex it filters out blocks, statements, comments and the rest. One of the problems is that a block statement needs to be on a single line. You often end up with super-long lines this way.

The parsing inside a block in Django happens in the block’s implementation. So it is very inconsistent and often re-implemented.

Extensions? Heavily discouraged in Jinja2. It is tricky to debug due to the compiled nature. It is encouraged in Django (“template tags”). Many people build them. It is easy to implement and debugging is quite doable. It is easier to implement due to the context and the render method.

A Jinja2 template compiles into a generator yielding string chunks. The django render() functions yield strings. Any form of recursive calls generate new strings. In the end, jinja can generate huge documents, django can run into problems.

Error handling is at the python level in Jinja2. jinja2 generates python code, so it simply prints a traceback if something goes wrong. Django has its own error handling that shows less information.

A huge difference is the context. In jinja2 it is a source of data. It only holds top-level variables. In django, it itself stores data. It holds all variables. Basically, it is a stack of dictionaries.

Autoescaping. In django, it is django-specific. It largely lives only in the template engine. Jinja2 uses markupsafe, which is pretty much standard in the python world. Escaping is handled in python. (Django also uses markupsafe one-directionally).

He showed some example django and jinja2 code. Fun to see the generated python code coming out of a short jinja2 template! He also showed some of the optimizations that jinja2 can do. Impressive.

Fun: he hacked the C python traceback functionality to get good tracebacks out of his templates. Imagine seeing simply {% for i in something %} in a traceback line…

Armin tried to use jinja in django for a google summer of code project. But in the end, it boils down to philosophical differences:

• People like jinja because of its python expressions and because of its speed.

• People like django’s templates because of the extensibility.

Why can’t django just use jinja2? Jinja needed to sacrifice certain functionality. Doing the same in Django would break everybody’s code.

The problem is solved to a sufficient degree that he thinks no big changes will happen.