Django under the hood: expressions - Josh Smeaton¶

Tags: django, djangocon

(One of my summaries of a talk at the 2015 django under the hood conference).

Josh Smeaton is a django core developer after his work on expressions.

What already existed for a long time in django are F expressions. There are used to send a computation to the database. A self-contained parcel of SQL. Like “take the price field and add the shipping costs to it”. Later aggregations were added. It is a bit the same, as it is “just a bit of sql” that gets send to the database.

Expressions in django are now much more refined. Multiple database backend support. Deep integration in the ORM to make writing expressions yourself in django easier. It almost makes .extra() and .raw() obsolete.

• .raw() is for writing an entire query in SQL. For those corner cases where you need to do weird tricks that the ORM doesn’t support.

• .extra() is for appending bits of SQL to the rest of your django query. It is evil and should go away.

Both are escape hatches that are hardly ever needed. One problem with them is that they are database backend specific.

Some examples of where you can use expressions:

.create(... username=Lower(username))
.annotate(title=F('price') + shipping)
.aggregate(sum_total=Sum('total'))
order_by(Coalesce('last_name', 'first_name))
.filter(name=Lower(Value(user_input)))

Batteries included! There are a couple of build-in functions like Coalesce, Concat, Lower, Upper.

Expressions can hide complexity. F(), Case(), When(). F() can refer to fields added by an aggregate, for instance. That goes much deeper than you could do with some custom .extra() SQL. And Case()/When() can be used to select different values out of the database depending on other values.

There are building bloks: Aggregate(), Func(), Value(). You can use those to make your own expressions.

Expressions in django 1.8 now has a proper public API with documentation. The espressions are composable: Sum(F('talk') + F('hold')) + Sum('wrap'). And the internals of the whole ORM are greatly simlified.

There’s one thing you can do with .extra() but not with expressions: custom joins. There are also still a few small bugs as the functionality is still pretty new. They’ve all been easy to fix till now.

He then showed some examples of using expressions and writing your own. Looks nice!

Image: German type-86 loco taking on coal on my in-progress ‘Eifelburgenbahn’ layout.