PyGrunn: creating abstraction between consumer and datastore - Marco Vellinga¶

Tags: pygrunn, django

(One of my summaries of a talk at the 2017 PyGrunn conference).

Marco Vellinga started his talk with train-related analogies and photos, which I of course liked. They have a big monolythic application. Like a big train: move one part (a locomotive) and the rest also moves. In an application you don’t want that: if you make a change in one part of the application, you don’t want to have to change lots of other parts. Decoupled is better.

With abstraction, for instance a user object with username, password and billing information, you can change the backend (splitting user and billing info, for instance) without needing to change anything in the frontend.

They created an abstraction layer containing

• core logic

• validation

• error handling

• data contracts

There isn’t really support for such a layer in Django, so you have to build it yourself. This was a lot of work and took a couple of tries to get it right.

They used Marshmallow for json serialization. For serializing queries (the equivalent of django’s ‘Q’ objects) they build filterql.

For (API) versioning they build their own function/class decorator to mark functions as “version 1” or “version 2”. They’re calling it versionary. It is kind of spooky, but it is fully tested and documented.

The main question: is it worth building? Well, it paid off instantly when they put it into production because they immediately found invalid data that was already in the system. Because the abstraction layer put all the validation in one place, all the data that previously slipped through was now found out :-)

Photo explanation: just a nice unrelated picture from the my work-in-progress german model railway

Dutch note: python+django programmeren in hartje Utrecht bij de oude gracht? Watersector, dus veel data en geo. Leuk! Nelen&Schuurmans is op zoek. Stuur mij maar een mailtje, want de vacaturetekst staat nog niet online :-)